Forem: Oliver Cole

Terraforming AWS: a serverless website backend, part 3

Oliver Cole — Wed, 19 Jul 2017 21:19:02 +0000

This is part three of my article series on using Terraform to build a serverless backend in AWS. Check out part one to get started.

Mailgun

Remember that Terraform supports a wide variety of cloud providers, and you can mix them together to produce the design you want.

The code we're deploying uses Mailgun.

We've already got the Mailgun API key configured as a Terraform variable - let's add an SMTP password to terraform.tfvars:

mailgun_smtp_password = "hunter2"

And then we can setup everything up in terraform.tf.

variable "mailgun_smtp_password" {}

provider "mailgun" {
  api_key = "${var.environment_configs["mail_api_key"]}"
}

resource "mailgun_domain" "serverless" {
  name          = "${var.environment_configs["mailgun_domain_name"]}"
  spam_action   = "disabled"
  smtp_password = "${var.mailgun_smtp_password}"
}

This is a good moment to touch on arguments and attributes - all the values you configured, like spam_action = "disabled", are arguments - just like calling constructors or functions in other languages.
Attributes are values on the resource that exist once it is created - for example an assigned IP address or URL, just like an instance property in other languages. We used it earlier when creating the aws_kms_alias for example:

   target_key_id = "${aws_kms_key.LambdaBackend_config.key_id}"

The mailgun_domain resource provides two attributes for the DNS records you need to create in order for Mailgun to send and receive email for your domain. There are two ways you can get this setup.

Manually setup Mailgun DNS records

The first way is simply to output the DNS records from Terraform and apply them manually at your DNS provider.

Back when we setup the SSM Parameter Store entries for our Lambda function, we used variables - specifically input variables that were passed from our terraform.tfvars file and onwards into ssm_parameter_map.tf. Terraform also supports output variables - either from modules like ssm_parameter_map or from our 'root' terraform.tf module.

Add these output variables to output the DNS records from the deployed mailgun_domain:

output "send" {
  value = "${mailgun_domain.serverless.sending_records}"
}

output "receive" {
  value = "${mailgun_domain.serverless.receiving_records}"
}

Then just terraform apply and the required DNS records will be shown at the end of the run. (If that didn't work - what might you have forgotten to do?)

Setup Mailgun DNS records using Terraform

Alternatively, at this point you probably know enough Terraform to try setting up the DNS records automatically! Terraform supports a variety of DNS providers, including AWS (you might consider using this module), DNSMadeEasy, DNSimple, Cloudflare, Dyn and generic RFC 2136 access.

If you aren't using anything more than your domain registrar's default settings, AWS has worked well for me personally, but does incur a small charge.

Make use of the sending_records and receiving_records attributes mentioned above to setup your provider, and make sure you've run terraform apply before proceeding. If you need to work with the list of records or adjust the format of any strings, check out the interpolation documentation for details.

Test Lambda

You can test your Lambda function by heading to the Lambda console, and using the test event:

{ "body-json": "email=user%40domain.com"}

Alternatively, you can do this on the command line:

aws lambda invoke --function-name LambdaBackend --payload "{ \"body-json\": \"email=user%40domain.com\"}" out.txt

Either way, you should receive an email at that address, and the result should look like this:

{
  "errorMessage": "Email.MovedPermanently : Redirecting.",
  "errorType": "http://your/redirect/URL",
  "stackTrace": [
    "redirect (/var/task/index.js:50:19)",
    "/var/task/index.js:111:20",
    "transporter.send (/var/task/node_modules/nodemailer/lib/mailer/index.js:187:21)",
    "/var/task/node_modules/nodemailer-mailgun-transport/src/mailgun-transport.js:149:9",
    "Request.finalCb [as callback] (/var/task/node_modules/nodemailer-mailgun-transport/node_modules/mailgun-js/lib/request.js:99:12)",
    "IncomingMessage.<anonymous> (/var/task/node_modules/nodemailer-mailgun-transport/node_modules/mailgun-js/lib/request.js:313:17)",
    "emitNone (events.js:91:20)",
    "IncomingMessage.emit (events.js:185:7)",
    "endReadableNT (_stream_readable.js:974:12)",
    "_combinedTickCallback (internal/process/next_tick.js:80:11)",
    "process._tickDomainCallback (internal/process/next_tick.js:128:9)"
  ]
}

The Lambda function returns an error for reasons you will see later.

If your function isn't working, take a look at the Cloudwatch logs for your function.

API gateway

Now that the function is working within AWS, we need to hook it up to the API gateway to expose it to the outside world.

This section sets up logging from the API Gateway:

resource "aws_api_gateway_account" "gateway" {
  cloudwatch_role_arn = "${aws_iam_role.cloudwatchlog.arn}"
}

resource "aws_iam_role" "cloudwatchlog" {
  name = "cloudwatchlog"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "apigateway.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}

resource "aws_iam_policy_attachment" "cloudwatchlog" {
  name       = "cloudwatchlog"
  roles      = ["${aws_iam_role.cloudwatchlog.name}"]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs"
}

This section sets up the actual API, resources and integrations. There are also Terraform modules available for this if you want to go down that route.

resource "aws_api_gateway_rest_api" "service" {
  name = "BackendService"
}

resource "aws_api_gateway_resource" "api" {
  rest_api_id = "${aws_api_gateway_rest_api.service.id}"
  parent_id   = "${aws_api_gateway_rest_api.service.root_resource_id}"
  path_part   = "api"
}

resource "aws_api_gateway_resource" "email" {
  rest_api_id = "${aws_api_gateway_rest_api.service.id}"
  parent_id   = "${aws_api_gateway_resource.api.id}"
  path_part   = "email"
}

resource "aws_api_gateway_method" "post" {
  rest_api_id   = "${aws_api_gateway_rest_api.service.id}"
  resource_id   = "${aws_api_gateway_resource.email.id}"
  http_method   = "POST"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "integration" {
  rest_api_id             = "${aws_api_gateway_rest_api.service.id}"
  resource_id             = "${aws_api_gateway_resource.email.id}"
  http_method             = "${aws_api_gateway_method.post.http_method}"
  integration_http_method = "POST"
  type                    = "AWS"
  uri                     = "arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${aws_lambda_function.LambdaBackend_lambda.arn}:$${stageVariables.alias}/invocations"
  passthrough_behavior    = "WHEN_NO_TEMPLATES"

  request_templates {
    "application/x-www-form-urlencoded" = <<EOF
##  See http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html
##  This template will pass through all parameters including path, querystring, header, stage variables, and context through to the integration endpoint via the body/payload
#set($allParams = $input.params())
{
"body-json" : $input.json('$'),
"params" : {
#foreach($type in $allParams.keySet())
    #set($params = $allParams.get($type))
"$type" : {
    #foreach($paramName in $params.keySet())
    "$paramName" : "$util.escapeJavaScript($params.get($paramName))"
        #if($foreach.hasNext),#end
    #end
}
    #if($foreach.hasNext),#end
#end
},
"stage-variables" : {
#foreach($key in $stageVariables.keySet())
"$key" : "$util.escapeJavaScript($stageVariables.get($key))"
    #if($foreach.hasNext),#end
#end
},
"context" : {
    "account-id" : "$context.identity.accountId",
    "api-id" : "$context.apiId",
    "api-key" : "$context.identity.apiKey",
    "authorizer-principal-id" : "$context.authorizer.principalId",
    "caller" : "$context.identity.caller",
    "cognito-authentication-provider" : "$context.identity.cognitoAuthenticationProvider",
    "cognito-authentication-type" : "$context.identity.cognitoAuthenticationType",
    "cognito-identity-id" : "$context.identity.cognitoIdentityId",
    "cognito-identity-pool-id" : "$context.identity.cognitoIdentityPoolId",
    "http-method" : "$context.httpMethod",
    "stage" : "$context.stage",
    "source-ip" : "$context.identity.sourceIp",
    "user" : "$context.identity.user",
    "user-agent" : "$context.identity.userAgent",
    "user-arn" : "$context.identity.userArn",
    "request-id" : "$context.requestId",
    "resource-id" : "$context.resourceId",
    "resource-path" : "$context.resourcePath"
    }
}

EOF
  }
}

resource "aws_api_gateway_method_response" "301" {
  rest_api_id = "${aws_api_gateway_rest_api.service.id}"
  resource_id = "${aws_api_gateway_resource.email.id}"
  http_method = "${aws_api_gateway_method.post.http_method}"
  status_code = "301"
  depends_on  = ["aws_api_gateway_integration.integration"]

  response_parameters = {
    "method.response.header.Location" = true
  }
}

resource "aws_api_gateway_integration_response" "default" {
  rest_api_id       = "${aws_api_gateway_rest_api.service.id}"
  resource_id       = "${aws_api_gateway_resource.email.id}"
  http_method       = "${aws_api_gateway_method.post.http_method}"
  status_code       = "${aws_api_gateway_method_response.301.status_code}"
  selection_pattern = "^Email.MovedPermanently.*"

  response_parameters = {
    "method.response.header.Location" = "integration.response.body.errorType"
  }

  depends_on = ["aws_api_gateway_integration.integration"]
}

resource "aws_api_gateway_deployment" "deploy" {
  depends_on = ["aws_api_gateway_method.post", "aws_api_gateway_integration.integration", "aws_api_gateway_integration_response.default"]

  rest_api_id = "${aws_api_gateway_rest_api.service.id}"
  stage_name  = "${terraform.env}"

  variables = {
    "alias" = "${terraform.env}"
  }
}

resource "aws_api_gateway_method_settings" "settings" {
  rest_api_id = "${aws_api_gateway_rest_api.service.id}"
  stage_name  = "${aws_api_gateway_deployment.deploy.stage_name}"
  method_path = "${aws_api_gateway_resource.api.path_part}/${aws_api_gateway_resource.email.path_part}/${aws_api_gateway_method.post.http_method}"

  settings {
    metrics_enabled = true
    logging_level   = "INFO"
  }
}

resource "aws_lambda_alias" "alias" {
  name             = "${terraform.env}"
  function_name    = "${aws_lambda_function.LambdaBackend_lambda.arn}"
  function_version = "$LATEST"
}

resource "aws_lambda_permission" "invoke" {
  statement_id  = "${terraform.env}Invoke"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.LambdaBackend_lambda.arn}"
  principal     = "apigateway.amazonaws.com"
  source_arn    = "arn:aws:execute-api:${var.region}:${data.aws_caller_identity.current.account_id}:${aws_api_gateway_rest_api.service.id}/*/${aws_api_gateway_method.post.http_method}/${aws_api_gateway_resource.api.path_part}/${aws_api_gateway_resource.email.path_part}"
  qualifier     = "${terraform.env}"
  depends_on    = ["aws_lambda_alias.alias"]
}

Add this output variable to output the path from the deployed API Gateway:

output "invoke_url" {
  value = "${aws_api_gateway_deployment.deploy.invoke_url}/${aws_api_gateway_resource.api.path_part}/${aws_api_gateway_resource.email.path_part}"
}

When you terraform apply you will get the value of the variable printed at the end of the output:

Outputs:

invoke_url = https://api_id_here.execute-api.eu-west-1.amazonaws.com/default/api/email

Test API

Now your Lambda function is exposed as an API.

Start up Postman, and enter the value from invoke_url as your URL.
Change the method to POST
On the Body tab, set the type to x-www-form-urlencoded
Add the key email and the target email address as the value.
Hit Send!

Postman should follow the redirect to your configured URL, and you should receive the email in your inbox. Congratulations!

Mayhem and destruction

If at any point you want to tear down the configuration of these services at AWS and Mailgun, you can simply run terraform destroy and confirm when asked. Your terraform.tf file will be unaffected.

DynamoDB provisioned throughput does incur a running cost for example, so you may wish to consider doing this when you have completed the article series.

Next steps

Rob's original article goes on to add much more functionality, and make use of additional AWS services. However, you probably don't need to know any more about Terraform to do so, so I am going to pause the article series here to gather some feedback. If enough people find this useful I will finish the series.

Please let me know in the comments if you have any questions or comments, and especially if you found this useful!

Terraforming AWS: a serverless website backend, part 2

Oliver Cole — Wed, 19 Jul 2017 21:18:49 +0000

This is part two of my article series on using Terraform to build a serverless backend in AWS. Check out part one to get started.

Data sources and encryption

Next up is a new concept: data sources. These allow you to pull in data from other external sources at runtime.
At the end of part one, we added this:

data "aws_caller_identity" "current" {}

aws_caller_identity allows you to get the AWS user ID of the account, and it's used in constructing policy documents and ARNs, using exactly the same interpolation as before:

resource "aws_kms_key" "LambdaBackend_config" {
  description             = "LambdaBackend_config_key"
  deletion_window_in_days = 7

  policy = <<POLICY
{
  "Version" : "2012-10-17",
  "Id" : "key-consolepolicy-3",
  "Statement" : [ {
    "Sid" : "Enable IAM User Permissions",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"
    },
    "Action" : "kms:*",
    "Resource" : "*"
  }, {
    "Sid" : "Allow use of the key",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "${aws_iam_role.LambdaBackend_master_lambda.arn}"
    },
    "Action" : [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ],
    "Resource" : "*"
  }, {
    "Sid" : "Allow attachment of persistent resources",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "${aws_iam_role.LambdaBackend_master_lambda.arn}"
    },
    "Action" : [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ],
    "Resource" : "*",
    "Condition" : {
      "Bool" : {
        "kms:GrantIsForAWSResource" : "true"
      }
    }
  } ]
}
POLICY
}

resource "aws_kms_alias" "LambdaBackend_config_alias" {
  name          = "alias/LambdaBackend_config"
  target_key_id = "${aws_kms_key.LambdaBackend_config.key_id}"
}

When we create some configuration for the service we'll be encrypting it - these resources allow us to do that!

You can terraform apply all of that.

Terraform is pre v1.0 software, and the aws_kms_key resource can sometimes throw an error. If this happens, just terraform apply again, and keep an eye on this issue!

Application config

Now we're going to define some configuration for our Lambda function. It's good practice to separate application code and configuration, and it's even better practice not to check the latter into source control. We're going to store our configuration using the AWS EC2 Systems Manager, known as SSM, and in the process going to learn about Terraform variables and modules.

We're going to be storing our configuration in the SSM Parameter Store. This is a key-value store useful for simple configuration strings, API keys and passwords - and it's free! You can configure content to be encrypted with the keys we just created, and control access to it with the IAM policies we also created.

Variables in Terraform work in a similar way to most programming languages. You declare them in the code, possibly with a value, and then you retrieve the value with the ${var.variable_name} syntax, similar to data sources above. You can also specify these variable values on the command line, or in separate files, meaning you don't have to check your secrets into source control. See How to Use Terraform Variables for more information.

Terraform maps are similar to hashtables, hashmaps, dictionaries or similar data structures in other languages. Go ahead and define it in terraform.tf.

variable "environment_configs" {
  type = "map"
}

Then you can create a new file called terraform.tfvars in the same directory as terraform.tf:

environment_configs = {
  site_callback = "CALLBACK TO RETURN TO FROM LAMBDA",
  email_table_name = "EMAIL DYNAMODB TABLE",
  aes_password = "AES PASSWORD",
  mail_api_key = "MAILGUN KEY",
  mailgun_domain_name = "MAILGUN DOMAIN",
  from_email = "OUTGOING EMAIL ACCOUNT"
}

Some tips on the contents of the variables:

site_callback - An http(s) URL.
email_table_name - Just a simple table name following the DynamoDB rules, like emails.
aes_password - A 32 character AES key. If you don't have one handy, check out keygen.js.
mail_api_key - From your Mailgun account.
mailgun_domain_name - A domain or subdomain you control DNS for.
from_email - An account on that domain. Just a name, no need to set anything up - Mailgun will work regardless!

Terraform supports the concept of modules - reusable collections of resources intended to work together: for example, a set of web servers and a load balancer. You can use this across multiple projects, even separating it out into separate source control or storage.

We're going to use a simple module to turn the environment_configs map into entries in the SSM Parameter Store. In the same directory as terraform.tf create a new directory called ssm_parameter_map, containing a file called ssm_parameter_map.tf, with this content:

variable "configs" {
  description = "Key/value pairs to create in the SSM Parameter Store"
  type        = "map"
}

variable "prefix" {
  description = "Prefix to apply to all key names"
}

variable "kms_key_id" {
  description = "ID of KMS key to use to encrypt values"
}

resource "aws_ssm_parameter" "configs" {
  count  = "${length(keys(var.configs))}"
  name   = "/${var.prefix}/${element(keys(var.configs),count.index)}"
  type   = "SecureString"
  value  = "${element(values(var.configs),count.index)}"
  key_id = "${var.kms_key_id}"
}

You will notice that this module has variables of its own, and the values get set when you call this module from your own code. For more details on the rest of the code, check out Gruntwork's article on loops and if statements.

Call your module from terraform.tf:

module "parameters" {
  source     = "/ssm_parameter_map"
  configs    = "${var.environment_configs}"
  prefix     = "${terraform.env}"
  kms_key_id = "${aws_kms_key.LambdaBackend_config.key_id}"
}

The source argument points to the module code, and the other arguments feed into the variables you saw in ssm_parameter_map.tf. terraform.env is currently the string default, but this will change later on when we delve into Terraform state environments.

Because Terraform modules could be located anywhere, you need to run terraform init to pull down your copy of the module code. This is the same command you ran earlier to download the AWS provider for Terraform. It should look like this:

λ terraform init 
Initializing modules... 
- module.parameters 
  Getting source "./ssm_parameter_map"

Then terraform apply as usual. Remember that you will need to run terraform get again if you move between machines or Git working copies.

Terraform is pre v1.0 software, and the aws_ssm_parameter resources in the module can sometimes throw a TooManyUpdates error. If this happens, just terraform apply again, and keep an eye on this issue!

Application code

Now we're going to put together code for our Lambda function. Once it's built and packaged for deployment, we'll switch back to Terraform and deploy it.

Create a sub-directory called email_lambda
Download index.js to this directory.
- On line 8, change us-west-1 to the same AWS region you configured your aws provider with.
- This is our main app code
Add a file called email.html to this directory containing the email content.
- If you're short on ideas, you can use this example file.
Install the email libraries we will use with npm. Make sure to do this from inside email_lambda:

npm install -prefix=./ nodemailer@4.0.1 nodemailer-mailgun-transport@1.3.5 aws-sdk@2.81.0 ssm-params@0.0.6

Now zip the whole directory into email_lambda.zip, making sure that the root of the zip file contains the files (index.js etc) and not a directory called email_lambda. It should look like this:

email_lambda.zip
├───etc
├───node_modules
├───email.html
└───index.js

If you have 7Zip installed on Windows, the command would be 7z.exe a -r email_lambda.zip .\email_lambda\*.

At this point, your directory structure should look something like this:

├───.terraform
│   └───modules
│       └───03f77d1ff66d94c49e171247a4234cd8
├───email_lambda
│   ├───etc
│   └───node_modules
│       ├───nodemailer
│       │   └─── ...
│       ├───nodemailer-mailgun-transport
│       │   ├─── ...
│       └───ssm-params
│           └─── ...
├───ssm_parameter_map
├───email_lambda.zip
├───terraform.tf
├───terraform.tfstate
├───terraform.tfstate.backup
└───terraform.tfvars

Now, you can add the Lambda function to terraform.tf:

resource "aws_lambda_function" "LambdaBackend_lambda" {
  filename         = "email_lambda.zip"
  function_name    = "LambdaBackend"
  role             = "${aws_iam_role.LambdaBackend_master_lambda.arn}"
  handler          = "index.handler"
  source_code_hash = "${base64sha256(file("email_lambda.zip"))}"
  runtime          = "nodejs6.10"
  timeout          = 15
  publish          = true

  environment {
    variables = {
      env = "${terraform.env}"
    }
  }
}

Apply that, and carry on to the final part.

Terraforming AWS: a serverless website backend, part 1

Oliver Cole — Wed, 19 Jul 2017 21:18:23 +0000

What if you could define all the infrastructure for your cloud application using code or text, apply your design and changes automatically, and then collaborate with your team in source control?

That's the promise of Terraform, an infrastructure-as-code tool from HashiCorp

Introduction

Rob Sherling wrote a great article on how to build a website backend on serverless infrastructure. This article will take you through the first stages of constructing Rob's infrastructure design using Terraform, and explain some Terraform concepts and rules along the way. Thanks in advance to Rob, who allowed me to write this article on top of his code and design!

This article, like Rob's original, assumes that:

You have an AWS account, know how to use it, and are comfortable with some small charges (though this generally fits into the free tier).
You have the AWS CLI installed and configured.
You have a domain name you can use for this project.
You're comfortable with basic Javascript - don't worry, we will just be modifying some existing code.
You have at least heard of the following (if not, read a bit on the ones you don’t know):
- AWS Lambda
- API Gateway
- CloudWatch
- DynamoDB
- KMS
- IAM
- The mail service Mailgun
- The API tool Postman

You might need more later on, but if you have the above you can build something today!

We're going to make use of all these different services, and some application code to build an entire application. If you would prefer to just learn the basics by starting up some VMs on EC2, you may prefer David Schmitz's article on Using Terraform for Cloud Deployments.

So what is Terraform?

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.

In other words, you describe your application infrastructure in human readable config/code, and Terraform handles all the detail of how to make that happen at your cloud provider.

Both Terraform and CloudFormation allow you to achieve similar results at AWS, but Terraform supports many more cloud providers than just AWS, as well as letting you work with scenarios like multiple AWS accounts, or composing different cloud providers into one deployed application.

What are we actually building?

We are going to make the backend for a website to be used in a marketing campaign - it will gather Twitter handles and email addresses for potential customers, and allow you to send them messages. To be hip and modern, we're going to make the backend 'serverless' by using entirely AWS cloud services (mainly Lambda and DynamoDB) to reduce costs and maintenance.

In part 1 of this article series, we will just tackle the initial email capture and storage.

Here's how the main pieces plug together:

What do I need?

You'll need a few things to complete the whole series, but for part 1, all you need is the following:

Terraform

Download it, and place it in your PATH.

Postman

Install it. We'll use this to test your backend API.

Mailgun account

Sign up for a free account. We'll use this to send the emails. Don't link your domain yet (if you did already, delete it). Make sure you set up some authorized recipients to use while testing.

Node.js

Install it. The infrastructure we build will also support Python, Java and C# (.NET Core), but the example code in this article series is all Node.js.

Optional: Text editor plugins

If you've graduated to something more advanced than Notepad (and you really should!), installing syntax highlighting and editor settings can make a big difference to your day:

Atom syntax highlighting
VS Code syntax highlighting, linting, formatting and validation
Sublime Text syntax highlighting and code snippets

Getting started

As we go along I will introduce various Terraform and AWS concepts, and link to their documentation. You can lay Terraform files out in whatever order you like - I've only chosen this order to slowly introduce new concepts.

Setup

Create a new directory for this project, and a new text file in there called terraform.tf.

Before we get started with setting up infrastructure in AWS, we need to tell the AWS provider built into Terraform how to talk to the AWS API. All we need to cover for now is the AWS region - the provider will pick up the rest of your credentials from the AWS CLI you already configured. Don't worry too much about the variable, we'll cover that later.

variable "region" {
  default = "eu-west-1"
}

provider "aws" {
  region = "${var.region}"
}

You'll notice the code looks pretty much like config files you're used to, eg JSON or YML. You can set the region value to your nearest if you wish.

After providers, Terraform is built around the concept of resources. Generally these correspond to 'things' on offer at your cloud provider, such as virtual machines or servers, databases, stored files, etc. Sometimes they actually refer to links or associations between two resources, or other pieces of configuration.

Database

We will start with the DynamoDB table - this has a 1:1 relationship with a Terraform resource. Add this into your terraform.tf:

resource "aws_dynamodb_table" "emails" {
  name           = "emails"
  read_capacity  = 2
  write_capacity = 2
  hash_key       = "email"

  attribute {
    name = "email"
    type = "S"
  }
}

Let's break this down:

aws_dynamodb_table is the resource provided by the AWS provider.
The first emails is the name for this resource - but in Terraform only. You can be as generic or descriptive as you like, but like in any software development, it's good practice to be able to understand what something is by just reading the name.
The second emails is the actual DynamoDB table name - you'll see this in the AWS console and refer to it in code.
The rest of the arguments are specific to the resource - check out the docs if you want to know more.

Init

Terraform is based on a pluggable design - the components that actually talk to cloud providers and create resources are downloaded automatically and stored in your project directory.
So before you can call AWS, you need to tell Terraform to download the AWS provider:

λ terraform init 

Initializing provider plugins... 
- Checking for available provider plugins on https://releases.hashicorp.com... 
- Downloading plugin for provider "aws" (1.11.0)... 

The following providers do not have any version constraints in configuration, 
so the latest version was installed. 

To prevent automatic upgrades to new major versions that may contain breaking 
changes, it is recommended to add version = "..." constraints to the 
corresponding provider blocks in configuration, with the constraint strings 
suggested below. 

* provider.aws: version = "~> 1.11"

Plan and apply

The next step is to let Terraform build a plan of the changes you've requested. Go ahead and run terraform plan - the AWS provider in Terraform will look at your current AWS setup and plan out what it needs to do to deploy your infrastructure. It should look something like this:

λ terraform plan
[...]
An execution plan has been generated and is shown below. 
Resource actions are indicated with the following symbols: 
  + create 

Terraform will perform the following actions: 

  + aws_dynamodb_table.emails 
      id:                        <computed> 
      arn:                       <computed> 
      attribute.#:               "1" 
      attribute.3226637473.name: "email" 
      attribute.3226637473.type: "S" 
      hash_key:                  "email" 
      name:                      "emails" 
      read_capacity:             "2" 
      server_side_encryption.#:  <computed> 
      stream_arn:                <computed> 
      stream_label:              <computed> 
      stream_view_type:          <computed> 
      write_capacity:            "2" 


Plan: 1 to add, 0 to change, 0 to destroy.

What does this show?:

Your new resource, aws_dynamodb_table.emails is shown, with all of its properties. As + indicates, this is a resource due for creation.
The values marked <computed> will be assigned by AWS when the table is created. Remember this is only Terraform's plan of action, so these don't exist yet.

By the way - if you have anything else already setup in your AWS account you will notice it does not appear here. You can mix manually managed resources with Terraform resources safely, and even multiple Terraform projects.

At this point you can go ahead with terraform apply:

λ terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_dynamodb_table.emails
      id:                        <computed>
      arn:                       <computed>
      attribute.#:               "1"
      attribute.3226637473.name: "email"
      attribute.3226637473.type: "S"
      hash_key:                  "email"
      name:                      "emails"
      read_capacity:             "2"
      server_side_encryption.#:  <computed>
      stream_arn:                <computed>
      stream_label:              <computed>
      stream_view_type:          <computed>
      write_capacity:            "2"


Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_dynamodb_table.emails: Creating...
  arn:                       "" => "<computed>"
  attribute.#:               "" => "1"
  attribute.3226637473.name: "" => "email"
  attribute.3226637473.type: "" => "S"
  hash_key:                  "" => "email"
  name:                      "" => "emails"
  read_capacity:             "" => "2"
  server_side_encryption.#:  "" => "<computed>"
  stream_arn:                "" => "<computed>"
  stream_label:              "" => "<computed>"
  stream_view_type:          "" => "<computed>"
  write_capacity:            "" => "2"
aws_dynamodb_table.emails: Still creating... (10s elapsed)
aws_dynamodb_table.emails: Creation complete after 14s (ID: emails)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

If you get tired of typing yes, you can always add a -auto-approve flag to skip confirmation.

You might have noticed that the table was provisioned with a throughput capacity of 2. That's probably overkill for this project, so edit terraform.tf to make that 1, and terraform plan.

~ aws_dynamodb_table.emails
    read_capacity:  "2" => "1"
    write_capacity: "2" => "1"

This time around you can see that the table is due for a change (marked with ~) - Terraform won't destroy and recreate it. Arguments changed will be shown underneath. At this point, terraform apply should be self explanatory:

aws_dynamodb_table.emails: Modifying... (ID: emails)
  read_capacity:  "2" => "1"
  write_capacity: "2" => "1"

If you run terraform state list, you will see a list of everything that Terraform manages in your AWS account. You can check terraform state show aws_dynamodb_table.emails to get the details of a specific resource.

Permissions and security

IAM

We'll be running the backend server logic in a Lambda function, but before we build that we need to setup an IAM role and attach some policies to it. This will give our function permissions to access other AWS services.

data "aws_caller_identity" "current" {}

resource "aws_iam_role" "LambdaBackend_master_lambda" {
  name = "LambdaBackend_master_lambda"
  path = "/"

  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
POLICY
}

resource "aws_iam_role_policy_attachment" "LambdaBackend_master_lambda_AmazonDynamoDBFullAccess" {
  role       = "${aws_iam_role.LambdaBackend_master_lambda.name}"
  policy_arn = "arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess"
}

resource "aws_iam_role_policy_attachment" "LambdaBackend_master_lambda_CloudWatchFullAccess" {
  role       = "${aws_iam_role.LambdaBackend_master_lambda.name}"
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchFullAccess"
}

resource "aws_iam_role_policy" "ssm_read" {
  name = "ssm_read"
  role = "${aws_iam_role.LambdaBackend_master_lambda.id}"

  policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
          "ssm:DescribeParameters"
      ],
      "Resource": "*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "ssm:GetParameters"
        ],
        "Resource": "arn:aws:ssm:${var.region}:${data.aws_caller_identity.current.account_id}:parameter/${terraform.env}/*"
    }
  ]
}
POLICY
}

There's some new syntax going on here, so let's explain:

The << syntax allows you to include a multi-line string or document. In this case we're using it for the IAM policy document. After << we have POLICY, telling Terraform to read from that point until the next POLICY marker.
- The name POLICY isn't special, anything will do.
- If you've ever scripted a Unix terminal, this may sound familiar.
- As you may have guessed, anything inside the angle brackets isn't for Terraform, so if you don't understand what it means, don't worry for now.
If you don't want to keep your AWS policy documents in your Terraform code, check out the file() built-in function to load them from disk.
The role argument relies on Terraform interpolation. This is a fancy way of saying it allows you to get values from elsewhere, whether that is another resource, or some static configuration.
- If you're used to an OO programming language, think of this like accessing class members.
- All we're really doing here is making sure the policy attachment works, no matter how you name the aws_iam_role. This is an example of Don't Repeat Yourself, a good practice in any language.
Don't worry about the data line for now - we'll cover that in part two.

When you're happy, go ahead and terraform apply this, and carry on to the next part.