Forem: Matt Jarrett

Build an Express like app on AWS Lambda

Matt Jarrett — Tue, 06 Oct 2020 13:52:52 +0000

A dead simple Express-like Lambda hello world example ⚡ ☁️

What

I recently had a need to build a backend REST app and wanted to use a few simple routes via Express to serve my needs. Express is a commonly used backend in JavaScript/ Node.js. I wanted to run this on AWS via Lambda for all of the benefits of having a serverless architecture. This article is the result of what I learned making it work. You should be able to follow this example and fall well within the AWS free tier.

I had two goals in mind:

I want to create and manage my infrastructure with Terraform.
I want to use simple Express-like back end.

How much work is it?

The JavaScript portion of this hello world example is simple with more of the lines of code being Terraform to create and wire up the AWS services. Both are outlined below. I also have all of the code for this example on my GitHub.

Prerequisites

You'll need Node.js installed and an AWS account.

lambda-api

lambda-api offers a simple and lightweight solution that will look familiar to anyone that has spent time with Express. Building a solution with lambda-api provides a single dependency solution that is tiny at 28 kB.

Start a new Node project with npm init.

npm init

Install lambda-api

npm install lambda-api

Now create a index.js file in project with this content.

index.js

// Require the framework and instantiate it
const api = require("lambda-api")()

// Define a route
api.get("/", async (req, res) => {
  console.log("hello world")
  return "hello world"
})

api.get("/foo", async (req, res) => {
  console.log("/foo hit")
  return "/foo hit"
})

api.get("/bar", async (req, res) => {
  console.log("/bar hit")
  return "/bar hit"
})

// Declare your Lambda handler
exports.handler = async (event, context) => {
  return await api.run(event, context)
}

lambda-api makes the routes simple via get(), post(), put() to name a few options.

Terraform

In order to make this work with AWS Lambda, you need to expose the Lambda through an API Gateway. I wanted to use Terraform for building, deploying, and deleting my infrastructure. This is known as infrastructure as code (IaC). Using Terraform will provides us several benifits:

Orchestration, not merely configuration
Immutable infrastructure
Declarative, not procedural code
Speed to create, re-create, change, or delete infrastructure.

For simplicity in this hello world example, I will use my local machine to store the Terraform state but would recommend Terraform Cloud for actual app infrastructure state storage. Never upload your state to GitHub or your choice of repository.

Create a terraform folder in your project. In that folder, run terraform init to initialize a working directory containing Terraform configuration files. This is the first command that should be run after writing a new Terraform configuration or cloning an existing one from version control. It is safe to run this command multiple times.

provider.tf

Create a provider.tf file with this content.

provider.tf

provider "aws" {
  version = "~> 3.0"
  region = var.aws-region
}

variables.tf

Create a variables.tf file with this content.

variables.tf

variable "aws-region" {
  description = "AWS region for the infrastructure"
  type = string
  default = "us-east-1"
}

Modules

We are going to use modules to organize the IaC. Inside the terraform folder, create a modules folder. We'll create a handful of modules within that.

archive_file

Inside of the terraform/modules folder, let's create a folder called archive.

In the terraform/modules/archive folder create a main.tf file with this content.

main.tf

data "archive_file" "placeholder" {
  type = "zip"
  output_path = "${path.module}/lambda-function-payload.zip"

  source {
    content  = "placeholder"
    filename = "placeholder.txt"
  }
}

output "data-archive-file-placeholder-output-path" {
  value = data.archive_file.placeholder.output_path
}

We use archive_file which generates an archive from content, a file, or directory of files. It will hold a placeholder text file used when creating the Lambda below. This is done to separate the creation, updating, and deletion of the infrastructure from the deployment of the code in the deployment stage of a CI/CD pipeline. Yay, clean seperation 🎉!

iam

We will use AWS IAM to manage access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.

Inside of the terraform/modules folder, let's create a folder called iam.

In the terraform/modules/iam folder create a main.tf file with this content.

main.tf

resource "aws_iam_role" "express-like-lambda-example" {
  name               = "express-like-lambda-example"
  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": {
    "Action": "sts:AssumeRole",
    "Principal": {
      "Service": "lambda.amazonaws.com"
    },
    "Effect": "Allow"
  }
}
POLICY
}

resource "aws_iam_policy" "express-like-lambda-example-logs" {
  name        = "express-like-lambda-example-logs"
  description = "Adds logging access"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*"
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment" "attach-logs" {
  role       = aws_iam_role.express-like-lambda-example.name
  policy_arn = aws_iam_policy.express-like-lambda-example-logs.arn
}

output "aws-iam-role-express-like-lambda-example-arn" {
  value = aws_iam_role.express-like-lambda-example.arn
}

The aws_iam_role express-like-lambda-example sets up the role for the Lambda we're going to use. After that we set a aws_iam_policy express-like-lambda-example-logs which adds logging access to the Lambda. We use a aws_iam_role_policy_attachment called attach-logs to attach the policy to the role. Lastly, we output the arn of the role for use in another module a little later.

lambda

We will use AWS Lambda to run our code without provisioning or managing servers. You pay only for the compute time you consume.

With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Lambda takes care of everything required to run and scale your code with high availability.

Inside of the terraform/modules folder, let's create a folder called lambda.

In the terraform/modules/lambda create a variables.tf file with this content.

variables.tf

variable "aws-iam-role-express-like-lambda-example-arn" {
  description = "IAM role ARN"
  type = string
}

variable "data-archive-file-placeholder-output-path" {
  description = "Placeholder content for Lambda"
  type = string
}

The first variable is the arn of the iam role from above. The second variable is the output path of the archive file from above. Both are needed in this example to create the Lambda.

In the terraform/modules/lambda folder create a main.tf file with this content.

main.tf

resource "aws_lambda_function" "express-like-lambda-example" {
  filename = var.data-archive-file-placeholder-output-path
  function_name = "express-like-lambda-example"
  handler       = "index.handler"
  role          = var.aws-iam-role-express-like-lambda-example-arn
  runtime       = "nodejs12.x"
  memory_size   = 128
  timeout       = 1
}

resource "aws_lambda_function_event_invoke_config" "express-like-lambda-example-event-invoke-config" {
  function_name = aws_lambda_function.express-like-lambda-example.arn
  maximum_event_age_in_seconds = 60
  maximum_retry_attempts       = 0
}

resource "aws_lambda_permission" "express-like-lambda-example" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.express-like-lambda-example.arn
  principal     = "apigateway.amazonaws.com"
}

output "aws-lambda-function-express-like-lambda-example-arn" {
  value = aws_lambda_function.express-like-lambda-example.arn
}

output "aws-lambda-function-express-like-lambda-example-invoke-arn" {
  value = aws_lambda_function.express-like-lambda-example.invoke_arn
}

The aws_lambda_function express-like-lambda-example creates the Lambda function. The filename used is from the archive above by using the variable we defined in terraform/modules/lambda/variables.tf. The aws_lambda_function_event_invoke_config express-like-lambda-example-event-invoke-config allows us to define the maximim age in seconds to allow the function to run for and the maximum retry attempts. The aws_lambda_permissionexpress-like-lambda-example allows the Lambda to be to be executed via API Gateway. Finally, we output Lambda arn and invoke_arn to be used later when we set up the API Gateway.

api-gateway

Home stretch, hang in there...

We will use AWS API Gateway to create our RESTful API. API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management. API Gateway has no minimum fees or startup costs. You pay for the API calls you receive and the amount of data transferred out.

Inside of the terraform/modules folder, let's create a folder called api-gateway.

In the terraform/modules/api-gateway create a variables.tf file with this content.

variables.tf

variable "aws-lambda-function-express-like-lambda-example-arn" {
  description = "express-like-lambda-example Lambda ARN"
  type = string
}

variable "aws-lambda-function-express-like-lambda-example-invoke-arn" {
  description = "express-like-lambda-example Lambda invoke ARN"
  type = string
}

The first variable specifies the Lambda arn and the second specifies the Lambda invoke_arn.

In the terraform/modules/iam-gateway folder create a main.tf file with this content.

main.tf

resource "aws_api_gateway_rest_api" "express-like-lambda-example" {
  name = "express-like-lambda-example"
}

resource "aws_api_gateway_method" "proxy-root" {
  rest_api_id   = aws_api_gateway_rest_api.express-like-lambda-example.id
  resource_id   = aws_api_gateway_rest_api.express-like-lambda-example.root_resource_id
  http_method   = "ANY"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "express-like-lambda-example" {
  rest_api_id             = aws_api_gateway_rest_api.express-like-lambda-example.id
  resource_id             = aws_api_gateway_method.proxy-root.resource_id
  http_method             = aws_api_gateway_method.proxy-root.http_method
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = var.aws-lambda-function-express-like-lambda-example-invoke-arn
}

resource "aws_api_gateway_resource" "proxy" {
  rest_api_id = aws_api_gateway_rest_api.express-like-lambda-example.id
  parent_id   = aws_api_gateway_rest_api.express-like-lambda-example.root_resource_id
  path_part   = "{proxy+}"
}

resource "aws_api_gateway_method" "proxy" {
  rest_api_id   = aws_api_gateway_rest_api.express-like-lambda-example.id
  resource_id   = aws_api_gateway_resource.proxy.id
  http_method   = "ANY"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "lambda" {
  rest_api_id             = aws_api_gateway_rest_api.express-like-lambda-example.id
  resource_id             = aws_api_gateway_method.proxy.resource_id
  http_method             = aws_api_gateway_method.proxy.http_method
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = var.aws-lambda-function-express-like-lambda-example-invoke-arn
}

resource "aws_api_gateway_deployment" "express-like-lambda-example_v1" {
  depends_on = [
    aws_api_gateway_integration.express-like-lambda-example
  ]
  rest_api_id = aws_api_gateway_rest_api.express-like-lambda-example.id
  stage_name  = "v1"
}

output "endpoint" {
  value = aws_api_gateway_deployment.express-like-lambda-example_v1.invoke_url
}

Here we are setting up the Lambda Proxy Integration option in API Gateway that allows the details of an API request to be passed as the event parameter of a Lambda function.

lambda-api automatically parses this information to create a normalized REQUEST object. The request can then be routed using lambda-api's methods.

aws_api_gateway_rest_api provides an API Gateway REST API. aws_api_gateway_method provides a HTTP Method for an API Gateway Resource. aws_api_gateway_integration provides an HTTP Method Integration for an API Gateway Integration. aws_api_gateway_resource provides an API Gateway Resource. aws_api_gateway_deployment provides an API Gateway REST Deployment. Lastly, we output the URL to invoke the API.

main.tf

We now need to tie all of the Terraform we've made together. In the terraform folder create a main.tf file with this content.

module "archive" {
  source = "./modules/archive"
}

module "iam" {
  source = "./modules/iam"
}

module "lambda" {
  source = "./modules/lambda"
  data-archive-file-placeholder-output-path = module.archive.data-archive-file-placeholder-output-path
  aws-iam-role-express-like-lambda-example-arn = module.iam.aws-iam-role-express-like-lambda-example-arn
}

module "api-gateway" {
  source = "./modules/api-gateway"
  aws-lambda-function-express-like-lambda-example-arn = module.lambda.aws-lambda-function-express-like-lambda-example-arn
  aws-lambda-function-express-like-lambda-example-invoke-arn = module.lambda.aws-lambda-function-express-like-lambda-example-invoke-arn
}

# Set the generated URL as an output. Run `terraform output url` to get this.
output "endpoint" {
  value = module.api-gateway.endpoint
}

This chains together all of the modules we've written and completes the declarative infrastructure with Terraform.

Running the code

Deploying the Infrastructure

🎉 You made it this far! Let's play with the code you've made! 🎉

We're going to run some Terraform commands to deploy the infrastructure.

terraform plan

The terraform plan command is used to create an execution plan. This command is a convenient way to check whether the execution plan for a set of changes matches your expectations without making any changes to real resources or to the state.

That should work without issue so you can move onto applying this Terraform plan.

terraform apply

The terraform apply command is used to apply the changes required to reach the desired state of the configuration, or the pre-determined set of actions generated by a terraform plan execution plan.

You will need to confirm this apply with a yes when prompted. Take the time to read what is about to be created before you enter yes. It will show you what is about to be created.

For example:

terraform apply

...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.iam.aws_iam_policy.express-like-lambda-example-logs: Creating...
module.iam.aws_iam_role.express-like-lambda-example: Creating...
module.api-gateway.aws_api_gateway_rest_api.express-like-lambda-example: Creating...
module.iam.aws_iam_role.express-like-lambda-example: Creation complete after 0s [id=express-like-lambda-example]
module.lambda.aws_lambda_function.express-like-lambda-example: Creating...
module.iam.aws_iam_policy.express-like-lambda-example-logs: Creation complete after 1s [id=arn:aws:iam::REDACTED:policy/express-like-lambda-example-logs]
module.api-gateway.aws_api_gateway_rest_api.express-like-lambda-example: Creation complete after 1s [id=REDACTED]
module.iam.aws_iam_role_policy_attachment.attach-logs: Creating...
module.api-gateway.aws_api_gateway_resource.proxy: Creating...
module.api-gateway.aws_api_gateway_method.proxy-root: Creating...
module.api-gateway.aws_api_gateway_method.proxy-root: Creation complete after 0s [id=REDACTED-ANY]
module.iam.aws_iam_role_policy_attachment.attach-logs: Creation complete after 0s [id=express-like-lambda-example-REDACTED]
module.api-gateway.aws_api_gateway_resource.proxy: Creation complete after 1s [id=REDACTED]
module.api-gateway.aws_api_gateway_method.proxy: Creating...
module.api-gateway.aws_api_gateway_method.proxy: Creation complete after 0s [id=REDACTED-ANY]
module.lambda.aws_lambda_function.express-like-lambda-example: Still creating... [10s elapsed]
module.lambda.aws_lambda_function.express-like-lambda-example: Creation complete after 16s [id=express-like-lambda-example]
module.lambda.aws_lambda_permission.express-like-lambda-example: Creating...
module.lambda.aws_lambda_function_event_invoke_config.express-like-lambda-example-event-invoke-config: Creating...
module.api-gateway.aws_api_gateway_integration.lambda: Creating...
module.api-gateway.aws_api_gateway_integration.express-like-lambda-example: Creating...
module.lambda.aws_lambda_permission.express-like-lambda-example: Creation complete after 0s [id=AllowAPIGatewayInvoke]
module.api-gateway.aws_api_gateway_integration.express-like-lambda-example: Creation complete after 0s [id=REDACTED-ANY]
module.api-gateway.aws_api_gateway_deployment.express-like-lambda-example_v1: Creating...
module.api-gateway.aws_api_gateway_integration.lambda: Creation complete after 0s [id=REDACTED-ANY]
module.lambda.aws_lambda_function_event_invoke_config.express-like-lambda-example-event-invoke-config: Creation complete after 0s [id=arn:aws:lambda:us-east-1:REDACTED:function:express-like-lambda-example]
module.api-gateway.aws_api_gateway_deployment.express-like-lambda-example_v1: Creation complete after 1s [id=REDACTED]

Apply complete! Resources: 13 added, 0 changed, 0 destroyed.

Outputs:

endpoint = https://REDACTED.execute-api.us-east-1.amazonaws.com/v1

Copy or remember the endpoint from the output for use in a bit.

Deploying the App

Open the package.json and create this npm script.

"scripts": {
    "build": "npm install --production && rm -rf build && mkdir build && zip -r -q -x='*terraform*' -x='*.md' -x='LICENSE' -x='*build*' -x='*.DS_Store*' -x='*.git*' build/express-like-lambda-example.zip . && du -sh build"
  },

Now in root of the project you can run the build command to build the zip file in preparation to deploy it to the Lambda we created.

npm run build

For example:

npm run build

> express-like-lambda-example@1.0.0 build /Users/REDACTED/Development/express-like-lambda-example
> npm install --production && rm -rf build && mkdir build && zip -r -q -x='*media*' -x='*terraform*' -x=*coverage* -x='*.md' -x='LICENSE' -x='*build*' -x='*.DS_Store*' -x='*.git*' build/express-like-lambda-example.zip . && du -sh build

audited 1 package in 0.916s
found 0 vulnerabilities

 28K    build

Now we can deploy our zipped app to Lambda. For that I use this command.

aws lambda update-function-code --function-name=express-like-lambda-example --zip-file=fileb://build/express-like-lambda-example.zip --region=us-east-1 1> /dev/null

Calling the API

Now we can hit our API 🎉

curl https://REDACTED.execute-api.us-east-1.amazonaws.com/v1
hello world

More example use:

curl https://REDACTED.execute-api.us-east-1.amazonaws.com/v1/foo
/foo hit

curl https://REDACTED.execute-api.us-east-1.amazonaws.com/v1/bar
/bar hit

curl https://REDACTED.execute-api.us-east-1.amazonaws.com/v1/baz
{"error":"Route not found"}

Note, your URL will differ from the one above. It's unique each deployment. Your URL will come from the output of the terraform apply.

Conclusion

I had fun. I learned a little more Terraform and about a rad Node.js package lambda-api. If I made any mistakes I can learn from I'm happy to learn of those in the comments. If you have any questions please feel free to ask.

How I Built a Resume API w/ Go, Terraform, and AWS

Matt Jarrett — Sat, 27 Jun 2020 22:18:32 +0000

Background

I've been inspired by Sara Vieira to build more dumb stuff on the Internet to recharge my soul and learn while I'm doing it. I wanted to learn Go this year and after taking Stephen Grider's awesome Go: The Complete Developer's Guide (Golang) I wanted to build something simple to continue my Go journey. I decided to build an open source REST API for my JSON-based standard format resume. I'm using Go for my app, Terraform for my Infrastructure as Code (IaC) with Terraform Cloud to provision my infrastructure, either on demand or in response to various events. I'm deploying the app to AWS Lambda with a Circle CI CI/CD pipeline.

All of the code for this can be found in my GitHub.

Demo

You can curl the REST API to view my resume

curl -L https://resume.mattjarrett.dev

Setup

You'll need to set up Go and Terraform locally as well as have an account on Terraform Cloud and AWS if you want to deploy your code.

Creating your Resume in JSON

I copied the JSON-based standard format resume schema that I would edit in my info. After I was done editing that JSON, I ended up with this.

{
  "basics": {
    "name": "Matt Jarrett",
    "label": "Full Stack Developer",
    "picture": "https://mattjarrett.dev/img/header.jpg",
    "email": "",
    "phone": "",
    "website": "https://mattjarrett.dev/",
    "summary": "Dad. Husband. Full Stack Developer. JavaScript Enthusiast. Aspiring Designer. Cyclist. I'm a passionate and detail oriented full stack software developer based in Dallas, Texas. Experienced leader where I enjoy design, clean code, and building positive team environments and strong developer experiences. I enjoy turning problems into solutions that make people smile. I enjoy continuous learning and exploring new technologies. When I'm not coding at home or work, I enjoy time with my family and photography.",
    "location": {
      "address": "",
      "postalCode": "",
      "city": "Dallas",
      "countryCode": "US",
      "region": "Texas"
    },
    "profiles": [
      {
        "network": "Linkedin",
        "username": "Matt Jarrett",
        "url": "https://www.linkedin.com/in/matt-jarrett-303a75144/"
      },
      {
        "network": "GitHub",
        "username": "cujarrett",
        "url": "https://github.com/cujarrett"
      },
      {
        "network": "Twitter",
        "username": "cujarrett",
        "url": "https://twitter.com/home"
      }
    ]
  },
  "work": [
    {
      "company": "State Farm",
      "position": "Technology Engineer",
      "website": "https://www.statefarm.com",
      "startDate": "2011-11-01",
      "endDate": "N/A",
      "summary": "State Farm is the largest property and casualty insurance provider in the United States. It is also the largest auto insurance provider in the United States. State Farm is ranked 36th in the 2019 Fortune 500, which lists American companies by revenue.",
      "highlights": [
        "Advancing Cloud Native and DevOps transformations in the telematics platform space. Working with an exciting mix of technology including public cloud, mobile, and legacy systems. Working daily on not only technology but also culture.",
        "Built new solutions using React, Node, NOSQL, REST, Kubernetes, GitLab, CI/CD, Grafana, and Prometheus for enablement of large high priority enterprise initiatives. Mentored several team members in modern technology. Designed solutions for implementation. Taught classes on modern JavaScript.",
        "Designed and implemented many successful web services allowing developers on demand access to safe fabricated data for a variety of needs and automation. It's since been used millions of times across the enterprise, enabling development teams to focus on improving customer experience.",
        "Designed and implemented an automated infrastructure solution offering a complete stand up and tear down process accomplished in minutes compared to days.",
        "Taught Java in Enterprise Java classes to spread knowledge. Active member of the college recruitment team including creation of campus events, hacks, and interviews. Mentored many college interns through a successful internships resulting in all receiving and accepting full time positions. Designed and implemented multiple solutions in Java."
      ]
    }
  ],
  "volunteer": [
    {
      "organization": "Tech Titans",
      "position": "Speaker",
      "website": "https://techtitans.org/",
      "startDate": "2017-01-01",
      "endDate": "N/A",
      "summary": "Tech Titans is a forum that leverages the regional technology community to collaborate, share and inspire creative thinking that fuels tomorrow’s innovations.",
      "highlights": [
        "I Spoke at many events over the years with audiences including teachers and students."
      ]
    }
  ],
  "education": [
    {
      "institution": "Illinois State University",
      "area": "Information Technology",
      "studyType": "Bachelor",
      "startDate": "2009-06-01",
      "endDate": "2011-01-01",
      "gpa": "3.87/4.0",
      "courses": []
    }
  ],
  "awards": [],
  "publications": [],
  "skills": [
    {
      "name": "AWS",
      "level": "Intermediate",
      "keywords": [
        "Cloud",
        "Lambda",
        "S3",
        "CloudFront"
      ]
    },
    {
      "name": "JavaScript",
      "level": "Expert",
      "keywords": [
        "ECMAScript",
        "ES6",
        "Node.js",
        "Web",
        "Front End"
      ]
    },
    {
      "name": "React",
      "level": "Pro",
      "keywords": [
        "SPA",
        "Web",
        "Front End"
      ]
    },
    {
      "name": "Vue",
      "level": "Intermediate",
      "keywords": [
        "SPA",
        "Web",
        "Front End"
      ]
    },
    {
      "name": "Angular",
      "level": "Intermediate",
      "keywords": [
        "SPA",
        "Web",
        "Front End"
      ]
    },
    {
      "name": "Go",
      "level": "Intermediate",
      "keywords": [
        "Golang"
      ]
    },
    {
      "name": "Kubernetes",
      "level": "Pro",
      "keywords": [
        "K8s",
        "CKAD"
      ]
    },
    {
      "name": "Docker",
      "level": "Intermediate",
      "keywords": [
        "Containers"
      ]
    },
    {
      "name": "Design",
      "level": "Intermediate",
      "keywords": [
        "Web",
        "Front End"
      ]
    },
    {
      "name": "Linux",
      "level": "Intermediate",
      "keywords": [
        "Bash",
        "Shell"
      ]
    },
    {
      "name": "NOSQL",
      "level": "Pro",
      "keywords": [
        "Couch",
        "Dynamo"
      ]
    },
    {
      "name": "SQL",
      "level": "Pro",
      "keywords": [
        "PostgreSQL",
        "Aurora"
      ]
    },
    {
      "name": "GitHub",
      "level": "Expert",
      "keywords": [
        "SDLC",
        "OSS",
        "CI/CD"
      ]
    },
    {
      "name": "GitLab",
      "level": "Expert",
      "keywords": [
        "SDLC",
        "OSS",
        "CI/CD"
      ]
    },
    {
      "name": "Pipelines",
      "level": "Expert",
      "keywords": [
        "DevOps"
      ]
    },
    {
      "name": "Grafana",
      "level": "Expert",
      "keywords": [
        "DevOps",
        "Observability",
        "Dashboards",
        "SRE",
        "SLO"
      ]
    },
    {
      "name": "Prometheus",
      "level": "Expert",
      "keywords": [
        "DevOps",
        "Observability",
        "Dashboards",
        "SRE",
        "SLO"
      ]
    },
    {
      "name": "Java",
      "level": "Pro",
      "keywords": []
    },
    {
      "name": "Photoshop",
      "level": "Expert",
      "keywords": []
    },
    {
      "name": "Python",
      "level": "Intermediate",
      "keywords": []
    }
  ],
  "languages": [
    {
      "language": "English",
      "fluency": "Native speaker"
    }
  ],
  "interests": [
    {
      "name": "Cycling",
      "keywords": [
        "MTB",
        "Gravel",
        "Road"
      ]
    },
    {
      "name": "Photography",
      "keywords": [
        "Weddings",
        "Landscape",
        "Senior"
      ]
    }
  ],
  "references": [
    {
      "name": "Caleb Lemoine",
      "reference": "Matt is one of the best engineers I've had the pleasure of working with and knowing personally. He would be a highly valued asset anywhere."
    },
    {
      "name": "Lucas Reardon",
      "reference": "Matt Jarrett's passion for technology is infectious. His brand as a forward-thinking leader and influencer raises the performance of those around him as well as the products and capabilities he touches. If you're building a team, you want Matt. Full stop."
    }
  ]
}

Creating the Structs

Once I was done with my edits I used this rad JSON to Go struct app created by Matt Holt to convert my JSON to to Go's equivalent of JSON, the struct.

After I made struct's for each section.

type Resume struct {
    Basics       Basics      `json:"basics"`
    Work         []Work      `json:"work"`
    Volunteer    []Volunteer `json:"volunteer"`
    Education    []Education `json:"education"`
    Awards       []string    `json:"awards"`
    Publications []string    `json:"publications"`
    Skills       []Skill     `json:"skills"`
    Languages    []Language  `json:"languages"`
    Interests    []Interest  `json:"interests"`
    References   []Reference `json:"references"`
}

type Basics struct {
    Name     string    `json:"name"`
    Label    string    `json:"label"`
    Picture  string    `json:"picture"`
    Email    string    `json:"email"`
    Phone    string    `json:"phone"`
    Website  string    `json:"website"`
    Summary  string    `json:"summary"`
    Location Location  `json:"location"`
    Profiles []Profile `json:"profiles"`
}

type Location struct {
    Address     string `json:"address"`
    PostalCode  string `json:"postalCode"`
    City        string `json:"city"`
    CountryCode string `json:"countryCode"`
    Region      string `json:"region"`
}

type Profile struct {
    Network  string `json:"network"`
    Username string `json:"username"`
    URL      string `json:"url"`
}

type Work struct {
    Company    string   `json:"company"`
    Position   string   `json:"position"`
    Website    string   `json:"website"`
    StartDate  string   `json:"startDate"`
    EndDate    string   `json:"endDate"`
    Summary    string   `json:"summary"`
    Highlights []string `json:"highlights"`
}

type Volunteer struct {
    Organization string   `json:"organization"`
    Position     string   `json:"position"`
    Website      string   `json:"website"`
    StartDate    string   `json:"startDate"`
    EndDate      string   `json:"endDate"`
    Summary      string   `json:"summary"`
    Highlights   []string `json:"highlights"`
}

type Education struct {
    Institution string   `json:"institution"`
    Area        string   `json:"area"`
    StudyType   string   `json:"studyType"`
    StartDate   string   `json:"startDate"`
    EndDate     string   `json:"endDate"`
    Gpa         string   `json:"gpa"`
    Courses     []string `json:"courses"`
}

type Skill struct {
    Name     string   `json:"name"`
    Level    string   `json:"level"`
    Keywords []string `json:"keywords"`
}

type Language struct {
    Language string `json:"language"`
    Fluency  string `json:"fluency"`
}

type Interest struct {
    Name     string   `json:"name"`
    Keywords []string `json:"keywords"`
}

type Reference struct {
    Name      string `json:"name"`
    Reference string `json:"reference"`
}

Next I made a func that filled the Structs

func getResume() *Resume {
    return &Resume{
        Basics: Basics{
            Name:    "Matt Jarrett",
            Label:   "Full Stack Developer",
            Picture: "https://mattjarrett.dev/img/header.jpg",
            Email:   "",
            Phone:   "",
            Website: "https://mattjarrett.dev/",
            Summary: "Dad. Husband. Full Stack Developer. JavaScript Enthusiast. Aspiring Designer. Cyclist. I'm a passionate and detail oriented full stack software developer based in Dallas, Texas. Experienced leader where I enjoy design, clean code, and building positive team environments and strong developer experiences. I enjoy turning problems into solutions that make people smile. I enjoy continuous learning and exploring new technologies. When I'm not coding at home or work, I enjoy time with my family and photography.",
            Location: Location{
                Address:     "",
                PostalCode:  "",
                City:        "Dallas",
                CountryCode: "US",
                Region:      "Texas",
            },
            Profiles: []Profile{
                {
                    Network:  "Linkedin",
                    Username: "Matt Jarrett",
                    URL:      "https://www.linkedin.com/in/matt-jarrett-303a75144/",
                }, {
                    Network:  "GitHub",
                    Username: "cujarrett",
                    URL:      "https://github.com/cujarrett",
                }, {
                    Network:  "Twitter",
                    Username: "cujarrett",
                    URL:      "https://twitter.com/home",
                },
            },
        },
        Work: []Work{
            {
                Company:   "State Farm",
                Position:  "Technology Engineer",
                Website:   "https://www.statefarm.com",
                StartDate: "2011-11-01",
                EndDate:   "N/A",
                Summary:   "State Farm is the largest property and casualty insurance provider in the United States. It is also the largest auto insurance provider in the United States. State Farm is ranked 36th in the 2019 Fortune 500, which lists American companies by revenue.",
                Highlights: []string{
                    "Advancing Cloud Native and DevOps transformations in the telematics platform space. Working with an exciting mix of technology including public cloud, mobile, and legacy systems. Working daily on not only technology but also culture.",
                    "Built new solutions using React, Node, NOSQL, REST, Kubernetes, GitLab, CI/CD, Grafana, and Prometheus for enablement of large high priority enterprise initiatives. Mentored several team members in modern technology. Designed solutions for implementation. Taught classes on modern JavaScript.",
                    "Designed and implemented many successful web services allowing developers on demand access to safe fabricated data for a variety of needs and automation. It's since been used millions of times across the enterprise, enabling development teams to focus on improving customer experience.",
                    "Designed and implemented an automated infrastructure solution offering a complete stand up and tear down process accomplished in minutes compared to days.",
                    "Taught Java in Enterprise Java classes to spread knowledge. Active member of the college recruitment team including creation of campus events, hacks, and interviews. Mentored many college interns through a successful internships resulting in all receiving and accepting full time positions. Designed and implemented multiple solutions in Java."},
            },
        },
        Volunteer: []Volunteer{
            {
                Organization: "Tech Titans",
                Position:     "Speaker",
                Website:      "https://techtitans.org/",
                StartDate:    "2017-01-01",
                EndDate:      "N/A",
                Summary:      "Tech Titans is a forum that leverages the regional technology community to collaborate, share and inspire creative thinking that fuels tomorrow’s innovations.",
                Highlights:   []string{"I Spoke at many events over the years with audiences including teachers and students."},
            },
        },
        Education: []Education{
            {
                Institution: "Illinois State University",
                Area:        "Information Technology",
                StudyType:   "Bachelor",
                StartDate:   "2009-06-01",
                EndDate:     "2011-01-01",
                Gpa:         "3.87/4.0",
                Courses:     []string{},
            },
        },
        Awards:       []string{},
        Publications: []string{},
        Skills: []Skill{
            {
                Name:  "AWS",
                Level: "Intermediate",
                Keywords: []string{
                    "Cloud", "Lambda", "S3", "CloudFront",
                },
            }, {
                Name:  "JavaScript",
                Level: "Expert",
                Keywords: []string{
                    "ECMAScript", "ES6", "Node.js", "Web", "Front End",
                },
            }, {
                Name:  "React",
                Level: "Pro",
                Keywords: []string{
                    "SPA", "Web", "Front End",
                },
            }, {
                Name:  "Vue",
                Level: "Intermediate",
                Keywords: []string{
                    "SPA", "Web", "Front End",
                },
            }, {
                Name:  "Angular",
                Level: "Intermediate",
                Keywords: []string{
                    "SPA", "Web", "Front End",
                },
            }, {
                Name:  "Go",
                Level: "Intermediate",
                Keywords: []string{
                    "Golang",
                },
            }, {
                Name:  "Kubernetes",
                Level: "Pro",
                Keywords: []string{
                    "K8s", "CKAD",
                },
            }, {
                Name:  "Docker",
                Level: "Intermediate",
                Keywords: []string{
                    "Containers",
                },
            }, {
                Name:  "Design",
                Level: "Intermediate",
                Keywords: []string{
                    "Web", "Front End",
                },
            }, {
                Name:  "Linux",
                Level: "Intermediate",
                Keywords: []string{
                    "Bash", "Shell",
                },
            }, {
                Name:  "NOSQL",
                Level: "Pro",
                Keywords: []string{
                    "Couch", "Dynamo",
                },
            }, {
                Name:  "SQL",
                Level: "Pro",
                Keywords: []string{
                    "PostgreSQL", "Aurora",
                },
            }, {
                Name:  "GitHub",
                Level: "Expert",
                Keywords: []string{
                    "SDLC", "OSS", "CI/CD",
                },
            }, {
                Name:  "GitLab",
                Level: "Expert",
                Keywords: []string{
                    "SDLC", "OSS", "CI/CD",
                },
            }, {
                Name:  "Pipelines",
                Level: "Expert",
                Keywords: []string{
                    "DevOps",
                },
            }, {
                Name:  "Grafana",
                Level: "Expert",
                Keywords: []string{
                    "DevOps", "Observability", "Dashboards", "SRE", "SLO",
                },
            }, {
                Name:  "Prometheus",
                Level: "Expert",
                Keywords: []string{
                    "DevOps", "Observability", "Dashboards", "SRE", "SLO",
                },
            }, {
                Name:     "Java",
                Level:    "Pro",
                Keywords: []string{},
            }, {
                Name:     "Photoshop",
                Level:    "Expert",
                Keywords: []string{},
            }, {
                Name:     "Python",
                Level:    "Intermediate",
                Keywords: []string{},
            },
        },
        Languages: []Language{
            {
                Language: "English",
                Fluency:  "Native speaker",
            },
        },
        Interests: []Interest{
            {
                Name: "Cycling",
                Keywords: []string{
                    "MTB", "Gravel", "Road",
                },
            }, {
                Name: "Photography",
                Keywords: []string{
                    "Weddings", "Landscape", "Senior",
                },
            },
        },
        References: []Reference{
            {
                Name:      "Caleb Lemoine",
                Reference: "Matt is one of the best engineers I've had the pleasure of working with and knowing personally. He would be a highly valued asset anywhere.",
            },
            {
                Name:      "Lucas Reardon",
                Reference: "Matt Jarrett's passion for technology is infectious. His brand as a forward-thinking leader and influencer raises the performance of those around him as well as the products and capabilities he touches. If you're building a team, you want Matt. Full stop.",
            },
        },
    }
}

Formatting the response JSON

I wanted to return a formatted JSON for readability. I wrote this func to do so.

func (input Resume) formatResume() string {
    bytesBuffer := new(bytes.Buffer)
    json.NewEncoder(bytesBuffer).Encode(&input)

    responseBytes := bytesBuffer.Bytes()

    var prettyJSON bytes.Buffer
    error := json.Indent(&prettyJSON, responseBytes, "", "  ")
    if error != nil {
        log.Println("JSON parse error: ", error)
    }
    formattedResume := string(prettyJSON.Bytes())
    return formattedResume
}

Serving the API

I used AWS's API Gateway to handle requests via Lambda with this func.

func handleRequest(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    resume := getResume()
    formattedResume := resume.formatResume()

    response := events.APIGatewayProxyResponse{
        StatusCode: http.StatusOK,
        Headers:    map[string]string{"Content-Type": "application/json"},
        Body:       formattedResume,
    }
    return response, nil
}

Main

I set up my main to handle requests by calling handleRequest.

func main() {
    lambda.Start(handleRequest)
}

Provisioning the Infrastructure with Terraform

I used Terraform for my Infrastructure as Code (IaC). I'm using AWS Lambda for compute and serving that compute via AWS API Gateway.

As I journey into the cloud and Terraform use I sometimes find the line blurred between provisioning infrastructure vs applications. My mind prefers separation between provisioning my infrastructure, either on demand or in response to various events and my CI/CD pipeline for running continuous integration and continuous deployments. To accomplish this for my resume-api, I had to work around AWS Lambda wanting the compute code at time of creation. I choose to handle this by using a placeholder at infrastructure provisioning and then using a Circle CI CI/CD pipeline for building and updating the Lambda with the actual code as changes occur with pull requests from GitHub.

I am using Terraform Cloud to provision my infrastructure, either on demand or in response to various events. After setting up my Terraform Cloud project workspace I updated my resume-api workspace to have all applies be done manually so I still get automated plans on each change, but infrastructure is only updated when I manually click apply in Terraform Cloud.

This is what the Terraform Cloud runs look like.

variable "aws_region" {
  description = "AWS region for the infrastructure"
  type = string
  default = "us-east-1"
}

data "archive_file" "placeholder" {
  type = "zip"
  output_path = "${path.module}/lambda-function-payload.zip"

  source {
    content = "placeholder"
    filename = "placeholder.txt"
  }
}

provider "aws" {
  region = var.aws_region
}

# Define a Lambda function.
#
# The handler is the name of the executable for go1.x runtime.
resource "aws_lambda_function" "resume-api" {
  filename = data.archive_file.placeholder.output_path
  function_name = "resume-api"
  handler       = "resume-api"
  role          = aws_iam_role.resume-api.arn
  runtime       = "go1.x"
  memory_size   = 128
  timeout       = 1
}

# A Lambda function may access to other AWS resources such as S3 bucket. So an
# IAM role needs to be defined. This example does not access to any resource,
# so the role is empty.
#
# The date 2012-10-17 is just the version of the policy language used here [1].
#
# [1]: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_version.html
resource "aws_iam_role" "resume-api" {
  name               = "resume-api"
  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": {
    "Action": "sts:AssumeRole",
    "Principal": {
      "Service": "lambda.amazonaws.com"
    },
    "Effect": "Allow"
  }
}
POLICY
}

# Allow API gateway to invoke the resume-api Lambda function.
resource "aws_lambda_permission" "resume-api" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.resume-api.arn
  principal     = "apigateway.amazonaws.com"
}

# A Lambda function is not a usual public REST API. We need to use AWS API
# Gateway to map a Lambda function to an HTTP endpoint.
resource "aws_api_gateway_resource" "resume-api" {
  rest_api_id = aws_api_gateway_rest_api.resume-api.id
  parent_id   = aws_api_gateway_rest_api.resume-api.root_resource_id
  path_part   = "resume-api"
}

resource "aws_api_gateway_rest_api" "resume-api" {
  name = "resume-api"
}

#           GET
# Internet -----> API Gateway
resource "aws_api_gateway_method" "resume-api" {
  rest_api_id   = aws_api_gateway_rest_api.resume-api.id
  resource_id   = aws_api_gateway_resource.resume-api.id
  http_method   = "GET"
  authorization = "NONE"
}

#              POST
# API Gateway ------> Lambda
# For Lambda the method is always POST and the type is always AWS_PROXY.
#
# The date 2015-03-31 in the URI is just the version of AWS Lambda.
resource "aws_api_gateway_integration" "resume-api" {
  rest_api_id             = aws_api_gateway_rest_api.resume-api.id
  resource_id             = aws_api_gateway_resource.resume-api.id
  http_method             = aws_api_gateway_method.resume-api.http_method
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = "arn:aws:apigateway:${var.aws_region}:lambda:path/2015-03-31/functions/${aws_lambda_function.resume-api.arn}/invocations"
}

# This resource defines the URL of the API Gateway.
resource "aws_api_gateway_deployment" "resume-api_v1" {
  depends_on = [
    aws_api_gateway_integration.resume-api
  ]
  rest_api_id = aws_api_gateway_rest_api.resume-api.id
  stage_name  = "v1"
}

# Set the generated URL as an output. Run `terraform output url` to get this.
output "url" {
  value = "${aws_api_gateway_deployment.resume-api_v1.invoke_url}${aws_api_gateway_resource.resume-api.path}"
}

Makefile

I made a Makefile for running project related tasks.

clean:
    go clean

install:
    go get -v -t -d ./...

test:
    go test -v ./...

compile:
    GOOS=linux go build -o resume-api main.go

CI/CD With Circle CI

After I've provisioned the infrastructure I am free to update the Lambda code with a Circle CI/CD pipeline after automated quality tests pass.

# Golang CircleCI 2.0 configuration file
# Check https://circleci.com/docs/2.0/language-go/ for more details
version: 2
jobs:

  test:
    docker:
      - image: circleci/golang:1.14
    working_directory: /go/src/github.com/cujarrett/resume-api
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: make install
      - run:
          name: Run tests
          command:  make test

  build:
    docker:
      - image: circleci/golang:1.14
    working_directory: ~/temp
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: make install
      - run:
          name: Build executable
          command: make compile
      - run:
          name: Make temp build folder
          command: mkdir build
      - run:
          name: Zip executable
          command: zip build/resume-api.zip -q resume-api
      - persist_to_workspace:
          root: .
          paths:
            - build/resume-api.zip

  deploy:
    docker:
      - image: 'circleci/python:3.7.6'
    working_directory: ~/temp
    steps:
      - attach_workspace:
          at: ~/temp
      - run:
          name: Install AWS CLI
          command: sudo pip install awscli
      - deploy:
          name: Deploy to AWS S3
          command: |
            aws lambda update-function-code \
              --function-name=resume-api \
              --zip-file=fileb://build/resume-api.zip 1> /dev/null \
              --region=us-east-1

workflows:
  version: 2
  cicd:
    jobs:
      - test
      - build:
          requires:
            - test
      - deploy:
          requires:
            - build
          filters:
            branches:
              only: master

Now every branch's pull request triggers a terraform plan and all quality tests as well as deploying the app out to AWS when it is merged.

Conclusion

I had fun. I learned a little more Go, Terraform, and Terraform Cloud than I knew before. If I made any mistakes I can learn from I'm happy to learn of those

Forem: Matt Jarrett

Build an Express like app on AWS Lambda

What

How much work is it?

Prerequisites

lambda-api

Terraform

provider.tf

variables.tf

Modules

archive_file

iam

lambda

api-gateway

main.tf

Running the code

Deploying the Infrastructure

Deploying the App

Calling the API

Conclusion

How I Built a Resume API w/ Go, Terraform, and AWS

Background

Demo

Setup

Creating your Resume in JSON

Creating the Structs

Formatting the response JSON

Serving the API

Main

Provisioning the Infrastructure with Terraform

Makefile

CI/CD With Circle CI

Conclusion

Links