Forem: Piotr Mionskowski

Autoscaling AWS EKS cluster with custom Gitlab runner configuration

Piotr Mionskowski — Thu, 18 Jun 2020 05:07:42 +0000

Gitlab offers a robust CI/CD solution suitable for most projects.
The documentation depicts the easiest way to connect your Kubernetes cluster to Gitlab.
Once connected Gitlab can install and configure Gitlab runner.

However, such approach does not allow for many adjustments to Gitlab runner configuration.
For example when using Kubernetes executor providing build level resource requests and limits does not work.

In this post we'll use aws-cdk to:

provision an AWS EKS cluster
install cluster autoscaler
install Gitlab runner using Helm chart

Prerequisite

Create an aws-cdk typescript project with necessary dependencies using

mkdir cicd && cd cicd
npx cdk init --language typescript
npm i --save @aws-cdk/aws-eks

Create EKS Cluster with aws-cdk

First we add an AWS EKS cluster with a managed node group.
A managed node group provides a convenient way to provision and manage nodes.

export class CICDStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const clusterAdmin = new iam.Role(this, "Cluster Master Role", {
      assumedBy: new iam.AccountRootPrincipal(),
    });

    const clusterName = "cicd";
    const cluster = new eks.Cluster(this, "CICD Cluster", {
      mastersRole: clusterAdmin,
      clusterName: clusterName,
      defaultCapacity: 0,
    });

    const mainNodeGroup = cluster.addNodegroup("main", {
      desiredSize: 1,
      instanceType: ec2.InstanceType.of(InstanceClass.M5, InstanceSize.XLARGE2),
      nodegroupName: "main",
      maxSize: 10
    });

    ...

  }
}

The clusterAdmin IAM Role is going to be use to manage the cluster with kubectl.
We do not need it to install and configure Gitlab runner. However, it is essential for debugging purposes.

Install Gitlab runner using Helm chart

For our EKS cluster to perform any CI work we need to install Gitlab runner.
Thankfully Gitlab provides an official Helm chart.

cluster.addChart("gitlab-runner", {
  chart: "gitlab-runner",
  repository: "https://charts.gitlab.io/",
  version: "v0.17.1",
  wait: true,
  values: {
    gitlabUrl: "https://gitlab.com/",
    runnerRegistrationToken: "<runner registration token>",
    rbac: {
      create: true,
    },
    runners: {
      // https://gitlab.com/gitlab-org/charts/gitlab-runner/blob/master/values.yaml
      // required for dind
      privileged: true,
      builds: {
        cpuRequests: "1",
        cpuRequestsOverwriteMaxAllowed: "16",
        cpuLimitOverwriteMaxAllowed: "16",
        memoryRequests: "4Gi",
        memoryLimitOverwriteMaxAllowed: "16Gi",
        memoryRequestsOverwriteMaxAllowed: "16Gi",
      },
    },
  },
});

For the Gitlab runner to register in Gitlab server we need to get a runner registration token.
You can find the runner registration token on a project or group CI/CD settings page.
In the above example I passed the value as is to the runnerRegistrationToken setting.
However, you should not commit that into source control and instead use an environment variable,
or a parameter store reference.

Note that we have configured a couple of values defined in the Helm chart.
Most notably we set the required cpu*OverwriteMaxAllowed and memory*OverwriteMaxAllowed options.
We have to configure them otherwise it will not be possible to set Kubernetes CPU and memory allocations for requests and limits on the .gitlab-ci.yml.
With the above set we can request a certain amount of CPU and memory for build:

 variables:
   KUBERNETES_CPU_REQUEST: 3
   KUBERNETES_CPU_LIMIT: 5
   KUBERNETES_MEMORY_REQUEST: 2Gi
   KUBERNETES_MEMORY_LIMIT: 4Gi

Install Cluster autoscaler using aws-cdk construct

In order for our AWS EKS cluster to scale dynamically we need to install Cluster autoscaler.
With autoscaler installed, whenever the amount of declared requested CPU or memory resources exceeds those available on nodes, a new node will be added to the cluster.

I've wrapped the Cluster autoscaler setup into aws-cdk construct
With the ClusterAutoscaler construct we can simply enhance our EKS Cluster by adding the following to the CICDStack definition:

new ClusterAutoscaler(this, "cluster-autoscaler", {
  cluster: cluster,
  nodeGroups: [mainNodeGroup],
  clusterName,
});

Connect to the cluster with kubectl

After a deployment the new Gitlab runner becomes available for a project or Gitlab group.
The cdk deploy command will also output a hint on how to configure kubectl to connect to the cluster.
It will look somewhat similar to the example below:

aws eks update-kubeconfig --name cicd --region eu-west-1 \
  --role-arn arn:aws:iam::XXXXXXXXXXX:role/CICD-ClusterMasterRoleXXXX-XXXX

You only need to execute the above command once. It will store the configuration for kubectl on your local disk.
Afterwards you can inspect the EKS Kubernetes cluster with kubectl e.g:

> kubectl get pods 
NAME                                                              READY   STATUS    RESTARTS   AGE
cicdcicdclusterchartgitlabrunnerdXXXXX-gitlab-runner-XXXXXXXXXX   1/1     Running   0          26h

You can find full working example under the github repository.

Deploy aws-cdk app to different environments

Piotr Mionskowski — Sun, 07 Jun 2020 18:42:26 +0000

Infrastructure as a code greatly improves traceability, repeatability and thus overall quality of the software systems built with it.
With a correct setup development teams can now more easily spin up short-lived clones of systems or their parts.
This gives a profound boost in both productivity but also ability to run automated regression or performance tests.

In this post I will show how to use aws-cdk to deploy multiple copies of the same application:

1. Using multiple AWS accounts.

2. Using a single AWS account.

Using multiple AWS accounts to deploy copies of the same aws-cdk application

By default, every aws-cdk Stack you create is environment-agnostic.
You can deploy an environment-agnostic stack with any AWS account granted you have enough entitlements.
Note that AWS account does not mean IAM user. A single AWS account can have multiple IAM users.

The cdk deploy command uses specified AWS profile or a default one.

Let us start with a bare bone app multiple-accounts generated with:

mkdir multiple-accounts && cd multiple-accounts
npx cdk init --language typescript

To make our example more entertaining I added a single S3 Bucket to the MultipleAccountsStack:

import * as cdk from "@aws-cdk/core";
import * as s3 from "@aws-cdk/aws-s3";

export class MultipleAccountsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new s3.Bucket(this, "MyBucket");
  }
}

I also created 2 AWS profiles for 2 AWS accounts, using aws configure, named blog-staging and blog-production for demonstration.

Deploy aws-cdk app using explicitly specified profile

For start let's use --profile parameter to deploy our stacks:

> npx cdk deploy --profile blog-staging

MultipleAccountsStack: deploying...
MultipleAccountsStack: creating CloudFormation changeset...
 ...

 ✅  MultipleAccountsStack

Stack ARN:
arn:aws:cloudformation:eu-central-1:XXXXXXXXXX:stack/MultipleAccountsStack/0d3c7530-a318-11ea-ab8d-02b8022d5b40

> npx cdk deploy --profile blog-production

MultipleAccountsStack: deploying...
MultipleAccountsStack: creating CloudFormation changeset...
...

 ✅  MultipleAccountsStack

Stack ARN:
arn:aws:cloudformation:eu-central-1:YYYYYYYYYY:stack/MultipleAccountsStack/3a5cc690-a319-11ea-b00b-023cc2de05b4

Note how XXXXXXXXXX and YYYYYYYYYY identifies, which AWS account deployed the stack.
The stack has the same name in both cases: MultipleAccountsStack

You can specify the AWS profile to use with AWS_PROFILE environment variable:

AWS_PROFILE=blog-production npx cdk deploy

Last but not least the standard AWS CLI environment variables work as well:

export AWS_ACCESS_KEY_ID=xxxx
export AWS_SECRET_ACCESS_KEY=yyy
export AWS_DEFAULT_REGION=eu-central-1
npx cdk deploy

Please refer to the documentation for further details.

Benefits of using multiple AWS accounts to deploy aws-cdk app

The first advantage is how using multiple AWS accounts keeps aws-cdk code simple.
The infrastructure code ignores details of the environment it runs in. This makes it possible to re-use it in other contexts.
This approach also improves security.
With multiple AWS accounts we minimise risk of aws-cdk installations affecting each other.

Downsides of using multiple AWS accounts to deploy aws-cdk app

The following example presents a first downside, or in reality a gotcha easy to miss.
Say someone not well versed in AWS services decided to name the bucket:

new s3.Bucket(this, "MyBucket", {
  bucketName: "important-assets",
});

It will work fine when deployed for the first time. However, as soon as we try to deploy it to a second AWS account we get:

> npx cdk deploy --profile blog-production
(node:61539) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time
MultipleAccountsStack: deploying...
MultipleAccountsStack: creating CloudFormation changeset...
 0/3 | 10:56:27 AM | CREATE_IN_PROGRESS   | AWS::S3::Bucket    | MyBucket (MyBucketF68F3FF0)
 0/3 | 10:56:27 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata | CDKMetadata
 1/3 | 10:56:27 AM | CREATE_FAILED        | AWS::S3::Bucket    | MyBucket (MyBucketF68F3FF0) important-assets already exists

Some AWS resources names need a unique name in a given region across all AWS accounts.

The difficulty to create AWS accounts on a developer whim is the second downside.
Remember that want to re-create environments with changed parts.
Forcing a new AWS account may turn out problematic and create a management headache too big to ignore.

Using single AWS account to deploy copies of the same aws-cdk application

As we saw above by default the Stack name gets derived from the class name.
Thankfully one can easily override that behaviour.
In my projects I prefer using a DEPLOY_ENV environment variable to specify the deployment target.

DEPLOY_ENV=staging npx cdk deploy

We need to adjust our app definition so that DEPLOY_ENV is recognized.

Using DEPLOY_ENV variable inside aws-cdk entry point

The aws-cdk app typically resided as bin folder. In our example it is the bin/multiple-accounts.ts file.
With the following adjustment we can deploy our app to many targets:

#!/usr/bin/env node
import "source-map-support/register";
import * as cdk from "@aws-cdk/core";
import { MultipleAccountsStack } from "../lib/multiple-accounts-stack";

const deployEnv = process.env.DEPLOY_ENV || "dev";
const app = new cdk.App();
new MultipleAccountsStack(app, deployEnv + "-MultipleAccountsStack");

During deployment aws-cdk will create a CloudFormation stack named ${process.env.DEPLOY_ENV}-MultipleAccountsStack:

Using DEPLOY_ENV consistently inside aws-cdk application

As I mentioned above one can easily fall into unique name traps.
In order to avoid naming conflicts and to apply consistent naming scheme I often define a helper similar to the following.

function deployEnv() {
  return process.env.DEPLOY_ENV || "dev";
}

function envSpecific(logicalName: string | Function) {
  const suffix =
    typeof logicalName === "function" 
    ? logicalName.name 
    : logicalName;

  return `${deployEnv()}-${suffix}`;
}

export class MultipleAccountsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, props?: cdk.StackProps) {
    super(scope, envSpecific(MultipleAccountsStack), props);

    new s3.Bucket(this, "MyBucket", {
      bucketName: envSpecific("important-assets"),
    });
  }
}

With the above change our cdk app entry point no longer decides how to name the stack.
Note however, that you can still allow passing the name through constructor when you need it.

Using envSpecific helper keeps important resource names consistent.
More importantly it allows deploying as many copies of aws-cdk app as we like.

Combine AWS Step Functions with CloudWatch Events using aws-cdk

Piotr Mionskowski — Sun, 24 May 2020 20:11:13 +0000

AWS Step Functions allow one to execute & coordinate long-running processes.
Step Functions fall into serverless AWS services, and the platform manages the function execution state completely.

In the example below we will use the following AWS services:

aws-cdk
Step Function
Lambda

The example demonstrates how Step Functions manage execution of a process, which involves external events e.g. human interaction.

Define step function

I will use blank infrastructure project created with

mkdir step-function && cd step-function
npx cdk init --language typescript

Next we need to add required packages:

cd step-function
npm i --save @aws-cdk/aws-stepfunctions \
  @aws-cdk/aws-stepfunctions-tasks \
  @aws-cdk/aws-lambda-nodejs \
  @aws-cdk/aws-dynamodb

We start with a state machine that contains a single Wait step as follows.

import { WaitTime } from "@aws-cdk/aws-stepfunctions";
import { Duration } from "@aws-cdk/core";
import * as cdk from "@aws-cdk/core";
import * as sfn from "@aws-cdk/aws-stepfunctions";

export class StepFunctionStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const wait = new sfn.Wait(this, "Wait for Event", {
      time: WaitTime.duration(Duration.seconds(5)),
    });

    new sfn.StateMachine(this, "Step Function", {
      definition: wait,
    });
  }
}

We deploy it with the usual aws-cdk deploy invocation

AWS_PROFILE=my-aws-profile npm run cdk deploy

After a few moments we can find our brand new AWS Step Function visual representation in AWS Web Console:

Define workflow in AWS Step Function

In order to make our example more interesting let's add a bit complexity to it.
Imagine we handle a document flow, which requires two separate actors to sign it off.

The actors act independently. The first signer, Alice, signs documents quickly.
The other, Bob, can take a couple of days to process a document.

Once both actors sign the document our flow succeeds. However, if we do not collect the signatures
until some time elapses the flow fails.

In order for the `@aws-cdk/aws-lambda-nodejs` to work properly you need to have Docker running

Alice's part consists of a single task along with a lambda function.

const signDocument = new lambda.NodejsFunction(this, "Sign Document", {
  entry: __dirname + "/sign-document.ts",
  handler: "signDocument",
});

const aliceSignature = new sfn.Task(this, "Alice Signature", {
  task: new tasks.RunLambdaTask(signDocument, {
    payload: sfn.TaskInput.fromObject({
      signer: "Alice",
      documentId: Data.stringAt("$.documentId"),
    }),
  }),
});

The signDocument lambda function handler defined in step-function/lib/sign-document.ts looks as follows:

import { Handler } from "aws-lambda";

const delay = (millis: number) =>
  new Promise((resolve) => setTimeout(resolve, millis));

export const signDocument: Handler<{
  documentId: string;
  signer: string;
}> = async (event) => {
  await delay(1000);
  console.log("Sign document", event);
  return {
    result: `${event.signer} signed ${event.documentId}`,
  };
};

Please note that the lambda function can be async as long as it completes within the regular AWS Lambda function rules.

Bob's signature is a more complex job. Since Bob can take several days to process a request to sign a document we cannot simply use lambda.

In order to solve that we will split the processing into 2 lambda functions using ServiceIntegrationPattern.WAIT_FOR_TASK_TOKEN.
The first one will receive step function token and store it in a DynamoDb table. That simulates reaching out to Bob e.g. through email.

The second one will retrieve the token from the DynamoDb table and mark the Step function task as complete.

const signRequestsTable = new dynamodb.Table(this, "Signature Requests", {
  billingMode: BillingMode.PAY_PER_REQUEST,
  partitionKey: { name: "documentId", type: AttributeType.STRING },
});

const requestDocumentSignature = new lambda.NodejsFunction(
  this,
  "Request Document Signature",
  {
    entry: __dirname + "/sign-document.ts",
    handler: "requestDocumentSignature",
    environment: {
      signRequestsTableName: signRequestsTable.tableName,
    },
  }
);

const completeSignatureRequest = new lambda.NodejsFunction(
  this,
  "Complete Document Signature",
  {
    entry: __dirname + "/sign-document.ts",
    handler: "completeSignatureRequest",
    environment: {
      signRequestsTableName: signRequestsTable.tableName,
    },
  }
);

signRequestsTable.grantReadWriteData(requestDocumentSignature);
signRequestsTable.grantReadWriteData(completeSignatureRequest);

const bobSignature = new sfn.Task(this, "Bob Signature", {
  task: new tasks.RunLambdaTask(requestDocumentSignature, {
    integrationPattern: ServiceIntegrationPattern.WAIT_FOR_TASK_TOKEN,
    payload: sfn.TaskInput.fromObject({
      stepFunctionToken: sfn.Context.taskToken,
      signer: "Bob",
      documentId: Data.stringAt("$.documentId"),
    }),
  }),
});

Last but not least we need to allow the completeSignatureRequest to adjust step function state through IAM Policy Statement.

const waitForSignaturesTimeout = Duration.seconds(60);
const stateMachine = new sfn.StateMachine(this, "Step Function", {
  definition: flow,
  timeout: waitForSignaturesTimeout,
});

completeSignatureRequest.addToRolePolicy(
  new iam.PolicyStatement({
    actions: ["states:SendTaskSuccess", "states:SendTaskFailure"],
    resources: [stateMachine.stateMachineArn],
  })
);

After we deploy our updated stack the function definition should look as follows:

Call Step Function callback from AWS Lambda

For our workflow to complete we need to implement Bob's lambda part.
Let's start with a function that simulates starting a long-running business process.
The important bit in sendSignatureRequest is that it stores state function task token.
We will use the token later on to call AWS Step Function back.
The callback will include result. In our case a fake signature of a document.

First add 2 more packages to our project:

npm i --save @types/aws-lambda aws-sdk

Then update the AWS Lambda sendSignatureRequest definition:

import { Handler } from "aws-lambda";
const DynamoDb = require("aws-sdk/clients/dynamodb");

interface SignDocumentEvent {
  documentId: string;
  signer: string;
  stepFunctionToken: string;
}

const documentClient = new DynamoDb.DocumentClient();
const signRequestsTableName = process.env.signRequestsTableName;

export const requestDocumentSignature: Handler<SignDocumentEvent> = async (
  event
) => {
  console.log("Request signature for", event);

  await documentClient
    .put({
      TableName: signRequestsTableName,
      Item: event,
    })
    .promise();
};

The last missing piece is to implement the completeSignatureRequest lambda function.
The important bit is to call sendTaskSuccess to indicate that processing completed successfully.

import { Handler } from "aws-lambda";
const StepFunctions = require("aws-sdk/clients/stepfunctions");
const DynamoDb = require("aws-sdk/clients/dynamodb");
const documentClient = new DynamoDb.DocumentClient();
const signRequestsTableName = process.env.signRequestsTableName;
const stepFunctions = new StepFunctions();
export const completeSignatureRequest: Handler<{ documentId: string }> = async (
  event
) => {
  const tableItem = await documentClient
    .get({
      TableName: signRequestsTableName,
      Key: { documentId: event.documentId },
    })
    .promise();

  const signRequest: SignDocumentEvent = tableItem.Item;
  console.log("Received signature for", signRequest);

  const output = JSON.stringify({
    signature: `${signRequest.signer} signed ${signRequest.documentId}`,
  });

  await stepFunctions
    .sendTaskSuccess({
      output: output,
      taskToken: signRequest.stepFunctionToken,
    })
    .promise();
};

Let's test the execution by requesting a document with id equal to 3.
As expected Alice signs the document after couple of seconds.
However, for in order to get Bob's signature we invoke complete signature lambda.
This simulates an action that involves human interaction.
Note that the whole process could take months to complete!

The AWS Step Function console depicts the current phase of processing pretty neatly.

You can find full working example in Github repository.

Leave a comment if you like the article and would like to see more!

HTTPs on CloudFront using Certificate Manager and aws-cdk

Piotr Mionskowski — Fri, 22 May 2020 22:20:40 +0000

The static website deployed with aws-cdk to CloudFront post
shows how to deploy static content with aws-cdk. We still miss a custom domain configuration for
our website though. The following example shows how to:

setup a custom domain name for a CloudFront distribution
enable https using AWS issued trusted certificate

Get a trusted certificate from AWS Certificate Manager using aws-cdk

Using the example from previous post as a base, inside infrastructure directory, let's add
additional package:



npm i --save @aws-cdk/aws-certificatemanager

CloudFront can only use AWS Certificate Manager issued certificates inside us-east-1 region (N. Virginia).
We can create our FrontendStack in us-east-1 region. Alternatively we can create another stack with the certificate only.
Below I show how to use the second option.



...
import { ValidationMethod } from "@aws-cdk/aws-certificatemanager";
import * as cm from "@aws-cdk/aws-certificatemanager";

const domainName = "fast-static-website.miensol.pl";

export class CertificateStack extends cdk.Stack {
  constructor(scope: cdk.Construct) {
    super(scope, "CertificateStack", {
      env: { region: "us-east-1" },
    });

    const certificate = new cm.Certificate(this, "CustomDomainCertificate", {
      domainName: domainName,
      validationMethod: ValidationMethod.DNS,
    });

    const certificateArn = certificate.certificateArn;
    new CfnOutput(this, "CertificateArn", {
      value: certificateArn,
    });
  }
}
...

We request a certificate for a given domain name in us-east-1 region.
Before AWS Certificate Manager issues a trusted TLS certificate it needs to verify that you own
the domain. The validation happens using either:

email: AWS CM sends a message to admin email addresses for the domain
DNS: AWS CM requests a particular DNS record configuration

In the example above I use fast-static-website.miensol.pl with DNS validation method. Since I own
the domain I can prove that to AWS Certificate Manager.

We also update the bin/infrastructure.ts file to include our new stack:



#!/usr/bin/env node
import "source-map-support/register";
import * as cdk from "@aws-cdk/core";
import { CertificateStack, FrontendStack } from "../lib/frontend-stack";

const app = new cdk.App();
new CertificateStack(app);
new FrontendStack(app);

Let's trigger next deployment with:



npm run cdk deploy '*'

The process will wait for a certificate request validation. The requested DNS
record will show up in the deployment command output:



> npm run cdk deploy
...
1/5 | 6:03:37 PM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate
 | CustomDomainCertificate (CustomDomainCertificateXXXXX) Content of DNS Record is: {Name: XXXXXX.fast-static-website.miensol.pl.,Type: CNAME,Value: XXXXX.auiqqraehs.acm-validations.aws.}
1/5 Currently in progress: CustomDomainCertificateXXXXX

You can also find the request in Certificate Manager inside AWS Web console. After
you configure the DNS record, the process will continue. In the command output you should see a message similar to:



CertificateStack.CertificateArn = arn:aws:acm:us-east-1:XXXXXXX:certificate/XXXXX-XXXXX-XXXX-XXXX-XXXXXXXX

Please note the above value, we will need it in the following step.

Attach a certificate to CloudFront distribution using aws-cdk

We need to tell the CloudFront distribution to use the new certificate. Let's modify the FrontendStack
to accept parameters:




interface FrontendProps {
  certificateArn: string;
}

export class FrontendStack extends cdk.Stack {
  constructor(scope: cdk.Construct, props: FrontendProps) {
    super(scope, "FrontendStack");

    const frontBucket = new s3.Bucket(this, "FrontendBucket", {
      websiteIndexDocument: "index.html",
      publicReadAccess: true,
    });

    const distribution = new cloudfront.CloudFrontWebDistribution(
      this,
      "Distribution",
      {
        viewerCertificate: {
          aliases: [domainName],
          props: {
            acmCertificateArn: props.certificateArn,
            sslSupportMethod: "sni-only",
          },
        },
...

In the viewerCertificate section we specify domain name as well as certificate CloudFront distribution should use.
Next we pass CertificateStack.CertificateArn value noted above as a parameter:



const app = new cdk.App();
new CertificateStack(app);
new FrontendStack(app, {
  certificateArn:
    "arn:aws:acm:us-east-1:XXXXXXX:certificate/XXXXX-XXXXX-XXXX-XXXX-XXXXXXXX",
});

The deployment will take a while to complete since the CF distribution has to update.
We can use that time to configure fast-static-website.miensol.pl DNS record.
The record needs to point to FrontendStack.DistributionDomainName.
I use CloudFlare so in my case this means creating a CNAME record:

We can now load our static website using HTTPS!

It is a shame that it is not possible to create cross region references using aws-cdk:

There is not, unfortunately. You're going to have to thread the API URL across regions manually. That means:

Deploy the ApiStackIreland first. Note the API URL (maybe make it an Output so that it's easy to get to).
Deploy the EdgeStack and plug in the value of API URL from the first step.
There are a number of ways you could do this, and honestly I'm not sure which one to recommend.

I guess a pragmatic one would be to use this.node.getContext() and pass the value on the command-line when deploying the second stack (using the -c KEY=VALUE fflag), or put it into cdk.json under the context: {} section.

The full working example is available in the github repository.

Deploy fast, static website to AWS like a pro

Piotr Mionskowski — Sun, 17 May 2020 14:02:06 +0000

In this post we'll see how to create and deploy a static website using IaC using the following technologies:

aws-cdk
AWS CloudFormation
AWS Lambda
AWS S3

Have a static website

For the sake of simplicity we'll use Create React App in a directory called my-project:

cd my-project
npx create-react-app frontend --template typescript

Create the static website content inside frontend/build directory with:

cd frontend && yarn build

Add aws-cdk to the project

Create infrastructure project with aws-cdk:

mkdir infrastructure && cd infrastructure
npx cdk init --language typescript

Note that in order to deploy you need to have AWS API credentials configured.
I prefer using AWS profiles, which you can set up using AWS CLI:

aws configure --profile my-profile-name

The command will ask you couple of questions.
Please find the AWS Access Key Id and AWS Secret Access Key for your AWS account in AWS Web Console IAM section.
The AWS CLI stores configuration under ~/.aws (Linux or Mac) or %USERPROFILE%\.aws (Windows).

Please follow aws-cdk Getting Started to get more details.

Deploy static content deployment to S3 website

We will deploy static content to AWS using @aws-cdk/aws-s3-deployment package. We need to add it to our project:

cd infrastructure
npm i --save @aws-cdk/aws-s3-deployment

Next, let's tell aws-cdk to deploy contents of frontend/build directory to AWS.
In our template we have InfrastructureStack available under lib/infrastructure-stack.ts.
Since it makes sense to have more focused CloudFormation stacks let's rename it to FrontendStackStack.

import * as cdk from '@aws-cdk/core';
import * as s3 from '@aws-cdk/aws-s3'
import * as s3deploy from '@aws-cdk/aws-s3-deployment'
import * as cloudfront from '@aws-cdk/aws-cloudfront'
import * as path from 'path'

export class FrontendStack extends cdk.Stack {
  constructor(scope: cdk.Construct) {
    super(scope, 'FrontendStack');
  }
}

All files, which our static content consists of, will be stored on S3, so we need to create a bucket there:

super(scope, 'FrontendStack');

const frontBucket = new s3.Bucket(this, 'FrontendBucket', {
  websiteIndexDocument: 'index.html',
  publicReadAccess: true
});

Next, we need to tell @aws-cdk/aws-s3-deployment to push contents of frontent/build to the bucket created above:

new s3deploy.BucketDeployment(this, 'DeployWebsite', {
  sources: [s3deploy.Source.asset(
    path.join(__dirname, '..', '..', 'frontend', 'build')
  )],
  destinationBucket: frontBucket
});

Since our aws-cdk stack uses assets we need to bootstrap our AWS account before first deployment:

cd infrastructure
AWS_PROFILE=my-profile-name npx cdk bootstrap

Once the above command completes we can now finally deploy our website:

cd infrastructure
AWS_PROFILE=my-profile-name npm run cdk deploy

The command will prompt for confirmation before it creates the necessary security rules.
You will see what resources get created or modified while the command runs.
Afterwards all resources created are part of AWS CloudFormation FrontendStack stack.
Most importantly the command copied all files to S3 Bucket, which in turn can host the website.

One you find the S3 bucket created in the AWS CloudFormation stack:

Your website is now available under a URL similar to:

http://frontendstack-frontendbucketefe2e19c-ixsoxq891rl1.s3-website.eu-central-1.amazonaws.com/

Improve performance of static website with CloudFront

The website loads files directly from S3 bucket. The S3 bucket, by default, lives inside a single AWS Region,
which means that visitors from far away may suffer from subpar loading speeds.

For production setup we can use AWS CloudFront CDN, which distributes our content across over 100 locations around the globe.
Using this solution with @aws-cdk/aws-s3-deployment is a piece of cake:

const distribution = new cloudfront.CloudFrontWebDistribution(this, 'Distribution', {
  originConfigs: [{
    s3OriginSource: {
      s3BucketSource: frontBucket
    },
    behaviors : [ {isDefaultBehavior: true}]
  }]
});

new cdk.CfnOutput(this, 'DistributionDomainName', {
  value: distribution.domainName
})

new s3deploy.BucketDeployment(this, 'DeployWebsite', {
  sources: [s3deploy.Source.asset(
      path.join(__dirname, '..', '..', 'frontend', 'build')
  )],
  destinationBucket: frontBucket,
  distribution: distribution,
});

After you update the stack with AWS_PROFILE=my-profile-name npm run cdk deploy you will find a new CloudFront distribution created.
Note that it make take few minutes to complete.
We added CfnOutput to make it easier to find the CloudFront distribution autogenerated domain name.

Outputs:
FrontendStack.DistributionDomainName = d3a9wcc4fxw2d.cloudfront.net

Now the static website uses global content delivery network!

You can find full example used in this post in the Github repository

Originally published at https://miensol.pl.