Single CloudFront distribution for S3 web app and API Gateway

Even Zhang — Wed, 07 Oct 2020 17:40:17 +0000

This post outlines how to use a single Amazon CloudFront distribution for both your web app hosted on S3 and also your backend API.

The outcomes I want to achieve:

website.com will load my web app
website.com/non-existent-page will give me a human friendly 404 page
website.com/api/* will be routed to my backend API
website.com/api/non-existent-endpoint will return me a machine friendly error response from my backend and NOT my human friendly 404 page

I will be using Amazon API GatewayV2 as my backend for this example but the principles should apply to any other backends.

Why
Summary
Architecture
Step by Step Tutorial Using CDK
- CDK Setup
- API Gateway with a Lambda Backend
- S3 Bucket and CloudFront Distribution
- Add API Gateway as Another CloudFront Origin
- Lambda@Edge for Handling Redirects
- Test It Out
Key Take Aways
Alternatives

Why

Why use a single CloudFront distribution for both my web app and API when I can use a separate distribution for both? E.g. why can't I just use website.com and api.website.com?

In short, subdomains can be difficult to manage and in some organisations it may even be difficult to request for a new subdomain.

An example where subdomain management can be difficult is different environments. Lets say you have a dev environment with the following subdomain dev.website.com; would your API be api.dev.website.com or would it be dev.api.website.com? Would you create separate SSL certificates for them? What about your CORS rules? What about CSP? How are you going to manage this in your codebase?

...and I can go on forever.

So some people may consider just using /api/ and call it a day. This post is for those people.

Summary

Essentially we will have CloudFront serve from multiple origins based on path patterns. In this case we will have Cloudfront forward all /api/* requests to the API Gateway and have all other requests forwarded to S3. This alone will achieve outcomes 1, 3 and 4.

However, if someone tries to access /non-existent-page you will get a "NoSuchKey" error from S3 which is not our human friendly 404 error.

What about using CloudFront custom error configurations to overwrite all 404 errors to index.html or a dedicated 404 page? Unfortunately, CloudFront does not yet have the capability to set custom error configurations per origin today, so in our case it would overwrite our backend API error response as well (basically the core of the problem we are trying to solve here).

So with custom error configurations out of the question, we are left with...

Yep you guessed it, Lambda@edge.

We can use Lambda@Edge for /non-existent-page routes to your S3 origin, redirecting any 404 responses back to our index.html or to a dedicated 404 page.

Architecture

Pretty straight forward, we just need a Lambda@Edge between our S3 origin and our distribution.

Step by Step Tutorial Using CDK

Below is a step by step tutorial building out a working example using CDK.

For reference this is what I have installed locally:



$ node --version
v14.13.1
$ yarn --version # you can use npm
1.22.5
$ cdk --version
1.67.0 (build 2b4dd71)
$ docker --version # used by CDK to compile typescript lambdas
Docker version 19.03.13, build 4484c46d9d

You can also find all the source code here with instructions on deploying a working example:

evnz / blog-example-single-cf-distribution

Example of single Amazon CloudFront distribution for both your web app hosted on S3 bucket and API Gateway backend

CDK Setup

Let's create a typescript CDK project and install some dependencies:



$ cdk init app --language typescript
...
$ yarn add \
    @aws-cdk/aws-cloudfront \
    @aws-cdk/aws-apigatewayv2 \
    @aws-cdk/aws-s3 \
    @aws-cdk/aws-s3-deployment \
    @aws-cdk/aws-lambda \
    @aws-cdk/aws-lambda-nodejs \
    @aws-cdk/aws-iam
...
$ yarn add --dev --exact parcel@2.0.0-beta.1 # for compiling typescript lambdas

API Gateway with a Lambda Backend

Inside the lib folder of your CDK app, create a folder called backend and put a dummy lambda function (index.ts) that returns 200 inside:



export const handler = async (event: any): Promise<any> => {
  return {
    statusCode: 200,
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ key: "Machine friendly hello world" }),
  };
};

Add the imports we need in your lib/{your-stack-name}.ts file:



import * as cdk from "@aws-cdk/core";
import { NodejsFunction } from "@aws-cdk/aws-lambda-nodejs";
import { Runtime } from "@aws-cdk/aws-lambda";
import * as apigatewayv2 from "@aws-cdk/aws-apigatewayv2";

Now we can add the following code to our CDK stack to create the lambda function and integrate it with an API GatewayV2:



const httpApi = new apigatewayv2.HttpApi(this, "MyApiGateway");

const helloWorldLambda = new NodejsFunction(this, "HelloWorldLambda", {
  entry: `${__dirname}/backend/index.ts`,
  handler: "handler",
  runtime: Runtime.NODEJS_12_X,
});

const lambdaIntegration = new apigatewayv2.LambdaProxyIntegration({
  handler: helloWorldLambda,
});

httpApi.addRoutes({
  path: "/api/helloworld", // You must include the `/api/` since CloudFront will not truncate it
  methods: [apigatewayv2.HttpMethod.GET],
  integration: lambdaIntegration,
});

3. S3 Bucket and CloudFront Distribution

Inside the lib folder of your CDK app, create a folder called frontend and create an index.html file inside with some html content:



<html>
  <body>
    Hello world
  </body>
</html>

Import the following CDK dependencies in you lib/{your-stack-name}.ts file:



import * as cloudfront from "@aws-cdk/aws-cloudfront";
import * as s3 from "@aws-cdk/aws-s3";
import * as iam from "@aws-cdk/aws-iam";
import { Duration } from "@aws-cdk/core";
import * as s3deploy from "@aws-cdk/aws-s3-deployment";

Now lets create a standard static website using a private S3 bucket, a CloudFront distribution and everything inbetween (IAMs, OAIs etc):



const cloudfrontOAI = new cloudfront.OriginAccessIdentity(
  this,
  "cloudfrontOAI",
  {
    comment: `Allows CloudFront access to S3 bucket`,
  }
);

const websiteBucket = new s3.Bucket(this, "S3BucketForWebsiteContent", {
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  cors: [
    {
      allowedOrigins: ["*"],
      allowedMethods: [s3.HttpMethods.GET],
      maxAge: 3000,
    },
  ],
});

// uploads index.html to s3 bucket
new s3deploy.BucketDeployment(this, "DeployWebsite", {
  sources: [s3deploy.Source.asset(`${__dirname}/frontend`)], // folder containing your html files
  destinationBucket: websiteBucket,
});

websiteBucket.addToResourcePolicy(
  new iam.PolicyStatement({
    sid: "Grant Cloudfront Origin Access Identity access to S3 bucket",
    actions: ["s3:GetObject"],
    resources: [websiteBucket.bucketArn + "/*"],
    principals: [cloudfrontOAI.grantPrincipal],
  })
);

const cloudfrontDistribution = new cloudfront.CloudFrontWebDistribution(
  this,
  "MyDistribution",
  {
    comment: "CDN for Web App",
    defaultRootObject: "index.html",
    viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
    priceClass: cloudfront.PriceClass.PRICE_CLASS_ALL,
    originConfigs: [
      {
        s3OriginSource: {
          s3BucketSource: websiteBucket,
          originAccessIdentity: cloudfrontOAI,
        },
        behaviors: [
          {
            compress: true,
            isDefaultBehavior: true,
            defaultTtl: Duration.seconds(0),
            allowedMethods:
              cloudfront.CloudFrontAllowedMethods.GET_HEAD_OPTIONS,
          },
        ],
      },
    ],
  }
);

Add API Gateway as Another CloudFront Origin

Add API Gateway as another origin with a path pattern of /api/* to the CloudFront distribution (make sure the API Gateway origin is above the S3 origin to ensure the path matching for /api/* takes precendence):



originConfigs: [
  {
    // make sure your backend origin is first in the originConfigs list so it takes precedence over the S3 origin
    customOriginSource: {
      domainName: `${httpApi.httpApiId}.execute-api.${this.region}.amazonaws.com`,
    },
    behaviors: [
      {
        pathPattern: "/api/*", // CloudFront will forward `/api/*` to the backend so make sure all your routes are prepended with `/api/`
        allowedMethods: cloudfront.CloudFrontAllowedMethods.ALL,
        defaultTtl: Duration.seconds(0),
        forwardedValues: {
          queryString: true,
          headers: ["Authorization"], // By default CloudFront will not forward any headers through so if your API needs authentication make sure you forward auth headers across
        },
      },
    ],
  },
  {
    s3OriginSource: {
      s3BucketSource: websiteBucket,
      originAccessIdentity: cloudfrontOAI,
    },
    behaviors: [
      {
        compress: true,
        isDefaultBehavior: true,
        defaultTtl: Duration.seconds(0),
        allowedMethods: cloudfront.CloudFrontAllowedMethods.GET_HEAD_OPTIONS,
      },
    ],
  },
];

Lambda Edge for Handling Redirects

Caution: Once you have created a Lambda@Edge, deleting it can take some time.

Inside the lib folder of your CDK app, create a folder called redirect and create an index.ts file inside with the following content (this will be our redirect Lambda@Edge):



"use strict";

exports.handler = (event: any, context: any, callback: any) => {
  const response = event.Records[0].cf.response;
  const request = event.Records[0].cf.request;

  /**
   * This function updates the HTTP status code in the response to 302, to redirect to another
   * path (cache behavior) that has a different origin configured. Note the following:
   * 1. The function is triggered in an origin response
   * 2. The response status from the origin server is an error status code (4xx or 5xx)
   */

  if (response.status == 404) {
    const redirect_path = `/`; //redirects back to root so to index.html

    response.status = 302;
    response.statusDescription = "Found";

    /* Drop the body, as it is not required for redirects */
    response.body = "";
    response.headers["location"] = [{ key: "Location", value: redirect_path }];
  }

  callback(null, response);
};

Now add the Lambda resource to your CDK (make sure this is before the CloudFront distribution resource as we will be referencing this Lambda inside the distribution later):



const redirectLambda = new NodejsFunction(this, "redirectLambda", {
  entry: `${__dirname}/redirect/index.ts`,
  handler: "handler",
  runtime: Runtime.NODEJS_12_X,
});

Lastly lets associate this Lambda@Edge with our S3 origin:



{

    s3OriginSource: {

        s3BucketSource: websiteBucket,

        originAccessIdentity: cloudfrontOAI,

    },

    behaviors: [

        {

            compress: true,

            isDefaultBehavior: true,

            defaultTtl: Duration.seconds(0),

            allowedMethods:

            cloudfront.CloudFrontAllowedMethods.GET_HEAD_OPTIONS,

            lambdaFunctionAssociations: [

                {

                    lambdaFunction: redirectLambda.currentVersion,

                    eventType: LambdaEdgeEventType.ORIGIN_RESPONSE,

                },

            ],

        },

    ],

},

Test It Out

Using the cloudfront URL try visiting the following endpoints

/ should return our hello world index.html page

/api/helloworld should return us a machine friendly message

/api/non-existent-endpoint should be handled by the backend API

and finally /non-existent-page should redirect us back to our index.html so we can handle it inside our web app

Key Take Aways

Use CloudFront to serve from multiple origins based on path-patterns
CloudFront will forward the whole path to your origin so make sure your API paths are the same as the ones set in CloudFront
CloudFront by default will not forward any headers to your API so be explicit about what headers are forwarded
Do not use CloudFront custom error configurations since they will overrwrite your API error responses as well
Use Lambda@Edge to redirect 404 NoSuchKey errors from S3

Alternatives

If Lambda@Edge does not suit your taste, you may want to explore the following alternatives instead:

Use a public S3 bucket static website as a third origin for redirecting
Serve your S3 content from your API Gateway or ALB

Hope this was helpful!

Feedback and questions welcome in the comment section below :)

Forem: Even Zhang