Forem: Pierre Chollet

Very useful to think about APIGTW limits it before you run into them, very insightful! 💡

Pierre Chollet — Fri, 06 Dec 2024 17:25:23 +0000

Avoiding API Gateway’s integrations hard limit: scaling serverless architectures efficiently

Anaïs Schlienger for Serverless By Theodo ・ Nov 26

Your AWS app in depth like never before with sls-mentor

Pierre Chollet — Tue, 11 Jun 2024 15:00:36 +0000

Your serverless app like you've never seen it before with sls-mentor

Ever dreamed of being able to visualize and analyze your entire AWS application at a glance? With the new 3.0 (alpha) of sls-mentor, it is now possible!

// Detect dark theme var iframe = document.getElementById('tweet-1754504552861024582-725'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1754504552861024582&theme=dark" }

sls-mentor is a free and open-source tool that generates an interactive graph of your AWS application. This graph contains all the interactions between components of your app (Lambda functions, DynamoDB tables, S3 buckets...).

sls-mentor also has a brand new feature: Dashboards. With dashboards, you have access to stats about your app such as :

Lambda Cold start duration 🏎
Lambda Bundle size 📦
S3 bucket size 🪣
DynamoDB table size 📊
And more to come! ✨

How to run sls-mentor?

You only need your CLI to run the new 3.0 of sls-mentor, simply use:

npx sls-mentor@alpha -p <AWS_CLI_PROFILE> -r <AWS_REGION>

Star sls-mentor on Github ⭐️

sls-mentor will perform its analysis live, on the AWS Account associated with the CLI profile.

There are also filtering options: -c to specify cloudformation stacks, -t for tags

We need you!

If you enjoyed trying sls-mentor 3.0, your feedback is valuable! Feel free to comment or to contact me on twitter

Contact me on twitter 🚀

We are also open to contributions!

Contribute on Github ⭐️

Visualize your AWS app like never before with sls-mentor

Pierre Chollet — Wed, 14 Feb 2024 12:33:19 +0000

Your serverless app like you've never seen it before with sls-mentor

Ever dreamed of being able to visualise your entire AWS application at a glance? With the new 3.0 (alpha) of sls-mentor, it is now possible!

// Detect dark theme var iframe = document.getElementById('tweet-1754504552861024582-83'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1754504552861024582&theme=dark" }

The end-game of sls-mentor is to allow developers and tech-leads to be able to take informed decisions relative to the architecture of their AWS App, by providing them with user-friendly data and best-practices compatible with their situation.

How to run sls-mentor?

You only need your CLI to run the new 3.0 of sls-mentor, simply use:

npx sls-mentor@alpha -p <AWS_CLI_PROFILE> -r <AWS_REGION>

Star sls-mentor on Github ⭐️

sls-mentor will perform its analysis live, on the AWS Account associated with the CLI profile.

There are also filtering options: -c to specify cloudformation stacks, -t for tags

What can sls-mentor do?

sls-mentor 3.0 is still in alpha, but it is already quite capable!

Visualize a graph snapshot of your AWS app

Some resources or interactions are not yet supported yet, we are working on them!

Rank Lambdas/DynamoDBs by stats (duration, size...)

More options coming soon!

Edit the graph

Have fun by moving around the components of your app, or pin them (by pressing SPACE) to make it clearer.

Short-term roadmap

We are currently working hard on making the alpha better!

// Detect dark theme var iframe = document.getElementById('tweet-1757058192696086817-228'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1757058192696086817&theme=dark" }

Here are some of the features we want to implement during the following weeks:

AppSync APIs and integrations
Clustering (By Cloudformation stack, by tags...)
Filtering options (By Cloudformation stack, by tags...)
More stats (S3 buckets...)
First best-practices

We need you!

If you enjoyed trying sls-mentor 3.0, your feedback is valuable! Feel free to comment or to contact me on twitter

Contact me on twitter 🚀

We are also open to contributions!

Contribute on Github ⭐️

Learn serverless on AWS step-by-step - Schedule tasks with EventBridge Scheduler

Pierre Chollet — Tue, 16 Jan 2024 14:12:20 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. With last article, we discovered how to Lambda function destinations to avoid losing data when an asynchronous Lambda function fails. In this article, we will discover how to schedule tasks with EventBridge Scheduler.

What will we do today?

Create a small memo application, where we can create a memo, and execute it at a specific date and time
Create a Lambda function that creates a memo
Create a Lambda function that is triggered at a specific date and time, and executes the memo

The architecture of our application will look like this:

⬇️ I post serverless content very regularly, if you want more ⬇️

Follow me on twitter 🚀

Quick announcement: I also work on a library called 🛡 sls-mentor 🛡. It is a compilation of 40 serverless best-practices, that are automatically checked on your AWS serverless projects (no matter the framework). It is free and open source, feel free to check it out!

Find sls-mentor on Github ⭐️

How to schedule tasks with AWS EventBridge Scheduler?

AWS EventBridge Scheduler is a service that allows you to schedule tasks in the future. It can be compared to AWS EventBridge Rules: while rules allow you to trigger tasks based on rates or cron expressions, scheduler allows you to trigger tasks at a specific date and time (they also support cron expressions and rates btw...).

Using the AWS EventBridge Scheduler programmatically to trigger a Lambda function has an easy and a not-so-easy step:

😁 The easy step is to specify a target, this can easily be done using the ARN of the Lambda function you want to trigger
🥵 The not-so-easy step is to give the permission to the scheduler to invoke your Lambda function. This is done by:
- Creating a role with a scheduler.amazonaws.com service principal (assumed by the scheduler)
- Giving the permission to the scheduler to invoke your Lambda function lambda:InvokeFunction
- Giving the permission to your input Lambda function (addMemo) to pass the role to the scheduler iam:PassRole

This can be summarized in the following diagram:

Now, let's see how this works in practice, with a real code example!

Create a small memo application using AWS EventBridge Scheduler

To create our app, we will use the AWS CDK. If you are not familiar with it, I invite you to read the first article of this series where I explain properly how to setup a CDK project.

npx cdk init app --language typescript
npm i @aws-sdk/client-scheduler
npm i uuid # always useful to generate unique ids
npm i -D esbuild # Needed to bundle our Lambdas!

First, let's create the Infrastructure as Code (IAC) of the application. This is done by updating the CDK stack:

// stack.ts - Infrastructure as code
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

import * as cdk from 'aws-cdk-lib';
import { join } from 'path';

export class Part17SchedulerStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Lambda function triggered by scheduler
    const executeMemo = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ExecuteMemo', {
      entry: join(__dirname, 'executeMemo.ts'),
      handler: 'handler',
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
      bundling: {
        externalModules: ['@aws-sdk'],
      },
    });

    // Create role for scheduler to invoke executeMemo
    const invokeExecuteMemoRole = new cdk.aws_iam.Role(this, 'InvokeMemoRole', {
      assumedBy: new cdk.aws_iam.ServicePrincipal('scheduler.amazonaws.com'),
    });
    invokeExecuteMemoRole.addToPolicy(
      new cdk.aws_iam.PolicyStatement({
        actions: ['lambda:InvokeFunction'],
        resources: [executeMemo.functionArn],
      }),
    );

    // Lambda function that schedules executeMemo
    const addMemo = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'AddMemo', {
      entry: join(__dirname, 'addMemo.ts'),
      handler: 'handler',
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
      bundling: {
        externalModules: ['@aws-sdk'],
      },
      environment: {
        SCHEDULE_TARGET_ARN: executeMemo.functionArn,
        SCHEDULE_ROLE_ARN: invokeExecuteMemoRole.roleArn,
      },
    });

    // Allow addMemo to create a scheduler
    addMemo.addToRolePolicy(
      new cdk.aws_iam.PolicyStatement({
        actions: ['scheduler:CreateSchedule'],
        resources: ['*'],
      }),
    );

    // Allow addMemo to pass the invokeExecuteMemoRole to the scheduler
    addMemo.addToRolePolicy(
      new cdk.aws_iam.PolicyStatement({
        actions: ['iam:PassRole'],
        resources: [invokeExecuteMemoRole.roleArn],
      }),
    );

    // Trigger addMemo via API Gateway
    const api = new cdk.aws_apigateway.RestApi(this, 'Api', {
      restApiName: 'Part17Service',
    });
    api.root.addResource('addMemo').addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(addMemo));
  }
}

What is happening here?

1️⃣ We create the executeMemo Lambda function, that will be triggered by the scheduler
2️⃣ We create a role that will be assumed by the scheduler, and that will allow it to invoke the executeMemo Lambda function (see intro diagram)
3️⃣ We create the addMemo Lambda function, that will create a scheduler
4️⃣ We allow the addMemo Lambda function to create a scheduler by adding the scheduler:CreateSchedule permission and the iam:PassRole (see intro diagram) permission to the addMemo role
5️⃣ We create an API Gateway endpoint that will trigger the addMemo Lambda function, and a POST method to trigger it

Now, let's create the Lambda functions that will be triggered by the API Gateway endpoint. First, let's create the addMemo Lambda function, that will create a schedule:

// addMemo.ts - Lambda function that creates a scheduler
import {
  ActionAfterCompletion,
  CreateScheduleCommand,
  FlexibleTimeWindowMode,
  SchedulerClient,
} from '@aws-sdk/client-scheduler';

import { v4 as uuidv4 } from 'uuid';

const client = new SchedulerClient({});
const scheduleTargetArn = process.env.SCHEDULE_TARGET_ARN as string;
const scheduleRoleArn = process.env.SCHEDULE_ROLE_ARN as string;

if (scheduleTargetArn === undefined || scheduleRoleArn === undefined) {
  throw new Error('Missing environment variables');
}

export const handler = async ({
  body,
}: {
  body: string;
}): Promise<{
  statusCode: number;
  body: string;
}> => {
  const {
    memo,
    date,
    time,
    timezone = 'Europe/Paris',
  } = JSON.parse(body) as { memo?: string; date?: string; timezone?: string; time?: string };

  if (memo === undefined || date === undefined) {
    return {
      statusCode: 400,
      body: 'Bad Request',
    };
  }

  await client.send(
    new CreateScheduleCommand({
      Name: uuidv4(),
      Target: {
        Arn: scheduleTargetArn,
        RoleArn: scheduleRoleArn,
        Input: JSON.stringify({ memo }),
      },
      ScheduleExpressionTimezone: timezone,
      ScheduleExpression: `at(${date}T${time})`,
      FlexibleTimeWindow: {
        Mode: FlexibleTimeWindowMode.OFF,
      },
      ActionAfterCompletion: ActionAfterCompletion.DELETE,
    }),
  );

  return {
    statusCode: 200,
    body: 'Memo scheduled',
  };
};

What is happening here?

1️⃣ We setup a client and parse environment variables (see IAC where we set them)
2️⃣ We parse the body of the request, and extract the memo, date, time and timezone (default to Europe/Paris (where I live 😅))
3️⃣ We create a schedule using the AWS SDK for Javascript
- 🅰️ We set the target, using the scheduleTargetArn and scheduleRoleArn environment variables
- 🅱️ We set the schedule expression, using the date and time provided in the request, along with the timezone, and we set the schedule to be automatically deleted after execution

Finally, let's create the executeMemo Lambda function, that will be triggered by the scheduler. This function will simply log the memo to the console:

// executeMemo.ts - Lambda function that executes a memo
export const handler = async ({ memo }: { memo: string }): Promise<void> => {
  console.log(memo);

  return Promise.resolve();
};

Very easy! Notice that the lambda input is the same as the input field specified in the CreateScheduleCommand of the addMemo Lambda function.

Test our app

We are done! Time to deploy and to test our API route /addMemo

npm run cdk bootstrap
npm run cdk deploy

I live in Paris 🇫🇷, so I used my default Europe/Paris timezone, but you can specify the timezone of your choice. I specified a time of 22:37, and it is 22:36, if I head to Cloudwatch, I should see my executeMemo Lambda function being triggered in 1 minute.

It worked! I can also see the payload of the event, and it contains the memo I created.

Going further

What could be improved in this app?

We could go further and add a /getMemos route, that would return all the memos that are not executed yet
We could also add a /executeMemo route, that would execute a memo immediately, and cancel the corresponding schedule
As always, implementing authentication would be a good idea 😅

Conclusion

This article was a basic introduction to AWS EventBridge Scheduler. We discovered how to create a scheduled task, and how to execute it. We also discovered how to use the AWS CDK to provision our infrastructure. I hope you enjoyed this article, and that you learned something new!

I plan to continue this series of articles on a bi-monthly basis. You can follow this progress on my repository! I will cover new topics in the future, if you have any suggestions, do not hesitate to contact me!

I would really appreciate if you could react and share this article with your friends and colleagues. It will help me a lot to grow my audience. Also, don't forget to subscribe to be updated when the next article comes out!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

5 Serverless Best Practices to Become a Skilled Cloud Architect

Pierre Chollet — Wed, 06 Dec 2023 13:58:11 +0000

Discover serverless best practices with sls-mentor

In this article, I will go through 5 low cost, high impact serverless best practices that will help you build faster, cheaper, greener, more secure and stable applications on AWS.

Each best practice is illustrated with an infographic, to make it easy to understand and share with your colleagues. I hope you will enjoy it!

This compilation is based on 🛡 sls-mentor 🛡, my free open-source tool that automatically checks 30 serverless best practices on your AWS serverless projects (no matter the framework). Feel free to check it out!

Find sls-mentor on Github ⭐️

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

1 - When deploying a Lambda function, use a ARM64 architecture instead of a x86_64 architecture

Two Lambda function processors are available, how to choose wisely? 🤔

Basically, you should always go with ARM64 if your code is compatible! 🚀

🛡 sls-mentor has a rule that enforces the usage of ARM64 in your Lambda functions so that you never forget 🧠

2 - Use intelligent tiering on S3 Buckets to reduce storage costs

Which storage class fits the best for my #AWS app?

The choice is easy: use Intelligent Tiering! It automatically moves files between classes based on their usage.

🛡 sls-mentor automatically check that your buckets have Intelligent Tiering enabled, so that you never forget🧠

3 - Deploy fast Lambda functions with bundles smaller than 5MB

My Lambda function has insane cold starts, what should I do? 😿

Short answer: make its bundle smaller! On a given runtime, bundle size is the n°1 cause of long cold starts 🐢

🛡 sls-mentor lists every Lambda function with a bundle larger than 5MB, try it on your @awscloud account!

4 - Do not keep your CloudWatch logs forever to reduce storage costs

CloudWatch by @awscloud is known to be expensive 💸

There is a simple trick to reduce costs: 🚨Do not keep your logs forever🚨

Storage is cheap, but accumulating it over years quickly gets out of hands 📈

🛡 sls-mentor can list Log Groups with infinite retention, try it out 🚀

5 - Use an Authorizer on all your API Gateway endpoints to secure your API

When deploying an API, anyone with its URL can access your resources by default 😱

Authorizers allow you to secure your API, by relying on Cognito, IAM, or any custom integration.

🛡 sls-mentor can list all API Gateway endpoints without an Authorizer, try it out 🚀

Many more best practices to discover and automate!

🛡 sls-mentor is a compilation of 30 serverless best-practices, that are automatically checked on your AWS serverless projects (no matter the framework). It is free and open source, feel free to check it out!

Find sls-mentor on Github ⭐️

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless: Lambda Destinations

Pierre Chollet — Wed, 22 Nov 2023 13:52:12 +0000

TL;DR

When working with asynchronous Lambda functions, it can be hard to know whether a function invocation succeeded or failed. Furthermore, it can be important to retain a trace of the failed invocation, to be able to debug it or retry it later. Don't worry, AWS has a solution for that: Lambda destinations!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

What are we going to build?

Today, we will take advantage of Lambda destinations to be notified when an asynchronous Lambda function fails, and store the failed invocation to retry it later. To do so, we will:

Build a Lambda function that fails randomly, triggered every minute by a CRON
Plug a SNS topic to the failure destination of the Lambda function
Use the SNS topic to send an email, and store the event in a SQS queue

The architecture will look like this:

Quick announcement: I also work on a library called 🛡 sls-mentor 🛡. It is a compilation of 30 serverless best-practices, that are automatically checked on your AWS serverless projects (no matter the framework). It is free and open source, feel free to check it out!

Find sls-mentor on Github ⭐️

What are Lambda destinations?

The answer to this question is really simple: there are two types of Lambda destinations:

Success destinations: they are triggered when a asynchronous Lambda function succeeds
Failure destinations: they are triggered when a asynchronous Lambda function fails

Each time, the destination is triggered with information about the Lambda function invocation. Among this information, you can find the request payload, the response payload, the error message, the error type, etc...

Destinations can be multiple AWS services, such as SNS, SQS, EventBridge, etc... In this article, we will use SNS as a destination to be able to dispatch the failed invocations to an email address and a SQS queue.

Add a failure destination to a Lambda function

To build this tiny project, I will use (as in the previous articles) the AWS CDK combined with Typescript. If you are not familiar with the CDK, I invite you to read the first article of this series, where I explain how to set up a CDK project.

The CDK code for this architecture looks like this:



// lambda-destinations-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from 'path';

const DEVELOPERS_EMAILS = ['pchol.pro@gmail.com'];

export class LambdaDestinationsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create a CRON trigger, that will trigger the Lambda function every minute
    const cronTrigger = new cdk.aws_events.Rule(this, 'CronTrigger', {
      schedule: cdk.aws_events.Schedule.expression('rate(1 minute)'),
    });

    // Create a SNS topic that will be used as a failure destination
    const onFailureTopic = new cdk.aws_sns.Topic(this, 'OnFailureTopic');

    // Create the Lambda function that will fail randomly
    const lambdaFunction = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'LambdaFunction', {
      entry: join(__dirname, 'lambda.ts'),
      handler: 'handler',
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
      // Add the failure destination to the Lambda function
      onFailure: new cdk.aws_lambda_destinations.SnsDestination(onFailureTopic),
    });

    // Add the Lambda function as a target of the CRON trigger
    cronTrigger.addTarget(
      new cdk.aws_events_targets.LambdaFunction(lambdaFunction, {
        event: cdk.aws_events.RuleTargetInput.fromObject({
          // Add some data to the event payload, you will see that we will be able to retrieve it later
          veryImportantData: 'this should not be lost !!!',
        }),
        retryAttempts: 1, // 1 retry attempt
      }),
    );

    // Create a SQS queue that will be used to store the failed events
    const failedEventsQueue = new cdk.aws_sqs.Queue(this, 'FailedEventsQueue');

    // Redirect the failed events to the SQS queue
    onFailureTopic.addSubscription(new cdk.aws_sns_subscriptions.SqsSubscription(failedEventsQueue));

    // Also send an email to the developers with the same information to notify them
    DEVELOPERS_EMAILS.forEach(email => {
      onFailureTopic.addSubscription(new cdk.aws_sns_subscriptions.EmailSubscription(email));
    });
  }
}

There is not so much code so I wrote it in one block. Let's go through it step by step:

First, we create a CRON trigger that will trigger the Lambda function every minute
Then, we create a SNS topic that will be used as a failure destination. We will later add subscriptions to this topic so that it can dispatch the failed events.
Then, we create the Lambda function that will fail randomly. We add the SNS topic as a failure destination to the Lambda function.
We add the Lambda function as a target of the CRON trigger. We also add some data to the event payload, to see later that we can retrieve it. We set the number of retries to 1. This means that we will only be notified if the Lambda function fails twice in a row.
We create a SQS queue that will be used to store the failed events.
Finally, we add subscriptions to the SNS topic. We add a subscription to the SQS queue, and a subscription to each developer email address.

Final step, let's write the code for the handler of the Lambda function:



// lambda.ts
export const handler = async (): Promise<void> => {
  // 80% chance of going wrong
  // 64% counting the retry
  const success = Math.random() > 0.8;

  if (!success) {
    throw new Error('Something went wrong!');
  }

  return Promise.resolve();
};

Nothing very interesting here. We throw an error 80% of the time. Combined with the 1 retry attempt, it means that the Lambda function will trigger the failure destination 64% of the time. It does not trigger the failure destination when there are still retries left

We are done with the code 🚀! Let's deploy it to AWS!



npm i -D esbuild # Esbuild is required because we use the NodejsFunction construct

npm run cdk bootstrap # Only the first time you deploy to an AWS account

npm run cdk deploy

Test our Lambda Destinations

Now that the application is deployed, we don't have much to do since it is automatically triggered every minute. Wait a little bit and you will eventually receive an email like this one:

In this email, we can see the error message, and most importantly, the request event payload: the data wasn't lost!

If you login to the AWS console and check the SQS queue, it should contain a message (or more if you waited a little bit) that looks similar to what you received by email:

You will later be able to re-use these messages, for example to retry the failed invocations once the issue is fixed.

What could we do next?

In this article, I only showed you the failure destinations, you can also use success destinations. They are useful to be notified when a Lambda function succeeds, or to create a pipeline of Lambda functions.

It is also possible to be smarter with our handling of the messages stored in the SQS queue. With a bit of development, it should be possible to re-trigger the failed invocations automatically, without necessarily having to do it manually.

Other services like EventBridge can also be used as destinations, feel free to try them too!

Failure destinations best practices

It is considered best-practice to use failure destinations on all your asynchronous Lambda functions (for data retention and notification purposes). Remember sls-mentor? It automatically checks that you use failure destinations on all your asynchronous Lambda functions, so that you know where to put your efforts first.

Find sls-mentor on Github ⭐️

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Deploy your first Node 20 Lambda function on AWS!

Pierre Chollet — Mon, 13 Nov 2023 12:54:47 +0000

TL;DR

AWS released a public base image for Node 20 Lambda runtime last week. It is now possible to deploy Node 20 Lambda functions on AWS, by using this image with ECR (Elastic Container Registry). In this article, discover how to deploy your first Node 20 Lambda function on AWS, before the managed runtime is released! Stay tuned till the end to see the performance results.

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Find sls-mentor on Github ⭐️

🚨 Last minute update: Node 20 managed runtime is now available, no need to use container images anymore 🚨

If you still want to learn how to deploy a Lambda function using a container image, back to the article ⬇️

How to deploy a pre-release Node 20 Lambda function on AWS?

Node 20 isn't officially supported by AWS Lambda yet. However, AWS released a public base image for Node 20 Lambda runtime last week. It is possible to use this base image to build a custom runtime with docker, and use it to deploy Lambda functions on AWS. This is achieved in three simple steps:

Build a Node 20 Lambda docker image based on the public AWS base image (done locally)
Push this image to ECR (Elastic Container Registry)
Create a Lambda function using this image

Build a Node 20 Lambda docker image

First step: create a dockerfile to build a Node 20 Lambda docker image.



FROM public.ecr.aws/lambda/nodejs:20.2023.11.12.08-arm64
COPY app.js ./
CMD [ "app.handler" ]

This image is derived from the official public.ecr.aws/lambda/nodejs:20.2023.11.12.08-arm64 image. You can also use public.ecr.aws/lambda/nodejs:20.2023.11.12.08-x86_64 if the target Lambda architecture is x86_64. Then, it copies the app.js file to the image, and sets the CMD to app.handler. This means that we have to create a app.js file in the same directory as the dockerfile, and define a handler function in it.

To test if the image runs on Node 20, let's use a Node 20 only feature: the Array.prototype.toSorted method, which creates a new sorted array without modifying the original array. This new method is only available in Node 20, so if the image runs on Node 20, it should work. Here is the app.js file:



exports.handler = async () => {
  const table = [2, 3, 1, 9, 4];

  const sortedTable = table.toSorted(); // Only works on Node 20!

  return sortedTable; // [1, 2, 3, 4, 9]
};

We are done! Lets build the image! I add a lambda-image:node-20 tag to the image, so that I can easily find it later.



docker build --platform linux/amd64 -t lambda-image:node-20 . # tag the image with your own tag

Push the image to ECR (Elastic Container Registry)

Now that the image is built, we can push it to ECR. First, we need to create a repository in ECR. I will use the AWS CLI to do so. For all the following commands, replace <REPOSITORY_NAME> with the name you want to give to your repository, with your AWS account ID, and with the AWS region you want to use.



# create the ECR repository
aws ecr create-repository --repository-name <REPOSITORY_NAME> --image-scanning-configuration scanOnPush=true --region <AWS_REGION>

# login docker to ECR
aws ecr get-login-password | docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com

# tag the image with the ECR repository URI
docker tag lambda-image:node-20 <AWS_ACCOUNT_ID>.dkr.ecr.eu-west-1.amazonaws.com/<REPOSITORY_NAME>:latest

# push the image to ECR
docker push <AWS_ACCOUNT_ID>.dkr.ecr.eu-west-1.amazonaws.com/<REPOSITORY_NAME>:latest

We are done with Docker! Now, let's create a Lambda function using this image.

Create a Lambda function using this image

For this step, I will use the Web UI, for clarity. You can also use any programmatic way to create your Lambda function. The process is pretty straightforward:

You are done! You can now test your Lambda function. If everything went well, you should see the following result:

The Array.prototype.toSorted method worked as expected! This means that the Lambda function ran on Node 20.

Lambda Node 20 performance

In ran 50 cold and warm starts on this Lambda function, and measured the execution time. Here are the results, compared to a similar Lambda function running on the aws-managed Node 18 runtime:

Runtime	Median execution time	Average execution time	Cold ⛄️ or warm 🔥
Node 18	177ms	175ms	Cold ⛄️
Node 20	236ms	232ms	Cold ⛄️
Node 18	2ms	11ms	Warm 🔥
Node 20	2.5ms	28ms	Warm 🔥

What can we learn from these results? Node 20 is slower than Node 18 on cold starts, and has similar performance on warm starts. This is not surprising, as Node 20 is still in pre-release and is deployed using a container image, which has slower cold starts.

Expect the performance to be better when the managed Node 20 runtime is released!

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless: Upload files on S3

Pierre Chollet — Wed, 08 Nov 2023 12:45:02 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. During last article, we tackled some more advanced DynamoDB concepts, like the use of a DocumentClient or dynamodb-toolbox to improve the developer experience. Today, we will learn how to upload files on S3 from a React application, using a Lambda function and a presigned URL.

I already wrote an article that is an introduction to S3. In this old article, we tackled the basics of working with S3, by storing simple text files on AWS. Today, let's go further and code a real-world use case: uploading files and downloading files from a React application.

Follow me on twitter 🚀

What will we do today?

Create a simple backend allowing a user to upload a file on S3, and list all the files he uploaded
Create a React application that will use this backend to upload and list files
To do this, we will discover together the concept of presigned URLs, and how to use them to upload files on S3

If you have difficulties to follow, you can find the code at the end of the article here.

Find sls-mentor on Github ⭐️

How to upload files on S3?

The new concept I want to introduce today is the concept of presigned URLs. A presigned URL is a URL that is generated by AWS, and that gives temporary access to a resource. For example, a Lambda function in your backend can generate a presigned URL that allows you to download or upload a file directly from your frontend to S3.

The basic idea is to request a presigned URL from your backend, and then use it in your frontend to upload or download a file. The presigned URL is generated by AWS, and contains all the information needed to access the resource. It is signed by your backend, so that AWS can verify that the request comes from your backend.

The application we are going to build will look like this:

It will contain two lambda functions:

UploadFile will add the name of the file uploaded by the user in a DynamoDB table (to keep track of the files uploaded by each user), and will generate a presigned URL that will allow the user to upload the file on S3.
ListFiles will list all the files uploaded by the user, by querying the DynamoDB table, and generate a presigned URL for each file, that will allow the user to download the file from S3.

Create a AWS backend to upload and download files

First, let's start by creating the backend of our application. We will use the AWS CDK with Typescript to create our infrastructure. If you are not familiar with it, go back to my first article to learn how to get started.

Provisioning the infrastructure

Based on the architecture diagram above, we have to create a DynamoDB table, an API, two Lambda functions and a S3 bucket. We also have to give the right permissions to the Lambda functions to access the DynamoDB table and the S3 bucket.



import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from 'path';

export class UploadS3BackStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create an API Gateway REST API
    const api = new cdk.aws_apigateway.RestApi(this, 'Api');

    // Create a DynamoDB table to store files
    const filesTable = new cdk.aws_dynamodb.Table(this, 'Files', {
      partitionKey: { name: 'PK', type: cdk.aws_dynamodb.AttributeType.STRING },
      sortKey: { name: 'SK', type: cdk.aws_dynamodb.AttributeType.STRING },
      billingMode: cdk.aws_dynamodb.BillingMode.PAY_PER_REQUEST,
    });

    // Create an S3 bucket with CORS enabled
    const bucket = new cdk.aws_s3.Bucket(this, 'Bucket', {
      cors: [
        {
          allowedMethods: [
            cdk.aws_s3.HttpMethods.GET,
            cdk.aws_s3.HttpMethods.PUT,
            cdk.aws_s3.HttpMethods.POST,
            cdk.aws_s3.HttpMethods.DELETE,
          ],
          allowedOrigins: ['*'],
          allowedHeaders: ['*'],
        },
      ],
    });

    // Create a Lambda function to upload a file
    const uploadFileLambda = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'UploadFile', {
      entry: join(__dirname, 'uploadFile.ts'),
      handler: 'handler',
      environment: {
        TABLE_NAME: filesTable.tableName,
        BUCKET_NAME: bucket.bucketName,
      },
    });
    filesTable.grantReadWriteData(uploadFileLambda);
    bucket.grantPut(uploadFileLambda);
    bucket.grantPutAcl(uploadFileLambda);

    // Plug the Lambda function into API Gateway, and enable CORS
    const uploadFileResource = api.root.addResource('uploadFile');
    uploadFileResource.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(uploadFileLambda));
    uploadFileResource.addCorsPreflight({
      allowOrigins: ['*'],
      allowMethods: ['POST'],
    });

    // Create a Lambda function to list files
    const listFilesLambda = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ListFiles', {
      entry: join(__dirname, 'listFiles.ts'),
      handler: 'handler',
      environment: {
        TABLE_NAME: filesTable.tableName,
        BUCKET_NAME: bucket.bucketName,
      },
    });
    filesTable.grantReadData(listFilesLambda);
    bucket.grantRead(listFilesLambda);

    // Plug the Lambda function into API Gateway, and enable CORSs
    const listFilesResource = api.root.addResource('listFiles');
    listFilesResource.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(listFilesLambda));
    listFilesResource.addCorsPreflight({
      allowOrigins: ['*'],
      allowMethods: ['POST'],
    });
  }
}

In the code above:

We create an API and a DynamoDB table (classic business)
We create a S3 Bucket. We also have to enable CORS on the bucket, so that we can upload files from our frontend using presigned URLs. You can restrict the CORS configuration to your frontend domain if you want, but for the sake of simplicity, we will allow all origins.
We create two Lambda functions. We will see the code of these functions later. For now, we just have to know that the first one will allow us to upload a file on S3, and the second one will allow us to list all the files uploaded by a user.
We plug the Lambda functions into API Gateway. We also enable CORS on the API Gateway resources, so that we can call the API from our frontend.
We give the right permissions to the Lambda functions. The first Lambda function needs to read and write in the DynamoDB table, and to write in the S3 bucket. The second Lambda function needs to read in the DynamoDB table and in the S3 bucket.

Generate an upload presigned URL from a Lambda function

Now, let's write the code for the uploadFile Lambda function. Based on our IAC, we have to write the code inside the uploadFile.ts file.



import { DynamoDBClient, PutItemCommand } from '@aws-sdk/client-dynamodb';
import { PutObjectCommand, S3Client } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const dynamoDBClient = new DynamoDBClient({});
const s3Client = new S3Client({});

export const handler = async (event: {
  body: string;
}): Promise<{ statusCode: number; body: string; headers: unknown }> => {
  const tableName = process.env.TABLE_NAME;
  const bucketName = process.env.BUCKET_NAME;

  if (tableName === undefined || bucketName === undefined) {
    throw new Error('Missing environment variable');
  }

  const { userId, fileName } = JSON.parse(event.body) as { userId?: string; fileName?: string };

  if (userId === undefined || fileName === undefined) {
    return Promise.resolve({
      statusCode: 400,
      body: JSON.stringify({ message: 'Missing userId or fileName' }),
      headers: {
        'Access-Control-Allow-Origin': '*',
      },
    });
  }

  await dynamoDBClient.send(
    new PutItemCommand({
      TableName: tableName,
      Item: {
        PK: { S: userId },
        SK: { S: fileName },
      },
    }),
  );

  const uploadUrl = await getSignedUrl(
    s3Client,
    new PutObjectCommand({
      Bucket: bucketName,
      Key: `${userId}/${fileName}`,
    }),
    { expiresIn: 60 },
  );

  return {
    statusCode: 200,
    body: JSON.stringify({ uploadUrl }),
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
  };
};

In the code above, we:

Read the environment variables. We need the name of the DynamoDB table and the name of the S3 bucket to perform our operations.
Read the body of the request. The body of the request contains the userId and the fileName of the file we want to upload. We will use this information to store the file in the DynamoDB table.
Put an item in the DynamoDB table. We store the userId and the fileName in the DynamoDB table, so that we can list all the files uploaded by a user later.
Generate a presigned URL. We generate a presigned URL that will allow the user to upload a file on S3. The presigned URL is valid for 60 seconds, and is signed by our backend, so that AWS can verify that the request comes from our backend. During the next 60 seconds, any person using this url will be able to upload a file on S3, without having to go through our backend. This is the magic of presigned URLs!
Return the presigned URL. We return the presigned URL to the frontend, so that it can upload the file on S3. Don't forget to enable CORS on the API Gateway resource, so that the frontend can call the API.

Generate download presigned URLs from a Lambda function

Now, let's write the code for the listFiles Lambda function. Based on our IAC, we have to write the code inside the listFiles.ts file.



import { DynamoDBClient, QueryCommand } from '@aws-sdk/client-dynamodb';
import { GetObjectCommand, S3Client } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const dynamoDBClient = new DynamoDBClient({});
const s3Client = new S3Client({});

export const handler = async ({
  body,
}: {
  body: string;
}): Promise<{ statusCode: number; body: string; headers: unknown }> => {
  const tableName = process.env.TABLE_NAME;
  const bucketName = process.env.BUCKET_NAME;

  if (tableName === undefined || bucketName === undefined) {
    throw new Error('Missing environment variable');
  }

  const { userId } = JSON.parse(body) as { userId?: string };

  if (userId === undefined) {
    return Promise.resolve({
      statusCode: 400,
      body: JSON.stringify({ message: 'Missing userId' }),
      headers: {
        'Access-Control-Allow-Origin': '*',
      },
    });
  }

  const { Items: files = [] } = await dynamoDBClient.send(
    new QueryCommand({
      TableName: tableName,
      KeyConditionExpression: 'PK = :pk',
      ExpressionAttributeValues: {
        ':pk': { S: userId },
      },
    }),
  );

  const filesWithPresignedUrls = await Promise.all(
    files.map(async ({ SK }) => {
      const fileName = SK?.S ?? '';

      const downloadUrl = await getSignedUrl(
        s3Client,
        new GetObjectCommand({
          Bucket: bucketName,
          Key: `${userId}/${fileName}`,
        }),
        { expiresIn: 60 },
      );

      return { fileName, downloadUrl };
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({ files: filesWithPresignedUrls }),
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
  };
};

In the code above, we:

Read the environment variables. Same as before.
Read the body of the request. Same as before.
Query the DynamoDB table. We query the DynamoDB table to get all the files uploaded by the user.
Generate a presigned URL for each file. For each file, we generate a presigned URL that will allow the user to download the file from S3. These presigned URLs allow each file to be downloaded by the frontend, without having to go through our backend.
Return the presigned URLs. We return the presigned URLs to the frontend, so that it can download the files from S3. Don't forget to enable CORS on the API Gateway resource, so that the frontend can call the API.

We are done with the backend! Final step: deploy the backend on AWS.



npm i @aws-sdk/client-dynamodb @aws-sdk/client-s3 @aws-sdk/s3-request-presigner # Install required dependencies
npm i -D esbuild # We need esbuild as a devDep to bundle our Lambda functions
npm run cdk bootstrap # Only the first time you use CDK on your AWS account
npm run cdk deploy

Create a React application to upload and download files

Finally, let's create a React application that will use our backend to upload and download files. The quickest way to create a React application is to use vite, it allows you to create a simple SPA in seconds.



npm create vite

Let's create a very simple frontend, consisting of only one page containing :

A userId input to "Login" the user (you can do much better than this, but it is not the point of this article)
A List files button to list all the files uploaded by the user. This button will call the listFiles API endpoint.
A File input to chose a file and a Upload file button to upload the file on S3. This button will sequentially call the uploadFile API endpoint, and then upload the file on S3 using the presigned URL returned by the API.
An array of images, each image having a source equal to the presigned URL of the file

The code for this page looks like this:



import { useState } from 'react';

function App() {
  const [files, setFiles] = useState<{ fileName: string; downloadUrl: string }[]>([]);
  const [userId, setUserId] = useState<string>('');
  const [file, setFile] = useState<File | null>(null);

  const listFiles = async (userId: string) => {
    const response = await fetch(`${import.meta.env.VITE_API_URL}/listFiles`, {
      method: 'POST',
      body: JSON.stringify({ userId }),
    });
    const { files } = (await response.json()) as { files: { fileName: string; downloadUrl: string }[] };
    setFiles(files);
  };

  return (
    <div>
      <div>
        <input type="text" value={userId} onChange={e => setUserId(e.target.value)} />
        <button onClick={() => listFiles(userId)} disabled={userId === ''}>
          List files
        </button>
      </div>
      {userId !== '' && (
        <div>
          <p>Upload File</p>
          <input
            type="file"
            onChange={e => {
              if (e.target.files) {
                setFile(e.target.files[0]);
              }
            }}
            max={1}
            accept="image/png, image/jpeg"
          />
          <button
            onClick={async () => {
              if (file === null) {
                throw new Error('File is null');
              }

              const fileName = file.name;
              const response = await fetch(`${import.meta.env.VITE_API_URL}/uploadFile`, {
                method: 'POST',
                body: JSON.stringify({ userId, fileName }),
              });
              const { uploadUrl } = (await response.json()) as { uploadUrl: string };

              await fetch(uploadUrl, {
                method: 'PUT',
                body: file,
                headers: {
                  'Content-Type': file.type,
                },
              });

              await listFiles(userId);
            }}
            disabled={file === null}
          >
            Upload
          </button>
        </div>
      )}
      <div>
        {files.map(({ downloadUrl, fileName }) => (
          <img src={downloadUrl} key={fileName} height={300} />
        ))}
      </div>
    </div>
  );
}

export default App;

Notice I use the import.meta.env.VITE_API_URL variable to call the API. You must define this variable in the .env file of the project, based on the URL of the API you just deployed.



# .env (root of the frontend project)

VITE_API_URL=https://.......

Demo time!

When we start the app, we type a userId, and are able to list the files uploaded by this user. We can also upload a file, and see it appear in the list of files. For now, there is no file in the list, because we haven't uploaded any file yet.

I grant you that the UI is not very pretty 🤣 (this is not the point today). Now, we can use the file input to chose a file to upload.

When we click on the Upload button, the file is uploaded on S3, and we can see it appear in the list of files. The list of files is refreshed automatically after the upload!

What's next?

There are a lot of things we could do to improve this application, for example:

Implement a real authentication system
Use this authentication to restrict who can upload and download files (user1 should not be able to download files uploaded by user2!!!)
Restrict in the backend the file types that can be uploaded (extension, size, etc...)
Implement a folder system, to recreate your own Google Drive!
Improve the UI 🤣

You can find the code of this article here.

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless: Master DynamoDB!

Pierre Chollet — Thu, 26 Oct 2023 12:05:27 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. During last article, I showed you how to use contracts to use 🐝 swarmion 🐝 to create Lambda functions easily and type them strongly. Today, let's continue with the same idea to improve our developer experience with DynamoDB!

I already wrote an article that is an introduction to DynamoDB. If you go and read it, you will quickly see that while being a great database, DynamoDB is painful to use in Lambda functions: items have a weird format, are not typed and building commands like queries does not feel natural. In this article, follow me through the process of improving our code step by step!

Follow me on twitter 🚀

What will we do today?

Create two very simple Lambda functions: createUser and listUsers, using the classic DynamoDB SDK
- Users type will be : { username: string; email: string; age: number; adult: boolean }
Replace the DynamoDB SDK with the DocumentClient SDK, to get rid of the weird .S .N .BOOL ... format
Replace the DocumentClient SDK with dynamodb-toolbox, to get a more natural API and strongly typed queries

If you have difficulties to follow, you can find the code at the end of the article here.

Find sls-mentor on Github ⭐️

The classic (and painful) DynamoDB experience

Setting up a DynamoDB table and Lambda functions

Like all the articles of this series, we will use the AWS CDK with Typescript to create our infrastructure. If you are not familiar with it, go back to my first article to learn how to get started.

First, let's start by writing the Infrastructure as code in the CDK stack. We will create a DynamoDB table, an API and two Lambda functions, while not forgetting to give the right permissions and environment variables to the Lambda functions to access the DynamoDB table.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from 'path';

// Use the Node18 runtime which provides the aws-sdk v3 natively
// This way our Lambda functions bundles will be smaller
const sharedLambdaConfig = {
  runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
  bundling: {
    externalModules: ['@aws-sdk'],
  },
};

export class BackendStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const api = new cdk.aws_apigateway.RestApi(this, 'Api', {});

    const table = new cdk.aws_dynamodb.Table(this, 'DdbTable', {
      partitionKey: { name: 'PK', type: cdk.aws_dynamodb.AttributeType.STRING },
      sortKey: { name: 'SK', type: cdk.aws_dynamodb.AttributeType.STRING },
      billingMode: cdk.aws_dynamodb.BillingMode.PAY_PER_REQUEST,
    });

    const classicRoute = api.root.addResource('classic');

    const classicCreateUser = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ClassicCreateUser', {
      entry: join(__dirname, 'classic.ts'),
      handler: 'createUser',
      environment: {
        TABLE_NAME: table.tableName,
      },
      ...sharedLambdaConfig,
    });
    table.grantWriteData(classicCreateUser);
    classicRoute.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(classicCreateUser));

    const classicListUsers = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ClassicListUsers', {
      entry: join(__dirname, 'classic.ts'),
      handler: 'listUsers',
      environment: {
        TABLE_NAME: table.tableName,
      },
      ...sharedLambdaConfig,
    });
    table.grantReadData(classicListUsers);
    classicRoute.addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(classicListUsers));
  }
}

The NodeJsFunction construct requires esbuild to be installed in your project -> npm i -D esbuild

Compared with my previous articles, nothing really new here! The only new improvement is the clever use of the Node18 runtime, which provides the aws-sdk v3 natively. This way, we don't need to bundle the aws-sdk in our Lambda functions, which makes them smaller. We do this by specifying the externalModules option in the bundling property of the NodeJsFunction construct.

The classic DynamoDB Lambda function code

Based on the IAC we just created, we have to create a file named classic.ts in the same directory as the stack, and write the code of the lambda handlers inside. Using the classic DynamoDB SDK, it looks like this:

import { DynamoDBClient, PutItemCommand, QueryCommand } from '@aws-sdk/client-dynamodb';

const ddbClient = new DynamoDBClient({});
const tableName = process.env.TABLE_NAME ?? '';

export const createUser = async (event: { body: string }) => {
  const { username, email, age, adult } = JSON.parse(event.body) as {
    username: string;
    email: string;
    age: number;
    adult: boolean;
  };

  await ddbClient.send(
    new PutItemCommand({
      TableName: tableName,
      Item: {
        PK: { S: 'USER' },
        SK: { S: username },
        email: { S: email },
        age: { N: age.toString() },
        adult: { BOOL: adult },
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'User created successfully',
    }),
  };
};

export const listUsers = async () => {
  const { Items = [] } = await ddbClient.send(
    new QueryCommand({
      TableName: tableName,
      KeyConditionExpression: 'PK = :pk',
      ExpressionAttributeValues: {
        ':pk': { S: 'USER' },
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      users: Items.map(user => ({
        username: user.SK.S,
        email: user.email.S,
        age: +(user.age.N ?? '0'),
        adult: user.adult.BOOL ?? false,
      })),
    }),
  };
};

If you read my first dynamodb article, you will recognize the same structure. We create a DynamoDBClient and use it to send commands to DynamoDB.

CreateUser: We have to be cautious to use the .N .S .BOOL format to interact with DynamoDB, to represent string, number and boolean attributes. This is not very natural and can be confusing.
ListUser: We must use the KeyConditionExpression and ExpressionAttributeValues properties to setup our query. This is also quite painful!

Furthermore, we have to autocompletion and type safety when writing our code! This is OK in small projects, but can quickly become a nightmare in bigger projects.

But don't worry, I got you covered! First, let's replace the DynamoDB SDK with the DocumentClient SDK to get rid of the .N .S .BOOL format.

Using DocumentClient instead of the classic DynamoDB SDK

The DocumentClient SDK is another SDK developed by AWS. Basically, it is a wrapper around the classic DynamoDB SDK, which automatically marshalls and unmarshalls (serializes and deserializes) the data for you. This way, you don't have to use the .N .S .BOOL format anymore, and you can use the native types of your programming language.

Create two new Lambda functions

First, let's add two new Lambda functions to the IAC of the project in our CDK stack:

// ... previous code
const documentRoute = api.root.addResource('document');

const documentCreateUser = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'DocumentCreateUser', {
  entry: join(__dirname, 'document.ts'),
  handler: 'createUser',
  environment: {
    TABLE_NAME: table.tableName,
  },
  ...sharedLambdaConfig,
});
table.grantWriteData(documentCreateUser);
documentRoute.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(documentCreateUser));

const documentListUsers = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'DocumentListUsers', {
  entry: join(__dirname, 'document.ts'),
  handler: 'listUsers',
  environment: {
    TABLE_NAME: table.tableName,
  },
  ...sharedLambdaConfig,
});
table.grantReadData(documentListUsers);
documentRoute.addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(documentListUsers));

These two lambda functions have basically the same configuration as the previous ones, except that they use the document.ts file as entry point, and are linked to a new API route.

Use the DocumentClient SDK inside Lambda functions

Let's create a new file named document.ts in the same directory as the stack, and write the code of the lambda handlers inside. Using the DocumentClient SDK, it looks like this:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand, QueryCommand } from '@aws-sdk/lib-dynamodb';

const ddbClient = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(ddbClient);
const tableName = process.env.TABLE_NAME ?? '';

export const createUser = async (event: { body: string }) => {
  const { username, email, age, adult } = JSON.parse(event.body) as {
    username: string;
    email: string;
    age: number;
    adult: boolean;
  };

  await documentClient.send(
    new PutCommand({
      TableName: tableName,
      Item: {
        PK: 'USER',
        SK: username,
        email,
        age,
        adult,
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'User created successfully',
    }),
  };
};

export const listUsers = async () => {
  const { Items = [] } = await documentClient.send(
    new QueryCommand({
      TableName: tableName,
      KeyConditionExpression: 'PK = :pk',
      ExpressionAttributeValues: {
        ':pk': 'USER',
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      users: Items.map(user => ({
        username: user.SK,
        email: user.email,
        age: user.age,
        adult: user.adult,
      })),
    }),
  };
};

Let's break down the differences with the classic approach:

First, we setup a documentClient that will send commands. It is built from the DynamoDBClient we already had (clearly showing how it is only a wrapper around the classic SDK).
In the createUser Lambda handler, we can create a user using the PutCommand. This time, the code is easier to write and understand as the format is more natural.
Same business in the listUsers Lambda handler. The ExpressionAttributeValues is cleaner and we don't have to use the .S .N .BOOL format when mapping the results.

With little effort, we already improved our developer experience! But there is still no type safety and autocompletion. All attributes are typed as any. It would also be cool if we could avoid writing technical attributes like PK and SK in our code. Let's see how we can do that using dynamodb-toolbox!

Using dynamodb-toolbox to abstract DynamoDB technicalities

What is dynamodb-toolbox? In the documentation it is described as: "A set of tools that makes it easy to work with Amazon DynamoDB and the DocumentClient". Furthermore, it says, This is NOT an ORM!: it is a set of tools that help write commands and queries in a more natural way, but it is not an heavy abstraction layer like an ORM.

Create the last two Lambda functions

Like before, let's create the last two Lambda functions in our CDK stack:

const toolboxRoute = api.root.addResource('toolbox');

const toolboxCreateUser = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ToolboxCreateUser', {
  entry: join(__dirname, 'toolbox.ts'),
  handler: 'createUser',
  environment: {
    TABLE_NAME: table.tableName,
  },
  ...sharedLambdaConfig,
});
table.grantWriteData(toolboxCreateUser);
toolboxRoute.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(toolboxCreateUser));

const toolboxListUsers = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ToolboxListUsers', {
  entry: join(__dirname, 'toolbox.ts'),
  handler: 'listUsers',
  environment: {
    TABLE_NAME: table.tableName,
  },
  ...sharedLambdaConfig,
});
table.grantReadData(toolboxListUsers);
toolboxRoute.addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(toolboxListUsers));

Use dynamodb-toolbox inside Lambda functions

dynamodb-toolbox works by creating entities. Entities are a way to describe a format for your items. You can define multiple entities inside a single table (it's called single-table-design), but here we only need to create a UserEntity. Then, you can use Entities to perform actions like creating, updating, deleting and querying items with a defined format inside the table.

Let's create a userEntity.ts file containing the definition of our UserEntity:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
import { Entity, Table } from 'dynamodb-toolbox';

const ddbClient = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(ddbClient);

const table = new Table({
  // WARNING: Only import this file in Lambda functions that have this environment variable
  name: process.env.TABLE_NAME ?? '',
  partitionKey: 'PK',
  sortKey: 'SK',
  DocumentClient: documentClient,
});

export const UserEntity = new Entity({
  table,
  name: 'USER',
  attributes: {
    username: { type: 'string', required: true },
    email: { type: 'string', required: true },
    age: { type: 'number', required: true },
    adult: { type: 'boolean', required: true },
    PK: { partitionKey: true, default: 'USER', hidden: true },
    SK: { sortKey: true, default: ({ username }: { username: string }) => username, hidden: true },
  },
});

As you can see at the beginning of the code, dynamodb-toolbox is built over the DocumentClient we previously covered. Here, the interesting part of the code is the definition of the attributes of the entity.

username, email, age and adult are classical attributes, that have string, number and boolean types. We also specify that they are required, so that Typescript checks for us that we don't forget to set them when we create a user.
PK is the partition key of the table. We know that it is a constant value, so we set it by default (this way there is no risk of making a typo inside Lambda handlers). Furthermore, we make it hidden so that it is not returned when we query the table. (we already know that queried items are users because we use the UserEntity)
SK is the sort key of the table. We can set it by default dynamically, using the username attribute specified above. This way, when using the UserEntity to create a user, we don't have to set the SK attribute. It hides technical details from us, and makes our code cleaner. We also hide it so that returned items only contain the attributes we care about.

Now, let's use the UserEntity inside our Lambda handlers, by creating a new file named toolbox.ts in the same directory as the stack:

import { UserEntity } from './userEntity';

export const createUser = async (event: { body: string }) => {
  const { username, email, age, adult } = JSON.parse(event.body) as {
    username: string;
    email: string;
    age: number;
    adult: boolean;
  };

  await UserEntity.put({
    username,
    email,
    age,
    adult,
  });

  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'User created successfully',
    }),
  };
};

export const listUsers = async () => {
  const { Items = [] } = await UserEntity.query('USER');

  return {
    statusCode: 200,
    body: JSON.stringify({
      users: Items.map(user => ({
        username: user.username,
        email: user.email,
        age: user.age,
        adult: user.adult,
      })),
    }),
  };
};

The code has never been so clean! No more mentions to PK and SK, no more .S .N .BOOL format, no more ExpressionAttributeValues and KeyConditionExpression. The code is also strongly typed! If you forget to set an attribute when putting an item, Typescript will tell you. The result of the query is also typed, and you can use autocompletion to access the attributes of the returned items.

⬆️ If we forget to set adult, Typescript will tell us!

⬆️ The result of the query is typed

Finally, as you can see in the last screenshot, dynamodb-toolbox automatically generates attributes created and modified, that are the date of creation and last modification of the item. This is very useful to keep track of the history of your items!

Stay tuned for dynamodb-toolbox v1 🚀

dynamodb-toolbox is currently in its 0.9.x version, many things can be improved like the typing of complex LIST and MAP attributes for example. Be reassured, the author is working on a v1 that will be released soon! It will use a simpler syntax and will be 100% type-safe, check this article to learn more about the beta version of this V1!

Conclusion

There is no demo in this article as it would be quite trivial, but if you deploy the code, you can test the three API endpoints and see that they work the same way. The only difference is the code of the Lambda handlers, which is cleaner and more natural when using DocumentClient, and even better, dynamodb-toolbox.

dynamodb-toolbox has much more to offer, like the possibility to use indexes, perform updates, scans, get etc... I encourage you to read the documentation to learn more about it!

You can find the code of this article here.

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless: Strong Lambda Types!

Pierre Chollet — Thu, 05 Oct 2023 09:33:42 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. During last article, I showed you how to deploy a frontend on AWS, and how to make it interact with your serverless backend.

Maybe you noticed that we didn't share any types between our frontend and our backend: we weren't taking advantage of the fact that we were using the same language on both sides. In this article, we will see how to share types between our frontend and our backend, how to use them to make our code more robust, and how to use contracts to speed up our development!

⚠️ This article is a direct follow-up of the previous one. If you haven't read it yet, I strongly recommend you to do so before reading this one! You can find the code where I left it here.

⬇️ I post serverless content very regularly, if you want more ⬇️

Follow me on twitter 🚀

What will we do today?

Create contracts to share types between our frontend and our backend
Use them to define lambda functions faster and more robustly
Use them to make API calls from our frontend faster and more robustly

If you have difficulties to follow, you can find the code at the end of the article here.

Find sls-mentor on Github ⭐️

Where did we leave last time?

Last time, we created a simple backend consisting of two Lambda functions: createUser and listUsers. This lambda functions were interacting with an API and a DynamoDB table. We also created a frontend that was calling our backend to create and list users.

The architecture looked like this:

The repository looked like this:

On the code side, the handlers of our lambda functions were poorly typed: every thing was hardcoded, there was no relationship between the backend and the frontend: the code was hard to maintain (multiple sources of truth).

Using contracts to share types between our frontend and our backend

What are contracts? Contracts are a piece of code that defines an interface between two parts of your application. In our case, we will use contracts to define the interface between our frontend and our backend. Our two routes will be defined by a contract, containing information such as :

The path of the route
The HTTP method of the route
The input of the route
The output of the route

This information will be available to both our frontend and our backend, so that we can use it to make our code more robust and faster to write.

Creating serverless contracts

We want our contracts to be available to many packages, so taking advantage of NX, all we have to do is to create a new package called contracts, that will be shared between our frontend and our backend.

cd packages
mkdir contracts && cd contracts
npm init -y # We create a new package, and we initialize it
cp ../backend/tsconfig.json ./tsconfig.json # We re-use the TS config of our backend

But how to create a contract? The best library I know is called @swarmion/serverless-contracts, created by team Swarmion. It exports super useful utils that will make the contracts we create very powerful in the scope of an AWS serverless application. Follow the article and you will understand why!

Let's install the library in our contracts package:

npm install @swarmion/serverless-contracts -S

Now, let's create a index.ts file in our contracts package, and let's create our first contract:

import { ApiGatewayContract } from '@swarmion/serverless-contracts';

export const listUsersContract = new ApiGatewayContract({
  id: 'listUsers',
  path: '/users',
  method: 'GET',
  integrationType: 'restApi',
  outputSchemas: {
    [200]: {
      type: 'object',
      properties: {
        users: {
          type: 'array',
          items: {
            type: 'object',
            properties: {
              firstName: { type: 'string' },
              lastName: { type: 'string' },
              email: { type: 'string' },
            },
            required: ['firstName', 'lastName', 'email'],
            additionalProperties: false,
          },
        },
      },
      required: ['users'],
      additionalProperties: false,
    } as const, // This as const is important, it will make sure that the type of the object is inferred correctly
  },
});

Let's break down this code:

First 4 keys are used to define the route of our contract: the path, the method, the integration type (restApi or httpApi).
The outputSchemas key is used to define the output of our route. In our case, we define that the output of our route will be a 200 response, containing an object with a users key, containing an array of users. Each user will be an object with a firstName, a lastName and an email.

You can define as many outputs as you want, for many different HTTP codes. This can allow you to correctly handle errors in your frontend, for example.

The syntax used to define the output is called JSON Schema. It is a standard way to define the structure of a JSON object. If you know zod, the spirit is similar. Based on Swarmion's roadmap, it will be possible to use zod schemas to defined contracts in the future, which will be super cool!

Let's add the contract for createUser in the same file:

// Previous code

export const createUserContract = new ApiGatewayContract({
  id: 'createUser',
  path: '/users',
  method: 'POST',
  integrationType: 'restApi',
  bodySchema: {
    type: 'object',
    properties: {
      email: { type: 'string' },
      firstName: { type: 'string' },
      lastName: { type: 'string' },
    },
    additionalProperties: false,
    required: ['email', 'firstName', 'lastName'],
  } as const,
  outputSchemas: {
    [200]: {
      type: 'object',
      properties: {
        message: { type: 'string' },
      },
      additionalProperties: false,
      required: ['message'],
    } as const,
  },
});

Like in the previous contract, we define information relative to the api path, the outputs of the route, and we also define the input of the route, using the bodySchema key. Here the body of the request will be an object with an email, a firstName and a lastName, with no additional properties allowed.

We are done with the creation of our contracts! 🎉 Now, time to refactor our backend and our frontend to use them, and to see how they can help us!

Using serverless contracts to define lambda functions

Additionally to ways to define contracts, Swarmion also export its killer feature: the getHandler function. This function generates a fully typed lambda handler, based on the contract you pass to it. Let's see how it works!

First, let's install the library in our backend package:

npm install @swarmion/serverless-contracts -S
npm install @middy/core @middy/http-cors -S # Will be useful for CORS managementt

Then, let's refactor our lambda functions to use the contracts we created:

import { DynamoDBClient, QueryCommand } from '@aws-sdk/client-dynamodb';
import { listUsersContract } from '../../../contracts';
import { getHandler } from '@swarmion/serverless-contracts';
import middy from '@middy/core';
import cors from '@middy/http-cors';

const client = new DynamoDBClient({});

const main = getHandler(listUsersContract, {
  validateInput: false,
  validateOutput: false,
  returnValidationErrors: false,
})(async () => {
  const tableName = process.env.TABLE_NAME;

  if (!tableName) {
    throw new Error('Missing TABLE_NAME');
  }

  const { Items } = await client.send(
    new QueryCommand({
      TableName: tableName,
      KeyConditions: {
        PK: {
          ComparisonOperator: 'EQ',
          AttributeValueList: [{ S: 'USER' }],
        },
      },
    }),
  );

  const users = (Items ?? []).map(item => ({
    email: item.SK.S ?? '',
    firstName: item.firstName.S ?? '',
    lastName: item.lastName.S ?? '',
  }));

  return {
    statusCode: 200,
    body: {
      users,
    },
  };
});

export const handler = middy(main).use(cors());

If you compare this snippet with the code before refactoring, you will see that barely nothing changed. The only difference is that we are using the getHandler function to generate our handler. This makes it so that our handler now fully validates the input and the output of our lambda function, based on the contract we passed to it. Furthermore, there is no more need to do some JSON.parse and JSON.stringify, as the handler does it for us now!

I also decided to use the middy library to add CORS management to our lambda function. This will allow us to call our lambda function from our frontend, without having to worry about CORS.

Notice that I disabled the validation of the input and the output of our lambda function. For now, the lambda doesn't validate at runtime that the input and the output of the lambda function are correct. If you want it to do so, you can remove the validateInput and validateOutput keys from the options and instead pass a ajv instance to the ajv key. This will make the lambda validate the input and the output of the lambda function at runtime, and throw an error if the input or the output is not correct (see swarmion docs for more info).

The handler of createUser is quite similar:

import { DynamoDBClient, PutItemCommand } from '@aws-sdk/client-dynamodb';
import { getHandler } from '@swarmion/serverless-contracts';
import { createUserContract } from '../../../contracts';
import middy from '@middy/core';
import cors from '@middy/http-cors';

const client = new DynamoDBClient({});

const main = getHandler(createUserContract, {
  validateInput: false,
  validateOutput: false,
  returnValidationErrors: false,
})(async event => {
  const tableName = process.env.TABLE_NAME;

  if (!tableName) {
    throw new Error('Missing TABLE_NAME');
  }

  const { email, firstName, lastName } = event.body;

  await client.send(
    new PutItemCommand({
      TableName: tableName,
      Item: {
        PK: { S: 'USER' },
        SK: { S: email },
        firstName: { S: firstName },
        lastName: { S: lastName },
      },
    }),
  );

  return {
    statusCode: 200,
    body: {
      message: 'User created',
    },
  };
});

export const handler = middy(main).use(cors());

Here, the real power of Swarmion contracts comes from the fact that I can access the body of the request directly from the event object, without having to parse it. The body is also fully typed, based on the contract we passed to the getHandler function.

Using serverless contracts in the Infrastructure as Code

Now that we have contracts, we can use them to define our infrastructure as code (the CDK code). This will ensure that lambda functions are configured on the correct path and with the correct HTTP method.

In my backend-stack.ts file, I can simplify some code to write:

api.root.resourceForPath(createUserContract.path).addCorsPreflight({
  allowOrigins: cdk.aws_apigateway.Cors.ALL_ORIGINS,
  allowMethods: cdk.aws_apigateway.Cors.ALL_METHODS,
  allowHeaders: cdk.aws_apigateway.Cors.DEFAULT_HEADERS,
});
api.root
  .resourceForPath(listUsersContract.path)
  .addMethod(listUsersContract.method, new cdk.aws_apigateway.LambdaIntegration(listUsers));
api.root
  .resourceForPath(createUserContract.path)
  .addMethod(createUserContract.method, new cdk.aws_apigateway.LambdaIntegration(createUser));

Using serverless contracts in the frontend

The second place where contracts will be extremely useful is in the frontend. Indeed, we can use them to generate the API calls we make to our backend.

First, let's install the library in our frontend package:

npm install @swarmion/serverless-contracts -S

Then, in our frontend, we can replace the old calls to fetch with calls generated from contracts:

import { getFetchRequest } from '@swarmion/serverless-contracts';
import { createUserContract, listUsersContract } from '../../contracts';

// ... rest of the component

const syncUsers = async () => {
  const {
    body: { users },
  } = await getFetchRequest(listUsersContract, fetch, {
    baseUrl: import.meta.env.VITE_API_URL,
  });

  setUsers(users);
};

You can't see it on the article, but the users variable is correctly typed: it is an array of users, as defined in the contract. Plus we don't have to specify the path of the route or the HTTP method!

Same for the submit of the form:

const onFormSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
  e.preventDefault();
  await getFetchRequest(createUserContract, fetch, {
    baseUrl: import.meta.env.VITE_API_URL,
    body: {
      firstName,
      lastName,
      email,
    },
  });
  setUsers([...users, { firstName, lastName, email }]);
  await syncUsers();
};

We don't have to specify that it is a POST request, and the body of the request is correctly typed, based on the contract we passed to the getFetchRequest function.

Conclusion

In this article, we saw how to use contracts to share types between our frontend and our backend, and how to use them to make our code more robust and faster to write:

Type validation of the input and the output of our lambda functions
No more need to parse the body of the request
No more need to specify the path of the route or the HTTP method in the frontend
Etc...

There are so many other things we can do with contracts, such as:

Generating a Swagger documentation of our API
Doing runtime validation of the input and the output of our lambda functions
Using contracts from other services such as EventBridge

⭐️ If you enjoyed this article, go give a huge star to the Swarmion repository on Github, they truly deserve it! ⭐️

You can find the code of this article here.

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless: Deploy a frontend!

Pierre Chollet — Wed, 20 Sep 2023 12:34:08 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. During the last 11 articles, we tackled together a lot of "backend" services, creating small apps and interacting with them with HTTP requests. Today, let's start to close the loop and create a frontend, deployed on AWS, and able to interact with our backend services.

What will we do today?

Create a simple monorepo containing a frontend and a backend
Create a minimalistic backend
Create a frontend interacting with it, and deploy it on AWS!

All the code of this article is available on this repository.

⬇️ I post serverless content very regularly, if you want more ⬇️

Follow me on twitter 🚀

Find sls-mentor on Github ⭐️

How to bring frontend and backend together: the monorepo

Since the beginning of this series, I made the choice to create my backend apps using TypeScript. One of the underlying reasons was to be able to share code between my frontend and my backend, because the main frontend frameworks: react, next, vue, etc... are all compatible with TypeScript. This will pay off today, because we will be able to create a monorepo containing both our frontend and our backend, plus the code able to deploy the frontend on AWS.

The architecture of the monorepo will look like this:

It will contain three packages:

Backend: deploy an API, some lambdas, and a database
Frontend: a simple react app, built using vite. The frontend will rely on the API url defined in the backend package
Frontend deploy: a simple package that will deploy the built code of the frontend on AWS, using S3 and CloudFront

Everything will interact together thanks to nx, a tool that allows to have multiple package.json files in the same repository, and to run commands on all of them at once. It is a very powerful tool, and I strongly advise you to check it out!

Build a monorepo containing a frontend and a serverless backend

Create the monorepo

To get started, run the following command (be sure to use node 18 to follow this article)



npx create-nx-workspace@latest

Chose a name for your workspace and select default options for the rest. This command creates a monorepo. This monorepo is designed to contain packages into a packages folder. This is where we will create our backend, frontend and frontend-deploy packages.

Create the backend package

To build our backend, let's use the AWS CDK, as always in this series. If you need a refresher, you can check the other articles of my series. Run these commands in your cli to create the backend package:



mkdir packages && cd packages
mkdir backend && cd backend
npx cdk init app --language typescript

This creates a new CDK project. The important files in this projects are: bin/backend.ts and lib/backend-stack.ts. The first one is the entry point of the CDK app where you can specify the region and account you want to deploy to. The second one contains the stack that will be deployed, here is where we will write our code.

Lets create a very simple app, a REST API with 2 routes: createUser and listUsers. It will be made of 2 lambdas, and a DynamoDB table and a REST API from API Gateway.

Here is the code of the stack:



import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from 'path';

export class BackendStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create the API
    const api = new cdk.aws_apigateway.RestApi(this, 'RestApi', {});

    // Create the /users route, add CORS to it to allow the front to call it
    const users = api.root.addResource('users');
    users.addCorsPreflight({
      allowOrigins: cdk.aws_apigateway.Cors.ALL_ORIGINS,
      allowMethods: cdk.aws_apigateway.Cors.ALL_METHODS,
      allowHeaders: cdk.aws_apigateway.Cors.DEFAULT_HEADERS,
    });

    // Create the DynamoDB table
    const table = new cdk.aws_dynamodb.Table(this, 'UsersTable', {
      partitionKey: {
        name: 'PK',
        type: cdk.aws_dynamodb.AttributeType.STRING,
      },
      sortKey: {
        name: 'SK',
        type: cdk.aws_dynamodb.AttributeType.STRING,
      },
      billingMode: cdk.aws_dynamodb.BillingMode.PAY_PER_REQUEST,
    });

    // Create the listUsers lambda, link it to the API
    const listUsers = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ListUsers', {
      entry: join(__dirname, 'functions', 'listUsers.ts'),
      handler: 'handler',
      environment: {
        TABLE_NAME: table.tableName,
      },
      bundling: {
        minify: true,
        externalModules: ['@aws-sdk/client-dynamodb'],
      },
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
    });
    table.grantReadData(listUsers);
    users.addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(listUsers));

    // Create the createUser lambda, link it to the API
    const createUser = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'CreateUser', {
      entry: join(__dirname, 'functions', 'createUser.ts'),
      handler: 'handler',
      environment: {
        TABLE_NAME: table.tableName,
      },
      bundling: {
        minify: true,
        externalModules: ['@aws-sdk/client-dynamodb'],
      },
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
    });
    table.grantWriteData(createUser);
    users.addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(createUser));
  }
}

Nothing really new in this code snippet if you already read the beginning of my series. We create a DynamoDB table, and two lambdas. The lambdas are linked to the API, and the DynamoDB table is linked to the lambdas.

The only new thing is the CORS configuration of the /users route. In my previous articles, I did not need it as I was using Postman to call my API. Now, I need to be able to call it from my frontend, so I need to allow CORS. To stay simple, I allow all origins, all methods and all headers. TO go further, you should only allow your frontend origin, and only the methods and headers you need.

Final step of the backend development, create the code of the 2 lambda functions. Create a functions folder in the backend package, and create the two following files:



// createUser.ts
import { DynamoDBClient, PutItemCommand } from '@aws-sdk/client-dynamodb';

const client = new DynamoDBClient({});

export const handler = async (event: {
  body: string;
}): Promise<{ statusCode: number; body: string; headers?: Record<string, string> }> => {
  const tableName = process.env.TABLE_NAME;

  if (!tableName) {
    throw new Error('Missing TABLE_NAME');
  }

  const { email, firstName, lastName } = JSON.parse(event.body) as {
    email: string;
    firstName: string;
    lastName: string;
  };

  if (!email || !firstName || !lastName) {
    return {
      statusCode: 400,
      body: 'Missing parameters',
    };
  }

  await client.send(
    new PutItemCommand({
      TableName: tableName,
      Item: {
        PK: { S: 'USER' },
        SK: { S: email },
        firstName: { S: firstName },
        lastName: { S: lastName },
      },
    }),
  );

  return {
    statusCode: 200,
    body: 'User created',
    headers: {
      'Content-Type': 'application/json',
      'Access-Control-Allow-Origin': '*',
    },
  };
};



// listUsers.ts
import { DynamoDBClient, QueryCommand } from '@aws-sdk/client-dynamodb';

const client = new DynamoDBClient({});

export const handler = async (): Promise<{ statusCode: number; body: string; headers: Record<string, string> }> => {
  const tableName = process.env.TABLE_NAME;

  if (!tableName) {
    throw new Error('Missing TABLE_NAME');
  }

  const { Items } = await client.send(
    new QueryCommand({
      TableName: tableName,
      KeyConditions: {
        PK: {
          ComparisonOperator: 'EQ',
          AttributeValueList: [{ S: 'USER' }],
        },
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify(
      Items?.map(item => ({
        email: item.SK.S,
        firstName: item.firstName.S,
        lastName: item.lastName.S,
      })),
    ),
    headers: {
      'Content-Type': 'application/json',
      'Access-Control-Allow-Origin': '*',
    },
  };
};

One more time, nothing new in these two handlers, except the CORS configuration. They only use the aws sdk to interact with DynamoDB, and return a 200 or 400 status code depending on the result of the request. The CORS headers in the returned object allow the frontend to call the API. Like in the CDK code, you should restrict the origin, methods and headers to the ones you need.

Time to deploy our backend! Run the following commands:



npm run cdk bootstrap
npm run cdk deploy

Take notes of the API url, we will need it when we will create the frontend.

Create the frontend package

Time to create the frontend package! We will use vite as it is a very simple way to create a react app. Run the following commands:



cd packages
npm create vite
cd frontend
npm i

Chose react as framework, typescript as language, and default options for the rest. This creates a new react app in the frontend folder. First thing to do is to add the API url to a .env file at the root of packages/frontend



VITE_API_URL="THE API URL YOU NOTED EARLIER"

Now, let's delete every file inside the src folder, except main.tsx, App.tsx and vite-env.ts. We want to create an app as simple as possible to focus on the deployment. Here is the code of App.tsx:



import { useEffect, useState } from 'react';

function App() {
  const [users, setUsers] = useState<{ firstName: string; lastName: string; email: string }[]>([]);
  const [firstName, setFirstName] = useState<string>('');
  const [lastName, setLastName] = useState<string>('');
  const [email, setEmail] = useState<string>('');

  const syncUsers = async () => {
    const res = await fetch(`${import.meta.env.VITE_API_URL}/users`);
    const body = (await res.json()) as { firstName: string; lastName: string; email: string }[];

    setUsers(body);
  };

  useEffect(() => {
    void syncUsers();
  }, []);

  const onFormSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    await fetch(`${import.meta.env.VITE_API_URL}/users`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ firstName, lastName, email }),
    });
    setUsers([...users, { firstName, lastName, email }]);
    await syncUsers();
  };

  return (
    <div>
      <div>
        <h1>Users</h1>
        <form onSubmit={onFormSubmit}>
          <input
            type="text"
            name="firstName"
            value={firstName}
            onChange={e => setFirstName(e.target.value)}
            placeholder="First Name"
          />
          <input
            type="text"
            name="lastName"
            value={lastName}
            onChange={e => setLastName(e.target.value)}
            placeholder="Last Name"
          />
          <input
            type="text"
            name="email"
            value={email}
            onChange={e => setEmail(e.target.value)}
            placeholder="Email Address"
          />
          <button type="submit">Submit</button>
        </form>
      </div>
      <table>
        <thead>
          <tr>
            <th>First Name</th>
            <th>Last Name</th>
            <th>Email Address</th>
          </tr>
        </thead>
        <tbody>
          {users.map(({ email, firstName, lastName }) => (
            <tr key={email}>
              <td>{firstName}</td>
              <td>{lastName}</td>
              <td>{email}</td>
            </tr>
          ))}
        </tbody>
      </table>
    </div>
  );
}

export default App;

The App component does 2 things: it displays a form to create a new user, and it displays a table with all the users. I use fetch to interact with the API, and I use the API url from the .env file, by using import.meta.env.VITE_API_URL. This is a very simple app, but it is enough for our purpose.

Now, we can already test the frontend locally. Run npm run dev in your CLI and you will be able to interact from localhost with the AWS backend.

It works! We can create users, and they are displayed in the table. No data is lost when we refresh the page, because the data is stored in the DynamoDB table on AWS!

Before leaving the frontend, build it using the npm run build command. We will need the built code (in the dist folder) to deploy it on AWS.

Deploy the frontend

All this code is really cool, be how can you become the next Mark Zuckerberg if you can't deploy your app on the internet? Let's fix that!

One more time, we will use the AWS CDK to deploy our frontend. We will use S3 to store the built code, and CloudFront to serve it. To do this, we will create a second CDK project package, named frontend-deploy. Run the following commands:



cd packages
mkdir frontend-deploy && cd frontend-deploy
npx cdk init app --language typescript

Like for the backend, we have to modify the lib/frontend-deploy-stack.ts file to deploy resources to AWS. Here is the code:



import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from 'path';

export class FrontendDeployStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create a bucket to store the built code
    // This bucket is public
    // It contains a website index document to serve the index.html file
    const staticSiteBucket = new cdk.aws_s3.Bucket(this, 'StaticSiteBucket', {
      websiteIndexDocument: 'index.html',
      blockPublicAccess: {
        blockPublicAcls: false,
        blockPublicPolicy: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
      },
      publicReadAccess: true,
    });

    // Deploy the built code of packages/frontend
    // to the bucket automatically on every code change
    new cdk.aws_s3_deployment.BucketDeployment(this, 'DeployStaticSite', {
      sources: [cdk.aws_s3_deployment.Source.asset(join(__dirname, '../..', 'frontend', 'dist'))],
      destinationBucket: staticSiteBucket,
    });

    // Create a CloudFront distribution to serve the website
    new cdk.aws_cloudfront.Distribution(this, 'StaticSiteDistribution', {
      defaultBehavior: {
        origin: new cdk.aws_cloudfront_origins.S3Origin(staticSiteBucket),
        viewerProtocolPolicy: cdk.aws_cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
      },
    });
  }
}

In this code snippet, we create 3 resources:

A bucket to store the built code of the frontend
A bucket deployment to automatically deploy the built code to the bucket
A CloudFront distribution to serve the website

I will not go to far into the details of these resources. I published a very detailed article about deploying static website on AWS a few months ago, it goes further, tackling additional topics like custom domains, SSL certificates, etc... If you want to go further, I strongly advise you to check it out!

Now, we can deploy our frontend! Run the following commands:



npm run cdk bootstrap
npm run cdk deploy

Time to test the full app! Go to the CloudFront distribution url, and you should see the same website as before, but deployed on AWS!

Want to deploy the website on your own domain? Check out my article about it!

Conclusion

All the code of this article is available on this repository.

What could be done better?

If you read this article with attention, you probably noticed that I did not use any type safety in the frontend. My monorepo doesn't really share code between its packages. This is because I wanted to keep the code as simple as possible, to focus on the deployment.

Be sure that I will cover these topics in future articles! Type-safety and code sharing are the main reasons why I chose TypeScript for this series, and I will not let you down!

Now it's your time to shine 🌟

This series will continue, but you are at a point where you can already build your own serverless apps! I encourage you to try to build your own apps, and to deploy them on AWS.

Try to implement some funny ideas, taking advantage of AWS services like SES or SNS to send emails or SMS, or using S3 to store files. You can also try to build a more complex app, with a frontend and a backend, and deploy it on AWS. The possibilities are endless!

Let's connect!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀

Getting started with AWS serverless - DynamoDB Streams

Pierre Chollet — Thu, 31 Aug 2023 12:16:50 +0000

TL;DR

In this series, I try to explain the basics of serverless on AWS, to enable you to build your own serverless applications. With last article, we discovered SNS topics and how to use them to send notifications. In this article, we will tackle DynamoDB streams, which allow to react to changes in a DynamoDB table!

What will we do today?

I will show you how to react to items being inserted or modified inside a DynamoDB table.
We will create a simple restaurant booking application powered by DynamoDB streams.
Everything will be event-driven!

There is a lot of code in this article, because I wanted to provide an example close to a real-world application. If you only want to see the DynamoDB stream part, check the Create a DynamoDB stream linked to a Lambda function and Third Lambda function: streamTarget parts.

⬇️ I post serverless content very regularly, if you want more ⬇️

Follow me on twitter 🚀

Find sls-mentor on Github ⭐️

What are DynamoDB streams and why are they useful?

Basically, DynamoDB streams allow a target to listen to changes being made inside a DynamoDB table. When doing serverless, this target is often a Lambda function, that has side-effects based on the changes made to the table.

This feature is very powerful: it allows you to build event-driven applications, where the components are loosely coupled: side-effects are triggered by changes in the database, and not by direct calls to the functions, that allows you to build very scalable applications.

Going further, DynamoDB streams are the major building block of serverless event-sourcing patterns on AWS. They allow you to build applications that are very resilient to failures, and that can be easily scaled. I won't go into details here, but check this article by Maxime Vivier if you want to learn more about this topic.

Let's build a restaurant booking application!

In this article, we will build a simple restaurant booking application. Bookings can either be pending confirmation or confirmed. When a booking is created, it is pending and an email is sent to the restaurant with a link to confirm the booking. When the restaurant confirms the booking, the booking is marked as confirmed, and the customer receives an email with the confirmation.

To meet these requirements, we will store bookings inside a DynamoDB table, and react to items being inserted (bookings are created) or modified (bookings are confirmed) to send emails to the right people. The architecture of our application will look like this:

The most important part is in the middle of the diagram: a DynamoDB stream forwarding changes inside the table to a Lambda function. Depending on the situation, the Lambda function will dispatch events to the right Lambda to send the right email to the right person.

We will also create a link inside the email sent to the restaurant to confirm the booking, this link will trigger the confirmation process.

To develop this application, I will use the AWS CDK. I already used it in all the articles of this series. Moreover, every concept expect DynamoDB streams was already tackled before. So if you need a refresher, feel free check older articles!

Setup the resources of our application

Let's start by creating a new CDK stack containing all the resources we will need, except for the lambda functions.

import { Construct } from 'constructs';
import * as cdk from 'aws-cdk-lib';
import path from 'path';

import { restaurantBookedTemplateHtml } from './onRestaurantBooked/template';
import { reservationConfirmedTemplateHtml } from './onReservationConfirmed/template';

export class ArticleDDBStream extends Construct {
  constructor(scope: Construct, id: string, api: cdk.aws_apigateway.RestApi, identity: cdk.aws_ses.EmailIdentity) {
    super(scope, id);

    // Api to create and confirm bookings
    const api = new cdk.aws_apigateway.RestApi(this, 'api', {});

    // Table to store reservations
    const table = new cdk.aws_dynamodb.Table(this, 'ReservationsTable', {
      partitionKey: { name: 'SK', type: cdk.aws_dynamodb.AttributeType.STRING },
      sortKey: { name: 'PK', type: cdk.aws_dynamodb.AttributeType.STRING },
      billingMode: cdk.aws_dynamodb.BillingMode.PAY_PER_REQUEST,
      // We need to enable streams on the table
      stream: cdk.aws_dynamodb.StreamViewType.NEW_IMAGE,
    });

    // Event bus to dispatch events
    const bus = new cdk.aws_events.EventBus(this, 'EventBus');

    // SES Identity to send emails
    // You can also use an email address identity
    // Check my SES article for more details
    const DOMAIN_NAME = '<YOUR_DOMAIN_NAME>';
    const hostedZone = cdk.aws_route53.HostedZone.fromLookup(this, 'hostedZone', {
      domainName: DOMAIN_NAME,
    });
    const identity = new cdk.aws_ses.EmailIdentity(this, 'sesIdentity', {
      identity: cdk.aws_ses.Identity.publicHostedZone(hostedZone),
    });

    // Email template when a restaurant is booked
    const restaurantBookedTemplate = new cdk.aws_ses.CfnTemplate(this, 'RestaurantBookedTemplate', {
      template: {
        templateName: 'restaurantBookedTemplate',
        subjectPart: 'Restaurant booked',
        htmlPart: restaurantBookedTemplateHtml,
      },
    });

    // Email template when a reservation is confirmed
    const reservationConfirmedTemplate = new cdk.aws_ses.CfnTemplate(this, 'ReservationConfirmedTemplate', {
      template: {
        templateName: 'reservationConfirmedTemplate',
        subjectPart: 'Reservation confirmed',
        htmlPart: reservationConfirmedTemplateHtml,
      },
    });
  }
}

Nothing really new here, we create an api, a DynamoDB table, an event bus, an SES identity, and two SES templates. The only new thing is the stream property of the table. It allows us to create a DynamoDB stream linked to the table. The NEW_IMAGE value means that the stream will contain the new version of the item after it has been modified.

To stay short, I didn't include the content of the email templates. You can find them here and here. If you need a refresher on SES and templated emails, check this article.

Create a DynamoDB stream linked to a Lambda function

Now that we are set up, let's create a Lambda function that will be triggered by the DynamoDB stream.

// ... previous code
const streamTarget = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'StreamTarget', {
  entry: path.join(__dirname, 'streamTarget', 'handler.ts'),
  handler: 'handler',
  environment: {
    EVENT_BUS_NAME: bus.eventBusName,
  },
});

table.grantStreamRead(streamTarget);
streamTarget.addEventSourceMapping('StreamSource', {
  eventSourceArn: table.tableStreamArn,
  startingPosition: cdk.aws_lambda.StartingPosition.LATEST,
  batchSize: 1,
});
streamTarget.addToRolePolicy(
  new cdk.aws_iam.PolicyStatement({
    actions: ['events:PutEvents'],
    resources: [bus.eventBusArn],
  }),
);

Linking a Lambda function to a DynamoDB stream consists of creating an event source mapping. This mapping will trigger the Lambda function every time an item is inserted or modified inside the table.

The batchSize property allows you to specify how many items will be sent to the Lambda function at once. Here, we set it to 1, so the Lambda function will be triggered for each item.

Do not forget to grant the Lambda function the right to read the stream (input), and to add the right to put events on the event bus (output).

Create the rest of the Lambda functions

Finally, let's create the other 4 lambda functions:

2 to enable API calls to modify the table
2 to react to events dispatched by the stream target and send emails

const bookRestaurant = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'BookRestaurant', {
  entry: path.join(__dirname, 'bookRestaurant', 'handler.ts'),
  handler: 'handler',
  environment: {
    TABLE_NAME: table.tableName,
  },
});

table.grantWriteData(bookRestaurant);
api.root.addResource('bookRestaurant').addMethod('POST', new cdk.aws_apigateway.LambdaIntegration(bookRestaurant));

const confirmReservation = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'ConfirmReservation', {
  entry: path.join(__dirname, 'confirmReservation', 'handler.ts'),
  handler: 'handler',
  environment: {
    TABLE_NAME: table.tableName,
  },
});

table.grantWriteData(confirmReservation);
api.root
  .addResource('confirmReservation')
  .addResource('{reservationId}')
  .addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(confirmReservation));

const onRestaurantBookedRule = new cdk.aws_events.Rule(this, 'OnRestaurantBookedRule', {
  eventBus: bus,
  eventPattern: {
    source: ['StreamTarget'],
    detailType: ['OnRestaurantBooked'],
  },
});

const onRestaurantBooked = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'OnRestaurantBookedLambda', {
  entry: path.join(__dirname, 'onRestaurantBooked', 'handler.ts'),
  handler: 'handler',
  environment: {
    FROM_EMAIL_ADDRESS: `notifications@${identity.emailIdentityName}`,
    API_URL: api.url,
    TEMPLATE_NAME: restaurantBookedTemplate.ref,
  },
});

onRestaurantBookedRule.addTarget(new cdk.aws_events_targets.LambdaFunction(onRestaurantBooked));
onRestaurantBooked.addToRolePolicy(
  new cdk.aws_iam.PolicyStatement({
    actions: ['ses:SendTemplatedEmail'],
    resources: ['*'],
  }),
);

const onReservationConfirmedRule = new cdk.aws_events.Rule(this, 'OnReservationConfirmedRule', {
  eventBus: bus,
  eventPattern: {
    source: ['StreamTarget'],
    detailType: ['OnReservationConfirmed'],
  },
});

const onReservationConfirmed = new cdk.aws_lambda_nodejs.NodejsFunction(this, 'OnReservationConfirmedLambda', {
  entry: path.join(__dirname, 'onReservationConfirmed', 'handler.ts'),
  handler: 'handler',
  environment: {
    FROM_EMAIL_ADDRESS: `notifications@${identity.emailIdentityName}`,
    TEMPLATE_NAME: reservationConfirmedTemplate.ref,
  },
});

onReservationConfirmedRule.addTarget(new cdk.aws_events_targets.LambdaFunction(onReservationConfirmed));
onReservationConfirmed.addToRolePolicy(
  new cdk.aws_iam.PolicyStatement({
    actions: ['ses:SendTemplatedEmail'],
    resources: ['*'],
  }),
);

The first two functions are very simple: they allow to create bookings and confirm them. They are linked to the API, so they can be called by anyone. They have the permission to edit the DynamoDB table. I used a GET endpoint to confirm a booking, in order to be able to use a link inside the email sent to the restaurant.

The last two functions are linked to the event bus. They will be triggered by the stream target, and will send emails to the right people. I used SES templated emails to send the emails, so I had to add the right to send templated emails to the Lambda functions.

Implement the Lambda functions

Now that we have all the resources we need, let's implement the Lambda functions! To get started, let's settle on the format of the data we will store inside the table. It will simplify the code.

// types.ts
export type Reservation = {
  // PK='RESERVATION'
  id: string; // SK
  firstName: string; // S
  lastName: string; // S
  email: string; // S
  dateTime: string; // S
  partySize: number; // N
  // status: S
};

First Lambda function: `bookRestaurant`

import { DynamoDBClient, PutItemCommand } from '@aws-sdk/client-dynamodb';

import { v4 as uuid } from 'uuid';
import { Reservation } from '../types';

const client = new DynamoDBClient({});

export const handler = async ({ body }: { body: string }): Promise<{ statusCode: number; body: string }> => {
  const tableName = process.env.TABLE_NAME;

  if (tableName === undefined) {
    throw new Error('TABLE_NAME environment variable must be defined');
  }

  const { firstName, lastName, email, dateTime, partySize } = JSON.parse(body) as Partial<Reservation>;

  if (
    firstName === undefined ||
    lastName === undefined ||
    email === undefined ||
    dateTime === undefined ||
    partySize === undefined
  ) {
    return {
      statusCode: 400,
      body: 'Bad request',
    };
  }

  const reservationId = uuid();

  await client.send(
    new PutItemCommand({
      TableName: tableName,
      Item: {
        PK: { S: `RESERVATION` },
        SK: { S: reservationId },
        firstName: { S: firstName },
        lastName: { S: lastName },
        email: { S: email },
        partySize: { N: partySize.toString() },
        dateTime: { S: dateTime },
        status: { S: 'PENDING' },
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      reservationId,
    }),
  };
};

Easy: we parse the body of the request, and store the reservation inside the table. We set the status to PENDING by default. Check my series of articles if you need a refresher on how to interact with DynamoDB tables and API Gateway.

Second Lambda function: `confirmReservation`

import { DynamoDBClient, UpdateItemCommand } from '@aws-sdk/client-dynamodb';

const client = new DynamoDBClient({});

export const handler = async ({
  pathParameters,
}: {
  pathParameters: { reservationId: string };
}): Promise<{ statusCode: number; body: string; headers: unknown }> => {
  const tableName = process.env.TABLE_NAME;

  if (tableName === undefined) {
    throw new Error('TABLE_NAME environment variable must be defined');
  }

  await client.send(
    new UpdateItemCommand({
      TableName: tableName,
      Key: {
        PK: { S: `RESERVATION` },
        SK: { S: pathParameters.reservationId },
      },
      UpdateExpression: 'SET #status = :status',
      ExpressionAttributeNames: {
        '#status': 'status',
      },
      ExpressionAttributeValues: {
        ':status': { S: 'CONFIRMED' },
      },
    }),
  );

  return {
    statusCode: 200,
    body: JSON.stringify({
      reservationId: pathParameters.reservationId,
    }),
    headers: {
      'Content-Type': 'application/json',
      'Access-Control-Allow-Origin': '*',
    },
  };
};

This function is also very simple: it updates the status of the reservation to CONFIRMED. There are two little tricks here:

The status attribute is a reserved keyword in DynamoDB, so we need to use an expression attribute name to update it.
We need to add the Access-Control-Allow-Origin header to the response to allow the browser to call the API. (We want the link inside the email to work)

Third Lambda function: `streamTarget`

import { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';
import { Reservation } from '../types';

type InputProps = {
  Records: {
    eventName: string;
    dynamodb: {
      NewImage: {
        PK: { S: string };
        SK: { S: string };
        email: { S: string };
        firstName: { S: string };
        lastName: { S: string };
        dateTime: { S: string };
        partySize: { N: string };
        status: { S: string };
      };
    };
  }[];
};

const client = new EventBridgeClient({});

export const handler = async ({ Records }: InputProps): Promise<void> => {
  const eventBusName = process.env.EVENT_BUS_NAME;

  if (eventBusName === undefined) {
    throw new Error('EVENT_BUS_NAME environment variable is not set');
  }

  await Promise.all(
    Records.map(async ({ dynamodb, eventName }) => {
      if (eventName !== 'INSERT' && eventName !== 'MODIFY') {
        return;
      }

      const { SK, email, firstName, lastName, dateTime, partySize } = dynamodb.NewImage;

      const eventDetail: Reservation = {
        id: SK.S,
        firstName: firstName.S,
        lastName: lastName.S,
        email: email.S,
        dateTime: dateTime.S,
        partySize: +partySize.N,
      };

      await client.send(
        new PutEventsCommand({
          Entries: [
            {
              EventBusName: eventBusName,
              Source: 'StreamTarget',
              DetailType: eventName === 'INSERT' ? 'OnRestaurantBooked' : 'OnReservationConfirmed',
              Detail: JSON.stringify(eventDetail),
            },
          ],
        }),
      );
    }),
  );
};

This is the important part of the article: the Lambda function that will be triggered by the DynamoDB stream. The function receives a list of records, each record containing the new version of an item inside the table. We filter the records to only keep the ones that are INSERT or MODIFY events, and we dispatch events on the event bus based on the event type.

We also map back the DynamoDB item to a Reservation object, to simplify the code of the other Lambda functions.

Fourth Lambda function: `onRestaurantBooked`

import { SESv2Client, SendEmailCommand } from '@aws-sdk/client-sesv2';
import { Reservation } from '../types';

const client = new SESv2Client({});
const RESTAURANT_OWNER_EMAIL_ADDRESS = 'pierrech@theodo.fr';

export const handler = async ({ detail }: { detail: Reservation }): Promise<void> => {
  const templateName = process.env.TEMPLATE_NAME;
  const apiURL = process.env.API_URL;
  const fromEmailAddress = process.env.FROM_EMAIL_ADDRESS;

  if (templateName === undefined || apiURL === undefined || fromEmailAddress === undefined) {
    throw new Error('TEMPLATE_NAME, API_URL and FROM_EMAIL_ADDRESS environment variables must be defined');
  }

  await client.send(
    new SendEmailCommand({
      FromEmailAddress: fromEmailAddress,
      Destination: {
        ToAddresses: [RESTAURANT_OWNER_EMAIL_ADDRESS],
      },
      Content: {
        Template: {
          TemplateName: templateName,
          TemplateData: JSON.stringify({
            firstName: detail.firstName,
            lastName: detail.lastName,
            dateTime: detail.dateTime,
            partySize: detail.partySize,
            apiURL,
            reservationId: detail.id,
          }),
        },
      },
    }),
  );
};

This lambda reacts to an EventBridge event. It has access to the detail of the event, which is a Reservation object. It uses this data to hydrate the SES template and send an email to the restaurant owner. It also uses an API_URL environment variable to create the link to confirm the reservation. (The creation of the link is done inside the template, check it here)

Notice the email is sent to a fixed email address. In a real application, you would probably want to store the email address of the restaurant elsewhere, and retrieve it here.

Fifth Lambda function: `onReservationConfirmed`

import { SESv2Client, SendEmailCommand } from '@aws-sdk/client-sesv2';
import { Reservation } from '../types';

const client = new SESv2Client({});

export const handler = async ({ detail }: { detail: Reservation }): Promise<void> => {
  const templateName = process.env.TEMPLATE_NAME;
  const fromEmailAddress = process.env.FROM_EMAIL_ADDRESS;

  if (templateName === undefined || fromEmailAddress === undefined) {
    throw new Error('TEMPLATE_NAME and FROM_EMAIL_ADDRESS environment variables must be defined');
  }

  await client.send(
    new SendEmailCommand({
      FromEmailAddress: fromEmailAddress,
      Destination: {
        ToAddresses: [detail.email],
      },
      Content: {
        Template: {
          TemplateName: templateName,
          TemplateData: JSON.stringify({
            firstName: detail.firstName,
            lastName: detail.lastName,
            dateTime: detail.dateTime,
            partySize: detail.partySize,
          }),
        },
      },
    }),
  );
};

Basically the same thing here, except that the email template is simpler (no link). We send the email to the customer.

And that's it!

Test the application

To test the application, we only need to execute a single API call from Postman. Let's create a booking:

Seconds later, I received an email (as a restaurant owner):

When I click on the link, the booking is confirmed. I received the confirmation email (as a customer):

It's as simple as that! The application is very scalable, because it is event-driven: we can add external API calls when the booking is confirmed, send receipts, process payments, etc... without modifying the underlying architecture of the application.

Conclusion

This was a long article, but I hope you learned a lot! DynamoDB streams are a very powerful feature of AWS, next step in this direction would be to use them to create event-sourcing applications. One more time, I recommend you to check this article if you want to learn more about this topic.

I plan to continue this series of articles on a bi-monthly basis. I already covered the creation of simple lambda functions and REST APIs, as well as interacting with DynamoDB databases and S3 buckets. You can follow this progress on my repository! I will cover new topics like front-end deployment, type safety, more advanced patterns, and more... If you have any suggestions, do not hesitate to contact me!

I you want to stay in touch here is my twitter account. I often post or re-post interesting stuff about AWS and serverless, feel free to follow me!

Follow me on twitter 🚀