Forem: Thomas Aribart

MCP Client: Building a Smart and Robust Integration to DynamoDB with DynamoDB-Toolbox

Thomas Aribart — Tue, 03 Jun 2025 16:39:57 +0000

MCP Clients are rapidly gaining traction, making backend interactions more intuitive and powerful than ever!

However, integrating them with a database like DynamoDB can be challenging. DynamoDB’s schema-less design makes schema discovery and querying difficult, and its strict reliance on well-defined access patterns means that even a small misstep can break your application.

Enter DynamoDB-Toolbox — a lightweight and type-safe query builder for DynamoDB. It already knows everything about your table schemas and access patterns, so why not use that to supercharge your MCP Server?

Well, now you can! Thanks to the new MCPTookit feature, you can launch a fully functional MCP Server in just a few lines of code — ready to query and insert data with zero extra setup 🙌

The best part? Your MCP Server is:

Smart: Runs only efficient and cost-effective queries
Robust: Catches bad input before it ever hits your database
DRY: All the power with a few lines of code

Let’s dive into a real-world example and see how DynamoDB-Toolbox can power up your MCP Server in no time.

1. Preparing our Models

This section focuses on model definition, not MCP. If you’re already using DynamoDB-Toolbox, feel free to skip ahead. Otherwise, it’s a great way to learn how the Toolbox works :)

We’ll model a Pokedex with two entities — Trainer and Pokemon — with the following requirements:

Query all trainers efficiently, ordered by birthDate
List all Pokemons for a specific trainer, ordered by captureDate

We’ll use a Single Table Design, using only a partition key in the primary index but a composite key in a GSI.

Create a Table

First, let’s create a table in AWS. Here’s an example using the AWS CDK:

import { AttributeType, TableV2 } from 'aws-cdk-lib/aws-dynamodb';

new TableV2(app, 'PokeTable', {
  tableName: 'poke-table',
  partitionKey: { name: 'PK', type: AttributeType.STRING },
  globalSecondaryIndexes: [
    {
      indexName: 'GSI1',
      partitionKey: { name: 'GSI1_PK', type: AttributeType.STRING },
      sortKey: { name: 'GSI1_SK', type: AttributeType.STRING },
    },
  ],
});

Create a DynamoDB-Toolbox `Table` Instance

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
import { Table } from 'dynamodb-toolbox/table';

const documentClient = DynamoDBDocumentClient.from(new DynamoDBClient());

const PokeTable = new Table({
  name: 'poke-table',
  partitionKey: { name: 'PK', type: 'string' },
  indexes: {
    GSI1: {
      type: 'global',
      partitionKey: { name: 'GSI1_PK', type: 'string' },
      sortKey: { name: 'GSI1_SK', type: 'string' },
    },
  },
  meta: {
    title: 'Pokedex',
    description: 'Single Table containing Trainers and their Pokemons',
  },
  documentClient,
});

Define Schemas and Entities

Example for the Trainer entity:

import { Entity } from 'dynamodb-toolbox/entity';
import { item, string } from 'dynamodb-toolbox/schema';

const TrainerEntity = new Entity({
  name: 'trainer',
  table: PokeTable,
  schema: item({
    trainerId: string().key().savedAs('PK'),
    name: string(),
    birthDate: string(),
    bio: string().optional(),
    profilePictureUrl: string(),
  }).and(prevSchema => ({
    GSI1_PK: string().default('trainers').hidden(),
    GSI1_SK: string()
      .link<typeof prevSchema>(({ birthDate }) => birthDate)
      .hidden(),
  })),
  meta: {
    title: 'Trainer',
    description: 'Represents a Trainer',
  },
});

And for the Pokemon entity:

import { Entity } from 'dynamodb-toolbox/entity';
import { item, string, number, boolean, list } from 'dynamodb-toolbox/schema';

const PokemonEntity = new Entity({
  name: 'pokemon',
  table: PokeTable,
  schema: item({
    pokemonId: string().key().savedAs('PK'),
    species: string(),
    pokeTypes: list(string()),
    level: number(),
    legendary: boolean().optional(),
    trainerId: string().optional(),
    captureDate: string().optional(),
  }).and(prevSchema => ({
    GSI1_PK: string()
      .optional()
      .link<typeof prevSchema>(
        ({ trainerId }) => `trainers/${trainerId}/pokemons`,
      )
      .hidden(),
    GSI1_SK: string()
      .optional()
      .link<typeof prevSchema>(({ captureDate }) => captureDate)
      .hidden(),
  })),
  meta: {
    title: 'Pokemon',
    description: 'Represents a Pokemon. Can be wild or captured.',
  },
});

Create Access Patterns

Now let’s define two Access Patterns based on our requirements:

import { any } from 'dynamodb-toolbox/schema';
import { AccessPattern } from 'dynamodb-toolbox/entity/actions/accessPattern';

const trainersAccessPattern = TrainerEntity.build(AccessPattern)
  .schema(any())
  .pattern(() => ({ index: 'GSI1', partition: 'trainers' }))
  .meta({
    title: 'Query all trainers',
    description: 'Returns all trainers (paginated)',
  });

const trainerPokemonsAccessPattern = PokemonEntity.build(AccessPattern)
  .schema(string())
  .pattern(trainerId => ({
    index: 'GSI1',
    partition: `trainers/${trainerId}/pokemons`,
  }))
  .meta({
    title: 'Trainer’s Pokemons',
    description:
      'Returns all Pokemons for a given trainer, ordered by capture date (paginated)',
  });

2. Launch Your MCP Server

Now that our Table, Entities, and AccessPatterns are ready, we're almost ready to launch our MCP Server!

Define a `Database`

Let’s bundle everything using the Registry and Database features:

import { Registry } from 'dynamodb-toolbox/table/actions/registry';
import { Database } from 'dynamodb-toolbox/database';

const PokeTableRegistry = PokeTable.build(Registry)
  .registerEntities(TrainerEntity, PokemonEntity)
  .registerAccessPatterns({
    trainers: trainersAccessPattern,
    trainerPokemons: trainerPokemonsAccessPattern,
  });

const PokeDB = new Database({
  pokedex: PokeTableRegistry,
  // Add more table registries here if needed
});

// Example usage:
PokeDB.tables.pokedex.entities.pokemon;
PokeDB.tables.pokedex.accessPatterns.trainers.query(...);

Authenticate to AWS

For local development, we’ll run a local MCP server, but you can also deploy over HTTP using Frederic Barthelet’s excellent Middy MCP middleware.

To authenticate via your SSO profile:

import { fromSSO } from '@aws-sdk/credential-providers';

const dynamoDbClient = new DynamoDBClient({
  region: '<AWS_REGION>',
  profile: '<SSO_PROFILE>',
  credentials: fromSSO(),
});
const documentClient = DynamoDBDocumentClient.from(dynamoDbClient);

Launch the `McpServer`

Finally, let’s launch the McpServer and use the MCPToolkit Action from DynamoDB-Toolbox to wire everything up:

import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import { MCPToolkit } from 'dynamodb-toolbox/database/actions/mcpToolkit';

const mcpServer = new McpServer({
  name: 'poke-server',
  version: '1.0.0',
});

// 👇 That's it!
PokeDB.build(MCPToolkit).addTools(mcpServer);

async function main() {
  const transport = new StdioServerTransport();
  await mcpServer.connect(transport);
  console.log('Poke-MCP Server running on stdio');
}

main().catch(error => {
  console.error('Fatal error:', error);
  process.exit(1);
});

You’re now ready to interact with your DynamoDB tables using natural language!

3. Run Your MCP Client 🙌

I used Claude as the client. Just build your code if needed, add your new Tool in the settings and restart the client.

Now we can query trainers, list Pokemons, and insert data — all interactively!

Claude was able to connect to DynamoDB 🙌 There are no trainers yet, but you can ask it to create one with some starter Pokemons:

Conclusion

With the new MCPToolkit, spinning up a fully functional MCP Server takes just a few lines of code — instantly ready to handle queries and data insertion without any extra hassle 🙌

What makes this MCP Server stand out?

Smart: Executes only the most efficient, fast, and cost-effective queries
Robust: Validates inputs to keep bad data from reaching your database
DRY: Packs all this power into just one simple line of code

Together, DynamoDB-Toolbox and MCP Clients simplify working with DynamoDB like never before. Try it out and experience a backend that’s both powerful and effortless!

Now go catch ’em all — and let your AI Assistant help! 🧠🎮

🌐 Building a multi-region Serverless API with the AWS CDK, Lambda, and DynamoDB

Thomas Aribart — Wed, 07 May 2025 16:20:44 +0000

While working on DynamoDB-Toolshack, I encountered a common challenge: ensuring fast and reliable performance for users worldwide. In today’s digital landscape, users expect snappy responses no matter their location. Meeting this demand often requires deploying globally distributed infrastructure.

For those unfamiliar, DynamoDB-Toolshack is the ultimate DynamoDB Studio for rapidly evolving applications. It elevates your DynamoDB experience with a schema-aware admin interface, data monitors and migration tools.

Discover DynamoDB-Toolshack

This guide walks you through building and deploying a multi-region serverless API using Amazon API Gateway, AWS Lambda, DynamoDB Global Tables, and the AWS CDK (Cloud Development Kit).

🧱 Architecture Overview

We’ll build the following architecture:

Amazon API Gateway – Public HTTP API to handle client requests.
AWS Lambda – Stateless compute service to process those requests.
Amazon DynamoDB Global Tables – A fully replicated database for low-latency data access across multiple regions.
AWS CDK – Infrastructure as code to define and deploy all resources.

Prerequisites

Before starting, make sure you:

Have a Route 53 hosted zone for a top-level domain (e.g., example.com), and plan to use a subdomain like api.example.com.
Write code in TypeScript.
Are familiar with the AWS CDK.

📦 Step 1: Set Up Your CDK Project

Start by initializing a new CDK app:

# Create the folder
mkdir global-api-cdk && cd global-api-cdk
# Init the CDK App
npx cdk init app --language typescript
# Add some dependencies
npm add @types/aws-lambda @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

🌍 Step 2: Choose Deployment Regions

Pick one primary region for global resources (e.g., us-east-1) and one or more additional regions for API and data replication.

// lib/regions.ts
export const globalResourcesRegion = 'us-east-1';
export const replicatedRegions = ['eu-west-1', 'sa-east-1'];

export const allRegions = [globalResourcesRegion, ...replicatedRegions];

Each replicated region will host a replica of the API and DynamoDB table. The Global Table ensures data sync across all regions.

🏗️ Step 3: Define a DynamoDB Global Table

Create a GlobalResourcesStack containing unique resources. This includes a TableV2 with replicationRegions specified:

// lib/global.ts
import { Construct } from 'constructs';
import { Stack, StackProps } from 'aws-cdk-lib';
import { AttributeType, TableV2 } from 'aws-cdk-lib/aws-dynamodb';

import { replicatedRegions } from './regions';

export class GlobalResourcesStack extends Stack {
  table: TableV2;

  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props);

    this.table = new TableV2(this, 'Table', {
      tableName: 'global-table',
      partitionKey: { name: 'id', type: AttributeType.STRING },
      replicas: replicatedRegions.map(region => ({ region })),
    });
  }
}

Create regional stacks for replicated resources. You can import the TableV2 with the fromTableName method:

// lib/local.ts
import { Construct } from 'constructs';
import { Stack, StackProps } from 'aws-cdk-lib';
import { ITableV2, TableV2 } from 'aws-cdk-lib/aws-dynamodb';

import { GlobalResourcesStack } from './global';

export class LocalResourcesStack extends Stack {
  // 👇 Use ITableV2 type as the table is in the global stack
  table: ITableV2;

  constructor(
    scope: Construct,
    id: string,
    {
      globalResources,
      ...props
    }: { globalResources: GlobalResourcesStack } & StackProps,
  ) {
    super(scope, id, props);

    // 👇 Use `fromTableName` to replace the stack region
    this.table = TableV2.fromTableName(
      this,
      'Table',
      globalResources.table.tableName,
    );
  }
}

Finally, let's define the CDK App like this:

// bin/global-api-cdk.ts
import { App } from 'aws-cdk-lib';

import { allRegions, globalResourcesRegion } from '../lib/regions';
import { GlobalResourcesStack } from '../lib/global';
import { LocalResourcesStack } from '../lib/local';

const app = new App();

const globalResources = new GlobalResourcesStack(app, 'GlobalResources', {
  env: { region: globalResourcesRegion },
  // 👇 Important to pass References from a stack to another
  crossRegionReferences: true,
});

for (const region of allRegions) {
  const localResources = new LocalResourcesStack(
    app,
    `LocalResources${region}`,
    {
      env: { region },
      globalResources,
      crossRegionReferences: region !== globalResourcesRegion,
    },
  );

  localResources.addDependency(globalResources);
}

🌐 Step 4: Set Up Custom Domains

Let's import the top domain Hosted Zone (example.com) and create a Hosted Zone for the API subdomain (api.example.com). Hosted zones are global, so let's add it to the GlobalResourcesStack:

import {
  IHostedZone,
  HostedZone,
  RecordSet,
  RecordTarget,
  RecordType,
} from 'aws-cdk-lib/aws-route53';

class GlobalResourcesStack extends Stack {
  ...
  topDomainHostedZone: IHostedZone;
  apiSubDomainHostedZone: HostedZone;
  apiSubDomainDelegationRecords: RecordSet;

  constructor(...) {
    ...

    this.topDomainHostedZone = HostedZone.fromHostedZoneAttributes(
      this,
      'TopDomainHostedZone',
      // 👇 Imported from your Top Domain configuration
      { zoneName: 'example.com', hostedZoneId: 'Z31867814UKQT2MAKXUTL' },
    );

    this.apiSubDomainHostedZone = new HostedZone(
      this,
      'ApiSubDomainHostedZone',
      { zoneName: 'api.example.com' },
    );

    // 👇 (optional) To automatically append the delegation records
    this.apiSubDomainDelegationRecords = new RecordSet(
      this,
      'ApiSubDomainDelegationRecords',
      {
        zone: this.topDomainHostedZone,
        recordType: RecordType.NS,
        recordName: this.apiSubDomainHostedZone.zoneName,
        target: RecordTarget.fromValues(
          ...this.apiSubDomainHostedZone.hostedZoneNameServers!,
        ),
      },
    );
  }
}

Next, let's create a Rest API and a regional Certificate for each replicated region:

import { IHostedZone } from 'aws-cdk-lib/aws-route53';
import {
  Certificate,
  CertificateValidation,
} from 'aws-cdk-lib/aws-certificatemanager';
import { RestApi } from 'aws-cdk-lib/aws-apigateway';

class LocalResourcesStack extends Stack {
  ...
  // 👇 Use IHostedZone type as the HostedZone is in the global stack
  apiSubDomainHostedZone: IHostedZone;
  apiCertificate: Certificate;
  api: RestApi;

  constructor(...) {
    ...
    this.apiSubDomainHostedZone = globalResources.apiSubDomainHostedZone;

    this.apiCertificate = new Certificate(this, 'ApiCertificate', {
      domainName: this.apiSubDomainHostedZone.zoneName,
      validation: CertificateValidation.fromDns(this.apiSubDomainHostedZone),
    });

    this.api = new RestApi(this, 'Api', {
      domainName: {
        domainName: this.apiSubDomainHostedZone.zoneName,
        certificate: this.apiCertificate,
      },
    });
  }
}

🛰️ Step 5: Add Latency-Based Routing

The CDK doesn’t natively support latency records, but we can use this workaround:

// lib/latencyRecord.ts
import { ARecordProps, CfnRecordSet, ARecord } from 'aws-cdk-lib/aws-route53';
import { Construct } from 'constructs';

export class LatencyRecord extends ARecord {
  constructor(
    scope: Construct,
    id: string,
    { region, ...props }: { region: string } & ARecordProps,
  ) {
    super(scope, id, props);

    const recordSet = this.node.defaultChild as CfnRecordSet;
    recordSet.region = region;
    recordSet.setIdentifier = region;
  }
}

And use it like so:

import { ApiGateway } from 'aws-cdk-lib/aws-route53-targets';
import { RecordTarget } from 'aws-cdk-lib/aws-route53';

import { LatencyRecord } from './latencyRecord';

class LocalResourcesStack extends Stack {
  ...

  constructor(...) {
    ...

    new LatencyRecord(this, 'ApiSubDomainLatencyRecord', {
      zone: this.apiSubDomainHostedZone,
      target: RecordTarget.fromAlias(new ApiGateway(this.api)),
      region: this.region,
    });
  }
}

⚙️ Step 6: Add a Lambda Function

import { NodejsFunction } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Runtime, Architecture } from 'aws-cdk-lib/aws-lambda';
import { LogGroup, RetentionDays } from 'aws-cdk-lib/aws-logs';
import { LambdaIntegration } from 'aws-cdk-lib/aws-apigateway';

class LocalResourcesStack extends Stack {
  ...
  lambda: NodejsFunction;

  constructor(...) {
    ...

    this.lambda = new NodejsFunction(this, 'Function', {
      entry: 'lib/handler.ts',
      handler: 'main',
      environment: {
        TABLE_NAME: this.table.tableName,
      },
      runtime: Runtime.NODEJS_20_X,
      architecture: Architecture.ARM_64,
      awsSdkConnectionReuse: true,
      logGroup: new LogGroup(this, 'FunctionLogGroup', {
        logGroupName: 'log-group',
        retention: RetentionDays.THREE_MONTHS,
      }),
      bundling: {
        minify: true,
        keepNames: true,
        sourceMap: true,
        mainFields: ['module', 'main'],
        externalModules: ['@aws-sdk/*'],
      },
    });

    // 👇 Add permissions to read data from the table
    this.table.grantReadData(this.lambda);

    // 👇 Set the API as Lambda trigger
    this.api.root
      .resourceForPath('/object/{id}')
      .addMethod('GET', new LambdaIntegration(this.lambda));
  }
}

Here’s a simple code file for our handler:

// lib/handler.ts
import type {
  APIGatewayProxyEventBase,
  APIGatewayProxyResult,
} from 'aws-lambda';

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';

const dynamoDbClient = new DynamoDBClient();
const documentClient = DynamoDBDocumentClient.from(dynamoDbClient);

export const main = async (
  event: APIGatewayProxyEventBase<unknown>,
): Promise<APIGatewayProxyResult> => {
  const { pathParameters } = event;
  const id = pathParameters!.id;

  const { Item } = await documentClient.send(
    new GetCommand({ TableName: process.env.TABLE_NAME, Key: { id } }),
  );

  if (Item === undefined) {
    return { statusCode: 404, body: JSON.stringify({ message: 'Not found' }) };
  }

  return { statusCode: 200, body: JSON.stringify(Item) };
};

🚀 Step 7: Deploy and Test

cdk bootstrap
cdk deploy --all

Once deployed, test your endpoint at https://api.example.com/object/{id}. Depending on your position, you should observe CloudWatch Logs in one region or another!

💭 Final Thoughts

Adding a Cognito UserPool for authentication is easy, but beyond the scope of this guide.

This architecture has many benefits:

Global low-latency access to data.
Fully managed infrastructure with high availability.
Pay-per-use with minimal operational overhead.
Easily extendable to more regions or endpoints.

However, beware of:

Eventual consistency: DynamoDB Global Tables introduce replication delays between regions.
API Gateway limitations: Some features like API keys are region-specific and not globally synchronized.

🛠️ Conclusion

Deploying a global serverless API is simpler than ever with AWS’s managed services and the CDK. With just a few lines of code, you can serve users across the world with fast response times and consistent data. Whether you're building for mobile, IoT, or global web apps, this setup gives you performance, reliability, and scale.

[UPDATED] The DynamoDB-Toolbox v1 beta is here 🙌 All you need to know!

Thomas Aribart — Mon, 09 Oct 2023 15:22:07 +0000

☝️ NOTE: This article is about the beta.1 release.

👉 If you need documentation for the beta.0 release, you may be looking for the previous version of this article.

👉 If you want to migrate from the beta.0 to the beta.1, there is a dedicated article summarizing the changes.

At Theodo, we are big fans of Jeremy Daly’s DynamoDB-Toolbox. We started using it as early as 2019 and grew fond of it... but were also well aware of its flaws 😅

One of them was that it had originally been coded in JavaScript. Although Jeremy rewrote the source code in TypeScript in 2020, it didn't handle type inference, a feature that I eventually came to implement myself in the v0.4.

However, there were still some features that we felt lacked: From declaring enums on primitives, to supporting recursive schemas and types (lists and maps sub-attributes) and polymorphism.

I was also wary of the object-oriented approach: I don’t have anything against classes, but they are not tree-shakable. Meaning that they should be kept relatively light in a serverless context. That’s what AWS went for with the v3 of their SDK, and for good reasons: Keep bundles tight!

That just wasn't the case for DynamoDB-Toolbox: I remember working on an .update method that was more than 1000 lines long... But why bundle it when you don't even need it?

So last year, I decided to throw myself into a complete overhaul of the code, with three main objectives:

Support the v3 of the AWS SDK (although support has been added in the v0.8)
Get the API and type inference on par with those of more "modern" tools like zod and electrodb
Use a more functional and tree-shakable approach

Today, I am happy to announce the v1 beta of dynamodb-toolbox is out 🙌 It includes reworked Table and Entity classes, as well as complete support for PutItem, GetItem, UpdateItem and DeleteItem commands (including conditions and projections), with Query and Scan commands soon to follow.

This article details how the new API works and the main breaking changes from previous versions - which, by the way, only concern the API: No data migration needed 🥳

Let's dive in!

Table of content

Installation
Tables
Entities
- Timestamps
- Matching the Table schema
- SavedItem and FormattedItem
Designing Entity schemas
- Schema definition
- Attributes types
- any
- <primitive>
- set
- list
- map
- record
- anyOf
- Looking forward
Computed defaults
Commands
- PutItemCommand
- GetItemCommand
- DeleteItemCommand
- UpdateItemCommand
Mocking entities in tests
Utility helpers and types
- formatSavedItem
- Condition and parseCondition
- Projection and parseProjection
- KeyInput and PrimaryKey
Errors
Conclusion

Installation

### npm
npm i dynamodb-toolbox@1.0.0-beta.1

## yarn
yarn add dynamodb-toolbox@1.0.0-beta.1

## ...and so on

The v1 is built on top the v3 of the AWS SDK. It has @aws-sdk/client-dynamodb and @aws-sdk/lib-dynamodb as peer dependencies so you’ll have to install them as well:

## npm
npm i @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

## yarn
yarn add @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

## ...and so on

Tables

Tables are defined pretty much the same way as in previous versions, but the key attributes now have a type along with their name:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
// Will be renamed Table in the official release 😉
import { TableV2 } from 'dynamodb-toolbox';

const dynamoDBClient = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(dynamoDBClient);

const myTable = new TableV2({
  name: 'MySuperTable',
  partitionKey: {
    name: 'PK',
    type: 'string', // 'string' | 'number' | 'binary'
  },
  sortKey: {
    name: 'SK',
    type: 'string',
  },
  documentClient,
});

☝️ The v1 does not support indexes yet as queries are not yet available.

The table name can be provided with a getter, which can be useful in some contexts where you may want to use the class without actually running any command (e.g. tests or deployments):

const myTable = new TableV2({
  ...
  // 👇 Only executed at command execution
  name: () => process.env.TABLE_NAME,
});

As in previous versions, the v1 classes tag your data with an entity identifier through an internal entity string attribute, saved as "_et" by default. This can be renamed at the Table level through the entityAttributeSavedAs argument:

const myTable = new TableV2({
  ...
  // 👇 defaults to "_et"
  entityAttributeSavedAs: '__entity__',
});

Entities

For Entities, the main change is that the attributes argument becomes schema:

// Will be renamed Entity in the official release 😉
import { EntityV2, schema } from 'dynamodb-toolbox';

const myEntity = new EntityV2({
  name: 'MyEntity',
  table: myTable,
  // Attributes definition
  schema: schema({ ... }),
});

Timestamps

The internal timestamp attributes are also there and behave similarly as in the previous versions. You can set the timestamps to false to disable them (default value is true), or fine-tune the created and modified attributes names:

const myEntity = new EntityV2({
  ...
  // 👇 de-activate timestamps altogether
  timestamps: false,
});

const myEntity = new EntityV2({
  ...
  timestamps: {
    // 👇 de-activate only `created` attribute
    created: false,
    modified: true,
  },
});

const myEntity = new EntityV2({
  ...
  timestamps: {
    created: {
      // 👇 defaults to "created"
      name: 'creationDate',
      // 👇 defaults to "_ct"
      savedAs: '__createdAt__',
    },
    modified: {
      // 👇 defaults to "modified"
      name: 'lastModificationDate',
      // 👇 defaults to "_md"
      savedAs: '__lastMod__',
    },
  },
});

Matching the Table schema

An important change from previous versions is that the EntityV2 key attributes are validated against the TableV2 schema, both through types and at run-time. There are two ways to match the table schema:

The simplest one is to have an entity schema that already matches the table schema (see "Designing Entity schemas"). The Entity is then considered valid and no other argument is required:

import { string } from 'dynamodb-toolbox';

const pokemonEntity = new EntityV2({
  name: 'Pokemon',
  table: myTable, // <= { PK: string, SK: string } primary key
  schema: schema({
    // Provide a schema that matches the primary key
    PK: string().key(),
    // 🙌 using "savedAs" will also work
    pokemonId: string().key().savedAs('SK'),
    ...
  }),
});

If the entity key attributes don't match the table schema, the Entity class will require you to add a computeKey property which must derive the primary key from them:

const pokemonEntity = new EntityV2({
  ...
  table: myTable, // <= { PK: string, SK: string } primary key
  schema: schema({
    pokemonClass: string().key(),
    pokemonId: string().key(),
    ...
  }),
  // 🙌 `computeKey` is correctly typed
  computeKey: ({ pokemonClass, pokemonId }) => ({
    PK: pokemonClass,
    SK: pokemonId,
  }),
});

SavedItem and FormattedItem

If you feel lost, you can always use the SavedItem and FormattedItem utility types to infer the type of your entity items:

import type { FormattedItem, SavedItem } from 'dynamodb-toolbox';

const pokemonEntity = new EntityV2({
  name: 'Pokemon',
  timestamps: true,
  table: myTable,
  schema: schema({
    pokemonClass: string().key().savedAs('PK'),
    pokemonId: string().key().savedAs('SK'),
    level: number().default(1),
    customName: string().optional(),
    internalField: string().hidden(),
  }),
});

// What Pokemons will look like in DynamoDB
type SavedPokemon = SavedItem<typeof pokemonEntity>;
// 🙌 Equivalent to:
// {
//   _et: "Pokemon",
//   _ct: string,
//   _md: string,
//   PK: string,
//   SK: string,
//   level: number,
//   customName?: string | undefined,
//   internalField: string | undefined,
// }

// What fetched Pokemons will look like in your code
type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// 🙌 Equivalent to:
// {
//   created: string,
//   modified: string,
//   pokemonClass: string,
//   pokemonId: string,
//   level: number,
//   customName?: string | undefined,
// }

Designing Entity schemas

Now let’s dive into the part that received the most significant overhaul: Schema definition.

Schema definition

Similarly to zod or yup, attributes are now defined through function builders. For TS users, this removes the need for the as const statement previously needed for type inference (so don't forget to remove it when you migrate 🙈).

You can either import the attribute builders through their dedicated imports, or through the attribute or attr shorthands. For instance, those declarations will output the same attribute schema:

import { string, attribute, attr } from 'dynamodb-toolbox';

// 👇 More tree-shakable
const pokemonName = string();
// 👇 Not tree-shakable, but single import
const pokemonName = attribute.string();
const pokemonName = attr.string();

Prior to being wrapped in a schema declaration, attributes are called warm: They are not validated (at run-time) and can be used to build other schemas. By inspecting their types, you will see that they are prefixed with $. Once frozen, validation is applied and building methods are stripped:

The main takeaway is that warm schemas can be composed while frozen schemas cannot:

import { schema } from 'dynamodb-toolbox';

const pokemonName = string();

const pokemonSchema = schema({
  // 👍 No problem
  pokemonName,
  ...
});

const pokedexSchema = schema({
  // ❌ Not possible
  pokemon: pokemonSchema,
  ...
});

You can create/update warm attributes by using dedicated methods or by providing option objects. The former provides a slick devX with autocomplete and shorthands, while the latter theoretically requires less compute time and memory usage, although it should be very minor (validation being only applied on freeze):

// Using methods
const pokemonName = string().required('always');
// Using options
const pokemonName = string({ required: 'always' });

All attributes share the following options:

required (string?="atLeastOnce") Tag a root attribute or Map sub-attribute as required. Possible values are:
- "atLeastOnce" Required in PutItem commands
- "never": Optional in all commands
- "always": Required in all commands

// Equivalent
const pokemonName = string().required();
const pokemonName = string({ required: 'atLeastOnce' });

// `.optional()` is a shorthand for `.required(”never”)`
const pokemonName = string().optional();
const pokemonName = string({ required: 'never' });

A very important breaking change from previous versions is that root attributes and Map sub-attributes are now required by default. This was made so composition and validation work better together.

💡 Outside of root attributes and Map sub-attributes, such as in a list of strings, it doesn’t make sense for sub-schemas to be optional. So, should I force users to write list(string().required()) every time OR make string validation and type inference aware of their context (ignore required in lists but not in maps)? It felt more elegant to enforce string() as required by default and prevent schemas such as list(string().optional()).

hidden (boolean?=true) Skip attribute when formatting the returned item of a command:

const pokemonName = string().hidden();
const pokemonName = string({ hidden: true });

key (boolean?=true) Tag attribute as needed to compute the primary key:

// Note: The method will also modify the `required` property to "always"
// (it is often the case in practice, you can still use `.optional()` if needed)
const pokemonName = string().key();
const pokemonName = string({ key: true, required: 'always' });

savedAs (string) Previously known as map. Rename a root or Map sub-attribute before sending commands:

const pokemonName = string().savedAs('_n');
const pokemonName = string({ savedAs: '_n' });

default: (ComputedDefault) See Computed defaults

Attributes types

Here’s the exhaustive list of available attribute types:

Any

Define an attribute of any value. No validation will be applied at run-time, and its type will be resolved as unknown:

import { any } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  metadata: any(),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   metadata: unknown
// }

You can provide default values through the defaults option or the keyDefault, putDefault and updateDefault methods. A simpler default method is also exposed. It acts similarly as putDefault, except if the attribute has been tagged as a key attribute, in which case it will act as keyDefault:

const metadata = any().default({ any: 'value' });
// 👇 Similar to
const metadata = any().putDefault({ any: 'value' });
// 👇 ...or
const metadata = any({
  defaults: {
    key: undefined,
    put: { any: 'value' },
    update: undefined,
  },
});

const keyPart = any().key().default('my-awesome-partition-key');
// 👇 Similar to
const metadata = any().key().keyDefault('my-awesome-partition-key');
// 👇 ...or
const metadata = any({
  key: true,
  defaults: {
    key: 'my-awesome-partition-key',
    // put & update defaults are not useful in `key` attributes
    put: undefined,
    update: undefined,
  },
});

const metadata = any().default(() => 'Getters also work!');

Primitives

Defines a string, number, boolean or binary attribute:

import { string, number, boolean, binary } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  pokemonType: string(),
  level: number(),
  isLegendary: boolean(),
  binEncoded: binary(),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   pokemonType: string
//   level: number
//   isLegendary: boolean
//   binEncoded: Buffer
// }

Similarly to any attributes, you can provide default values through the defaults option or the default methods:

// 🙌 Correctly typed!
const creationDate = string().default(() => new Date().toISOString());
// 👇 Similar to
const creationDate = string().putDefault(() => new Date().toISOString());
// 👇 ...or
const creationDate = string({
  defaults: {
    key: undefined,
    put: () => new Date().toISOString(),
    update: undefined,
  },
});

// 👇 Additionally fill 'creationDate' on updates if needed
import { $get } from 'dynamodb-toolbox';

const creationDate = string()
  .putDefault(() => new Date().toISOString())
  // (See UpdateItemCommand section for $get description)
  .updateDefault(() => $get('creationDate', new Date().toISOString()));
// 👇 Similar to
const creationDate = string({
  defaults: {
    key: undefined,
    put: () => new Date().toISOString(),
    update: () => $get('creationDate', new Date().toISOString()),
  },
});

const id = number().key().default(1);
// 👇 Similar to
const id = number().key().keyDefault(1);
// 👇 ...or
const id = number({
  defaults: {
    key: 1,
    // put & update defaults are not useful in `key` attributes
    put: undefined,
    update: undefined,
  },
});

Primitive types have an additional enum option. For instance, you could provide a finite list of pokemon types:

const pokemonTypeAttribute = string().enum('fire', 'grass', 'water');

// Shorthand for `.enum("POKEMON").default("POKEMON")`
const pokemonPartitionKey = string().const('POKEMON');

💡 For type inference reasons, the enum option is only available as a method, not as an object option

Set

Defines a set of strings, numbers or binaries. Unlike in previous versions, sets are kept as Set classes. Let me know if you would prefer using arrays (or being able to choose from both):

import { set } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  skills: set(string()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   skills: Set<string>
// }

Options can be provided as a 2nd argument:

const setAttr = set(string()).hidden();
const setAttr = set(string(), { hidden: true });

List

Defines a list of sub-schemas of any type:

import { list } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  skills: list(string()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   skills: string[]
// }

As in sets, options can be povided as a 2nd argument.

Map

Defines a finite list of key-value pairs. Keys must follow a string schema, while values can be sub-schema of any type:

import { map } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  nestedMagic: map({
    will: map({
      work: string().const('!'),
    }),
  }),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   nestedMagic: {
//     will: {
//       work: "!"
//     }
//   }
// }

As in sets and lists, options can be povided as a 2nd argument.

Record

A new attribute type that translates to Partial<Record<KeyType, ValueType>> in TypeScript. Records differ from maps as they can accept an infinite range of keys and are always partial:

import { record } from 'dynamodb-toolbox';

const pokemonType = string().enum(...);

const pokemonSchema = schema({
  ...
  weaknessesByPokemonType: record(pokemonType, number()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   weaknessesByPokemonType: {
//     [key in PokemonType]?: number
//   }
// }

Options can be provided as a 3rd argument:

const recordAttr = record(string(), number()).hidden();
const recordAttr = record(string(), number(), { hidden: true });

AnyOf

A new meta-attribute type that represents a union of types, i.e. a range of possible types:

import { anyOf } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  pokemonType: anyOf([
    string().const('fire'),
    string().const('grass'),
    string().const('water'),
  ]),
});

In this particular case, an enum would have done the trick. However, anyOf becomes particularly powerful when used in conjunction with a map and the enum or const directives of a primitive attribute, to implement polymorphism:

const pokemonSchema = schema({
  ...
  captureState: anyOf([
    map({
      status: string().const('caught'),
      // 👇 captureState.trainerId exists if status is "caught"...
      trainerId: string(),
    }),
    // ...but not otherwise! 🙌
    map({ status: string().const('wild') }),
  ]),
});

type CaptureState = FormattedItem<typeof pokemonEntity>['captureState'];
// 🙌 Equivalent to:
// | { status: "wild" }
// | { status: "caught", trainerId: string }

As in sets, lists and maps, options can be povided as a 2nd argument.

Looking forward

That’s all for now! I’m planning on including new null, tuple and allOf attributes at some point.

If there are other types you’d like to see, feel free to leave a comment on this article and/or open a discussion on the official repo with the v1 label 👍

Computed defaults

In previous versions, default was used to compute attribute from other attributes values. This feature was very handy for "technical" attributes such as composite indexes.

However, it was just impossible to type correctly in TypeScript:

const pokemonSchema = schema({
  ...
  level: number(),
  levelPlusOne: number().default(
    // ❌ No way to retrieve the caller context
    input => input.level + 1,
  ),
});

It means the input was typed as any and it fell to the developper to type it correctly, which just didn’t cut it for me.

The solution I committed to was to split computed defaults declaration into 2 steps:

First, declare that an attribute default should be derived from other attributes:

import { ComputedDefault } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  level: number(),
  levelPlusOne: number().default(ComputedDefault),
});

💡 ComputedDefault is a JavaScript Symbol (TLDR: A sort of unique and custom null), so it cannot possibly conflict with an actual desired default value.

Then, declare a way to compute this attribute at the entity level, through the putDefaults property:

const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  putDefaults: {
    // 🙌 Correctly typed!
    levelPlusOne: ({ level }) => level + 1,
  },
});

The same can be applied to updateDefaults (or both):

import { $get } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  level: number(),
  previousLevel: number().updateDefault(ComputedDefault),
});

const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  updateDefaults: {
    // 🙌 Correctly typed!
    previousLevel: ({ level }) =>
      // Update 'previousLevel' only if 'level' is updated
      level !== undefined ? $get('level') : undefined,
  },
});

In the tricky case of nested attributes, putDefaults and updateDefaults become objects with an _attributes or _elements property to emphasize that the computing is local:

const pokemonSchema = schema({
  ...
  defaultLevel: number(),
  // 👇 Defaulted Map attribute
  levelHistory: map({
    currentLevel: number(),
    // 👇 Defaulted sub-attribute
    nextLevel: number().default(ComputedDefault),
  }).default(ComputedDefault),
});

const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  putDefaults: {
    levelHistory: {
      // Defaulted value of Map attribute
      _map: item => ({
        currentLevel: item.defaultLevel,
        nextLevel: item.defaultLevel,
      }),
      _attributes: {
        // Defaulted value of sub-attribute
        nextLevel: (levelHistory, item) => levelHistory.currentLevel + 1,
      },
    },
  },
});

Commands

Now that we know how to design entities, let’s take a look at how we can leverage them to craft commands 👍

💡 The beta only supports the PutItem, GetItem, UpdateItem and DeleteItem commands. If you need to run Query or Scan commands, my advice is to run native SDK commands and format their output with the formatSavedItem util.

As mentioned in the intro, I searched for a syntax that favored tree-shaking. Here's an example of it, with the PutItem command:

// v0.x Not tree-shakable
const response = await pokemonEntity.put(pokemonItem, options);

// v1 Tree-shakable 🙌
import { PutItemCommand } from 'dynamodb-toolbox';

const command = new PutItemCommand(
  pokemonEntity,
  // 🙌 Correctly typed!
  pokemonItem,
  // 👇 Optional
  putItemOptions,
);

// Get command params
const params = command.params();
// Send command
const response = await command.send();

pokemonItem can be provided later or edited:

import { PutItemCommand } from 'dynamodb-toolbox';

const incompleteCommand = new PutItemCommand(pokemonEntity);

// (will return a new command and not mutate the original one)
const completeCommand = incompleteCommand.item(pokemonItem);

// (can be chained by design)
const response = await incompleteCommand
  .item(pokemonItem)
  .options(options)
  .send();

You can also use the .build method of the entity to craft a command directly hydrated with your entity:

// 🙌 We get a syntax closer to v0.x... but tree-shakable!
const response = await pokemonEntity
  .build(PutItemCommand)
  .item(pokemonItem)
  .options(options)
  .send();

PutItemCommand

The capacity, metrics and returnValues options behave exactly the same as in previous versions. The condition option benefits from improved typing, and clearer logical combinations:

import { PutItemCommand } from 'dynamodb-toolbox';

const { Attributes } = await pokemonEntity
  .build(PutItemCommand)
  .item(pokemonItem)
  .options({
    capacity: 'TOTAL',
    metrics: 'SIZE',
    // 👇 Will type the response `Attributes`
    returnValues: 'ALL_OLD',
    condition: {
      or: [
        { attr: 'pokemonId', exists: false },
        // 🙌 "lte" is correcly typed
        { attr: 'level', lte: 99 },
        // 🙌 You can nest logical combinations
        { and: [{ not: { ... } }, ...] },
      ],
    },
  })
  .send();

GetItemCommand

The attributes option behaves the same as in previous versions, but benefits from improved typing as well:

import { GetItemCommand } from 'dynamodb-toolbox';

const { Item } = await pokemonEntity
  .build(GetItemCommand)
  .key(pokemonKey)
  .options({
    capacity: 'TOTAL',
    consistent: true,
    // 👇 Will type the response `Item`
    attributes: ['pokemonId', 'pokemonType', 'level'],
  })
  .send();

DeleteItemCommand

The DeleteItem command is pretty much a mix between the PutItem and GetItem commands ones, options wise:

import { DeleteItemCommand } from 'dynamodb-toolbox';

const { Attributes } = await pokemonEntity
  .build(DeleteItemCommand)
  .key(pokemonKey)
  .options({
    capacity: 'TOTAL',
    metrics: 'SIZE',
    // 👇 Will type the response `Attributes`
    returnValues: 'ALL_OLD',
    condition: {
      or: [
        { attr: 'level', lte: 99 },
        ...
      ],
    },
  })
  .send();

UpdateItemCommand

The UpdateCommand is the richest of all DynamoDB commands. Its options are similar to the PutItemCommand options:

import { UpdateItemCommand } from 'dynamodb-toolbox';

const { Item } = await pokemonEntity
  .build(UpdateItemCommand)
  .item(pokemonItem)
  .options({
    capacity: 'TOTAL',
    metrics: 'SIZE',
    // 👇 Will type the response `Attributes`
    returnValues: 'ALL_NEW',
    condition: {
      or: [
        { attr: 'level', lte: 99 },
        ...
      ],
    },
  })
  .send();

However, its item method offers many more possibilities 🙌 Let's explore them:

Removing an attribute

Any optional attribute can be removed with the $remove util:

import { $remove } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  isLegendary: boolean().optional(),
});

pokemonEntity.build(UpdateItemCommand).item({
  ...
  isLegendary: $remove(),
});

Referencing saved values

You can reference a saved attribute value by using the $get util:

import { $get } from 'dynamodb-toolbox';

pokemonEntity.build(UpdateItemCommand).item({
  ...
  // 👇 Resolved by DynamoDB at write time
  previousLevel: $get('level'),
});

Self-references are possible. You can also provide a fallback value as 2nd argument in case the specified attribute path misses from the saved item:

pokemonEntity.build(UpdateItemCommand).item({
  ...
  previousLevel: $get('level', 1),
  // 👇 fallback can also be a reference!
  chainedRefs: $get(
    'firstRef',
    $get('secondRef', 'Sky is the limit!'),
  ),
});

Note that the attribute path is type-checked, but wether its attribute value extends the updated attribute value is not for the moment, so be extra-careful:

const pokemonSchema = schema({
  ...
  name: string(),
  level: number(),
});

pokemonEntity.build(UpdateItemCommand).item({
  // ❌ Will be caught
  name: $get('non.existing[0].attribute'),
  // 🙈 Will NOT be caught
  level: $get('name'),
});

Non-recursive attributes

In the case of non-recursive attributes, e.g. primitives and sets, updates will completely override their previous values:

const pokemonSchema = schema({
  ...
  isLegendary: boolean(),
  level: number(),
  name: string(),
  binEncoded: binary(),
  skills: set(string()),
});

pokemonEntity.build(UpdateItemCommand)
  .item({
    ...
    // 👇 Set fields to desired values
    isLegendary: true,
    nextLevel: 42,
    name: 'Pikachu',
    binEncoded: Buffer.from(...),
    skills: new Set(['thunder'])
  })

number attributes benefit from additional $sum, $subtract and $add operations, which can use references:

import { $add, $subtract, $get } from 'dynamodb-toolbox';

await pokemonEntity.build(UpdateItemCommand)
  .item({
    ...
    health: $subtract($get('health'), 20),
    level: $sum($get('level', 0), 1),
    // 👇 Similar to
    level: $add(1),
  })
  .send();

To add or remove specific values from a set, you can use the $add and $delete utils:

pokemonEntity.build(UpdateItemCommand)
  .item({
    ...
    skills: $add('thunder', 'dragon-tail'),
    types: $delete('flight'),
  })

Recursive attributes

In the case of recursive attributes, e.g. lists, maps and records, updates are partial by default. You can use the $set to specify a complete override:

const pokemonSchema = schema({
  ...
  types: list(string()),
  skills: list(string()),
  some: map({
    nested: map({
      field: string(),
      otherField: number(),
    }),
  }),
  bestSkillByType: record(string(), string()),
});

// 👇 Partial overrides
pokemonEntity.build(UpdateItemCommand).item({
  ...
  // 👇 Indexes 0 and 2 will be updated
  skills: ['thunder', undefined, $remove()],
  // 👇 Similar to
  skills: {
    0: 'thunder',
    2: $remove(),
  },
  some: {
    nested: {
      field: 'foo',
    },
  },
  bestSkillByType: {
    electric: 'thunder',
    flight: $remove(),
  },
});

import { $set } from 'dynamodb-toolbox';

// 👇 Complete overrides
pokemonEntity.build(UpdateItemCommand).item({
  ...
  skills: $set(['thunder']),
  some: $set({
    nested: {
      field: 'foo',
      otherField: 42,
    },
  }),
  bestSkillByType: $set({
    electric: 'thunder',
  }),
});

lists attributes benefit from additional $append and $prepend operations, which can use references:

pokemonEntity.build(UpdateItemCommand).item({
  ...
  skills: $append(['thunder', 'dragon-tail']),
  levelHistory: $append($get('level')),
  types: $prepend(['flight']),
});

Any and anyOf attributes

The any attribute supports all the syntaxes specified above. anyOf attributes updates are not supported yet.

Mocking entities in tests

As much as I appreciate this chained syntax, it makes mocking hard in unit tests. For this reason, the v1 exposes a mockEntity util to help you mock commands:

import { mockEntity } from 'dynamodb-toolbox';

const mockedPokemonEntity = mockEntity(pokemonEntity);

mockedPokemonEntity.on(GetItemCommand).resolve({
  // 🙌 Type-safe!
  Item: {
    pokemonId: 'pikachu1',
    name: 'Pikachu',
    level: 42,
    ...
  },
});

// 👇 For more fine-grained control
mockedPokemonEntity
  .on(GetItemCommand)
  .mockImplementation((key, options) => ({
    // 🙌 Still type-safe!
    Item: {
      pokemonId: 'pikachu1',
      ...
    },
  }));

//👇 To simulate errors
mockedPokemonEntity.on(GetItemCommand).reject('Something bad happened');

You can then make assertions on received commands:

await pokemonEntity
  .build(GetItemCommand)
  .key({ pokemonId: 'pikachu1' })
  .options({ consistent: true })
  .send();
// => Will return mocked values!

mockedPokemonEntity.received(GetItemCommand).count();
// => 1
mockedPokemonEntity.received(GetItemCommand).args(0);
// => [{ pokemonId: 'pikachu1' }, { consistent: true }]
mockedEntity.received(GetItemCommand).allArgs();
// => [[{ pokemonId: 'pikachu1' }, { consistent: true }], ...anyOtherCall]

Utility helpers and types

In addition to the SavedItem and FormattedItem types, the v1 exposes a bunch of useful helpers and utility types:

formatSavedItem

formatSavedItem transforms a saved item returned by the DynamoDB client to it’s formatted counterpart:

import { formatSavedItem } from 'dynamodb-toolbox';

// 🙌 Typed as FormattedItem<typeof pokemonEntity>
const formattedPokemon = formatSavedItem(
  pokemonEntity,
  savedPokemon,
  // As in GetItem commands, attributes will filter the formatted item
  { attributes: [...] },
);

Note that it is a parsing operation, i.e. it does not require the item to be typed as SavedItem<typeof myEntity>, but will throw an error if the saved item is invalid:

const formattedPokemon = formatSavedItem(pokemonEntity, {
  ...
  level: 'not a number',
});
// ❌ Will raise error:
// => "Invalid attribute in saved item: level. Should be a number"

Condition and parseCondition

The Condition type and parseCondition util are useful to type conditions and build condition expressions:

import { Condition, parseCondition } from 'dynamodb-toolbox';

const condition: Condition<typeof pokemonEntity> = {
  attr: 'level',
  lte: 42,
};

const parsedCondition = parseCondition(pokemonEntity, condition);
// => {
//   ConditionExpression: "#c1 <= :c1",
//   ExpressionAttributeNames: { "#c1": "level" },
//   ExpressionAttributeValues: { ":c1": 42 },
// }

Projection and parseProjection

The AnyAttributePath type and parseProjection util are useful to type attribute paths and build projection expressions:

import { AnyAttributePath, parseProjection } from 'dynamodb-toolbox';

const attributes: AnyAttributePath<typeof pokemonEntity>[] = [
  'pokemonType',
  'levelHistory.currentLevel',
];

const parsedProjection = parseProjection(pokemonEntity, attributes);
// => {
//   ProjectionExpression: '#p1, #p2.#p3',
//   ExpressionAttributeNames: {
//     '#p1': 'pokemonType',
//     '#p2': 'levelHistory',
//     '#p3': 'currentLevel',
//   },
// }

KeyInput and PrimaryKey

Both types are useful to type item primary keys:

import type { KeyInput, PrimaryKey } from 'dynamodb-toolbox';

type PokemonKeyInput = KeyInput<typeof pokemonEntity>;
// => { pokemonClass: string, pokemonId: string }

type MyTablePrimaryKey = PrimaryKey<typeof myTable>;
// => { PK: string, SK: string }

Errors

Finally, let’s take a quick look at error management. When DynamoDB-Toolbox encounters an unexpected input, it will throw an instance of DynamoDBToolboxError, which itself extends the native Error class with a code property:

await pokemonEntity
  .build(PutItemCommand)
  .item({ ..., level: 'not a number' })
  .send();
// ❌ [parsing.invalidAttributeInput] Attribute level should be a number

Some DynamoDBToolboxErrors also expose a path property (mostly in validations) and/or a payload property for additional context. If you need to handle them, TypeScript is your best friend, as the code property will correctly discriminate the DynamoDBToolboxError type:

import { DynamoDBToolboxError } from 'dynamodb-toolbox';

const handleError = (error: Error) => {
  if (!error instanceof DynamoDBToolboxError) throw error;

  switch (error.code) {
    case 'parsing.invalidAttributeInput':
      const path = error.path;
      // => "level"
      const payload = error.payload;
      // => { received: "not a number", expected: "number" }
      break;
      ...
    case 'entity.invalidItemSchema':
      const path = error.path; // ❌ error does not have path property
      const payload = error.payload; // ❌ same goes with payload
      ...
  }
};

Conclusion

And that’s it for now! I hope you’re as excited as I am about this new release 🙌

If you have features that I've missed in mind, or would like to see some of the ones I mentioned prioritized, please leave a comment on this article and/or create an issue or open a discussion on the official repo with the v1 label 👍

See you soon!

New DynamoDB-Toolbox v1 beta: Features and breaking changes

Thomas Aribart — Mon, 09 Oct 2023 15:22:05 +0000

☝️ NOTE: This article details how to migrate from the beta.0 to the beta.1 of the new DynamoDB-Toolbox major.

👉 If you need the full documentation of the beta.1 release, you may be looking for this article.

👉 If you need the full documentation of the beta.0 release, you may be looking for its previous version.

A new v1 beta of DynamoDB-Toolbox is out 🙌

Simply named beta.1, the aim of this article is to succinctly describe its new features and breaking changes from the beta.0.

New features

UpdateItemCommand

The main update from this release is that the UpdateItemCommand is now available 🥳

It's clearly one of of the hardest feature I had to develop in my life (thank you, DynamoDB reference for being very unclear on soooooo many edge cases 😳). I'm glad it is now shipped!

See the updated documentation article for an exhaustive documentation, but here's a sneak peek:

import { UpdateItemCommand, $add, $get } from 'dynamodb-toolbox';

const { Item } = await pokemonEntity
  .build(UpdateItemCommand)
  .item({
    pokemonId,
    ...
    name: newName,
    // Save current 'level' value to 'previousLevel'
    previousLevel: $get('level'),
    // Add 1 to the current 'level'
    level: $add(1),
  })
  .options({
    // Only if level is < 99
    condition: { attr: 'level', lt: 99 },
    // Will type the response `Attributes`
    returnValues: 'ALL_NEW',
    ...
  })
  .send();

MockEntity

Also, a new mockEntity util is now available to help you write unit tests and make assertions:

import { mockEntity } from 'dynamodb-toolbox';

const mockedPokemonEntity = mockEntity(pokemonEntity);

mockedPokemonEntity.on(GetItemCommand).resolve({
  // 🙌 Type-safe!
  Item: {
    pokemonId: 'pikachu1',
    name: 'Pikachu',
    level: 42,
    ...
  },
});

await pokemonEntity
  .build(GetItemCommand)
  .key({ pokemonId: 'pikachu1' })
  .options({ consistent: true })
  .send();
// => Will return mocked values!

mockedPokemonEntity.received(GetItemCommand).args(0);
// => [{ pokemonId: 'pikachu1' }, { consistent: true }]

Providing getters for table names

Last small improvement: Table names can now be provided with getters. This can be useful in some contexts where you may want to use the Table class without actually running any command (e.g. tests or deployments):

const myTable = new TableV2({
  ...
  // 👇 Only executed at command execution
  name: () => process.env.TABLE_NAME,
});

Breaking changes

`default` options

With the appearance of the UpdateItemCommand, the default option had to be reworked.

I disliked it for being ambiguous and impractical in some cases (like the internal created timestamp attribute). Well, it is now split into three options:

defaults.key option (keyDefault method): Fills undefined key attributes during key computing. Used in all commands for now, we'll see about Queries and Scans.
defaults.put option (putDefault method): Used for non-key attributes in PutItemCommands
defaults.update option (updateDefault method): Used for non-key attributes in UpdateItemCommands

// 👇 Regular attribute
const lastUpdateAttribute = string({
  defaults: {
    key: undefined,
    put: () => new Date().toISOString(),
    update: () => new Date().toISOString(),
  },
});
// ...or
const lastUpdateAttribute = string()
  .putDefault(() => new Date().toISOString())
  .updateDefault(() => new Date().toISOString());

// 👇 Key attribute
const keyAttribute = string({
  key: true,
  defaults: {
    key: 'my-awesome-partition-key',
    // put & update defaults are not useful in `key` attributes
    put: undefined,
    update: undefined,
  },
});
// ...or
const keyAttribute = string().key().keyDefault('my-awesome-partition-key');

Note that the default method is still there. It acts similarly as putDefault, except if the attribute has been tagged as a key attribute, in which case it acts as keyDefault:

const metadata = any().default({ any: 'value' });
// 👇 Similar to
const metadata = any().putDefault({ any: 'value' });
// 👇 ...or
const metadata = any({
  defaults: {
    key: undefined,
    put: { any: 'value' },
    update: undefined,
  },
});

const keyPart = any().key().default('my-awesome-partition-key');
// 👇 Similar to
const metadata = any().key().keyDefault('my-awesome-partition-key');
// 👇 ...or
const metadata = any({
  key: true,
  defaults: {
    key: 'my-awesome-partition-key',
    // put & update defaults are not useful in `key` attributes
    put: undefined,
    update: undefined,
  },
});

Also, the const shorthand still acts enum(MY_CONST).default(MY_CONST) and does NOT provide update defaults. Let me know if it's something you'd need.

`computeDefaults`

The same split had to be applied to computed defaults: In the same spirit, the computeDefaults property has been renamed putDefaults, and a new updateDefaults property has appeared (with computeKey still being available to compute the primary key):

import { schema, EntityV2, ComputedDefault } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  level: number(),
  levelPlusOne: number().default(ComputedDefault),
  previousLevel: number().updateDefault(ComputedDefault),
});

const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  putDefaults: {
    // 🙌 Correctly typed!
    levelPlusOne: ({ level }) => level + 1,
  },
  updateDefaults: {
    // 🙌 Correctly typed!
    previousLevel: ({ level }) =>
      // Update 'previousLevel' only if 'level' is updated
      level !== undefined ? $get('level') : undefined,
  },
});

Conclusion

That's it 🙌 I hope you like this new version... and YES, Queries and Scans are next!

See you in a few months for the official release! (before the end of the year I hope 🤞)

The DynamoDB-Toolbox v1 beta is here 🙌 All you need to know!

Thomas Aribart — Fri, 09 Jun 2023 17:29:54 +0000

At Theodo, we are big fans of Jeremy Daly’s DynamoDB-Toolbox. We started using it as early as 2019 and grew fond of it... but were also well aware of its flaws 😅

That just wasn't the case for DynamoDB-Toolbox: I remember working on an .update method that was more than 1000 lines long... But why bundle it when you don't even need it?

So last year, I decided to throw myself into a complete overhaul of the code, with three main objectives:

Support the v3 of the AWS SDK (although support has been added in the v0.8)
Get the API and type inference on par with those of more "modern" tools like zod and electrodb
Use a more functional and tree-shakable approach

Today, I am happy to announce the v1 beta of dynamodb-toolbox is out 🙌 It includes reworked Table and Entity classes, as well as complete support for PutItem, GetItem and DeleteItem commands (including conditions and projections), with UpdateItem, Query and Scan commands soon to follow.

This article details how the new API works and the main breaking changes from previous versions - which, by the way, only concern the API: No data migration needed 🥳

Let's dive in!

Table of content

Installation
Tables
Entities
- Timestamps
- Matching the Table schema
- SavedItem and FormattedItem
Designing Entity schemas
- Schema definition
- Attributes types
- any
- <primitive>
- set
- list
- map
- record
- anyOf
- Looking forward
Computed defaults
Commands
- PutItemCommand
- GetItemCommand
- DeleteItemCommand
Utility helpers and types
- formatSavedItem
- Condition and parseCondition
- Projection and parseProjection
- KeyInput and PrimaryKey
Errors
Conclusion

Installation



### npm
npm i dynamodb-toolbox@1.0.0-beta.0

## yarn
yarn add dynamodb-toolbox@1.0.0-beta.0

## ...and so on

☝️ Stay up to date with the patches by following the project GitHub releases

The v1 is built on top the v3 of the AWS SDK. It has @aws-sdk/client-dynamodb and @aws-sdk/lib-dynamodb as peer dependencies so you’ll have to install them as well:



## npm
npm i @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

## yarn
yarn add @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

## ...and so on

Tables

Tables are defined pretty much the same way as in previous versions, but the key attributes now have a type along with their name:



import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
// Will be renamed Table in the official release 😉
import { TableV2 } from 'dynamodb-toolbox';

const dynamoDBClient = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(dynamoDBClient);

const myTable = new TableV2({
  name: 'MySuperTable',
  partitionKey: {
    name: 'PK',
    type: 'string', // 'string' | 'number' | 'binary'
  },
  sortKey: {
    name: 'SK',
    type: 'string',
  },
  documentClient,
});

☝️ The v1 does not support indexes yet as queries are not yet available.



const myTable = new TableV2({
  ...
  // 👇 defaults to "_et"
  entityAttributeSavedAs: '__entity__',
});

Entities

For Entities, the main change is that the attributes argument becomes schema:



// Will be renamed Entity in the official release 😉
import { EntityV2, schema } from 'dynamodb-toolbox';

const myEntity = new EntityV2({
  name: 'MyEntity',
  table: myTable,
  // Attributes definition
  schema: schema({ ... }),
});

Timestamps



const myEntity = new EntityV2({
  ...
  // 👇 de-activate timestamps altogether
  timestamps: false,
});

const myEntity = new EntityV2({
  ...
  timestamps: {
    // 👇 de-activate only `created` attribute
    created: false,
    modified: true,
  },
});

const myEntity = new EntityV2({
  ...
  timestamps: {
    created: {
      // 👇 defaults to "created"
      name: 'creationDate',
      // 👇 defaults to "_ct"
      savedAs: '__createdAt__',
    },
    modified: {
      // 👇 defaults to "modified"
      name: 'lastModificationDate',
      // 👇 defaults to "_md"
      savedAs: '__lastMod__',
    },
  },
});

Matching the Table schema

An important change from previous versions is that the EntityV2 key attributes are validated against the TableV2 schema, both through types and at runtime. There are two ways to match the table schema:

The simplest one is to have an entity schema that already matches the table schema (see "Designing Entity schemas"). The Entity is then considered valid and no other argument is required:



import { string } from 'dynamodb-toolbox';

const pokemonEntity = new EntityV2({
  name: 'Pokemon',
  table: myTable, // <= { PK: string, SK: string } primary key
  schema: schema({
    // Provide a schema that matches the primary key
    PK: string().key(),
    // 🙌 using "savedAs" will also work
    pokemonId: string().key().savedAs('SK'),
    ...
  }),
});

If the entity key attributes don't match the table schema, the Entity class will require you to add a computeKey property which must derive the primary key from them:



const pokemonEntity = new EntityV2({
  ...
  table: myTable, // <= { PK: string, SK: string } primary key
  schema: schema({
    pokemonClass: string().key(),
    pokemonId: string().key(),
    ...
  }),
  // 🙌 `computeKey` is correctly typed
  computeKey: ({ pokemonClass, pokemonId }) => ({
    PK: pokemonClass,
    SK: pokemonId,
  }),
});

SavedItem and FormattedItem

If you feel lost, you can always use the SavedItem and FormattedItem utility types to infer the type of your entity items:



import type { FormattedItem, SavedItem } from 'dynamodb-toolbox';

const pokemonEntity = new EntityV2({
  name: 'Pokemon',
  timestamps: true,
  table: myTable,
  schema: schema({
    pokemonClass: string().key().savedAs('PK'),
    pokemonId: string().key().savedAs('SK'),
    level: number().default(1),
    customName: string().optional(),
    internalField: string().hidden(),
  }),
});

// What Pokemons will look like in DynamoDB
type SavedPokemon = SavedItem<typeof pokemonEntity>;
// 🙌 Equivalent to:
// {
//   _et: "Pokemon",
//   _ct: string,
//   _md: string,
//   PK: string,
//   SK: string,
//   level: number,
//   customName?: string | undefined,
//   internalField: string | undefined,
// }

// What fetched Pokemons will look like in your code
type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// 🙌 Equivalent to:
// {
//   created: string,
//   modified: string,
//   pokemonClass: string,
//   pokemonId: string,
//   level: number,
//   customName?: string | undefined,
// }

Designing Entity schemas

Now let’s dive into the part that received the most significant overhaul: Schema definition.

Schema definition

You can either import the attribute builders through their dedicated imports, or through the attribute or attr shorthands. For instance, those declarations will output the same attribute schema:



import { string, attribute, attr } from 'dynamodb-toolbox';

// 👇 More tree-shakable
const pokemonName = string();
// 👇 Not tree-shakable, but single import
const pokemonName = attribute.string();
const pokemonName = attr.string();

The main takeaway is that warm schemas can be composed while frozen schemas cannot:



import { schema } from 'dynamodb-toolbox';

const pokemonName = string();

const pokemonSchema = schema({
  // 👍 No problem
  pokemonName,
  ...
});

const pokedexSchema = schema({
  // ❌ Not possible
  pokemon: pokemonSchema,
  ...
});



// Using methods
const pokemonName = string().required('always');
// Using options
const pokemonName = string({ required: 'always' });

All attributes share the following options:

required (string?="atLeastOnce") Tag a root attribute or Map sub-attribute as required. Possible values are:
- "atLeastOnce" Required in PutItem commands
- "never": Optional in all commands
- "always": Required in PutItem, GetItem and DeleteItem commands



// Equivalent
const pokemonName = string().required();
const pokemonName = string({ required: 'atLeastOnce' });

// `.optional()` is a shorthand for `.required(”never”)`
const pokemonName = string().optional();
const pokemonName = string({ required: 'never' });

💡 Outside of root attributes and Map sub-attributes, such as in a list of strings, it doesn’t make sense for sub-schemas to be optional. So, should I force users to write list(string().required()) every time OR make string validation and type inference aware of their context (ignore required in lists but not in maps)? It felt more elegant to enforce string() as required by default and prevent schemas such as list(string().optional()).

hidden (boolean?=true) Skip attribute when formatting the returned item of a command:



const pokemonName = string().hidden();
const pokemonName = string({ hidden: true });

key (boolean?=true) Tag attribute as needed to compute the primary key:



// Note: The method will also modify the `required` property to "always"
// (it is often the case in practice, you can still use `.optional()` if needed)
const pokemonName = string().key();
const pokemonName = string({ key: true });

savedAs (string) Previously known as map. Rename a root or Map sub-attribute before sending commands:



const pokemonName = string().savedAs('_n');
const pokemonName = string({ savedAs: '_n' });

default: (ComputedDefault) See Computed defaults

Attributes types

Here’s the exhaustive list of available attribute types:

Any

Define an attribute of any value. No validation will be applied at runtime, and its type will be resolved as unknown:



import { any } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  metadata: any(),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   metadata: unknown
// }

You can provide default values through the default option or method:



const metadata = any().default({ any: 'value' });
const metadata = any({
  default: () => 'Getters also work!',
});

Primitives

Defines a string, number, boolean or binary attribute:



import { string, number, boolean, binary } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  pokemonType: string(),
  level: number(),
  isLegendary: boolean(),
  binEncoded: binary(),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   pokemonType: string
//   level: number
//   isLegendary: boolean
//   binEncoded: Buffer
// }

You can provide default values through the default option or method:



// 🙌 Correctly typed!
const level = number().default(42);
const date = string().default(() => new Date().toISOString());

const level = number({ default: 42 });
const date = string({
  default: () => new Date().toISOString(),
});

Primitive types have an additional enum option. For instance, you could provide a finite list of pokemon types:



const pokemonTypeAttribute = string().enum('fire', 'grass', 'water');

// Shorthand for `.enum("POKEMON").default("POKEMON")`
const pokemonPartitionKey = string().const('POKEMON');

💡 For type inference reasons, the enum option is only available as a method, not as an object option

Set

Defines a set of strings, numbers or binaries. Unlike in previous versions, sets are kept as Set classes. Let me know if you would prefer using arrays (or being able to choose from both):



import { set } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  skills: set(string()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   skills: Set<string>
// }

Options can be provided as a 2nd argument:



const setAttr = set(string()).hidden();
const setAttr = set(string(), { hidden: true });

List

Defines a list of sub-schemas of any type:



import { list } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  skills: list(string()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   skills: string[]
// }

As in sets, options can be povided as a 2nd argument.

Map

Defines a finite list of key-value pairs. Keys must follow a string schema, while values can be sub-schema of any type:



import { map } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  nestedMagic: map({
    will: map({
      work: string().const('!'),
    }),
  }),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   nestedMagic: {
//     will: {
//       work: "!"
//     }
//   }
// }

As in sets and lists, options can be povided as a 2nd argument.

Record

A new attribute type that translates to Partial<Record<KeyType, ValueType>> in TypeScript. Records differ from maps as they can accept an infinite range of keys:



import { record } from 'dynamodb-toolbox';

const pokemonType = string().enum(...);

const pokemonSchema = schema({
  ...
  weaknessesByPokemonType: record(pokemonType, number()),
});

type FormattedPokemon = FormattedItem<typeof pokemonEntity>;
// => {
//   ...
//   weaknessesByPokemonType: {
//     [key in PokemonType]?: number
//   }
// }

Options can be provided as a 3rd argument:



const recordAttr = record(string(), number()).hidden();
const recordAttr = record(string(), number(), { hidden: true });

AnyOf

A new meta-attribute type that represents a union of types, i.e. a range of possible types:



import { anyOf } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  pokemonType: anyOf([
    string().const('fire'),
    string().const('grass'),
    string().const('water'),
  ]),
});



const pokemonSchema = schema({
  ...
  captureState: anyOf([
    map({
      status: string().const('caught'),
      // 👇 captureState.trainerId exists if status is "caught"...
      trainerId: string(),
    }),
    // ...but not otherwise! 🙌
    map({ status: string().const('wild') }),
  ]),
});

type CaptureState = FormattedItem<typeof pokemonEntity>['captureState'];
// 🙌 Equivalent to:
// | { status: "wild" }
// | { status: "caught", trainerId: string }

As in sets, lists and maps, options can be povided as a 2nd argument.

Looking forward

That’s all for now! I’m planning on including new tuple and allOf attributes at some point.

If there are other types you’d like to see, feel free to leave a comment on this article and/or open a discussion on the official repo with the v1 label 👍

Computed defaults

In previous versions, default was used to compute attribute from other attributes values. This feature was very handy for "technical" attributes such as composite indexes.

However, it was just impossible to type correctly in TypeScript:



const pokemonSchema = schema({
  ...
  level: number(),
  levelPlusOne: number().default(
    // ❌ No way to retrieve the caller context
    input => input.level + 1,
  ),
});

It means the input was typed as any and it fell to the developper to type it correctly, which just didn’t cut it for me.

The solution I committed to was to split computed defaults declaration into 2 steps:

First, declare that an attribute default should be derived from other attributes:



import { ComputedDefault } from 'dynamodb-toolbox';

const pokemonSchema = schema({
  ...
  level: number(),
  levelPlusOne: number().default(ComputedDefault),
});

💡 ComputedDefault is a JavaScript Symbol (TLDR: A sort of unique and custom null), so it cannot possibly conflict with an actual desired default value.

Then, declare a way to compute this attribute at the entity level, through the computeDefaults property:



const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  computeDefaults: {
    // 🙌 Correctly typed!
    levelPlusOne: ({ level }) => level + 1,
  },
});

In the tricky case of nested attributes, computeDefaults becomes an object with an _attributes or _elements property to emphasize that the computing is local:



const pokemonSchema = schema({
  ...
  defaultLevel: number(),
  // 👇 Defaulted Map attribute
  levelHistory: map({
    currentLevel: number(),
    // 👇 Defaulted sub-attribute
    nextLevel: number().default(ComputedDefault),
  }).default(ComputedDefault),
});

const pokemonEntity = new EntityV2({
  ...
  schema: pokemonSchema,
  computeDefaults: {
    levelHistory: {
      // Defaulted value of Map attribute
      _map: item => ({
        currentLevel: item.defaultLevel,
        nextLevel: item.defaultLevel,
      }),
      _attributes: {
        // Defaulted value of sub-attribute
        nextLevel: (levelHistory, item) => levelHistory.currentLevel + 1,
      },
    },
  },
});

Note that there is (and has always been) an ambiguity as to when default values are actually used, that I hope to solve soon by splitting it into getDefault, putDefault, updateDefault and so on (default being the one to rule them all). For the moment, defaults are only used in putItem commands.

Commands

Now that we know how to design entities, let’s take a look at how we can leverage them to craft commands 👍

💡 The beta only supports the PutItem, GetItem, and DeleteItem commands. If you need to run UpdateItem, Query or Scan commands, our advice is to run native SDK commands and format their output with the formatSavedItem util.

As mentioned in the intro, I searched for a syntax that favored tree-shaking. Here's an example of it, with the PutItem command:



// v0.x Not tree-shakable
const response = await pokemonEntity.putItem(pokemonItem, options);

// v1 Tree-shakable 🙌
import { PutItemCommand } from 'dynamodb-toolbox';

const command = new PutItemCommand(
  pokemonEntity,
  // 🙌 Correctly typed!
  pokemonItem,
  // 👇 Optional
  putItemOptions,
);

// Get command params
const params = command.params();
// Send command
const response = await command.send();

pokemonItem can be provided later or edited, which can be useful if the command is built in several steps (at execution, an error will be thrown if no item has been provided):



import { PutItemCommand } from 'dynamodb-toolbox';

const incompleteCommand = new PutItemCommand(pokemonEntity);

// (will return a new command and not mutate the original one)
const completeCommand = incompleteCommand.item(pokemonItem);

// (can be chained by design)
const response = await incompleteCommand
  .item(pokemonItem)
  .options(options)
  .send();

You can also use the .build method of the entity to craft a command directly hydrated with your entity:



// 🙌 We get a syntax closer to v0.x... but tree-shakable!
const response = await pokemonEntity
  .build(PutItemCommand)
  .item(pokemonItem)
  .options(options)
  .send();

💡 As much as I appreciate this syntax, it makes mocking hard in unit tests. I'm already working on a mockEntity helper, inspired by the awesome aws-sdk-client-mock. This will probably make another article soon.

PutItemCommand

The capacity, metrics and returnValues options behave exactly the same as in previous versions. The condition option benefits from improved typing, and clearer logical combinations:



import { PutItemCommand } from 'dynamodb-toolbox';

const { Attributes } = await pokemonEntity
  .build(PutItemCommand)
  .item(pokemonItem)
  .options({
    capacity: 'TOTAL',
    metrics: 'SIZE',
    // 👇 Will type the response `Attributes`
    returnValues: 'ALL_OLD',
    condition: {
      or: [
        { attr: 'pokemonId', exists: false },
        // 🙌 "lte" is correcly typed
        { attr: 'level', lte: 99 },
        // 🙌 You can nest logical combinations
        { and: [{ not: { ... } }, ...] },
      ],
    },
  })
  .send();

❗️The "UPDATED_OLD" and "UPDATED_NEW" return values options are not fully supported yet so I do not recommend using them for now

GetItemCommand

The attributes option behaves the same as in previous versions, but benefits from improved typing as well:



import { GetItemCommand } from 'dynamodb-toolbox';

const { Item } = await pokemonEntity
  .build(GetItemCommand)
  .key(pokemonKey)
  .options({
    capacity: 'TOTAL',
    consistent: true,
    // 👇 Will type the response `Item`
    attributes: ['pokemonId', 'pokemonType', 'level'],
  })
  .send();

DeleteItemCommand

The DeleteItem command is pretty much a mix between the two previous ones, options wise:



import { DeleteItemCommand } from 'dynamodb-toolbox';

const { Attributes } = await pokemonEntity
  .build(DeleteItemCommand)
  .key(pokemonKey)
  .options({
    capacity: 'TOTAL',
    metrics: 'SIZE',
    // 👇 Will type the response `Attributes`
    returnValues: 'ALL_OLD',
    condition: {
      or: [
        { attr: 'level', lte: 99 },
        ...
      ],
    },
  })
  .send();

Utility helpers and types

In addition to the SavedItem and FormattedItem types, the v1 exposes a bunch of useful helpers and utility types:

formatSavedItem

formatSavedItem transforms a saved item returned by the DynamoDB client to it’s formatted counterpart:



import { formatSavedItem } from 'dynamodb-toolbox';

// 🙌 Typed as FormattedItem<typeof pokemonEntity>
const formattedPokemon = formatSavedItem(
  pokemonEntity,
  savedPokemon,
  // As in GetItem commands, attributes will filter the formatted item
  { attributes: [...] },
);

Note that it is a parsing operation, i.e. it does not require the item to be typed as SavedItem<typeof myEntity>, but will throw an error if the saved item is invalid:



const formattedPokemon = formatSavedItem(pokemonEntity, {
  ...
  level: 'not a number',
});
// ❌ Will raise error:
// => "Invalid attribute in saved item: level. Should be a number"

Condition and parseCondition

The Condition type and parseCondition util are useful to type conditions and build condition expressions:



import { Condition, parseCondition } from 'dynamodb-toolbox';

const condition: Condition<typeof pokemonEntity> = {
  attr: 'level',
  lte: 42,
};

const parsedCondition = parseCondition(pokemonEntity, condition);
// => {
//   ConditionExpression: "#1 <= :1",
//   ExpressionAttributeNames: { "#1": "level" },
//   ExpressionAttributeValues: { ":1": 42 },
// }

Projection and parseProjection

The AnyAttributePath type and parseProjection util are useful to type attribute paths and build projection expressions:



import { AnyAttributePath, parseProjection } from 'dynamodb-toolbox';

const attributes: AnyAttributePath<typeof pokemonEntity>[] = [
  'pokemonType',
  'levelHistory.currentLevel',
];

const parsedProjection = parseProjection(pokemonEntity, attributes);
// => {
//   ProjectionExpression: '#1, #2.#3',
//   ExpressionAttributeNames: {
//     '#1': 'pokemonType',
//     '#2': 'levelHistory',
//     '#3': 'currentLevel',
//   },
// }

KeyInput and PrimaryKey

Both types are useful to type item primary keys:



import type { KeyInput, PrimaryKey } from 'dynamodb-toolbox';

type PokemonKeyInput = KeyInput<typeof pokemonEntity>;
// => { pokemonClass: string, pokemonId: string }

type MyTablePrimaryKey = PrimaryKey<typeof myTable>;
// => { PK: string, SK: string }

Errors



await pokemonEntity
  .build(PutItemCommand)
  .item({ ..., level: 'not a number' })
  .send();
// ❌ [parsing.invalidAttributeInput] Attribute level should be a number



import { DynamoDBToolboxError } from 'dynamodb-toolbox';

const handleError = (error: Error) => {

  if (!error instanceof DynamoDBToolboxError) throw error;

switch (error.code) {

    case 'parsing.invalidAttributeInput':

      const path = error.path;

      // => "level"

      const payload = error.payload;

      // => { received: "not a number", expected: "number" }

      break;

      ...

    case 'entity.invalidItemSchema':

      const path = error.path; // ❌ error does not have path property

      const payload = error.payload; // ❌ same goes with payload

      ...

  }

};

Conclusion

And that’s it for now! I hope you’re as excited as I am about this new release 🙌

See you soon!

An in-depth comparison of the most popular DynamoDB wrappers

Thomas Aribart — Wed, 19 Apr 2023 08:24:44 +0000

⚠️ Update ⚠️: Since the v1 of DynamoDB-Toolbox has been released, I strongly recommend using it over ElectroDB:

Slick zod-like schema declaration

Support for polymorphism

Better tree-shaking

Type-safe dependent defaults and more!

Check it out 👉 https://www.dynamodbtoolbox.com/

AWS DynamoDB is a key-value database designed to run high-performance applications at any scale. It automatically scales up and down based on your current traffic, and does not require maintaining connections (as requests are sent over HTTP), which makes it the go-to DB service for serverless developers on AWS.

Because it’s 2023 and no-one writes HTTP requests anymore, AWS published a SDK called the document client to craft said requests. However, if you’ve ever used it, you will know that it’s still very painful to use.

For instance, let’s look at this UpdateCommand example straight from the DynamoDB docs itself:

await DocumentClient.send(
  new UpdateCommand({
    TableName: 'TABLE_NAME',
    Key: {
      // 👇 No type-safety on the primary key
      title: 'MOVIE_NAME',
      year: 'MOVIE_YEAR',
    },
    // 👇 String expressions hard to build (and still no type-safety)
    UpdateExpression: 'set info.plot = :p, info.#r = :r',
    // 👇 When used in Expressions, attribute names have to be provided separately
    ExpressionAttributeNames: {
      '#r': 'rank',
    },
    // 🤦‍♂️ List of attribute names as strings separated by commas
    ProjectionExpression: '#r',
    // 👇 Attribute values have to be provided separately
    ExpressionAttributeValues: {
      // 👇 No validation or type-safety to enforce DB schema
      ':p': 'MOVIE_PLOT',
      ':r': 'MOVIE_RANK',
    },
  }),
);

It is a very simple example (updating two fields of a Movie item), yet already very verbose 😅 And things only get messier as the complexity of your business logic grows: What if your items have 20 attributes? With some of them deeply nested? Or optional? What if you want to index an item or not depending on its values (e.g. a status attribute)? What about polymorphism?

In those cases (which are quite common) the required code to generate those requests can get very hard to maintain. That is why, very early on, developers built open-source libraries to “wrap” the DynamoDB client, with two goals in mind:

🏋️‍♀️ Simplifying the writing of DynamoDB requests
📐 Adding run-time data validation, i.e. “artificial” schemas to a schema-less DB (and more recently type-safety)

For instance, here is an example of the same UpdateCommand with one of those wrappers, DynamoDB-Toolbox:

import { Table, Entity } from 'dynamodb-toolbox';

// Provided some schema specifications...
const MovieTable = new Table({
  name: 'TABLE_NAME',
  partitionKey: 'title',
  sortKey: 'year',
  DocumentClient, // <= the original DocumentClient
});

const MovieEntity = new Entity({
  name: 'Customer',
  attributes: {
    title: { partitionKey: true, type: 'string' },
    year: { sortKey: true, type: 'string' },
    info: { type: 'map' },
  },
  table: MovieTable,
} as const);

// ...we get a validated AND type-safe request method 🙌
await MovieEntity.update({
  title: 'MOVIE_NAME',
  year: 'MOVIE_YEAR',
  info: {
    plot: 'MOVIE_PLOT',
    rank: 'MOVIE_RANK',
  },
});

And just like that, we went from an obscure 18-line object to a readable and elegant 8-liner 🤩 Not bad, don't you think?

DynamoDB-Toolbox is not the only SDK wrapper out there. If you browse Alex DeBrie’s awesome-dynamodb-tools repo, you’ll actually find a bunch of them. So, which one should you chose?

In this article, we took an in depth comparison of the 4 most popular DynamoDB wrappers:

🦌 Dynamoose: Dynamoose is the OG of DynamoDB wrappers. Created in 2015, it provides a syntax that closely mirrors that of the popular Mongoose library for MongoDB.
🧰 DynamoDB Toolbox: DynamoDB-Toolbox was created by Jeremy Daly, AWS Serverless Hero, and writer of the newsletter off-by-none. It was first released in September 2018 and has gained in popularity ever since.
⚡️ ElectroDB: Released in April 2020 by Tyler W. Walsh, ElectroDB benefits from having been created after Typescript won over the JS ecosystem. So it has a higher focus on type-safety than its predecessors.
💍 DynamoDB-OneTable: First released in January 2021, DynamoDB-OneTable is maintained by Sensedeep and is part of its broader Serverless Developer Studio offer.

We ranked them based on the following criteria:

📣 Library state: Classic open-source KPIs such as number of downloads, community, documentation etc.
🏗️ Data modeling: The broadness of their Entity definition API. Do they allow attribute name re-mapping (useful for single-table design)? Do they support enums? Nested attributes definitions? Computing indexes from other attributes?
✨ Typescript support: Type-safety is all the rage these days! All libraries come with some sort of type-safety, but type-inference (i.e. inferring tailored types from custom schemas) is still hard to get right.
🤖 API: How easy it is to do common DynamoDB requests like put, get or query... with secondary indexes, filters and conditions (You can find examples for each wrapper in our dedicated repo).

📣 Library state

(*as of 2023/02)	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
First release date	2014-02-27	2019-11-20	2020-03-11	2021-01-12
Last release date	✅ Jan. 6, 2023	✅ Jan. 8, 2023	✅ Jan. 20, 2023	✅ Jan. 25, 2023
Github ⭐️	1900	1400	530	505
NPM weekly downloads	86 k	38 k	4 k	16 k
Bundle size	🟡 382 kB	✅ 64.1 kB	🟡 176.7 kB	✅ 64.3 kB
Documentation	✅	✅	🟡 No global search	❌ Can be improved
DynamoDB Client v3 compatibility	✅	❌	✅	✅

All four libraries are well maintained, and have enough GitHub stars and npm downloads to be considered “battle tested”.

Dynamoose has the highest stats (probably from being the first one around). However, it’s the heaviest, with a size of 382KB, which is not negligible, considering that bundles above 5MB negatively impact Lambdas cold starts.

The main takeaways are that DynamoDB-Toolbox is not compatible with the V3 of the DynamoDB client (though it should be coming soon), and that the DynamoDB-OneTable documentation leaves to be desired.

🏗️ Data modeling

We initially started with a very broad scope of features useful for Entity definition (like specifying attributes as required, or aliasing attributes). However, most of them were already implemented by all libraries. For the sake of simplicity, we removed them and kept the following ones:

Nested attributes definition: Could we type nested fields of lists and maps attributes (like plot and rank in our first Movie example)?
Enum support: Could we specify a finite range of values for a primitive attribute?
Default values: Could we provide default values for an attribute? That is especially useful for entities with “simple” access patterns like fixed strings. We differentiated independent defaults (fixed or derived from context such as timestamps or env variables) from dependent defaults (computed from other attributes).
Pre-save/post-fetch attribute transformation: This can be needed for technical reasons, such as prefixing attributes. When possible, it’s best to hide such details from your code and let your wrapper handle the heavy-lifting.
Polymorphism support: Sometimes, items can switch between different statuses and shapes that go with them. We tested how easy it was to translate to in each library.

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Nested attributes definition	✅	❌	✅	✅
Enum support	✅	❌	✅	✅
Independent defaults	✅	✅	✅	✅
Dependent defaults	❌	✅	🟡 Via the `set` option	🟡 Via string templates like `"user#${email}"`
Attribute value transformation	✅	✅	✅	❌
Polymorphism	❌	❌	❌	❌

Overall, Dynamoose and ElectroDB have the upper hand, with ElectroDB being slightly ahead as it allows deriving attributes default values from other attributes.

Surprisingly, none of those libraries handles polymorphism and type-safe dependent defaults. As a maintainer of DynamoDB-Toolbox, I know for sure that those features are coming in the next major, so if you’re already using it, do not consider migrating to ElectroDB just yet 🙂

✨ Typescript support

All libraries support Typescript at a basic level, so we mostly focused on type inference. We looked for type inference in:

DynamoDB requests: Root and nested level attributes
Dependent defaults definition
Expressions: Conditions, filters and projections

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Requests (Root attributes)	❌	✅	✅	✅
Requests (Nested attributes)	❌	❌	✅	✅
Dependent defaults	❌	❌	❌	❌
Expressions	❌	🟡 yes but only at root level	✅	❌ Via string templates like `"(${role} = {admin})"`
IDE performances	❌	❌ Slow	✅	✅

Once again ElectroDB has the upper hand here. Very nice job, Tyler W. Walsh 😎

🤖 API

Finally we compared the APIs of each solution for sending DynamoDB requests, i.e. how easy they were to write and understand (verbosity, complexity etc).

Single Item Requests: For usual PUT, GET and DELETE requests.
Updates: For UPDATE requests. We tested updating nested fields, adding values to numbers and elements to lists.
Queries: For querying and using secondary indexes.
Conditions & filters: For conditioned write operations and filtered query results.

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Single Item Requests	🟡 Clunky	✅	✅	🟡 Hard to override an existing item (`PUT`)
Updates	🟡 Clunky	🟡 Operations (`$delete`, `$add`...) mixed with attributes	✅	🟡 Operations (`delete`, `add`...) are far from attributes (in `options`)
Queries	❌ Uses only FilterExpression	🟡 Confusing query formalism	✅	✅
Conditions & filters	🟡 Verbose, ORM-ish	🟡 Confusing `OR` clauses	🟡 Easy to use but incomplete (e.g. `AND`/`OR` clauses)	❌ String template 🙄

If the previous section wasn't enough, this one should convince you NOT to use Dynamoose: I found it hard to believe, but it doesn't even support basic (efficient) queries, and uses only filter expressions under the hood 🙄.

That being said, ElectroDB has a the nicest API. I especially liked the possibility to name queries according to their business meaning, but not the find and match methods, which are not native DynamoDB requests and can build costly and inefficient scan requests without you being aware.

Conclusion

Overall, as of march 2023, ElectroDB looks like the best DynamoDB client wrapper. Although it is newer and less “battle tested”, it is better than its concurrent on every other criteria: It has the same data modeling features, a more complete type inference, and a nicer API.

Nevertheless, there are some parts that could be improved:

Type inference still has some blind spots
There's no support for polymorphism
Entity definition autocompletion could be more helpful (it would benefit from a zod-like approach)

Finally, I would not rule out DynamoDB-Toolbox just yet! Its next major is just around the corner, with many new capabilities that even ElectroDB doesn’t have (such as type-safe dependent defaults and polymorphism). So expect a round 2 of this article in the next few months… 😉

An in-depth comparison of the most popular DynamoDB wrappers

Thomas Aribart — Tue, 21 Mar 2023 12:09:39 +0000

⚠️ Update ⚠️: Since the v1 of DynamoDB-Toolbox has been released, I strongly recommend using it over ElectroDB:

Slick zod-like schema declaration

Support for polymorphism

Better tree-shaking

Type-safe dependent defaults and more!

Check it out 👉 https://www.dynamodbtoolbox.com/

For instance, let’s look at this UpdateCommand example from the DynamoDB docs itself:

await DocumentClient.send(
  new UpdateCommand({
    TableName: 'TABLE_NAME',
    Key: {
      // 👇 No type-safety on the primary key
      title: 'MOVIE_NAME',
      year: 'MOVIE_YEAR',
    },
    // 👇 String expressions hard to build (and still no type-safety)
    UpdateExpression: 'set info.plot = :p, info.#r = :r',
    // 👇 When used in Expressions, attribute names have to be provided separately
    ExpressionAttributeNames: {
      '#r': 'rank',
    },
    // 🤦‍♂️ List of attribute names as strings separated by commas
    ProjectionExpression: '#r',
    // 👇 Attribute values have to be provided separately
    ExpressionAttributeValues: {
      // 👇 No validation or type-safety to enforce DB schema
      ':p': 'MOVIE_PLOT',
      ':r': 'MOVIE_RANK',
    },
  }),
);

🏋️‍♀️ Simplifying the writing of DynamoDB requests
📐 Adding run-time data validation, i.e. “artificial” schemas to a schema-less DB (and more recently type-safety)

For instance, here is an example of the same UpdateCommand with one of those wrappers, DynamoDB-Toolbox:

import { Table, Entity } from 'dynamodb-toolbox';

// Provided some schema specifications...
const MovieTable = new Table({
  name: 'TABLE_NAME',
  partitionKey: 'title',
  sortKey: 'year',
  DocumentClient, // <= the original DocumentClient
});

const MovieEntity = new Entity({
  name: 'Customer',
  attributes: {
    title: { partitionKey: true, type: 'string' },
    year: { sortKey: true, type: 'string' },
    info: { type: 'map' },
  },
  table: MovieTable,
} as const);

// ...we get a validated AND type-safe request method 🙌
await MovieEntity.update({
  title: 'MOVIE_NAME',
  year: 'MOVIE_YEAR',
  info: {
    plot: 'MOVIE_PLOT',
    rank: 'MOVIE_RANK',
  },
});

And just like that, we went from an obscure 18-line object to a readable and elegant 8-liner 🤩 Not bad, don't you think?

DynamoDB-Toolbox is not the only SDK wrapper out there. If you browse Alex DeBrie’s awesome-dynamodb-tools repo, you’ll actually find a bunch of them. So, which one should you chose?

In this article, we took an in depth comparison of the 4 most popular DynamoDB wrappers:

🦌 Dynamoose: Dynamoose is the OG of DynamoDB wrappers. Created in 2015, it provides a syntax that closely mirrors that of the popular Mongoose library for MongoDB.
🧰 DynamoDB Toolbox: DynamoDB-Toolbox was created by Jeremy Daly, AWS Serverless Hero, and writer of the newsletter off-by-none. It was first released in September 2018 and has gained in popularity ever since.
⚡️ ElectroDB: Released in April 2020 by Tyler W. Walsh, ElectroDB benefits from having been created after Typescript won over the JS ecosystem. So it has a higher focus on type-safety than its predecessors.
💍 DynamoDB-OneTable: First released in January 2021, DynamoDB-OneTable is maintained by Sensedeep and is part of its broader Serverless Developer Studio offer.

We ranked them based on the following criteria:

📣 Library state: Classic open-source KPIs such as number of downloads, community, documentation etc.
🏗️ Data modeling: The broadness of their Entity definition API. Do they allow attribute name re-mapping (useful for single-table design)? Do they support enums? Nested attributes definitions? Computing indexes from other attributes?
✨ Typescript support: Type-safety is all the rage these days! All libraries come with some sort of type-safety, but type-inference (i.e. inferring tailored types from custom schemas) is still hard to get right.
🤖 API: How easy it is to do common DynamoDB requests like put, get or query... with secondary indexes, filters and conditions (You can find examples for each wrapper in our dedicated repo).

📣 Library state

(*as of 2023/02)	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
First release date	2014-02-27	2019-11-20	2020-03-11	2021-01-12
Last release date	✅ Jan. 6, 2023	✅ Jan. 8, 2023	✅ Jan. 20, 2023	✅ Jan. 25, 2023
Github ⭐️	1900	1400	530	505
NPM weekly downloads	86 k	38 k	4 k	16 k
Bundle size	🟡 382 kB	✅ 64.1 kB	🟡 176.7 kB	✅ 64.3 kB
Documentation	✅	✅	🟡 No global search	❌ Can be improved
DynamoDB Client v3 compatibility	✅	❌	✅	✅

All four libraries are well maintained, and have enough GitHub stars and npm downloads to be considered “battle tested”.

The main takeaways are that DynamoDB-Toolbox is not compatible with the V3 of the DynamoDB client (though it should be coming soon), and that the DynamoDB-OneTable documentation leaves to be desired.

🏗️ Data modeling

Nested attributes definition: Could we type nested fields of lists and maps attributes (like plot and rank in our first Movie example)?
Enum support: Could we specify a finite range of values for a primitive attribute?
Default values: Could we provide default values for an attribute? That is especially useful for entities with “simple” access patterns like fixed strings. We differentiated independent defaults (fixed or derived from context such as timestamps or env variables) from dependent defaults (computed from other attributes).
Pre-save/post-fetch attribute transformation: This can be needed for technical reasons, such as prefixing attributes. When possible, it’s best to hide such details from your code and let your wrapper handle the heavy-lifting.
Polymorphism support: Sometimes, items can switch between different statuses and shapes that go with them. We tested how easy it was to translate to in each library.

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Nested attributes definition	✅	❌	✅	✅
Enum support	✅	❌	✅	✅
Independent defaults	✅	✅	✅	✅
Dependent defaults	❌	✅	🟡 Via the `set` option	🟡 Via string templates like `"user#${email}"`
Attribute value transformation	✅	✅	✅	❌
Polymorphism	❌	❌	❌	❌

Overall, Dynamoose and ElectroDB have the upper hand, with ElectroDB being slightly ahead as it allows deriving attributes default values from other attributes.

✨ Typescript support

All libraries support Typescript at a basic level, so we mostly focused on type inference. We looked for type inference in:

DynamoDB requests: Root and nested level attributes
Dependent defaults definition
Expressions: Conditions, filters and projections

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Requests (Root attributes)	❌	✅	✅	✅
Requests (Nested attributes)	❌	❌	✅	✅
Dependent defaults	❌	❌	❌	❌
Expressions	❌	🟡 yes but only at root level	✅	❌ Via string templates like `"(${role} = {admin})"`
IDE performances	❌	❌ Slow	✅	✅

Once again ElectroDB has the upper hand here. Very nice job, Tyler W. Walsh 😎

🤖 API

Finally we compared the APIs of each solution for sending DynamoDB requests, i.e. how easy they were to write and understand (verbosity, complexity etc).

Single Item Requests: For usual PUT, GET and DELETE requests.
Updates: For UPDATE requests. We tested updating nested fields, adding values to numbers and elements to lists.
Queries: For querying and using secondary indexes.
Conditions & filters: For conditioned write operations and filtered query results.

	🦌 Dynamoose	🧰 DynamoDB-Toolbox	⚡️ ElectroDB	💍 DynamoDB-OneTable
Single Item Requests	🟡 Clunky	✅	✅	🟡 Hard to override an existing item (`PUT`)
Updates	🟡 Clunky	🟡 Operations (`$delete`, `$add`...) mixed with attributes	✅	🟡 Operations (`delete`, `add`...) are far from attributes (in `options`)
Queries	❌ Uses only FilterExpression	🟡 Confusing query formalism	✅	✅
Conditions & filters	🟡 Verbose, ORM-ish	🟡 Confusing `OR` clauses	🟡 Easy to use but incomplete (e.g. `AND`/`OR` clauses)	❌ String template 🙄

Conclusion

Nevertheless, there are some parts that could be improved:

Type inference still has some blind spots
There's no support for polymorphism
Entity definition autocompletion could be more helpful (it would benefit from a zod-like approach)

Type-safe S3 Select queries with Kysely

Thomas Aribart — Wed, 22 Feb 2023 12:42:53 +0000

S3 Select is an Amazon S3 feature that enables retrieving subsets of S3 Objects content via SQL expressions. You can use clauses like SELECT and WHERE to fetch data from CSVs, JSONs, or Apache Parquet files, even if they are compressed with GZIP and/or server-side encrypted.

S3 Select is simple to use, cost-effective and can drastically improve the performances of your application depending on your query. It is overall a great addition to the Serverless developer toolkit, particularly when:

You need to access a large amount of data (which would make it unfit for other storage solutions like DynamoDB)
You need to query and filter it in a complex and/or dynamic way
You don’t need to use a JOIN clause (as S3 Select can only query a single file) or to update a single record of the data (it is called S3 Select, not S3 Insert 🙂)

In this article, we’ll learn how to run S3 Select commands in Typescript, and how we can improve our DevX and type-safety using Kysely.

Querying with S3 Select

Let’s say that we have a DB of Pokemons in the shape of a CSV, stored somewhere in a S3 Bucket:

id ; name      ; customName ; type     ; level ; generation
1  ; pikachu   ;            ; electric ; 42    ; 1
2  ; charizard ;            ; fire     ; 54    ; 1
3  ; meganium  ; plantyDino ; grass    ; 26    ; 2
...

What if we want to retrieve the fire Pokemons from generation 1 and 2? Well, S3 Select let us do that with the following query:

import { S3Client, SelectObjectContentCommand } from '@aws-sdk/client-s3';

export const s3Client = new S3Client({
  region: 'us-east-1', // <= Your region here
});

const { Payload } = await s3Client.send(
  new SelectObjectContentCommand({
    // 👇 Those first params are required but don't mind them
    ExpressionType: 'SQL',
    OutputSerialization: {
      JSON: {
        RecordDelimiter: ',',
      },
    },
    // 👇 Those depends on the CSV
    InputSerialization: {
      CSV: {
        FileHeaderInfo: 'USE',
        FieldDelimiter: ';',
        QuoteCharacter: '"',
      },
    },
    // 👇 Those are the most important
    Bucket: 'my-super-bucket-name',
    Key: 'pokedex.csv',
    Expression: `
            select "id", "name", "customName", "type" as "pokemonType", "level"
                from "S3Object"
                where
                    "generation" in ('1', '2')
                    and "type" = 'fire'
      `,
  }),
);

// Note that this command requires the s3:GetObject permission

This is great and all but we can make it better:

📝 We need to add some custom parsing to actually use the response Payload
👍 We can use Kysely to produce the SQL expression in a type-safe and devX-friendly way
🌈 We can also use it to type the command result

Parsing the query response

The Payload is not a plain Javascript object but an instance of AsyncInterable. Now what the hell is an AsyncIterable you ask me? Well, you could look for the official Async Iterator documentation on MDN… or you can just stop asking questions and use the following helper:

import type { SelectObjectContentEventStream } from '@aws-sdk/client-s3';

// 👇 TextDecoder is a native Node/Browser class
const textDecoder = new TextDecoder();

const parseS3SelectEventStream = async (
  s3SelectEventStream:
    | AsyncIterable<SelectObjectContentEventStream>
    | undefined,
): Promise<unknown[]> => {
  if (!s3SelectEventStream) {
    return [];
  }

  const stringifiedJSONOutputs: string[] = [];

  for await (const event of s3SelectEventStream) {
    if (event.Records) {
      const stringifiedJSONOutput = textDecoder.decode(event.Records.Payload);
      stringifiedJSONOutputs.push(stringifiedJSONOutput);
    }
  }

  const rows = JSON.parse(
    '[' + stringifiedJSONOutputs.join('').slice(0, -1) + ']',
  ) as unknown[];

  return rows;
};

Now we can parse the output like this:

const { Payload: s3SelectEventStream } = await s3Client.send(
  new SelectObjectContentCommand({
    // ...
  }),
);

// 🎉 Ta-da!
const myPokemons: unknown[] = await parseS3SelectEventStream(
  s3SelectEventStream,
);

Building the query with Kysely

Let’s face it, writing SQL queries by hand is a pain and very error prone. Besides, wouldn't it be nice to have some type error if the CSV suddenly changes shape?

That’s where Kysely comes to the rescue: Kysely is a type-safe and devX-friendly typescript SQL query builder. It was designed to work with PostgreSQL and MySQL, but it exposes a few classes that can let us write queries without being connected to an actual relational database.

Let’s begin by designing our CSV type:

enum PokemonType {
  Water = 'water',
  Grass = 'grass',
  Fire = 'fire',
  // ...
}

interface PokemonCSV {
  id: string;
  name: string;
  customName?: string;
  type: PokemonType;
  // 👇 In CSVs everything is a string
  level: string;
  generation: '1' | '2'; // ...up to 9
}

Next, let’s install Kysely and instanciate a Kysely database:

# npm
npm install kysely

# yarn
yarn add kysely

import {
  Kysely,
  DummyDriver,
  SqliteAdapter,
  SqliteIntrospector,
  SqliteQueryCompiler,
} from 'kysely';

interface Database {
  S3Object: PokemonCSV;
}

const db = new Kysely<Database>({
  dialect: {
    createAdapter: () => new SqliteAdapter(),
    createDriver: () => new DummyDriver(),
    createIntrospector: ($db: Kysely<unknown>) => new SqliteIntrospector($db),
    createQueryCompiler: () => new SqliteQueryCompiler(),
  },
});

// That’s it 🎉

Simple, no? Now, let’s write the same query, but while enjoying some type-safety and auto-completion:

const kyselyQuery = db
  .selectFrom('S3Object')
  .select([
    'id',
    'name',
    'customName',
    // 🙌 You can rename columns as you like
    'type as pokemonType',
    'level',
    // 💥 Will trigger an error:
    'unexistingColumn',
  ])
  // 🙌 Every method is type-safe!
  .where('generation', 'in', ['1', '2'])
  .where('type', '=', PokemonType.Fire);

To protect us from nasty SQL injections, Kysely doesn’t directly provide us with the SQL expression but with a sql string and parameters array:

const {
  sql, // 👈 SQL query with '?' as placeholders
  parameters, // 👈 Array of parameters
} = kyselyQuery.compile();

Because S3 Select doesn’t accept parameters in its API, we have to hydrate the parameters ourselves:

const dangerouslyHydrateSQLParameters = (
  sql: string,
  parameters: readonly unknown[],
): string => {
  for (const parameter of parameters) {
    sql = sql.replace('?', `'${String(parameter)}'`);
  }

  return sql;
};

const { sql, parameters } = kyselyQuery.compile();

// ⛔️ **BE SURE TO VALIDATE DYNAMIC PARAMETERS FIRST** ⛔️
const sqlExpression = dangerouslyHydrateSQLParameters(sql, parameters);
console.log(sqlExpression);

// 👇 We retrieve the same expression as above:
// select "id", "name", "customName", "type" as "pokemonType", "level"
//   from "S3Object"
//   where
//     "generation" in ('1', '2')
//     and "type" = 'fire'

We just have to provide it to our S3 Select command and voilà! We’re done!

Well almost: Notice that we only typed our SQL query expression! What about the response that comes from S3?

Inferring the response type

Initially, Kysely was not only made to build queries but also execute them. We can just benefit from the inferred type by inspecting the execute property of our Kysely query. That can be done with the help of some TS wizardry:

import type { SelectQueryBuilder } from 'kysely';

type QueryResponseRow<
  // 👇 Add a large type constraint
  KyselyQuery extends SelectQueryBuilder<
    Record<string, unknown>,
    string,
    unknown
  >,
> =
  // 👇 Remove the Promise wrapper
  Awaited<
    // 👇 Get the return type of the execute method
    ReturnType<KyselyQuery['execute']>
    // 👇 Unpack the array
  >[number];

type Pokemon = QueryResponseRow<typeof kyselyQuery>;

// 👇 Equivalent to:
type Pokemon = {
  id: string;
  name: string;
  // 👍 customName is indeed possibly undefined
  customName: string | undefined;
  level: string;
  // 🙌 "type" property has been renamed
  pokemonType: PokemonType;
};

Now we can just use this type the response of our parseS3SelectEventStream util and voilà! We’re done! For good this time 🙂

Conclusion

Both S3 Select and Kysely are awesome tools. By joining both of them, we can run performant, scalable and cost-effective queries while benefitting from strong and DRY type-safety and inference 🥳

Note that there is one minor drawback, though: Kysely will add 120KB in your Lambdas bundles (props to for serverless-analyze-bundle-plugin for helping me out with this 🙌). It is not a lot, but not negligible either as NodeJS Lambdas bundles above 5MB negatively impacts their cold starts. So you might want to re-evaluate adding Kysely to your bundles if your query is not changing often.

Monitor the CO2 emissions of your AWS application with Cloud Carbon Footprint 🌱

Thomas Aribart — Wed, 06 Apr 2022 08:48:26 +0000

I think I don’t need to remind you that climate change is a big deal and that humanity must urgently reduce its CO2 emissions.

We, web developers, must do our part: It has been estimated data centers produce 2% of the global greenhouse gases emissions.

To help us take data-driven actions, cloud providers have started building monitoring dashboards of our cloud usage carbon emissions:

Microsoft released the Emissions Impact Dashboard in 2020
Google released the Carbon Footprint service on GCP in 2021
And AWS just released their Customer Carbon Footprint Tool last month (feb. 2022)

At Kumo, we develop Serverless applications (which, by the way, seem to be inherently greener...). So this article will focus on AWS, which has always been ahead regarding Serverless.

The sad thing is that AWS have been quite late regarding CO2 emissions monitoring: Their Customer Carbon Footprint Tool has just been just released (2 years after Azure), and, for now, suffers from strong limitations:

Results have high granularity (100 kgCO2 equivalent versus 1 for GCP’s Carbon Footprint)
Only the S3 and EC2 services are available in the emissions breakdown by service : In the case of a Serverless application, most of the emissions will fall... in the “Other” category

So we’ve searched for alternatives, and found Cloud Carbon Footprint 🎉

Cloud Carbon Footprint is an open-source application that enables projects to monitor their cloud usage CO2 emissions on GCP, Azure... and AWS! It also offer nice features like breakdowns by region, account or AWS service:

Feel free to test the online demo for more details!

The emissions are derived from billing. The billing-to-CO2 coefficients are debated by the Cloud Carbon Footprint community (and overridable if needed). You can find more infos on the project methodology page, and contribute to their Github project or Google Group.

This article explains how set up Cloud Carbon Footprint on your project in only a few clicks, so you can start taking actions now! Let’s dive in 🙌

📖 Create a Cost & Usage Reports

Before setting Cloud Carbon Footprint up, let’s take a quick look at how it works.

Technically, Cloud Carbon Footprint uses Athena to run queries on a S3 containing the data of Cost & Usage Reports made by AWS Billing. It’s a mouthful, so let’s draw it:

One consequence is that you cannot install Cloud Carbon Footprint and have it work out-of-the-box: You’ll have to do some setup in your AWS account first.

Setting up all those resources by hand can be quite long... Thankfully, you won’t have to do it, because we made a CloudFormation template just for you ❤️

To use it:

Download the JSON template from my Github repository
Open the Cloudformation service in the us-east-1 region (we’ll see why later)
Select Create stack > With new resources and Template is ready

Select Upload a template file, upload the JSON template and select Next

Choose a stack name (like cloud-carbon-footprint) and select Next until you reach the “Review” page

In the “Review” page, accept the check boxes and select Create stack

And voilà! For the curious, this template will deploy:

🪣 A S3 Bucket and a S3 Bucket Policy to store the Cost & Usage Report (”BillingDataBucket”)
📖 The Cost & Usage Report definition (”CurDefinition”)
🏊‍♀️ The necessary Glue, Lambda and IAM resources to allow Athena to crawl the Cost & Usage Report
🪣 A S3 bucket to store Athena query results (”AthenaQueryResultsBucket”)
🔑 The necessary IAM resources to allow Cloud Carbon Footprint to perform said queries (”CloudCarbonFootprintUserPolicy” and “CloudCarbonFootprintUser”)

Two important notes:

It is important to base this stack in the us-east-1 region as the AWS::CUR::ReportDefinition resource type is only available there (don’t worry, reports are still cross-regions)
If your account is not the paying account (like a linked account in AWS Organization), the paying account should enable Cost & Usage Reports first in the AWS Billing Console

Once the required resources are deployed, simply wait: It can take up to 24h for AWS Billing to create the first Cost & Usage Report.

🚀 Launch Cloud Carbon Footprint

Once a first Cost & Usage Report is available (it will be visible in the AWS Billing Console > check the “Data last refreshed” column), you are ready to launch Cloud Carbon Footprint!

Open the IAM Console and select Create Access Key on the cloud-carbon-footprint user: Write down the Access key ID and the Secret access key (you will only see the latter once)
If you haven’t done it, install the AWS CLI
Configure a new profile and set it as your default user



aws configure --profile <PROFILE_NAME>
# AWS Access Key ID: <ACCESS_KEY_ID>
# AWS Secret Access Key: <SECRET_ACCESS_KEY>
# Default region name: Your default region (ex: eu-west-1)
# Default output format: Your default output format (ex: json)
export AWS_PROFILE=<PROFILE_NAME>

Install the cloud-carbon-footprint package via npx



npx @cloud-carbon-footprint/create-app
# Submit your PROJECT_NAME in the wizard
cd <PROJECT_NAME>
yarn install
yarn guided-install
# Cloud provider: AWS
# AWS account id: <YOUR_PROJECT_ACCOUNT_ID> (e.g. 628019955336)
# AWS account name: <YOUR_PROJECT_NAME> (e.g. "[PRODUCTION] Master Account")
# AWS region: us-east-1 (should be the region of the CUR bucket)
# Ensure your AWS account has the correct permission: Yes
# AWS role name: <ANY_VALUE> (doesn't work)
# Enable the Cost and Usage Reports Billing AWS feature: Yes
# Setup Athena DB to save the Cost and Usage Reports: Yes
# Athena DB name: costandusagereportdatabase
# Athena DB table: costandusagereport
# Athena DB query result location: s3://<STACK_NAME>-athenaqueryresultsbucket-<SOME_HASH>
# (can be found in the s3 service)
# Install the AWS CLI: Yes
# Configure aws credentials locally: No
# Run `aws configure`: Yes
yarn start

And voilà! You should have your dashboard ready to go 🚀

The data will go back up to the oldest data available in the Cost and Usage Report, which should be the beginning of its creation month, and will be refreshed at least daily. It is important not to remove the Report as long as you want to monitor your emissions.

Now you can start taking actions in favor of climate 💪

Cheers!

Forem: Thomas Aribart

MCP Client: Building a Smart and Robust Integration to DynamoDB with DynamoDB-Toolbox

1. Preparing our Models

Create a Table

Create a DynamoDB-Toolbox Table Instance

Define Schemas and Entities

Create Access Patterns

2. Launch Your MCP Server

Define a Database

Authenticate to AWS

Launch the McpServer

3. Run Your MCP Client 🙌

Conclusion

🌐 Building a multi-region Serverless API with the AWS CDK, Lambda, and DynamoDB

🧱 Architecture Overview

Prerequisites

📦 Step 1: Set Up Your CDK Project

🌍 Step 2: Choose Deployment Regions

🏗️ Step 3: Define a DynamoDB Global Table

🌐 Step 4: Set Up Custom Domains

🛰️ Step 5: Add Latency-Based Routing

⚙️ Step 6: Add a Lambda Function

🚀 Step 7: Deploy and Test

💭 Final Thoughts

🛠️ Conclusion

[UPDATED] The DynamoDB-Toolbox v1 beta is here 🙌 All you need to know!

Table of content

Installation

Tables

Entities

Timestamps

Matching the Table schema

SavedItem and FormattedItem

Designing Entity schemas

Schema definition

Attributes types

Any

Primitives

Set

List

Map

Record

AnyOf

Looking forward

Computed defaults

Commands

PutItemCommand

GetItemCommand

DeleteItemCommand

UpdateItemCommand

Removing an attribute

Referencing saved values

Non-recursive attributes

Recursive attributes

Any and anyOf attributes

Mocking entities in tests

Utility helpers and types

formatSavedItem

Condition and parseCondition

Projection and parseProjection

KeyInput and PrimaryKey

Errors

Conclusion

New DynamoDB-Toolbox v1 beta: Features and breaking changes

New features

UpdateItemCommand

MockEntity

Providing getters for table names

Breaking changes

default options

computeDefaults

Conclusion

The DynamoDB-Toolbox v1 beta is here 🙌 All you need to know!

Table of content

Installation

Tables

Entities

Timestamps

Matching the Table schema

SavedItem and FormattedItem

Create a DynamoDB-Toolbox `Table` Instance

Define a `Database`

Launch the `McpServer`

`default` options

`computeDefaults`