Forem: Daniel Worsnup

Directory-Based Resolver Mapping for AWS AppSync

Daniel Worsnup — Thu, 13 Mar 2025 23:02:26 +0000

Cross-posted from my tech blog.

Managing AWS AppSync Pipeline Resolvers at Scale

AWS AppSync pipeline resolvers enable multi-step processing for GraphQL field resolution. However, as GraphQL schemas grow, several challenges emerge:

Disorganized Code Placement: Resolvers stored within the AWS console or as scattered files become difficult to locate in schemas with hundreds of fields.
Inconsistent Naming: Without standardization, resolver files develop inconsistent patterns that impede discovery and maintenance.
Implicit Dependencies: Pipeline resolvers often depend on specific data sources without explicit links in the code.
Team Scaling Issues: Multiple developers contribute divergent implementation patterns without a consistent structure.
Deployment Challenges: Coordinating resolver changes across diverse data sources introduces errors.
Complex Troubleshooting: Identifying specific functions causing errors becomes time-consuming without clear organization.

Directory-Based Schema Mapping

This approach organizes resolvers by mirroring the GraphQL schema structure in your filesystem:

pipelines/
  ├── {TypeName}/
  │     └── {FieldName}/
  │           ├── meta.json
  │           ├── 0.resolver.js
  │           └── 1.resolver.js (optional additional functions)

For example, this GraphQL query:

type Query {
  listRestaurants(input: ListRestaurantsInput!): RestaurantPage!
}

Maps to:

pipelines/Query/listRestaurants/

Implementation Components

1. Resolver Functions

Each resolver function is a numbered JavaScript file implementing AppSync's request/response pattern:

// pipelines/Query/listRestaurants/0.resolver.js
export function request(ctx) {
  const { args } = ctx;
  return {
    operation: 'Query',
    query: {
      expression: '#location = :location',
      expressionNames: { '#location': 'location' },
      expressionValues: { ':location': args.input.location }
    },
  };
}

export function response(ctx) {
  return ctx.result;
}

2. Data Source Configuration

A meta.json file specifies the data source configuration. Each element of the dataSourceConfigs array defines the data source for the resolver with the same index. The example below specifies that the resolver defined by the file 0.resolver.js uses a DynamoDB data source and requires permissions for the RestaurantsTable.

// pipelines/Query/listRestaurants/meta.json
{
  "dataSourceConfigs": [
    {
      "type": "ddb",
      "tableArns": [
        "arn:aws:dynamodb:us-east-1:123456789012:table/RestaurantsTable"
      ]
    }
  ]
}

3. Automated Resolver Registration

The infrastructure code scans the directory structure to register resolvers:

export function defineResolvers({ scope, api }) {
  const typeNamePaths = globSync(path.join(__dirname, `./pipelines/*`), { withFileTypes: true });

  for (const typeNamePath of typeNamePaths) {
    if (!typeNamePath.isDirectory()) continue;

    const fieldNamePaths = globSync(`${typeNamePath.fullpath()}/*`, { withFileTypes: true });
    for (const fieldNamePath of fieldNamePaths) {
      if (!fieldNamePath.isDirectory()) continue;

      definePipelineResolver({
        scope,
        api,
        typeName: typeNamePath.name,
        fieldName: fieldNamePath.name,
        path: fieldNamePath,
      });
    }
  }
}

For each resolver, the infrastructure code scans the resolver directory for {n}.resolver.js source files and the meta.json file:

function definePipelineResolver({
  scope,
  api,
  typeName,
  fieldName,
  path,
}) {
  const pipelineName = `${typeName}${upperFirst(fieldName)}Pipeline`;
  const functionDefs = [];
  let meta = null;

  // Scan resolver directory for files and configuration
  for (const child of path.readdirSync()) {
    if (child.isDirectory()) continue;

    const match = child.name.match(/^([0-9]+)\.resolver\.js$/i);
    if (match) {
      const index = +match[1];
      functionDefs.push({ index, path: child });
    } else if (child.name === 'meta.json') {
      meta = JSON.parse(fs.readFileSync(child.fullpath(), 'utf8'));
    }
  }

  // Sort functions by numerical prefix
  const sortedFunctionDefs = _.sortBy(functionDefs, ({ index }) => index);

  // Create pipeline configuration
  const pipelineConfig = sortedFunctionDefs.map(({ path, index }) => {
    const dataSourceConfig = meta.dataSourceConfigs[index];
    const dataSource = defineDataSource({ 
      scope, 
      name: `${pipelineName}DS${index}`, 
      api, 
      config: dataSourceConfig 
    });

    return defineFunction({
      name: `${pipelineName}Function${index}`,
      path,
      dataSource,
      scope,
      api,
    });
  });

  // Register the AppSync resolver
  new appsync.Resolver(scope, pipelineName, {
    api,
    typeName,
    fieldName,
    pipelineConfig,
    code: Code.fromInline(`
      function request(ctx) { return ctx; }
      function response(ctx) { return ctx.result; }
      export { request, response };
    `),
    runtime: FunctionRuntime.JS_1_0_0,
  });
}

4. Data Source Definition

Function to create data sources with appropriate IAM permissions:

function defineDdbDataSourceRole({
  scope,
  name,
  tableArns,
}: {
  scope: Construct;
  name: string;
  tableArns: string[];
}): Role | undefined {
  return new iam.Role(scope, name, {
    assumedBy: new iam.ServicePrincipal('appsync.amazonaws.com'),
    inlinePolicies: {
      AccessDynamoDbTables: new iam.PolicyDocument({
        statements: [
          new iam.PolicyStatement({
            actions: ['dynamodb:GetItem', 'dynamodb:Query', 'dynamodb:Scan' /* add actions as needed by resolvers */],
            resources: tableArns.reduce((collector: string[], tableArn: string) => {
              collector.push(tableArn);
              collector.push(`${tableArn}/index/*`);
              return collector;
            }, []),
          }),
        ],
      }),
    },
  });
}

function defineDataSource({
  scope,
  name,
  api,
  config,
}: {
  scope: Construct;
  name: string;
  api: IGraphqlApi;
  config: { type: string; tableArns: string[] };
}): BaseDataSource {
  if (config.type === 'ddb') {
    const serviceRole = defineDdbDataSourceRole({
      scope,
      name: `${name}ServiceRole`,
      tableArns: config.tableArns,
    });

    return new appsync.DynamoDbDataSource(scope, name, {
      api,
      table: dynamodb.Table.fromTableArn(scope, `${name}Table`, config.tableArns[0]),
      serviceRole,
    });
  }

  // Implement other types of data sources as needed

  throw new Error(`Unsupported data source type: ${config.type}`);
}

5. CDK Integration

Integrate the resolver mapping into your CDK stack:

import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as path from 'path';
import * as appsync from 'aws-cdk-lib/aws-appsync';
import { defineResolvers } from './resolvers';

export class ApiStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props);

    const graphqlApi = new appsync.GraphqlApi(this, 'GraphqlApi', {
      name: 'Api',
      definition: appsync.Definition.fromSchema(
        appsync.SchemaFile.fromAsset(path.join(__dirname, './shared/gql/schema.gql'))
      ),
    });

    // Register all resolvers from the directory structure
    defineResolvers({
      scope: this,
      api: graphqlApi,
    });
  }
}

Dependencies

This implementation relies on several key libraries and frameworks:

AWS CDK (Cloud Development Kit) - Infrastructure as code framework that enables defining AWS resources using TypeScript.
- aws-cdk-lib - Core CDK library
- aws-cdk-lib/aws-appsync - CDK constructs for AppSync
- aws-cdk-lib/aws-dynamodb - CDK constructs for DynamoDB
- aws-cdk-lib/aws-iam - CDK constructs for IAM
Constructs - Programming model for composing cloud resources.
- constructs - The base construct library
Lodash - JavaScript utility library used for its array and object manipulation functions.
- lodash - Specifically, _.sortBy() for sorting function definitions by index
Node.js Built-ins - Native Node.js modules used for filesystem operations.
- fs - For reading files and directories
- path - For path manipulation and joining
Glob - Pattern matching library for directory scanning.
- glob - The globSync function used for finding files matching patterns

These dependencies should be included in your package.json file, except for the Node.js built-ins which are available by default in Node.js environments.

Key Benefits

Schema-Aligned Organization: Filesystem mirrors GraphQL schema for intuitive resolver discovery.
Pipeline Step Ordering: Numbered files (e.g., 0.resolver.js) explicitly reflect execution order.
Logical Separation: Each file handles a distinct responsibility in the pipeline.
Simplified Maintenance: Easy location and modification of resolver logic through directory navigation.
Declarative Data Sources: Explicit data source configurations in meta.json files.
Automated Registration: New resolvers automatically register when added to the directory structure.

Conclusion

This directory-based approach creates a self-documenting system where filesystem structure reflects GraphQL schema organization. The pattern maintains clarity even as APIs scale to hundreds of fields and multiple data sources, while enabling automated infrastructure provisioning through AWS CDK.

By establishing this convention, team members can quickly locate, understand, and modify resolvers without hunting through disparate files or console UIs. The result is a maintainable, scalable architecture for complex AWS AppSync GraphQL APIs.

Like this post?

Follow me on Twitter where I tweet about frontend things: @thesnups

Shortcut Linking with AWS Identity Center SSO

Daniel Worsnup — Thu, 06 Mar 2025 00:18:55 +0000

Cross-posted from my tech blog.

When an alarm goes off at 3 AM, every second counts. Navigating through the AWS Console can be a frustrating experience when troubleshooting a critical issue. What if you could send your on-call engineers directly to the exact CloudWatch alarm page they need, bypassing the maze of AWS Console navigation?

This post demonstrates how to build a system that creates shortcut links to AWS resources through AWS Identity Center (formerly AWS SSO), enabling your team to go from alert notification to viewing the relevant resource in seconds.

The Problem with Traditional AWS Alerts

Traditional CloudWatch alarm notifications contain basic information such as:

Alarm name
State change (OK, ALARM, INSUFFICIENT_DATA)
Timestamp
Reason for the alarm

However, they don't provide a direct way to access the resource. Engineers receiving these alerts must:

Log in to the AWS Console
Navigate to the correct AWS account
Switch to the right region
Find the CloudWatch service
Locate the specific alarm

This process consumes valuable minutes that could be better spent addressing the actual issue.

The Solution: Shortcut Linking with AWS Identity Center

This solution streamlines incident response by creating authenticated shortcut links to specific AWS resources. These links handle the entire authentication process and navigation, taking engineers directly to the relevant resource with a single click.

Here's the high-level workflow:

CloudWatch alarm triggers and sends a notification to an SNS topic
A Lambda function processes the SNS message
The Lambda extracts the resource details (region, account ID, alarm name)
It constructs an AWS Identity Center shortcut link that points directly to the resource
The function sends a formatted message with this shortcut link to the on-call channel

The result? Engineers can view the exact resource in seconds rather than minutes.

What Does It Look Like?

The implementation suggested in this article will generate the shortcut link and send it to a Discord channel, which looks like this:

However, you can modify the Lambda code to send the alert to a different channel such as Slack, PagerDuty, or email.

Understanding AWS IAM Identity Center

Before diving into implementation, it's important to understand the technology that makes this possible. AWS IAM Identity Center provides a centralized way to manage access across multiple AWS accounts. For our shortcut linking solution, it offers three key capabilities:

Single Authentication Point: Users authenticate once and can access multiple AWS accounts
Role-Based Access: Automatically assumes the appropriate IAM role in the target account
Direct Resource Navigation: Can link directly to specific resources across accounts

When an engineer clicks our shortcut link, here's what happens behind the scenes:

They're directed to the Identity Center login page (if not already authenticated)
After authentication, Identity Center automatically:
- Federates them into the specified AWS account
- Assumes the designated IAM role
- Redirects to the exact resource URL

This eliminates all manual navigation steps, saving critical time during incidents.

Finding Your Identity Center Values

To create shortcut links that work with your AWS Identity Center setup, you'll need two key values:

SSO Start URL (ssoStartUrl): The entry point to your AWS Identity Center portal.
- Log in to the AWS Management Console
- Go to the AWS IAM Identity Center service
- In the "Dashboard" section, find the "AWS access portal URL" - this is your ssoStartUrl
- It typically looks like https://d-xxxxxxxxxx.awsapps.com/start
SSO Role Name (ssoRoleName): The permission set name that users will assume in the target AWS account.
- In the IAM Identity Center console, go to "AWS accounts"
- Select a permission set that your users have been assigned
- The name of this permission set (e.g., "AWSAdministratorAccess", "ReadOnlyAccess") is your ssoRoleName
- This role must have sufficient permissions to view the resources you're linking to

These values will be passed to our Lambda function as environment variables, making them easy to update if your Identity Center configuration changes.

Setting Up the Infrastructure with AWS CDK

Let's build this solution using AWS CDK. We'll create these components:

A Discord webhook integration for receiving alert notifications
An SNS topic and Lambda function to process alarms and send shortcut links
CloudWatch alarms configured to publish to our SNS topic

Let's break down each component:

1. Setting Up Discord Webhooks

First, set up Discord to receive our enhanced notifications via a Webhook:

Open Discord and navigate to your server
Click on the server name and select Server Settings
Select Integrations → Webhooks
Click New Webhook, then customize the name and destination channel
Copy the webhook URL (we'll store this securely in AWS Secrets Manager)

Note that webhook URLs should be treated as secrets since they allow posting to your Discord channel.

2. Building the CDK Infrastructure

Let's build our solution with CDK by creating a MonitoringStack that contains our Discord webhook secret, SNS topic, Lambda function, and CloudWatch alarm for monitoring DynamoDB throttling:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subs from 'aws-cdk-lib/aws-sns-subscriptions';
import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import { SnsAction } from 'aws-cdk-lib/aws-cloudwatch-actions';

interface MonitoringStackProps extends cdk.StackProps {
  tableName: string;
  ssoStartUrl: string;
  ssoRoleName: string;
}

export class MonitoringStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MonitoringStackProps) {
    super(scope, id, props);

    // Store Discord webhook URL securely in Secrets Manager
    const discordWebhookSecret = new secretsmanager.Secret(this, 'DiscordWebhookSecret', {
      secretName: 'discord/webhook',
    });

    // Create SNS topic that will receive all our CloudWatch alarms
    const alarmTopic = new sns.Topic(this, 'AlarmTopic', {
      topicName: 'cloud-watch-alarms',
    });

    // Create Lambda function to process notifications and create shortcut links
    const notifyLambda = new lambda.Function(this, 'NotifyLambda', {
      runtime: lambda.Runtime.NODEJS_22_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('lambda'), 
      environment: {
        // Pass the secret ARN and SSO values to the Lambda
        DISCORD_WEBHOOK_SECRET_ARN: discordWebhookSecret.secretArn,
        SSO_START_URL: props.ssoStartUrl,
        SSO_ROLE_NAME: props.ssoRoleName,
      },
    });

    // Grant the Lambda permission to read the webhook secret
    discordWebhookSecret.grantRead(notifyLambda);

    // Subscribe the Lambda to the SNS topic
    alarmTopic.addSubscription(new subs.LambdaSubscription(notifyLambda));

    // As an example, monitor one of your DynamoDB tables
    const table = dynamodb.Table.fromTableName(
      this,
      'MonitoredTable',
      props.tableName
    );

    // Set an alarm on the ReadThrottleEvents metric
    const readThrottleAlarm = new cloudwatch.Alarm(this, 'TableReadThrottleAlarm', {
      metric: new cloudwatch.Metric({
        namespace: 'AWS/DynamoDB',
        metricName: 'ReadThrottleEvents',
        dimensionsMap: {
          TableName: props.tableName,
        },
        statistic: 'Sum',
      }),
      threshold: 1, // Alarm on a single throttle event
      evaluationPeriods: 1, // In a single period
      alarmName: `${props.tableName}-ReadThrottleAlarm`,
    });

    // When the alarm triggers, publish a message to the SNS topic
    readThrottleAlarm.addAlarmAction(new SnsAction(alarmTopic));
  }
}

This CDK application:

Creates the notification infrastructure:
- Securely stores the Discord webhook URL in AWS Secrets Manager
- Sets up an SNS topic as the collection point for all alarms
- Deploys a Lambda function that transforms alarms into shortcut-linked notifications
- Passes Identity Center configuration to the Lambda as environment variables
Sets up DynamoDB monitoring:
- References an existing DynamoDB table
- Creates an alarm for read throttling events
- Routes the alarm to our SNS topic

Here's how to deploy this stack with your specific SSO values:

// In your bin/<app-name>.ts file or equivalent
const app = new cdk.App();
new MonitoringStack(app, 'MonitoringStack', {
  tableName: 'your-dynamodb-table-name',
  ssoStartUrl: 'https://d-xxxxxxxxxx.awsapps.com/start',
  ssoRoleName: 'AWSAdministratorAccess', // Or another permission set name
});

After deployment, add your Discord Webhook URL as the secret value in AWS Secrets Manager.

The Lambda Function: Creating Shortcut Links

Now for the core of our solution—the Lambda function that processes CloudWatch alarm notifications and creates shortcut links. Note that you can spend time customizing how the alert renders in Discord to meet your specific needs. Choose your own colors, emojis, layout, and more by customizing the Lambda function!

import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";

// Emoji for Discord message based on alarm state
const getStateDiscordEmoji = (state) => {
  if (state === 'OK') return ':green_square:';
  if (state === 'ALARM') return ':red_square:';
  if (state === 'INSUFFICIENT_DATA') return ':yellow_square:';
  return ':grey_question:';
};

// Color for Discord message based on alarm state
const getStateDiscordColor = (state) => {
  if (state === 'OK') return 65280;
  if (state === 'ALARM') return 16711680;
  if (state === 'INSUFFICIENT_DATA') return 16776960;
  return 13421772;
};

// Helper to extract parts of the alarm ARN
const parseAlarmArn = (alarmArn) => {
  const match = alarmArn.match(/^arn:aws:cloudwatch:(.+):(.+):alarm:(.+)$/);

  if (!match) {
    throw new Error(`Invalid alarm ARN: ${alarmArn}`);
  }

  return {
    region: match[1],
    accountId: match[2],
    alarmName: match[3],
  };
};

// Helper functions for building properly formatted alarm URL
const getAlarmUrl = (alarmName, region) => {
  const alarmUrl = new URL('https://console.aws.amazon.com/cloudwatch/home');
  alarmUrl.searchParams.set('region', region);
  alarmUrl.hash = `alarmsV2:alarm/${encodeURIComponent(alarmName).replaceAll('%', '$')}`;
  return alarmUrl.href;
};

// Helper function for building properly formatted SSO shortcut URL
const getSsoShortcutUrl = (accountId, roleName, destinationUrl) => {
  const hashParams = new URLSearchParams({
    account_id: accountId,
    role_name: roleName,
    destination: destinationUrl,
  });

  const ssoShortcutUrl = new URL(`${process.env.SSO_START_URL}/`);
  ssoShortcutUrl.hash = `/console?${hashParams}`;
  return ssoShortcutUrl.href;
};

// Initialize the client outside the handler for better performance
const secretsManagerClient = new SecretsManagerClient();

export const handler = async (event) => {
  // Parse the SNS message from CloudWatch
  const {
    AWSAccountId,
    AlarmArn,
    AlarmName,
    NewStateReason,
    NewStateValue,
    StateChangeTime,
    Trigger,
  } = JSON.parse(event.Records[0].Sns.Message);
  const { region } = parseAlarmArn(AlarmArn);
  const stateEmoji = getStateDiscordEmoji(NewStateValue);
  const stateColor = getStateDiscordColor(NewStateValue);

  // Build the direct link to this alarm in CloudWatch
  const alarmUrl = getAlarmUrl(AlarmName, region);

  // Create the Identity Center shortcut link
  const ssoRoleName = process.env.SSO_ROLE_NAME;
  const alarmSsoShortcutUrl = getSsoShortcutUrl(AWSAccountId, ssoRoleName, alarmUrl);

  // Retrieve the Discord webhook URL from Secrets Manager
  const { SecretString: webhookUrl } = await secretsManagerClient.send(
    new GetSecretValueCommand({
      SecretId: process.env.DISCORD_WEBHOOK_SECRET_ARN
    })
  );

  const title = `${stateEmoji} **${NewStateValue}**`;
  const description = `Alarm **${AlarmName}** entered state **${NewStateValue}** due to "_${NewStateReason}_"`;

  // Create a formatted Discord message with the shortcut link
  const message = {
    content: null,
    embeds: [{
      title,
      description,
      url: alarmSsoShortcutUrl,
      color: stateColor,
      fields: [
        {
          name: 'Account',
          value: AWSAccountId,
          inline: true
        },
        {
          name: 'Region',
          value: region,
          inline: true
        },
        {
          name: 'Time',
          value: StateChangeTime,
          inline: true
        },
        {
          name: 'Alarm',
          value: AlarmName,
          inline: true
        },
        {
          name: 'State',
          value: NewStateValue,
          inline: true
        },
        {
          name: 'Reason',
          value: NewStateReason,
          inline: true
        },
        {
          name: 'View Alarm',
          value: `[Open Alarm in CloudWatch Console](${alarmSsoShortcutUrl})`
        },
      ],
    }],
  };

  // Send the notification to Discord
  await fetch(webhookUrl, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(message)
  });
};

How It All Works Together

Here's how these components create a seamless incident response workflow:

Alert Triggering: A CloudWatch alarm changes state and publishes a notification to the SNS topic.
Link Creation: The Lambda function:
- Extracts essential information from the alert
- Creates a direct link to the specific resource
- Wraps it in an Identity Center authentication link
- Formats a user-friendly Discord message
Engineer Response: When an engineer receives the notification:
- They click the shortcut link
- If needed, they authenticate with AWS Identity Center
- They're automatically redirected to the exact resource
- They can immediately begin troubleshooting

When an alert comes in, engineers see a nicely formatted Discord message and can access the resource within seconds—dramatically faster than the traditional multi-step navigation process:

Extending the Solution

This approach isn't limited to CloudWatch alarms. You can adapt the URL-building logic above to create shortcut links to any AWS resource:

EC2 instances: Link directly to specific instances having issues
Lambda functions: Jump to logs or metrics for failing functions
DynamoDB tables: Link to tables with throttling or capacity issues

You can also integrate with other notification channels like Slack, Microsoft Teams, or PagerDuty.

Security Best Practices

When implementing this solution, adhere to these security considerations:

Least Privilege: Configure the Lambda function with only the permissions needed to read from Secrets Manager and receive SNS messages.
Secret Management: Store sensitive information like webhook URLs in AWS Secrets Manager.
IAM Roles: Ensure the role specified in your SSO shortcut links has appropriate permissions for your team members.

Conclusion

By combining CloudWatch alarms with AWS Identity Center shortcut linking, you create a powerful incident response tool that:

Dramatically reduces time to resolution
Eliminates navigation friction during critical incidents
Provides engineers with context-rich notifications
Works seamlessly across multiple AWS accounts

This approach not only improves your team's efficiency but also reduces the stress of on-call rotations by streamlining the response process.

The next time an alarm wakes someone up at 3 AM, they'll be one click away from the exact resource they need to fix—and that makes all the difference.

Happy building!

Like this post?

Follow me on Twitter where I tweet about frontend things: @thesnups

Objects as Keys: A JavaScript Adventure

Daniel Worsnup — Sat, 19 Oct 2019 03:09:21 +0000

Cross-posted from my website's blog.

Let's dive right into some code:

const key1 = {}
const key2 = {}

const obj = {
  [key1]: 1,
  [key2]: 2
}

console.log(obj[key1], obj[key2]) // Prints 2, 2 instead of 1, 2

In this snippet we're using two objects (key1 and key2) as keys in another object (obj). It doesn't quite work as expected, and the reason for the actual behavior is simple: objects do not uniquely identify themselves when used as object keys. We'll dig into the details of this in a moment.

In JavaScript, objects were never intended to operate as keys in other objects. It isn't the way the language is designed, and it's impossible to use them this way out-of-the-box, as demonstrated by the previous code snippet. In the event that we do need this type of behavior, we can leverage a Map and be done with it:

const key1 = {}
const key2 = {}

const map = new Map()
map.set(key1, 1)
map.set(key2, 2)

console.log(map.get(key1), map.get(key2)) // Prints 1, 2

You're probably thinking, "Right. So why is the topic even open for discussion?" I'm glad you asked!

Exploring alternate, unorthodox solutions to problems, even when they involve some practices that are not recommended for production code, can lead to unexpected learning and insight. There is a time and place for asking the questions "What if?" and "If so, how?" This is why we are here. What if we could use objects as keys? How might it work?

In this post we will dig into this idea of using objects as keys without using a Map or Set. We will start with a rough working solution, identify issues, and iterate toward a robust implementation that covers a wide range of use cases. Along the way we will stumble upon and discuss a number of lesser-known language features, including automatic type coercion, prototype chains, property descriptors, and symbols.

If you're ready to learn more about JavaScript, let's get started!

Object Properties

Perhaps the simplest thing you can do to an object is to give it a property with some value. As with anything in software development, there are a number of ways do so. You can declare initial properties when you create an object:

const obj = {
  awesomenessLevel: 9001
}

Or, you can intialize properties after object creation using the assignment operator:

const obj = {}
obj.awesomenessLevel = 9001
// or
obj['awesomenessLevel'] = 9001

And a third way would be to call Object.defineProperty or Reflect.defineProperty, passing the object, a property name, and a property descriptor:

const obj = {}
Reflect.defineProperty(obj, 'awesomenessLevel', { value: 9001 })

In all of these cases, we would say that the string 'awesomenessLevel' is the property key and the number 9001 is the property value.

Note: The property created by the property descriptor above differs in a few significant ways from properties created by assignment and object instantiation. More on this later.

Key Types and Automatic Coercion

While a property's value can be any type, its key must be one of only two types: a string or a symbol. When using any other key type, the JavaScript runtime will first try to coerce, or force, the key to a string before using it as a property key:

const obj = {}
obj[1] = true
key = Object.keys(obj)[0]

console.log(key, typeof key) // '1', 'string'

As you can see, when we use the number 1 as a key, it gets coerced to the string '1' before being set as a property key.

When a key is not a string and cannot be coerced to a string, the JS runtime will throw a good ole TypeError your way. In practice, however, this is actually pretty rare because almost every value in JavaScript can be coerced to a string. Consider objects, for example.

By default, an object's prototype points to the global Object.prototype, which has a generic toString implementation. When trying to coerce an object to a string, the JS runtime will check the object and its prototype chain for toString functions. It will ultimately land on Object.prototype.toString if it doesn't find one any earlier in the prototype chain:

const key = {}
Object.getPrototypeOf(key) === Object.prototype // true
key.toString === Object.prototype.toString // true

console.log(key.toString()) // '[object Object]'

const obj = {
  [key]: true
}

console.log(obj) // { '[object Object]': true }

Object.prototype.toString returns the string '[object Object]', meaning that by default all objects are coerced to '[object Object]' when a string representation is needed. Unfortunately, this coercion behavior isn't very helpful when using objects as keys because it results in all objects having the same key! As a result, each time we assign a value to an object key it will overwrite the previous value, and only the most recent value will actually be stored:

const key1 = {}
const key2 = {}
const key3 = {}
const obj = {
  [key1]: 1,
  [key2]: 2,
  [key3]: 3
}

console.log(obj) // { '[object Object]': 3 }

This explains why the first code snippet did not work as expected.

Overriding the Default Coercion Behavior

Because the JS runtime coerces objects to strings when they are used as keys in other objects, we need every unique object to be coerced to a unique string (instead of being coerced to the default '[object Object]'). In order to accomplish this, we need a way to alter an object's default coercion behavior. There are a number of ways to do so, for example:

We can create the object without a prototype, thereby severing the inheritance relationship with Object.prototype and removing the default toString implementation. As a result, the JS runtime's ability to coerce the object to a string is taken away, and we get a TypeError when using the object as a key:
```
const key = Object.create(null)
key.toString // undefined

const obj = {
  [key]: true // TypeError
}
```

We can change the object's prototype to one with a different toString implementation:

const myAmazingPrototype = {
  toString() { return 'hello' }
}

const key = {}
Object.setPrototypeOf(key, myAmazingPrototype)

const obj = {
  [key]: true
}

console.log(obj) // { 'hello': true }

We could have also used Object.create here:

const key = Object.create(myAmazingPrototype)

Notice that the object continues to inherit other default object properties from Object.prototype because it's still connected through myAmazingPrototype's prototype. We've simply added one more level to the chain of prototypes:

Object.getPrototypeOf(key) === Object.prototype // false
Object.getPrototypeOf(Object.getPrototypeOf(key)) === Object.prototype // true
key.valueOf // ƒ valueOf() { [native code] }

We can define toString directly on the object. The object's own implementation of toString takes priority over that of Object.prototype because the JS runtime finds it first when searching the object and its prototype chain:
```
const key = {
  toString() { return 'hello' }
}
const obj = {
  [key]: true
}

console.log(obj) // { 'hello': true }
```
We can define Symbol.toPrimitive on the object. This symbol enables us to completely bypass the JS runtime's default algorithm for coercing an object to a primitive. This default algorithm searches the object and its prototype chain for toString or valueOf functions and throws a TypeError when neither can be found.
```
const key = {
  [Symbol.toPrimitive]() { return 'hello' }
}
const obj = {
  [key]: true
}

console.log(obj) // { 'hello': true }
```

All of these approaches enable us to override object-to-primitive coercion behavior on individual objects, but we still don't quite have what we need.

Overriding Coercion For All Objects

Instead of overriding the behavior on individual objects, we want all objects to inherit the overridden behavior by default. We can then create objects with object literal syntax and use those objects as keys without having to make changes to the object or its prototype chain. To that end, let's define Symbol.toPrimitive on Object.prototype:

Object.prototype[Symbol.toPrimitive] = () => 'hello'
const key = {}
const obj = {
  [key]: true
}

console.log(obj) // { 'hello': true }

As you can see, the key object was coerced to 'hello' without us having to do anything special to it.

Note: In the introduction to this post, I mentioned that our solution would incorporate practices that are not recommended for production code, and I was referring specifically to this. I don't recommend making changes to Object.prototype in real-world applications except when polyfilling standard features. Recall that we embarked on this adventure with the primary goal of learning more about JavaScript.

The next step is the fun part!

Generating Unique IDs

Our Symbol.toPrimitive function needs to return a unique string for each unique object. In order to accomplish this, we need some way to generate a unique identifier for every object. Let's call this identifier the object ID. The question is, how do we get such an ID?

When I first tried to solve this problem, my initial thought was that these IDs could be derived simply by "stringifying" the objects:

Object.prototype[Symbol.toPrimitive] = function() {
  return JSON.stringify(this)
}

This solution indeed works for some use cases:

const key1 = { a: 1 }
const key2 = { b: 2 }

const obj = {
  [key1]: 1,
  [key2]: 2
}

console.log(obj[key1], obj[key2]) // 1, 2

But it has the following major limitations:

Objects that contain identical key/value properties produce the same ID because they stringify to the same string. Hence we don't have guaranteed unique keys.
An object's ID will change over time as its keys and values change. As a result, every object would have to be treated as immutable.
Objects that contain functions or circular references cannot be used as keys because these objects cannot be stringified.
Stringification becomes more expensive as the size of an object grows. There could be use cases where JSON.stringify is called repeatedly because objects are being coerced to strings repeatedly.

We need something much better. Instead of trying to derive the ID from an object's contents, we can assign an ID to an object the first time that Symbol.toPrimitive is called on it, and we can store this ID on the object itself so that it can be recalled and returned for future coercions.

Let's start by assigning the same ID to every object that Symbol.toPrimitive is called on:

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    this.id = 'hello'
    return this.id
  }
}

Notice a few things about these changes:

Our function is a regular function expression rather than an arrow function. This is extremely important because of how arrow functions affect the this context of the function. We need this to refer to the object that Symbol.toPrimitive was called on, but using an arrow function causes this to refer to whatever this refers to outside of the function. In fancy terms, this would get inherited from the surrounding lexical context.
We've added a check around Symbol.toPrimitive's hint parameter. Because the JS runtime uses Symbol.toPrimitive for more than just string coercion, hint can be any of the values 'string', 'number', and 'default'. For our use case, we only need to handle the string case.

Let's see what happens when our new function is invoked:

const obj = {}
console.log(obj) // {}
console.log(`${obj}`) // 'hello'
console.log(obj) // { 'id': 'hello' }

It works, but we're going to want to find a better way to associate the ID with the object. We'll look at why and how in a moment. First, let's start assigning unique IDs!

We can use an integer for the object ID and use a global variable to track what the next object ID will be. Each time we assign an object ID, we increment the "global ID counter", which we'll name nextObjectId:

let nextObjectId = 0

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (this.id === undefined) {
      this.id = nextObjectId++ // assign first, then increment
    }
    return this.id
  }
}

The if(this.id === undefined) { ... } conditional ensures that we only assign an ID (and increment the global counter) the first time that Symbol.toPrimitive is called on the object. If an ID is already assigned, we skip this step and immediately return the ID.

Let's see how things are looking by running our first code snippet again:

const key1 = {}
const key2 = {}
const obj = {
  [key1]: 1,
  [key2]: 2
}
console.log(obj[key1], obj[key2]) // 1, 2

It works! Even with this simple Symbol.toPrimitive function we are already able to use objects as keys. However, there are still a few significant improvements to be made. Let's take a closer look at key1 and key2 after the previous snippet ran:

console.log(key1) // { 'id': 2 }
console.log(key2) // { 'id': 3 }

In my console the assigned IDs ended up being 2 and 3. Your results may differ, but the important detail is that they should be two unique, consecutive integers.

We'll make three improvements before calling it a day.

Hiding the Object ID From Enumeration

First, it's not ideal that an object's ID is stored as a normal property on the object. The ID will show up when enumerating the object's keys and will, for instance, get copied to another object when spreading:

const anotherKey = { ...key1 }
console.log(anotherKey) // { 'id': 2 }

We're now in a situation where two objects have the same ID. According to our Symbol.toPrimitive implementation, key1 and anotherKey coerce to the same string, and thus we no longer have a unique ID for each object:

obj[anotherKey] = 3
console.log(obj[key1]) // should be 1, but we get 3

To fix this, we need Symbol.toPrimitive to associate the ID with the object in a way that is as "invisible" as possible. The ID shouldn't show up during key enumeration (i.e. Object.keys) or when shallow cloning an object using Object.assign or spread syntax. We could make the ID completely invisible by storing it in a Map or WeakMap:

const objectIdsMap = new WeakMap()

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (!objectIdsMap.has(this)) {
      objectIdsMap.set(this, nextObjectId++)
    }
    return objectIdsMap.get(this)
  }
}

However, the whole point of this post is to mimic the behavior of Maps and Sets without using Maps and Sets. And so we resort to the next best thing: property descriptors! Let's use Reflect.defineProperty to create an ID property that doesn't show up during enumeration:

let nextObjectId = 0

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (this.id === undefined) {
      Reflect.defineProperty(this, 'id', {
        value: nextObjectId++,
        enumerable: false // this is the default
      })
    }
    return this.id
  }
}

We actually don't have to specify enumerable: false in the descriptor object because false is the default value for enumerable. We'll leave this part out of upcoming code snippets.

Now that our ID is non-enumerable, it's much more invisible than it was before and won't get copied to other objects:

const key = {}
`${key}` // 2
Object.keys(key) // []

const anotherKey = { ...key }
`${anotherKey}` // 3
Object.keys(anotherKey) // []

There are still ways to see the ID property, and without using a Map/WeakMap we will not be able to get around this:

When you inspect an object in the browser's developer tools, you'll see the non-enumerable properties along with an indicator to distinguish them from enumerable properties. In Chrome, for example, non-enumerable properties are slightly faded out. These features are useful for debugging code that utilizes non-enumerable properties.
The Object.getOwnPropertyNames function returns all of an object's own property names, including those that are non-enumerable:
```
Object.getOwnPropertyNames(key) // ['id']
```

Nevertheless, we've hidden the ID property well enough for most use cases. Let's move on to the second improvement.

Preventing Collisions With the Hidden ID Property

There is another problem with the ID property that we need to fix: we've made it impossible for any other code to utilize an id property on objects that are used as keys. There are many use cases for having a property named id on an object, and we need to support them. Consider the following code:

const key = {}
`${key}` // 2
key.id = 'anotherId'
console.log(key.id) // 2

The assignment of a new value to the id property failed, and what's worse is that no error was thrown to indicate that it failed! If you aren't careful, behavior like this can result in bugs that are very difficult to troubleshoot. Let's examine why the assignment failed. Recall the property descriptor that our Symbol.toPrimitive function uses:

Reflect.defineProperty(this, 'id', {
  value: nextObjectId++,
})

Because we did not include writable in the property descriptor, it defaulted to false, meaning that the id property cannot be changed with an assignment operator. There are two ways to detect when property assignments fails:

In strict mode, assigning to a non-writable property throws a TypeError:

(() => {
  'use strict'
  var key = {}
  Reflect.defineProperty(key, 'id', { value: 2 })
  key.id = 'anotherId' // TypeError
})()

Reflect.set will return false when a property assignment fails:
```
Reflect.set(key, 'id', 'anotherId') // false
```

We could make id a writable property, but this not ideal because it enables other code to potentially change an ID assigned by Symbol.toPrimitive. As a result, we lose the guarantee that object IDs are unique and stable.

The issue we're facing isn't that other code cannot assign to the id property—it's that our object keying system is utilizing a property named id in the first place. We need to prevent collisions with our ID property and enable other code to use any property name without conflicting with the object ID. We have several options for doing so:

Use a more obscure property name for the object ID. This isn't a terrible idea. We could minimize the risk of collisions significantly by using a name like __object_id__ab8dfbbd4bed__, where we've even included some random characters in the property name. But let's see if we can do even better!
Use a symbol for the ID property. Because symbols are guaranteed to be unique, this is the perfect use case for one.

Here's the updated code, adjusted to use a symbol for the object ID:

const ID_SYMBOL = Symbol()
let nextObjectId = 0

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (this[ID_SYMBOL] === undefined) {
      Reflect.defineProperty(this, ID_SYMBOL, {
        value: nextObjectId++,
      })
    }
    return this[ID_SYMBOL]
  }
}

With this change, other parts of the codebase and even other libraries are free to use any object property name without risking a collision with our ID property. Also, the ID property will now be returned from Object.getOwnPropertySymbols instead of Object.getOwnPropertyNames:

Object.getOwnPropertyNames(key) // []
Object.getOwnPropertySymbols(key) // [Symbol()]

Giving our symbol a "description" can help with debugging without impacting the behavior or uniqueness of the symbol:

const ID_SYMBOL = Symbol('Object ID')

// ...

Object.getOwnPropertySymbols(key) // [Symbol(Object ID)]

Now that our object ID property is safe from being seen or altered by other code, let's move on to the third improvement.

Mitigating Collisions With Non-Object Keys

The IDs produced by Symbol.toPrimitive are guaranteed to uniquely identify an object as a key as long as only objects are used as keys. However, there may be use cases that require an object to contain both object and non-object keys. Consider the following code:

const key1 = {}
const key2 = {}
const key3 = 2

const obj = {
  [key1]: 1,
  [key2]: 2,
  [key3]: 3
}

console.log(obj[key1], obj[key2], obj[key3]) // Prints 3, 2, 3 instead of 1, 2, 3

The code doesn't behave as expected because key1 and key3 coerce to the same string, '2'. To fix this, objects need to coerce to keys that are globally unique across all data types, instead of being unique only among other objects. Let's look at two solutions.

Object ID Prefix

We can make our object IDs more globally unique by prefixing them with an obscure string, such as '__object_id__' or '__object_id__ab8dfbbd4bed__':

const ID_SYMBOL = Symbol('Object ID')
const ID_PREFIX = `__object_id__`
let nextObjectId = 0

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (this[ID_SYMBOL] === undefined) {
      Reflect.defineProperty(this, ID_SYMBOL, {
        value: `${ID_PREFIX}${nextObjectId++}`
      })
    }
    return this[ID_SYMBOL]
  }
}

Running the last code snippet now produces the expected results because key1 now coerces to '__object_id__2' and no longer conflicts with key3, which coerces to '2'.

The ID prefix is a solid solution and suffices for most use cases. However, while this solution significantly reduces the probability of collisions, it doesn't eliminate the issue entirely. Some of you may already know where this is going!

Object ID Symbols

Instead of using a global ID counter (nextObjectId), each object's ID can be its own unique symbol:

const ID_SYMBOL = Symbol('Object ID')

Object.prototype[Symbol.toPrimitive] = function(hint) {
  if (hint === 'string') {
    if (this[ID_SYMBOL] === undefined) {
      Reflect.defineProperty(this, ID_SYMBOL, {
        value: Symbol()
      })
    }
    return this[ID_SYMBOL]
  }
}

By using symbols, we have handed off to the browser the responsibility of creating unique keys. These keys will be unique across the space of all primitives coerced from JavaScript values.

Unfortunately, there is one major caveat to this approach: property symbols are not included in the return value of Object.keys:

const key1 = {}
const key2 = {}

const obj = {
  [key1]: 1,
  [key2]: 2
}

console.log(obj[key1], obj[key2]) // 1, 2 👍
console.log(Object.keys(obj)) // [] 👎

For this reason, the ID prefix approach may be superior.

And that's all! It took us a number of iterations to get here, but we landed on a simple Symbol.toPrimitive function that, when applied to Object.prototype, enables the use of objects as keys.

The Journey Is More Important Than The Destination

We can learn a lot from delving into an unconventional approach to solving a problem. Even though the final code for our object keying system shouldn't be used in production, I hope you learned from the journey we took to build it.

We discussed a number of ideas and language features. We saw problems with our solution and made improvements until we ended up with something robust and functional. Let's revisit the highlights:

Object keys must be strings or symbols. Keys of any other type will be coerced to an allowable type, and an error will be thrown if this fails.
We can use Symbol.toPrimitive to override the default algorithm for coercing an object to a primitive, and we can apply this symbol to Object.prototype to declare the override for all objects.
We can use Reflect.defineProperty or Object.defineProperty to define non-enumerable properties on an object. These properties won't be returned from Object.keys or get copied when using Object.assign/spread. Property descriptors enable a number of other behaviors as well. See the MDN docs for details.
When we need truly unique property keys with a zero chance of collisions, symbols are the way to go! But keep in mind that they are not included in the return value of Object.keys.

One Last Thing

The polyfill library core-js is commonly used when developing for browsers that do not natively support certain standard JavaScript features. Not surprisingly, it contains polyfills for Map and Set. Our final approach for enabling objects as keys is strongly based on the implementation of these polyfills. Specifically, core-js utilizes property descriptors and non-enumerable properties to store an object's ID on the object itself. Further, the IDs are generated by an incrementing counter variable!

That's All!

We covered a lot of ground in this post. If you made it all the way through, thanks for reading! I sincerely hope you learned a thing or two about JavaScript that you didn't know before.

Happy coding!

Like this post?

Follow me on Twitter where I tweet about frontend things: @thesnups

How to Build a Simple Markdown Plugin for Your Gatsby Site

Daniel Worsnup — Mon, 30 Sep 2019 03:07:05 +0000

Cross-posted from my website's blog.

The majority of software development boils down to automating tasks and processes that would otherwise consume valuable time, require manual effort, and be prone to accidental errors. Whenever you find yourself repeating some task or process repeatedly, there are a few questions you should immediately start asking yourself:

Can this be automated?
Is it worth the time to automate this?
Is it worth the financial investment (if any) to automate this?

Most of the time, the answer to all of the above will be an astounding yes. There's rarely a reason to waste time doing something that a computer can do for you much more quickly and with a much lower risk of error. Let's consider a few examples of common automations that many of us rely on today:

JavaScript transpilers, such as Babel and TypeScript. These enable us to write modern JavaScript code with bleeding edge language features and maintain confidence that our code will run properly in a broad range of browsers and browser versions.
Development tools, such as IDEs, IntelliSense, bundlers, debuggers, linters, and formatters. This category speaks for itself.
Automated testing, which enables us to deploy our changes with confidence that core user flows have not been impacted.
Continuous integration and delivery, which enable us build, test, and deploy code to production at the push of a single button.

Because we've become so accustomed to the value of automations such as these, it can be hard to imagine our lives without them. But this is how it used to be! Let's try not to take these things for granted, and instead remind ourselves that we are fortunate to be develop software in today's technological ecosystem.

With all of this in mind, it's important to note that there are legitimately good reasons for not building an automation, such as when:

You have higher priority work that requires more immediate attention, such as critical bug fixes or an imminent feature work deadline.
You aren't sure yet if the task or process will be needed long-term and need more time to verify that it will be.

Let me know in the comments or on Twitter of other examples of powerful automations that we tend to take for granted, or other good reasons for postponing an automation!

The remainder of this post focuses on a simple automation I built for my Gatsby blog. We'll take a look at why I built it, what it does, and how it works!

Cross-Posting Blogs

If you are a blogger then you are more than familiar with the struggle for visibility. You want people to read your work (why else would you have created it?), but there are a lot of other bloggers out there, some of which have the advantage of being sponsored by well-known, respected organizations. Though our motives may differ, we all share a common desire to grow a readership, and so we are forced to figure out how to survive in a competitive market.

Making your content accessible in more places than just your personal website is an easy way to increase your visibility, and you can do so by cross-posting to other blogging platforms, such as DEV and Medium. These platforms let you categorize and tag your content and will automatically point interested readers in your direction. On Medium you can even get paid!

Note: When cross-posting, be sure use canonical URLs to point back to the original content on your website so that search engines don't penalize you for duplicate content.

If your personal blog is built with Gatsby's blog starter (as mine is), you can cross-post to DEV extremely easily. Both platforms are powered by Markdown (for post content) and Front Matter (for post metadata), and although there are a few adjustments necessary, it's about as close to 1:1 as it gets. One notable difference, however, is the syntax each platform uses for third-party video embeds.

Embedded Videos

Adding videos to your content can be extremely useful for conveying information that is hard to capture in another form, such as text or images. Here's a somewhat meta video demonstrating how it looks in one of my blog posts:

Out-of-the-box, Gatsby's blog starter does not support video embeds, but it's easy to add support by installing the gatsby-remark-embed-video plugin. With this plugin, you can embed videos into your posts using the following syntax:

# An Awesome Video

Check out this awesome video:

`youtube: 12345abcde`

This will embed the Youtube video with ID 12345abcde. On DEV, however, embedding the same Youtube video is done like this:

# An Awesome Video

Check out this awesome video:

{% youtube 12345abcde %}

This is because DEV's third party embed syntax is based on Liquid's templating language, which DEV also supports in their Markdown.

A Few Solutions

As with any problem, there are multiple approaches we could take to solve this issue. Two main ideas came to my mind:

Write all video embeds using gatsby-remark-embed-video syntax. Before cross-posting, go through and update all video embeds to use DEV syntax. These updates could be made manually, but it would be better to automate this with a Regex find/replace, which would mitigate the risk of errors.
Write all video embeds using DEV syntax and figure out how to support this syntax in a Gatsby blog.

Option 2 is better for a few reasons:

The embed syntax becomes consistent across both platforms.
No extra update step is needed when preparing a blog post for cross-posting, which both saves time and prevents errors in the future.
I get to learn how to write a Gatsby markdown plugin!

This brings us to the meat of the post: building a custom plugin that lets us embed Youtube videos using DEV's embed syntax. Before diving into the implementation, let's first briefly look at how Gatsby works with your markdown source files.

Gatsby and Markdown

Thanks to Gatsby's flexible plugin architecture, populating a blog from markdown source files is a breeze. For a detailed tutorial on how to do this, check out Creating a Blog with Gatsby. There are a few core plugins involved, and the remainder of this post assumes that these plugins are installed and configured:

gatsby-source-filesystem - Reads markdown files from the filesystem and produces one Markdown node for each
gatsby-transformer-remark - Transforms these Markdown nodes into HTML that is ready to be rendered in a browser

By default, gatsby-transformer-remark's HTML output isn't much more than a 1:1 representation of the Markdown input, for example:

An h1 for each #
An li for each 1. or * in a list
A wrapping ol/ul around each set of lis

For most types of blog post content this is exactly what we want, but there are situations where the compiled HTML needs to be either 1) more sophisticated or 2) changed completely. Our Youtube video embed case is an example of the latter, but let's briefly take a look at an example of the former! Consider the following Markdown, which renders an image with some alt text:

![I'm the alt text](my-amazing-image.png)

By default, gatsby-transformer-remark will produce the following HTML output for this Markdown input:

<p>
  <img src="my-amazing-image.png" alt="I'm the alt text">
</p>

Whilst this output is completely functional, it isn't optimized for the modern web. Instead of producing a simple img tag with a single src attribute, it would be much better to produce a fully responsive image, complete with srcset and sizes that will ensure the best experience for a broad range of devices:

<p>
  <img
    srcset="my-amazing-image-320w.jpg 320w,
            my-amazing-image-480w.jpg 480w,
            my-amazing-image-800w.jpg 800w"
    sizes="(max-width: 320px) 280px,
            (max-width: 480px) 440px,
            800px"
    src="my-amazing-image-800w.jpg"
    alt="I'm the alt text">
</p>

Responsive images are far superior to simple images.

Instead of the developer deciding at implementation time which image to load, responsive images enable the browser to decide at run time.

The reason we want to hand off this decision to the browser is because the browser is in the best position to make it! The browser has the most information about the user's browsing context and can take into account any or all of the following factors:

Screen size
Device orientation
Pixel density
Current network conditions
The user's current data-saver preferences

Ultimately, the user benefits. However, getting Gatsby to render a functional responsive image requires not only an extra transformation step when compiling Markdown to HTML, but also an image processing step that produces all of the required image sizes.

How can we do this? With more plugins!

Plugins Within Plugins

Customizing the behavior of gatsby-transformer-remark requires hooking into its internals. Luckily for us, the gatsby-transformer-remark plugin itself can be customized with plugins! For example, we can easily solve the responsive image problem by leveraging the great gatsby-remark-images plugin. In addition to providing the srcset and sizes attributes and resizing the original image, it also renders an elastic container to prevent layout jumps and supports the "blur up" placeholder loading effect. Amazing!

With all of our responsive image needs not only met but exceeded, we can return our focus to the Youtube video embed problem.

How a gatsby-transformer-remark Plugin Works

Before jumping into the code for our custom plugin, we need to know a bit more about how plugins for gatsby-transformer-remark work.

Plugins written for gatsby-transformer-remark define additional transformations that should be applied to your Markdown before it gets compiled to the final HTML that is rendered on your Gatsby site.

Thankfully, we don't have to apply these transformations to raw Markdown source strings, which would be messy and unperformant.

Abstract Syntax Trees

gatsby-transformer-remark does the heavy lifting of parsing the raw Markdown source strings into Abstract Syntax Trees, or ASTs. If you aren't familiar with the concept of an AST, don't be intimidated! It's just a fancy name for a simple idea:

An Abstract Syntax Tree is a tree representation of a source code string. Each node in the tree represents a construct from the source code.

An AST begins as a 1:1 reflection of the source code string from which it was built, meaning that it could be traversed and compiled back into the original string if needed. Sometimes it's useful to operate on an unaltered AST. For example, our good friend ESLint examines your source code's unaltered AST—rather than the source code itself—for issues. Other times it's useful to have ASTs undergo mutating transformations to produce new ASTs, which are no longer equivalent to the original source. For example, many compilers will automatically optimize code by identifying and fixing parts of the code's AST that can perform more efficiently.

Our plugin is an example of scenario #2. We want to transform our Markdown ASTs in such a way that instances of the Youtube video embed string are replaced with embed HTML for the specified video.

Markdown ASTs

Internally, gatsby-transformer-remark uses the remark processor to build ASTs that comply with the MDAST spec (short for Markdown Abstract Syntax Tree). Among other things, this spec defines the various node types that can exist in a Markdown AST, such as image, text and inlineCode. Consider the following Markdown:

I'm a paragraph containing `inline code`!

The resulting MDAST tree is as follows (with some irrelevant metadata removed for brevity):

{
  "type": "root",
  "children": [
    {
      "type": "paragraph",
      "children": [
        {
          "type": "text",
          "value": "I'm a paragraph containing "
        },
        {
          "type": "inlineCode",
          "value": "inline code"
        },
        {
          "type": "text",
          "value": "!"
        },
      ]
    }
  ]
}

Notice how the nodes in the AST map directly to the constructs in the Markdown source: text -> inlineCode -> text, nested together under a paragraph.

Writing a transformer plugin for gatsby-transformer-remark boils down to traversing Markdown ASTs (such as the one above) and making changes to relevant nodes. Our Youtube video embed plugin simply needs to do the following:

Traverse the AST looking for nodes of type text
If the node's value matches the DEV video embed syntax, transform it!

Now that we have an idea of how a gatsby-transformer-remark plugin works and what ours needs to do, let's jump into the implementation!

Building the Plugin

The Gatsby docs do a great job of explaining how to create custom plugins. For simplicity, the plugin we build here will only support Youtube video embeds, but a fun open source project would be a plugin that supports all of DEV's third party embed tags (they have a lot!) and possibly even the Liquid templating language. You heard it here first!

We'll create our embed plugin as a local plugin, meaning that it is scoped to a specific Gatsby project and lives in the project's repository under the plugins directory of the project root. Create a directory for the Youtube video embed plugin:

cd path/to/gatsby/project
mkdir plugins # If it doesn't exist already
cd plugins
mkdir youtube-video-embed

The only files needed to create a plugin are package.json and index.js. Create these in the plugin directory:

cd youtube-video-embed
npm init # You can accept all of the default values
touch index.js

A plugin for gatsby-transformer-remark is simply a function that receives a Markdown AST as a parameter and alters it. Once configured, gatsby-transformer-remark will invoke this function once for each Markdown node, and recall that gatsby-source-filesystem produces a Markdown node for each Markdown source file in our project.

We'll implement our plugin function in index.js. Let's open it for editing and add the following code:

module.exports = ({ markdownAST }) => {
  console.log('video embed!', JSON.stringify(markdownAST))
}

Notice how markdownAST can be conveniently destructed from the first parameter to the plugin function. To make sure things are working, we're currently just logging the AST of each Markdown node to the build console.

Configuring the Plugin

Next, we need to configure our Gatsby project to run the new plugin, which we accomplish by listing the plugin in gatsby-config.js. Since this is a plugin for gatsby-transformer-remark—not Gatsby itself—we list it under gatsby-transformer-remark's own plugin list:

module.exports = {
  /* ... */
  plugins: [
    /* ... */
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          'youtube-video-embed'
        ],
      },
    },
    /* ... */
  ]
}

Note: Our plugin will soon render an iframe that loads the embedded video. Because of this, if your project also relies on the gatsby-remark-responsive-iframe plugin, you have to list our plugin first:

plugins: [
  'youtube-video-embed',
  'gatsby-remark-responsive-iframe'
]

With the configuration change in place, you should be able to run your Gatsby site (npm run dev) and see the AST of each Markdown node logged to the build console. If so, things are working! Now let's make the plugin do something useful.

Traversing the AST

As mentioned earlier, we need to search markdownAST for nodes of type text so that we can transform them. We could write our own loops and recursion to do this, but instead let's have the unist-util-visit library do it for us:

npm i unist-util-visit

This library exposes a visit function, which allows us traverse a Markdown AST by specifying the following:

The type of node we want to visit (text), and
A function to be called once for each node of the specified type

In index.js, import the library and call visit:

const visit = require(`unist-util-visit`)

module.exports = ({ markdownAST }) => {
  visit(markdownAST, 'text', (node) => {
    // We're at a text node!
  })
}

The next step is to check the value of each visited text node to see if it matches DEV's Youtube video embed syntax. Recall that this syntax is {% youtube 12345abcde %}, where 12345abcde is the ID of the Youtube video to embed.

Let's define a simple regular expression that matches the syntax and use it to check node.value. Each time we find a match, we log the video ID (which we can get from a match group) to the console:

const YOUTUBE_REGEX = /^{% youtube (\w+) %}$/

module.exports = ({ markdownAST }) => {
  visit(markdownAST, 'text', (node) => {
    const match = YOUTUBE_REGEX.exec(node.value)
    if (match) {
      console.log('Found one! The ID is', match[1])
    }
  })
}

Assuming that you have added some Youtube video embeds to your Markdown files, you should see the video IDs logged to the console when you run this:

Found one! The ID is zcjuXR8obvI
Found one! The ID is Q2CNno4JuJM
Found one! The ID is CpYLXl0Rm74

We're so close! We have identified the text nodes we care about, and all that remains is to transform them.

Transforming the AST

Remember earlier when we discussed mutating ASTs? That's exactly what we're going to do! The MDAST spec defines an html node type that represents raw HTML within a Markdown source file. Let's change the type of each video embed node to html and change the value to an HTML string that defines an iframe pointing to the embedded Youtube video:

module.exports = ({ markdownAST }) => {
  visit(markdownAST, 'text', (node) => {
    const match = YOUTUBE_REGEX.exec(node.value)
    if (match) {
      const videoId = match[1]

      node.type = 'html'
      node.value = `
        <iframe
          type="text/html"
          width="640"
          height="360"
          frameborder="0"
          src="https://www.youtube.com/embed/${videoId}"
        ></iframe>
      `
    }
  })
}

And that's it! If you rebuild your Gatsby site and open it in the browser, you will see embedded Youtube videos powered by the DEV embed syntax.

Possible Improvements

The final plugin implementation above is intentionally minimal in order to be as digestible as possible, but there are many improvements that could be made when integrating this into your project. Here are a few thoughts:

The HTML string only passes a few parameters to the Youtube iframe player, but there are a variety of parameters you can use to configure the embedded video to your liking.
Whilst this implementation works fine for only supporting Youtube video embeds, it doesn't scale well in its current state. To expand support to other video providers (such as Vimeo or Twitch) or other embed types (such as code snippets, music, tweets, etc), we would want to build a more generic system in which provider-specific details and behavior are abstracted. To see an example, check out the source code for gatsby-remark-embed-video, which I mentioned earlier in this post.
This implementation will transform text nodes anywhere in the Markdown AST, but we probably only want the transformation to apply to top-level paragraphs that only contain a single text node.
This implementation does not allow different video embed instances to be customized. For example, we may want some videos to autoplay and others to loop infinitely. In order to enable this, we'd want to extend the embed syntax to allow different parameters to be specified: {% youtube 12345abcde loop=true autoplay=false %}, for example.

Beyond Gatsby Plugins

One of my favorite things about being a programmer is having the ability to make my life and others' lives easier by automating processes with code. Building automations not only saves precious time, but also prevents the silly errors often caused by manual grunt work on repetitive tasks. There are few better feelings than experiencing the fruits of these efforts and then fondly thanking the you-of-the-past for anticipating their value and setting aside time to build them.

The simple Markdown transformer plugin we built is just a small example of this. With the plugin in place, we can embed Youtube videos in our Gatsby blogs and cross-post to DEV without thinking twice about whether the video embeds will work properly.

Tell me in the comments or on Twitter about an automation that you have written for yourself or have shared with the world!

Happy coding!

Thanks for reading!

I've been planning on starting a tech blog for several years now, which is a feeling that I'm sure some of you can relate to. Only recently have I started taking it seriously, and so far the reception has been very positive. I'd like to thank everyone for reading, liking, commenting, and (re)tweeting, etc. Let me know what I can do to help you on your journey!

Like this post?

Follow me on Twitter where I (re)tweet about frontend things: @thesnups

Why You Should Be Using the Web Share API in Your PWA

Daniel Worsnup — Wed, 18 Sep 2019 23:04:57 +0000

Cross-posted from my website's blog.

Progressive web apps (PWAs) are all the hype these days, and for good reason! PWAs provide significant user experience benefits over traditional web apps and web sites. If you're unfamiliar with the idea of a progressive web app, I'd recommend taking a look at this article on Google Developers.

The primary goal of PWAs is to blur the line between the UI/UX of web apps and that of native apps (apps that are built with native technologies to target specific operating systems, such as iOS and Android). One of the ways a PWA can blur this line is by providing the same, native-feeling functionality as a native app. Luckily for those of us trying to build great PWAs, browser vendors have been working hard to unlock these capabilities for us, including the ability to add to home screen, re-engage with push notifications, and even connect to a bluetooth device!

Shout out to all the great browser vendors out there for continuing to push the web platform forward!

This post will focus on a small piece of functionality that, when implemented properly, will enhance the native feel of your app and contribute to an enhanced user experience: sharing.

Sharing

In today's world, enabling your users to share your content effortlessly is more important than ever. But it doesn't only enhance their experience—when users are able to share your content easily across a wide range of platforms, the ultimate result is the increased visibility of your content. You benefit as well!

If you want to grow your audience and your reach, your content needs to be easy to share.

Traditionally, we web developers have been responsible for building custom share UIs into our web apps, either manually or by leveraging third-party libraries or services. For example, my website's blog utilizes several react-share buttons and icons for its custom share UI, as seen in the following video:

In the absence of alternatives, the custom share UI approach is great! However, it has a few major drawbacks:

We have no way of knowing which share targets an individual user needs. Different users have different (often industry/domain-specific) sharing needs, and so we are forced to guess the list of most commonly needed share targets, such as Facebook, Twitter, Reddit, etc. As a fallback we sometimes enable users to copy a URL to their clipboard.
Share UIs look different on different websites, forcing users to take a moment to understand the capabilities of the share UI on the site they are currently looking at. Furthermore, there is a frustrating disconnect for users that are accustomed to their device's native share UI.
We are limited in the number of share targets we can support. For example, we cannot create a button that enables users to share content directly to a Facebook Messenger conversation.
We have to build the UI ourselves or rely on a third-party library or service.

Is there a way we can solve all of these problems at once? There is! Please welcome to the stage the Web Share API!

The Web Share API

In 2016, the Chrome team first launched the Web Share API in Chrome 61 for Android. Since then, it has been picked up by a few more browsers (more on that to come). The Web Share API unlocks the power of the device's native sharing UI and makes it accessible to web apps.

Instead of web apps themselves being responsible for the share UI, the Web Share API enables a web app to instruct the browser to show the native share UI for a specific piece of shareable content.

This approach to showing share UIs solves all the problems mentioned above:

The browser hands off the rendering of the share UI to the operating system (OS), which is aware of all of the installed apps that can act as share targets. The OS can then prioritize certain targets over others based on the user's sharing habits, their preferences, and the specific type of content being shared. Amazing!
The user is presented with the same share UI that they are used to seeing in native contexts on their device.
All of the installed apps that can act as share targets are shown as options in the native share UI.
We don't have to build anything manually or rely on anything third-party. The browser and OS take care of everything!

Here's how it looks on my website's blog:

Browser Support

At the time of this writing, the Web Share API is implemented in a few mobile browsers but hasn't yet seen widespread adoption across both desktop and mobile. Check the Web Share API Can I Use page for up-to-date browser support information.

Because of poor browser support, it's best practice to use the Web Share API when it's available and fall back to a custom share UI when it isn't. I take this approach on my website. In the video above, notice how the native UI is triggered by the same button that would trigger the custom UI if Web Share were not supported. This approach is also taken by The Dev Community.

Because the Web Share API is so easy to use, adding it to your web app can be an extremely quick win UX-wise for users with Web Share support in their browser. Let's take a look at just how easy it is.

Using the Web Share API

Because all the hard work of building and showing the share UI has already been taken care of by the browser and OS, we have to do very little to enable our users to share content with Web Share. There are only two steps involved:

Verify that Web Share is supported by the current browser
If it is, tell it what to share!

When Web Share is supported, the browser exposes a share function on the global window.navigator object. The MDN docs do a great job of describing this function. Take a look!

All we need to do to check for Web Share is to check for the existence of this function:

if (navigator.share) {
  // Web Share is supported
} else {
  // Web Share is not supported
}

In order to mitigate abuse, browsers that support Web Share require navigator.share to be called in response to a user gesture, such as clicking a share button:

myShareButton.addEventListener('click', () => {
  if (navigator.share) {
    // We can call navigator.share here!
  }
})

// We can't call it here

The navigator.share function expects you tell it what the user is trying to share. You specify this information as an object with a few optional keys: url, text, and title. Although your requirements may be different depending on the nature of your app, most of the time you want to enable users to share the page they are currently looking at. To do this, you only need to define the url and title keys, which we have easy access to in a browser environment:

myShareButton.addEventListener('click', () => {
  if (navigator.share) {
    navigator.share({
      url: location.href,
      title: document.title
    })
  } else {
    // Show custom share UI
  }
})

Canonical URLs

If your app uses canonical URLs, you can query the document for a canonical URL and have that URL take priority over location.href:

const canonicalLink = document.querySelector('link[rel=canonical]')
const url = canonicalLink ? canonicalLink.href : location.href

navigator.share({
  url,
  title: document.title
})

Reacting to the share

If your app needs to react to the share action, navigator.share returns a promise for your convenience. The following example uses await to handle the promise, but you can just as easily chain the usual .then() and .catch() calls.

try {
  await navigator.share({ url, title })
  console.log('Thanks for sharing!')
} catch (e) {
  console.error('Could not share!', e)
}

That's pretty much all there is to it!

Summary

The Web Share API is the future of sharing on the web. Regardless of whether you consider your web app to be a PWA, it should use the Web Share API when it is available. It only takes a few lines of code to check for browser support and conditionally make a call to the API!

Beyond Web Share

It takes a lot of work to build a web app that is perceived by users as providing an equal or comparable experience to a native app. This checklist from Google enumerates an exhaustive list of PWA features, divided into baseline and exemplary feature categories.

Some of you may already be in the process of building new PWAs or migrating existing web apps to be PWAs. If that's you, don't let yourself be intimidated by the amount of work involved. Instead of focusing on the enormity of the entire task, prioritize features by anticipated impact, focus on one feature at a time, and identify and execute quick wins (such as Web Share!) in order to move your app in the right direction at an amplified rate. Don't forget to re-evaluate your priorities as you go!

Thanks for reading and happy coding!

Like this post?

Follow me on Twitter where I (re)tweet about frontend things: @thesnups

Everything You Need to Know About Array#reduce

Daniel Worsnup — Sun, 15 Sep 2019 00:45:16 +0000

Cross-posted from my website's blog.

Array#reduce, or Array.prototype.reduce (referred to simply as reduce from here on), is my favorite function in the JavaScript standard library. Although it can take some time to get used to, it's 100% worth the effort. Once the power of reduce is fully grasped, it will enable you to redefine complex logic in a declarative, readable way.

This post is divided into two main sections: 1) an explanation of what reduce is and how it works, and 2) a demonstration of some interesting applications of reduce that you may not have previously considered. If you are a seasoned veteran with reduce, the explanation section will be review. Feel free to skip to the demonstration section.

What is reduce?

Simply put, reduce is a function that lets you reduce an array down to a single value. This value, which we'll call the reduced value, can be any type you want. You will often find yourself needing to reduce an array down to one of the many JavaScript primitive types, such as object, number, boolean, or even another array (we'll see some examples of this later!), depending on the circumstances. However, you are not limited to reducing to the primitive types. The reduced value can be any type you want, such as a Map, Set, or any custom type defined by your project.

In JavaScript, a reduce function is defined on the Array prototype (Array.prototype.reduce), which means you can call it on any array instance:

const myArray = [1, 2, 3];
const reducedValue = myArray.reduce(/* args */);

How does reduce work?

The array you call reduce on describes what you want to reduce, and the parameters passed into reduce describe how you want to build the reduced value from the array. The MDN documentation for reduce does a great job of detailing the inputs and outputs of reduce. Go take a look! I'll do a high-level overview here.

Parameters

The reducer function. Don't confuse this with a state management reducer function such as those used with Redux. Though the concepts are similar, they are not the same.
The initial value for the reduce loop.

The reducer function

When you call reduce on an array, reduce will iterate over the array one element at a time, invoking the reducer function once for each element. When reduce calls your reducer function, it passes the following four parameters in:

Accumulator
Current element
Current index
Source array

Don't worry too much about the last two parameters for now. In practice, I rarely find myself needing to use them.

The accumulator (sometimes called the collector) is the value that represents the results of invoking the reducer function on each element of the array up to, but not including, the current element. It's effectively the "reduced value so far". This is the essence of the reducer function:

The reducer function describes how to combine the accumulator (the reduced value so far) with the current element of the array to produce the accumulator for the next iteration.

The initial value (reduce's second parameter) acts as the accumulator for the first invocation of the reducer function, and the value returned from the final invocation of the reducer function is the final reduced value that is ultimately returned from the reduce call.

Case study: the `sum` function

We're all familiar with the sum function. Let's take a look at a simple implementation:

function sum(numbers) {
  let sumSoFar = 0;

  for (const number of numbers) {
    sumSoFar += number;
  }

  return sumSoFar;
}

What may not be obvious about the sum function is that it is actually just a special case of reduce. The sumSoFar variable acts as the accumulator:

function sum(numbers) {
  let accumulator = 0;

  for (const number of numbers) {
    accumulator += number;
  }

  return accumulator;
}

The body of the for loop is describing how to combine the current element (number) with the current accumulator to produce the next accumulator for the next iteration. This should sound familiar! With reduce, this is the job of the reducer function:

function sum(numbers) {
  let accumulator = 0;

  for (const number of numbers) {
    accumulator = reducer(accumulator, number);
  }

  return accumulator;
}

function reducer(accumulator, currentElement) {
  return accumulator + currentElement;
}

Notice how we've created a layer of abstraction by moving the logic for computing the next accumulator into a reducer function. At this point we're very close to having an actual reduce implementation. Let's finish it up by renaming a few things and allowing the reducer function and initial value to be passed in:

function reduce(array, reducer, initialValue) {
  let accumulator = initialValue;

  for (const currentElement of array) {
    accumulator = reducer(accumulator, currentElement);
  }

  return accumulator;
}

Supporting the last two parameters of the reducer function (the array index and the array itself) is trivial. To keep track of the current array index we can switch to a standard for loop instead of a for...of:

function reduce(array, reducer, initialValue) {
  let accumulator = initialValue;

  for (let i = 0; i < array.length; ++i) {
    accumulator = reducer(accumulator, array[i], i, array);
  }

  return accumulator;
}

Last but not least, with native reduce we don't need to pass in the array because we're calling reduce on the array. For illustrative purposes it looks something like the following, but keep in mind that we wouldn't run this code in production. There usually isn't a good reason to overwrite the behavior of native JavaScript functions:

Array.prototype.reduce = function(reducer, initialValue) {
  let accumulator = initialValue;

  for (let i = 0; i < this.length; ++i) {
    accumulator = reducer(accumulator, this[i], i, this);
  }

  return accumulator;
}

Notice that when the function is defined on Array.prototype, we can refer to the array itself as this.

What are some applications of reduce?

Let's take a look at some examples of reduce functions in the wild!

Some of the upcoming examples show functions defined on Array.prototype. Please note that it is not my intent to recommend that code like this be run in production. These examples are intended to demonstrate how some of the native Array.prototype methods could be implemented. In practice, we always want to use the existing native implementations rather than overwrite them with our own.

The `sum` function

We already saw how a simple sum function can be modified slightly to become the actual reduce function, but let's revisit sum to see how it is written using reduce:

function sum(numbers) {
  return numbers.reduce((accumulator, currentElement) => {
    return accumulator + currentElement;
  }, 0);
}

Notice the initial value, 0, and how the reducer function simply adds the current element to the accumulator to produce the next accumulator. By utilizing reduce we unlock an extremely declarative way to write this sum loop.

Although accumulator and currentElement are reasonable variable names to use in the context of a reduce loop, you'll find that in practice there are usually better names that are more appropriate to the context of the code being written. For example, in the case of the sum function, the names sumSoFar and number convey more circumstantial meaning and will probably more helpful for someone else (or even you!) reading the code during a code review or in the future:

function sum(numbers) {
  return numbers.reduce((sumSoFar, number) => {
    return sumSoFar + number;
  }, 0);
}

The `map` function

The map function is an extremely useful function that should be hanging from your toolbelt for quick and easy access. If it's not, go read about Array.prototype.map on MDN.

map calls a function (the "mapper" function, defined by you) on each element of an array, and returns a new array containing the results of each function call.

Here's an example of map in action:

function addOneToEach(numbers) {
  return numbers.map((number) => number + 1);
}

addOneToEach([1, 2, 3]) // [2, 3, 4]

What most probably haven't realized about map is that is actually just a special case of reduce! Unlike sum, where we reduce an array down to a number, with map we reduce an array down to another array. Because of this, we pass empty array as the initial value. Here's what it looks like:

Array.prototype.map = function(mapperFn) {
  return this.reduce((accumulator, currentElement) => {
    const mappedCurrentElement = mapperFn(currentElement);

    return [...accumulator, mappedCurrentElement];
  }, []);
}

Notice that the only thing the reducer function needs to do is run the current element through the passed-in mapper function and then add it to the end of the accumulator, which is initialized to an empty array.

The above implementation of map will have serious performance problems as the size of the input array grows. This is because the reducer function is creating a new array on each iteration and then copying the elements of the accumulator into it before finally appending the newly mapped current value. If you do the relevant math you will discover that the time complexity of this approach (assuming the time complexity of the mapper function is constant) is on the order of O(n²).

This is bad, so let's fix it! Instead of creating a new array on each iteration, there's no reason we can't keep using the same array through the entire reduction. On each iteration, we can push the mapped current element onto the array, and return it for the next iteration:

Array.prototype.map = function(mapper) {
  return this.reduce((accumulator, currentElement) => {
    const mappedCurrentElement = mapper(currentElement);

    accumulator.push(mappedCurrentElement);

    return accumulator;
  }, []);
}

This approach has two benefits:

We've improved the time complexity to linear (or O(n)) time, and
The array passed in as the initial value is the same array that is ultimately returned.

The `filter` function

This is another one to be familiar with! If you aren't, go check it out on MDN.

filter calls a function (the "filter" function, defined by you) on each element of an array, and returns a new array containing all of the elements of the original array for which the filter function returned a truthy value.

Here's an example of 'filter' in action:

function removeUndefined(array) {
  return array.filter((x) => x !== undefined);
}

removeUndefined([1, true, undefined, 'hi']); // [1, true, 'hi']

What may not be completely apparent is that filter is also just a special case of reduce! Its implementation using a reduce loop is very similar to that of map. The only difference is that map's reducer function unconditionally appends the mapped element to the accumulator, whereas filter's reducer function conditionally appends the original element to the accumulator depending on the result of calling the filter function with that element. Here's what it looks like:

Array.prototype.filter = function(filterFn) {
  return this.reduce((accumulator, currentElement) => {
    if (filterFn(currentElement)) {
      accumulator.push(currentElement);
    }
    return accumulator;
  }, []);
}

Cool!

The `some` function

Not to be confused with the sum function that we've already spent some time talking about. The some function tends to be a little less well-known than map and filter, but it has use cases and definitely deserves a minor-supporting role in your toolbelt. Go take a look if you're new to some.

some calls a function (the "test" function, defined by you) on each element of an array, and returns a boolean indicating whether or not the test function returned a truthy value for any element in the array. It "short circuits" as much computation as possible by terminating the array iteration as soon as an element is encountered for which the test function returns a truthy value.

Here's an example of some in action:

function gotMilk(array) {
 return array.some((x) => x === 'milk');
}

gotMilk(['juice', 'water']); // false
gotMilk(['juice', 'milk', 'water']); // true

You've probably already guessed where this is going... Yes—some is actually just a special case of reduce. Unlike sum (where we reduce down to a number) and map and filter (where we reduce down to an array), with some we reduce down to a boolean. The boolean accumulator indicates whether or not any value of the array so far has returned truthy from the test function. Because of this, we initialize the accumulator to false, and once it gets flipped to true we stop calling the test function on the rest of the array:

Array.prototype.some = function(testFn) {
 return this.reduce((accumulator, currentElement) => {
   if (accumulator) { return accumulator; }
   return testFn(currentElement);
 }, false);
}

The reduce implementation of some is slightly less performant than the native implementation. The native implementation stops iterating as soon as a truthy value is encountered, whereas the reduce implementation only stops calling the test function but does not stop iterating. We could fix this by throwing an exception from the reducer function when we reach a truthy value, catch the exception outside, and return true. However, this defeats the purpose of using reduce in the first place.

The reason for showing an implementation of some that uses reduce is to illustrate that the idea of the some function is a special case of the reduce function, even though a performant implementation of some cannot be easily written using reduce.

And also these!

Similar to some, the following Array.prototype methods are all special cases of reduce and can be implemented using simple reducer functions:

every
find
findIndex
indexOf
flat

As we saw with some, a few of these functions are able to terminate the array iteration early and therefore cannot be performantly implemented using reduce. In spite of this, it is valuable to observe that they all are specific situations in which we want to reduce an array down to a single value.

So what?

The reduce function represents a simple idea: the reduction of an array down to a single value. Not surprisingly, it also boasts a simple implementation. So simple, in fact, that we can achieve it by making a few minor changes to a simple sum function!

But we shouldn't be fooled by reduce's simplicity in these regards. The power and applicability of reduce is evident in the sheer number of functions on the Array prototype (such as map, filter, and some) that are just special cases of reduce and can be implemented with simple reduce loops. This is not to suggest that we should use reduce instead of these more specific functions. Using the special cases of reduce (instead of reduce itself) improves the readability of your code! Rather, I'm pointing this out to showcase the power of reduce.

Power and beauty exist in simplicity. They do not require complexity. On the contrary, complexity should be avoided as much as possible! Think of it this way: a simple solution to a problem is going to be much easier to implement. It will be harder to accidentally write bugs into. It will be easier for another programmer to take over and build upon or change. It will be easier to test. The list goes on!

In the words of the great Edsger W. Dijkstra:

Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better.

And:

Simplicity is prerequisite for reliability.

Simple solutions are better than complex ones is almost every way imaginable. What's difficult is coming up with simple solutions. This is a skill that you will spend your entire career developing and never perfect.

That's all I've got for now! Hopefully you've been inspired not only to look for opportunities to reduce in your own code, but also to pursue simpler solutions when you have the bandwidth to. It will pay off in the long run!

Happy coding!

Like this post?

Follow me on Twitter where I (re)tweet about frontend things: @thesnups

Forem: Daniel Worsnup

Directory-Based Resolver Mapping for AWS AppSync

Managing AWS AppSync Pipeline Resolvers at Scale

Directory-Based Schema Mapping

Implementation Components

1. Resolver Functions

2. Data Source Configuration

3. Automated Resolver Registration

4. Data Source Definition

5. CDK Integration

Dependencies

Key Benefits

Conclusion

Like this post?

Shortcut Linking with AWS Identity Center SSO

The Problem with Traditional AWS Alerts

The Solution: Shortcut Linking with AWS Identity Center

What Does It Look Like?

Understanding AWS IAM Identity Center

Finding Your Identity Center Values

Setting Up the Infrastructure with AWS CDK

1. Setting Up Discord Webhooks

2. Building the CDK Infrastructure

The Lambda Function: Creating Shortcut Links

How It All Works Together

Extending the Solution

Security Best Practices

Conclusion

Like this post?

Objects as Keys: A JavaScript Adventure

Object Properties

Key Types and Automatic Coercion

Overriding the Default Coercion Behavior

Overriding Coercion For All Objects

Generating Unique IDs

Hiding the Object ID From Enumeration

Preventing Collisions With the Hidden ID Property

Mitigating Collisions With Non-Object Keys

Object ID Prefix

Object ID Symbols

The Journey Is More Important Than The Destination

One Last Thing

That's All!

Like this post?

How to Build a Simple Markdown Plugin for Your Gatsby Site

Cross-Posting Blogs

Embedded Videos

A Few Solutions

Gatsby and Markdown

Plugins Within Plugins

How a gatsby-transformer-remark Plugin Works

Abstract Syntax Trees

Markdown ASTs

Building the Plugin

Configuring the Plugin

Traversing the AST

Transforming the AST

Possible Improvements

Beyond Gatsby Plugins

Thanks for reading!

Like this post?

Why You Should Be Using the Web Share API in Your PWA

Sharing

The Web Share API

Browser Support

Using the Web Share API

Canonical URLs

Reacting to the share

Summary

Beyond Web Share

Like this post?

Everything You Need to Know About Array#reduce

What is reduce?

How does reduce work?

Parameters

The reducer function

Case study: the sum function

What are some applications of reduce?

The sum function

The map function

Case study: the `sum` function

The `sum` function

The `map` function

The `filter` function

The `some` function