Forem: Simon Plenderleith

Guidelines for choosing a Node.js framework

Simon Plenderleith — Mon, 04 Oct 2021 09:07:00 +0000

I often see the question "What’s the best Node.js framework?" crop up on message boards and social media. The replies tend to be full of strong opinions. Some developers will even get into arguments about it. If you’re trying to figure out the right framework for your project, none of this is going to be much help.

With lots of frameworks to choose from, and all those strong opinions, it’s easy to feel a little lost. Comparing frameworks based on the features they list can be a headache, and features are only part of the picture. It would be awesome if you had a clear checklist you could evaluate Node.js frameworks against.

I can’t provide you with an exhaustive checklist, as every project, team and developer have different needs. I do have some guidelines though to help you think about what matters to you when choosing a Node.js framework.

Thank you to the following folks who shared on Twitter what they consider when choosing a Node.js framework: Matt Hinchcliffe, Alex Wilson, Alexey Antipov and Anthony Bouch.

What’s your use case?

Context is everything. Try and get clear on your requirements before you start looking at frameworks and their features. This will help you you figure out if a framework is designed to support your intended use case. If it’s not, you could find yourself fighting against it when building your application.

Here are some use cases you might want to consider:

Full stack application vs API. Will your application be serving up HTML pages or will it be an API sending JSON responses? If it’s an API, will it be REST or GraphQL based?
Server-side rendering. Do you plan to use React or Vue components in your application? Some frameworks provide helpers to simplify integrating these frameworks in your applications.
Serverless. Will you be deploying your application to a serverless platform? For some serverless platforms like AWS Lambda, cold start time really matters. If a framework has a lot of initialisation to do before it can handle a request, it could have a big impact on your response times.
Real-time. If you want to use WebSockets, look for built-in framework support, or a community maintained library which you integrate with the framework.
TypeScript. Do you want to write your application in TypeScript? Some frameworks are designed only for TypeScript, some include type definitions, and others only have third-party types available which are maintained by the community. Even if you don’t intend to write your application in TypeScript, your code editor might be able to give you helpful hints based on a framework’s type definitions.

Framework styles

Some developers love frameworks which come with "batteries included", whereas others loathe them. "Batteries" in this context refers to features beyond HTTP request/response handling and routing. This might include things like validation, logging, authentication, database abstractions and dependency injection. Frameworks in this category tend to be highly opinionated about how applications should be built. They require you to structure things in a specific way so you can take advantage of the features which they offer.

At the other end of the spectrum you’ll find minimal frameworks. They tend to offer HTTP request/response handling, routing, and not a whole lot more. They’re not particularly opinionated and provide you with the freedom to structure your applications as you wish. As they provide a limited amount of functionality "out-of-the-box", you will need to choose and integrate other packages to provide any other functionality you require.

There are some frameworks which sit in the middle. They have some opinions and provide some additional functionality beyond the basics, such as logging and validation.

The style of framework you choose can be guided by the use cases I’ve mentioned above. It’s also likely to come down to your personal preferences (or those of your team). You might not have any preferences yet, but it will help if you pay attention to each framework’s style.

Support for promises and async / await

As Node.js and ECMAScript have evolved, applications designed around the callback pattern have faded away. Instead, we can now happily write applications with promises and async / await. This means it’s important for Node.js frameworks to be promise-aware. They should allow you to write async route handler functions and safely handle uncaught promise rejections for you.

If a framework doesn’t meet these basic requirements, it increases the risk of memory leaks or crashes in your applications. I’ve written an in-depth article which explains why this matters and what to look out for.

Documentation

Firstly, does the framework have documentation?! You’re likely to struggle building an application with a framework if it doesn’t have documentation. If you don’t see any, you should probably avoid using it.

Once you’ve established that a framework has documentation, try and get a sense of its quality. Not all documentation is created equal, so here are some things to consider:

Can you comfortably navigate and search it? The structure of documentation really matters. It can also be a big help if there’s a built-in search feature.
Does it make sense to you? There might be lots of documentation, but if it’s poorly written it’s not going to be much help.
Do you think you will be able to apply it when writing your own code? It’s one thing to read and understand how to do something, and totally another to apply it yourself when building a real application.

Practical examples

Reading through endless documentation to learn how to use a framework can feel overwhelming. Once you’ve got a general sense of what a framework can do, it really helps if there are practical examples available which show you how to use it. A "hello world" example might show you the basics of how to use a framework, but it’s often not much help if you want to do something more complex. Practical examples will show solutions for real problems you might need to solve. Ideally they’ll also demonstrate the "best practices" for using that particular framework.

You might find practical examples in the framework documentation, or perhaps in an examples folder in the project’s GitHub repository. Some frameworks even have full example applications on GitHub which you can browse, run and pick apart.

Seeing how the people who created a framework use it can be an excellent way of learning how to use it yourself. Instead of just grinding your way through documentation, practical examples will help you accelerate your learning curve with a new framework.

Community and ecosystem

The community which exists around a framework really matters. When shit hits the fan and you get really stuck or run into a weird bug, it’s important to know where you can go for help. Check if there’s a message board, Slack or Discord server for the framework you’re looking at. Dip into it and get a feel for the community. Do the people there seem welcoming and supportive? Are they happy to accept contributions? The community around a framework might not matter to you on day one, but you’ll definitely need it in the long term.

Framework popularity isn’t everything, but you’ll be building your whole application around it, so it’s important to check that it’s used by other developers. If a framework has widespread adoption it will make it easier to find libraries which have been written to work with it (e.g. middleware or plugins). It can be fairly straightforward to swap out one library for another in your application, but that’s generally not the case with a framework.

The stars for a repository on GitHub are often cited as an indicator of popularity, but I think they’re better viewed as a measure of general interest rather than actual usage. A simple way to determine if a framework is being used is by looking at its package download counts. They can’t be taken as a measure of how many projects are using a framework — many frameworks are bundled with other libraries but not used — but they can give you an idea of whether a framework is widely used. You can view weekly downloads on a package’s npm page, and the tool npm trends shows package downloads over time. It also allows you to compare packages.

Project health

When choosing a framework to build your application with, you want to be confident that the project is in good health and will continue to be maintained for the foreseeable future. Here are some indicators to look for:

Releases. Some developers hold the opinion that a framework doesn’t need new releases if it’s already "feature complete", but security updates and bug fixes are necessary. There is always the potential that major vulnerabilities exist in a framework (or one of its dependencies). It’s wise to avoid any framework which is unmaintained and no longer putting out releases.
Activity on issues. Lots of issues with no recent responses from maintainers could be an indicator that the project is unmaintained. On the flip side, if there are no issues at all, it could suggest that usage of the framework is very low.
Pull requests. A healthy project will typically have some recent pull request activity. Lots of old and inactive open pull requests could indicate that the project is no longer being maintained.
Contributors. If there are only one or two contributors to a framework it could suggest there isn’t much of a community around it. It also creates a risk around the long term maintenance of that framework if those contributors step back from the project. A healthy project will have numerous contributors, adding changes large and small.
Dependency graph. The more dependencies a framework has, the larger the attack surface area. It can also make debugging issues in your applications much more difficult. You don’t need to find a framework with zero dependencies, but you should have some awareness of a framework’s dependency graph. The tool npmgraph can provide you with an excellent overview.

The following tools can help you determine the health of a framework project:

Snyk Open Source Advisor. This tool generates a "health score" for packages. It pulls in data from several difference sources and summarises them to help you determine the health of a project.
The ‘Insights’ tab on GitHub repositories. This tab provides a comprehensive overview of recent project activity (releases, pull requests, issues and contributors).
Moiva. This is an open source tool which provides similar data to the Snyk and GitHub tools, but it conveniently allows you to compare metrics between frameworks.

If you’re looking for a list of Node.js frameworks, check out the list on Awesome Node.js. It includes popular Node.js frameworks, as well as some lesser known alternatives.

Conclusion

It turns out there are plenty of things to consider when choosing a Node.js framework. There’s one last thing, however, which you might want to ask yourself when evaluating a framework: do you think you’ll enjoy writing applications with it? Personally, if I don’t enjoy using a framework, every time I need to work with it becomes a chore. The sweet spot is a framework which meets your requirements and which you enjoy using.

Choosing a Node.js framework can be tricky, but hopefully the guidelines I’ve shared will help you refine your own personal checklist. Good luck choosing the "best" framework for your project!

Want to create better Node.js applications?

I write articles to help you level up as a Node.js developer. I'll send practical advice straight to your inbox every couple of weeks if you get on the list!

How does middleware work in Express?

Simon Plenderleith — Mon, 10 May 2021 12:11:00 +0000

This article is an adapted excerpt from my book, Express API Validation Essentials. It teaches you a complete API validation strategy which you can start applying in your Express applications today.

The Express documentation tells us that "an Express application is essentially a series of middleware function calls". It sounds simple on the surface, but honestly, middleware can get pretty confusing. You've probably found yourself wondering:

Where is the right place to add this middleware in my application?
When should I call the next callback function, and what happens when I do?
Why does the order of middleware matter?
How can I write my own code for handling errors?

The middleware pattern is fundamental to building applications with Express, so you want to have a solid understanding of what middleware is and how it works.

In this article we're going to dig into the middleware pattern. We'll also look at the different types of Express middleware and how to effectively combine them when we build our applications.

Jump links

The middleware pattern
- Middleware syntax
The two types of middleware
- Plain middleware
- Error handling middleware
Using middleware
- At the route level
- At the router level
- At the application level
Wrapping up

The middleware pattern

In Express, middleware are a specific style of function which you configure your application to use. They can run any code you like, but they typically take care of processing incoming requests, sending responses and handling errors. They are the building blocks of every Express application.

When you define a route in Express, the route handler function which you specify for that route is a middleware function:

app.get("/user", function routeHandlerMiddleware(request, response, next) {
    // execute something
});

(Example 1.1)

Middleware is flexible. You can tell Express to run the same middleware function for different routes, enabling you to do things like making a common check across different API endpoints.

As well as writing your own middleware functions, you can also install third-party middleware to use in your application. The Express documentation lists some popular middleware modules. There are also a wide variety of Express middleware modules available on npm.

Middleware syntax

Here is the syntax for a middleware function:

/**
 * @param {Object} request - Express request object (commonly named `req`)
 * @param {Object} response - Express response object (commonly named `res`)
 * @param {Function} next - Express `next()` function
 */
function middlewareFunction(request, response, next) {
    // execute something
}

(Example 1.2)

Note: You might have noticed that I refer to req as request and res as response. You can name the parameters for your middleware functions whatever you like, but I prefer verbose variable names as I think that it makes it easier for other developers to understand what your code is doing, even if they're not familiar with the Express framework.

When Express runs a middleware function, it is passed three arguments:

An Express request object (commonly named req) - this is an extended instance of Node.js' built-in http.IncomingMessage class.
An Express response object (commonly named res) - this is an extended instance of Node.js' built-in http.ServerResponse class.
An Express next() function - Once the middleware function has completed its tasks, it must call the next() function to hand off control to the next middleware. If you pass an argument to it, Express assumes it to be an error. It will skip any remaining non-error handling middleware functions and start executing error handling middleware.

Middleware functions should not return a value. Any value returned by middleware will not be used by Express.

The two types of middleware

Plain middleware

Most middleware functions that you will work with in an Express application are what I call "plain" middleware (the Express documentation doesn't have a specific term for them). They look like the function defined in the middleware syntax example above (Example 1.2).

Here is an example of a plain middleware function:

function plainMiddlewareFunction(request, response, next) {
    console.log(`The request method is ${request.method}`);

    /**
     * Ensure the next middleware function is called.
     */
    next();
}

(Example 1.3)

Error handling middleware

The difference between error handling middleware and plain middleware is that error handler middleware functions specify four parameters instead of three i.e. (error, request, response, next).

Here is an example of an error handling middleware function:

function errorHandlingMiddlewareFunction(error, request, response, next) {
    console.log(error.message);

    /**
     * Ensure the next error handling middleware is called.
     */
    next(error);
}

(Example 1.4)

This error handling middleware function will be executed when another middleware function calls the next() function with an error object e.g.

function anotherMiddlewareFunction(request, response, next) {
    const error = new Error("Something is wrong");

    /**
     * This will cause Express to start executing error
     * handling middleware.
     */
    next(error);
}

(Example 1.5)

Using middleware

The order in which middleware are configured is important. You can apply them at three different levels in your application:

The route level
The router level
The application level

If you want a route (or routes) to have errors which they raise handled by an error handling middleware, you must add it after the route has been defined.

Let's look at what configuring middleware looks like at each level.

At the route level

This is the most specific level: any middleware you configure at the route level will only run for that specific route.

app.get("/", someMiddleware, routeHandlerMiddleware, errorHandlerMiddleware);

(Example 1.6)

At the router level

Express allows you to create Router objects. They allow you to scope middleware to a specific set of routes. If you want the same middleware to run for multiple routes, but not for all routes in your application, they can be very useful.

import express from "express";

const router = express.Router();

router.use(someMiddleware);

router.post("/user", createUserRouteHandler);
router.get("/user/:user_id", getUserRouteHandler);
router.put("/user/:user_id", updateUserRouteHandler);
router.delete("/user/:user_id", deleteUserRouteHandler);

router.use(errorHandlerMiddleware);

(Example 1.7)

At the application level

This is the least specific level. Any middleware configured at this level will be run for all routes.

app.use(someMiddleware);

// define routes

app.use(errorHandlerMiddleware);

(Example 1.8)

Technically you can define some routes, call app.use(someMiddleware) , then define some other routes which you want someMiddleware to be run for. I don't recommend this approach as it tends to result in a confusing and hard to debug application structure.

You should only configure middleware at the application level if absolutely necessary i.e. it really must be run for every single route in your application. Every middleware function, no matter how small, takes some time to execute. The more middleware functions that need to be run for a route, the slower requests to that route will be. This really adds up as your application grows and is configured with lots of middleware. Try to scope middleware to the route or router levels when you can.

Wrapping up

In this article we've learnt about the middleware pattern in Express. We've also learnt about the different types of middleware and how we can combine them when building an application with Express.

If you'd like to read more about middleware, there are a couple of guides in the Express documentation:

Tired of wasting time reading Node.js blog posts which don't actually help you improve your projects?

Sign up to my weekly-ish newsletter and I'll let you know when I publish a new blog post which helps solve your real developer problems. I'll also send you an awesome tip so we can level up together, as well as a handful of excellent things by other people.

Learn how to handle validation in Express

Simon Plenderleith — Thu, 22 Apr 2021 14:56:11 +0000

These two free lessons are from 'Express API Validation Essentials'. Get the book.

Integrating Ajv into your Express application

You can use the Ajv library directly, however for an Express based API it's nice to be able to use middleware to validate request data which has been sent to an endpoint before that endpoint's route handler is run. This allows you to prevent things like accidentally storing invalid data in your database. It also means that you can handle validation errors and send a useful error response back to the client. The express-json-validator-middleware package can help you with all of this.

The express-json-validator-middleware package uses Ajv and allows you to pass configuration options to it. This is great as it means you have full control to configure Ajv as if you were using it directly.

Before we integrate this middleware into our application, let's get it installed:

npm install express-json-validator-middleware

Once you have it installed you need to require it in your application and configure it:

// src/middleware/json-validator.js

import { Validator } from "express-json-validator-middleware";

/**
 * Create a new instance of the `express-json-validator-middleware`
 * `Validator` class and pass in Ajv options if needed.
 *
 * @see https://github.com/ajv-validator/ajv/blob/master/docs/api.md#options
 */
const validator = new Validator();

export default validator.validate;

(Example 2.6)

Using a JSON schema to validate a response body

In this next piece of code we're going to do two things:

Define a JSON schema to describe the data we expect to receive when a client calls our API endpoint to create a new user. We want the data to be an object which always has a first_name and a last_name property. This object can optionally include an age property, and if it does, the value of that property must be an integer which is greater than or equal to 18.
Use the user schema which we've defined to validate requests to our POST /user API endpoint.

// src/schemas/user.schema.js

const userSchema = {
    type: "object",
    required: ["first_name", "last_name", "age"],
    properties: {
        first_name: {
            type: "string",
            minLength: 1,
        },
        last_name: {
            type: "string",
            minLength: 1,
        },
        age: {
            type: "number",
        },
    },
};

export default userSchema;

// src/server.js

import validate from "./middleware/json-validator.js";

import userSchema from "./schemas/user.schema.js";

app.post(
    "/user",
    validate({ body: userSchema }),
    function createUserRouteHandler(request, response, next) {
        /**
         * Normally you'd save the data you've received to a database,
         * but for this example we'll just send it back in the response.
         */
        response.json(request.body);

        next();
    }
);

(Example 2.7)

In the route definition above we're calling the validate() method from our Validator instance. We pass it an object telling it which request properties we want to validate, and what JSON schema we want to validate the value of each property against. It is configured to validate the body property of any requests to the POST /user endpoint against our userSchema JSON schema.

The validate() method compiles the JSON schema with Ajv, and then returns a middleware function which will be run every time a request is made to this endpoint. This middleware function will take care of running the validation which we've configured.

If the request body validates against our userSchema, the middleware function will call the next() Express function which was passed to it and our route handler function will be run. If Ajv returns a validation error, the middleware will call the next() Express function with an error object which has a validationErrors property containing an array of validation errors. Our route handler function will not be run. We'll look at where that error object gets passed to and how we can handle it in the next part of this book.

If you've enjoyed these free lessons from 'Express API Validation Essentials', get the book.

Are you using promises and async / await safely in Node.js?

Simon Plenderleith — Tue, 09 Mar 2021 18:15:48 +0000

You're building your Node.js server applications with a framework, and using promises and async / await. Mostly everything's working well, but it can get confusing at times, especially when you need to handle errors. How do you know if you're writing your async code correctly? And what happens if you're not?!

Promises and async / await are a fundamental part of writing Node.js applications with modern JavaScript. If only you could be confident that you're using them correctly in your applications. And wouldn't it be great to know that your framework has got you covered when you mess up? (hey, we all do!)

If a framework has "native support" for promises and async / await then things are in good shape, but what does that actually mean? In this article I'll bring you up to speed on what you need from your framework. Then we'll run through which frameworks have native support, and look at how you can write async code safely in Node.js.

In this article I'm going to refer to promises and async / await collectively as "async code".

Jump links

Support for async code in Node.js
What does framework "native support" for async code look like?
What happens when a framework doesn't natively support async code?
Which Node.js frameworks natively support async code?
How to write async code safely in Node.js
- Build new projects with a framework which natively supports it
- Migrate existing applications to a framework which natively supports it
- Let unhandled promise rejections crash your node process!
Learn more

Support for async code in Node.js

Node.js has had support for promises since v4.0.0 (released September 2015) and async / await since v7.6.0 (released February 2017). These JavaScript language features are now widely used across the Node.js ecosystem.

What does framework "native support" for async code look like?

In this context, "native support" means that a framework supports a certain set of behaviours or features by default, without the need for you to add any extra code.

Here are the things which a Node.js framework should natively support for your async code to function safely and correctly:

Your route handler functions can use async / await - The framework will explicitly wait for the promise wrapping the route handler function to resolve or reject (all JavaScript functions using the async keyword are automatically wrapped in a promise).
You can throw errors from within a route handler function - The error will be caught and gracefully handled by the framework i.e. sending back an appropriate error response to the client and logging the error.
Uncaught promise rejections from a route handler function will be handled for you - When a rejected promise is not caught and handled by your code within a route handler function, the framework will catch it and gracefully handle it in the same way as if an error has been thrown.
Bonus points: Return values from route handler functions are sent as a response body - This means you don't need to explicitly call a method to send a response e.g. response.send(). This is a nice to have though.

What happens when a framework doesn't natively support async code?

The asynchronous control flow of your application will behave unpredictably, especially when things go wrong i.e. if there's a promise rejection which you haven't caught. Because the framework has no awareness of promises, it won't catch promise rejections for you. If an unhandled promise rejection occurs, it's most likely that a response will never be sent back to the client. In most cases this will cause the client's request to hang.

Here is an example route defined in the style supported by some Node.js frameworks:

app.get("/user/:user_id", async (request, response) => {
    const user = await getUser(request.params.user_id);
    response.json(user);
});

If the route above is defined on a server instance created by a framework which doesn't natively support async code, you're very likely to run into problems. The route handler function doesn't implement any error handling code (either intentionally or by accident). If the promise returned by the getUser() function rejects, it will result in an unhandled promise rejection.

Unhandled promise rejections can cause memory leaks in your application. If the memory leak is bad enough, the node process will eventually run out of memory and your application will no longer be able to handle requests.

From Node.js v15.0.0 onwards an unhandled promise rejection will throw an error, causing the node process to exit i.e. your application will crash (in previous versions a warning message would be emitted to stderr).

Which Node.js frameworks natively support async code?

Many Node.js frameworks now support async code, so I think it's more useful to highlight popular frameworks which don't support async code:

Express - No native support for async code. There's not been a release of Express for almost 2 years now, so it looks unlikely that this framework will support async code any time soon. If you are unable to move away from Express at the moment, you can monkey patch in support for async / await.
Restify - No native support yet. Support merged mid-2020, scheduled for a v9 release.

An excellent alternative to Express or Restify is the Fastify framework. It has full native support for async code and is in active development. There is also a fastify-express plugin available which can help ease your migration path away from Express.

How to write async code safely in Node.js

Build new projects with a framework which natively supports it

If you can, you should absolutely use a Node.js framework which has native support for async code. This should be one of your minimum requirements when choosing a Node.js framework for a new project.

Migrate existing applications to a framework which natively supports it

If your existing Node.js applications are using a framework which doesn't natively support async code, you should strongly consider migrating them to one which does.

If you're using Express, the 'Which Node.js frameworks natively support async code?' section in this article has tips on migrating away from it.

Let unhandled promise rejections crash your node process!

This is really important.

You shouldn't set an event handler for unhandledRejection events, unless the event handler will exit the node process. You risk memory leaks in your application if the node process doesn't exit when an unhandled promise rejection occurs.

Until you start using Node.js >= v15.0.0, you should use the make-promises-safe module in your Node.js applications. When an unhandled promise rejection occurs in your application, this module "prints the stacktrace and exits the process with an exit code of 1" (i.e. crashes your application).

Learn more

If you want to read more on the subject, the official Learn Node.js website has an excellent page which covers the path from callbacks ➜ promises ➜ async / await in JavaScript: Modern Asynchronous JavaScript with Async and Await.

If you want to learn more about how to use promises correctly in Node.js and avoid some common pitfalls, I recommend watching the talk Broken Promises by James Snell.

Meanwhile, on a less technical note...

Tired of wasting time reading Node.js blog posts which don't actually help you improve your projects?

Sign up to my weekly-ish newsletter and I'll let you know when I publish a new blog post which helps solve real developer problems ✨

How to securely call an authenticated API from your front end

Simon Plenderleith — Fri, 05 Feb 2021 14:20:00 +0000

Credit: Key icon by Gregor Cresnar from the Noun Project

Your front end needs to access data from an API which requires an API key. If you put this API key in your client side JavaScript, you know that anyone viewing your website could view this API key (with a little help from their browser's Developer Tools). This doesn't seem secure at all, but what can you do instead?

Only applications running on a server - i.e. a back end - should have access to secrets like an API key. This means that requests with an API key can only be made from the server side. The thing is, you want your front end to be able to request and use data from the API in a secure way.

One way of achieving this is to create a "proxy" server. Instead of directly calling the API, your client side JavaScript will make requests to the proxy server. The proxy server can add an API key to every request and forward it on to the API. This keeps the API key secure and away from your front end.

In this article I'll show you how to use the Fastify framework and the fastify-http-proxy plugin to create a proxy server.

The full code for this article is on GitHub.

This article uses ECMAScript (ES) module syntax. ES modules are well supported in v12.20.0 / v14.13.0 (and higher) releases of Node.js. Learn more about ES module support in Node.js.

Jump links

Request flow with a proxy server
Create a server with Fastify
Add and configure the fastify-http-proxy plugin
Add an API key to proxied requests
- HTTP request header
- URL query string
Conclusion

Request flow with a proxy server

Let's assume that we have some client side JavaScript running on a web page - it could be a React application, or "vanilla" JavaScript (no framework or library). This client side JavaScript needs to retrieve data from an API which requires an API key to be sent in the request.

As we don't want our client side JavaScript to contain the API key for security reasons, we're going to create a proxy server in Node.js which can receive a request from the client side JavaScript (made with fetch, or a request library like Axios). This proxy server will add the required API key to the request and forward it on to the API server.

The request flow from the client (JavaScript running on a web page in a user's browser) through to the API server will look like this:

Request from client side JavaScript to our proxy server
 ↓
Proxy server receives request, adds the API key, forwards request to API server
 ↓
API server receives request, sends response back to proxy server

When the proxy server receives a response from the API server it will send it back to the client. At no point will the API key be exposed to the client side JavaScript.

Instead of making requests to https://some-api.com/some/path from our client side JavaScript, we'll now make requests to our proxy server: https://my-proxy.com/some/path. Neat, right?

Create a server with Fastify

We're going to use the Fastify framework and the fastify-http-proxy plugin to create our proxy server in Node.js.

First let's install the dependencies which our proxy server application will require:

npm install fastify fastify-http-proxy

We're now going to create and configure a Fastify server instance:

// src/server.js

import createFastifyServer from "fastify";

/**
 * Create a Fastify server instance with logging enabled.
 * Fastify uses the library `pino` for logging.
 *
 * @see https://www.fastify.io/docs/latest/Logging/
 * @see https://github.com/pinojs/pino/
 */
const fastify = createFastifyServer({
    logger: true,
});

try {
    /**
     * Make use of top-level `await` i.e. outside of an `async` function.
     *
     * @see https://nodejs.org/docs/latest-v14.x/api/esm.html#esm_top_level_await
     */
    await fastify.listen(3000);
} catch (error) {
    fastify.log.error(error);
    process.exit(1);
}

If we run this code (node src/server.js), we'll have an HTTP server listening on port 3000.

Our server doesn't provide any endpoints which you can make a request to, so making a request to http://localhost:3000/ will result in a 404 error response. We need to add and configure the fastify-http-proxy plugin in order for our server to be able to handle requests.

Add and configure the fastify-http-proxy plugin

We want to configure our Fastify server to proxy requests which it receives to https://some-api.com. In order to do this, let's import the fastify-http-proxy plugin and configure it:

// src/server.js

import fastifyHttpProxy from "fastify-http-proxy";

/**
 * Register and configure the `fastify-http-proxy` plugin.
 *
 * This plugin supports all the options of `fastify-reply-from`,
 * as well as a few additional options e.g. `upstream`.
 *
 * @see https://github.com/fastify/fastify-http-proxy#options
 * @see https://github.com/fastify/fastify-reply-from
 */
fastify.register(fastifyHttpProxy, {
    upstream: "https://some-api.com",
    undici: true,
});

Our Fastify server is now configured as a proxy server. It will forward all requests it receives to https://some-api.com (our "upstream" server).

Our proxy server will use the HTTP client library undici to make requests to the upstream server. The undici library is a dependency of fastify-reply-from, which fastify-http-proxy is using under the hood. undici is much faster than the native HTTP client provided by Node.js.

Now that we have our proxy server set up, we need to configure it to add an API key to proxied requests.

Add an API key to proxied requests

There are several different ways that APIs can implement authentication. One of the most common methods is for the client to pass an API key in the request. Typically APIs require the API key to be sent in a request header e.g. X-Api-Key: abc123. Some APIs might require the API key in a query string parameter e.g. ?apiKey=abc123.

fastify-http-proxy accepts a replyOptions object which it passes through to fastify-reply-from. These options give us full control to modify requests and responses as they pass through our proxy server.

Let's take a look at how we can modify requests and add an API key before our proxy server forwards it on to the API server (our "upstream").

HTTP request header

In order to add an API key to the HTTP request headers, we're going to set a replyOptions.rewriteRequestHeaders function. We will access our API key from an environment variable and set it as the value of an X-Api-Key request header. This code builds on our initial configuration for the fastify-http-proxy plugin:

// src/server.js

const CONFIG = {
    apiKey: process.env.API_KEY,
};

fastify.register(fastifyHttpProxy, {
    upstream: "https://some-api.com",
    undici: true,
    replyOptions: {
        rewriteRequestHeaders: (originalRequest, headers) => {
            return {
                /**
                 * Preserve the existing request headers.
                 */
                ...headers,
                /**
                 * Add the header which the API we're proxying requests
                 * to requires to authenticate the request.
                 */
                'X-Api-Key': CONFIG.apiKey,
            };
        },
    },
});

With a little extra configuration our server is now adding an API key to every request that it is proxying.

While X-Api-Key is a commonly used request header name, the API you are making requests to might require a different header e.g. Authorization: Bearer <TOKEN>. The replyOptions.rewriteRequestHeaders option allows us to add any request headers that we need.

View the full code for an example proxy server which authenticates using an API key request header

URL query string

I do not recommend that you design your own APIs to accept an API key via a URL query string. An API key is a "secret", just like a password is. When you put an API key in a URL it is much easier to accidentally leak it than if you send it via a request header e.g. by an accidental copy and paste, or by logging it in server request logs.

Unfortunately, some APIs do require you to send an API key in the URL query string. If this is the only way for you to authenticate with the API you are making requests to, you can use the replyOptions.queryString option provided by fastify-http-proxy.

Conclusion

In this article we've learnt how we can use Fastify and the fastify-http-proxy plugin to proxy requests and add an API key to them. This allows us to keep our API key secure and away from our client side JavaScript.

While fastify-http-proxy is very powerful and allows us to set up a proxy with minimal configuration, there are cases in which you might want to take a different approach e.g.

You need to proxy complex API requests
You want to create your own abstraction over another API
You have an existing back end Node.js server application

In these cases you might want to consider creating your own API endpoints which then make requests to an upstream API. The node-fetch library is a popular choice for making requests from Node.js. However, if you are interested in the features offered by undici, I recommend keeping an eye on the undici-fetch library. It is being developed as a WHATWG Fetch implementation based on undici.

Tired of wasting time reading Node.js blog posts which don't actually help you improve your projects?

Sign up to my weekly-ish newsletter and I'll let you know when I publish a new blog post which helps solve real developer problems ✨

What you need to know about ES modules in Node.js

Simon Plenderleith — Mon, 18 Jan 2021 12:03:26 +0000

ECMAScript modules are the official standard format to package JavaScript code for reuse.

— Source: Node.js documentation for ECMAScript modules

Up until recently I've tended to avoid using ES modules and import/export syntax in Node.js. I've found it hard to keep track of which versions of Node.js support this syntax out-of-the-box, as well as which ES module features they support. I've put together this short guide in the hope that I can help clear up any confusion that other folks might also have around ES modules in Node.js.

This blog post isn't going to cover much about what ECMAScript (ES) modules are or how to use them. There are lots of resources out there that already do a great job of this. If you want to learn more about ES modules, check out my top links for learning more at the end of this post.

As the quote from the Node.js documentation above mentions, ECMAScript modules are the official standard format for modules in JavaScript. The browser JavaScript ecosystem (tooling and browsers) has had good support for ES modules for quite a while now. Meanwhile, there have been tremendous efforts over the past few years to bring ES module support to Node.js too, which has historically only used the CommonJS module format.

Let's take a look at what you need to know about ES modules in Node.js today.

I intend to update and expand this blog post as needed. If you spot anything incorrect, out-of-date, or which you feel is missing from this post, please drop me a message on Twitter. I want to make sure this is a useful and accurate resource!

Jump links

Several names for the same thing
Which versions of Node.js have full support for ES modules?
Why are ECMAScript modules marked as 'Experimental' in the Node.js documentation?
ES and CommonJS modules play nice together (mostly)
Use ES modules when you start a new Node.js project
It's probably not worth migrating your existing Node.js applications to use ES modules
Potential pain points
- Limited test library support for mocking ES modules
- You can't import JSON files without using an experimental flag
My top links for learning more

Several names for the same thing

It took me a while to understand that there are several names which refer to the same thing. In this blog post I'm referring to them as "ES modules", but they are more generally known as "JavaScript modules". This is because they are a language-level module syntax for JavaScript (hence the standard I referenced earlier).

In the Node.js documentation you will generally see JavaScript modules referred to as "ECMAScript modules". In the wider world, this is often abbreviated to "ES modules", or simply "ESM". These are all a reference to the same thing.

JavaScript modules = ECMAScript modules = ES modules = ESM

Which versions of Node.js have full support for ES modules?

ES module support was added to Node.js in v8.5.0 (yes, really!), and released in September 2017. At that stage it was highly experimental and missing a lot of the ES module features which Node.js now supports.

Node.js now has full support for ES modules and you can happily use them in Node.js 12.x versions and above. I would however recommend that you use newer versions of Node.js 12.x and 14.x as they have have more mature and stable ES module support. Importantly, they also have improved interoperability with older style CommonJS modules, which are still widely used. I explain this in more detail in my article 'Node.js now supports named imports from CommonJS modules, but what does that mean?'.

Here are the versions I recommend using for the LTS release lines of Node.js:

14.x — v14.13.0 or higher
- Why? Support for detection of CommonJS named exports, unflagged top-level await.
12.x — v12.20.0 or higher
- Why? Support for detection of CommonJS named exports, Loading ECMAScript modules no longer requires a command-line flag.
10.x — Don't use ES modules in Node.js v10.x releases
- Why? It's missing some key ES module features. It's also end-of-life on 2021-04-30, meaning this release line will no longer receive bug fixes after that date.

It's not a deal breaker - you can use v12.0.0 or v14.0.0 if you like, but using the versions I've mentioned above (or higher) will make your life easier.

Why are ECMAScript modules marked as 'Experimental' in the Node.js documentation?

"Hold up, Simon, the Node.js 12.x and 14.x documentation for ECMAScript modules says that this feature is Experimental!"

ECMAScript modules are marked as 'Stable' in the Node.js 15.x documentation for ECMAScript modules, however in the 12.x and 14.x documentation they are marked as 'Experimental'.

If Node.js 12.x and 14.x releases have full support for ES modules, what gives? I was wondering the same, so I asked Matteo Collina on Twitter (he's a member of the Node.js TSC). Myles Borins (also a member of the TSC) chimed in on the thread to explain the rationale behind ES modules being marked as 'Experimental' in the 12.x and 14.x release lines:

We've explicitly not marked it as stable yet just in case we need to make breaking changes between 15 -> 16 that we may want to backport. I don't think 12 will ever get marked stable due to being maintenance.

(Source tweet)

If we are ready to make the commitment of not breaking anything then yes, it is stable. The 0th hour deprecation of subpath folders does leave me pause though. Also keep in mind that for LTS being experimental makes it easier to make non-breaking changes too.

(Source tweet)

My interpretation of this is: you can safely go ahead and use ES modules in Node.js 12.x or 14.x, as the core implementation is highly unlikely to change at this stage. Keeping ES modules marked as 'Experimental' in these Node.js release lines gives the developers working on Node.js some wiggle room to change things if needed. On an open source project of this scale, that makes a lot of sense to me.

Thanks to Matteo and Myles for helping clarify the situation.

ES and CommonJS modules play nice together (mostly)

Given that the npm registry lists almost 1.5 million packages, many of which only expose a CommonJS module, it's safe to say that interoperability between the ES and CommonJS module formats is really important. The good news is that they generally work pretty well together.

You can reliably import a CommonJS module in an ES module e.g.

import someModule from "someModule";

Thanks to some amazing work by Guy Bedford, Geoffrey Booth, and host of other contributors, you can also import named exports from CommonJS modules (in most cases - I've written about this in detail) e.g.

import { someFunction, someObject } from "someModule";

What all of this means is that, in general, you can import CommonJS modules in your ES modules and Node.js will take care of all the module format interoperability for you. I think that's pretty damn impressive.

The Node.js ECMAScript module documentation has a section which covers all the details about Interoperability with CommonJS.

Use ES modules when you start a new Node.js project

A new Node.js project is a great opportunity to start using ES modules. However, Node.js treats JavaScript code as CommonJS modules by default. This means you must tell it when it should treat JavaScript code as ES modules.

If all of your project's code will be in ES modules you can add "type": "module" in the project's package.json file and you're good to go. Node.js will treat every module in your project as an ES module.

Note: This doesn't mean the dependencies your project uses must use ES modules. ES and CommonJS modules have good interoperability, as explained above.

If only specific modules in your project will be ES modules, give the files they're contained in an .mjs extension (instead of .js) and Node.js will use the ES module loader for those scripts.

To learn more about configuring ES modules in Node.js I recommend reading the Node.js Packages documentation on this subject.

It's probably not worth migrating your existing Node.js applications to use ES modules

There's no single hugely obvious benefit to migrating an existing Node.js application from using CommonJS modules to ES modules. Unless you have a good reason to, migrating to ES modules just for the sake of it is probably not worth it. Given that modules are imported when an application starts, you're not going to see performance improvements when your application is running e.g. handling requests and serving responses.

If you're working with a small application, by all means go ahead and migrate it to use ES modules for the learning experience. For large applications though, there are almost certainly other things you could better spend your time / money on (and it's likely that your employer or clients will agree).

Node.js packages should potentially be updated to use ES modules. They can be changed to only use ES modules, or they can be configured to offer both CommonJS and ES modules (a "dual" package). The Node.js documentation dives into the pros and cons of the "dual" package approach.

Sindre Sorhus, author of over 1,000 Node.js packages, is planning to convert all of their packages to only use ES modules in order to "rip off the bandaid and push the ecosystem forward".

Potential pain points

While Node.js support for ES modules is really good, there are a couple of potential pain points you should be aware of. I expect these potential pain points to go away over time as the wider Node.js ecosystem support for ES modules improves.

Limited test library support for mocking ES modules

Jest and Sinon don't yet support mocking ES modules:

Jest has experimental support for ES modules.
Sinon does not have support for ES modules, but there is a recently re-opened GitHub issue which offers a workaround.

If you need an alternative for mocking, testdouble.js has full ES module support thanks to some excellent work by Gil Tayar.

The testdouble library is test-framework agnostic, meaning you can happily use it with any framework you're already using e.g. Mocha, Jest, Jasmine, Tape.

You can't import JSON files without using an experimental flag

You can't import JSON files - e.g. some-file.json - without using an experimental flag when you run node. This isn't a big deal for many applications, but if you're currently doing require("some-file.json") in your application, this is something to be aware of.

The Node.js documentation details a workaround for loading JSON files in ES modules. You can also enable the experimental JSON module loading if you really need to.

My top links for learning more

Node.js documentation for ECMAScript modules - The latest Node.js documentation for ES modules. It's really good and very thorough.
MDN Guide on JavaScript modules - This is a fantastic guide which provides some useful background on modules in JavaScript, along with a wealth of practical JavaScript module examples. I ❤️ MDN so much.
NodeConf Remote 2020 - Gil Tayar - ES Modules in Node.JS - An excellent introduction to ES modules in Node.js (companion code + link to slides, my notes from this talk).
Node Modules at War: Why CommonJS and ES Modules Can’t Get Along - This blog post is now a little bit out of date - e.g. ES modules can now import named CommonJS exports from many packages - but it's still well worth a read. When I read this article last year it really helped me understand the differences between the two module types and made a lot of things click for me.

Send awesome structured error responses with Express

Simon Plenderleith — Wed, 06 Jan 2021 16:05:00 +0000

When you’re creating an Express API it can be difficult to know how to handle error cases and send consistent error responses. It becomes even more complicated if you want to send helpful error responses with extra details about what went wrong.

You know these extra details are needed because they’ll also be super helpful for debugging requests to your API, but before you know it, you find yourself designing your own error response format. It all feels awkward, and like it’s probably something you shouldn’t be doing, but what alternative is there?

Thankfully, there’s an awesome alternative, and you can find it in the ‘Problem Details for HTTP APIs’ specification (RFC7807). Don’t worry though, I don’t expect you to go and read the whole RFC (Request for Comments) document. I know that RFCs aren’t always the easiest of reads, but I think the ideas in this one are so good that I’ve done the RFC reading for you and pulled out all of the good stuff that can help you with formatting your API error responses.

In this article we’ll explore the Problem Details specification and how it can help you build better APIs. By learning how to apply this well-defined and structured approach, your struggles with creating API error responses will be a thing of the past.

Jump links

Introducing the ‘Problem Details for HTTP APIs’ specification
Problem types and Problem details objects
Example problem details response
- More details, clearer problems
- Breakdown of a problem details object
How to send problem details responses with Express
- Define problem types and map them to JavaScript error classes
- Look up the problem details for an error
- Create an error handler to send a problem details response
- Use the problem details response error handler
- Example problem details error responses
Next steps

Introducing the ‘Problem Details for HTTP APIs’ specification

The aim of the problem details specification is to define a common error format which you can use for the error responses from your API. This avoids having to invent your own error response format or, even worse, attempting to redefine the meaning of existing HTTP status codes. Seriously, don’t do this! The meaning of HTTP status codes are well documented and commonly understood for a reason.

The status codes defined in the HTTP specification are very useful, and often provide enough context to the client as to what went wrong, but they don’t always convey enough information about an error to be helpful.

Take for example the status code 422 (Unprocessable Entity) – as defined in the HTTP specification, it tells a client that the server understood the request body and its structure, but was unable to process it. However, that alone doesn’t tell the client specifically what was wrong with the JSON that was sent in the request body. Problem details can help you solve this problem.

The specification describes a problem detail as "a way to carry machine-readable details of errors in a HTTP response". Let’s take a look at how the problem details specification defines them.

Problem types and Problem details objects

The problem details specification defines what a "problem type" and a "problem details object" are, and their relationship:

Problem type – A problem type definition must include a type URI (typically a URL), a short title to describe it and the HTTP status code for it to be used with.

If required, the definition can also specify additional properties to be included on problem details objects which use this type e.g. balance and accounts in the example above. These additional properties are referred to as "extensions" by the specification.

The type URI is effectively the namespace for the problem type definition. If the definition changes, the type should also change.

You should avoid defining a new problem type when the response HTTP status code provides enough context by itself. The specification gives the following example: "a ‘write access disallowed’ problem is probably unnecessary, since a 403 Forbidden status code in response to a PUT request is self-explanatory".

Problem details object – An object which includes the type, title and status properties for a problem type. This object represents a specific occurrence of that problem type. It can optionally contain a detail property – a human-readable explanation specific to this occurrence of the problem – and an instance property – a URI reference that identifies the specific occurrence of the problem.

A problem details object should include values for any extensions specified by the problem type definition.

Problem detail objects can be formatted as XML or JSON. For the purpose of this article we’ll be using JSON formatted problem details.

Example problem details response

The response body in this example contains a problem details object of the type https://example.com/probs/out-of-credit:

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en

{
    "type": "https://example.com/probs/out-of-credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but that costs 50.",
    "instance": "/account/12345/msgs/abc",
    "balance": 30,
    "accounts": ["/account/12345", "/account/67890"]
}

— Source: RFC7807 – Problem Details for HTTP APIs.

Note how the example response above contains the header Content-Type: application/problem+json. This is the media type for JSON problem details which is defined by the problem details specification. Clients can use the Content-Type header in a response to determine what is contained in the response body. This allows them to handle different types of response bodies in different ways.

Any response containing a problem details object must also contain the Content-Type: application/problem+json header.

More details, clearer problems

Including problem details in the response body allows the client to derive more information about what went wrong, and gives it a better chance of being able to handle the error appropriately. Every problem details object must have a type property. The client can then use the value of the type to determine the specific type of problem which occurred.

In the example problem details object above (Example 3.1), the problem can be identified as an "out of credit" problem when the client checks the value of the type field: https://example.com/probs/out-of-credit

The type for a problem can be specific to your API, or you can potentially reuse existing ones if you wish.

Breakdown of a problem details object

To better understand the properties which make up a problem details object, let’s break it down and look at each property. Let’s start with our example problem details object:

{
    "type": "https://example.com/probs/out-of-credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but that costs 50.",
    "instance": "/account/12345/msgs/abc",
    "balance": 30,
    "accounts": ["/account/12345", "/account/67890"]
}

Now let’s go through this line by line:

"type": "https://example.com/probs/out-of-credit",

The type URI for the problem type being used by this problem details object. The specification encourages that this is a real URL which provides human-readable documentation in HTML format. The client should use the value of this field as the primary identifier for the problem.

"title": "You do not have enough credit.",

The title defined by the problem type.

"status": 403,

The HTTP status code defined by the problem type. Should be the same as the status code sent in the response from the API.

As intermediaries between the client and the server (e.g. a proxy or a cache) might modify the response status code, this value can be used by the client to determine the original status code of the response. Also useful in situations where the response body is the only available part of the response e.g. in logs.

"detail": "Your current balance is 30, but that costs 50.",

A human-readable explanation of the problem. It should focus on helping the client correct the problem. Machine-readable information should be added in extensions (see below). Specific to this occurrence of the problem.

"instance": "/account/12345/msgs/abc",

A URI reference for the specific problem occurrence. Typically a URL, optionally containing more information. Specific to this occurrence of the problem.

"balance": 30,
"accounts": ["/account/12345", "/account/67890"]

Extensions specified by the problem type. Specific to this occurrence of the problem.

The type, title and status – as defined by a problem type – should be the same for every occurrence of the problem.

Note: As with any response you send from your API, you should be careful when creating problem details objects that you don’t expose any of the implementation details of your application, as this can make it potentially vulnerable to attack.

How to send problem details responses with Express

Now that we’ve covered the concepts and conventions of problem details, we can write some code. This code will allow us to send problem details error responses from our Express API.

Note: This article uses ECMAScript (ES) module syntax. ES modules are well supported in the v12 and v14 releases of Node.js.

For Node.js 12.x, I recommend using v12.20.0 or higher. For Node.js 14.x, I recommend using v14.13.0 or higher. These releases of Node.js have stable ES module support, as well as improved compatibility with the older style CommonJS modules, which are still widely used.

Define problem types and map them to JavaScript error classes

In this code we’re going to define two different problem types and map them to JavaScript error classes – in this case, ones that are provided by the http-errors library. We’ll use these problem types later on when we create an error handler middleware.

// src/middleware/problem-details-response.js

import createHttpError from "http-errors";

const defaultProblemDetails = {
    /**
     * This is the only URI reserved as a problem type in the
     * problem details spec. It indicates that the problem has
     * no additional semantics beyond that of the HTTP status code.
     */
    type: "about:blank",
    status: 500,
};

const problemTypes = [
    {
        matchErrorClass: createHttpError.BadRequest,
        details: {
            type: "https://example-api.com/problem/invalid-user-id",
            title: "User ID must be a number",
            status: 400,
        },
    },
    {
        matchErrorClass: createHttpError.Forbidden,
        details: {
            type: "https://example-api.com/problem/user-locked",
            title: "User has been locked",
            status: 403,
        },
    },
];

Look up the problem details for an error

Now let’s create a function which, when passed an error object, will look through our array of problemTypes for one which has been mapped to the type of error it has received:

// src/middleware/problem-details-response.js

/**
 * Get the problem details which have been defined for an error.
 *
 * @param {Error} error
 * @return {Object} - Problem details (type, title, status)
 */
function getProblemDetailsForError(error) {
    const problemType = problemTypes.find((problemType) => {
        /**
         * Test if the error object is an instance of the error
         * class specified by the problem type.
         *
         * @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof
         */
        return error instanceof problemType.matchErrorClass;
    });

    if (!problemType) {
        /**
         * A problem type hasn't been defined for the type of error 
         * this function has received so return fallback problem details.
         */
        return defaultProblemDetails;
    }

    return problemType.details;
}

Create an error handler to send a problem details response

This error handler middleware is going to call the getProblemDetailsByError() function which we just defined, and then send the problem details that it returns as a response body, along with the correct HTTP status code and Content-Type header:

// src/middleware/problem-details-response.js

/**
 * Send an error response using the problem details format.
 *
 * @see https://tools.ietf.org/html/rfc7807
 *
 * @param {Error} error
 * @param {Object} request - Express request object
 * @param {Object} response - Express response object
 * @param {Function} next - Express callback function
 */
function problemDetailsResponseMiddleware(
    error,
    request,
    response,
    next
) {
    /**
     * If response headers have already been sent,
     * delegate to the default Express error handler.
     */
    if (response.headersSent) {
        return next(error);
    }

    const problemDetails = getProblemDetailsForError(error);

    /**
     * If the problem details don't contain an HTTP status code,
     * let's check the error object for a status code. If the
     * error object doesn't have one then we'll fall back to a
     * generic 500 (Internal Server Error) status code.
     */
    if (!problemDetails.status) {
        problemDetails.status = error.statusCode || 500;
    }

    /**
     * Set the correct media type for a response containing a
     * JSON formatted problem details object.
     *
     * @see https://tools.ietf.org/html/rfc7807#section-3
     */
    response.set("Content-Type", "application/problem+json");

    /**
     * Set the response status code and a JSON formatted body
     * containing the problem details.
     */
    response.status(problemDetails.status).json(problemDetails);

    /**
     * Ensure any remaining middleware are run.
     */
    next();
};

export default problemDetailsResponseMiddleware;

Use the problem details response error handler

Our error handling middlware will be run when an error object is passed to a next() Express function. When the next() function is called with with an error object, it automatically stops calling all regular middleware for the current request. It then starts calling any error handler middleware which has been configured.

It’s time to pull everything together. Here is a complete example Express API application, configured to use our problem details error handler middleware:

// src/server.js

import express from "express";
import createHttpError from "http-errors";

import problemDetailsResponseMiddleware from "./middleware/problem-details-response.js";

/**
 * Express configuration and routes
 */

const PORT = 3000;
const app = express();

/**
 * In a real application this would run a query against a
 * database, but for this example it's returning a `Promise`
 * which randomly either resolves with an example user object
 * or rejects with an error.
 */
function getUserData() {
    return new Promise((resolve, reject) => {
        const randomlyFail = Math.random() < 0.5;
        if (randomlyFail) {
            reject(
                "An error occurred while attempting to run the database query."
            );
        } else {
            resolve({
                id: 1234,
                first_name: "Bobo",
                is_locked: true,
            });
        }
    });
}

/**
 * This route demonstrates:
 *
 * - Creating an error when the user ID in the URL is not numeric.
 * - Creating an error when the (faked) user object from the database
 * is locked.
 * - Catching a (randomly faked) database error (see `getUserData()`
 * function above).
 * - Passing all error objects to the `next()` callback so our problem
 * details response error handler can take care of them.
 */
app.get("/user/:user_id", (request, response, next) => {
    const userIdIsNumeric = !isNaN(request.params.user_id);

    if (!userIdIsNumeric) {
        const error = new createHttpError.BadRequest();

        return next(error);
    }

    getUserData()
        .then((user) => {
            if (user.is_locked) {
                const error = new createHttpError.Forbidden();

                return next(error);
            }

            response.json(user);
        })
        .catch(next);
});

app.use(problemDetailsResponseMiddleware);

app.listen(PORT, () =>
    console.log(`Example app listening at http://localhost:${PORT}`)
);

Example problem details error responses

Here are the error responses that are produced by the code that we’ve just put together:

< HTTP/1.1 400 Bad Request
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 106

{
    "type": "https://example-api.com/problem/invalid-user-id",
    "title": "User ID must be a number",
    "status": 400
}

< HTTP/1.1 403 Forbidden
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 98

{
    "type": "https://example-api.com/problem/user-locked",
    "title": "User has been locked",
    "status": 403
}

< HTTP/1.1 500 Internal Server Error
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 35

{
    "type": "about:blank",
    "status": 500
}

Just look at those beautiful structured error responses!

Next steps

Now that you've learnt all about the clarity that problem details can bring to your error responses, I hope you're excited to start using them in your own APIs!

Want to learn more about how you can build robust APIs with Express? Take a look at some of my other articles:

How to create an error handler for your Express API

Simon Plenderleith — Mon, 07 Dec 2020 09:30:00 +0000

Express provides a default error handler, which seems great until you realise that it will only send an HTML formatted error response. This is no good for your API as you want it to always send JSON formatted responses. You start handling errors and sending error responses directly in your Express route handler functions.

Before you know it, you have error handling code that’s logging errors in development to help you debug, and doing extra handling of the error object in production so you don’t accidentally leak details about your application’s internals. Even with just a few routes, your error handling code is getting messy, and even worse, it’s duplicated in each of your route handler functions. Argh!

Wouldn’t it be great if you could send JSON error responses from your API and have your error handling code abstracted into one place, leaving your route handlers nice and tidy? The good news is that you can, by creating your own error handler middleware.

In this article you’ll learn how to create an error handler middleware function which behaves in a similar way to Express’ default error handler, but sends a JSON response. Just the error handler that your API needs!

Jump links

Getting errors to error handler middleware
Creating an error handler
- Error handler concerns
- Error handler middleware function
- Error handler helper functions
Applying the error handler middleware
- Example error response
Next steps

Getting errors to error handler middleware

The Express documentation has examples of an error being thrown e.g. throw new Error('..'), however this only works well when all of your code is synchronous, which is almost never in Node.js. If you do throw error objects in your Express application, you will need to be very careful about wrapping things so that next() is always called and that the error object is passed to it.

There are workarounds for error handling with asynchronous code in Express – where Promise chains are used, or async/await – however the fact is that Express does not have proper support built in for asynchronous code.

Error handling in Express is a broad and complex topic, and I plan to write more about this in the future, but for the purpose of this article we’ll stick with the most reliable way to handle errors in Express: always explicitly call next() with an error object e.g.

app.get("/user", (request, response, next) => {
    const sort = request.query.sort;

    if (!sort) {
        const error = new error("'sort' parameter missing from query string.");

        return next(error);
    }

    // ...
});

Creating an error handler

You can create and apply multiple error handler middleware in your application e.g. one error handler for validation errors, another error handler for database errors, however we’re going to create a generic error handler for our API. This generic error handler will send a JSON formatted response, and we’ll be applying the best practices that are detailed in the official Express Error handling guide. If you want, you’ll then be able to build on this generic error handler to create more specific error handlers.

Ok, let’s get stuck in!

Error handler concerns

Here are the things we’re going to take care of with our error handler middleware:

Log an error message to standard error (stderr) – in all environments e.g. development, production.
Delegate to the default Express error handler if headers have already been sent – The default error handler handles closing the connection and failing the request if you call next() with an error after you’ve started writing the response, so it’s important to delegate to the default error handler if headers have already been sent (source).
Extract an error HTTP status code – from an Error object or the Express response object.
Extract an error message – from an Error object, in all environments except production so that we don’t leak details about our application or the servers it runs on. In production the response body will be empty and the HTTP status code will be what clients use to determine the type of error that has occurred.
Send the HTTP status code and the error message as a response – the body will be formatted as JSON and we’ll send a Content-Type: application/json header.
Ensure remaining middleware is run – we might end up adding middleware after our error handler middleware in future e.g. to send request timing metrics to another server, so it’s important that our error handler middleware calls next(), otherwise we could end up in debugging hell in the future.

Error handler middleware function

In Express, error handling middleware are middleware functions that accept four arguments: (error, request, response, next). That first error argument is typically an Error object which the middleware will then handle.

As we saw above, there are quite a few concerns that our error handler needs to cover, so let’s first take a look at the error handler middleware function. Afterwards we’ll dig into the helper functions which it calls.

// src/middleware/error-handler.js

const NODE_ENVIRONMENT = process.env.NODE_ENV || "development";

/**
 * Generic Express error handler middleware.
 *
 * @param {Error} error - An Error object.
 * @param {Object} request - Express request object
 * @param {Object} response - Express response object
 * @param {Function} next - Express `next()` function
 */
function errorHandlerMiddleware(error, request, response, next) {
    const errorMessage = getErrorMessage(error);

    logErrorMessage(errorMessage);

    /**
     * If response headers have already been sent,
     * delegate to the default Express error handler.
     */
    if (response.headersSent) {
        return next(error);
    }

    const errorResponse = {
        statusCode: getHttpStatusCode({ error, response }),
        body: undefined
    };

    /**
     * Error messages and error stacks often reveal details
     * about the internals of your application, potentially
     * making it vulnerable to attack, so these parts of an
     * Error object should never be sent in a response when
     * your application is running in production.
     */
    if (NODE_ENVIRONMENT !== "production") {
        errorResponse.body = errorMessage;
    }

    /**
     * Set the response status code.
     */
    response.status(errorResponse.statusCode);

    /**
     * Send an appropriately formatted response.
     *
     * The Express `res.format()` method automatically
     * sets `Content-Type` and `Vary: Accept` response headers.
     *
     * @see https://expressjs.com/en/api.html#res.format
     *
     * This method performs content negotation.
     *
     * @see https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation
     */
    response.format({
        //
        // Callback to run when `Accept` header contains either
        // `application/json` or `*/*`, or if it isn't set at all.
        //
        "application/json": () => {
            /**
             * Set a JSON formatted response body.
             * Response header: `Content-Type: `application/json`
             */
            response.json({ message: errorResponse.body });
        },
        /**
         * Callback to run when none of the others are matched.
         */
        default: () => {
            /**
             * Set a plain text response body.
             * Response header: `Content-Type: text/plain`
             */
            response.type("text/plain").send(errorResponse.body);
        },
    });

    /**
     * Ensure any remaining middleware are run.
     */
    next();
}

module.exports = errorHandlerMiddleware;

Error handler helper functions

There are three helper functions which are called by our error handler middleware function above:

getErrorMessage()
logErrorMessage()
getHttpStatusCode()

The benefit of creating these individual helper functions is that in future if we decide to create more specific error handling middleware e.g. to handle validation errors, we can use these helper functions as the basis for that new middleware.

Each of these helper functions are quite short, but they contain some important logic:

// src/middleware/error-handler.js

/**
 * Extract an error stack or error message from an Error object.
 *
 * @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error
 *
 * @param {Error} error
 * @return {string} - String representation of the error object.
 */
function getErrorMessage(error) {
    /**
     * If it exists, prefer the error stack as it usually
     * contains the most detail about an error:
     * an error message and a function call stack.
     */
    if (error.stack) {
        return error.stack;
    }

    if (typeof error.toString === "function") {
        return error.toString();
    }

    return "";
}

/**
 * Log an error message to stderr.
 *
 * @see https://nodejs.org/dist/latest-v14.x/docs/api/console.html#console_console_error_data_args
 *
 * @param {string} error
 */
function logErrorMessage(error) {
    console.error(error);
}

/**
 * Determines if an HTTP status code falls in the 4xx or 5xx error ranges.
 *
 * @param {number} statusCode - HTTP status code
 * @return {boolean}
 */
function isErrorStatusCode(statusCode) {
    return statusCode >= 400 && statusCode < 600;
}

/**
 * Look for an error HTTP status code (in order of preference):
 *
 * - Error object (`status` or `statusCode`)
 * - Express response object (`statusCode`)
 *
 * Falls back to a 500 (Internal Server Error) HTTP status code.
 *
 * @param {Object} options
 * @param {Error} options.error
 * @param {Object} options.response - Express response object
 * @return {number} - HTTP status code
 */
function getHttpStatusCode({ error, response }) {
    /**
     * Check if the error object specifies an HTTP
     * status code which we can use.
     */
    const statusCodeFromError = error.status || error.statusCode;
    if (isErrorStatusCode(statusCodeFromError)) {
        return statusCodeFromError;
    }

    /**
     * The existing response `statusCode`. This is 200 (OK)
     * by default in Express, but a route handler or
     * middleware might already have set an error HTTP
     * status code (4xx or 5xx).
     */
    const statusCodeFromResponse = response.statusCode;
    if (isErrorStatusCode(statusCodeFromResponse)) {
        return statusCodeFromResponse;
    }

    /**
     * Fall back to a generic error HTTP status code.
     * 500 (Internal Server Error).
     *
     * @see https://httpstatuses.com/500
     */
    return 500;
}

Now that we've created our error handler middleware, it's time to apply it in our application.

Applying the error handler middleware

Here is a complete example Express API application. It uses the http-errors library to add an HTTP status code to an error object and then passes it to the next() callback function. Express will then call our error handler middleware with the error object.

// src/server.js

const express = require("express");
const createHttpError = require("http-errors");

const errorHandlerMiddleware = require("./middleware/error-handler.js");

/**
 * In a real application this would run a query against a
 * database, but for this example it's always returning a
 * rejected `Promise` with an error message.
 */
function getUserData() {
    return Promise.reject(
        "An error occurred while attempting to run the database query."
    );
}

/**
 * Express configuration and routes
 */

const PORT = 3000;
const app = express();

/**
 * This route demonstrates:
 *
 * - Catching a (faked) database error (see `getUserData()` function above).
 * - Using the `http-errors` library to extend the error object with
 * an HTTP status code.
 * - Passing the error object to the `next()` callback so our generic
 * error handler can take care of it.
 */
app.get("/user", (request, response, next) => {
    getUserData()
        .then(userData => response.json(userData))
        .catch(error => {
            /**
             * 500 (Internal Server Error) - Something has gone wrong in your application.
             */
            const httpError = createHttpError(500, error);

            next(httpError);
        });
});

/**
 * Any error handler middleware must be added AFTER you define your routes.
 */
app.use(errorHandlerMiddleware);

app.listen(PORT, () =>
    console.log(`Example app listening at http://localhost:${PORT}`)
);

You can learn how to use the http-errors library in my article on 'How to send consistent error responses from your Express API'.

Example error response

Here's an example GET request with cURL to our /user endpoint, with the corresponding error response generated by our error handler middleware (in development):

$ curl -v http://localhost:3000/user

> GET /user HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.68.0
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< X-Powered-By: Express
< Vary: Accept
< Content-Type: application/json; charset=utf-8
< Content-Length: 279
< Connection: keep-alive
< 

{"message":"InternalServerError: An error occurred while attempting to run the database query.\n at /dev/example/src/server.js:262:22\n at processTicksAndRejections (internal/process/task_queues.js:97:5)"}

Next steps

You might have noticed that we're not sending a response body in production. This is due to the fact that sending the error object's message or call stack would leak details about our application, making it vulnerable to potential attackers. As we've created a generic error handler middleware here, the best that we can do is send back a suitable error HTTP status code in production.

If you know the types of the errors that your error handler middleware will be receiving (which you can check for example with error instanceof ErrorClass), you could define some production safe error messages which correspond with those error types. These production safe error messages could then be sent in the response body, providing more useful context about the error which has occurred. Give it a try!

How to handle request validation in your Express API

Simon Plenderleith — Fri, 20 Nov 2020 10:15:00 +0000

Let’s be real, adding request validation to your Express based API isn’t particularly exciting, but you know that it is an important foundational part of building an API, so you sit down to figure out what you’re going to do.

You try and pick a validation library, but it’s more difficult than you expect because they’re all quite different from each other, and it’s not clear what benefits one has over another. Perhaps you start to build your own custom validation, but it quickly starts to feel very messy. You just want to be able to put something reliable in place for validation and move on to building the interesting stuff in your API. You wonder to yourself, is adding request validation to an Express API really this difficult?!

In this article I’ll introduce you to JSON Schema, which allows you to describe the format that you expect data to be in and then validate data against it. I’ll then show you how to use JSON Schema to validate requests to your Express based API and send validation errors back in the response. By the time we’re done you won’t have to waste time figuring out how to handle request validation ever again.

Jump links

Getting to grips with JSON Schema
Why should I use JSON Schema and not validation library X?
How to integrate validation with JSON schemas into your application
Pulling it all together
Wrapping things up
Handy JSON Schema links

Getting to grips with JSON Schema

JSON Schema is very powerful, but for now we’ll only use a few of its features so that we can get comfortable with how it works.

Here’s an example JSON schema showing some of the types and keywords which you can use to describe how an object should be structured:

{
    "type": "object",
    "required": ["name"],
    "properties": {
        "name": {
            "type": "string",
            "minLength": 1
        },
        "age": {
            "type": "integer",
            "minimum": 18
        }
    }
}

The nice thing about JSON Schema is that it tends to be self-documenting, which is great for us humans who want to quickly understand what’s going on. At the same time, JSON schemas are also machine readable, meaning that we can use a JSON Schema validator library to validate the data which our application receives against a schema.

I recommend you finish reading this article before diving deeper into all of the features of JSON Schema, but if you’re keen to learn more about them right now you can jump to the handy links I’ve collected at the end.

Why should I use JSON Schema and not validation library X?

Here are the things which I think make JSON Schema a uniquely ideal tool for data validation in your Node.js application.

No library, framework or language lock-in

There are JSON Schema validation libraries available for every popular programming language.

JSON Schema doesn’t tie you to a library or a framework e.g. Joi, Yup, validate.js. These Node.js libraries all take their own approach to defining validation rules and error messages, so the things you need to learn to use them will become obsolete if they stop being developed or become deprecated.

This almost happened with the Joi validation library earlier this year, when the lead maintainer of the Hapi.js framework which it was a part of announced plans to deprecate all modules. Fortunately Joi itself seems to have been adopted by some kind souls, but it should make you think twice about committing to a specific library when more widely supported tooling is available.

Move between Node.js frameworks, or even languages, and take your schemas with you

Because JSON schemas aren’t tied to a framework, it’s one less thing to worry about if you decide to migrate away from Express to something else e.g. Fastify, which has built in support for request validation and response serialization with JSON Schema.

Because JSON Schema itself is language agnostic and widely supported, if you ever decide to rewrite your Node.js applications in a completely different language e.g. Go or Rust, you won’t need to rewrite all of the validation – you can take your JSON schemas with you!

Active and supportive community

There is an active community of folks on Slack who are very willing to help you out. The official JSON Schema website has a link which you can use to join.

JSON Schema is on a path to becoming a standard

JSON Schema is on its way to becoming a standard. It’s currently defined in a collection of IETF Internet-Draft documents, with the intention that they will be adopted by an IETF Working Group and shepherded through to RFC status, making them eligible to become an Internet Standard.

How to integrate validation with JSON schemas into your application

First things first, parse that JSON request body

Your application will need to be able to handle POST requests with a JSON body, where the Content-Type header is application/json. Here’s an example of how you can make a request like this on the command line with cURL:

curl --request POST \
  --url http://localhost:3000/user \
  --header 'Content-Type: application/json' \
  --data '{
    "first_name": "Test",
    "last_name": "Person",
    "age": true
}'

The package most commonly used for handling the JSON body of a POST request in Express based applications is body-parser. If you already have it installed and configured in your application, that’s great, and you can skip on to the next section, otherwise let’s get it set up:

npm install body-parser

And then add it into your application:

const bodyParserMiddleware = require("body-parser");

/**
 * You can add the `body-parser` middleware anywhere after
 * you've created your Express application, but you must do
 * it before you define your routes.
 *
 * By using the `json()` method, if a request comes into your
 * application with a `Content-Type: application/json` header,
 * this middleware will treat the request body as a JSON string.
 * It will attempt to parse it with `JSON.parse()` and set the
 * resulting object (or array) on a `body` property of the request
 * object, which you can access in your route handlers, or other
 * general middleware.
 */
app.use(bodyParserMiddleware.json());

Integrate Ajv (Another JSON Schema Validator) into your application

The Ajv (Another JSON Schema Validator) library is the most popular JSON Schema validator written for JavaScript (Node.js and browser). You can use Ajv directly, however for an Express based API it’s nice to be able to use middleware to validate request data which has been sent to an endpoint before that endpoint’s route handler is run. This allows you to prevent things like accidentally storing invalid data in your database. It also means that you can handle validation errors and send a useful error response back to the client. The express-json-validator-middleware package can help you with all of this.

Before we integrate this middleware into our application, let’s get it installed:

npm install express-json-validator-middleware

Once you have it installed you need to require it in your application and configure it:

const { Validator } = require("express-json-validator-middleware");

/**
 * Create a new instance of the `express-json-validator-middleware`
 * `Validator` class and pass in Ajv options.
 *
 * `allErrors` = Check all schema rules and collect all errors.
 * The Ajv default is for it to return after the first error.
 *
 * @see https://github.com/ajv-validator/ajv/blob/master/docs/api.md#options
 */
const { validate } = new Validator({ allErrors: true });

Using a JSON schema to validate a response

In this next code snippet we’re going to do two things:

Define a JSON schema which describes the data which we expect to receive when a client calls our API endpoint to create a new user. We want the data to be an object which always has a first_name and a last_name property. This object can optionally include an age property, and if it does, the value of that property must be an integer which is greater than or equal to 18.
We’re going to use the user schema which we’ve defined to validate requests to our POST /user API endpoint.

const userSchema = {
    type: "object",
    required: ["first_name", "last_name"],
    properties: {
        first_name: {
            type: "string",
            minLength: 1,
        },
        last_name: {
            type: "string",
            minLength: 1,
        },
        age: {
            type: "integer",
            minimum: 18,
        },
    },
};

/**
 * Here we're using the `validate()` method from our `Validator`
 * instance. We pass it an object telling it which request properties
 * we want to validate, and what JSON schema we want to validate the
 * value of each property against. In this example we are going to
 * validate the `body` property of any requests to the POST /user
 * endpoint against our `userSchema` JSON schema.
 *
 * The `validate()` method compiles the JSON schema with Ajv, and
 * then returns a middleware function which will be run every time a
 * request is made to this endpoint. This middleware function will
 * take care of running the validation which we've configured.
 *
 * If the request `body` validates against our `userSchema`, the
 * middleware function will call the `next()` Express function which
 * was passed to it and our route handler function will be run. If Ajv
 * returns validation errors, the middleware will call the `next()`
 * Express function with an error object which has a `validationErrors`
 * property containing an array of validation errors, and our route handler
 * function will NOT be run. We'll look at where that error object gets
 * passed to and how we can handle it in the next step.
 */
app.post(
    "/user",
    validate({ body: userSchema }),
    function createUserRouteHandler(request, response, next) {
        /**
         * Normally you'd save the data you've received to a database,
         * but for this example we'll just send it back in the response.
         */
        response.json(request.body);

        next();
    }
);

Sending validation errors in a response

In the previous code snippet we learnt how to integrate the express-json-validator-middleware so that it will validate a request body against our user schema. If there are validation errors, the middleware will call the next() Express function with an error object. This error object has a validationErrors property containing an array of validation errors. When an error object is passed to a next() Express function, it automatically stops calling all regular middleware for the current request, and starts calling any error handler middleware which has been configured.

The difference between error handler middleware and regular middleware is that error handler middleware functions specify four parameters instead of three i.e. (error, request, response, next). To be able to handle the error created by express-json-validator-middleware and send a useful error response back to the client we need to create our own error handler middleware and configure our Express application to use.

/**
 * Error handler middleware for handling errors of the
 * `ValidationError` type which are created by
 * `express-json-validator-middleware`. Will pass on
 * any other type of error to be handled by subsequent
 * error handling middleware.
 *
 * @see https://expressjs.com/en/guide/error-handling.html
 *
 * @param {Error} error - Error object
 * @param {Object} request - Express request object
 * @param {Object} response - Express response object
 * @param {Function} next - Express next function
 */
function validationErrorMiddleware(error, request, response, next) {
    /**
     * If the `error` object is not a `ValidationError` created
     * by `express-json-validator-middleware`, we'll pass it in
     * to the `next()` Express function and let any other error
     * handler middleware take care of it. In our case this is
     * the only error handler middleware, so any errors which
     * aren't of the `ValidationError` type will be handled by
     * the default Express error handler.
     *
     * @see https://expressjs.com/en/guide/error-handling.html#the-default-error-handler
     */
    const isValidationError = error instanceof ValidationError;
    if (!isValidationError) {
        return next(error);
    }

    /**
     * We'll send a 400 (Bad Request) HTTP status code in the response.
     * This let's the client know that there was a problem with the
     * request they sent. They will normally implement some error handling
     * for this situation.
     *
     * We'll also grab the `validationErrors` array from the error object
     * which `express-json-validator-middleware` created for us and send
     * it as a JSON formatted response body.
     *
     * @see https://httpstatuses.com/400
     */
    response.status(400).json({
        errors: error.validationErrors,
    });

    next();
}

This allows us to send back error responses like this when there is an error validating the request body against our user schema:

< HTTP/1.1 400 Bad Request
< Content-Type: application/json; charset=utf-8
< Content-Length: 187

{
    "errors": {
        "body": [
            {
                "keyword": "minimum",
                "dataPath": ".age",
                "schemaPath": "#/properties/age/minimum",
                "params": {
                    "comparison": ">=",
                    "limit": 18,
                    "exclusive": false
                },
                "message": "should be >= 18"
            }
        ]
    }
}

Pulling it all together

Here are all of the code snippets in this article combined into a complete Express API application:

const express = require("express");
const bodyParserMiddleware = require("body-parser");

const {
    Validator,
    ValidationError,
} = require("express-json-validator-middleware");

const { validate } = new Validator({ allErrors: true });

function validationErrorMiddleware(error, request, response, next) {
    const isValidationError = error instanceof ValidationError;
    if (!isValidationError) {
        return next(error);
    }

    response.status(400).json({
        errors: error.validationErrors,
    });

    next();
}

const userSchema = {
    type: "object",
    required: ["first_name", "last_name"],
    properties: {
        first_name: {
            type: "string",
            minLength: 1,
        },
        last_name: {
            type: "string",
            minLength: 1,
        },
        age: {
            type: "integer",
            minimum: 18,
        },
    },
};

const app = express();
app.use(bodyParserMiddleware.json());

app.post(
    "/user",
    validate({ body: userSchema }),
    function createUserRouteHandler(request, response, next) {
        response.json(request.body);

        next();
    }
);

app.use(validationErrorMiddleware);

const PORT = process.env.PORT || 3000;

app.listen(PORT, () =>
    console.log(`Example app listening at http://localhost:${PORT}`)
);

Note: For the purpose of this article I’ve combined everything into one block of code, but in a real application I would recommend separating the concerns into separate files.

Wrapping things up

You might have guessed from this article that I’m a big fan of JSON Schema. I think that it’s an excellent way to approach request validation, and I hope that you’re now ready to give it a try in your Express based applications.

In my next article I’ll be showing you how you to transform that raw errors array from Ajv into an even more helpful error response by applying the "problem detail" specification. If you want to know when I publish this new article hit the 'Follow' button at the top of this page!

Handy JSON Schema links

Understanding JSON Schema book – An excellent free online book which will teach you the fundamentals and help you make the most of JSON Schema (also available in PDF format).
JSON Schema Specification Links – The latest specifications for JSON Schema.
ajv-errors – An Ajv plugin for defining custom error messages in your schemas.
fluent-schema – Writing large JSON schemas is sometimes overwhelming, but this powerful little library allows you to write JavaScript to generate them.

How to restore changes which you’ve reverted from your main git branch

Simon Plenderleith — Thu, 12 Nov 2020 17:59:56 +0000

Reverting code changes and restoring them with git is one of those things that feels like it’s going to be straightforward, but often ends up turning your git history into a hot mess. A messy history is one thing, but not even being able to restore the code changes you reverted? That’s a nightmare!

When you start looking at the documentation for git you’ll probably end up despairing at how complicated it seems to do something which you know should be simple. Wouldn’t it be great if you could revert and restore changes whenever you need to with git? The good news is that it’s not difficult to do when you know how, but first we need to understand why your code changes aren’t being restored in the way that you expect them to be.

Why aren’t my code changes being restored?!

If you’ve ever reverted code changes from your main branch and then later tried to restore them, this git workflow is probably familiar to you:

The reason that the code changes made before the bug fix are "missing" from main when you try to merge feature-branch in again is because the commits containing those changes were already merged in to main earlier.

To bring the changes back in to main which you reverted, as well as any bug fix, you need to create a fresh branch and revert the commit which reverted the changes. Huh?! Let’s head to the command line and see what that looks like.

Put ya revert down flip it and revert it

Ok, now for the good stuff: the git commands which will restore order to the universe and let you get on with your life. The following commands assume you’re in a situation where your main branch has had feature-branch merged in, and that the code changes introduced by that merge have then been reverted.

# Make sure your local main branch is up-to-date.
git checkout main
git pull

# Create a new branch off of main.
# This is where you're going to bring back the changes which were reverted.
git checkout -b restore-feature-x

# You need to look at your git log and copy the SHA hash for the commit that
# you created when you reverted the changes e.g. c4780a2bf0e22a93ece0b42cf5e42a9a21cfcbca
# You'll use this commit SHA hash which you've copied in the next command.
git log

# Revert the commit where you reverted the changes.
git revert COMMIT_SHA_HASH_YOU_JUST_COPIED

# This branch will now contain the changes which you originally reverted,
# and you can proceed to make any bug fixes or further changes that are required.
#
# You can diff this branch against the main branch to confirm that when you merge
# this branch in, main will contain all the changes that you want it to.
git diff main..

I recommend that you follow this one simple rule for restoring changes that you’ve reverted with git: revert the revert, every time. If you don’t, you’re very likely to end up fighting git, which nobody wants to do, not even on a good day.

Notes from NodeConf Remote 2020

Simon Plenderleith — Mon, 09 Nov 2020 11:36:38 +0000

NodeConf Remote 2020 was held on November 2nd – 6th as a free online event, standing in for the annual NodeConf EU conference, which for obvious reasons couldn’t be held in person due to the pandemic. I tuned in when I could during the two days of excellent talks and I wrote up notes during the talks I watched.

Fair warning: The talk notes I’m sharing here are in a fairly raw state and provided with no warranty – I tried to make sure I noted down all the details accurately, but I can’t guarantee that it’s all 100% correct!

You can watch all of the talks from NodeConf Remote 2020 on the NearForm YouTube channel.

Jump links

Talk: Aaaaaaaaaaaaaah, They’re Here! ES Modules in Node.JS
Talk: Can we double HTTP client throughput?
Talk: AsyncLocalStorage: usage and best practices
Talk: Examining Observability in Node.js
Talk: Node.js startup performance

Talk: Aaaaaaaaaaaaaah, They’re Here! ES Modules in Node.JS

Speaker

Gil Tayar (@giltayar)

One big takeaway

The ECMAScript modules (ESM) implementation in Node.js is in a very mature state, so it’s a great time to start gradually migrating your packages and applications from CommonJS modules. You might also potentially be able to ditch Babel too if you’re only using it for import/export support.

Talk abstract

Yes, they’re here. Node v13.2.0 marked the first version of Node.JS where ESM support is unflagged, meaning you can start using ES Modules. It’s been a long, four year journey from defining them in the spec (June 2015!) till they could be used in Node.JS (November 2019).

Why did it take so long? What were the major hurdles? Should we migrate? How does the migration path look like? Are they really better than CommonJS Modules? What is in store for the future?

Gil Tayar, a former member of the Node.JS Modules Working Group, and now just a passionate observer of it, will try and navigate these confusing waters, and hopefully make you understand why, when, and how to migrate your Node.JS code to use ES Modules.

My notes

Unfortunately I missed a big chunk of this talk, but my main takeaway was the reasons why ECMAScript modules (ESM) are better than CommonJS (CJS) modules or Babel transforms. It’s because they’re:

Strict
Browser compatible – whoop, standards!
Statically parsed
Async + supports top-level await
Native 🎉

Talk: Can we double HTTP client throughput?

Speaker

Matteo Collina (@matteocollina)

One big takeaway

If you’re running Node.js microservices which make HTTP requests to each other, to keep things fast, you should 1. always create an HTTP agent with keepAlive, 2. use HTTP pipelining. Undici can take care of both of these things for you, and is capable of a much higher throughput than the Node.js core http module.

Talk abstract

The Node.js HTTP client is a fundamental part of any application, yet many think it cannot be improved. I took this as a challenge and I’m now ready to present a new HTTP client for Node.js, undici, that doubles the throughput of your application.

The story behind this improvement begins with the birth of TCP/IP and it is rooted in one of the fundamental limitations of networking: head-of-line blocking (HOL blocking). HOL blocking is one of those topics that developers blissfully ignore and yet it deeply impacts the runtime experience of the distributed applications that they build every day. Undici is a HTTP/1.1 client that avoids HOL blocking by using keep-alive and pipelining, resulting in a doubling of your application throughput.

My notes

I missed a bunch of this talk too 🙈 I’m planning to watch the full talk video (link below) to learn more about the TCP fundamentals which affect HTTP request performance in Node.js, but here’s what I noted live on the day:

Microservices typically communicate with each other over HTTP/1.1 – without any tuning, requests can get slow ← I’ve personally experienced this in projects I’ve worked on in the past.
To have decent request throughput you should always create an HTTP agent with keep alive enabled – this allows for connection reuse between requests.
You should also use HTTP pipelining so you can send concurrent HTTP requests over a single connection.
Undici HTTP/1.1 client allows you to create a "pool" which you can then make requests through. Using Undici with pool + pipelining is FAST – over three times throughput of node http agent with keep alive 🚀
Main takeaways:
- Always use an http(s).Agent
- Undici can drastically reduce the overhead of your distributed system

Talk: AsyncLocalStorage: usage and best practices

Speaker

Vladimir de Turckheim (@poledesfetes)

One big takeaway

Node.js is constantly evolving and there are some powerful new APIs being implemented that don’t always make headlines. The AsyncLocalStorage API is one of those, and I’m hoping I’ll have an opportunity soon to give it a try.

Talk abstract

During Spring, a curious API was added to Node.js core: AsyncLocalStorage. Most Node.js users are not familiar with the power of such tool.

That’s too bad: it can be used to drastically improve an application’s code and allow building powerful tooling around Node.js applications.

So, let’s discover what this API is and how to use it to leverage the unlimited powers of AsyncLocalStorage (ok, I might have exagerated it a bit here).

My notes

Given that efforts are being made to align Node.js more closely with browser standards, it seems odd that the AsyncLocalStorage API is named as it is: it has nothing to do with the browser Local Storage API 🤔

In threaded languages e.g. PHP

A request enters the process → a thread is created
The request has its own thread – it’s basically a thread-singleton

In single-threaded world e.g. Node.js, a single thread handles multiple requests.

Exception handling is weird in Node.js, nextTick async operations will lose the call stack.

Let’s create contexts for asynchronous environments: AsyncLocalStorage – asynchronous-proof store, can create async contexts for you to use.

Basic example:

const { AsyncLocalStorage } = require("async_hooks");

const context = new AsyncLocalStorage();

context.run(new Map(), () => {
    // Do stuff
});

You can always know what the current request context is. Using process.on('uncaughtException'), which is normally advised against, however AsyncLocalStorage allows us to create an application state. Allows for unified error handling.

Other use cases:

User management – store current user and use in DB abstraction
Monitoring – build your own monitoring tool to log/track/monitor what your apps
Single DB transaction for HTTP request

Key points:

Memory safe and pretty fast.
It’s experimental, but production ready.
It won’t work with queues.
Don’t share AsyncLocalStorage instances.
Don’t create too many AsyncLocalStorage instances.
Consider the store as immutable if using basic types
Use a Map for everything else
Use the run method, but enterWith only if you need to
Call exit() if you are not sure if it will be GCed

Talk: Examining Observability in Node.js

Speaker

Liz Parody (@lizparody23)

One big takeaway

Observing = Exposing internal state of an application so it can be viewed externally and continuously analysed.

Monitoring = Waiting for problems to happen.

Talk abstract

Imagine your productivity and confidence developing web apps without chrome dev tools. Many do exactly that with Node.js.

It is important to observe and learn what’s happening in your app to stay competitive and create the most performant and efficient Node.js applications, following the best practices.

In this talk, we will explore useful tools to examine your Node.js applications and how observability will speed up development, produce better code while improving reliability and uptime.

My notes

What is observability? It’s a measure of how well the internal state of a system can be determined from the outside.

Observing or asking questions from outside the system – no new code should be needed.

Tools to the rescue!

Software becoming exponentially more complex: microservices, Docker, Kubernetes etc. Great for products, hard for humans.

Big growth in observability tools, but hard to choose one.

Why is observability important? Just monitoring for problems not enough – new issues could be "unknown unknowns".

A good observability tool:

Helps you find where problem is
Doesn’t add overhead to app
Has great security
Flexible integrations
Doesn’t require code changes

Observing = Exposing internal state to be externally accessed.

Monitoring = Waiting for problems to happen.

Layers of observability:

Cloud/Network
Service/Host
Node.js
Internals

Node.js + Internals tools

A. Node.js Performance Hooks

Performance monitoring should be part of development process, not an afterthought when problems arise.

Using perf_hooks module allow you to collect performance metrics from the running Node.js application.

Requires code to implement in your application.

B. Profiling

Flame graphs can be very useful, but they’re very intensive to collect so cannot be captured in production.

C. Trace Events

Enable with —trace-event—categories trace_events

node, node.async_hooks, v8 – enabled by default

To get the output of several events —trace-event-enabled

Connect to the locally running application: chrome://tracing

Tracing has less overhead, but it can become tricky to work with as it exposes a lot of Node.js internals.

D. Heap Snapshot

Is a static snapshot of memory usage details at point in time, glimpse into V8 heap usage

Useful for finding and fixing memory + performance issues in Node.js applications.

Built in heap snapshots signal flag --heapshot-signal

Chrome DevTools allow you to compare snapshots.

E. The V8 Inspector

Chrome DevTools was integrated directly into Node.js a few years ago.

--inspect flag, listens by default on 127.0.0.1:9229

--inspect-brk for using the inspector with breakpoints

Go to chrome://inspect so you can connect DevTools to your Node.js application

Allows you to… edit code on-the-fly, diagnose problems quickly, access sourcemaps for transpiled code, LiveEdit, console evaluation, sampling JavaScript profiler with flame graph, heap snapshot inspection, async stacks for native promises.

Only suitable for development, not for production.

Problems with Node.js internals tools

Tells you there’s a problem, but not where.

Not easy to implement, not enough information.

Not presented in user-friendly way, data overload.

Significant overhead, not viable in production.

External Tools for Node.js Observability

A. Blocked Library

Available in Node.js 8+. Helps you checked if event loop is blocked, provides stacktrace pointing to blocking function. blocked() function reports every value over configured threshold.

B. New Relic (hosted service)

Offers application performance monitoring (APM).

C. DataDog (hosted service)

Similar service to New Relic.

D. Instana (hosted service)

APM for microservices – trace every distributed request, map all service dependencies, profile every production process.

E. Dynatrace (hosted service)

Another APM, with a focus on "advanced observability".

F. Google Cloud Stackdriver

Another APM, for Google Cloud and Amazon Web Services.

Problems with APMs

They have to be integrated into your applications and can cause a significant amount of overhead.

Accuracy might be questionable as the APM modules themselves can have

N|Solid

Native C++ agent which runs alongside your application, doesn’t require integration with your application code, resulting in minimal overhead on application performance. [N|Solid is a product of NodeSource, the speaker’s employer]

Talk: Node.js startup performance

Speaker

Joyee Cheung (@JoyeeCheung)

One big takeaway

There is a tremendous amount of important work being done in the background by developers like Joyee who are working on the Node.js core. If you want to get a deeper understanding of what Node.js is doing under the hood, and why improvements to the Node.js core are so important, I thoroughly recommend that you watch Joyee’s talk.

Talk abstract

In this talk, we will break down how Node.js spends its time starting up from scratch, and look into recent changes in the Node.js code base that have improved the startup performance by using different tricks and various V8 APIs.

My notes

The journey of Node.js startup performance

Refactoring to avoid unnecessary work
Implement code caching
Integrating V8 startup snapshot

Used to take ~60ms on a modern server.

After optimisations ended, startup time on same server dropped to 21ms.

Between Node.js v10 – v15 – startup time time reduced by 40 – 50%

Overview of the Node.js bootstrap process

Around half of the Node.js core is written in JavaScript, the rest in C++.

Initialize the process e.g. processing command line flags, setting up signal handlers, creating default event loop etc. (C++)
Initialize V8 isolate (C++)
- v8::Isolate is instance of the v8 JavaScript engine
- Encapsulates the JS heap, microtask queue, pending exceptions…
Initialize V8 context (JavaScript)
- Sandboxed execution context
- Encapsulates JavaScript builtins (primordials) e.g. globalThis, Array, Object
- Node.js copies original JS built-ins at beginning of bootstrap for built-in modules to use.
- In Node.js userland JS executed in main V8 context by default, shares same context as one used by built-ins of Node.js
Initialize Node.js environment (JavaScript and C++)
- Initialize runtime-independent states (JavaScript)
- Initialize event loop (C++)
- Initialize V8 inspector (C++) – can only debug JavaScript once the inspector is initialized
- Initialize runtime dependent states (JavaScript)
Load main script (JavaScript)
- Execution from CLI (node index.js) – Create + initialize environment, select a main script → Load run_main_module.js, detect module type → Read and compile ${cwd}/index.js with CommonJS or ECMAScript module loader → Start event loop
- Execution for worker intialiized by code in main thread – Create + initialize environment, select a main script → Load worker_thread.js, setup message port and start listening → Start event loop → Compile and run the script sent from the port
Start the event loop – will be kept running until nothing is keeping it open.

Refactoring

Lazy-load builtins that are not always used
- Lots of builtin modules depend on each other
- Caveat: more time would be spent loading them on demand later
- Can be reverted when startup snapshot covers these modules
Initializing runtime states were cleanly separated as part of the refactoring work.

Code caching

This speeds up JS compilation.
Previously: parse and compile source code of JS native modules at Node.js run time and execute them to make them available as built-in modules.
Now: parse and compile source code of JS native modules at Node.js executable build time, then deserialize them from the Node.js executable in the Node.js process (run time), and execute them to make them available as built-in modules.

Refactoring for snapshot integration

This was enabled by splitting runtime initialization into two separate phases (as mentioned earlier).
Before: At Node.js process run time: Array, Object, String etc. → Runs through initialization scripts → Initialize Primordials: process, URL, Buffer etc. → Node.js process
After: At Node.js executable build time: Array, Object, String etc. → Runs through initialization scripts → Initialize Primordials: process, URL, Buffer etc. → Serialize and compile into snapshot blob. At Node.js process run time, deserialize snapshot blob from executable.
Saves quite a lot of time at startup.

Ongoing work

During this refactoring work for Node.js, contributions were made to V8
Supporting more language features in the V8 snapshot
- JSMap and JSSet rehashing (previously disabled in Node.js v8)
- Class field initializers

Future work

Userland snapshotting

Take a snapshot of an application and write it to disk
Load from file system or build into an executable
Tracking issue: https://github.com/nodejs/node/issues/35711

Questions & Answers

What’s inside the startup snapshots?

Two types:

Isolate snapshots – e.g. V8 strings, numbers
Context snapshots – e.g. objects you create

What are runtime dependent states?

Runtime dependent states = things configured with command line flags or environment variables

How to send consistent error responses from your Express API

Simon Plenderleith — Thu, 29 Oct 2020 11:12:46 +0000

When you’re building an API with Express it can be difficult to know how to send consistent error responses. The framework doesn’t seem to provide any special features for this, so you’re left to figure it out on your own. At some point you’ll probably find yourself wondering if you’re doing it "the right way".

As I mentioned in my ‘5 best practices for building a modern API with Express‘ blog post:

It’s very tempting when building an API to invent your own format for error responses, but HTTP response status codes are a great starting point as they can communicate a specific error state.

If you invent your own format you’ll have to build a bunch of extra logic into your API, and you’ll probably want to make sure it’s thoroughly tested too. Nobody wants errors in their error response code, right?! On top of that, it will also require clients – e.g. front end JavaScript – to implement additional code for handling the special format of your API’s error responses.

Wouldn’t it be nice if there was a simpler way, a tried and tested standard way, of sending error responses? As luck would have it, there is! The HTTP standard defines status codes which you can use in the responses from your API to indicate whether the request was successful, or whether an error occurred.

Here’s an example HTTP error response with the 400 status code, which indicates a ‘Bad Request’ from the client:

< HTTP/1.1 400 Bad Request
< Content-Type: text/html; charset=utf-8
< Content-Length: 138
< Date: Wed, 28 Oct 2020 20:11:07 GMT

If you want to send an error response like this, you could use Express' res.status() method:

res.status(400).end();

Follow the yellow brick road HTTP standard

By default the HTTP status code sent in a response from an Express application is 200 (OK). This tells the client that the request was successful and that they can proceed to parse and extract whatever data they require from the response. To indicate an error when sending a response, you should use an HTTP status code from one of the two error ranges defined by the HTTP standard:

Client error 4xx - The client has done something wrong.
Server error 5xx - Something has gone wrong in your application.

The MDN web docs have a great reference with all of the HTTP response status codes that are defined by the HTTP standard, along with explanations of what they mean.

Once you've figured out what error status codes your API should send in different situations, you need a way of getting those status codes into an error - this is where the http-errors library comes in.

How to create errors with the http-errors library

Setting it up

First of all you will need to install the http-errors library:

npm install http-errors

And then you'll want to require() it in your application (after you require express is fine):

const createHttpError = require("http-errors");

The http-errors library offers two different ways to create an error object.

Way #1: Specify a numeric HTTP status code

The first way is to specify a numeric HTTP status code e.g.

const error = createHttpError(400, "Invalid filter");

If you want, instead of passing an error message string, you can pass an existing error object to be extended e.g.

const error = new Error("Invalid filter");

const httpError = createHttpError(400, error);

Warning! Be careful whenever you pass an existing error as you could potentially expose sensitive details about your application if it gets sent through in an error response. As you'll see in the next section, the default error handler in Express has safeguards built in to help you avoid this, but it's important to be careful about what details you expose in your API responses.

If you want to specify extra headers to be added when the error is sent in a response, http-errors allows you to do this by passing in a properties object e.g.

const error = createHttpError(400, "Invalid filter", {
    headers: {
        "X-Custom-Header": "Value"
    }
});

Way #2: Use a named HTTP error constructor

The second way of creating an error object is to use one of the named HTTP error constructors which http-errors provides e.g.

const error = new createHttpError.BadRequest("Invalid filter");

The difference with this second approach is that you can only pass in an error message string - it doesn't allow you to pass in an existing error object or a properties object. For situations where you don't need them, I think this second approach is easier to maintain. It means don't need to keep looking up HTTP status codes to find out what they mean every time you revisit the code.

What's in these error objects?

Here are the properties which will always exist on an error object created with http-errors, along with example values:

{
    message: "Invalid filter",
    // This statusCode property is going to come in very handy!
    statusCode: 400,
    stack: `BadRequestError: Invalid filter
        at /home/simonplend/dev/express-error-responses/app.js:33:17
        at Layer.handle [as handle_request] (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/layer.js:95:5)
        at next (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/route.js:137:13)`
}

Now let's take a look at what you can do with an error object once you've created it.

The default error handler in Express

Express provides a default error handler. This error handler is called when you call the next() callback function from a middleware or route handler and you pass an error object to it e.g. next(error).

There are two important things to know about the behaviour of the default error handler in Express:

It will look for for a statusCode property on the error object (error.statusCode) - that's right, just like the one which exists on errors created with http-error. If statusCode is within the 4xx or 5xx range, it will set that as the status code of the response, otherwise it will set the status code to 500 (Internal Server Error).
In development it will send the full stack trace of the error that it receives (error.stack) in the response e.g.

BadRequestError: Invalid sort parameter, must be either: first_name, last_name
    at /home/simonplend/dev/express-error-responses/app.js:17:17
    at Layer.handle [as handle_request] (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/layer.js:95:5)
    at next (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/layer.js:95:5)
    at /home/simonplend/dev/express-error-responses/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/index.js:335:12)
    at next (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/index.js:275:10)
    at expressInit (/home/simonplend/dev/express-error-responses/node_modules/express/lib/middleware/init.js:40:5)
    at Layer.handle [as handle_request] (/home/simonplend/dev/express-error-responses/node_modules/express/lib/router/layer.js:95:5)

In production i.e. when the environment variable NODE_ENV is set to production, it will omit the stack trace and only send the name which corresponds with the HTTP status code e.g. Bad Request.

Displaying a stack trace in production is a bad thing to do: it creates a security risk as it reveals internal details about your application, which makes it more vulnerable to a potential attacker, as they will have details about how your application is structured and what libraries you are using.

Putting it all together

Ok, so we know about HTTP status codes, how to create JavaScript error objects which contain a status code, and how the default error handler in Express can use them. Let's put it all together!

const express = require("express");

/**
 * We'll be using this library to help us create errors with
 * HTTP status codes.
 */
const createHttpError = require("http-errors");

/**
 * In a real application this would run a query against a
 * database, but for this example it's always returning a
 * rejected `Promise` with an error message.
 */
function getUserData() {
    return Promise.reject(
        "An error occurred while attempting to run the database query."
    );
}

/**
 * Express configuration and routes
 */

const PORT = 3000;
const app = express();

/**
 * This example route will potentially respond with two
 * different types of error:
 * 
 * - 400 (Bad Request) - The client has done something wrong.
 * - 500 (Internal Server Error) - Something has gone wrong in your application.
 */
app.get("/user", (request, response, next) => {
    const validSort = ["first_name", "last_name"];

    const sort = request.query.sort;
    const sortIsValid = sort && validSort.includes(sort);

    if (!sortIsValid) {
        /**
         * This error is created by specifying a numeric HTTP status code.
         * 
         * 400 (Bad Request) - The client has done something wrong.
         */
        const error = new createHttpError.BadRequest(
            `Invalid sort parameter, must be either: ${validSort.join(", ")}`
        );

        /**
         * Because we're passing an error object into the `next()` function,
         * the default error handler in Express will kick in and take
         * care of sending an error response for us.
         * 
         * It's important that we return here so that none of the
         * other code in this route handler function is run.
         */
        return next(error);
    }

    getUserData()
        .then(userData => response.json(userData))
        .catch(error => {
            /**
             * This error is created by using a named HTTP error constructor.
             *
             * An existing error is being passsed in and extra headers are
             * being specified that will be sent with the response.
             *
             * 500 (Internal Server Error) - Something has gone wrong in your application.
             */
            const httpError = createHttpError(500, error, {
                headers: {
                    "X-Custom-Header": "Value",
                }
            });

            /**
             * Once again, the default error handler in Express will kick
             * in and take care of sending an error response for us when
             * we pass our error object to `next()`.
             * 
             * We don't technically need to return here, but it's
             * good to get into the habit of doing this when calling
             * `next()` so you don't end up with weird bugs where
             * an error response has been sent but your handler function
             * is continuing to run as if everything is ok.
             */
            return next(httpError);
        });
});

app.listen(PORT, () =>
    console.log(`Example app listening at http://localhost:${PORT}`)
);

Note: The default error handler in Express sends an HTML response body. If you want to send a JSON response body, you will need to write your own error handler. I'll be covering this in a future blog post!