Forem: Omonigho Kenneth Jimmy

Security Best Practices for Your Node.js Application

Omonigho Kenneth Jimmy — Wed, 17 Jul 2024 13:54:20 +0000

The widespread adoption of Node.js continues to grow, making it a prime target for XSS, DoS, and brute force attacks. Therefore, protecting your Node application from possible vulnerabilities and threats is crucial.

In this guide, we'll uncover common security threats and explore best practices for preventing them. You don't have to be a cybersecurity expert to implement fundamental security measures for your Node.js application.

So, are you ready? Let's go! 🚀

Understanding Node.js Security Risks and Implementing Security Best Practices

Let's explore the common security risks associated with Node.js applications and practical ways to mitigate them.

Threat: Injection Attacks (SQL, NoSQL)

Sensitive data lives in your database. If an attacker inserts malicious queries or commands into input fields and finds their way in, they can gain unauthorized access to users' passwords, restricted data, or worse. This type of attack is known as SQL or NoSQL injection.

For instance, suppose your application exposes an endpoint that accepts a movie category from a user input (through a query param):

https://api.my-site.com/v1/movies?category=action

This prompts the application to execute the following SQL query and fetch all released action movies from the database:

SELECT * FROM movies WHERE category = 'action' AND isReleased = true;

An attacker can construct an attack by injecting malicious SQL into the input like so:

https://api.my-site.com/v1/movies?category=action'--

This results in the following SQL query:

SELECT * FROM movies WHERE category = 'action'--' AND isReleased = true;

Because -- is a comment indicator in SQL, the rest of the statement after it, AND isReleased = true, will be ignored. As a result, all movies are displayed, including those not yet released.

Mitigate Injection Attacks

Let's now see what we can do to mitigate such attacks.

Use of Parameterized Queries

Use parameterized queries instead of directly concatenating user input into the SQL query string. These techniques, also known as prepared statements, separate SQL logic from data, ensuring that user input is treated as a value rather than executable SQL code.

For instance, based on the movie app example above, don't process the user input like this:

import * as mysql from 'mysql';

const connection = mysql.createConnection({
  // db credentials
});
connection.connect();

const userInput = 'action'--';
const query = `SELECT * FROM movies WHERE category = '${input}' AND isReleased = true`;

connection.query(query, (error, results, fields) => {
  // handle results
});

Do this instead:

import * as mysql from 'mysql';

const connection = mysql.createConnection({
  // db credentials
});
connection.connect();

const userInput = 'action'--';
const query = 'SELECT * FROM movies WHERE category = ? AND isReleased = true';

connection.query(query, [input], (error, results, fields) => {
  // handle results
});

The second example uses a prepared statement, where the ? placeholder is the input value rather than a direct concatenation of the user input into the SQL query string (as shown in the first example). The user input is, therefore, passed separately as parameters to the prepared statement, preventing it from being interpreted as SQL code.

It's noteworthy that while prepared statements can handle string and number values effectively, they cannot and should not be used to parameterize table names, column names, SQL keywords, or other structural components of an SQL statement. If we have to make our query more dynamic, adding operators and identifiers as well as values, we should use other strategies like whitelisting. This implies that every dynamic parameter should be manually set in your script and selected from that predefined set.

For instance, suppose you want to add ORDER BY dynamically into your SQL query. You can whitelist a set of allowed parameters and match the user input against this whitelist before constructing the query like so:

const orders = ["name", "price", "quantity"]; // whitelisted columns; you may use a hash map

const userInput = " UNION SELECT username, password FROM users--"; // malicious input

if (!orders.includes(userInput)) throw new Error("Invalid input!");

const query = `SELECT * FROM movies ORDER BY ${userInput}`;

This technique ensures that the query only utilizes valid and anticipated identifiers.

Sanitize User Input

Sanitizing input against database injection attacks involves removing or escaping keywords and characters that could be interpreted as part of an SQL/NoSQL query. Unsanitized user input is risky because, as demonstrated in the movie app example, attackers can leverage it to send malicious queries through your application to your database and steal important data.

There are several known npm packages available for sanitizing SQL/NoSQL queries:

sqlstring provides utility functions for escaping and formatting MySQL. For example, it provides methods like escape(), which replaces special characters within the given input string with escape characters to ensure they are treated as literal values rather than parts of the SQL syntax.
pg-format formats SQL queries in PostgreSQL applications and provides functionality similar to that of sqlstring.
express-mongo-sanitize is popularly used in MongoDB applications to prevent Operator Injection. It works by analyzing request data and removing any MongoDB operators or characters such as $gt, $lt, $eq, $ne, $regex, $where, and others that could potentially be used to execute malicious queries.

Use of ORMs/ODMs

If your application doesn't necessitate raw SQL/NoSQL, opt for Object-Relational Mappers (ORMs) like Sequelize or Object-Document Mappers (ODMs) like Mongoose for database queries. They feature built-in protection against injection attacks, such as parameterized queries, automatic escaping, and schema validation, and adhere to some security best practices.

Threat: Cross-site Scripting (XSS)

Using an input field, an attacker can store a malicious script (typically written in JavaScript) in your database through your Node.js app. The script is then served to multiple users whenever they access the affected web page or resource, and then executed in the victim's browser. This allows the attacker to steal sensitive information, manipulate web page content, or perform unauthorized actions on behalf of the user. This type of attack is known as cross-site scripting.

How to Prevent Cross-site Scripting

We'll now turn to how we can avoid cross-site scripting.

Validate User Input

Validate all user-supplied input to ensure that it conforms to expected formats and does not contain malicious code. Use libraries like joi or class-validator for data validation.

For example:

If a value is expected to be numeric, validate that it's numeric, and likewise for other data types.
If a user submits a URL that you intend to include in responses, validate that it starts with a secure protocol, like HTTP or HTTPS.

Output Encoding

Encode output data before user-controllable data is written to a page to prevent it from being interpreted as HTML or JavaScript. You can use tools like xss for this purpose.

In an HTML context, you should use HTML entities like so:

< converts to &lt; > converts to &gt;

In a JavaScript string context, non-alphanumeric values should be Unicode-escaped:

< converts to \u003c
> converts to \u003e

Use of `HTTPOnly` Cookies

If using cookies, set the HTTPOnly flag to prevent access from client-side JavaScript. You can also use the secure flag to ensure that cookies are only sent over HTTPS connections.

Setting HTTP Headers

Setting special headers in your Node.js application is essential for controlling access and ensuring data integrity, thereby enhancing protection against XSS attacks.

Use libraries such as helmet to set special HTTP headers, like X-XSS-Protection for mitigating XSS attacks.
Set the Content-Disposition header to an attachment for file downloads to prevent browsers from executing downloaded files as scripts or HTML content.

Threat: Brute Force

What would you do if you wanted to unlock a phone but didn't know the password? You might try multiple combinations, but how many combinations can you make in a day if that would take hundreds, thousands, or even millions of guesses?

What does it take to pull this off in minutes? Brute force. This type of attack is a forceful and repetitive trial-and-error method used to guess all possible combinations of a password, encryption key, or any other login information to gain unauthorized access.

How to Mitigate Brute Force

Here's how we can help prevent brute-force attacks.

Rate Limiting

Use middleware or third-party packages such as express-rate-limit to enforce rate limits and restrict the number of API calls from a single IP address within a specific time frame.

Application of Maximum Login Attempts

Implement maximum login attempts and enforce account lockout when the threshold is exceeded.

Application of Multi-Factor Authentication (MFA)

Implement Multi-Factor Authentication (MFA). Require users to provide a second form of authentication, such as a one-time passcode sent via SMS or email, in addition to their password.

Threat: Cross-Site Request Forgery (CSRF)

An attacker could build a webpage form that creates a request against your site like this:

<form action="https://my-vulnerable-site.com/email/change" method="POST">
  <input type="hidden" name="email" value="user@evil-site.com" />
</form>
<script>
  document.forms[0].submit();
</script>

Suppose a user visits the attacker's page and is logged in to your site, assuming that your site uses a session cookie for authentication. In that case, their browser automatically includes their session cookie in the request.

The vulnerable website will process the request in the normal way, treating it as though it had been made by the user, and proceed to change their email address.

Subsequently, the attacker can change the user's password and gain access to the user's account. This type of attack — enabling an attacker to manipulate users into executing unintended actions — is known as Cross-Site Request Forgery (CSRF).

Mitigating CSRF Attacks

The strongest defense against CSRF attacks is incorporating a CSRF token into relevant requests. The CSRF token strategy usually works like this:

The server sends the client a token.
The client submits a form with the token.
The server rejects the request if the token is invalid.

The token must meet the following criteria:

Unpredictable and possessing high complexity, much like session tokens in general.
Associated with the user's session.
Thoroughly verified before executing the relevant action in each case.

You could create a custom CSRF middleware leveraging package like csrf for generating and validating CSRF tokens.

Threat: Insecure Dependencies

Let's face it, most Node.js applications need third-party libraries from npm or yarn for ease and speed of development. Sadly, this convenience comes with the risk of introducing unknown vulnerabilities to your application. (Granted, most maintainers address vulnerabilities in their packages and regularly release updated versions.)

Here are some steps you can take to try to avoid introducing insecure dependencies into your application.

Constantly Update Your Dependencies

You could use npm audit or snyk to analyze your project’s dependencies tree and provide insights into any known vulnerabilities.

Verify Packages Before Use

Before integrating a library into your code, ensure it is stable and actively maintained, and review GitHub issues for that package. Additionally, examine its dependencies, reputation, reviews, and community support to guarantee the security and safety of your application. While these efforts can be demanding, they are essential to ensure the integrity of your integrated packages.

Additional Security Best Practices

Let's end with a few other general best practices to keep your Node app secure.

Logging and Monitoring

Providing visibility into system activities can enhance security best practices. I highly recommend using AppSignal to log and monitor your Node.js application because it is efficient and easy to integrate.

Check out my previous post on Async Stack Traces in Node.js where I demonstrated how to set up AppSignal for Express.js applications easily.

How can logging and monitoring your system help? You'll gain insights into:

Unusual login attempts
Unauthorized access
Suspicious network traffic
Data passing through a network (both tracking and analysis)
Log entries

Run Node.js As a Non-root User

If your Node.js app operates with root user privileges, attackers can exploit any vulnerabilities, granting them full control over your machine. They could then carry out destructive actions against your application.

So, whether your backend runs on a dedicated server or Docker container, avoid using root users for Node.js. Olivier Lalonde's insights in the post Don't run Node.js as root! are noteworthy here.

Deny Access To JWT After A Password Reset

If a user is alerted and responds to unfamiliar access into their account by changing their password, blacklist the JWT previously issued to the client to avoid further access by the unknown actor.

Don't Expose Error Details In the Response Payload

Never send 500 error details to the client. Attackers can use that information to exploit the vulnerabilities in your application. Instead, send a generic message like "Something went wrong 😔".

Denial of Service (DoS)

Mitigate DoS attacks by using rate limiters for network throttling and libraries such as body-parser to limit body payload size.

Avoid Using Default Cookie Names

Don't use default names for cookies, as they might unintentionally disclose the technology stack your backend relies upon. By discerning the framework in use, attackers can exploit any particular vulnerabilities associated with it.

Instead, opt for custom cookie names.

And that's it!

Wrapping Up

In this post, we explored common security threats in Node.js applications and practical ways to mitigate them for production environments.

You can fortify your applications against evolving security risks by implementing these security measures and remaining vigilant.

Happy coding!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

Job Schedulers for Node: Bull or Agenda?

Omonigho Kenneth Jimmy — Wed, 13 Sep 2023 14:00:00 +0000

Whether you're familiar with job schedulers or new to the concept, two popular libraries in the Node.js ecosystem stand out: Bull and Agenda.

These libraries provide unique features and advantages when it comes to efficient job scheduling. Let's explore the ins and outs of Bull and Agenda! 🚀

Scheduling Jobs in Node.js: Conducting An Orchestra

A job scheduler can be likened to an orchestra conductor in a Node.js application. Just as a conductor leads and coordinates musicians to produce a harmonious symphony, a job scheduler orchestrates the execution of tasks in your application.

Imagine your application as a grand orchestra, with each task representing a skilled musician. Without a conductor, each musician might play their part at their own pace, resulting in a disjointed and chaotic performance. Similarly, without a job scheduler, tasks in your application may run independently, leading to resource conflicts, delays, and suboptimal performance.

Just as a conductor adapts to unforeseen circumstances during a live performance, a job scheduler handles exceptions and failures in your application. It gracefully manages errors, reschedules failed tasks, and ensures that the show goes on smoothly, maintaining the reliability and resilience of your application.

Now that we've summarised the role of a job scheduler, let's turn our attention to what Bull can do.

Bull for Node.js

Bull claims to be the fastest, most reliable Redis-based queue for Node.js. It is known for its high performance, scalability, and robustness. With Bull, you can create queues to manage and process jobs asynchronously, making it ideal for handling time-consuming or resource-intensive tasks.

Benefits of Bull as a Job Scheduler

Using Bull as a job scheduler brings several benefits to your Node.js application:

High performance and scalability: Bull leverages efficient algorithms and data structures to process jobs quickly. It is designed to handle high workloads and scales horizontally.
Reliable job processing and job queues: Bull ensures reliable job processing by providing features such as job persistence, automatic retries, error handling, and priority queues.
Advanced job scheduling options: You can schedule jobs to run at specific times, set recurring jobs using cron-like expressions, or define intervals between job executions.
Monitoring and insights: Bull provides a built-in monitoring dashboard to track job progress, view statistics, and monitor performance metrics. This feature allows you to gain insights into the job processing pipeline, identify bottlenecks, and optimize your application's performance.
Integration and compatibility: Bull integrates well with popular frameworks like Express or Nest.js and can easily integrate with different data stores, message brokers, and external services.
Extensive ecosystem and community support: Bull is actively maintained and regularly updated, ensuring compatibility with the latest versions of Node.js. The community provides helpful resources, documentation, and plugins to extend Bull's functionality and address various use cases.

Take note, however, of the following information from the Bull documentation:

Bull is currently in maintenance mode, we are only fixing bugs. For new features check BullMQ, a modern rewritten implementation in Typescript. You are still very welcome to use Bull if it suits your needs, which is a safe, battle-tested library.

Real-world Use Cases Where Bull Shines

Bull excels in real-world use cases where efficient job scheduling and task management are critical. In distributed systems or microservice architectures, Bull particularly shines as a job scheduler for coordinating and synchronizing tasks across multiple services or nodes. Here's a code example:

// Service A - Producer

const Queue = require("bull");

// Create a Bull queue
const queue = new Queue("taskQueue", {
  redis: {
    host: "localhost",
    port: 6379,
  },
});

// Producer service adds jobs to the queue
async function addToQueue(data) {
  await queue.add(data);
  console.log("Job added to the queue:", data);
}

// Service B - Consumer

// Create a Bull queue with the same name as the producer
const queue = new Queue("taskQueue", {
  redis: {
    host: "localhost",
    port: 6379,
  },
});

// Consumer service processes jobs from the queue
queue.process(async (job) => {
  console.log("Processing job:", job.data);
  // Perform the necessary actions for the job
  // ...

  // Mark the job as completed
  return Promise.resolve();
});

// Start producing jobs
addToQueue({ data: "Task 1" });
addToQueue({ data: "Task 2" });
addToQueue({ data: "Task 3" });

In this example, we have two services, Service A (Producer) and Service B (Consumer). They both share a Bull queue named taskQueue. Service A produces jobs by adding data to the queue using the addToQueue function. Service B consumes these jobs and processes them using the queue.process method.

For instance, when addToQueue({ data: 'Task 1' }) is called, it adds a job to the Bull queue named taskQueue. The job contains data { data: 'Task 1' } representing the specific task or job to be processed.

The queue.add(data) method adds the job to the queue, and Bull takes care of persisting the job details in the underlying Redis database. The job is then available for consumption by any connected consumer service processing jobs from the same queue.

When queue.process is called in the consumer service, it sets up a worker that listens to the queue and starts processing jobs as they become available. In this case, the worker will process the job containing { data: 'Task 1' } once it reaches the front of the queue.

The job processing involves invoking the function provided to queue.process. In the code example, the function logs the job data (console.log('Processing job:', job.data)) and performs any necessary actions specific to the job.

By calling addToQueue with different data for each task, you can add multiple jobs to the queue. The consumer service will process them one by one based on the order in which they were added.

Other use cases where Bull shines include:

Background processing and task offloading
Task scheduling and automation
Job queues and workload balancing
Delayed and retry mechanisms

But, as the saying goes: "Every rose has its thorn". Bull, while an exciting choice of job scheduler, also has some noteworthy drawbacks.

Potential Drawbacks of Bull

Dependency on Redis: Bull relies on Redis as its underlying data store, requiring the setup and maintenance of a Redis server. This can introduce added complexity and infrastructure demands if you're not already using Redis in your application.
Steep learning curve: Bull's wide range of features and advanced options may require developers new to job scheduling and asynchronous task management to invest time in learning and experimenting to utilize its capabilities effectively.
Memory consumption: Bull's memory usage can increase with large job queues, necessitating careful monitoring and optimization to stay within acceptable limits, particularly in memory-constrained or high-throughput environments.
Persistence options: Bull's reliance on Redis for persistence may not align with your preferred data store, potentially necessitating custom solutions or alternative job scheduler libraries to integrate with a different database system.
Overkill for simple use cases: If your application has straightforward scheduling requirements or doesn't require advanced job processing features, integrating Bull may introduce unnecessary complexity. In such cases, a simpler job scheduler or task queue solution might be more suitable.

This brings us to Bull’s alternative — Agenda.

Agenda for Node.js

Agenda is a lightweight job scheduling library for Node.js, built on top of MongoDB. It focuses on simplicity and ease of use, making it an excellent choice for applications that require basic job scheduling capabilities without the need for complex features.

Benefits of Agenda as a Job Scheduler

Here are a few notable benefits of using Agenda:

Simple and lightweight: Agenda is designed to be lightweight and easy to use, making it a great choice for applications with simpler job scheduling requirements.
Persistence options: Unlike Bull, Agenda supports multiple databases, including MongoDB as the default choice. This gives you the flexibility to use your preferred database system for storing job data.
Event-driven architecture: Agenda's event-driven architecture enables seamless integration and efficient communication between application components through custom event-triggered job execution.
Error handling and job recovery: Agenda includes built-in error handling, job recovery, retries, and concurrency control, ensuring effective management of job execution errors and recovery from failures in your application.
Job scheduling flexibility: Agenda provides comprehensive scheduling capabilities, human-readable syntax, and complex recurring schedules for executing tasks at regular intervals or specific times.
Active community and maintenance: Agenda has an active community and ongoing maintenance, ensuring regular updates, bug fixes, ample resources, documentation, and community support for any issues or assistance you may need.

Examples Where Agenda Is a Good Fit

Agenda is well-suited for a range of scenarios, including recurring tasks, cron jobs, event-driven workflows, and lightweight use cases that demand a straightforward API and a minimalistic solution for job scheduling in your application.

The following piece of code uses Agenda to execute tasks based on complex time patterns (using cron syntax):

const Agenda = require("agenda");
const agenda = new Agenda();

// Define a job to be executed based on cron schedule
agenda.define("sendEmail", async (job) => {
  const { to, subject, body } = job.attrs.data;
  // Send email logic here
  console.log(`Sending email to ${to} with subject: ${subject}`);
});

// Define the cron schedule for the job
agenda.every("0 8 * * *", "sendEmail", {
  to: "example@example.com",
  subject: "Daily Report",
  body: "Hello, this is your daily report.",
});

// Start the agenda instance and execute jobs
(async () => {
  await agenda.start();
  console.log("Agenda started");

  // Optionally, you can define additional jobs and schedules here

  // Gracefully shut down the agenda instance
  await agenda.stop();
})();

In this example, we define an Agenda instance and then define a job named sendEmail. We specify the job's logic inside the async function. The every method is used to schedule the job with the cron expression 0 8 * * *, which means that the job will run every day at 8:00 AM. The job data includes the email recipient, subject, and body.

When the agenda starts, it will execute the job according to the defined schedule. You can add more jobs and schedules as needed within the async block. Finally, the optional await agenda.stop(); is called to gracefully stop the Agenda instance when the application is finished.

Limitations of Agenda

While Agenda offers several advantages, it also comes with some trade-offs to consider:

Lack of advanced features: If your application requires complex job orchestration, prioritization, or extensive queue management, you may find Agenda's feature set to be less comprehensive than Bull's.
Single process execution: By default, Agenda operates within a single process, which simplifies deployment and setup but may not scale as effectively as Bull in terms of handling high job concurrency and distributed processing.
Less active community: While Agenda has an active community, it may have a smaller user base compared to other popular alternatives like Bull, potentially resulting in fewer available resources, documentation, and community support.
Learning curve for complex scenarios: Agenda is user-friendly for simpler use cases, but can have a steeper learning curve for complex job scheduling needs. If your application requires intricate workflows, advanced scheduling patterns, or sophisticated error handling, a deeper understanding of Agenda's capabilities may be necessary.

Feature Comparison between Bull and Agenda for Node.js

Here's a concise table comparing the features of each library to help you choose the optimal job scheduler for your Node.js projects.

Let's dive into the key similarities and differences between Bull and Agenda to make an informed choice! 🚀

Feature	🐂 Bull	🗓️ Agenda
Job Persistence	Redis	MongoDB (with support for other databases)
Scalability	Suitable for high job concurrency	Single process, may not scale as effectively
Learning Curve	Moderate	Relatively easy for simpler use cases
Advanced Features	Extensive options for job management	Limited compared to more feature-rich schedulers
Persistence Flexibility	Tightly integrated with Redis	Supports multiple databases, including MongoDB
Event-Driven Architecture	Not emphasized	Embraces event-driven workflows
Error Handling	Customizable error-handling mechanisms	Built-in error handling and job recovery options
Community Support	Widely adopted, active community	Active community but with a smaller user base
Scheduling Flexibility	Versatile scheduling options	Rich set of scheduling features, including cron
Resource Consumption	Potentially higher memory consumption	Lightweight and optimized for lower resource usage
Integration	Works well with existing Redis setups	Flexible integration with various tech stacks
Use Case Suitability	Complex job orchestration and queueing	Simpler use cases and straightforward scheduling

Ultimately, the choice between Bull and Agenda boils down to your project's specific needs and complexity. Whether you prioritize advanced features, scalability, simplicity, or specific integration requirements, use the insights gained from this comparison to make an informed decision that aligns with your development goals.

Remember, your choice of job scheduler plays a vital role in efficient task management. Choose wisely and enjoy seamless job scheduling in your Node.js applications.

Wrapping Up

In this post, we explored Bull and Agenda as job scheduler libraries for Node.js applications.

We learned about the benefits of Bull, such as reliable job persistence and extensive features, and its real-world use cases. Additionally, we discovered Agenda's advantages, including its support for multiple databases, event-driven architecture, and rich scheduling options.

By comparing their features, potential drawbacks, and use case suitability, we can now make informed decisions when choosing between Bull and Agenda for our specific application requirements.

Happy job scheduling!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

An Introduction to Async Stack Traces in Node.js

Omonigho Kenneth Jimmy — Wed, 24 May 2023 11:19:30 +0000

Node.js 12.x onwards introduced async stack traces. Async stack traces allow developers to view the call stack of asynchronous code, making it easier to trace and debug code issues.

In this post, we’ll see how async stack traces work, how you can use them to debug code, and how to track them using AppSignal. Ready?

Let's get going!

How Async Stack Traces Work in Node.js

In JavaScript, asynchronous code is executed through callbacks, promises, and async/await. When an error occurs in asynchronous code, it can be difficult to trace its origin because the call stack is not captured at the point of the error. Async stack traces capture the call stack when the asynchronous code is scheduled rather than when it is executed, allowing developers to see an error's full context.

Node.js uses an event-driven, non-blocking I/O model for handling asynchronous code execution. An event loop constantly checks for new tasks, executes them one by one, and registers callback functions for tasks that do not block the execution. This allows Node.js to handle multiple tasks simultaneously, improving performance and scalability.

Async stack traces capture the call stack when an async operation begins using "long stack traces". When an async operation starts, Node.js creates a new execution context that includes information about the current call stack and the location where the async operation was initiated. This execution context is then passed along with the async operation as it is scheduled and executed.

When an error occurs, Node.js can reference the execution context to determine the entire call stack at the point where the async operation began. It enriches the stack property of Error instances with async stack frames, i.e., await locations in the code. These async frames are marked with async in the stack string. You can see the full context of an error, including the function calls and variables that led to the error.

Node.js Async Stack Traces: A Code Example

Consider the following code. An async function init() calls funcOne(), which then calls funcTwo(), which then calls and throws funcThree(). Async stack traces will print a stack trace with init(), funcOne(), funcTwo(), and funcThree().

const funcThree = async () => {
  await Promise.resolve();
  throw new Error("Oops");
};

const funcTwo = async () => {
  await Promise.resolve();
  await funcThree();
};

const funcOne = async () => {
  await new Promise((resolve) => setTimeout(resolve, 10));
  await funcTwo();
};

const init = async () => {
  await new Promise((resolve) => setTimeout(resolve, 10));
  await funcOne();
};

init().then(
  () => console.log("success"),
  (error) => console.error(error.stack)
);

This results in an output like so:

node ./asyncStackTraces.js

Error: Oops
    at funcThree (/Users/kennethjimmy/Desktop/asyncStackTraces.js:3:9)
    at async funcTwo (/Users/kennethjimmy/Desktop/asyncStackTraces.js:8:3)
    at async funcOne (/Users/kennethjimmy/Desktop/asyncStackTraces.js:13:3)
    at async init (/Users/kennethjimmy/Desktop/asyncStackTraces.js:18:3)

If we run the same code on a version of Node.js older than 12, the output looks like this:

node ./asyncStackTraces.js

Error: Oops
    at funcThree (/Users/kennethjimmy/Desktop/asyncStackTraces.js:3:9)
    at <anonymous>

In this case, at <anonymous> isn’t very helpful in revealing the full context of the error.

However, this works differently with async stack traces. When an error occurs in a callback function, the call stack is captured at the point when a callback is scheduled (the init() function in this case), rather than when it is executed.

Set Up AppSignal for Your Express.js App

AppSignal can assist you in identifying and troubleshooting problems with your app, including issues related to slow performance, errors, crashes, and downtime. You can resolve these issues quickly before they negatively impact your users.

In this tutorial, we will walk you through how to integrate AppSignal into your Express.js app.

Note: You can follow along with the source code for the following tutorial here.

Firstly, sign up for an account on appsignal.com and create a new app by following these guides.

Then start a new Node.js project:

npm init -y

Install AppSignal for Node.js:

npm install --save @appsignal/nodejs

In your repo’s root directory, create an appsignal.cjs file and paste the following code to require and configure AppSignal:

require("dotenv").config();
const { Appsignal } = require("@appsignal/nodejs");

new Appsignal({
  active: true,
  name: process.env.APPSIGNAL_APP_NAME,
  pushApiKey: process.env.APPSIGNAL_PUSH_API_KEY, // Note: renamed from `apiKey` in version 2.2.5
});

Note: An application's Push API key is given during setup and can be found in the App settings > Push & deploy tab.

In your package.json file, edit the start script to load AppSignal's configuration before any other library using the --require flag, like so:

{
  "scripts": {
    "start": "node --require ‘./appsignal.cjs’ app.mjs"
  }
}

Integrating AppSignal into your Express app is very easy. You just need to require the expressErrorHandler module and then use it in your app with app.use, as shown below:

import { expressErrorHandler } from "@appsignal/nodejs";

const app = express();

// add this after all routes, but before any other error handlers
app.use(expressErrorHandler());

The expressErrorHandler module consists of middleware designed to forward any errors in your Express application to AppSignal.

Tracking Async Callback Errors with AppSignal

Let's create an error inside an asynchronous callback to see how AppSignal handles it:

const funcThree = async () => {
  await Promise.resolve();
  throw new Error("Oops");
};

const funcTwo = async () => {
  await Promise.resolve();
  await funcThree();
};

const funcOne = async () => {
  await new Promise((resolve) => setTimeout(resolve, 10));
  await funcTwo();
};

app.get(
  "/trigger_error",
  catchAsync(async (req, res) => {
    await new Promise((resolve) => setTimeout(resolve, 10));
    await funcOne();

    res.status(200).json({
      statusbar: "success",
      data: {
        mentors,
      },
    });
  })
);

When accessing GET /trigger_error, an error is triggered. The server will return the error along with an asynchronous stack trace, as shown below:

{
  "status": "error",
  "error": {
      "statusCode": 500,
      "status": "error"
  },
  "message": "Oops",
  "stack": "Error: Oops\n    at funcThree (file:///Users/kennethjimmy/Desktop/app.mjs:29:9)\n    at async funcTwo (file:///Users/kennethjimmy/Desktop/app.mjs:34:3)\n    at async funcOne (file:///Users/kennethjimmy/Desktop/app.mjs:39:3)\n    at async file:///Users/kennethjimmy/Desktop/app.mjs:55:5"
}

After the error is thrown, AppSignal captures and reports the exception. To view your application's errors and stack traces, navigate to the AppSignal Dashboard, select "Errors", and click on "Issue list."

Click on a particular error to see its details, including the error message and stack trace.

Increasing Stack Trace Size in Node.js

By default, stack trace size is limited, making it difficult to pinpoint the exact source of an error. In development, we want as much context as we can get. Fortunately, it's possible to increase stack trace size in Node.js.

To do so, you can use the stackTraceLimit property of the Error object/class. By default, this property is set to 10, so only the last 10 calls will be included in the stack trace. If the value is set to 0, this stops the collection of stack traces. However, any positive integer can be assigned as the maximum limit for collecting frames. Alternatively, setting the property to Infinity will collect all the frames.

For example, to set the stack trace limit to Infinity in an Express.js application in development mode, you can use the following code:

// Set the stack trace limit to Infinity in development mode
if (process.env.NODE_ENV !== 'production') {
  Error.stackTraceLimit = Infinity
}

// Initialize Express.js app
const express = require('express');
const app = express();

// Define routes
// ...

// Start the server
const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`Server started on port ${port}`);
});

By increasing stack trace size, you can get more information about the execution of your code and better understand the context in which an error occurs. However, it's important to consider the following:

Performance impact: Increasing stack trace size can negatively impact performance, particularly in cases where a large number of errors occur, or the stack trace size is set to an excessively high value.
Memory usage: A larger stack trace size means more memory usage, which can be a concern if your application is already memory-intensive.
Security risks: Longer stack traces can potentially expose sensitive information, such as credentials or source code, if not properly handled.
Complexity: It can be more difficult to navigate and analyze a larger stack trace, particularly for complex applications.

With that in mind, you should use a reasonable limit that provides the necessary information without causing performance issues.

Wrapping Up

In this post, we've seen how async stack traces in Node.js can help you debug and trace errors in your asynchronous code by capturing call stacks.

We also explored how integrating AppSignal into your Node.js application makes it a lot easier to detect and resolve async callback errors in real time.

Finally, we discussed how expanding stack trace size in Node.js can offer more insights into the execution of your code and help you understand an error's context. However, it's important to use this feature judiciously, taking into account its possible impact on performance, memory usage, security, and navigability.

Happy coding!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

Forem: Omonigho Kenneth Jimmy

Security Best Practices for Your Node.js Application

Understanding Node.js Security Risks and Implementing Security Best Practices

Threat: Injection Attacks (SQL, NoSQL)

Mitigate Injection Attacks

Use of Parameterized Queries

Sanitize User Input

Use of ORMs/ODMs

Threat: Cross-site Scripting (XSS)

How to Prevent Cross-site Scripting

Validate User Input

Output Encoding

Use of HTTPOnly Cookies

Setting HTTP Headers

Threat: Brute Force

How to Mitigate Brute Force

Rate Limiting

Application of Maximum Login Attempts

Application of Multi-Factor Authentication (MFA)

Threat: Cross-Site Request Forgery (CSRF)

Mitigating CSRF Attacks

Threat: Insecure Dependencies

Constantly Update Your Dependencies

Verify Packages Before Use

Additional Security Best Practices

Logging and Monitoring

Run Node.js As a Non-root User

Deny Access To JWT After A Password Reset

Don't Expose Error Details In the Response Payload

Denial of Service (DoS)

Avoid Using Default Cookie Names

Wrapping Up

Job Schedulers for Node: Bull or Agenda?

Scheduling Jobs in Node.js: Conducting An Orchestra

Bull for Node.js

Benefits of Bull as a Job Scheduler

Real-world Use Cases Where Bull Shines

Potential Drawbacks of Bull

Agenda for Node.js

Benefits of Agenda as a Job Scheduler

Examples Where Agenda Is a Good Fit

Limitations of Agenda

Feature Comparison between Bull and Agenda for Node.js

Wrapping Up

An Introduction to Async Stack Traces in Node.js

How Async Stack Traces Work in Node.js

Node.js Async Stack Traces: A Code Example

Set Up AppSignal for Your Express.js App

Tracking Async Callback Errors with AppSignal

Increasing Stack Trace Size in Node.js

Wrapping Up

Use of `HTTPOnly` Cookies