🚀 Handling Cron Jobs in NestJS with Multiple Instances using Bull

#nestjs #javascript #programming #tutorial

Introduction

Cron jobs are essential when you need to execute scheduled tasks in your application. But what happens when you have multiple instances running? 🤯 Without proper handling, each instance might execute the job simultaneously, causing duplicate processing and potential data inconsistencies.

A solid solution is using Bull (a Node.js queue library based on Redis) to ensure that only one instance executes the cron job at a time. In this article, we'll explore how to achieve this using NestJS and Bull.

🛠 Setting Up Bull in NestJS

1️⃣ Install Dependencies

First, install Bull and Redis client:

npm install --save @nestjs/bull bull ioredis

2️⃣ Configure BullModule

In your app.module.ts, configure Bull to use Redis:

import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
import { MyCronJobProcessor } from './cron.processor';
import { MyCronJobService } from './cron.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost', // Use the Redis host
        port: 6379, // Default Redis port
      },
    }),
    BullModule.registerQueue({
      name: 'cronQueue',
    }),
  ],
  providers: [MyCronJobProcessor, MyCronJobService],
})
export class AppModule {}

Note: Ensure Redis is running locally or use a cloud-hosted Redis service.

🎯 Implementing the Cron Job

3️⃣ Creating the Cron Job Service

We’ll create a service that adds jobs to the queue at scheduled intervals.

import { Injectable } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { Cron } from '@nestjs/schedule';

@Injectable()
export class MyCronJobService {
  constructor(@InjectQueue('cronQueue') private cronQueue: Queue) {}

  @Cron('*/5 * * * * *') // Runs every 5 seconds
  async scheduleJob() {
    await this.cronQueue.add('processData', {});
    console.log('Cron job added to the queue ✅');
  }
}

The @Cron decorator schedules the job at a fixed interval, ensuring that the task is queued rather than executed by every instance.

4️⃣ Processing the Cron Job

Only one instance should process the job at a time. Bull takes care of that for us! 🎉

import { Process, Processor } from '@nestjs/bull';
import { Job } from 'bull';

@Processor('cronQueue')
export class MyCronJobProcessor {
  @Process('processData')
  async handleCronJob(job: Job) {
    console.log('Processing cron job... 🤖', job.id);
    // Your task logic here (e.g., database cleanup, report generation)
  }
}

📦 Create a Docker compose

If you run multiple instances of your NestJS app, only one instance will process the queued job, thanks to Redis locking mechanisms in Bull.

version: '3.8'

services:
  redis:
    image: redis:6.2
    restart: always
    ports:
      - '6379:6379'
    networks:
      - app-network

  worker:
    image: node:18
    working_dir: /app
    volumes:
      - .:/app
    command: ["npm", "run", "start"]
    environment:
      - NODE_ENV=production
      - REDIS_HOST=redis
    networks:
      - app-network
    deploy:
      replicas: 3  # Example: Run 3 instances of the worker
      restart_policy:
        condition: on-failure

networks:
  app-network:
    driver: bridge

Run:

docker-compose up -d

Now, your Redis instance is ready to use! 🚀

✅ Conclusion

Using Bull with Redis in NestJS, we’ve ensured that:

✔️ Cron jobs are scheduled only once per interval
✔️ Multiple instances don’t trigger duplicate executions
✔️ Scalability is achieved with a queue-based approach

Now you’re ready to handle scheduled tasks like a pro! 💪🔥

Happy coding! 🚀

"Please fix this..."

Focus on creating stellar experiences without email headaches. Postmark's reliable API and detailed analytics make your transactional emails as polished as your product.

Start free