Forem: Boyinbode Ebenezer Ayomide

The Worst Morning of My Developer Life — A Patient Hacker, a Fake AI Tool, and 150 Deleted Repos — My Story

Boyinbode Ebenezer Ayomide — Sat, 04 Apr 2026 08:47:09 +0000

I Woke Up to 150+ Deleted GitHub Repos and a Ransom Note — This Is What Happened

A real developer. A real attack. A real recovery.

I almost didn't write this.

Partly because it's embarrassing. Partly because I'm still angry about it. But mostly because I know there are developers out there running the exact same tools I was running, making the exact same mistakes I was making — and I don't want them to wake up to what I woke up to on the morning of March 24, 2026.

So here it is. The full story. No sugar coating.

The Morning Everything Was Gone

I opened my laptop like any other morning. Grabbed my coffee, opened GitHub, and just… froze.

My repositories were gone.

Not some of them. Not a few. Over 150 repositories across multiple organizations — personal projects, client work, production backends — all gone. Replaced by repositories with random UUID names, each containing a single file with this message:

All production database backups have been deleted from your systems 
and securely stored in our databases. 
Contact for Backup: https://t.me/cyberluffy

My stomach dropped.

I checked my GitHub audit log. The damage was methodical and deliberate. Someone had:

Removed all my repos from every integration — SonarCloud, Vercel, AWS Amplify, AWS Connector for GitHub, Cursor — all of it
Then deleted the repositories themselves
The activity was traced to IP 188.241.177.181 — São Paulo, Brazil

I had never been to Brazil.

My First Reaction (And What You Should Do First)

I panicked. I won't lie. But I forced myself to slow down and think.

The first thing I did was change my GitHub password and start revoking tokens. But here's what I didn't fully understand yet — changing your password means nothing if the attacker is using a token. Tokens are independent of your password. I had to revoke everything.

If this is happening to you right now, do this immediately:

Go to GitHub → Settings → Developer Settings → Personal Access Tokens → Revoke ALL
Settings → SSH and GPG Keys → Delete ALL
Settings → Applications → Remove anything suspicious
Settings → Security → Sessions → Sign out all other sessions
Enable 2FA if you haven't already — use an authenticator app, not SMS
Go to support.github.com and open an emergency ticket — time matters for repo restoration

I opened a GitHub support ticket within the first hour. That was one of the best decisions I made that day.

How Did They Even Get In?

This is the part that took me a while to figure out. And honestly, the answer surprised me.

They didn't brute force my password. They didn't phish me. They didn't exploit some obscure vulnerability.

I let them in myself. Without knowing it.

A few weeks earlier I had come across an AI agent tool called OpenClaw. It was marketed as a developer productivity tool — an AI gateway that could help automate workflows. I installed it, tried it out briefly, and kind of forgot about it.

npm install -g openclaw

That one command changed everything.

What I didn't know was that OpenClaw installed a persistent background service on my Mac — a gateway that ran 24/7, survived reboots, and had full access to my filesystem. I found it hiding in my LaunchAgents:

~/Library/LaunchAgents/ai.openclaw.gateway.plist

Opening that file made my blood run cold:

<key>RunAtLoad</key>
<true/>

<key>KeepAlive</key>
<true/>

<key>ProgramArguments</key>
<array>
  <string>/opt/homebrew/bin/node</string>
  <string>/opt/homebrew/lib/node_modules/openclaw/dist/index.js</string>
  <string>gateway</string>
  <string>--port</string>
  <string>18789</string>
</array>

RunAtLoad: true. KeepAlive: true. It was designed to always be running. Always listening. On port 18789.

The gateway logs told the rest of the story. This thing had been running since March 17 — a full 7 days before I noticed anything. And it wasn't just sitting there. It was connected to my WhatsApp. It was advertising itself on my local network. It had access to everything on my machine.

But the real damage came from something embarrassingly simple.

The $0 Mistake That Actually Caused This

While digging through my shell history, I found this:

git remote set-url origin https://drbenzene:ghp_WT********oIYa57uNoVz3kkDKWXMGgOAW5N0xhm5u@github.com/company/backend_code.git

There it was. My GitHub Personal Access Token. Sitting in plain text in my bash history.

I had done what so many developers do — embedded a token directly in a git remote URL to avoid being prompted for credentials. It's quick. It's convenient. And it is catastrophically dangerous.

That token had full repository access. OpenClaw read my shell history, extracted the token, and handed it to whoever was on the other end.

The full attack chain was:

I install OpenClaw to try it out
        ↓
OpenClaw runs silently for 7 days with full filesystem access
        ↓
Reads ~/.bash_history and ~/.zsh_history
        ↓
Finds GitHub PAT stored in plain text in remote URL
        ↓
Attacker uses token to access GitHub API
        ↓
Removes repos from all integrations
        ↓
Deletes all 150+ repositories
        ↓
Creates ransom note repos
        ↓
I wake up to a nightmare

Seven days. One npm install. One bad habit with git tokens.

The Investigation — What I Checked and How

I want to be thorough here because this is the part most blog posts skip. Here's exactly what I checked to understand the full scope of the breach.

Checking My Mac

LaunchAgents and LaunchDaemons — where malware hides on Mac:

ls ~/Library/LaunchAgents/
ls /Library/LaunchAgents/
ls /Library/LaunchDaemons/

This is where I found OpenClaw.

Active network connections:

lsof -i | grep ESTABLISHED

Running processes:

ps aux

Cron jobs:

crontab -l
sudo crontab -l

Shell history for exposed secrets:

cat ~/.zsh_history | grep -E "ghp_|token|password|secret|key"

This is where I found a second exposed token — for another GitHub organization. I revoked it immediately.

Login history:

last

Good news here — no remote logins to my Mac. The attacker never directly SSH'd into my machine. They operated entirely through the OpenClaw gateway.

Checking My Production Servers

I SSH'd into my live backend server and ran:

# Who has logged in?
last -n 50

# Any unauthorized users?
cat /etc/passwd | grep -v nologin | grep -v false

# Recently modified files?
find /etc -mtime -7 -type f 2>/dev/null

# What's in authorized_keys?
cat ~/.ssh/authorized_keys

The server was clean — no unauthorized logins, no backdoor users, no modified files. But my server's bash history also had an exposed token in it. I revoked that too.

The Cleanup — Exactly What I Did

1. Killed the Backdoor

# Stop OpenClaw immediately
launchctl unload ~/Library/LaunchAgents/ai.openclaw.gateway.plist

# Delete the LaunchAgent
rm ~/Library/LaunchAgents/ai.openclaw.gateway.plist

# Uninstall the package
npm uninstall -g openclaw

# Remove all its data
rm -rf ~/.openclaw

2. Deleted All Old SSH Keys

# Remove all old keys from my Mac
rm ~/.ssh/id_rsa ~/.ssh/id_rsa.pub
rm ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub
rm ~/.ssh/company_key
rm ~/.ssh/prodevs_key ~/.ssh/prodevs_key.pub
rm ~/.ssh/azure_new_key ~/.ssh/azure_new_key.pub
rm ~/.ssh/your_key_name ~/.ssh/your_key_name.pub

Also removed the old key from my production server:

# On the server - remove the old compromised key
sed -i '/generated-by-azure/d' ~/.ssh/authorized_keys

3. Generated Fresh SSH Keys

# Dedicated key for GitHub
ssh-keygen -t ed25519 -C "my@email.com" -f ~/.ssh/github_key

# One key for all servers
ssh-keygen -t ed25519 -C "my@email.com" -f ~/.ssh/servers_key

Why two separate keys? If one gets compromised, the other is still safe. At minimum, keep your GitHub key separate from your server keys.

4. Set Up SSH Config Properly

nano ~/.ssh/config

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/github_key

Host Company-Live
  HostName YOUR_SERVER_IP
  User azureuser
  IdentityFile ~/.ssh/servers_key

5. Switched All Repos From HTTPS to SSH

# Never again will I use this format
# git remote set-url origin https://user:TOKEN@github.com/org/repo.git

# This is the right way
git remote set-url origin git@github.com:username/repo.git

6. Cleared My Shell History

cat /dev/null > ~/.bash_history
cat /dev/null > ~/.zsh_history
history -c

7. Ran Security Scans

I downloaded KnockKnock from Objective-See (objective-see.org) — it scans every persistence mechanism on your Mac. All 17 items it found were legitimate. Clean.

I also ran Malwarebytes for Mac. Clean.

What the Security Scans Showed

KnockKnock scanned every category:

Launch Items: 17 found — all legitimate (Docker, MySQL, Nginx, Redis, etc.)
Login Items: 3 found — all legitimate (Electron app, PDFgear, Docker)
Kernel Extensions: 0
Library Inserts: 0
Library Proxies: 0
Login/Logout Hooks: 0

No traces of OpenClaw remained. The cleanup was complete.

About the Ransom — Don't Pay It

The ransom note claimed they had "securely stored backups" of all my deleted repositories. Let me be very direct: this is almost certainly a lie.

GitHub repository deletion is immediate and irreversible from the attacker's side. They deleted my repos and left a note — they almost certainly do not have my code.

Do not contact them. Do not pay. Instead:

Contact GitHub support immediately — they may be able to restore recently deleted repos
Check if any collaborators have local clones
Check Vercel, AWS Amplify, Netlify — they often cache your code
Check your own local machine for cloned copies

The Lessons I'm Taking Forward

Never Embed Tokens in Git URLs

# ❌ This stores your token in shell history forever
git remote set-url origin https://username:ghp_TOKEN@github.com/org/repo.git

# ✅ Use SSH — clean, secure, no tokens exposed
git remote set-url origin git@github.com:org/repo.git

Vet Every Tool You Install — Especially AI Agents

I installed OpenClaw because it looked interesting and useful. I didn't check:

Who published the npm package
What permissions it was requesting
Why it needed to run as a persistent background service
Whether it was open source and auditable

AI agent tools are a new category of developer tool that often requires deep system access — filesystem, network, environment variables. That power can be abused.

Before installing any tool like this:

# Check who published it and when
npm info PACKAGE_NAME

# Look at the package contents before installing
npx --yes npm-package-inspect PACKAGE_NAME

Audit Your LaunchAgents Regularly

# Run this monthly
ls ~/Library/LaunchAgents/
ls /Library/LaunchAgents/
ls /Library/LaunchDaemons/

Download KnockKnock and run it monthly. It takes 2 minutes and could save you everything.

Use Separate SSH Keys for Separate Purposes

~/.ssh/github_key        → GitHub only
~/.ssh/servers_key       → All production servers

If you manage multiple GitHub accounts:

Host github-personal
  HostName github.com
  User git
  IdentityFile ~/.ssh/github_personal_key

Host github-work
  HostName github.com
  User git
  IdentityFile ~/.ssh/github_work_key

Enable 2FA — But Not SMS

SMS 2FA can be SIM-swapped. Use an authenticator app:

Google Authenticator
Authy
1Password (built-in TOTP)

Never Store Secrets in Shell History

Add this to your .zshrc or .bashrc:

# Prevent specific commands from being saved to history
export HISTIGNORE="*TOKEN*:*token*:*password*:*secret*:*ghp_*"

# Or increase history size but add timestamps for auditing
export HISTTIMEFORMAT="%F %T "

Better yet — use a secrets manager like 1Password or Bitwarden and never type tokens directly into the terminal.

The Developer Security Checklist I Now Follow

I built this after the incident. Review it monthly:

GitHub:
[ ] 2FA enabled with authenticator app (not SMS)
[ ] No tokens embedded in git remote URLs
[ ] SSH used instead of HTTPS for all repos
[ ] Audit log checked for suspicious activity
[ ] Old/unused tokens revoked
[ ] SSH keys rotated every 6 months

Mac:
[ ] LaunchAgents audited monthly
[ ] KnockKnock scan run monthly  
[ ] Malwarebytes scan run monthly
[ ] All new tools vetted before installation

Servers:
[ ] authorized_keys reviewed regularly
[ ] Login history checked
[ ] No unknown user accounts
[ ] .env secrets rotated after any compromise

General:
[ ] Secrets manager in use (1Password, Bitwarden)
[ ] No plaintext tokens anywhere in shell history
[ ] .env in global .gitignore
[ ] 2FA on every critical service

Where Things Stand Now

GitHub support is working on repo restoration. My Mac is confirmed clean by both KnockKnock and Malwarebytes. All credentials have been rotated. SSH is set up properly. My production server is secure.

It was a brutal day. Honestly one of the worst days I've had as a developer. But looking back, every single thing that went wrong was preventable. And that's exactly why I wrote this.

The attack wasn't sophisticated in some magical, unstoppable way. It worked because:

I installed a tool without fully vetting it
I had a bad habit of embedding tokens in git URLs
I didn't regularly audit what was running on my machine

All fixable. All preventable. Starting today.

One Last Thing

If you're reading this and you have tokens embedded in git remote URLs right now — please go fix them. Right now. Before you finish reading this.

# Find all repos using HTTPS with tokens
grep -r "https://.*:.*@github.com" ~/.gitconfig
grep -r "https://.*:.*@github.com" ~/*/. git/config 2>/dev/null

# Fix them
git remote set-url origin git@github.com:username/repo.git

It takes 30 seconds. It could save you everything.

If this helped you or if you've had a similar experience, I'd love to hear about it in the comments. Let's help each other stay safe out there.

And if you found this useful — share it. Another developer somewhere needs to read this today.

Let's Connect

Linkedln | Github | Twitter |

Tags: security github developer ssh hacking incidentresponse nodejs devops macsecurity career

How I Built an AI-Driven Job Automation Engine: My Hardest Engineering Lessons

Boyinbode Ebenezer Ayomide — Sat, 14 Feb 2026 10:07:54 +0000

Building a job automation engine sounds like a fun project until you hit the first React-based form that ignores your scripts.

After months of building for one a client on a Job automation core, I've realized that modern web automation isn't about "botting"—it's about teaching code to behave with human-level nuance. If you're looking to build something similar or just want to level up your browser automation game, here is the blueprint I followed and the "gotchas" I learned along the way.

1. Don't Build a Scraper. Build a System.

The biggest mistake I made early on was trying to run automation directly from my API routes. Browser instances are heavy; they spike memory and can kill your main service.

My Solution: I moved everything to a distributed microservice architecture using NestJS and Bull (Redis).

When you're building this, treat every automation as a background job. This gives you:

Retries: If a page fails to load, the system retries with exponential backoff.
Concurrency Control: You can limit how many browsers run at once so you don't melt your server.

2. Dealing with Brittle Selectors (The AI Bridge)

If you rely on id="first_name", your code will break next week. Modern UIs change too fast.

What I learned: Use LLMs as a "semantic translator." Instead of hardcoding selectors, I wrote a service that crawls the page, extracts all visible inputs/labels, and sends that "schema" to an AI.

Here’s how you can implement this "discovery" phase:

// How I extract form metadata for the AI to "understand"
const fields = await page.evaluate(() => {
  return Array.from(document.querySelectorAll('input, select, textarea')).map(el => ({
    id: el.id,
    label: el.closest('div')?.querySelector('label')?.innerText || '',
    type: el.tagName
  }));
});

// Then, I ask the AI to map the user's resume data to these IDs
const answers = await aiService.generateAnswers(fields, resumeData);

By doing this, I stopped caring if the developer changed the CSS or renamed a field. The AI just "gets it."

3. Outsmarting the "React Void"

This was my biggest "aha!" moment. I noticed that high-level commands like page.fill() often failed to trigger React state updates. The value would appear on the screen, but when I clicked "Submit," the form acted like it was empty.

The Fix: You have to simulate the actual hardware events. I learned that focusing, typing manually, and blurring the input is the only way to satisfy modern framework listeners (like Redux or React Hook Form).

// The "Human" way to fill a field
async function humanFill(page: Page, selector: string, value: string) {
    const input = await page.$(selector);
    await input.focus();
    await input.click();

    // Clear the field first
    await page.keyboard.press('Control+A');
    await page.keyboard.press('Backspace');

    // Type with a slight delay to mimic a human
    await page.keyboard.type(value, { delay: 40 });

    // Explicitly trigger the 'change' and 'input' events
    await input.evaluate(el => {
        el.dispatchEvent(new Event('input', { bubbles: true }));
        el.dispatchEvent(new Event('change', { bubbles: true }));
    });
}

4. The Stealth Layer (Staying Under the Radar)

Job boards don't like bots. If you use a default headless browser, you’ll get blocked.

My Tip: Always remove the navigator.webdriver flag and use a real User-Agent. But the real "pro tip" I discovered? CSP Bypassing. Sometimes site security blocks reCAPTCHA assets on certain network configurations. I used Playwright's route interceptor to fix this:

// A little trick to ensure reCAPTCHA always loads correctly
await page.route('**/*.recaptcha.net/**', route => {
    const url = route.request().url().replace('www.recaptcha.net', 'www.google.com');
    route.continue({ url });
});

5. Trust but Verify (Success Detection)

A common mistake is assuming that if the URL changed, the submission worked. In my experience, job boards are full of "noise"—like diversity surveys that show a "Thank You" message even if your main application crashed.

I learned to clone the DOM, strip out the noise (footers, survey sections), and perform a strict text search.

Final Thoughts
The web is messy. Automation is the art of handling that mess. My biggest takeaway is that human-centric automation (keystrokes, AI-mapping, event-dispatching) beats brute-force scraping every single time.

If you’re building your own engine, start small, capture screenshots of every failure, and treat the browser like a living person interacting with your code.

Have questions about how I handled specific anti-bot blocks or AI prompts? Drop them in the comments!

Let's Connect

Linkedln | Github | Twitter |

Scaling Cron Job Performance from a Few Thousands to Millions Of Process

Boyinbode Ebenezer Ayomide — Tue, 07 Oct 2025 10:39:02 +0000

As business applications grow and more user retention, one of the most challenging aspects is maintaining performance while serving an ever-increasing user base. In this article, we'll explore how I transformed our portfolio performance processing system from a simple linear approach to a sophisticated batching system capable of handling over 1,000,000 users efficiently.

WHERE DOES THE PROBLEM BEGINS

Initially, during the early days of this startup, our system processed portfolio performance reports cron jobs, and other transactional or async processes in a straightforward manner - iterate through all users and send their monthly summaries. This worked fine when we had thousands of users, but as we scaled to over 1,000,000 active users, several critical issues emerged

Memory Exhaustion - Loading all users into memory simultaneously
Database Connection Overload - Too many concurrent database queries
Email Service Rate Limiting - Overwhelming our email provider
Processing Timeouts - Jobs taking hours to complete
Poor Error Recovery - One failed user could break the entire process.

THE WAY FORWARD

My approach was to implements a two-tier batching strategy that processes users in manageable chunks while distributing the load across time intervals.

import { Injectable, Logger } from '@nestjs/common';

interface User {
  id: number;
  email: string;
  first_name: string;
  last_name: string;
  created_at: Date;
}

interface JobLog {
  id: number;
  job_name: string;
  job_status: JobLogStatus;
  job_response?: string;
  created_at: Date;
}

enum JobLogStatus {
  PENDING = 'PENDING',
  IN_PROGRESS = 'IN_PROGRESS',
  COMPLETED = 'COMPLETED',
  FAILED = 'FAILED',
}

interface TimeInterval {
  from: Date;
  to: Date;
}

interface BatchProcessingConfig {
  batchSize: number;
  delayBetweenBatches: number;
  maxRetries: number;
  concurrencyLimit: number;
}

@Injectable()
export class PortfolioPerformanceProcessor {
  private readonly logger = new Logger(PortfolioPerformanceProcessor.name);

  // Configuration for batch processing
  private readonly config: BatchProcessingConfig = {
    batchSize: 100,
    delayBetweenBatches: 2000, // 2 seconds
    maxRetries: 3,
    concurrencyLimit: 5,
  };

  constructor(
    private readonly userRepository: UserRepository,
    private readonly statisticService: StatisticService,
    private readonly mailService: MailService,
    private readonly jobLogRepository: JobLogRepository,
    private readonly loggerRepository: LoggerRepository,
  ) {}

  /**
   * Main entry point for processing monthly portfolio performance
   * Uses temporal segmentation and batching for optimal performance
   */
  async processMonthlyPortfolioPerformanceInBatches(): Promise<string> {
    const newJobLog = await this.initializeJobLog();

    try {
      // Generate monthly intervals from start date to now
      const intervals = this.generateProcessingIntervals();
      let totalProcessed = 0;
      let totalErrors = 0;

      this.logger.log(`Starting portfolio processing for ${intervals.length} time intervals`);

      // Process each time interval sequentially to manage load
      for (const [index, interval] of intervals.entries()) {
        this.logger.log(`Processing interval ${index + 1}/${intervals.length}: ${interval.from.toISOString()} to ${interval.to.toISOString()}`);

        const intervalResult = await this.processTimeInterval(interval);
        totalProcessed += intervalResult.processed;
        totalErrors += intervalResult.errors;

        // Add delay between intervals to prevent overwhelming the system
        if (index < intervals.length - 1) {
          await this.delay(1000);
        }
      }

      await this.completeJobLog(newJobLog.id, totalProcessed, totalErrors);
      return `Portfolio processing completed. Processed: ${totalProcessed}, Errors: ${totalErrors}`;

    } catch (error) {
      await this.failJobLog(newJobLog.id, error);
      throw error;
    }
  }

  /**
   * Process all users within a specific time interval using cursor-based pagination
   */
  private async processTimeInterval(interval: TimeInterval): Promise<{ processed: number; errors: number }> {
    let lastId: number | undefined = undefined;
    let totalProcessed = 0;
    let totalErrors = 0;
    let batch: User[];

    do {
      // Fetch users in batches using cursor-based pagination
      batch = await this.userRepository.getUsersByDateRangeInBatches(
        interval.from,
        interval.to,
        this.config.batchSize,
        lastId,
      );

      if (batch.length === 0) break;

      this.logger.debug(`Processing batch of ${batch.length} users`);

      // Process current batch with error handling
      const batchResult = await this.processBatch(batch);
      totalProcessed += batchResult.processed;
      totalErrors += batchResult.errors;

      // Update cursor for next iteration
      lastId = batch[batch.length - 1].id;

      // Add delay between batches to prevent rate limiting
      if (batch.length === this.config.batchSize) {
        await this.delay(this.config.delayBetweenBatches);
      }

    } while (batch.length === this.config.batchSize);

    return { processed: totalProcessed, errors: totalErrors };
  }

Instead of processing all over 1 Million users at once, I segment users by their registration date into monthly intervals. This approach ensures each interval contains a manageable number of users. If one interval fails, others continue processing.

/**
   * Process a single batch of users with concurrent processing and error handling
   */
  private async processBatch(users: User[]): Promise<{ processed: number; errors: number }> {
    const promises = users.map(user => this.processUserWithRetry(user));
    const results = await Promise.allSettled(promises);

    const processed = results.filter(r => r.status === 'fulfilled').length;
    const errors = results.filter(r => r.status === 'rejected').length;

    // Log any rejections for monitoring
    results.forEach((result, index) => {
      if (result.status === 'rejected') {
        this.logError(`Failed to process user ${users[index].id}`, result.reason);
      }
    });

    return { processed, errors };
  }

  /**
   * Process individual user with retry logic
   */
  private async processUserWithRetry(user: User, attempt: number = 1): Promise<void> {
    try {
      await this.processUser(user);
    } catch (error) {
      if (attempt < this.config.maxRetries) {
        this.logger.warn(`Retry ${attempt}/${this.config.maxRetries} for user ${user.id}`);
        await this.delay(1000 * attempt); // Exponential backoff
        return this.processUserWithRetry(user, attempt + 1);
      }
      throw error;
    }
  }

  /**
   * Core user processing logic
   */
  private async processUser(user: User): Promise<void> {
    try {
      // Fetch portfolio statistics
      const portfolio = await this.statisticService.getWeeklyStats(user.id);

      // Send email if user has valid email
      if (this.isValidEmail(user.email)) {
        const displayName = this.formatUserDisplayName(user);
        await this.mailService.sendWeeklyPortfolioSummary(
          user.email,
          displayName,
          portfolio,
        );
      }
    } catch (error) {
      this.logError(`Error processing portfolio for user ${user.id}`, error);
      throw error;
    }
  }

  /**
   * Generate monthly intervals for processing
   */
  private generateProcessingIntervals(): TimeInterval[] {
    const start = new Date('2022-01-01');
    const now = new Date();
    return UTILITIES.generateMonthlyIntervals(start, now);
  }

  /**
   * Initialize job logging
   */
  private async initializeJobLog(): Promise<JobLog> {
    return this.jobLogRepository.saveJobLog({
      job_name: 'processMonthlyPortfolioPerformanceInBatches',
      job_status: JobLogStatus.IN_PROGRESS,
    });
  }

  /**
   * Complete job logging with success status
   */
  private async completeJobLog(jobId: number, processed: number, errors: number): Promise<void> {
    await this.jobLogRepository.updateJobLog(jobId, {
      job_status: JobLogStatus.COMPLETED,
      job_response: `Portfolio processing completed. Processed: ${processed} users, Errors: ${errors}`,
    });
  }

  /**
   * Mark job as failed
   */
  private async failJobLog(jobId: number, error: any): Promise<void> {
    await this.jobLogRepository.updateJobLog(jobId, {
      job_status: JobLogStatus.FAILED,
      job_response: `Job failed: ${error.message}`,
    });
  }

  /**
   * Enhanced error logging with structured data
   */
  private async logError(message: string, error: any): Promise<void> {
    this.logger.error(message, error.stack);

    await this.loggerRepository.saveLogger({
      log_message: `${message}: ${error.message} - ${JSON.stringify({
        stack: error.stack,
        timestamp: new Date().toISOString(),
      })}`,
      log_level: 'ERROR',
      log_path: 'src/crons/portfolio-processor.service.ts',
    });
  }

  /**
   * Utility methods
   */
  private async delay(ms: number): Promise<void> {
    return new Promise(resolve => setTimeout(resolve, ms));
  }

  private isValidEmail(email: string): boolean {
    return email && email.includes('@') && email.length > 5;
  }

  private formatUserDisplayName(user: User): string {
    return user.first_name && user.last_name 
      ? `${user.first_name} ${user.last_name}` 
      : 'There';
  }
}

We process users in batches of 100, which provides the optimal balance between:

Memory usage (keeping it low)
Database connection efficiency
Processing speed
Error isolation

// Enhanced Repository with optimized queries
@Injectable()
export class UserRepository {
  constructor(private readonly db: DatabaseConnection) {}

  /**
   * Optimized cursor-based pagination for large datasets
   * Uses indexed columns for efficient querying
   */
  async getUsersByDateRangeInBatches(
    fromDate: Date,
    toDate: Date,
    batchSize: number,
    lastId?: number,
  ): Promise<User[]> {
    const query = `
      SELECT id, email, first_name, last_name, created_at
      FROM users 
      WHERE created_at >= $1 
        AND created_at < $2 
        ${lastId ? 'AND id > $4' : ''}
        AND email IS NOT NULL 
        AND email != ''
      ORDER BY id ASC 
      LIMIT $3
    `;

    const params = lastId 
      ? [fromDate, toDate, batchSize, lastId]
      : [fromDate, toDate, batchSize];

    return this.db.query(query, params);
  }
}

// Utility class for date and batch management
export class UTILITIES {
  /**
   * Generate monthly intervals between two dates
   */
  static generateMonthlyIntervals(start: Date, end: Date): TimeInterval[] {
    const intervals: TimeInterval[] = [];
    const current = new Date(start);

    while (current < end) {
      const from = new Date(current);
      const to = new Date(current.getFullYear(), current.getMonth() + 1, 1);

      intervals.push({
        from,
        to: to > end ? end : to,
      });

      current.setMonth(current.getMonth() + 1);
    }

    return intervals;
  }

  /**
   * Async delay utility
   */
  static delay(ms: number): Promise<void> {
    return new Promise(resolve => setTimeout(resolve, ms));
  }
}

// Enhanced mail service with rate limiting and retry logic
@Injectable()
export class MailService {
  private readonly logger = new Logger(MailService.name);
  private emailQueue: Array<{ email: string; name: string; portfolio: any }> = [];
  private isProcessingQueue = false;

  /**
   * Queue-based email sending to handle rate limits
   */
  async sendWeeklyPortfolioSummary(
    email: string, 
    name: string, 
    portfolio: any
  ): Promise<void> {
    // Add to queue instead of immediate sending
    this.emailQueue.push({ email, name, portfolio });

    // Start processing if not already running
    if (!this.isProcessingQueue) {
      this.processEmailQueue();
    }
  }

  /**
   * Process email queue with rate limiting
   */
  private async processEmailQueue(): Promise<void> {
    this.isProcessingQueue = true;

    while (this.emailQueue.length > 0) {
      const batch = this.emailQueue.splice(0, 10); // Process 10 emails at once

      const promises = batch.map(({ email, name, portfolio }) => 
        this.sendEmailWithRetry(email, name, portfolio)
      );

      await Promise.allSettled(promises);

      // Rate limiting delay
      if (this.emailQueue.length > 0) {
        await UTILITIES.delay(1000); // 1 second between batches
      }
    }

    this.isProcessingQueue = false;
  }

  /**
   * Send individual email with retry logic
   */
  private async sendEmailWithRetry(
    email: string, 
    name: string, 
    portfolio: any,
    attempt: number = 1
  ): Promise<void> {
    try {
      // Your actual email sending logic here
      await this.actualEmailSend(email, name, portfolio);
    } catch (error) {
      if (attempt < 3) {
        this.logger.warn(`Email retry ${attempt}/3 for ${email}`);
        await UTILITIES.delay(2000 * attempt);
        return this.sendEmailWithRetry(email, name, portfolio, attempt + 1);
      }
      this.logger.error(`Failed to send email to ${email} after 3 attempts`, error.stack);
      throw error;
    }
  }

  private async actualEmailSend(email: string, name: string, portfolio: any): Promise<void> {
    // Implementation depends on your email provider
    // This could be SendGrid, AWS SES, etc.
  }
}

// Advanced monitoring and metrics
@Injectable()
export class ProcessingMetrics {
  private processingStats = {
    totalUsers: 0,
    processedUsers: 0,
    failedUsers: 0,
    averageProcessingTime: 0,
    emailsSent: 0,
    startTime: null as Date | null,
  };

  startProcessing(): void {
    this.processingStats.startTime = new Date();
    this.resetCounters();
  }

  recordUserProcessed(): void {
    this.processingStats.processedUsers++;
  }

  recordUserFailed(): void {
    this.processingStats.failedUsers++;
  }

  recordEmailSent(): void {
    this.processingStats.emailsSent++;
  }

  getProcessingReport(): string {
    const duration = this.processingStats.startTime 
      ? Date.now() - this.processingStats.startTime.getTime()
      : 0;

    return `
Processing Report:
- Duration: ${(duration / 1000 / 60).toFixed(2)} minutes
- Users Processed: ${this.processingStats.processedUsers}
- Users Failed: ${this.processingStats.failedUsers}
- Emails Sent: ${this.processingStats.emailsSent}
- Success Rate: ${((this.processingStats.processedUsers / (this.processingStats.processedUsers + this.processingStats.failedUsers)) * 100).toFixed(2)}%
- Processing Rate: ${(this.processingStats.processedUsers / (duration / 1000 / 60)).toFixed(2)} users/minute
    `;
  }

  private resetCounters(): void {
    this.processingStats.processedUsers = 0;
    this.processingStats.failedUsers = 0;
    this.processingStats.emailsSent = 0;
  }
}

// Database optimization utilities
export class DatabaseOptimizations {
  /**
   * Example of an optimized query with proper indexing
   */
  static getOptimizedUserQuery(): string {
    return `
      -- Ensure these indexes exist for optimal performance:
      -- CREATE INDEX CONCURRENTLY idx_users_created_at_id ON users(created_at, id);
      -- CREATE INDEX CONCURRENTLY idx_users_email_not_null ON users(email) WHERE email IS NOT NULL;

      SELECT id, email, first_name, last_name, created_at
      FROM users 
      WHERE created_at >= $1 
        AND created_at < $2 
        AND id > COALESCE($4, 0)
        AND email IS NOT NULL 
        AND email != ''
      ORDER BY id ASC 
      LIMIT $3
    `;
  }
}

// Configuration management
export interface SystemConfig {
  database: {
    maxConnections: number;
    queryTimeout: number;
  };
  email: {
    rateLimit: number;
    batchSize: number;
  };
  processing: {
    userBatchSize: number;
    intervalDelayMs: number;
    maxConcurrentJobs: number;
  };
}

export const PRODUCTION_CONFIG: SystemConfig = {
  database: {
    maxConnections: 20,
    queryTimeout: 30000,
  },
  email: {
    rateLimit: 100, // emails per minute
    batchSize: 10,
  },
  processing: {
    userBatchSize: 100,
    intervalDelayMs: 2000,
    maxConcurrentJobs: 3,
  },
};

Scaling from thousands to millions of users requires fundamental changes in how we approach batch processing. The key is breaking down large problems into manageable chunks while maintaining system reliability and performance.

The batching strategy reduced our processing time from over 6 hours to under 45 minutes while improving reliability and monitoring capabilities. Most importantly, it's designed to scale further as our user base continues to grow.

Always remember, premature optimization is the root of all evil, but once you hit scale, proper optimization becomes essential for business continuity.

Let's Connect

Linkedln | Github | Twitter | Youtube

Advanced NestJS: Hidden Gems for Senior Backend Engineers

Boyinbode Ebenezer Ayomide — Mon, 30 Jun 2025 08:53:15 +0000

Boyinbode Ebenezer Ayomide

Jun 30 '25

Advanced NestJS: Hidden Gems for Senior Backend Engineers

Comments

9 min read

Advanced NestJS: Hidden Gems for Senior Backend Engineers

Boyinbode Ebenezer Ayomide — Mon, 30 Jun 2025 08:52:38 +0000

Most times when we think of NestJS, we picture decorators, dependency injection, and clean architecture. But beneath the surface lies a treasure trove of advanced features that can transform how you build scalable backend applications. After years of building enterprise-grade NestJS applications, I've discovered patterns and techniques that rarely make it into tutorials but are essential for senior-level development.

Custom Metadata Reflection: Building Intelligent Decorators

Standard decorators are limited in their ability to store and retrieve complex configuration data. You need a way to attach sophisticated metadata to classes and methods that can be accessed at runtime.

// metadata.constants.ts
export const CACHE_CONFIG_METADATA = Symbol('cache-config');
export const RATE_LIMIT_METADATA = Symbol('rate-limit');

// cache-config.decorator.ts
import { SetMetadata } from '@nestjs/common';

export interface CacheConfig {
  ttl: number;
  key?: string;
  condition?: (args: any[]) => boolean;
  tags?: string[];
}

export const CacheConfig = (config: CacheConfig) => 
  SetMetadata(CACHE_CONFIG_METADATA, config);

// Advanced usage with conditional caching
@CacheConfig({
  ttl: 300,
  key: 'user-profile-{{userId}}',
  condition: (args) => args[0]?.cacheEnabled !== false,
  tags: ['user', 'profile']
})
async getUserProfile(userId: string, options?: GetUserOptions) {
  // method implementation
}

Smart Interceptor Using Metadata

// cache.interceptor.ts
import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Reflector } from '@nestjs/core';
import { Observable, of } from 'rxjs';
import { tap } from 'rxjs/operators';

@Injectable()
export class SmartCacheInterceptor implements NestInterceptor {
  constructor(
    private reflector: Reflector,
    private cacheService: CacheService
  ) {}

  intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
    const cacheConfig = this.reflector.get<CacheConfig>(
      CACHE_CONFIG_METADATA,
      context.getHandler()
    );

    if (!cacheConfig) {
      return next.handle();
    }

    const request = context.switchToHttp().getRequest();
    const args = context.getArgs();

    // Check condition
    if (cacheConfig.condition && !cacheConfig.condition(args)) {
      return next.handle();
    }

    // Generate cache key with template replacement
    const cacheKey = this.generateCacheKey(cacheConfig.key, request, args);

    // Try to get from cache
    const cachedResult = this.cacheService.get(cacheKey);
    if (cachedResult) {
      return of(cachedResult);
    }

    // Execute and cache result
    return next.handle().pipe(
      tap(result => {
        this.cacheService.set(cacheKey, result, {
          ttl: cacheConfig.ttl,
          tags: cacheConfig.tags
        });
      })
    );
  }

  private generateCacheKey(template: string, request: any, args: any[]): string {
    let key = template;

    // Replace template variables
    key = key.replace(/\{\{(\w+)\}\}/g, (match, prop) => {
      return request.params?.[prop] || 
             request.query?.[prop] || 
             args[0]?.[prop] || 
             match;
    });

    return key;
  }
}

Advanced Execution Context Manipulation

The ExecutionContext is more powerful than you realize. Here's how to leverage it for sophisticated request handling.

// advanced-auth.guard.ts
import { Injectable, CanActivate, ExecutionContext, UnauthorizedException } from '@nestjs/common';
import { Reflector } from '@nestjs/core';

export interface AuthContext {
  user?: any;
  permissions?: string[];
  rateLimit?: {
    remaining: number;
    resetTime: number;
  };
}

@Injectable()
export class AdvancedAuthGuard implements CanActivate {
  constructor(
    private reflector: Reflector,
    private authService: AuthService,
    private rateLimitService: RateLimitService
  ) {}

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const request = context.switchToHttp().getRequest();
    const response = context.switchToHttp().getResponse();

    // Get handler and class metadata
    const handler = context.getHandler();
    const controllerClass = context.getClass();

    // Check if route is public
    const isPublic = this.reflector.getAllAndOverride<boolean>('isPublic', [
      handler,
      controllerClass,
    ]);

    if (isPublic) {
      return true;
    }

    // Extract and validate token
    const token = this.extractTokenFromHeader(request);
    if (!token) {
      throw new UnauthorizedException('Token not found');
    }

    try {
      // Validate token and get user
      const user = await this.authService.validateToken(token);

      // Check rate limiting
      const rateLimitKey = `rate_limit:${user.id}:${handler.name}`;
      const rateLimit = await this.rateLimitService.checkLimit(rateLimitKey);

      if (!rateLimit.allowed) {
        response.set('X-RateLimit-Remaining', '0');
        response.set('X-RateLimit-Reset', rateLimit.resetTime.toString());
        throw new UnauthorizedException('Rate limit exceeded');
      }

      // Create enhanced auth context
      const authContext: AuthContext = {
        user,
        permissions: user.permissions,
        rateLimit: {
          remaining: rateLimit.remaining,
          resetTime: rateLimit.resetTime
        }
      };

      // Attach to request for use in downstream handlers
      request.authContext = authContext;

      // Set rate limit headers
      response.set('X-RateLimit-Remaining', rateLimit.remaining.toString());
      response.set('X-RateLimit-Reset', rateLimit.resetTime.toString());

      return true;
    } catch (error) {
      throw new UnauthorizedException('Invalid token');
    }
  }

  private extractTokenFromHeader(request: any): string | undefined {
    const [type, token] = request.headers.authorization?.split(' ') ?? [];
    return type === 'Bearer' ? token : undefined;
  }
}

Dynamic Module Factory Patterns

Creating modules that adapt their behavior based on runtime conditions is a powerful pattern for building flexible applications.

// feature-flag.interface.ts
export interface FeatureFlagConfig {
  provider: 'redis' | 'database' | 'memory';
  refreshInterval?: number;
  defaultFlags?: Record<string, boolean>;
  remoteConfig?: {
    url: string;
    apiKey: string;
  };
}

// feature-flag.module.ts
import { DynamicModule, Module, Provider } from '@nestjs/common';
import { FeatureFlagService } from './feature-flag.service';

@Module({})
export class FeatureFlagModule {
  static forRoot(config: FeatureFlagConfig): DynamicModule {
    const providers: Provider[] = [
      {
        provide: 'FEATURE_FLAG_CONFIG',
        useValue: config,
      },
    ];

    // Conditionally add providers based on configuration
    switch (config.provider) {
      case 'redis':
        providers.push({
          provide: 'FEATURE_FLAG_STORAGE',
          useClass: RedisFeatureFlagStorage,
        });
        break;
      case 'database':
        providers.push({
          provide: 'FEATURE_FLAG_STORAGE',
          useClass: DatabaseFeatureFlagStorage,
        });
        break;
      default:
        providers.push({
          provide: 'FEATURE_FLAG_STORAGE',
          useClass: MemoryFeatureFlagStorage,
        });
    }

    // Add remote config provider if configured
    if (config.remoteConfig) {
      providers.push({
        provide: 'REMOTE_CONFIG_CLIENT',
        useFactory: () => new RemoteConfigClient(config.remoteConfig),
      });
    }

    providers.push(FeatureFlagService);

    return {
      module: FeatureFlagModule,
      providers,
      exports: [FeatureFlagService],
      global: true,
    };
  }

  static forRootAsync(options: {
    useFactory: (...args: any[]) => Promise<FeatureFlagConfig> | FeatureFlagConfig;
    inject?: any[];
  }): DynamicModule {
    return {
      module: FeatureFlagModule,
      providers: [
        {
          provide: 'FEATURE_FLAG_CONFIG',
          useFactory: options.useFactory,
          inject: options.inject || [],
        },
        {
          provide: 'FEATURE_FLAG_STORAGE',
          useFactory: async (config: FeatureFlagConfig) => {
            switch (config.provider) {
              case 'redis':
                return new RedisFeatureFlagStorage();
              case 'database':
                return new DatabaseFeatureFlagStorage();
              default:
                return new MemoryFeatureFlagStorage();
            }
          },
          inject: ['FEATURE_FLAG_CONFIG'],
        },
        FeatureFlagService,
      ],
      exports: [FeatureFlagService],
      global: true,
    };
  }
}

// Usage in AppModule
@Module({
  imports: [
    FeatureFlagModule.forRootAsync({
      useFactory: (configService: ConfigService) => ({
        provider: configService.get('FEATURE_FLAG_PROVIDER') as 'redis' | 'database',
        refreshInterval: configService.get('FEATURE_FLAG_REFRESH_INTERVAL', 30000),
        remoteConfig: configService.get('FEATURE_FLAG_REMOTE_URL') ? {
          url: configService.get('FEATURE_FLAG_REMOTE_URL'),
          apiKey: configService.get('FEATURE_FLAG_API_KEY'),
        } : undefined,
      }),
      inject: [ConfigService],
    }),
  ],
})
export class AppModule {}

Request Scope Memory Management

Understanding REQUEST scope deeply is crucial for preventing memory leaks in high-traffic applications.

// user-context.service.ts
import { Injectable, Scope, OnModuleDestroy } from '@nestjs/common';
import { EventEmitter } from 'events';

@Injectable({ scope: Scope.REQUEST })
export class UserContextService implements OnModuleDestroy {
  private readonly eventEmitter = new EventEmitter();
  private readonly subscriptions: (() => void)[] = [];
  private userData: Map<string, any> = new Map();

  constructor() {
    // Set max listeners to prevent memory leak warnings
    this.eventEmitter.setMaxListeners(100);
  }

  setUserData(key: string, value: any): void {
    this.userData.set(key, value);
    this.eventEmitter.emit('userDataChanged', { key, value });
  }

  getUserData(key: string): any {
    return this.userData.get(key);
  }

  onUserDataChange(callback: (data: { key: string; value: any }) => void): void {
    this.eventEmitter.on('userDataChanged', callback);

    // Store cleanup function
    const cleanup = () => this.eventEmitter.off('userDataChanged', callback);
    this.subscriptions.push(cleanup);
  }

  // Critical: Clean up resources when request ends
  onModuleDestroy(): void {
    // Remove all event listeners
    this.subscriptions.forEach(cleanup => cleanup());
    this.eventEmitter.removeAllListeners();

    // Clear data
    this.userData.clear();

    console.log('UserContextService destroyed for request');
  }
}

// Usage with proper cleanup
@Injectable()
export class UserService {
  constructor(private userContext: UserContextService) {}

  async processUser(userId: string): Promise<void> {
    // This will be automatically cleaned up when request ends
    this.userContext.onUserDataChange((data) => {
      console.log(`User data changed: ${data.key} = ${data.value}`);
    });

    this.userContext.setUserData('userId', userId);
    this.userContext.setUserData('lastActivity', new Date());
  }
}

Advanced Exception Filter Chaining

Create sophisticated error handling with hierarchical exception filters.

// base-exception.filter.ts
import { ExceptionFilter, Catch, ArgumentsHost, HttpException, Logger } from '@nestjs/common';
import { Request, Response } from 'express';

export interface ErrorContext {
  correlationId: string;
  userId?: string;
  endpoint: string;
  userAgent?: string;
  ip: string;
}

@Catch()
export abstract class BaseExceptionFilter implements ExceptionFilter {
  protected readonly logger = new Logger(this.constructor.name);

  abstract canHandle(exception: any): boolean;
  abstract handleException(exception: any, host: ArgumentsHost): void;

  catch(exception: any, host: ArgumentsHost): void {
    if (this.canHandle(exception)) {
      this.handleException(exception, host);
    } else {
      // Pass to next filter in chain
      this.delegateToNext(exception, host);
    }
  }

  protected delegateToNext(exception: any, host: ArgumentsHost): void {
    // This would be handled by the next filter in the chain
    // or the default NestJS exception handler
    throw exception;
  }

  protected createErrorContext(host: ArgumentsHost): ErrorContext {
    const ctx = host.switchToHttp();
    const request = ctx.getRequest<Request>();

    return {
      correlationId: request.headers['x-correlation-id'] as string || 
                    Math.random().toString(36).substring(7),
      userId: (request as any).authContext?.user?.id,
      endpoint: `${request.method} ${request.url}`,
      userAgent: request.headers['user-agent'],
      ip: request.ip,
    };
  }
}

// validation-exception.filter.ts
@Catch(ValidationException)
export class ValidationExceptionFilter extends BaseExceptionFilter {
  canHandle(exception: any): boolean {
    return exception instanceof ValidationException;
  }

  handleException(exception: ValidationException, host: ArgumentsHost): void {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const errorContext = this.createErrorContext(host);

    this.logger.warn('Validation error', {
      ...errorContext,
      errors: exception.getErrors(),
    });

    response.status(400).json({
      statusCode: 400,
      message: 'Validation failed',
      errors: exception.getErrors(),
      correlationId: errorContext.correlationId,
      timestamp: new Date().toISOString(),
    });
  }
}

// business-exception.filter.ts
@Catch(BusinessException)
export class BusinessExceptionFilter extends BaseExceptionFilter {
  canHandle(exception: any): boolean {
    return exception instanceof BusinessException;
  }

  handleException(exception: BusinessException, host: ArgumentsHost): void {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const errorContext = this.createErrorContext(host);

    this.logger.error('Business logic error', {
      ...errorContext,
      errorCode: exception.getErrorCode(),
      message: exception.message,
    });

    response.status(422).json({
      statusCode: 422,
      message: exception.message,
      errorCode: exception.getErrorCode(),
      correlationId: errorContext.correlationId,
      timestamp: new Date().toISOString(),
    });
  }
}

// global-exception.filter.ts
@Catch()
export class GlobalExceptionFilter extends BaseExceptionFilter {
  canHandle(exception: any): boolean {
    return true; // Global filter handles everything
  }

  handleException(exception: any, host: ArgumentsHost): void {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const errorContext = this.createErrorContext(host);

    // Handle different types of exceptions
    if (exception instanceof HttpException) {
      this.handleHttpException(exception, response, errorContext);
    } else {
      this.handleUnknownException(exception, response, errorContext);
    }
  }

  private handleHttpException(
    exception: HttpException, 
    response: Response, 
    context: ErrorContext
  ): void {
    const status = exception.getStatus();
    const exceptionResponse = exception.getResponse();

    this.logger.error('HTTP Exception', {
      ...context,
      status,
      response: exceptionResponse,
    });

    response.status(status).json({
      statusCode: status,
      message: exception.message,
      correlationId: context.correlationId,
      timestamp: new Date().toISOString(),
    });
  }

  private handleUnknownException(
    exception: any, 
    response: Response, 
    context: ErrorContext
  ): void {
    this.logger.error('Unhandled Exception', {
      ...context,
      error: exception.message,
      stack: exception.stack,
    });

    response.status(500).json({
      statusCode: 500,
      message: 'Internal server error',
      correlationId: context.correlationId,
      timestamp: new Date().toISOString(),
    });
  }
}

// Register filters in correct order
// main.ts
app.useGlobalFilters(
  new ValidationExceptionFilter(),
  new BusinessExceptionFilter(),
  new GlobalExceptionFilter(), // This should be last
);

Advanced Health Check Orchestration

Build sophisticated health monitoring that goes beyond simple HTTP checks.

// health-check.service.ts
import { Injectable } from '@nestjs/common';
import { HealthIndicator, HealthIndicatorResult, HealthCheckError } from '@nestjs/terminus';

@Injectable()
export class AdvancedHealthIndicator extends HealthIndicator {
  constructor(
    private readonly databaseService: DatabaseService,
    private readonly redisService: RedisService,
    private readonly externalApiService: ExternalApiService,
  ) {
    super();
  }

  async checkDatabase(key: string): Promise<HealthIndicatorResult> {
    try {
      const startTime = Date.now();
      await this.databaseService.executeQuery('SELECT 1');
      const responseTime = Date.now() - startTime;

      const isHealthy = responseTime < 1000; // 1 second threshold

      const result = this.getStatus(key, isHealthy, {
        responseTime: `${responseTime}ms`,
        threshold: '1000ms',
        timestamp: new Date().toISOString(),
      });

      if (!isHealthy) {
        throw new HealthCheckError('Database response time too high', result);
      }

      return result;
    } catch (error) {
      throw new HealthCheckError('Database connection failed', {
        [key]: {
          status: 'down',
          error: error.message,
          timestamp: new Date().toISOString(),
        },
      });
    }
  }

  async checkRedis(key: string): Promise<HealthIndicatorResult> {
    try {
      const startTime = Date.now();
      await this.redisService.ping();
      const responseTime = Date.now() - startTime;

      const result = this.getStatus(key, true, {
        responseTime: `${responseTime}ms`,
        timestamp: new Date().toISOString(),
      });

      return result;
    } catch (error) {
      throw new HealthCheckError('Redis connection failed', {
        [key]: {
          status: 'down',
          error: error.message,
          timestamp: new Date().toISOString(),
        },
      });
    }
  }

  async checkExternalDependencies(key: string): Promise<HealthIndicatorResult> {
    const checks = await Promise.allSettled([
      this.checkExternalApi('payment-gateway', 'https://api.payment.com/health'),
      this.checkExternalApi('notification-service', 'https://api.notifications.com/health'),
      this.checkExternalApi('analytics-service', 'https://api.analytics.com/health'),
    ]);

    const results = checks.map((check, index) => ({
      name: ['payment-gateway', 'notification-service', 'analytics-service'][index],
      status: check.status === 'fulfilled' ? 'up' : 'down',
      details: check.status === 'fulfilled' ? check.value : check.reason,
    }));

    const failedServices = results.filter(r => r.status === 'down');
    const isHealthy = failedServices.length === 0;

    const result = this.getStatus(key, isHealthy, {
      services: results,
      failedCount: failedServices.length,
      totalCount: results.length,
      timestamp: new Date().toISOString(),
    });

    if (!isHealthy) {
      throw new HealthCheckError('External dependencies failing', result);
    }

    return result;
  }

  private async checkExternalApi(name: string, url: string): Promise<any> {
    const startTime = Date.now();
    const response = await fetch(url, { 
      method: 'GET',
      timeout: 5000 // 5 second timeout
    });
    const responseTime = Date.now() - startTime;

    return {
      name,
      status: response.ok ? 'up' : 'down',
      responseTime: `${responseTime}ms`,
      statusCode: response.status,
    };
  }
}

// health.controller.ts
@Controller('health')
export class HealthController {
  constructor(
    private health: HealthCheckService,
    private advancedHealth: AdvancedHealthIndicator,
  ) {}

  @Get()
  @HealthCheck()
  check() {
    return this.health.check([
      () => this.advancedHealth.checkDatabase('database'),
      () => this.advancedHealth.checkRedis('redis'),
    ]);
  }

  @Get('detailed')
  @HealthCheck()
  detailedCheck() {
    return this.health.check([
      () => this.advancedHealth.checkDatabase('database'),
      () => this.advancedHealth.checkRedis('redis'),
      () => this.advancedHealth.checkExternalDependencies('external-services'),
    ]);
  }

  @Get('readiness')
  @HealthCheck()
  readinessCheck() {
    // More strict checks for readiness
    return this.health.check([
      () => this.advancedHealth.checkDatabase('database'),
      () => this.advancedHealth.checkRedis('redis'),
      () => this.advancedHealth.checkExternalDependencies('external-services'),
    ]);
  }

  @Get('liveness')
  @HealthCheck()
  livenessCheck() {
    // Basic checks for liveness (pod restart criteria)
    return this.health.check([
      () => this.advancedHealth.checkDatabase('database'),
    ]);
  }
}

Provider Overriding in Tests: Surgical Test Isolation

Advanced testing patterns that provide precise control over dependencies.

// user.service.spec.ts
describe('UserService', () => {
  let service: UserService;
  let app: TestingModule;
  let mockUserRepository: jest.Mocked<UserRepository>;
  let mockEventEmitter: jest.Mocked<EventEmitter2>;
  let mockCacheService: jest.Mocked<CacheService>;

  beforeEach(async () => {
    // Create sophisticated mocks
    mockUserRepository = {
      save: jest.fn(),
      findOne: jest.fn(),
      update: jest.fn(),
      delete: jest.fn(),
      find: jest.fn(),
    } as any;

    mockEventEmitter = {
      emit: jest.fn(),
      on: jest.fn(),
      off: jest.fn(),
    } as any;

    mockCacheService = {
      get: jest.fn(),
      set: jest.fn(),
      del: jest.fn(),
      reset: jest.fn(),
    } as any;

    app = await Test.createTestingModule({
      imports: [
        // Import actual modules but override specific providers
        DatabaseModule,
        CacheModule,
        EventEmitterModule.forRoot(),
      ],
      providers: [
        UserService,
        NotificationService,
      ],
    })
    // Override specific providers surgically
    .overrideProvider(getRepositoryToken(User))
    .useValue(mockUserRepository)

    .overrideProvider(EventEmitter2)
    .useValue(mockEventEmitter)

    .overrideProvider(CACHE_TOKENS.SESSION_CACHE)
    .useValue(mockCacheService)

    // Override guards for testing without authentication
    .overrideGuard(JwtAuthGuard)
    .useValue({ canActivate: () => true })

    // Override interceptors to disable caching during tests
    .overrideInterceptor(CacheInterceptor)
    .useValue({ intercept: (context, next) => next.handle() })

    .compile();

    service = app.get<UserService>(UserService);
  });

  describe('createUser', () => {
    it('should create user and emit event', async () => {
      // Arrange
      const userData = { email: 'test@example.com', name: 'Test User' };
      const createdUser = { id: '1', ...userData };
      mockUserRepository.save.mockResolvedValue(createdUser);

      // Act
      const result = await service.createUser(userData);

      // Assert
      expect(mockUserRepository.save).toHaveBeenCalledWith(userData);
      expect(mockEventEmitter.emit).toHaveBeenCalledWith('user.created', {
        userId: '1',
        email: 'test@example.com',
        preferences: undefined
      });
      expect(result).toEqual(createdUser);
    });

    it('should handle cache failure gracefully', async () => {
      // Arrange
      const userData = { email: 'test@example.com', name: 'Test User' };
      const createdUser = { id: '1', ...userData };
      mockUserRepository.save.mockResolvedValue(createdUser);
      mockCacheService.set.mockRejectedValue(new Error('Cache unavailable'));

      // Act & Assert - should not throw
      const result = await service.createUser(userData);
      expect(result).toEqual(createdUser);
    });
  });

  // Test with different provider overrides per test
  describe('with different cache configuration', () => {
    beforeEach(async () => {
      // Override with different cache implementation
      await app.close();

      app = await Test.createTestingModule({
        imports: [UserModule],
      })
      .overrideProvider(CACHE_TOKENS.SESSION_CACHE)
      .useFactory({
        factory: () => new MemoryCacheService({ maxSize: 10 }),
      })
      .compile();

      service = app.get<UserService>(UserService);
    });

    it('should work with memory cache', async () => {
      // Test implementation with actual memory cache
    });
  });

  afterEach(async () => {
    await app.close();
  });
});

The key insight is that NestJS provides the primitives, but senior engineers know how to compose them into powerful, maintainable systems. These patterns have been battle-tested in production environments handling millions of requests.
Master these techniques, and you'll find yourself building more robust, scalable, and maintainable backend applications that can handle enterprise-level complexity with ease.

Let's Connect

Linkedln | Github | Twitter |

ADVANCED NESTJS BRUTAL FORCE ATTACK TECHNIQUES AND PREVENTIONS

Boyinbode Ebenezer Ayomide — Fri, 14 Mar 2025 12:42:09 +0000

Understanding Brute Force Attacks
Advanced Techniques to Prevent Brute Force Attacks
Setting Up Rate Limiting
Implementing Account Lockout
Adding CAPTCHA Verification
Blocking Suspicious IPs
Best Practices
Conclusion

Brute force attacks are a common security threat where an attacker attempts to gain unauthorized access to a system by trying numerous username/password combinations. The attacker systematically trying all possible combinations of credentials until the correct one is found. This can lead to unauthorized access, data breaches, and other security issues. To prevent such attacks, you must implement multiple layers of defense.

This article will guide you through advanced techniques to mitigate brute force attacks, including rate limiting, account lockout mechanisms, and CAPTCHA integration in NestJS

Understanding Brute Force Attacks

Imagine a popular online banking platform, "SecureBank," which allows users to log in using their email addresses and passwords. The platform has thousands of users, including individuals and businesses, who rely on it for managing their finances. Unfortunately, SecureBank’s login system has a critical flaw: it doesn’t implement any mechanisms to prevent multiple login attempts. This oversight makes it an easy target for brute force attacks.

The attacker, a malicious individual with basic programming skills, discovers SecureBank’s login page at https://securebank.com/login. The page requires a username (email) and password for access. The attacker’s goal is to gain unauthorized access to user accounts, particularly those with high balances or administrative privileges.

The attacker also obtains a list of common email addresses (e.g., from a previous data breach) and a dictionary of frequently used passwords (e.g., "password123", "admin123", "12345678").

The attacker writes a simple Python script to automate the login attempts. The script reads email addresses and passwords from the prepared lists and sends them to the login endpoint. Here’s what the script looks like:

import requests

# List of common email addresses and passwords
emails = ["user1@example.com", "user2@example.com", "admin@securebank.com"]
passwords = ["password123", "admin123", "12345678", "qwerty"]

# Target login endpoint
login_url = "https://securebank.com/login"

# Launch the brute force attack
for email in emails:
    for password in passwords:
        payload = {
            "email": email,
            "password": password
        }
        response = requests.post(login_url, data=payload)

        if "Login successful" in response.text:
            print(f"Success! Email: {email}, Password: {password}")
            break

The attacker runs the script, which begins sending thousands of login requests to SecureBank’s login endpoint. Each request tries a different combination of email and password. Since there are no restrictions on the number of attempts, the script can run indefinitely until it finds a valid combination.

After several hours, the script successfully logs in with the following credentials:

Email: admin@securebank.com

Password: admin123

The attacker now has access to the admin account, which grants them full control over the banking platform. They can transfer funds, steal sensitive customer data, or even lock legitimate users out of their accounts.

Advanced Techniques to Prevent Brute Force Attacks

a. Rate Limiting
Rate limiting restricts the number of requests a user can make within a specific time frame. This prevents attackers from making too many login attempts in a short period.

NestJS provides a flexible way to implement rate limiting using the @nestjs/throttler package.

Install Dependencies

npm install @nestjs/throttler

Configure ThrottlerModule
In your app.module.ts, configure the ThrottlerModule:

import { ThrottlerModule } from '@nestjs/throttler';

@Module({
  imports: [
    ThrottlerModule.forRoot({
      ttl: 60, // Time-to-live in seconds
      limit: 10, // Maximum number of requests within TTL
    }),
  ],
})
export class AppModule {}

Apply Rate Limiting to Routes
Use the @Throttle() decorator to apply rate limiting to specific routes:

import { Throttle } from '@nestjs/throttler';

@Controller('auth')
export class AuthController {
  @Throttle(5, 60) // 5 requests per 60 seconds
  @Post('login')
  async login(@Body() credentials: any) {
    // Login logic
  }
}

b. Account Lockout Mechanism
After a certain number of failed login attempts, the account is temporarily locked. This slows down brute force attacks and alerts the user of suspicious activity.

Track Failed Login Attempts
Store the number of failed login attempts in your database or an in-memory store like Redis.

import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { User } from './user.entity';

@Injectable()
export class AuthService {
  constructor(
    @InjectRepository(User)
    private userRepository: Repository<User>,
  ) {}

  async incrementFailedAttempts(userId: number): Promise<void> {
    const user = await this.userRepository.findOne({ where: { id: userId } });
    user.failedAttempts += 1;
    await this.userRepository.save(user);
  }

  async resetFailedAttempts(userId: number): Promise<void> {
    const user = await this.userRepository.findOne({ where: { id: userId } });
    user.failedAttempts = 0;
    await this.userRepository.save(user);
  }

  async isAccountLocked(userId: number): Promise<boolean> {
    const user = await this.userRepository.findOne({ where: { id: userId } });
    return user.failedAttempts >= 5; // Lock after 5 failed attempts
  }
}

Lock Account After Too Many Failed Attempts
In your login logic, check if the account is locked:

@Post('login')
async login(@Body() credentials: any) {
  const user = await this.authService.validateUser(credentials);
  if (!user) {
    await this.authService.incrementFailedAttempts(user.id);
    throw new UnauthorizedException('Invalid credentials');
  }

  if (await this.authService.isAccountLocked(user.id)) {
    throw new UnauthorizedException('Account locked due to too many failed attempts');
  }

  await this.authService.resetFailedAttempts(user.id);
  // Proceed with login
}

c. CAPTCHA Integration
CAPTCHA challenges ensure that the login attempt is made by a human and not an automated script.

import axios from 'axios';

@Post('login')
async login(@Body() credentials: any) {
  const captchaResponse = await axios.post(
    `https://www.google.com/recaptcha/api/siteverify?secret=${process.env.RECAPTCHA_SECRET}&response=${credentials.captcha}`,
  );

  if (!captchaResponse.data.success) {
    throw new UnauthorizedException('CAPTCHA verification failed');
  }

  // Proceed with login
}

d. IP Blocking
Blocking IP addresses that exhibit suspicious behavior (e.g., too many failed login attempts) can effectively mitigate brute force attacks.

Track IPs with Failed Attempts
Use Redis to store IP addresses and their failed attempt counts.

import { Injectable } from '@nestjs/common';
import { InjectRedis, Redis } from '@nestjs-modules/ioredis';

@Injectable()
export class IpBlockingService {
  constructor(@InjectRedis() private readonly redis: Redis) {}

  async incrementFailedAttempts(ip: string): Promise<void> {
    const attempts = await this.redis.incr(`ip:${ip}:attempts`);
    if (attempts === 1) {
      await this.redis.expire(`ip:${ip}:attempts`, 3600); // Expire after 1 hour
    }
  }

  async isIpBlocked(ip: string): Promise<boolean> {
    const attempts = await this.redis.get(`ip:${ip}:attempts`);
    return attempts && parseInt(attempts) >= 10; // Block after 10 failed attempts
  }
}

Check if the IP is blocked before processing the login request:

@Post('login')
async login(@Body() credentials: any, @Req() req: Request) {
  const ip = req.ip;
  if (await this.ipBlockingService.isIpBlocked(ip)) {
    throw new UnauthorizedException('IP blocked due to suspicious activity');
  }

  // Proceed with login
}

Preventing brute force attacks requires a multi-layered approach. By combining these techniques, you can significantly enhance the security of your NestJS backend.

Remember that each project has its unique requirements, so adapt these techniques accordingly. Stay up-to-date with the NestJS documentation and the rapidly evolving Node.js ecosystem to make the most of this powerful framework. Happy coding!

Let's Connect

Linkedln | Github | Twitter |

Building A Scalable Advanced Email Templating System with @react-email and NestJS

Boyinbode Ebenezer Ayomide — Thu, 24 Oct 2024 12:50:06 +0000

Whether you're sending transactional emails or personalized messages, having a templating system in place can greatly enhance the efficiency and flexibility of your email handling process. In this tutorial, we will explore how to build an email templating system in NestJS using @react-email for building dynamic and visually appealing email templates.

This integration will allow you to leverage the power of React's components for email design while NestJS will handle the backend logic to send these emails. Let’s walk through the steps with sample code to make the process easier.

Why Use @react-email/components?
@react-email/components provides a powerful and flexible way to create reusable email templates. With React's component-driven architecture, you can break down your email layout into small, manageable pieces and use the same code for the frontend and backend. This library abstracts the complexity of HTML emails, which can be tedious to manage manually.

Prerequisites
Before we begin, ensure you have the following installed:

Node.js (v14 or later)
NestJS CLI
TypeScript

A basic understanding of React
Step 1: Set Up a New NestJS Project
Start by setting up a new NestJS project if you don't already have one.

$ nest new email-templating
$ cd email-templating

Once your project is set up, we will install the necessary packages.

Step 2: Install Required Dependencies
Next, install the required packages for email templating and sending.

React Email components
npm install @react-email/components react react-dom

Nodemailer for sending emails

npm install nodemailer

Required types

npm install --save-dev @types/nodemailer

Step 3: Create Email Templates Using @react-email/components
Now, let’s create a simple email template using @react-email/components. First, create a new directory called email-templates inside the src folder.

mkdir src/email-templates

To include Tailwind CSS for styling and a reusable layout wrapper for your emails, you’ll need to set up a global layout component that can be used across different email templates, while leveraging Tailwind's utility classes for styling.

$ npm install tailwindcss

Inside the email-templates folder, create a file called welcome-email.tsx. This will be our first email template.

Now, configure Tailwind CSS by creating a tailwind.config.js file in your root directory.

$ npx tailwindcss init

Next, we need to configure Tailwind to process styles within the email templates. Add the following content inside your tailwind.config.js file:

module.exports = {
  content: [
    './src/email-templates/**/*.{js,ts,jsx,tsx}',
    './src/layout/**/*.{js,ts,jsx,tsx}',
  ],
  theme: {
    extend: {},
  },
  plugins: [],
};

To wrap every email with a consistent layout and styling, create a Layout component in your src/layout directory. This layout will serve as the base for all emails.

$ mkdir src/layout
$ touch src/layout/EmailLayout.tsx

Here’s the EmailLayout component that uses Tailwind for styling:

// src/layout/EmailLayout.tsx
import React from 'react';
import { Html, Head, Body, Container } from '@react-email/components';

const EmailLayout = ({ children }: { children: React.ReactNode }) => {
  return (
    <Html>
      <Head />
      <Body className="bg-gray-100 p-6">
        <Container className="max-w-xl mx-auto bg-white p-6 shadow-md rounded-lg">
          {/* Header */}
          <div className="text-center mb-8">
            <h1 className="text-2xl font-semibold text-indigo-600">Your Company</h1>
          </div>

          {/* Email Content */}
          {children}

          {/* Footer */}
          <div className="text-center mt-10 text-gray-500 text-sm">
            <p>&copy; {new Date().getFullYear()} Your Company. All rights reserved.</p>
          </div>
        </Container>
      </Body>
    </Html>
  );
};

export default EmailLayout;

This layout provides a structured, reusable container for all emails with Tailwind’s utility classes. It wraps the content in a clean layout, including a header and footer, ensuring consistency across all emails.

// src/mail/mail.service.ts
import { Injectable } from '@nestjs/common';
import nodemailer from 'nodemailer';
import { render } from '@react-email/components';
import WelcomeEmail from '../email-templates/welcome-email';

@Injectable()
export class MailService {
  private transporter;

  constructor() {
    this.transporter = nodemailer.createTransport({
      service: 'Gmail',
      auth: {
        user: 'your-email@gmail.com',
        pass: 'your-email-password',
      },
    });
  }

  async sendWelcomeEmail(to: string) {
    const emailHtml = render(<WelcomeEmail />);

    const mailOptions = {
      from: 'your-email@gmail.com',
      to,
      subject: 'Welcome to Our Service!',
      html: emailHtml,
    };

    await this.transporter.sendMail(mailOptions);
  }
}

Step 4: Set Up Email Sending in NestJS
Now that we have the template ready, let's integrate it with the backend to send emails. We'll be using nodemailer for email delivery.

First, create a mail.service.ts file in the src/mail directory. This service will handle sending emails using nodemailer.

mkdir src/mail
touch src/mail/mail.service.ts

Here’s how you can set up the mail service:


// src/mail/mail.service.ts
import { Injectable } from '@nestjs/common';
import nodemailer from 'nodemailer';
import { render } from '@react-email/components';
import WelcomeEmail from '../email-templates/welcome-email';

@Injectable()
export class MailService {
  private transporter;

  constructor() {
    this.transporter = nodemailer.createTransport({
      service: 'Gmail',
      auth: {
        user: 'your-email@gmail.com',
        pass: 'your-email-password',
      },
    });
  }

  async sendWelcomeEmail(to: string) {
    const emailHtml = render(<WelcomeEmail />);

    const mailOptions = {
      from: 'your-email@gmail.com',
      to,
      subject: 'Welcome to Our Service!',
      html: emailHtml,
    };

    await this.transporter.sendMail(mailOptions);
  }
}

In this service, we set up nodemailer with Gmail as the transport service. Replace the authentication details with your own email credentials. The sendWelcomeEmail method renders the React component to HTML using the render method from @react-email/components and sends it as an email.

Step 5: Create a Mail Controller
To trigger the email sending from an endpoint, create a new controller called mail.controller.ts.

touch src/mail/mail.controller.ts

Here’s how you can set up the controller:

// src/mail/mail.controller.ts
import { Controller, Post, Body } from '@nestjs/common';
import { MailService } from './mail.service';

@Controller('mail')
export class MailController {
  constructor(private readonly mailService: MailService) {}

  @Post('send-welcome')
  async sendWelcomeEmail(@Body('email') email: string) {
    await this.mailService.sendWelcomeEmail(email);
    return { message: 'Welcome email sent successfully!' };
  }
}

This controller exposes a POST /mail/send-welcome endpoint that takes an email address as the body parameter and triggers the sendWelcomeEmail method in the MailService.

Step 6: Set Up the Mail Module
Next, we’ll set up a MailModule to register both the MailService and MailController.

// src/mail/mail.module.ts
import { Module } from '@nestjs/common';
import { MailService } from './mail.service';
import { MailController } from './mail.controller';

@Module({
  providers: [MailService],
  controllers: [MailController],
})
export class MailModule {}

Finally, import the MailModule in your main application module.

// src/app.module.ts
import { Module } from '@nestjs/common';
import { MailModule } from './mail/mail.module';

@Module({
  imports: [MailModule],
})
export class AppModule {}

Step 7: Test the Integration
Now, you can test the email sending functionality by running your NestJS application and making a POST request to the /mail/send-welcome endpoint with a valid email address in the request body.

npm run start:dev

Use a tool like Postman or curl to send the request:

POST http://localhost:3000/mail/send-welcome
{
  "email": "recipient@example.com"
}

If everything is set up correctly, you should receive the welcome email in the specified inbox.

Feel free to expand this by adding more dynamic content to your templates, handling different types of emails (password resets, notifications), and exploring advanced features of nodemailer.

Happy coding!

Let's Connect

Linkedln | Github | Twitter | Youtube

Best Security implementation Practices In NestJS. A Comprehensive Guide

Boyinbode Ebenezer Ayomide — Sat, 23 Mar 2024 10:21:39 +0000

Ensuring robust security measures is paramount. Authentication and authorization are crucial aspects of security implementation, safeguarding sensitive data and resources from unauthorized access. NestJS, with its modular and scalable architecture, provides an excellent framework for building secure applications. In this blog post, we will explore the best practices for implementing authentication, authorization, and security measures in NestJS applications, along with code examples to guide you through the process.

1 . Setting Up Authentication:

Authentication is the process of verifying the identity of users. NestJS offers various strategies for implementing authentication, including JWT (JSON Web Tokens), session-based authentication, and OAuth2. Here, we will focus on JWT authentication, which is widely used for its simplicity and scalability.

// auth.module.ts
import { Module } from '@nestjs/common';
import { JwtModule } from '@nestjs/jwt';
import { AuthService } from './auth.service';
import { JwtStrategy } from './jwt.strategy';
import { AuthController } from './auth.controller';

@Module({
  imports: [
    JwtModule.register({
      secret: process.env.JWT_SECRET,
      signOptions: { expiresIn: '1h' },
    }),
  ],
  controllers: [AuthController],
  providers: [AuthService, JwtStrategy],
})
export class AuthModule {}

2 . Implementing Authorization

Authorization defines what actions users are allowed to perform within the application. NestJS provides decorators like @Roles() and @UseGuards() to enforce authorization rules at the controller and route levels.

// roles.guard.ts
import { Injectable, CanActivate, ExecutionContext } from '@nestjs/common';
import { Reflector } from '@nestjs/core';

@Injectable()
export class RolesGuard implements CanActivate {
  constructor(private reflector: Reflector) {}

  canActivate(context: ExecutionContext): boolean {
    const roles = this.reflector.get<string[]>('roles', context.getHandler());
    if (!roles) {
      return true;
    }
    const request = context.switchToHttp().getRequest();
    const user = request.user;
    return user && user.roles && user.roles.some(role => roles.includes(role));
  }
}

3 . Securing Routes

Protecting routes with guards ensures that only authorized users can access them. We can apply guards at the controller or route level using @UseGuards() decorator.
typescript

// app.controller.ts
import { Controller, Get, UseGuards } from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
import { RolesGuard } from './roles.guard';
import { Roles } from './roles.decorator';

@Controller('app')
export class AppController {
  @Get()
  @UseGuards(AuthGuard('jwt'), RolesGuard)
  @Roles('admin')
  getApp(): string {
    return 'Protected Route Accessed Successfully!';
  }
}

4 . Handling Authentication

In NestJS, authentication logic is typically implemented in a service. This service is responsible for verifying user credentials and generating tokens.

// auth.service.ts
import { Injectable } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { UserService } from '../user/user.service';

@Injectable()
export class AuthService {
  constructor(
    private readonly userService: UserService,
    private readonly jwtService: JwtService,
  ) {}

  async validateUser(username: string, password: string): Promise<any> {
    const user = await this.userService.findByUsername(username);
    if (user && user.password === password) {
      const { password, ...result } = user;
      return result;
    }
    return null;
  }

  async login(user: any): Promise<any> {
    const payload = { username: user.username, sub: user.userId };
    return {
      access_token: this.jwtService.sign(payload),
    };
  }
}

5 . Protecting Sensitive Data

In addition to authentication and authorization, protecting sensitive data is crucial for maintaining security. NestJS provides various mechanisms for encrypting sensitive information, such as passwords and access tokens.

// user.service.ts
import { Injectable } from '@nestjs/common';
import { User } from './user.entity';
import * as bcrypt from 'bcrypt';

@Injectable()
export class UserService {
  async findByUsername(username: string): Promise<User | undefined> {
    // Fetch user by username from the database
    return await User.findOne({ where: { username } });
  }

  async hashPassword(password: string): Promise<string> {
    const saltRounds = 10;
    return await bcrypt.hash(password, saltRounds);
  }

  async comparePasswords(enteredPassword: string, hashedPassword: string): Promise<boolean> {
    return await bcrypt.compare(enteredPassword, hashedPassword);
  }
}

Securing Configuration and Secrets

Storing sensitive information like database credentials, API keys, and JWT secrets securely is crucial. NestJS provides a built-in configuration module and environment variables for managing sensitive data securely.

// .env
JWT_SECRET=mysecretkey
DATABASE_URL=your_database_url


// database.module.ts
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';

@Module({
  imports: [TypeOrmModule.forRoot({
    type: 'postgres',
    url: process.env.DATABASE_URL,
    // other database configurations
  })],
})
export class DatabaseModule {}

7 . Rate Limiting and CSRF Protection

Implementing rate limiting and CSRF protection further enhances security by preventing abuse and protecting against cross-site request forgery attacks. NestJS offers middleware and libraries such as express-rate-limit and csurf for implementing these security measures.

// main.ts
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import * as rateLimit from 'express-rate-limit';
import * as csurf from 'csurf';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  // Apply rate limiting middleware
  app.use(rateLimit({
    windowMs: 15 * 60 * 1000, // 15 minutes
    max: 100, // limit each IP to 100 requests per windowMs
  }));

  // Apply CSRF protection middleware
  app.use(csurf());

  await app.listen(3000);
}
bootstrap();

8 . Logging and Monitoring

Logging and monitoring are essential aspects of security implementation, enabling detection and response to security incidents. NestJS provides robust logging capabilities through various logging libraries such as winston or @nestjs/common. Additionally, integrating monitoring tools like Prometheus and Grafana can provide insights into application performance and security metrics.

// logger.service.ts
import { Injectable, Logger } from '@nestjs/common';

@Injectable()
export class LoggerService extends Logger {
  // Custom logging methods for different log levels
  log(message: string) {
    super.log(message);
    // Additional logging logic
  }

  error(message: string, trace: string) {
    super.error(message, trace);
    // Additional error logging logic
  }

  warn(message: string) {
    super.warn(message);
    // Additional warning logging logic
  }
}

9 . Input Validation and Sanitization

Input validation and sanitization are critical for preventing common security vulnerabilities such as SQL injection, XSS (Cross-Site Scripting), and CSRF (Cross-Site Request Forgery). NestJS provides built-in validation features using libraries like class-validator and class-transformer to validate incoming data against predefined rules.

// create-user.dto.ts
import { IsString, IsNotEmpty, IsEmail } from 'class-validator';

export class CreateUserDto {
  @IsString()
  @IsNotEmpty()
  username: string;

  @IsString()
  @IsNotEmpty()
  @IsEmail()
  email: string;

  // Add more validation rules for other fields
}

10 . Continuous Security Testing

Continuous security testing is crucial for identifying and fixing security vulnerabilities throughout the development lifecycle. Integrating security testing tools like OWASP ZAP, SonarQube, and Snyk into your CI/CD pipeline helps automate security testing and ensures that security is an integral part of the development process.

# Example CI/CD pipeline configuration (using GitLab CI)
stages:
  - build
  - test
  - deploy

security_scan:
  stage: test
  image: owasp/zap2docker-stable
  script:
    - zap-baseline.py -t http://your-app-url
  allow_failure: true

11 . Error Handling and Reporting

Proper error handling and reporting are crucial for security, as they help identify and mitigate potential vulnerabilities and security incidents. NestJS provides robust error handling mechanisms through interceptors and exception filters, allowing you to gracefully handle errors and log relevant information.

// error.interceptor.ts
import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Observable, throwError } from 'rxjs';
import { catchError } from 'rxjs/operators';
import { Logger } from '@nestjs/common';

@Injectable()
export class ErrorInterceptor implements NestInterceptor {
  private logger = new Logger('ErrorInterceptor');

  intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
    return next.handle().pipe(
      catchError(error => {
        this.logger.error(`Error occurred: ${error.message}`, error.stack);
        return throwError(error);
      }),
    );
  }
}

Secure Session Management

Session management is critical for web application security, ensuring that user sessions are securely managed and authenticated. NestJS provides built-in support for session management through libraries like express-session, allowing you to configure session settings and implement secure session storage mechanisms.

// session.config.ts
import * as session from 'express-session';
import { INestApplication } from '@nestjs/common';

export function configureSession(app: INestApplication) {
  app.use(
    session({
      secret: process.env.SESSION_SECRET,
      resave: false,
      saveUninitialized: false,
      cookie: { secure: true },
    }),
  );
}

13 . Security Headers and Content Security Policy (CSP)

Implementing security headers and Content Security Policy (CSP) helps mitigate common web security vulnerabilities such as XSS and clickjacking attacks. NestJS allows you to configure security headers and CSP using middleware and global filters.

// security.middleware.ts
import { Injectable, NestMiddleware } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';

@Injectable()
export class SecurityMiddleware implements NestMiddleware {
  use(req: Request, res: Response, next: NextFunction) {
    res.setHeader('X-Frame-Options', 'DENY');
    res.setHeader('X-XSS-Protection', '1; mode=block');
    res.setHeader('Content-Security-Policy', "default-src 'self'");
    next();
  }
}

14 . Secure File Uploads

Secure file uploads are essential for preventing malicious file uploads and protecting against security vulnerabilities such as file inclusion attacks. NestJS provides built-in support for file uploads using libraries like multer, allowing you to implement file upload validation and security measures.

// upload.service.ts
import { Injectable } from '@nestjs/common';
import { diskStorage } from 'multer';
import { extname } from 'path';

@Injectable()
export class UploadService {
  constructor() {}

  multerOptions = {
    storage: diskStorage({
      destination: './uploads',
      filename: (req, file, callback) => {
        const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1e9);
        callback(null, uniqueSuffix + extname(file.originalname));
      },
    }),
    fileFilter: (req, file, callback) => {
      if (!file.originalname.match(/\.(jpg|jpeg|png|gif)$/)) {
        return callback(new Error('Only image files are allowed!'), false);
      }
      callback(null, true);
    },
  };
}

Secure Session Storage

When using session-based authentication, ensure that session data is stored securely to prevent session hijacking and tampering. Use secure session storage mechanisms such as encrypted cookies or session stores with strong encryption and access controls to safeguard session data from unauthorized access.

// session.config.ts
import * as session from 'express-session';
import * as connectRedis from 'connect-redis';
import * as redis from 'redis';
import { INestApplication } from '@nestjs/common';

export function configureSession(app: INestApplication) {
  const RedisStore = connectRedis(session);
  const redisClient = redis.createClient();

  app.use(
    session({
      store: new RedisStore({ client: redisClient }),
      secret: process.env.SESSION_SECRET,
      resave: false,
      saveUninitialized: false,
      cookie: { secure: true },
    }),
  );
}

Incorporating these additional security measures into your NestJS applications, you can enhance overall security resilience and mitigate potential security risks and vulnerabilities effectively. From secure communication and session storage to regular patching and updates, these practices help establish a robust security posture and protect your applications and data from various security threats.

Also, Always Remember to stay informed about emerging security trends and best practices and continuously evaluate and enhance your security measures to address evolving security challenges effectively.

Let's Connect

Linkedln | Github | Twitter | Youtube

Best NestJS Practices and Advanced Techniques

Boyinbode Ebenezer Ayomide — Wed, 23 Aug 2023 19:11:10 +0000

NestJS has quickly gained popularity as a powerful framework for building scalable and maintainable Node.js applications. Its modular architecture, dependency injection system, and support for TypeScript make it an excellent choice for developing complex backend systems. In this blog post, we will explore some of the best practices and advanced techniques for working with NestJS to ensure that your projects are well-structured, performant, and easy to maintain.

Table of Contents

Setting Up a NestJS Project
Modular Architecture with Modules and Services
Dependency Injection and Providers
Controllers and Routing
Data Validation and DTOs
Error Handling and Logging
Authentication and Authorization
Testing with Jest
Performance Optimization
Deployment and Continuous Integration

Setting Up a NestJS Project
To start a new NestJS project, you can use the Nest CLI:

Start by installing the NestJS CLI globally on your local machine and also creating a new Project

$ npm install -g @nestjs/cli
$ nest new project-name

This will create a basic project structure with a default app module.

Modular Architecture with Modules and Services
NestJS encourages a modular architecture to keep your codebase organized. Modules encapsulate related components, and services handle business logic. Here's an example of creating a module and service:

// users.module.ts
import { Module } from '@nestjs/common';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';

@Module({
  controllers: [UsersController],
  providers: [UsersService],
})
export class UsersModule {}

// users.service.ts
import { Injectable } from '@nestjs/common';

@Injectable()
export class UsersService {
  private users = [];

  create(user) {
    this.users.push(user);
  }

  findAll() {
    return this.users;
  }
}

Dependency Injection and Providers
NestJS uses dependency injection to manage component dependencies. Providers are the building blocks of NestJS applications. They can be injected into other components like controllers and services.

// users.controller.ts
import { Controller, Post, Body } from '@nestjs/common';
import { UsersService } from './users.service';

@Controller('users')
export class UsersController {
  constructor(private readonly usersService: UsersService) {}

  @Post()
  create(@Body() user) {
    this.usersService.create(user);
  }
}

Controllers and Routing
Controllers define routes and request handling. They receive incoming requests, process them, and return responses.

// users.controller.ts
import { Controller, Get } from '@nestjs/common';
import { UsersService } from './users.service';

@Controller('users')
export class UsersController {
  constructor(private readonly usersService: UsersService) {}

  @Get()
  findAll() {
    return this.usersService.findAll();
  }
}

** Data Validation and DTOs**
Using Data Transfer Objects (DTOs) helps validate incoming data and ensure type safety.

// create-user.dto.ts
export class CreateUserDto {
  readonly name: string;
  readonly age: number;
  readonly email: string;
}

// users.controller.ts
import { Controller, Post, Body } from '@nestjs/common';
import { UsersService } from './users.service';
import { CreateUserDto } from './create-user.dto';

@Controller('users')
export class UsersController {
  // ...

  @Post()
  create(@Body() createUserDto: CreateUserDto) {
    this.usersService.create(createUserDto);
  }
}

Error Handling and Logging
Implement global exception filters for consistent error handling and use logging for better debugging.

// http-exception.filter.ts
import { ExceptionFilter, Catch, ArgumentsHost, HttpException } from '@nestjs/common';

@Catch(HttpException)
export class HttpExceptionFilter implements ExceptionFilter {
  catch(exception: HttpException, host: ArgumentsHost) {
    // Handle exception and log details
  }
}

Authentication and Authorization
Implement authentication and authorization using guards and strategies to ensure only authorized access to the controller.

// auth.guard.ts
import { Injectable, CanActivate, ExecutionContext } from '@nestjs/common';

@Injectable()
export class AuthGuard implements CanActivate {
  canActivate(context: ExecutionContext) {
    // Implement authentication logic
    return true;
  }
}

Testing with Jest
NestJS supports testing with Jest. Write unit tests for services, controllers, and more.

// users.service.spec.ts
import { Test, TestingModule } from '@nestjs/testing';
import { UsersService } from './users.service';

describe('UsersService', () => {
  let service: UsersService;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [UsersService],
    }).compile();

    service = module.get<UsersService>(UsersService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });
});

Performance Optimization
Performance optimization is crucial to ensuring that your NestJS application runs smoothly and efficiently. By following these techniques, you can enhance the speed, responsiveness, and scalability of your application.

Implement caching mechanisms to store frequently accessed data in memory, reducing the need to fetch it from the database repeatedly. The cache-manager package can be integrated seamlessly with NestJS to manage caching.

$ npm install cache-manager @nestjs/cache

import { CacheModule, Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [CacheModule.register()],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

import { CacheInterceptor, CacheTTL, Controller, Get, UseInterceptors } from '@nestjs/common';
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get('data')
  @UseInterceptors(CacheInterceptor)
  @CacheTTL(10) // Cache for 10 seconds
  getData() {
    return this.appService.getData();
  }
}

Deployment and Continuous Integration

Deploying your NestJS application and setting up a robust continuous integration (CI) pipeline are essential steps in the software development lifecycle. These processes ensure that your application is deployed reliably, tested thoroughly, and can scale seamlessly. Here's how to effectively deploy and manage your NestJS application.

Example Dockerfile:

FROM node:14

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 3000

CMD ["npm", "run", "start:prod"]

For multi-container applications, use Docker Compose to define and manage your application's services and dependencies.

version: '3'
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - '3000:3000'

Setting up CI ensures that your application is automatically built, tested, and deployed whenever changes are pushed to your version control repository. Tools like Jenkins, Travis CI, CircleCI, and GitLab CI/CD can help streamline this process.

Example .gitlab-ci.yml configuration:

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - npm install
    - npm run build

test:
  stage: test
  script:
    - npm run test

deploy:
  stage: deploy
  script:
    - npm install -g @nestjs/cli
    - nest build
    - docker build -t my-nest-app .
    - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
    - docker push my-nest-app

By following these best practices and advanced techniques, you can create robust and maintainable applications using NestJS. Its modular architecture, dependency injection system, and support for TypeScript enable you to build scalable and efficient backend systems with ease.

Let's Connect

Linkedln | Github | Twitter |