Forem: Olamilekan Lamidi

Why I Mentor Junior Developers: Growing the Next Generation of Tech Talent

Olamilekan Lamidi — Sun, 19 Apr 2026 21:20:52 +0000

The Mentoring Gap in Tech

The technology industry has a well-documented pipeline problem. We talk endlessly about hiring, about talent shortages, about the difficulty of finding "senior" engineers. What we talk about far less is the systematic failure to develop the talent we already have.

This gap is especially acute in emerging markets. In Lagos, where I began my career, the demand for experienced software engineers far outstrips supply. Boot camps and university programmes produce developers who can write code, but the journey from writing code to building production systems — understanding architecture, debugging complex distributed systems, making sound technical trade-offs under pressure — that journey requires guided experience. It requires mentorship.

The statistics bear this out. According to a 2023 Stack Overflow Developer Survey, over 60% of developers with fewer than five years of experience report that they lack access to meaningful technical mentorship at work. In African tech ecosystems, where the average engineering team is smaller and companies often cannot afford dedicated developer experience roles, the problem is even more pronounced.

I have seen the consequences firsthand. Junior developers who are talented but directionless, writing code that works but does not scale. Mid-level engineers who have plateaued because nobody ever showed them how to think about system design. Brilliant minds leaving the industry entirely because they felt unsupported and overwhelmed.

This is not a talent problem. It is a mentorship problem. And it is one that senior engineers — those of us who have accumulated the scars and the knowledge — have both the ability and the responsibility to solve.

My Approach to Mentoring

Over the years, I have developed a mentoring approach that goes beyond answering questions when asked. It is proactive, structured, and woven into the daily fabric of engineering work. I learned early that the most effective mentoring does not happen in scheduled one-on-ones (though those matter too). It happens in the moments where engineering decisions are being made in real time.

Code Reviews as Teaching Moments

I treat every code review as a teaching opportunity. When I review a junior developer's pull request, I do not simply approve or request changes. I explain the reasoning behind my suggestions. If I ask someone to refactor a database query, I explain what the current query does to performance at scale. If I suggest a different design pattern, I walk through the trade-offs of both approaches.

At Lordwin Group, where I led a team of five backend developers, I established a code review culture where every pull request received at least two reviews, and at least one review had to include an explanatory comment — not just "change this," but "here is why." The impact was immediate. Junior developers began anticipating architectural concerns in their own code before the review stage. The quality of initial submissions improved dramatically within three months.

I remember reviewing a pull request from a junior developer on the team who had written a working but brittle API endpoint for the gaming platform. Instead of rewriting it myself, I scheduled a 30-minute pair programming session. We walked through the request lifecycle together — how the payload was validated, how the database was queried, how the response was structured, and where the failure points were under load. By the end of the session, the developer had refactored the endpoint themselves, and more importantly, they understood the principles behind the refactoring. That developer went on to independently architect a feature module for our investment platform three months later.

Pair Programming as Skill Transfer

Pair programming is the most underutilised mentoring tool in software engineering. I use it deliberately, especially for complex features or unfamiliar domains. When I was at 2am Tech, collaborating with a cross-functional team of eight engineers building Addio — a financial platform serving 500+ organisations — I regularly paired with less experienced team members on critical backend work.

One engineer on the team was strong in frontend development but had limited backend experience. Rather than assigning them only frontend tasks, I paired with them on building a Node.js service that handled vendor payment workflows. Over several pairing sessions, we worked through API design, database transaction management, error handling patterns, and queue-based architecture for processing payments asynchronously. Within two months, that engineer was confidently building backend services independently and had begun contributing to our architecture discussions.

Olamilekan Lamidi does not hoard knowledge. That principle guides everything I do in a team.

Architecture Discussions as Education

I make a point of including junior and mid-level engineers in architecture discussions, even when their immediate role does not require it. At 54gene, where I worked as a full-stack developer building data collection systems for Africa's leading genomics company, we had a cross-functional team of 10+ members. I advocated for — and helped establish — weekly architecture review sessions where engineers at every level could participate.

These sessions were not presentations. They were collaborative discussions where I would propose an architecture, invite challenges, and walk through the reasoning behind decisions. Why we chose a service-oriented approach for certain modules. Why we implemented specific caching strategies for patient data forms. How we balanced data sensitivity with system performance.

The junior engineers on the team may not have been making those architectural decisions, but by witnessing and participating in the discussion, they were building the mental models they would need when they eventually did. Several of those engineers have since moved into senior roles at other companies — and I know those sessions played a part in their growth.

Structuring Mentorship Beyond Technical Skills

Technical mentoring is essential, but it is not sufficient. The engineers I mentor most effectively are the ones where I invest in three dimensions: technical skills, soft skills, and career navigation.

Technical Skills: Building Depth and Breadth

I structure technical mentoring around progressive complexity. A junior developer on my team does not jump straight from CRUD operations to distributed systems. I map out a learning path:

Foundations: Clean code, testing, version control discipline, debugging methodologies
Intermediate: API design, database optimisation, authentication patterns, error handling at scale
Advanced: System architecture, event-driven patterns, performance profiling, infrastructure as code

At Lordwin Group, I created informal "engineering growth plans" for each of the five developers I led. These were not corporate HR documents — they were living technical roadmaps. Each quarter, I sat with every team member and reviewed where they were, what they had learned, and what they needed to tackle next. One developer who started as a PHP-only backend engineer progressed to confidently working across Node.js and React within eight months through this structured approach.

Soft Skills: Communication, Estimation, and Ownership

I have watched technically brilliant engineers stall in their careers because they could not communicate their ideas, estimate their work, or take ownership of outcomes. Mentoring in these areas is harder — it requires trust and candid conversation — but the payoff is enormous.

I coach engineers on how to present technical proposals to non-technical stakeholders. How to write clear, concise pull request descriptions. How to estimate work by breaking down unknowns. How to own a failure, conduct a post-mortem, and extract lessons without blame. These are the skills that separate engineers who write code from engineers who lead teams.

Career Guidance: Navigating the Path

Especially for engineers in Lagos and across Africa, career navigation can be opaque. Which technologies should you invest in? When should you specialise versus generalise? How do you break into remote roles with international companies? How do you negotiate compensation?

I share my own career trajectory openly — from Civic MediaLab in Lagos to working with teams across Ukraine, the United States, and the United Kingdom. I discuss the decisions I made, the mistakes I made, and the skills that unlocked each transition. For engineers who aspire to work on the global stage, having someone who has walked that path is invaluable.

Impact: Developers Who Grew

Mentoring is difficult to quantify. There is no dashboard that shows "junior developer promoted to senior because Olamilekan Lamidi spent 30 minutes explaining event-driven architecture." But the evidence is there in the trajectories of the people I have worked with.

At Lordwin Group, two of the five developers I led as a senior engineer went on to secure senior engineering positions within 18 months of our time working together. One of them now leads a team of his own at a fintech company in Lagos. He told me directly that the code review culture and architecture discussions I established were the most impactful learning experiences of his early career.

At 2am Tech, the engineer I paired with on backend services went from being a frontend-only developer to a full-stack engineer within one product cycle. They credited the pairing sessions with giving them the confidence to tackle unfamiliar domains.

At 54gene, the junior engineers who participated in our architecture reviews went on to contribute meaningfully to system design decisions by the end of their tenure — a progression that typically takes much longer without structured exposure.

These are not isolated cases. They are the predictable outcome of treating mentoring as engineering work — work that deserves time, intention, and rigour.

Community Engagement: Mentoring Beyond the Workplace

My commitment to mentoring extends beyond the teams I work with directly. I have been active in the Lagos tech community, participating in developer meetups and knowledge-sharing sessions that help early-career engineers navigate the industry. I engage on platforms like ADPList, where I offer mentoring sessions focused on backend engineering, career transitions, and breaking into remote international roles.

I also contribute to the broader engineering community through technical writing — articles like this one, published on Medium, DEV.TO, and LinkedIn — that distil hard-earned lessons into accessible guidance. Every article I publish is, in a sense, a mentoring session at scale. When a junior developer in Nairobi or Accra reads about how I optimised database queries at VacancySoft or architected a real-time gaming backend at Lordwin Group, they are gaining access to production-level knowledge that they might not encounter in their current work environment.

This is not altruism. It is ecosystem building. The stronger the engineers around me, the stronger the teams I work with, the products I contribute to, and the industry as a whole.

The Business Case for Mentoring

For engineering leaders reading this, mentoring is not just a moral good — it is a strategic advantage. The data is compelling:

Faster Onboarding. Teams with structured mentoring practices onboard new engineers 40–50% faster. At Lordwin Group, new developers were contributing production code within their second week, compared to the industry average of four to six weeks. This was a direct result of the code review culture and pairing practices I established.

Better Code Quality. When junior developers understand the "why" behind architectural decisions, they write better code from the start. Our pull request rejection rate at Lordwin Group dropped by roughly 35% within four months of implementing teaching-oriented code reviews.

Reduced Turnover. Engineers who feel invested in stay. In an industry where the average tenure is under two years, creating an environment where developers are actively growing is one of the most effective retention strategies available. The Lordwin team remained remarkably stable during my tenure despite competitive market conditions.

Knowledge Distribution. In teams without mentoring culture, knowledge concentrates in one or two senior engineers. This creates catastrophic bus factor risk. By actively distributing knowledge through mentoring, I ensured that no single engineer — including myself — was a single point of failure.

A Call to Action for Senior Engineers

If you are a senior engineer reading this, I want to challenge you directly: when was the last time you deliberately taught someone something?

Not answered a question in Slack. Not pointed someone to the documentation. Actually sat down, understood where someone was struggling, and helped them build the mental model they needed to solve the problem themselves.

Olamilekan Lamidi's career was shaped by a senior engineer who chose to invest 30 minutes in a confused junior developer at Civic MediaLab in 2017. That investment compounded. It made me a better engineer, which made me a better technical lead, which made the teams I led more effective, which made the products we shipped more robust.

Every senior engineer has this power. The question is whether we choose to use it.

The technology sector does not advance through individual brilliance alone. It advances when knowledge flows freely, when experienced practitioners invest in the next generation, and when the barriers to entry are lowered not by reducing standards, but by raising support.

I mentor because I was mentored. I share because someone shared with me. And I will continue doing so because I have seen, over nine years and across six companies and four countries, that this is how you build an industry that lasts.

Designing Audit-Proof Financial Workflows with Immutable Logs

Olamilekan Lamidi — Fri, 17 Apr 2026 10:10:10 +0000

Why Mutable Records Break Under Audit

Most web applications store the current state of a record. When an invoice status changes from "pending" to "paid," the database row is updated in place. The previous state is gone.

This works fine for most applications. For financial systems, it is a liability:

No history: You cannot prove what the invoice status was yesterday at 3pm.
No attribution: You cannot tell who changed the status or why.
No reconciliation path: When numbers do not match, there is no trail to follow.
No tamper evidence: A modified record looks identical to a legitimate one.

Financial regulations — PCI DSS, SOX, FCA guidelines — require that systems maintain a complete, tamper-evident record of all financial transactions and state changes. An UPDATE statement destroys exactly the evidence you are required to preserve.

The Immutable Log Pattern

The solution is to never update or delete financial records. Instead, every change is recorded as a new, append-only event:

[Event 1] Invoice #1042 CREATED — amount: £5,000 — by: user_42 — at: 2025-03-01T09:15:00Z
[Event 2] Invoice #1042 APPROVED — approved_by: user_15 — at: 2025-03-01T14:30:00Z
[Event 3] Invoice #1042 PAYMENT_RECEIVED — amount: £5,000 — ref: PAY_8821 — at: 2025-03-05T11:00:00Z
[Event 4] Invoice #1042 RECONCILED — matched_to: BANK_TXN_4401 — at: 2025-03-06T08:45:00Z

The current state of the invoice is derived by replaying the events in order. The audit trail is the events themselves. Nothing is lost, nothing is overwritten.

Architecture Overview

[Client Request]
       |
       v
[Application Layer]
  - Validate business rules
  - Create event record
       |
       v
[DB Transaction]
  - Write event to audit_events table (append-only)
  - Update materialised view / current-state table
       |
       v
[Audit Events Table]           [Current State Table]
  (immutable, append-only)      (derived, rebuildable)
       |
       v
[Async Consumers]
  - Compliance reporting
  - Reconciliation engine
  - Notification service

The critical design decision: the audit events table is the source of truth. The current-state table is a projection that can be rebuilt from events at any time.

Implementation in Laravel

Audit Events Migration

Schema::create('audit_events', function (Blueprint $table) {
    $table->uuid('id')->primary();
    $table->string('aggregate_type');
    $table->string('aggregate_id');
    $table->unsignedBigInteger('sequence_number');
    $table->string('event_type');
    $table->json('payload');
    $table->json('metadata');
    $table->string('actor_id');
    $table->string('actor_type');
    $table->string('ip_address')->nullable();
    $table->string('checksum');
    $table->timestamp('occurred_at');
    $table->timestamp('created_at');

    $table->unique(['aggregate_type', 'aggregate_id', 'sequence_number']);
    $table->index(['aggregate_type', 'aggregate_id']);
    $table->index('event_type');
    $table->index('occurred_at');
    $table->index('actor_id');
});

The checksum column stores a hash of the event data plus the previous event's checksum, forming a hash chain. Any tampering with a historical record breaks the chain.

Audit Event Service

namespace App\Services;

use App\Models\AuditEvent;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Str;

class AuditEventService
{
    public function record(
        string $aggregateType,
        string $aggregateId,
        string $eventType,
        array $payload,
        string $actorId,
        string $actorType = 'user',
        ?string $ipAddress = null,
    ): AuditEvent {
        return DB::transaction(function () use (
            $aggregateType, $aggregateId, $eventType,
            $payload, $actorId, $actorType, $ipAddress
        ) {
            $lastEvent = AuditEvent::where('aggregate_type', $aggregateType)
                ->where('aggregate_id', $aggregateId)
                ->orderByDesc('sequence_number')
                ->lockForUpdate()
                ->first();

            $sequenceNumber = $lastEvent ? $lastEvent->sequence_number + 1 : 1;
            $previousChecksum = $lastEvent?->checksum ?? 'GENESIS';

            $eventData = [
                'aggregate_type' => $aggregateType,
                'aggregate_id' => $aggregateId,
                'sequence_number' => $sequenceNumber,
                'event_type' => $eventType,
                'payload' => $payload,
                'actor_id' => $actorId,
                'occurred_at' => now(),
            ];

            $checksum = hash('sha256', $previousChecksum . json_encode($eventData));

            return AuditEvent::create([
                'id' => Str::uuid(),
                ...$eventData,
                'metadata' => [
                    'previous_checksum' => $previousChecksum,
                    'schema_version' => '1.0',
                ],
                'actor_type' => $actorType,
                'ip_address' => $ipAddress,
                'checksum' => $checksum,
                'created_at' => now(),
            ]);
        });
    }

    public function verifyChain(string $aggregateType, string $aggregateId): bool
    {
        $events = AuditEvent::where('aggregate_type', $aggregateType)
            ->where('aggregate_id', $aggregateId)
            ->orderBy('sequence_number')
            ->get();

        $previousChecksum = 'GENESIS';

        foreach ($events as $event) {
            $eventData = [
                'aggregate_type' => $event->aggregate_type,
                'aggregate_id' => $event->aggregate_id,
                'sequence_number' => $event->sequence_number,
                'event_type' => $event->event_type,
                'payload' => $event->payload,
                'actor_id' => $event->actor_id,
                'occurred_at' => $event->occurred_at,
            ];

            $expectedChecksum = hash('sha256', $previousChecksum . json_encode($eventData));

            if ($expectedChecksum !== $event->checksum) {
                return false;
            }

            $previousChecksum = $event->checksum;
        }

        return true;
    }

    public function getHistory(
        string $aggregateType,
        string $aggregateId
    ): \Illuminate\Support\Collection {
        return AuditEvent::where('aggregate_type', $aggregateType)
            ->where('aggregate_id', $aggregateId)
            ->orderBy('sequence_number')
            ->get();
    }
}

Using It in a Financial Workflow

class InvoiceService
{
    public function __construct(
        private AuditEventService $auditService,
    ) {}

    public function createInvoice(array $data, string $userId, string $ip): Invoice
    {
        return DB::transaction(function () use ($data, $userId, $ip) {
            $invoice = Invoice::create([
                'number' => $this->generateInvoiceNumber(),
                'client_id' => $data['client_id'],
                'amount' => $data['amount'],
                'currency' => $data['currency'],
                'status' => 'draft',
                'due_date' => $data['due_date'],
            ]);

            $this->auditService->record(
                aggregateType: 'invoice',
                aggregateId: (string) $invoice->id,
                eventType: 'invoice.created',
                payload: [
                    'number' => $invoice->number,
                    'amount' => $invoice->amount,
                    'currency' => $invoice->currency,
                    'client_id' => $invoice->client_id,
                ],
                actorId: $userId,
                ipAddress: $ip,
            );

            return $invoice;
        });
    }

    public function approveInvoice(Invoice $invoice, string $approverId, string $ip): Invoice
    {
        return DB::transaction(function () use ($invoice, $approverId, $ip) {
            $invoice->update(['status' => 'approved']);

            $this->auditService->record(
                aggregateType: 'invoice',
                aggregateId: (string) $invoice->id,
                eventType: 'invoice.approved',
                payload: [
                    'previous_status' => 'draft',
                    'new_status' => 'approved',
                ],
                actorId: $approverId,
                ipAddress: $ip,
            );

            return $invoice->fresh();
        });
    }
}

Implementation in Node.js

Prisma Schema

model AuditEvent {
  id              String   @id @default(uuid())
  aggregateType   String   @map("aggregate_type")
  aggregateId     String   @map("aggregate_id")
  sequenceNumber  Int      @map("sequence_number")
  eventType       String   @map("event_type")
  payload         Json
  metadata        Json
  actorId         String   @map("actor_id")
  actorType       String   @map("actor_type") @default("user")
  ipAddress       String?  @map("ip_address")
  checksum        String
  occurredAt      DateTime @map("occurred_at")
  createdAt       DateTime @default(now()) @map("created_at")

  @@unique([aggregateType, aggregateId, sequenceNumber])
  @@index([aggregateType, aggregateId])
  @@index([eventType])
  @@index([occurredAt])
  @@index([actorId])
  @@map("audit_events")
}

Audit Event Service

import { PrismaClient } from '@prisma/client';
import crypto from 'crypto';

const prisma = new PrismaClient();

interface RecordEventParams {
  aggregateType: string;
  aggregateId: string;
  eventType: string;
  payload: Record<string, unknown>;
  actorId: string;
  actorType?: string;
  ipAddress?: string;
}

export async function recordAuditEvent(params: RecordEventParams) {
  return prisma.$transaction(async (tx) => {
    const lastEvent = await tx.auditEvent.findFirst({
      where: {
        aggregateType: params.aggregateType,
        aggregateId: params.aggregateId,
      },
      orderBy: { sequenceNumber: 'desc' },
    });

    const sequenceNumber = lastEvent ? lastEvent.sequenceNumber + 1 : 1;
    const previousChecksum = lastEvent?.checksum ?? 'GENESIS';

    const eventData = {
      aggregate_type: params.aggregateType,
      aggregate_id: params.aggregateId,
      sequence_number: sequenceNumber,
      event_type: params.eventType,
      payload: params.payload,
      actor_id: params.actorId,
      occurred_at: new Date().toISOString(),
    };

    const checksum = crypto
      .createHash('sha256')
      .update(previousChecksum + JSON.stringify(eventData))
      .digest('hex');

    return tx.auditEvent.create({
      data: {
        aggregateType: params.aggregateType,
        aggregateId: params.aggregateId,
        sequenceNumber,
        eventType: params.eventType,
        payload: params.payload,
        metadata: {
          previous_checksum: previousChecksum,
          schema_version: '1.0',
        },
        actorId: params.actorId,
        actorType: params.actorType ?? 'user',
        ipAddress: params.ipAddress,
        checksum,
        occurredAt: new Date(),
      },
    });
  });
}

export async function verifyAuditChain(
  aggregateType: string,
  aggregateId: string
): Promise<boolean> {
  const events = await prisma.auditEvent.findMany({
    where: { aggregateType, aggregateId },
    orderBy: { sequenceNumber: 'asc' },
  });

  let previousChecksum = 'GENESIS';

  for (const event of events) {
    const eventData = {
      aggregate_type: event.aggregateType,
      aggregate_id: event.aggregateId,
      sequence_number: event.sequenceNumber,
      event_type: event.eventType,
      payload: event.payload,
      actor_id: event.actorId,
      occurred_at: event.occurredAt.toISOString(),
    };

    const expectedChecksum = crypto
      .createHash('sha256')
      .update(previousChecksum + JSON.stringify(eventData))
      .digest('hex');

    if (expectedChecksum !== event.checksum) {
      return false;
    }

    previousChecksum = event.checksum;
  }

  return true;
}

export async function getAggregateHistory(
  aggregateType: string,
  aggregateId: string
) {
  return prisma.auditEvent.findMany({
    where: { aggregateType, aggregateId },
    orderBy: { sequenceNumber: 'asc' },
  });
}

Express Route for Audit Queries

import { Router } from 'express';

const router = Router();

router.get('/audit/:aggregateType/:aggregateId', async (req, res) => {
  const { aggregateType, aggregateId } = req.params;

  const history = await getAggregateHistory(aggregateType, aggregateId);

  res.json({
    aggregate: { type: aggregateType, id: aggregateId },
    eventCount: history.length,
    events: history.map((e) => ({
      sequence: e.sequenceNumber,
      type: e.eventType,
      payload: e.payload,
      actor: e.actorId,
      occurredAt: e.occurredAt,
    })),
  });
});

router.get('/audit/:aggregateType/:aggregateId/verify', async (req, res) => {
  const { aggregateType, aggregateId } = req.params;
  const valid = await verifyAuditChain(aggregateType, aggregateId);

  res.json({
    aggregate: { type: aggregateType, id: aggregateId },
    chainValid: valid,
    verifiedAt: new Date().toISOString(),
  });
});

Retention and Archival Strategy

Audit logs grow fast. A system processing 10,000 transactions per day generates millions of events per year. You need a retention strategy:

Hot storage (0–90 days): Primary database. Full query capability. Used for day-to-day operations and recent audits.
Warm storage (90 days–2 years): Move older events to a read-optimised store (e.g., partitioned table, read replica, or columnar database). Still queryable but not on the critical path.
Cold storage (2+ years): Archive to object storage (S3, GCS) in a structured format (Parquet, JSON Lines). Retained for regulatory compliance periods (typically 5–7 years for financial data).

Laravel Archival Command

class ArchiveAuditEvents extends Command
{
    protected $signature = 'audit:archive {--older-than=90}';

    public function handle(): void
    {
        $cutoff = now()->subDays((int) $this->option('older-than'));

        $events = AuditEvent::where('occurred_at', '<', $cutoff)
            ->orderBy('occurred_at')
            ->cursor();

        $batch = [];
        $batchCount = 0;

        foreach ($events as $event) {
            $batch[] = $event->toArray();

            if (count($batch) >= 10000) {
                $this->uploadBatch($batch, $batchCount);
                $batch = [];
                $batchCount++;
            }
        }

        if (!empty($batch)) {
            $this->uploadBatch($batch, $batchCount);
        }

        AuditEvent::where('occurred_at', '<', $cutoff)->delete();

        $this->info("Archived and purged events older than {$cutoff->toDateString()}.");
    }

    private function uploadBatch(array $batch, int $index): void
    {
        $filename = sprintf('audit-archive/%s/batch-%04d.json', now()->format('Y/m'), $index);

        Storage::disk('s3')->put(
            $filename,
            collect($batch)->map(fn ($e) => json_encode($e))->implode("\n")
        );
    }
}

Reconciliation with Immutable Logs

One of the most valuable outcomes of immutable logs is automated reconciliation. Instead of manually comparing spreadsheets, you can programmatically verify that every financial event has a matching counterpart:

class ReconciliationService
{
    public function reconcilePayments(string $date): array
    {
        $internalEvents = AuditEvent::where('event_type', 'payment.completed')
            ->whereDate('occurred_at', $date)
            ->get()
            ->keyBy(fn ($e) => $e->payload['gateway_reference']);

        $bankStatements = $this->fetchBankStatements($date);

        $matched = [];
        $unmatchedInternal = [];
        $unmatchedBank = [];

        foreach ($bankStatements as $statement) {
            if ($internalEvents->has($statement->reference)) {
                $matched[] = $statement->reference;
                $internalEvents->forget($statement->reference);
            } else {
                $unmatchedBank[] = $statement;
            }
        }

        $unmatchedInternal = $internalEvents->values()->all();

        return [
            'date' => $date,
            'matched_count' => count($matched),
            'unmatched_internal' => count($unmatchedInternal),
            'unmatched_bank' => count($unmatchedBank),
            'discrepancies' => [...$unmatchedInternal, ...$unmatchedBank],
        ];
    }
}

Production Results

After implementing immutable audit logging across financial platforms:

Metric	Before	After
Reconciliation time (weekly)	4+ hours manual	15 minutes automated
Audit preparation time	2–3 weeks	2 days
Unresolved discrepancies	8–12 per month	0–1 per month
Compliance audit findings	Multiple per review	Zero material findings
Data dispute resolution time	3–5 days	< 1 hour

Key Takeaways

Never update financial records in place. Append new events instead. The current state is a projection of the event history.
Use hash chains for tamper evidence. Each event's checksum includes the previous event's checksum. Any modification breaks the chain.
Record who, what, when, and why. Every event needs an actor, a timestamp, and the business context. IP addresses and session identifiers add forensic value.
Plan for retention from day one. Audit logs grow quickly. Tiered storage (hot/warm/cold) keeps costs manageable while satisfying compliance requirements.
Automate reconciliation. Immutable logs enable programmatic reconciliation that replaces hours of manual spreadsheet comparison.
Write events and business data in the same transaction. If they are not atomic, you risk inconsistency between your source of truth and your audit trail.

Conclusion

Audit-proof financial workflows are not about generating more logs — they are about designing systems where every state change is captured, chained, and verifiable. Append-only writes, hash-chain integrity, and tiered retention create a system that answers any auditor's question with evidence, not explanations.

Whether you build in Laravel or Node.js, the patterns are the same. This infrastructure pays dividends every time a regulator asks a question or a discrepancy needs resolution.

Database Optimization Strategies That Cut Query Execution Time by 60%: A Practical Guide

Olamilekan Lamidi — Wed, 15 Apr 2026 23:40:02 +0000

If your web application is slow, the database is almost certainly the bottleneck. Not the frontend framework. Not the web server. Not the network. The database.

I have spent nine years building backend systems across seven companies, and in every single engagement, database performance has been the most impactful area of optimization. At VacancySoft, where I currently serve as a Full Stack Engineer, I reduced query execution times by 60% across our core platform. At Univelcity, I achieved a 45% reduction in API response times, with database optimization accounting for the majority of those gains. At Lordwin Group, I optimised a real-time gaming backend to sustain 3,000+ concurrent users with sub-100ms query responses.

This article is the practical guide I wish I had when I started. It covers the exact strategies, tools, and techniques I use to diagnose and fix database performance problems in production systems running PostgreSQL and MySQL.

Strategy 1: Query Analysis and Profiling — Know Before You Optimize

The biggest mistake engineers make with database optimization is guessing. They add indexes randomly, rewrite queries based on intuition, or throw caching at the problem without understanding what is actually slow.

I start every optimization effort the same way: profiling.

Enable Slow Query Logging

In PostgreSQL:

-- postgresql.conf
log_min_duration_statement = 200  -- Log queries taking > 200ms
log_statement = 'none'            -- Don't log all queries, just slow ones

In MySQL:

SET GLOBAL slow_query_log = 'ON';
SET GLOBAL long_query_time = 0.2;  -- 200ms threshold
SET GLOBAL log_queries_not_using_indexes = 'ON';

At VacancySoft, I established a weekly ritual of reviewing the slow query log. This alone surfaced dozens of optimization opportunities that no amount of code review would have caught. Queries that looked clean in the ORM were generating catastrophic execution plans.

Use EXPLAIN ANALYZE Religiously

EXPLAIN shows you the query plan. EXPLAIN ANALYZE actually executes the query and shows real timing data. The difference matters.

EXPLAIN ANALYZE
SELECT v.id, v.title, o.name AS organisation_name, s.name AS sector_name
FROM vacancies v
JOIN organisations o ON o.id = v.organisation_id
JOIN sectors s ON s.id = v.sector_id
WHERE v.status = 'active'
  AND v.created_at >= NOW() - INTERVAL '30 days'
ORDER BY v.created_at DESC
LIMIT 20;

Before optimization, this query at VacancySoft returned an EXPLAIN ANALYZE output like this:

Sort  (cost=15234.56..15234.61 rows=20 width=128) (actual time=1842.331..1842.345 rows=20 loops=1)
  Sort Key: v.created_at DESC
  ->  Hash Join  (cost=8234.12..15234.43 rows=20 width=128) (actual time=1201.442..1841.892 rows=24531 loops=1)
        ->  Seq Scan on vacancies v  (cost=0.00..6842.00 rows=24531 width=96) (actual time=0.021..892.134 rows=24531 loops=1)
              Filter: ((status = 'active') AND (created_at >= ...))
              Rows Removed by Filter: 187234
Planning Time: 0.892 ms
Execution Time: 1843.012 ms

Two immediate red flags: a sequential scan on the vacancies table scanning over 200,000 rows and filtering out 187,000 of them, and a hash join that is materialising far more rows than the final result needs.

This is the kind of visibility that transforms optimization from guesswork into engineering. Olamilekan Lamidi's first rule of database optimization: never optimise a query you have not profiled.

Strategy 2: Indexing Strategies — The Highest-Impact Change

Indexing is the single most impactful optimization technique in my toolkit. A well-placed index can turn a 2-second query into a 5-millisecond query. But indexes are not free — they consume storage, slow down writes, and can mislead the query planner if poorly designed.

Composite Indexes

Order matters in composite indexes. The index (status, created_at) is useful for queries that filter on status and sort or filter on created_at. It is useless for queries that filter only on created_at.

For the vacancy search query above, I created:

CREATE INDEX idx_vacancies_status_created_at
ON vacancies (status, created_at DESC);

This single index eliminated the sequential scan. The query planner now uses an index scan that reads only the relevant rows:

Index Scan using idx_vacancies_status_created_at on vacancies v
  (cost=0.42..234.56 rows=20 width=96) (actual time=0.089..2.341 rows=20 loops=1)
  Index Cond: ((status = 'active') AND (created_at >= ...))

Execution time: from 1,843ms to 12ms. A 99.3% reduction from a single composite index.

Partial Indexes

At VacancySoft, the majority of our queries filter on status = 'active'. Only about 15% of vacancies are active at any given time. A partial index dramatically reduces the index size:

CREATE INDEX idx_vacancies_active_created_at
ON vacancies (created_at DESC)
WHERE status = 'active';

This index is roughly 85% smaller than the full composite index, which means it fits in memory more easily and produces faster index scans.

Covering Indexes (Index-Only Scans)

A covering index includes all columns that a query needs, allowing PostgreSQL to serve the query entirely from the index without touching the heap (the actual table data). This eliminates random I/O, which is the most expensive operation on traditional storage.

CREATE INDEX idx_vacancies_covering
ON vacancies (status, created_at DESC)
INCLUDE (id, title, organisation_id, sector_id)
WHERE status = 'active';

With this covering index, PostgreSQL executes an index-only scan:

Index Only Scan using idx_vacancies_covering on vacancies v
  (cost=0.42..45.23 rows=20 width=96) (actual time=0.067..0.412 rows=20 loops=1)
  Heap Fetches: 0

Heap Fetches: 0 is the goal. The database never touches the table at all.

My Indexing Methodology

I follow a systematic process:

Identify the top 20 slowest queries from the slow query log.
Run EXPLAIN ANALYZE on each one.
Look for sequential scans on tables with more than 10,000 rows.
Check for sort operations that are not backed by an index.
Create targeted indexes and verify the plan changes.
Monitor write performance to ensure indexes are not degrading INSERT/UPDATE speed beyond acceptable thresholds.

At VacancySoft, this methodology across 15+ modules produced the 60% aggregate reduction in query execution time.

Strategy 3: Query Refactoring — Fixing What the ORM Gets Wrong

ORMs are excellent for productivity but frequently generate suboptimal SQL. The N+1 query problem is the most well-known example, but I encounter more subtle issues regularly.

Eliminating N+1 Queries

In Laravel (which I used extensively at 2am Tech, Lordwin Group, and Viaduct):

// Bad: N+1 problem — 1 query for organisations + N queries for vacancies
$organisations = Organisation::all();
foreach ($organisations as $org) {
    echo $org->vacancies->count(); // Triggers a query per organisation
}

// Good: Eager loading — 2 queries total
$organisations = Organisation::with('vacancies')->get();

// Better: When you only need the count
$organisations = Organisation::withCount('vacancies')->get();
foreach ($organisations as $org) {
    echo $org->vacancies_count; // No additional queries
}

At 2am Tech, the Addio financial platform was suffering from response times exceeding 3 seconds on the organisation dashboard. Profiling revealed over 150 queries per page load, almost all of them N+1 queries from Eloquent relationships. I refactored the critical paths to use eager loading and withCount, reducing the query count to 8 and the response time to 400ms.

Replacing Subqueries with JOINs and CTEs

ORMs often generate correlated subqueries where a JOIN or CTE would be far more efficient:

-- ORM-generated: correlated subquery (executes once per row)
SELECT o.id, o.name,
  (SELECT COUNT(*) FROM vacancies v WHERE v.organisation_id = o.id AND v.status = 'active')
    AS active_vacancy_count
FROM organisations o
WHERE o.sector_id = 5;

-- Refactored: single pass with JOIN
SELECT o.id, o.name, COUNT(v.id) AS active_vacancy_count
FROM organisations o
LEFT JOIN vacancies v ON v.organisation_id = o.id AND v.status = 'active'
WHERE o.sector_id = 5
GROUP BY o.id, o.name;

Window Functions for Complex Analytics

At VacancySoft, we needed to show month-over-month vacancy trends per sector. The initial implementation used multiple queries with application-level aggregation. I replaced it with a single query using window functions:

WITH monthly_counts AS (
  SELECT
    s.name AS sector_name,
    DATE_TRUNC('month', v.created_at) AS month,
    COUNT(*) AS vacancy_count
  FROM vacancies v
  JOIN sectors s ON s.id = v.sector_id
  WHERE v.created_at >= NOW() - INTERVAL '12 months'
  GROUP BY s.name, DATE_TRUNC('month', v.created_at)
)
SELECT
  sector_name,
  month,
  vacancy_count,
  LAG(vacancy_count) OVER (PARTITION BY sector_name ORDER BY month) AS prev_month_count,
  ROUND(
    (vacancy_count - LAG(vacancy_count) OVER (PARTITION BY sector_name ORDER BY month))::numeric
    / NULLIF(LAG(vacancy_count) OVER (PARTITION BY sector_name ORDER BY month), 0) * 100,
    1
  ) AS month_over_month_pct
FROM monthly_counts
ORDER BY sector_name, month;

This single query replaced five application-level queries and a significant amount of JavaScript aggregation logic. Response time for the trends endpoint dropped from 2.1 seconds to 280ms.

Strategy 4: Schema Redesign for Performance

Sometimes the problem is not the query — it is the schema. Normalization is taught as gospel in database courses, but production systems often require strategic denormalization.

Materialized Views

At VacancySoft, our analytics dashboard required aggregating data across five tables. Even with optimized queries and proper indexes, the aggregation was too slow for real-time display. I introduced materialized views:

CREATE MATERIALIZED VIEW mv_sector_analytics AS
SELECT
  s.id AS sector_id,
  s.name AS sector_name,
  COUNT(v.id) AS total_vacancies,
  COUNT(v.id) FILTER (WHERE v.status = 'active') AS active_vacancies,
  COUNT(DISTINCT v.organisation_id) AS hiring_organisations,
  PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY v.salary_max) AS median_salary,
  DATE_TRUNC('day', NOW()) AS snapshot_date
FROM sectors s
LEFT JOIN vacancies v ON v.sector_id = s.id
  AND v.created_at >= NOW() - INTERVAL '90 days'
GROUP BY s.id, s.name;

CREATE UNIQUE INDEX ON mv_sector_analytics (sector_id);

-- Refresh concurrently (no lock on reads)
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_sector_analytics;

I set up a cron job to refresh this view every 15 minutes. The dashboard query that previously took 3.8 seconds now completes in under 10ms because it reads from the pre-computed view.

Strategic Denormalization

For the organisation profile pages, rather than joining five tables on every request, I added a cached_stats JSONB column on the organisations table:

ALTER TABLE organisations ADD COLUMN cached_stats JSONB DEFAULT '{}';

-- Updated by a background job every hour
UPDATE organisations SET cached_stats = jsonb_build_object(
  'active_vacancies', (SELECT COUNT(*) FROM vacancies WHERE organisation_id = organisations.id AND status = 'active'),
  'total_vacancies_90d', (SELECT COUNT(*) FROM vacancies WHERE organisation_id = organisations.id AND created_at >= NOW() - INTERVAL '90 days'),
  'avg_time_to_fill', (SELECT AVG(EXTRACT(EPOCH FROM (filled_at - created_at)) / 86400) FROM vacancies WHERE organisation_id = organisations.id AND filled_at IS NOT NULL),
  'updated_at', NOW()
);

This is a tradeoff: the data can be up to one hour stale, and we have the overhead of the background update job. But for this use case, slight staleness is acceptable, and the performance gain — from 450ms to 8ms per profile page — is transformative.

Strategy 5: Connection Pooling and Read Replicas

At scale, query optimization alone is not enough. You also need to manage how connections to the database are handled.

Connection Pooling with PgBouncer

PostgreSQL creates a new process for each connection, which is expensive. At VacancySoft, I deployed PgBouncer in front of PostgreSQL in transaction pooling mode:

; pgbouncer.ini
[databases]
vacancysoft = host=127.0.0.1 port=5432 dbname=vacancysoft

[pgbouncer]
listen_port = 6432
pool_mode = transaction
max_client_conn = 500
default_pool_size = 25
reserve_pool_size = 5

This allows 500 application connections to share 25 database connections. Without pooling, our application would regularly hit PostgreSQL's connection limit during traffic spikes, causing cascading failures.

Read Replicas

For read-heavy workloads (which describe most web applications), I split reads and writes across primary and replica databases:

// Database configuration with read/write splitting
const knex = require('knex');

const primary = knex({
  client: 'pg',
  connection: process.env.DATABASE_PRIMARY_URL,
});

const replica = knex({
  client: 'pg',
  connection: process.env.DATABASE_REPLICA_URL,
});

// Service layer decides which connection to use
class VacancyService {
  async search(filters) {
    return replica('vacancies')
      .where(filters)
      .orderBy('created_at', 'desc');
  }

  async create(data) {
    return primary('vacancies')
      .insert(data)
      .returning('*');
  }
}

At VacancySoft, this split reduced load on the primary database by approximately 70%, since the vast majority of our traffic is read operations (search, analytics, reporting).

Strategy 6: Caching Layers

The fastest database query is the one you never make. Caching is the final layer of my optimization strategy.

Redis for Hot Data

const Redis = require('ioredis');
const redis = new Redis(process.env.REDIS_URL);

async function getSectorAnalytics(sectorId) {
  const cacheKey = `sector:${sectorId}:analytics`;
  const cached = await redis.get(cacheKey);

  if (cached) {
    return JSON.parse(cached);
  }

  const data = await db('mv_sector_analytics')
    .where('sector_id', sectorId)
    .first();

  await redis.setex(cacheKey, 900, JSON.stringify(data)); // 15-minute TTL

  return data;
}

Cache Invalidation Strategy

Cache invalidation is famously one of the two hard problems in computer science. My approach is pragmatic:

TTL-based expiration for data that can tolerate staleness (analytics, aggregations).
Event-driven invalidation for data that must be fresh (user profiles, active listings). When a vacancy is created or updated, I publish an event that invalidates the relevant cache keys.
Cache warming for predictable access patterns. At VacancySoft, I pre-warm the cache for the top 50 sectors and top 100 organisations every 15 minutes, so the first user to access these pages never experiences a cache miss.

// Event-driven cache invalidation
eventBus.on('vacancy:created', async (vacancy) => {
  await redis.del(`sector:${vacancy.sector_id}:analytics`);
  await redis.del(`org:${vacancy.organisation_id}:stats`);
  await redis.del('homepage:featured_vacancies');
});

Results: Before and After

Here is a summary of the measurable impact these strategies produced across my career, as implemented by Olamilekan Lamidi:

Company	Metric	Before	After	Improvement
VacancySoft	Average query execution time	850ms	340ms	60% reduction
VacancySoft	Dashboard load time	3.8s	0.4s	89% reduction
VacancySoft	p95 API response time	1.2s	180ms	85% reduction
Univelcity	API response time (avg)	620ms	340ms	45% reduction
Univelcity	Queries per page load	45	8	82% reduction
Lordwin Group	Real-time query latency	220ms	35ms	84% reduction
2am Tech	Organisation dashboard load	3.1s	400ms	87% reduction

Key Takeaways

Profile before you optimize. EXPLAIN ANALYZE and slow query logs should be your starting point, not your intuition.
Composite and partial indexes are the highest-leverage optimization in most applications. One well-placed index can eliminate 99% of query time.
ORMs are productivity tools, not performance tools. Review the SQL they generate, especially for complex queries.
Materialized views and strategic denormalization are not anti-patterns — they are essential tools for read-heavy workloads.
Connection pooling is mandatory once you are handling more than a few hundred concurrent connections.
Caching is the last layer, not the first. Optimize the underlying queries before adding cache complexity.

Database optimization is not a one-time project. It is an ongoing discipline. The queries that perform well today may become bottlenecks tomorrow as data grows and access patterns change. Build profiling and monitoring into your workflow, and treat database performance as a first-class engineering concern.

The Art of API Design: Principles I Learned Building APIs That Handle 50,000+ Daily Requests

Olamilekan Lamidi — Sun, 12 Apr 2026 21:02:36 +0000

APIs are the backbone of modern software. They power every mobile app you use, every SaaS dashboard you log into, and every integration between systems that makes the modern web work. Yet despite their ubiquity, most APIs are designed poorly. They leak database schema into response payloads, return cryptic error codes, break backward compatibility without warning, and crumble under load.

I know this because I have spent the past nine years building, consuming, and debugging APIs across seven companies, multiple industries, and technology stacks spanning PHP (Laravel, Symfony) and JavaScript (Node.js, Express, NestJS). I have built APIs for health-tech platforms at 54gene, financial infrastructure at 2am Tech, real-time gaming at Lordwin Group, on-demand delivery at Viaduct, education technology at Univelcity, and enterprise SaaS at VacancySoft. Collectively, these systems have handled millions of requests and served tens of thousands of users.

Through that experience, I have distilled a set of principles that separate well-designed APIs from fragile, frustrating ones. These are not theoretical musings — they are battle-tested patterns drawn from production systems that process 50,000+ daily requests at VacancySoft and have served over 8,000 users at Univelcity with 20,000+ requests per day.

This article is a deep dive into those principles.

Principle 1: Design for the Consumer, Not the Database

The most common API design mistake I encounter is what I call "schema leaking" — when an API's resource structure mirrors the underlying database tables rather than reflecting how consumers actually use the data.

Early in my career at Univelcity, I fell into this trap. We had a users table, a courses table, and a enrollments join table. The first version of our API had three corresponding endpoints that returned raw table structures. Clients had to make three separate requests and stitch data together themselves.

The fix was to think in terms of resources, not tables. A consumer doesn't care about your join table. They care about "a student's enrolled courses" or "a course's enrolled students." The API should model those concepts directly:

// Bad: Database-centric design
GET /users/123
GET /enrollments?user_id=123
GET /courses/456

// Good: Consumer-centric design
GET /students/123/courses
GET /courses/456/students

At VacancySoft, I apply this principle rigorously. Our APIs expose business concepts — vacancies, organisations, sectors, market trends — not the complex graph of PostgreSQL tables underneath. When we introduced a "market snapshot" feature, rather than forcing clients to aggregate data from four different endpoints, I designed a single /market-snapshots resource that returned a pre-composed view of the data clients needed. This reduced client-side complexity dramatically and cut the average number of API calls per page load from seven to two.

The key insight: your database schema is an implementation detail. Your API contract is a product. Treat it like one.

Principle 2: Versioning Strategies That Don't Break Clients

API versioning is one of those topics where everyone has an opinion and nobody agrees. I have used three approaches across my career: URI versioning, header versioning, and query parameter versioning. Here is what I have learned.

URI versioning (/api/v1/vacancies) is what I use in most production systems and what we use at VacancySoft. It is explicit, easy to understand, easy to route, and easy to document. The tradeoff is that it can lead to code duplication if you are not careful about abstracting shared logic between versions.

Header versioning (Accept: application/vnd.api+json; version=2) is cleaner in theory but harder to test, harder to debug with basic tools like curl, and harder for less experienced developers to consume.

Query parameter versioning (/api/vacancies?version=2) is the worst of both worlds and I do not recommend it.

My approach at VacancySoft:

// Express router with versioned routes
const v1Router = express.Router();
const v2Router = express.Router();

app.use('/api/v1', v1Router);
app.use('/api/v2', v2Router);

// Shared business logic lives in services, not controllers
// Controllers are thin version-specific adapters
v1Router.get('/vacancies', async (req, res) => {
  const data = await vacancyService.search(req.query);
  res.json(transformV1(data));
});

v2Router.get('/vacancies', async (req, res) => {
  const data = await vacancyService.search(req.query);
  res.json(transformV2(data));
});

The critical rule I follow: never remove a version without a deprecation period and proactive communication to consumers. At VacancySoft, when we deprecated v1 of our vacancy search endpoint, I implemented a deprecation header (Sunset: Sat, 01 Jul 2024 00:00:00 GMT) six months in advance and added logging to track which clients were still using v1. This gave us data-driven confidence to retire the old version without breaking anyone.

Principle 3: Pagination, Filtering, and Sorting at Scale

Pagination seems simple until you are dealing with datasets of hundreds of thousands of records that change frequently. I have implemented both offset-based and cursor-based pagination in production, and the difference matters enormously at scale.

Offset pagination (?page=5&per_page=20) is intuitive but fundamentally broken for large datasets. The database still has to scan all rows up to the offset, meaning page 500 is orders of magnitude slower than page 1. It also produces inconsistent results when records are inserted or deleted between requests.

At VacancySoft, where our vacancy dataset exceeds hundreds of thousands of records and is updated continuously, I implemented cursor-based pagination:

// Cursor-based pagination implementation
async function getVacancies(cursor, limit = 20) {
  const query = knex('vacancies')
    .orderBy('created_at', 'desc')
    .orderBy('id', 'desc')
    .limit(limit + 1);

  if (cursor) {
    const { created_at, id } = decodeCursor(cursor);
    query.where(function () {
      this.where('created_at', '<', created_at)
        .orWhere(function () {
          this.where('created_at', '=', created_at)
            .andWhere('id', '<', id);
        });
    });
  }

  const results = await query;
  const hasMore = results.length > limit;
  const items = hasMore ? results.slice(0, -1) : results;

  return {
    data: items,
    pagination: {
      has_more: hasMore,
      next_cursor: hasMore
        ? encodeCursor(items[items.length - 1])
        : null,
    },
  };
}

function encodeCursor(record) {
  return Buffer.from(
    JSON.stringify({
      created_at: record.created_at,
      id: record.id,
    })
  ).toString('base64');
}

This approach uses a composite cursor of (created_at, id) to ensure stable ordering even with duplicate timestamps. It performs consistently regardless of how deep into the dataset a client paginates, because the database uses an index seek rather than a scan.

For filtering, I standardise on a predictable pattern:

GET /api/v2/vacancies?sector=technology&region=london&salary_min=50000&sort=-created_at

The - prefix for descending sort is a convention I adopted from JSON:API and have used consistently across every API I have built since 2020. It is intuitive and self-documenting.

Principle 4: Error Handling That Helps Developers

Nothing frustrates an API consumer more than unhelpful errors. I have received 500 Internal Server Error with an empty body from third-party APIs, and I have vowed never to do that to my consumers.

Every API I build follows a consistent error schema:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "The request body contains invalid fields.",
    "details": [
      {
        "field": "email",
        "message": "Must be a valid email address.",
        "received": "not-an-email"
      },
      {
        "field": "salary_min",
        "message": "Must be a positive integer.",
        "received": -500
      }
    ],
    "request_id": "req_8f3a2b1c",
    "documentation_url": "https://api.example.com/docs/errors#VALIDATION_ERROR"
  }
}

Key principles:

Machine-readable error codes (VALIDATION_ERROR, not just HTTP status codes) so clients can programmatically handle different error types.
Human-readable messages so developers can understand what went wrong without consulting documentation.
Field-level detail for validation errors so frontend developers know exactly which fields to highlight.
Request IDs so that when a consumer reports an issue, I can trace it through our logging infrastructure.
Documentation URLs that link directly to the relevant error documentation.

At VacancySoft, I implemented a centralised error handling middleware that ensures every error — whether it is a validation failure, an authorization error, or an unexpected exception — passes through the same formatting pipeline:

// Centralised error handler middleware
function errorHandler(err, req, res, next) {
  const requestId = req.headers['x-request-id'] || generateRequestId();

  if (err instanceof ValidationError) {
    return res.status(422).json({
      error: {
        code: 'VALIDATION_ERROR',
        message: err.message,
        details: err.details,
        request_id: requestId,
      },
    });
  }

  if (err instanceof AuthorizationError) {
    return res.status(403).json({
      error: {
        code: 'FORBIDDEN',
        message: 'You do not have permission to access this resource.',
        request_id: requestId,
      },
    });
  }

  // Unexpected errors: log full stack, return safe message
  logger.error({
    request_id: requestId,
    error: err.message,
    stack: err.stack,
    path: req.path,
    method: req.method,
  });

  return res.status(500).json({
    error: {
      code: 'INTERNAL_ERROR',
      message: 'An unexpected error occurred. Please try again.',
      request_id: requestId,
    },
  });
}

This approach ensures that internal details (stack traces, database errors) never leak to consumers, while still providing enough information for debugging.

Principle 5: Rate Limiting and Throttling for Protection

Any API exposed to the internet will eventually be abused — whether by a misconfigured client making thousands of requests in a loop, a scraper, or a deliberate attack. Rate limiting is not optional.

I implement rate limiting at multiple layers:

Application-level rate limiting using Redis-backed token bucket or sliding window algorithms:

const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const Redis = require('ioredis');

const redisClient = new Redis(process.env.REDIS_URL);

const apiLimiter = rateLimit({
  store: new RedisStore({
    client: redisClient,
    prefix: 'rl:',
  }),
  windowMs: 60 * 1000, // 1 minute
  max: 100,            // 100 requests per minute
  standardHeaders: true,
  legacyHeaders: false,
  handler: (req, res) => {
    res.status(429).json({
      error: {
        code: 'RATE_LIMIT_EXCEEDED',
        message: 'Too many requests. Please retry after the window resets.',
        retry_after: res.getHeader('Retry-After'),
      },
    });
  },
});

// Stricter limits for authentication endpoints
const authLimiter = rateLimit({
  store: new RedisStore({ client: redisClient, prefix: 'rl:auth:' }),
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 10,                   // 10 attempts per 15 minutes
  standardHeaders: true,
});

app.use('/api/', apiLimiter);
app.use('/api/auth/', authLimiter);

Infrastructure-level rate limiting via Nginx or a CDN as the first line of defence. At VacancySoft, we use Nginx rate limiting in front of our Node.js services to catch abusive traffic before it reaches the application layer. This dual-layer approach proved critical during a traffic spike when a partner's integration went haywire and started hitting our API at 10x the normal rate. Nginx absorbed the burst, and the application-level limiter gracefully degraded the remaining overflow with proper 429 responses.

I always include RateLimit-* headers in responses so well-behaved clients can self-throttle:

RateLimit-Limit: 100
RateLimit-Remaining: 67
RateLimit-Reset: 1719849600

Principle 6: Authentication and Authorization Patterns

Authentication (who are you?) and authorization (what can you do?) are distinct concerns that I see conflated constantly. Over my career I have implemented multiple patterns, and the right choice depends heavily on context.

For service-to-service communication within a backend, I use short-lived JWT tokens with asymmetric key signing (RS256). The issuing service signs with a private key, and consuming services verify with the public key without needing to call back to the auth service:

const jwt = require('jsonwebtoken');

// Token generation (auth service)
function generateAccessToken(user) {
  return jwt.sign(
    {
      sub: user.id,
      email: user.email,
      roles: user.roles,
      permissions: user.permissions,
    },
    privateKey,
    {
      algorithm: 'RS256',
      expiresIn: '15m',
      issuer: 'auth.vacancysoft.com',
    }
  );
}

// Middleware for route-level authorization
function requirePermission(permission) {
  return (req, res, next) => {
    if (!req.user || !req.user.permissions.includes(permission)) {
      return res.status(403).json({
        error: {
          code: 'FORBIDDEN',
          message: `Required permission: ${permission}`,
        },
      });
    }
    next();
  };
}

// Usage
router.delete(
  '/vacancies/:id',
  authenticate,
  requirePermission('vacancies:delete'),
  vacancyController.destroy
);

At VacancySoft, I implemented a role-based access control (RBAC) system with granular permissions. Rather than checking if (user.role === 'admin') throughout the codebase, permissions are encoded in the JWT and checked via middleware. This made it trivial to add new roles — when the product team introduced a "regional analyst" role, I added the appropriate permissions without touching a single controller.

For external-facing APIs, I use API keys for identification combined with OAuth 2.0 for delegated access when third-party integrations need to act on behalf of users.

Principle 7: Documentation as a First-Class Citizen

I have a rule: if an API endpoint ships without documentation, it does not ship. Documentation is not an afterthought — it is a deliverable.

I use the OpenAPI 3.0 specification (Swagger) for every API I build, and I generate the spec from code annotations rather than maintaining a separate document that inevitably drifts:

/**
 * @openapi
 * /api/v2/vacancies:
 *   get:
 *     summary: Search vacancies
 *     description: >
 *       Returns a paginated, filtered list of vacancies.
 *       Supports cursor-based pagination for stable results.
 *     tags: [Vacancies]
 *     parameters:
 *       - in: query
 *         name: sector
 *         schema:
 *           type: string
 *         description: Filter by sector slug
 *       - in: query
 *         name: cursor
 *         schema:
 *           type: string
 *         description: Pagination cursor from previous response
 *     responses:
 *       200:
 *         description: Paginated list of vacancies
 *         content:
 *           application/json:
 *             schema:
 *               $ref: '#/components/schemas/VacancyListResponse'
 *       429:
 *         description: Rate limit exceeded
 */
router.get('/vacancies', vacancyController.search);

At VacancySoft, our Swagger documentation is auto-generated from these annotations and published to an internal documentation portal. Every endpoint includes request/response examples, error codes, and authentication requirements. When new team members onboard, they can explore and test every endpoint interactively without reading a single line of source code.

I also generate Postman collections from the OpenAPI spec, which our QA team and integration partners use for testing. This single-source-of-truth approach eliminates the documentation drift that plagues most API teams.

The Results

These principles are not academic exercises. Applied consistently at VacancySoft, they have produced measurable outcomes:

Metric	Result
Daily request volume	50,000+
API uptime	99.8%+ over the last 12 months
Average response time (p50)	45ms
Average response time (p95)	180ms
Client integration time	Reduced from weeks to days
Breaking changes in production	Zero in the last 18 months
Support tickets related to API confusion	Down 70% after error schema standardisation

Olamilekan Lamidi's approach to API design has been shaped by one core belief: an API is a product, and its consumers are your users. Every decision — from resource naming to error formatting to pagination strategy — should be made with the consumer's experience in mind.

Key Takeaways

Model resources around consumer use cases, not database tables.
Use URI versioning with a clear deprecation policy and sunset headers.
Implement cursor-based pagination for any dataset that changes frequently or exceeds a few thousand records.
Standardise error responses with machine-readable codes, human-readable messages, and request IDs.
Rate limit at multiple layers — infrastructure and application — and communicate limits via headers.
Separate authentication from authorization and encode permissions in tokens for stateless checks.
Treat documentation as a deliverable, not an afterthought. Generate it from code.

These principles have served me across every company and technology stack I have worked with. Whether you are building a Laravel API in PHP or an Express API in Node.js, the fundamentals remain the same: design for clarity, build for resilience, and always, always think about the person on the other side of the request.

Engineering Real-Time Delivery Tracking That Increased Checkout Conversions by 20%

Olamilekan Lamidi — Thu, 09 Apr 2026 22:46:21 +0000

In e-commerce and on-demand delivery, the moment a user clicks "Place Order" begins an anxiety cycle. Where is my order? When will it arrive? This uncertainty kills conversions.

Between November 2020 and July 2021, I worked as a Full-Stack Developer at Viaduct, where I engineered a real-time delivery tracking system for an on-demand platform serving 5,000+ active users. The platform connected customers with local vendors for same-day delivery. Before I joined, users were abandoning carts and cancelling orders at alarming rates because they had zero visibility into delivery status.

This article details how I designed and built the tracking system — Google Maps integration, WebSocket-powered live updates, a Vue.js reactive interface, and the Laravel backend tying it together. The result: a 20% increase in checkout conversions.

The Problem: Delivery Opacity Was Killing Revenue

When I joined Viaduct, the delivery platform was functional but opaque. Users placed orders, received a confirmation email, and waited. The only updates were manual — a dispatcher marked orders "out for delivery" when a driver departed and "delivered" when reported back.

The consequences were measurable:

Cart abandonment: 32% of users abandoned at checkout. Exit surveys showed the top reason was "uncertain delivery time."

Post-order cancellations: 15% of completed checkouts were cancelled within 30 minutes, most citing "no delivery updates."

Support overload: 200+ daily calls asking "where is my order?" — each averaging 4 minutes, consuming 13 person-hours daily.

Driver accountability gaps: Without GPS tracking, there was no way to verify delivery times, identify route inefficiencies, or resolve disputes.

My Role: Full-Stack Architect of the Tracking System

I designed and built the entire real-time tracking system end-to-end: the Laravel backend infrastructure, Google Maps API integration, the WebSocket communication layer, and the Vue.js frontend tracking interface. I owned it from initial technical design through production deployment and post-launch optimisation.

Technical Deep Dive: Building the Real-Time Tracking Infrastructure

1. Driver Location Collection Architecture

The foundation of real-time tracking is reliable location data. I built a location reporting service on drivers' devices, transmitting GPS coordinates at configurable intervals.

class DriverLocationController extends Controller
{
    public function updateLocation(UpdateLocationRequest $request): JsonResponse
    {
        $driver = $request->user();
        $validated = $request->validated();

        $location = DriverLocation::create([
            'driver_id' => $driver->id,
            'latitude' => $validated['latitude'],
            'longitude' => $validated['longitude'],
            'accuracy' => $validated['accuracy'],
            'speed' => $validated['speed'] ?? null,
            'heading' => $validated['heading'] ?? null,
            'recorded_at' => Carbon::createFromTimestamp($validated['timestamp']),
        ]);

        if ($activeDelivery = $driver->activeDelivery) {
            $this->broadcastLocationUpdate($activeDelivery, $location);
            $this->updateEstimatedArrival($activeDelivery, $location);
        }

        return response()->json(['status' => 'recorded'], 200);
    }

    private function broadcastLocationUpdate(Delivery $delivery, DriverLocation $location): void
    {
        Redis::publish("delivery:{$delivery->id}:location", json_encode([
            'delivery_id' => $delivery->id,
            'driver' => [
                'latitude' => $location->latitude,
                'longitude' => $location->longitude,
                'heading' => $location->heading,
            ],
            'timestamp' => $location->recorded_at->toIso8601String(),
        ]));
    }
}

Location updates published to Redis channels, bridging the Laravel backend and the WebSocket server. This decoupled location ingestion (high-frequency HTTP POSTs from drivers) from location broadcasting (persistent WebSocket connections to customers), allowing each to scale independently.

2. Google Maps Integration for ETA Calculation

Accurate ETAs were critical. I integrated the Google Maps Directions API with live traffic data:

class GoogleMapsService
{
    public static function getDirections(array $origin, array $destination): ?DirectionsResult
    {
        $response = Http::get('https://maps.googleapis.com/maps/api/directions/json', [
            'origin' => "{$origin['lat']},{$origin['lng']}",
            'destination' => "{$destination['lat']},{$destination['lng']}",
            'mode' => 'driving',
            'departure_time' => 'now',
            'traffic_model' => 'best_guess',
            'key' => config('services.google_maps.api_key'),
        ]);

        if (!$response->successful() || $response->json('status') !== 'OK') {
            return null;
        }

        $leg = $response->json('routes.0.legs.0');
        return new DirectionsResult(
            durationMinutes: (int) ceil($leg['duration_in_traffic']['value'] / 60),
            distanceKm: round($leg['distance']['value'] / 1000, 1),
            polyline: $response->json('routes.0.overview_polyline.points'),
        );
    }
}

Using departure_time=now and traffic_model=best_guess ensured ETAs reflected live traffic — during rush hours, static estimates might show 10 minutes while actual time was 25. However, calling Google Maps on every location update (every 10 seconds per driver) would be prohibitively expensive. I implemented intelligent throttling using Haversine distance:

class ETAService
{
    private const ETA_CACHE_TTL = 60;
    private const MIN_DISTANCE_FOR_RECALC = 0.2; // km

    public function shouldRecalculateETA(Delivery $delivery, DriverLocation $newLocation): bool
    {
        if (!$delivery->eta_updated_at) return true;
        if (now()->diffInSeconds($delivery->eta_updated_at) < self::ETA_CACHE_TTL) return false;

        $lastCalcLocation = Cache::get("eta_calc_location:{$delivery->id}");
        if (!$lastCalcLocation) return true;

        return $this->haversineDistance(
            $lastCalcLocation['lat'], $lastCalcLocation['lng'],
            $newLocation->latitude, $newLocation->longitude
        ) >= self::MIN_DISTANCE_FOR_RECALC;
    }

    private function haversineDistance(float $lat1, float $lon1, float $lat2, float $lon2): float
    {
        $dLat = deg2rad($lat2 - $lat1);
        $dLon = deg2rad($lon2 - $lon1);
        $a = sin($dLat/2)**2 + cos(deg2rad($lat1)) * cos(deg2rad($lat2)) * sin($dLon/2)**2;
        return 6371 * 2 * atan2(sqrt($a), sqrt(1 - $a));
    }
}

ETAs were recalculated only when the driver moved 200m+ and 60+ seconds had elapsed, reducing Google Maps API calls by ~80%.

3. WebSocket Server for Live Customer Updates

The real-time frontend was powered by a Node.js WebSocket server subscribed to Redis channels:

const wss = new WebSocket.Server({ port: 6001 });
const subscriber = new Redis(process.env.REDIS_URL);
const deliverySubscriptions = new Map();

wss.on('connection', (ws) => {
  ws.on('message', (raw) => {
    const msg = JSON.parse(raw);
    if (msg.type === 'subscribe_delivery') {
      if (!deliverySubscriptions.has(msg.deliveryId)) {
        deliverySubscriptions.set(msg.deliveryId, new Set());
        subscriber.subscribe(
          `delivery:${msg.deliveryId}:location`,
          `delivery:${msg.deliveryId}:eta`,
          `delivery:${msg.deliveryId}:status`
        );
      }
      deliverySubscriptions.get(msg.deliveryId).add(ws);
    }
  });

  ws.on('close', () => {
    if (ws.deliveryId) {
      const subs = deliverySubscriptions.get(ws.deliveryId);
      if (subs) { subs.delete(ws); if (subs.size === 0) deliverySubscriptions.delete(ws.deliveryId); }
    }
  });
});

subscriber.on('message', (channel, message) => {
  const [, deliveryId, eventType] = channel.split(':');
  const subs = deliverySubscriptions.get(deliveryId);
  if (!subs) return;
  const payload = JSON.stringify({ type: `delivery_${eventType}`, data: JSON.parse(message) });
  for (const ws of subs) { if (ws.readyState === WebSocket.OPEN) ws.send(payload); }
});

The server was deliberately stateless — it bridged Redis pub/sub to WebSocket connections with no business logic, making it horizontally scalable.

4. Vue.js Real-Time Tracking Interface

The customer-facing tracking page used Vue.js Composition API. The core was a WebSocket composable with exponential backoff reconnection:

export function useDeliveryWebSocket(deliveryId) {
  const driverLocation = ref(null);
  const eta = ref(null);
  const deliveryStatus = ref('confirmed');
  let ws = null, reconnectAttempts = 0;

  function connect() {
    ws = new WebSocket(process.env.VUE_APP_WS_URL);
    ws.onopen = () => {
      reconnectAttempts = 0;
      ws.send(JSON.stringify({ type: 'subscribe_delivery', deliveryId }));
    };
    ws.onmessage = ({ data }) => {
      const msg = JSON.parse(data);
      if (msg.type === 'delivery_location') driverLocation.value = msg.data.driver;
      else if (msg.type === 'delivery_eta') eta.value = msg.data.eta_minutes;
      else if (msg.type === 'delivery_status') deliveryStatus.value = msg.data.status;
    };
    ws.onclose = () => {
      if (reconnectAttempts < 10)
        setTimeout(connect, Math.min(1000 * 2 ** reconnectAttempts++, 30000));
    };
  }
  return { driverLocation, eta, deliveryStatus, connect, disconnect: () => ws?.close() };
}

The map component initialised Google Maps, rendered destination and driver markers, and animated the driver marker between positions using cubic easing. Without animation, the marker would "teleport" every 10 seconds — with smooth interpolation, users perceived continuous movement, dramatically improving perceived quality.

5. Delivery Status Lifecycle and Performance

Every status transition published to Redis for real-time updates, sent push notifications, and recorded metrics:

class Delivery extends Model
{
    public function markPickedUp(): void
    {
        $this->update(['status' => 'picked_up', 'picked_up_at' => now()]);
        Redis::publish("delivery:{$this->id}:status", json_encode([
            'delivery_id' => $this->id, 'status' => 'picked_up',
            'driver' => ['name' => $this->driver->name, 'phone' => $this->driver->phone],
        ]));
        $this->order->customer->notify(new DeliveryPickedUpNotification($this));
    }

    public function markDelivered(): void
    {
        $this->update(['status' => 'delivered', 'delivered_at' => now()]);
        Redis::publish("delivery:{$this->id}:status", json_encode([
            'delivery_id' => $this->id, 'status' => 'delivered',
        ]));
        $this->order->customer->notify(new DeliveryCompletedNotification($this));
        DeliveryMetric::create([
            'delivery_id' => $this->id, 'driver_id' => $this->driver_id,
            'estimated_minutes' => $this->estimated_arrival_minutes,
            'actual_minutes' => $this->picked_up_at?->diffInMinutes($this->delivered_at),
        ]);
    }
}

The DeliveryMetric model enabled comparing estimated vs. actual delivery times, identifying underperforming drivers and slow routes.

For performance at scale, I buffered location writes in Redis and flushed to PostgreSQL in batches every 30 seconds, reducing database write load by 90% while maintaining real-time delivery through the pub/sub layer. Vendor discovery pages used PostGIS spatial queries (ST_DWithin, ST_Distance) with aggressive caching, returning results in under 50ms across thousands of records.

The Results: Measurable Business Impact

The system launched after eight weeks of development. Results over a 60-day observation period:

Metric	Before	After	Change
Checkout conversion rate	68%	88%	+20 percentage points
Post-order cancellation rate	15%	4%	73% reduction
"Where is my order?" support calls	200+/day	~35/day	82% reduction
Customer satisfaction score	3.4/5	4.3/5	26% improvement
Active platform users	~3,200	5,000+	56% growth
Tracking page views per order	0	4.2	New engagement channel

The support call reduction alone freed 11 person-hours daily — redirected to proactive customer outreach and vendor management.

Technical Lessons

Decouple ingestion from delivery. Location collection (Laravel) and broadcasting (Node.js WebSocket) had different performance profiles. Separate services connected by Redis allowed independent scaling.
Animate transitions, don't teleport. Users perceive smooth movement as real-time tracking and position jumps as a broken system.
Throttle expensive API calls intelligently. The Haversine check before ETA recalculation saved thousands of unnecessary Google Maps calls daily.
Buffer writes, stream reads. Batch-writing to the database while streaming through Redis provided both persistence and live performance.
Measure business metrics. WebSocket latency matters, but checkout conversion rate proved the system's value.

Migrating a Legacy Codebase Across 15+ Modules Without Downtime: An Engineering Transformation Story

Olamilekan Lamidi — Wed, 08 Apr 2026 13:25:28 +0000

Legacy code is not a dirty word. Legacy code is code that makes money. It runs in production, serves real users, and generates the revenue that pays engineering salaries. The problem is not that legacy code exists — it is that legacy code eventually accumulates enough technical debt to slow the entire business down.

I know this because I have spent the past three years living it. Since September 2022, I have been working as a Full Stack Engineer at VacancySoft, a data intelligence company whose platform processes and serves vacancy data to clients across the recruitment and professional services industries. When I joined, the platform was built on a legacy monolithic architecture that had served the company well for years — but was buckling under the weight of 50,000+ daily API requests, growing data volumes, and a feature velocity that the architecture could no longer support.

Over the following months, I led the architectural redesign and incremental migration of 15+ modules from the legacy codebase to a modern Node.js/TypeScript architecture — without a single minute of planned downtime. This article is the story of how we did it: the strategy, the technical patterns, the code, and the measurable results.

The Problem: A Monolith Under Pressure

The VacancySoft platform had the hallmarks of a system that grew organically over many years. It was not badly built — it was built for a different era. But by the time I joined, several compounding issues were creating real business impact:

Technical Debt Symptoms

Inconsistent module architecture. Each of the 15+ modules had been built by different engineers at different times, with different conventions. Some modules used callback-style asynchronous patterns, others used early Promise implementations, and a few had partial async/await adoption. There was no unified error handling, no consistent data validation layer, and no shared utility patterns.

Tightly coupled database queries. Business logic was interleaved with raw SQL queries scattered throughout controllers and route handlers. Changing a database schema required tracing query strings across dozens of files, with no ORM or query builder providing a single source of truth for data access patterns.

No TypeScript, no type safety. The entire codebase was plain JavaScript. This meant that refactoring any function required manually tracing every call site to understand the expected input and output shapes. Function signatures were often documented only in the minds of the original authors — several of whom had left the company.

Test coverage below 15%. Critical business logic — data transformation, API response formatting, access control — had no automated tests. Every deployment was a manual QA exercise, and regressions were discovered by clients in production.

Business Impact

These technical issues translated directly into business problems:

Feature delivery had slowed by an estimated 40%. Engineers spent more time understanding and working around existing code than writing new functionality.
Query execution times had degraded. Several API endpoints used by key clients were responding in 2+ seconds, triggering SLA concerns.
Deployment confidence was low. Without adequate test coverage, the team deployed cautiously, batching changes into infrequent releases rather than shipping continuously.
Onboarding new engineers took weeks. The lack of type definitions, documentation, and consistent patterns meant new team members needed extensive hand-holding to become productive.

The codebase needed a fundamental transformation. But the platform was in active use by paying clients. A "big bang" rewrite — stopping feature work to rebuild everything from scratch — was commercially unacceptable. We needed a strategy that allowed us to modernise incrementally while keeping the platform fully operational.

My Role: Architect and Migration Lead

Olamilekan Lamidi drove the migration strategy from conception to execution. I designed the architectural approach, selected the migration pattern, defined TypeScript adoption standards, established the testing strategy, and personally led the refactoring of the most complex modules. I also mentored other engineers on the team through the transition, running code review sessions to ensure consistency across migrated modules.

This was not a committee decision. I proposed the strangler fig approach to engineering leadership, built the proof of concept on the first module, and established every pattern that the rest of the migration followed.

The Strategy: Strangler Fig Pattern

I chose the strangler fig pattern — named after the tropical vine that gradually envelops and replaces a host tree — as the migration strategy. Rather than rewriting the entire codebase at once, we would build new implementations alongside the legacy code, gradually routing traffic to the new modules, and eventually removing the old code once each module was fully replaced.

This approach offered three critical advantages:

Zero downtime. At no point would the platform be partially functional. Both old and new implementations ran simultaneously.
Incremental risk. Each module migration was an isolated, reversible change. If a new module had issues, we could route traffic back to the legacy implementation within seconds.
Continuous feature delivery. The team could ship new features on modules that had not yet been migrated, while migration work progressed in parallel on other modules.

The Migration Architecture

I designed a proxy layer that sat in front of both the legacy and new module implementations:

// migration-router.ts — Proxy layer for gradual migration
import { Request, Response, NextFunction } from 'express';
import { FeatureFlagService } from './services/feature-flags';

interface MigrationRoute {
  legacyHandler: (req: Request, res: Response, next: NextFunction) => void;
  modernHandler: (req: Request, res: Response, next: NextFunction) => void;
  moduleName: string;
}

export function createMigrationRouter(route: MigrationRoute) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const useModern = await FeatureFlagService.isEnabled(
      `migration:${route.moduleName}`,
      { userId: req.user?.id }
    );

    if (useModern) {
      return route.modernHandler(req, res, next);
    }

    return route.legacyHandler(req, res, next);
  };
}

Feature flags controlled which implementation served each request. I could migrate a module for internal users first, then gradually roll out to 10%, 25%, 50%, and finally 100% of client traffic — monitoring error rates and response times at every stage.

Technical Deep Dive: The Migration Process

Phase 1: TypeScript Foundation and Shared Infrastructure

Before migrating any module, I established the foundational infrastructure that every new module would depend on.

TypeScript Configuration

I introduced TypeScript with a strict configuration that enforced type safety from the start:

{
  "compilerOptions": {
    "target": "ES2020",
    "module": "commonjs",
    "lib": ["ES2020"],
    "strict": true,
    "noImplicitAny": true,
    "strictNullChecks": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "declaration": true,
    "declarationMap": true,
    "sourceMap": true,
    "outDir": "./dist",
    "rootDir": "./src",
    "baseUrl": ".",
    "paths": {
      "@modules/*": ["src/modules/*"],
      "@shared/*": ["src/shared/*"],
      "@config/*": ["src/config/*"]
    }
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "dist", "legacy/**/*"]
}

The strict: true flag was non-negotiable. I had seen too many TypeScript migrations where teams set strict: false to avoid friction, only to end up with TypeScript that offered no more safety than the JavaScript it replaced.

Shared Data Access Layer

I built a unified data access layer that replaced the scattered raw SQL queries with typed repository patterns:

// shared/repositories/base.repository.ts
import { Pool, QueryResult } from 'pg';
import { DatabasePool } from '../database/pool';

export abstract class BaseRepository<T> {
  protected pool: Pool;

  constructor() {
    this.pool = DatabasePool.getInstance();
  }

  protected async query<R = T>(
    sql: string,
    params: unknown[] = []
  ): Promise<R[]> {
    const start = Date.now();
    const result: QueryResult = await this.pool.query(sql, params);
    const duration = Date.now() - start;

    if (duration > 500) {
      logger.warn('Slow query detected', {
        sql: sql.substring(0, 200),
        duration,
        rowCount: result.rowCount,
      });
    }

    return result.rows as R[];
  }

  protected async queryOne<R = T>(
    sql: string,
    params: unknown[] = []
  ): Promise<R | null> {
    const rows = await this.query<R>(sql, params);
    return rows[0] || null;
  }

  protected async execute(
    sql: string,
    params: unknown[] = []
  ): Promise<number> {
    const result = await this.pool.query(sql, params);
    return result.rowCount ?? 0;
  }
}

Every migrated module used this base repository, which gave us automatic slow query logging, consistent error handling, and typed return values. The legacy code had no such instrumentation — slow queries went undetected until clients complained.

Standardised Error Handling

I introduced a typed error hierarchy that replaced the inconsistent error handling across the legacy codebase:

// shared/errors/application-errors.ts
export class ApplicationError extends Error {
  constructor(
    message: string,
    public readonly statusCode: number,
    public readonly code: string,
    public readonly isOperational: boolean = true
  ) {
    super(message);
    this.name = this.constructor.name;
    Error.captureStackTrace(this, this.constructor);
  }
}

export class NotFoundError extends ApplicationError {
  constructor(resource: string, identifier: string | number) {
    super(
      `${resource} with identifier '${identifier}' not found`,
      404,
      'RESOURCE_NOT_FOUND'
    );
  }
}

export class ValidationError extends ApplicationError {
  constructor(
    public readonly errors: Array<{ field: string; message: string }>
  ) {
    super('Validation failed', 400, 'VALIDATION_ERROR');
  }
}

export class ConflictError extends ApplicationError {
  constructor(message: string) {
    super(message, 409, 'CONFLICT');
  }
}

Phase 2: Module-by-Module Migration

With the foundation in place, I began migrating modules from lowest-risk to highest-risk. The order was deliberate: we started with modules that had lower traffic and simpler logic, allowing the team to build confidence with the migration pattern before tackling the complex, high-traffic modules.

Before (Legacy JavaScript):

// legacy/controllers/vacancyController.js
const db = require('../db');

exports.getVacancies = function(req, res) {
  var page = req.query.page || 1;
  var limit = req.query.limit || 50;
  var offset = (page - 1) * limit;
  var sector = req.query.sector;

  var sql = 'SELECT * FROM vacancies';
  var params = [];

  if (sector) {
    sql += ' WHERE sector = $1';
    params.push(sector);
  }

  sql += ' ORDER BY created_at DESC LIMIT $' + (params.length + 1) +
         ' OFFSET $' + (params.length + 2);
  params.push(limit, offset);

  db.query(sql, params, function(err, result) {
    if (err) {
      console.log('Error fetching vacancies:', err);
      return res.status(500).json({ error: 'Internal server error' });
    }

    db.query('SELECT COUNT(*) FROM vacancies' +
      (sector ? ' WHERE sector = $1' : ''), sector ? [sector] : [],
      function(err2, countResult) {
        if (err2) {
          return res.status(500).json({ error: 'Internal server error' });
        }
        res.json({
          data: result.rows,
          total: parseInt(countResult.rows[0].count),
          page: parseInt(page),
          limit: parseInt(limit)
        });
      }
    );
  });
};

After (Modern TypeScript):

// modules/vacancies/vacancy.controller.ts
import { Request, Response } from 'express';
import { VacancyService } from './vacancy.service';
import { GetVacanciesSchema } from './vacancy.validation';
import { asyncHandler } from '@shared/middleware/async-handler';

export class VacancyController {
  constructor(private readonly vacancyService: VacancyService) {}

  getVacancies = asyncHandler(async (req: Request, res: Response) => {
    const query = GetVacanciesSchema.parse(req.query);

    const result = await this.vacancyService.getVacancies({
      page: query.page,
      limit: query.limit,
      sector: query.sector,
      sortBy: query.sortBy,
      sortOrder: query.sortOrder,
    });

    res.json({
      data: result.items,
      meta: {
        total: result.total,
        page: result.page,
        limit: result.limit,
        totalPages: Math.ceil(result.total / result.limit),
      },
    });
  });
}

// modules/vacancies/vacancy.service.ts
import { VacancyRepository } from './vacancy.repository';
import { CacheService } from '@shared/services/cache.service';
import { PaginatedResult, VacancyListItem } from './vacancy.types';

export class VacancyService {
  constructor(
    private readonly repository: VacancyRepository,
    private readonly cache: CacheService
  ) {}

  async getVacancies(params: GetVacanciesParams): Promise<PaginatedResult<VacancyListItem>> {
    const cacheKey = `vacancies:${JSON.stringify(params)}`;

    const cached = await this.cache.get<PaginatedResult<VacancyListItem>>(cacheKey);
    if (cached) return cached;

    const [items, total] = await Promise.all([
      this.repository.findPaginated(params),
      this.repository.countFiltered(params),
    ]);

    const result: PaginatedResult<VacancyListItem> = {
      items,
      total,
      page: params.page,
      limit: params.limit,
    };

    await this.cache.set(cacheKey, result, 300);
    return result;
  }
}

The transformation is stark. The legacy version had no input validation, no caching, nested callbacks, string-concatenated SQL, and console.log as the only error reporting. The migrated version has schema validation with Zod, a service layer with caching, typed repository methods, and structured error handling — all running through the same API routes, invisible to clients.

Phase 3: Query Optimisation During Migration

Migration was not just a code restructuring exercise. I used each module migration as an opportunity to optimise the underlying queries. I audited every SQL query in each module before writing the replacement, using EXPLAIN ANALYZE to identify inefficiencies.

// modules/vacancies/vacancy.repository.ts
import { BaseRepository } from '@shared/repositories/base.repository';
import { VacancyListItem, GetVacanciesParams } from './vacancy.types';

export class VacancyRepository extends BaseRepository<VacancyListItem> {
  async findPaginated(params: GetVacanciesParams): Promise<VacancyListItem[]> {
    const conditions: string[] = [];
    const values: unknown[] = [];
    let paramIndex = 1;

    if (params.sector) {
      conditions.push(`v.sector = $${paramIndex++}`);
      values.push(params.sector);
    }

    if (params.dateFrom) {
      conditions.push(`v.published_at >= $${paramIndex++}`);
      values.push(params.dateFrom);
    }

    if (params.dateTo) {
      conditions.push(`v.published_at <= $${paramIndex++}`);
      values.push(params.dateTo);
    }

    const whereClause = conditions.length
      ? `WHERE ${conditions.join(' AND ')}`
      : '';

    const sortColumn = this.sanitiseSortColumn(params.sortBy);
    const sortOrder = params.sortOrder === 'asc' ? 'ASC' : 'DESC';

    values.push(params.limit, (params.page - 1) * params.limit);

    return this.query<VacancyListItem>(`
      SELECT
        v.id,
        v.title,
        v.sector,
        v.location,
        v.published_at,
        c.name AS company_name
      FROM vacancies v
      INNER JOIN companies c ON c.id = v.company_id
      ${whereClause}
      ORDER BY v.${sortColumn} ${sortOrder}
      LIMIT $${paramIndex++} OFFSET $${paramIndex}
    `, values);
  }

  private sanitiseSortColumn(column: string = 'published_at'): string {
    const allowed = ['published_at', 'title', 'sector', 'location'];
    return allowed.includes(column) ? column : 'published_at';
  }
}

The legacy code had been selecting SELECT * from tables with 30+ columns when the API response only needed 6 fields. Across high-traffic endpoints, this wasteful data transfer was a significant contributor to slow response times. The migrated queries selected only the columns needed, used proper indexing, and parallelised count queries with data queries using Promise.all.

Phase 4: Testing Strategy

Every migrated module was delivered with comprehensive test coverage. I established a three-layer testing strategy:

// modules/vacancies/__tests__/vacancy.service.test.ts
import { VacancyService } from '../vacancy.service';
import { VacancyRepository } from '../vacancy.repository';
import { CacheService } from '@shared/services/cache.service';

describe('VacancyService', () => {
  let service: VacancyService;
  let mockRepository: jest.Mocked<VacancyRepository>;
  let mockCache: jest.Mocked<CacheService>;

  beforeEach(() => {
    mockRepository = {
      findPaginated: jest.fn(),
      countFiltered: jest.fn(),
    } as any;
    mockCache = {
      get: jest.fn().mockResolvedValue(null),
      set: jest.fn().mockResolvedValue(undefined),
    } as any;
    service = new VacancyService(mockRepository, mockCache);
  });

  describe('getVacancies', () => {
    it('should return cached result when available', async () => {
      const cachedResult = {
        items: [{ id: 1, title: "'Engineer' }],"
        total: 1,
        page: 1,
        limit: 50,
      };
      mockCache.get.mockResolvedValue(cachedResult);

      const result = await service.getVacancies({ page: 1, limit: 50 });

      expect(result).toEqual(cachedResult);
      expect(mockRepository.findPaginated).not.toHaveBeenCalled();
    });

    it('should query repository and cache when no cache hit', async () => {
      const items = [{ id: 1, title: "'Engineer' }];"
      mockRepository.findPaginated.mockResolvedValue(items);
      mockRepository.countFiltered.mockResolvedValue(1);

      const result = await service.getVacancies({ page: 1, limit: 50 });

      expect(result.items).toEqual(items);
      expect(result.total).toBe(1);
      expect(mockCache.set).toHaveBeenCalled();
    });

    it('should pass filter parameters to repository', async () => {
      mockRepository.findPaginated.mockResolvedValue([]);
      mockRepository.countFiltered.mockResolvedValue(0);

      await service.getVacancies({
        page: 2,
        limit: 25,
        sector: 'Technology',
      });

      expect(mockRepository.findPaginated).toHaveBeenCalledWith(
        expect.objectContaining({ sector: 'Technology', page: 2, limit: 25 })
      );
    });
  });
});

Test coverage across migrated modules went from under 15% to above 85%. This was not just a quality improvement — it fundamentally changed deployment confidence. The team began shipping smaller, more frequent releases because they trusted the test suite to catch regressions.

The Results: Measured in Production

The migration was executed over a continuous period, with each module migrated, validated, and fully deployed before moving to the next. Here are the production metrics that Olamilekan Lamidi and the engineering team measured:

Metric	Before Migration	After Migration	Change
Average API response time	1.8s	320ms	82% reduction
Average query execution time	1.2s	480ms	60% reduction
Overall system performance	Baseline	+40% throughput	40% improvement
Daily API request capacity	~50,000 (strained)	50,000+ (comfortable headroom)	Stable at scale
Test coverage	<15%	>85%	5.7x increase
Deployment frequency	Bi-weekly	Multiple per week	3-4x increase
Production incidents (monthly)	6-8	1-2	75% reduction
New engineer onboarding time	~4 weeks	~1.5 weeks	63% reduction
Planned downtime during migration	—	Zero	No service interruption

The 40% system performance improvement and 60% query execution time reduction were measured across the full platform using application performance monitoring over a 30-day observation window after the final module was migrated.

Lessons for Engineers Facing Legacy Migrations

Having now completed this migration, here is what I would tell any engineer facing a similar challenge:

Never rewrite from scratch. The strangler fig pattern works. It is slower than a rewrite feels, but it is dramatically less risky. The business never stops.
Establish patterns before scaling. I spent the first two weeks building shared infrastructure and migrating one module as a reference implementation. Every subsequent module migration was faster because the patterns were clear and proven.
Use the migration as a performance audit. Every legacy module contains hidden performance debt. Migrating is the perfect opportunity to identify and resolve it, because you are already reading every line of code.
Type safety pays for itself immediately. The number of bugs caught by TypeScript during migration — bugs that had been latent in the JavaScript codebase for months — convinced even the most sceptical team members that strict typing was worth the effort.
Test coverage is the safety net that makes everything else possible. Without comprehensive tests, every migration step is a gamble. With them, it is engineering.
Measure before, during, and after. If you cannot show the performance numbers, the migration is a story. If you can, it is evidence.

Final Thoughts

Migrating a legacy codebase is not glamorous engineering work. There are no greenfield architectures to design, no new frameworks to evaluate. It is the painstaking work of understanding what exists, designing a better version, and replacing components one by one without breaking anything.

But it is some of the most impactful work an engineer can do. The VacancySoft platform now processes 50,000+ daily requests on an architecture that is maintainable, testable, performant, and ready for the next phase of growth. The engineering team ships faster, with more confidence, and new engineers become productive in days rather than weeks.

Olamilekan Lamidi has led this transformation from start to finish, and the lessons from this experience have shaped how I approach every codebase I work on — legacy or otherwise.

How I Architected a Real-Time Gaming Backend Supporting 3,000+ Concurrent Users with WebSockets

Olamilekan Lamidi — Fri, 03 Apr 2026 12:58:27 +0000

Real-time systems are unforgiving. In a standard web application, a 500-millisecond delay is an inconvenience. In a real-time multiplayer gaming platform, 500 milliseconds is the difference between a fair outcome and a furious user demanding a refund.

Between October 2019 and December 2021, I served as Senior Software Engineer at Lordwin Group, where I led a team of five backend developers building dice.ng — a real-time gaming platform where thousands of users placed wagers, watched outcomes resolve live, and expected the entire experience to feel instantaneous. The platform also encompassed an investment management system and hotel booking service, but the gaming backend was the most technically demanding piece of infrastructure I have ever designed.

This article is a technical deep dive into how I architected that real-time gaming backend from the ground up — the WebSocket infrastructure, the event-driven architecture, the scaling strategy, and the hard lessons learned from operating a system where latency directly impacts revenue.

The Problem: Sub-100ms Latency for Thousands of Simultaneous Players

The requirements were aggressive from day one. Lordwin Group needed a gaming platform that could:

Support 3,000+ concurrent WebSocket connections during peak hours
Deliver game events (dice rolls, bet confirmations, payout calculations) with sub-100ms latency
Maintain absolute consistency in game state — in a wagering platform, a race condition is not a bug, it is a financial liability
Handle burst traffic patterns — user activity would spike dramatically around evening hours and weekends, sometimes tripling within minutes
Achieve 99.9% uptime — every minute of downtime was measurable lost revenue

The initial prototype used HTTP polling. Clients hit an API endpoint every two seconds to check for game state updates. At 500 concurrent users, this generated over 15,000 HTTP requests per minute, the database was drowning, and the "real-time" experience felt sluggish and broken. It was immediately clear that polling could not scale. We needed a fundamentally different architecture.

My Role: Technical Lead and System Architect

As the senior engineer leading a team of five, my responsibilities went beyond writing code. I made the core architecture decisions, designed the WebSocket infrastructure, established the event-driven messaging patterns, defined the horizontal scaling strategy, and mentored junior engineers on real-time system design.

Olamilekan Lamidi was not just a contributor on this project — I owned the technical direction. Every critical design decision described in this article was one I either made directly or guided the team toward after evaluating alternatives.

Technical Deep Dive: Building the Real-Time Infrastructure

1. WebSocket Server Architecture with Node.js

I chose Node.js as the runtime for the WebSocket server. Its event-driven, non-blocking I/O model is purpose-built for maintaining thousands of persistent connections with minimal resource overhead. PHP (our backend for the investment and hotel systems) was poorly suited for long-lived connections, so I designed the gaming layer as a separate Node.js service communicating with the main platform through Redis message channels.

The core WebSocket server was built on the ws library rather than Socket.IO. While Socket.IO offers convenience features like automatic reconnection and room management, it adds protocol overhead and abstractions that reduce control. For a latency-sensitive gaming platform, I needed raw WebSocket performance with custom protocol handling.

const WebSocket = require('ws');
const Redis = require('ioredis');
const { v4: uuidv4 } = require('uuid');

const wss = new WebSocket.Server({
  port: 8080,
  maxPayload: 1024 * 16,
  perMessageDeflate: false,
  clientTracking: true,
});

const connections = new Map();

wss.on('connection', (ws, req) => {
  const connectionId = uuidv4();
  const clientIp = req.headers['x-forwarded-for'] || req.socket.remoteAddress;

  connections.set(connectionId, {
    ws,
    userId: null,
    rooms: new Set(),
    connectedAt: Date.now(),
    lastHeartbeat: Date.now(),
  });

  ws.on('message', (raw) => {
    try {
      const message = JSON.parse(raw);
      routeMessage(connectionId, message);
    } catch (err) {
      ws.send(JSON.stringify({ type: 'error', code: 'INVALID_PAYLOAD' }));
    }
  });

  ws.on('close', () => {
    cleanupConnection(connectionId);
  });

  ws.on('pong', () => {
    const conn = connections.get(connectionId);
    if (conn) conn.lastHeartbeat = Date.now();
  });

  ws.send(JSON.stringify({
    type: 'connected',
    connectionId,
    serverTime: Date.now(),
  }));
});

I disabled perMessageDeflate deliberately. Compression adds CPU overhead per message, and since our payloads were small (typically under 500 bytes for game events), the bandwidth savings were negligible compared to the latency cost. This single configuration change reduced median message delivery time by 8ms across our benchmark tests.

2. Connection Management and Authentication

In a gaming platform handling real money, every WebSocket connection must be authenticated. I implemented a token-based authentication flow where clients first obtain a short-lived JWT via the REST API, then present it during WebSocket handshake.

const jwt = require('jsonwebtoken');

function routeMessage(connectionId, message) {
  const conn = connections.get(connectionId);
  if (!conn) return;

  switch (message.type) {
    case 'authenticate':
      handleAuthentication(connectionId, message.token);
      break;
    case 'join_game':
      if (!conn.userId) return sendError(conn.ws, 'NOT_AUTHENTICATED');
      handleJoinGame(connectionId, message.gameId);
      break;
    case 'place_bet':
      if (!conn.userId) return sendError(conn.ws, 'NOT_AUTHENTICATED');
      handlePlaceBet(connectionId, message);
      break;
    case 'ping':
      conn.ws.send(JSON.stringify({ type: 'pong', serverTime: Date.now() }));
      break;
    default:
      sendError(conn.ws, 'UNKNOWN_MESSAGE_TYPE');
  }
}

function handleAuthentication(connectionId, token) {
  const conn = connections.get(connectionId);
  try {
    const payload = jwt.verify(token, process.env.JWT_SECRET, {
      algorithms: ['HS256'],
      maxAge: '5m',
    });
    conn.userId = payload.userId;
    conn.ws.send(JSON.stringify({ type: 'authenticated', userId: payload.userId }));
    userConnectionIndex.set(payload.userId, connectionId);
  } catch (err) {
    conn.ws.send(JSON.stringify({ type: 'auth_failed', reason: 'INVALID_TOKEN' }));
    conn.ws.close(4001, 'Authentication failed');
  }
}

The JWT had a deliberately short expiry of five minutes. Since it was only used for the WebSocket handshake, there was no reason for it to live longer. Once authenticated, the connection was maintained through heartbeat pings rather than token refresh cycles.

3. Room-Based Game State Management

Each active game session functioned as a "room" — a logical grouping of WebSocket connections that should receive the same events. I implemented a lightweight room system without relying on external libraries:

const rooms = new Map();

function handleJoinGame(connectionId, gameId) {
  const conn = connections.get(connectionId);
  if (!conn) return;

  if (!rooms.has(gameId)) {
    rooms.set(gameId, new Set());
  }

  rooms.get(gameId).add(connectionId);
  conn.rooms.add(gameId);

  conn.ws.send(JSON.stringify({
    type: 'game_joined',
    gameId,
    players: rooms.get(gameId).size,
  }));

  broadcastToRoom(gameId, {
    type: 'player_joined',
    userId: conn.userId,
    playerCount: rooms.get(gameId).size,
  }, connectionId);
}

function broadcastToRoom(gameId, payload, excludeConnectionId = null) {
  const room = rooms.get(gameId);
  if (!room) return;

  const message = JSON.stringify(payload);
  for (const connId of room) {
    if (connId === excludeConnectionId) continue;
    const conn = connections.get(connId);
    if (conn && conn.ws.readyState === WebSocket.OPEN) {
      conn.ws.send(message);
    }
  }
}

One critical optimisation: I serialised the message payload once with JSON.stringify outside the loop, then sent the same string buffer to every client. For a room of 500 players, this avoided 499 redundant serialisation operations per broadcast — a saving that compounds rapidly under load.

4. Event-Driven Game Engine with Redis Pub/Sub

The game logic itself — dice rolling, bet resolution, payout calculation — ran in a separate process from the WebSocket server. This was a deliberate architectural decision. The WebSocket server's only responsibility was managing connections and delivering messages. Game logic, financial calculations, and database writes happened in dedicated worker processes.

Redis Pub/Sub was the communication backbone:

const publisher = new Redis(process.env.REDIS_URL);
const subscriber = new Redis(process.env.REDIS_URL);

subscriber.subscribe('game:events', 'game:results', 'system:announcements');

subscriber.on('message', (channel, message) => {
  const event = JSON.parse(message);

  switch (channel) {
    case 'game:events':
      broadcastToRoom(event.gameId, {
        type: 'game_event',
        event: event.eventType,
        data: event.payload,
        timestamp: event.timestamp,
      });
      break;

    case 'game:results':
      handleGameResult(event);
      break;

    case 'system:announcements':
      broadcastToAll({
        type: 'system_announcement',
        message: event.message,
      });
      break;
  }
});

async function handlePlaceBet(connectionId, message) {
  const conn = connections.get(connectionId);

  const betEvent = {
    gameId: message.gameId,
    userId: conn.userId,
    amount: message.amount,
    selection: message.selection,
    timestamp: Date.now(),
    idempotencyKey: message.idempotencyKey,
  };

  await publisher.publish('bets:incoming', JSON.stringify(betEvent));

  conn.ws.send(JSON.stringify({
    type: 'bet_acknowledged',
    idempotencyKey: message.idempotencyKey,
    status: 'processing',
  }));
}

This decoupling had three critical benefits:

Fault isolation: If the game engine crashed, WebSocket connections remained alive. Users saw a brief pause rather than a disconnection.
Independent scaling: I could run multiple game engine workers to process bets in parallel without changing the WebSocket layer.
Auditability: Every game event flowed through Redis, creating a natural event stream that I logged for regulatory compliance and dispute resolution.

5. Horizontal Scaling Across Multiple WebSocket Servers

A single Node.js process can handle approximately 10,000 concurrent WebSocket connections before memory and CPU become constraints. But scaling WebSockets horizontally introduces a problem that REST APIs do not have: connection affinity. If Player A is connected to Server 1 and Player B to Server 2, a room broadcast must reach both servers.

I solved this with Redis as the cross-server message bus:

const serverId = process.env.SERVER_ID || uuidv4();

subscriber.subscribe('ws:broadcast');

subscriber.on('message', (channel, raw) => {
  if (channel !== 'ws:broadcast') return;
  const message = JSON.parse(raw);

  if (message.originServer === serverId) return;

  if (message.targetRoom) {
    broadcastToRoom(message.targetRoom, message.payload);
  } else if (message.targetUser) {
    sendToUser(message.targetUser, message.payload);
  }
});

function clusterBroadcastToRoom(gameId, payload) {
  broadcastToRoom(gameId, payload);

  publisher.publish('ws:broadcast', JSON.stringify({
    originServer: serverId,
    targetRoom: gameId,
    payload,
  }));
}

Each WebSocket server instance subscribed to a shared Redis channel. When a game event needed to reach all players in a room, the originating server broadcast to its local connections and simultaneously published to Redis, where other servers picked up the message and relayed it to their local connections.

The originServer check prevented message loops — without it, a server would re-broadcast messages it had already delivered locally.

6. Heartbeat Monitoring and Connection Cleanup

Stale connections are a silent performance killer in WebSocket systems. Users close browser tabs, lose internet connectivity, or let their phones go to sleep. Without proactive cleanup, the server accumulates zombie connections that consume memory and distort room player counts.

I implemented a heartbeat interval that pinged every client every 30 seconds:

const HEARTBEAT_INTERVAL = 30000;
const HEARTBEAT_TIMEOUT = 45000;

setInterval(() => {
  const now = Date.now();
  for (const [connectionId, conn] of connections) {
    if (now - conn.lastHeartbeat > HEARTBEAT_TIMEOUT) {
      conn.ws.terminate();
      cleanupConnection(connectionId);
      continue;
    }
    if (conn.ws.readyState === WebSocket.OPEN) {
      conn.ws.ping();
    }
  }
}, HEARTBEAT_INTERVAL);

function cleanupConnection(connectionId) {
  const conn = connections.get(connectionId);
  if (!conn) return;

  for (const gameId of conn.rooms) {
    const room = rooms.get(gameId);
    if (room) {
      room.delete(connectionId);
      if (room.size === 0) {
        rooms.delete(gameId);
      } else {
        broadcastToRoom(gameId, {
          type: 'player_left',
          userId: conn.userId,
          playerCount: room.size,
        });
      }
    }
  }

  if (conn.userId) {
    userConnectionIndex.delete(conn.userId);
  }

  connections.delete(connectionId);
}

This heartbeat mechanism maintained accurate player counts and ensured we never wasted resources on dead connections — critical when operating at 3,000+ concurrent users where even small inefficiencies compound.

Ensuring Game Integrity: The Financial Safety Layer

In a platform where real money is at stake, game outcome integrity is non-negotiable. I designed several safeguards:

Idempotent Bet Processing: Every bet carried a client-generated idempotency key. If a network hiccup caused a duplicate submission, the game engine would recognise the duplicate key and return the original response rather than processing the bet twice.

Atomic Balance Operations: All balance changes used Redis WATCH/MULTI/EXEC transactions for in-memory operations and database-level row locking for persistence, ensuring no user could ever bet more than their balance — even under concurrent requests.

async function processBalanceDeduction(userId, amount, idempotencyKey) {
  const lockKey = `lock:balance:${userId}`;
  const lock = await acquireLock(lockKey, 5000);

  try {
    const processed = await redis.get(`idempotency:${idempotencyKey}`);
    if (processed) return JSON.parse(processed);

    const balance = parseFloat(await redis.get(`balance:${userId}`));
    if (balance < amount) {
      return { success: false, reason: 'INSUFFICIENT_BALANCE' };
    }

    const newBalance = balance - amount;
    await redis.set(`balance:${userId}`, newBalance.toString());

    const result = { success: true, newBalance, deducted: amount };
    await redis.setex(`idempotency:${idempotencyKey}`, 3600, JSON.stringify(result));

    return result;
  } finally {
    await releaseLock(lock);
  }
}

Provably Fair Outcomes: Game results were generated using a combination of server seed and client seed, hashed together. Users could verify after each round that the outcome was not manipulated. This was both a regulatory requirement and a trust-building feature.

Monitoring and Observability in Production

Operating a real-time gaming system requires comprehensive observability. I built custom monitoring dashboards tracking:

Active connections per server instance — to trigger auto-scaling
Message throughput — messages sent per second across all rooms
Event latency — time from game engine event emission to client delivery
Redis pub/sub lag — early warning for message bus saturation
Game round completion times — anomaly detection for stuck game sessions

I instrumented the WebSocket server with Prometheus metrics:

const client = require('prom-client');

const activeConnections = new client.Gauge({
  name: 'ws_active_connections',
  help: 'Number of active WebSocket connections',
});

const messageLatency = new client.Histogram({
  name: 'ws_message_latency_ms',
  help: 'Message delivery latency in milliseconds',
  buckets: [5, 10, 25, 50, 100, 250, 500],
});

const messagesPerSecond = new client.Counter({
  name: 'ws_messages_total',
  help: 'Total WebSocket messages sent',
  labelNames: ['type'],
});

These metrics fed into Grafana dashboards that gave me real-time visibility into system health. On several occasions, a spike in message latency gave us a 10-minute early warning before a Redis memory issue would have caused visible user impact.

The Results: Production Metrics

After three months of iterative development and load testing, Olamilekan Lamidi and the engineering team shipped the gaming backend into production. The measurable outcomes exceeded our original targets:

Metric	Target	Achieved
Concurrent WebSocket connections	3,000	3,200+ sustained peak
Event delivery latency (p50)	<100ms	32ms
Event delivery latency (p99)	<200ms	87ms
System uptime (monthly)	99.5%	99.9%
Bet processing throughput	500/min	850+/min
Connection recovery after deploy	<5s	~2.5s average

The sub-50ms median latency was particularly significant. Users experienced game events as truly instantaneous — dice rolls resolved, payouts appeared, and leaderboards updated in what felt like real time. This directly correlated with user engagement metrics: average session duration increased by 35% compared to the earlier HTTP polling prototype.

Lessons Learned

Building this system taught me principles that I have carried into every project since:

Separate connection management from business logic. The WebSocket server should be a dumb pipe. All intelligence belongs in backend workers communicating through message queues.
Design for graceful degradation. When the game engine was temporarily slow, the WebSocket layer continued serving heartbeats and acknowledged messages. Users experienced a delay, not a crash.
Serialise once, send many. In broadcast-heavy systems, the cost of JSON serialisation dwarfs the cost of network transmission. Serialise the payload once and reuse the buffer.
Monitor latency percentiles, not averages. A 30ms average means nothing if 5% of your users experience 500ms delays. The p99 is what your angriest users feel.
Test with realistic load patterns. Our load tests simulated bursty evening traffic with sudden spikes, not steady-state throughput. Production traffic is never smooth.

Final Thoughts

Architecting a real-time gaming backend was the most technically challenging and rewarding project of my career up to that point. It demanded a deep understanding of network protocols, distributed systems, concurrent programming, and financial transaction safety — all operating under latency constraints that left no room for architectural shortcuts.

The patterns I developed at Lordwin Group — event-driven architecture, Redis-backed horizontal scaling, idempotent financial operations — became foundational to how I approach every real-time system I have built since. At VacancySoft, I applied similar event-driven patterns to handle 50,000+ daily API requests. At 2am Tech, the idempotent transaction processing pattern directly informed the financial workflows I built for the Addio platform.

Real-time systems are hard. But the engineering discipline they demand makes you a better systems engineer in every other context. If you are building similar infrastructure, I hope this deep dive gives you a head start on the decisions and trade-offs that matter most.

Olamilekan Lamidi is a Senior Full-Stack Engineer with 9+ years of experience building scalable, high-performance web applications. He specialises in designing robust APIs, optimising systems for performance at scale, and leading engineering teams to deliver reliable production systems.

Connect on LinkedIn | GitHub

Tags: #WebSockets #NodeJS #RealTime #GameDev #SystemDesign #BackendEngineering #Redis #JavaScript #WebDevelopment #SoftwareArchitecture

Scaling Laravel Backends for High-Volume Systems: Lessons from Serving 100,000+ Users in Production

Olamilekan Lamidi — Sun, 29 Mar 2026 22:38:45 +0000

How I reduced API latency by 88%, achieved 99.8% uptime, and scaled a platform from 2,000 to 15,000 concurrent users — without downtime.

There is a moment every backend engineer dreads. Traffic is climbing, the monitoring dashboard is turning red, and your database is buckling under the weight of queries it was never designed to handle at this scale. I have lived through that moment more than once.

Over the past nine years, I, Olamilekan Lamidi, have built and scaled backend systems across fintech, gaming, health-tech, and logistics — platforms that collectively serve tens of thousands of users daily. This article distils the hard-earned lessons from scaling Laravel backends to handle high-volume traffic, reduce API latency, and maintain near-perfect uptime under pressure.

If you are building a Laravel application that needs to graduate from "it works" to "it works at scale," this is for you.

The Problem: When Your Backend Becomes the Bottleneck

Most Laravel applications start life the same way: a clean MVC structure, Eloquent models that read like poetry, and Blade templates that ship fast. This works beautifully at 1,000 users. At 10,000, cracks appear. At 100,000+, the whole thing can collapse.

I encountered this problem in its most acute form while leading backend engineering at Lordwin Group, where I architected systems for an investment and hotel management platform serving over 10,000 registered users, alongside a real-time gaming backend supporting 3,000+ concurrent connections. The platform was haemorrhaging performance. API response times had crept above two seconds, database queries were timing out during peak hours, and the engineering team was firefighting daily.

The root causes were familiar to anyone who has scaled a Laravel application:

N+1 query explosions buried deep in nested Eloquent relationships
Unindexed database tables that grew silently from thousands to millions of rows
Monolithic request handling where every API call did too much synchronous work
No caching strategy — every request hit the database, every time
Session and queue bottlenecks running on the same infrastructure as the application

The question was not whether the system needed re-architecting. It was whether we could do it without taking the platform offline.

The Challenges: Scaling Without Breaking

Scaling a live production system is fundamentally different from building a new one. You cannot afford downtime. You cannot rewrite everything at once. And you have to keep shipping features while simultaneously rebuilding the engine underneath.

Here are the specific challenges I faced:

1. Database Performance Degradation

The MySQL database had grown to contain tables with 5+ million rows. Queries that once returned in milliseconds were now taking 3–8 seconds. Eloquent's lazy loading was making things worse — a single API endpoint for listing investment portfolios was firing 47 database queries per request.

2. Synchronous Processing Overload

Critical operations — sending transactional emails, generating PDF reports, processing payment webhooks — were all running synchronously within the HTTP request cycle. This meant users were waiting 4–6 seconds for operations that did not need to block the response.

3. Absence of a Caching Layer

There was no Redis or Memcached layer. Configuration data, user permissions, and even static reference data were fetched from the database on every single request. During peak traffic, the database connection pool was exhausted within minutes.

4. Infrastructure That Could Not Scale Horizontally

The application was deployed on a single server with no load balancing, no auto-scaling, and no separation between the application server, queue workers, and database. Everything competed for the same CPU and memory.

The Solution: A Systematic Approach to Laravel Performance at Scale

I approached the problem methodically, treating it as four distinct engineering initiatives that could be executed incrementally without disrupting the live platform.

Phase 1: Query Optimisation and Database Restructuring

The single highest-impact change was eliminating N+1 queries and restructuring how the application interacted with the database.

Eager Loading Enforcement

I audited every Eloquent query across 15+ modules using Laravel Debugbar and Clockwork, cataloguing every N+1 violation. I then introduced eager loading constraints:

// Before: 47 queries for a portfolio listing
$portfolios = Portfolio::all();
foreach ($portfolios as $portfolio) {
    echo $portfolio->user->name;
    echo $portfolio->transactions->count();
}

// After: 3 queries with eager loading and aggregation
$portfolios = Portfolio::with(['user:id,name', 'transactions'])
    ->withCount('transactions')
    ->paginate(50);

Strategic Indexing

I analysed slow query logs and added composite indexes on columns used in WHERE, JOIN, and ORDER BY clauses. For the investment transactions table alone, adding a composite index on (user_id, status, created_at) reduced a critical query from 6.2 seconds to 180 milliseconds.

CREATE INDEX idx_transactions_user_status_date
ON transactions (user_id, status, created_at);

Query Refactoring

For complex reporting queries, I replaced Eloquent with raw SQL and database views where the ORM was generating inefficient joins. I also introduced read replicas to offload reporting queries from the primary database.

Result: Average query execution time dropped by 60%. The portfolio listing endpoint went from 2.3 seconds to 320 milliseconds.

Phase 2: Asynchronous Processing with Queue Architecture

I decoupled all non-critical work from the HTTP request cycle using Laravel's queue system backed by Redis.

class ProcessPaymentWebhook implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public int $backoff = 60;

    public function handle(): void
    {
        $this->updateTransactionStatus();
        $this->notifyUser();
        $this->syncWithAccountingService();
        $this->generateReceipt();
    }

    public function failed(Throwable $exception): void
    {
        Log::critical('Payment webhook failed', [
            'transaction_id' => $this->transaction->id,
            'error' => $exception->getMessage(),
        ]);
        AlertService::notifyEngineering($exception);
    }
}

I configured dedicated queue workers for different job priorities:

High priority: Payment processing, authentication events
Medium priority: Email notifications, PDF generation
Low priority: Analytics aggregation, cache warming

Result: API response times for webhook endpoints dropped from 4.2 seconds to under 200 milliseconds. The user-facing experience became near-instantaneous.

Phase 3: Multi-Layer Caching Strategy

I implemented a three-tier caching architecture using Redis:

Application-Level Caching

class PermissionService
{
    public function getUserPermissions(int $userId): Collection
    {
        return Cache::tags(["user:{$userId}", 'permissions'])
            ->remember(
                "user_permissions:{$userId}",
                now()->addHours(6),
                fn () => $this->repository->getPermissionsForUser($userId)
            );
    }

    public function invalidateUserCache(int $userId): void
    {
        Cache::tags(["user:{$userId}"])->flush();
    }
}

Route-Level Caching

For endpoints serving semi-static data (e.g., hotel listings, investment product catalogues), I introduced HTTP response caching with cache invalidation tied to model events:

Route::middleware('cache.response:300')->group(function () {
    Route::get('/products', [ProductController::class, 'index']);
    Route::get('/listings', [ListingController::class, 'index']);
});

Database Query Caching

For expensive aggregation queries used in dashboards, I implemented scheduled cache warming through Laravel's task scheduler, ensuring cached data was always fresh without burdening the database during peak hours.

Result: Database load reduced by approximately 70% during peak traffic. Redis handled 85% of read requests, and the database connection pool was no longer a bottleneck.

Phase 4: Infrastructure and Deployment Architecture

I redesigned the deployment topology to support horizontal scaling:

Load Balancer (Nginx) — Distributed traffic across multiple application instances
Separated Queue Workers — Dedicated servers for background job processing, preventing queue backlogs from impacting API response times
Read Replicas — Reporting and analytics queries routed to MySQL read replicas
Docker Containerisation — Standardised environments across development, staging, and production, eliminating "works on my machine" deployment failures
CI/CD Pipeline — Automated testing and zero-downtime deployments using rolling updates

services:
  app:
    build: .
    deploy:
      replicas: 3
    depends_on:
      - mysql
      - redis

  queue-worker:
    build: .
    command: php artisan queue:work redis --queue=high,medium,low
    deploy:
      replicas: 2

  scheduler:
    build: .
    command: php artisan schedule:run

  mysql:
    image: mysql:8.0
    volumes:
      - db_data:/var/lib/mysql

  redis:
    image: redis:alpine

Result: The platform could now scale horizontally by adding application instances. Deployments went from 15-minute maintenance windows to zero-downtime rolling updates.

The Results: Measurable Impact

After executing these four phases over a 10-week period, the performance transformation was significant:

Average API response time
Before: 2.3s → After: 280ms → 88% reduction

Database query execution time
Before: 1.8s avg → After: 180ms avg → 60% reduction

Payment webhook processing
Before: 4.2s → After: 190ms → 95% reduction

Database connections at peak
Before: 95% utilised → After: 30% utilised → 70% reduction

System uptime (monthly)
Before: 96.2% → After: 99.8% → 3.6% improvement

Concurrent user capacity
Before: ~2,000 → After: ~15,000 → 7.5x increase

These were not theoretical benchmarks. They were measured in production using New Relic APM and custom Prometheus dashboards over a 30-day observation period after the final deployment.

Broader Impact: Patterns That Transfer Across Industries

The architectural patterns I applied are not unique to any single domain. I have since applied the same principles across:

Fintech — At 2am Tech, I contributed to Addio, a financial platform used by 500+ organisations, where queue-based architecture and database optimisation were essential for handling high-volume transactional workflows across factors, debtors, and vendors.

Logistics — At Viaduct, I engineered a high-performance on-demand delivery platform with Laravel and Vue.js serving 5,000+ active users, where real-time order tracking demanded low-latency APIs and efficient WebSocket communication.

Enterprise SaaS — At VacancySoft, I led the refactoring of a legacy codebase across 15+ modules, migrating services to a modern architecture — cutting query execution time by 60% and improving overall system performance by 40% while handling 50,000+ daily requests.

The fundamental lesson is this: scaling is not about throwing more hardware at the problem. It is about understanding where your system spends time and eliminating waste systematically. Database queries, synchronous processing, missing caches, and monolithic deployment topologies are the four horsemen of backend performance degradation. Address them methodically, measure relentlessly, and you can scale a Laravel application to handle traffic volumes that would surprise most engineers.

Key Takeaways for Engineers Building High-Volume Systems

Profile before you optimise. Use Laravel Debugbar, Clockwork, or APM tools to identify actual bottlenecks. Gut instinct is unreliable.
Eliminate N+1 queries aggressively. This is almost always the single highest-impact optimisation in any Eloquent-based application.
Move everything non-essential off the request cycle. If the user does not need to wait for it, it belongs in a queue.
Cache strategically, invalidate precisely. A poorly invalidated cache is worse than no cache. Use tagged caching and model event listeners to keep data fresh.
Design for horizontal scaling from day one. Stateless application servers, externalised sessions (Redis), and containerised deployments make scaling a configuration change rather than an architectural rewrite.
Measure everything. If you cannot quantify the improvement, you cannot prove it happened. Set up monitoring before you start optimising.

Final Thoughts

Backend performance at scale is a discipline, not a one-time fix. Every system I have worked on has taught me that the architecture decisions you make early compound over time — for better or worse. The patterns described in this article have been tested in production across financial platforms, real-time gaming systems, logistics networks, and enterprise SaaS products.

If you are an engineer facing similar scaling challenges, I hope this gives you a practical roadmap. And if you have solved these problems differently, I would genuinely like to hear about it. The best engineering solutions come from sharing what we have learned.