Forem: Robins163

I Built a RAG App That Chats With Any PDF — Here's How

Robins163 — Tue, 14 Apr 2026 07:20:36 +0000

TL;DR: I built DocMind — a multimodal RAG (Retrieval-Augmented Generation) app that lets you upload any PDF, image, or document and ask questions about it in plain English. No API keys needed — it runs entirely locally using Ollama for the LLM and Xenova Transformers for embeddings. This post walks through the full architecture, the chunking strategy, the vector similarity search, and the code.

The Problem

You have a 200-page PDF — a legal contract, a research paper, your company's internal docs. You need to find specific information buried somewhere in it. Ctrl+F only works if you know the exact words. What you really want is to ask the document questions in plain language:

"What's the termination clause in this contract?"
"Summarize the results from section 4"
"What technologies does this company use?"

This is what RAG does. And unlike fine-tuning an LLM (which costs hundreds of dollars and takes hours), RAG is cheap, fast, and works with any document you throw at it.

I built DocMind to solve this for myself — and to have a real AI project in my portfolio that actually does something useful.

What Is RAG? (The 30-Second Version)

RAG = Retrieval-Augmented Generation. Instead of asking an LLM to answer from memory (which it often gets wrong — hallucinations), you:

Retrieve the relevant parts of your document
Augment the LLM's prompt with those chunks
Generate an answer grounded in actual content

  RAG vs PURE LLM
  ════════════════

  Pure LLM (without RAG):
  ┌────────┐   "What's the          ┌─────────┐
  │  User  │──termination clause?──▶│   LLM   │──▶ "I think it might be..."
  └────────┘                        │(guessing │    (HALLUCINATION ⚠️)
                                    │ from     │
                                    │ training)│
                                    └─────────┘

  RAG (with document context):
  ┌────────┐   "What's the          ┌──────────┐   relevant    ┌─────────┐
  │  User  │──termination clause?──▶│  Vector  │──paragraphs──▶│   LLM   │
  └────────┘                        │  Search  │               │(answers │
                                    │          │               │ from    │
                                    │ searches │               │ actual  │
                                    │ your PDF │               │ content)│
                                    └──────────┘               └─────────┘
                                                                    │
                                                                    ▼
                                                          "Section 12.3 states
                                                           the termination clause
                                                           requires 30 days..."
                                                           (GROUNDED ✅)

The Full Architecture

Here's how DocMind works end to end:

flowchart TD
    A[Upload Document] --> B{File Type?}
    B -->|PDF| C[pdf-parse]
    B -->|DOCX| D[mammoth]
    B -->|Image| E[Tesseract OCR]
    B -->|CSV| F[csv-parse]
    B -->|TXT/MD| G[Read directly]
    C --> H[Raw Text]
    D --> H
    E --> H
    F --> H
    G --> H
    H --> I[Chunk Text<br/>500 chars, 50 overlap]
    I --> J[Generate Embeddings<br/>all-MiniLM-L6-v2]
    J --> K[Store in Vector Index]

    L[User Question] --> M[Embed Question]
    M --> N[Cosine Similarity Search]
    K --> N
    N --> O[Top 5 Chunks Retrieved]
    O --> P[Build Prompt with Context]
    P --> Q[Ollama LLM generates answer]
    Q --> R[Return Answer + Sources]

    style K fill:#3b82f6,color:#fff
    style Q fill:#22c55e,color:#fff
    style E fill:#f59e0b,color:#fff

Two pipelines: Upload (extract → chunk → embed → store) and Query (embed question → similarity search → build prompt → LLM answer).

Step-by-Step Build

Step 1: Text Extraction — Getting Text Out of Anything

The first challenge is extracting text from 8 different file types. Each needs a different parser:

// extract.js — Extract text from any supported file type
const pdfParse = require('pdf-parse');
const mammoth = require('mammoth');
const Tesseract = require('tesseract.js');
const csvParse = require('csv-parse/sync');
const fs = require('fs');
const path = require('path');

async function extractText(filePath) {
  const ext = path.extname(filePath).toLowerCase();
  const buffer = fs.readFileSync(filePath);

  switch (ext) {
    case '.pdf':
      const pdf = await pdfParse(buffer);
      return pdf.text;

    case '.docx':
      const result = await mammoth.extractRawText({ buffer });
      return result.value;

    case '.txt':
    case '.md':
      return buffer.toString('utf-8');

    case '.csv':
      const records = csvParse.parse(buffer, { columns: true });
      return records.map(row => Object.values(row).join(' | ')).join('\n');

    case '.png':
    case '.jpg':
    case '.jpeg':
    case '.webp':
      const { data } = await Tesseract.recognize(buffer, 'eng');
      return data.text;

    default:
      throw new Error(`Unsupported file type: ${ext}`);
  }
}

Step 2: Chunking — Breaking Text Into Searchable Pieces

You can't embed a whole document as one vector — it loses meaning. Instead, you break it into small, overlapping chunks. The overlap ensures context isn't lost at chunk boundaries.

  CHUNKING STRATEGY
  ═════════════════

  Original text (1000 chars):
  ┌──────────────────────────────────────────────────────────┐
  │ The quick brown fox jumps over the lazy dog. The dog     │
  │ barked loudly at the mailman. Meanwhile, the cat slept   │
  │ on the windowsill, completely unbothered by the chaos... │
  └──────────────────────────────────────────────────────────┘

  Chunk 1 (500 chars):              Chunk 2 (500 chars):
  ┌──────────────────────────┐      ┌──────────────────────────┐
  │ The quick brown fox      │      │ ...barked loudly at the  │
  │ jumps over the lazy dog. │      │ mailman. Meanwhile, the  │
  │ The dog barked loudly at │      │ cat slept on the window- │
  │ the mailman. Meanwhile...│      │ sill, completely...      │
  └──────────────────────────┘      └──────────────────────────┘
           ▲                                  ▲
           │           OVERLAP (50 chars)     │
           │     ┌─────────────────────┐      │
           └─────│ ...loudly at the    │──────┘
                 │ mailman. Meanwhile..│
                 └─────────────────────┘

  Why overlap? Without it, a question about "the mailman and the cat"
  might miss context because they span a chunk boundary.

// chunk.js — Split text into overlapping chunks
function chunkText(text, chunkSize = 500, overlap = 50) {
  const chunks = [];
  let start = 0;

  while (start < text.length) {
    let end = start + chunkSize;

    // Try to break at a sentence boundary
    if (end < text.length) {
      const lastPeriod = text.lastIndexOf('.', end);
      if (lastPeriod > start + chunkSize * 0.5) {
        end = lastPeriod + 1;
      }
    }

    const chunk = text.slice(start, end).trim();
    if (chunk.length > 0) {
      chunks.push({
        text: chunk,
        startIndex: start,
        endIndex: end,
      });
    }

    // Move forward, but keep `overlap` chars for context continuity
    start = end - overlap;
  }

  return chunks;
}

// Example:
const chunks = chunkText(documentText, 500, 50);
console.log(`Created ${chunks.length} chunks from document`);

Step 3: Embeddings — Turning Text Into Numbers

This is the magic step. An embedding model converts text into a vector (a list of numbers) that captures the meaning of the text. Similar meanings → similar vectors.

  HOW EMBEDDINGS WORK
  ════════════════════

  Text:                              Vector (384 dimensions):
  "Redis caching improves            [0.23, -0.11, 0.87, 0.45, ..., 0.12]
   database performance"                          ↕
                                     SIMILAR direction as:
  "Using cache to speed up           [0.25, -0.09, 0.85, 0.43, ..., 0.14]
   database queries"

                                     DIFFERENT direction from:
  "The weather is sunny              [0.91, 0.34, -0.22, 0.05, ..., -0.67]
   in Mumbai today"


  Similarity = Cosine of angle between vectors
  ═══════════════════════════════════════════════

       "Redis caching"    "Speed up DB"
              ↗                ↗          Cosine similarity = 0.95
            /                /            (very similar!)
           /               /
          ╱───────────────╱──────────────▶ dimension 1
         ╱               ╱
        ╱               ╱
                       ╱
                      ╱  "Weather in Mumbai"
                                              Cosine similarity = 0.12
                                              (very different!)

// embeddings.js — Generate embeddings using Xenova Transformers (runs locally!)
const { pipeline } = require('@xenova/transformers');

let embedder = null;

async function getEmbedder() {
  if (!embedder) {
    // Downloads the model on first use (~80MB), then cached
    embedder = await pipeline(
      'feature-extraction',
      'Xenova/all-MiniLM-L6-v2'
    );
  }
  return embedder;
}

async function embedText(text) {
  const model = await getEmbedder();
  const output = await model(text, { pooling: 'mean', normalize: true });
  return Array.from(output.data);
}

async function embedChunks(chunks) {
  const model = await getEmbedder();
  const embeddings = [];

  for (const chunk of chunks) {
    const vector = await embedText(chunk.text);
    embeddings.push({
      ...chunk,
      vector,
    });
  }

  return embeddings;
}

// Each vector is 384 dimensions — captures semantic meaning of the text

Step 4: Vector Store — Finding Similar Chunks

When a user asks a question, we embed the question using the same model, then find the chunks with the most similar vectors using cosine similarity.

// vector-store.js — In-memory vector store with cosine similarity search
class VectorStore {
  constructor() {
    this.documents = new Map(); // docId → { name, chunks }
  }

  addDocument(docId, name, embeddedChunks) {
    this.documents.set(docId, { name, chunks: embeddedChunks });
  }

  search(queryVector, topK = 5) {
    const results = [];

    for (const [docId, doc] of this.documents) {
      for (const chunk of doc.chunks) {
        const similarity = cosineSimilarity(queryVector, chunk.vector);
        results.push({
          text: chunk.text,
          document: doc.name,
          similarity,
          startIndex: chunk.startIndex,
        });
      }
    }

    // Sort by similarity (highest first) and return top K
    return results
      .sort((a, b) => b.similarity - a.similarity)
      .slice(0, topK);
  }

  removeDocument(docId) {
    this.documents.delete(docId);
  }
}

function cosineSimilarity(vecA, vecB) {
  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] * vecA[i];
    normB += vecB[i] * vecB[i];
  }

  return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}

  VECTOR SEARCH — HOW IT WORKS
  ═════════════════════════════

  User question: "What is the termination clause?"
       │
       ▼
  Embed question → [0.34, -0.22, 0.91, ...]
       │
       ▼
  Compare against ALL chunk vectors:
  ┌───────────────────────────────────────────────┐
  │ Chunk 1: "Company overview..."     sim: 0.12  │
  │ Chunk 2: "Payment terms..."        sim: 0.28  │
  │ Chunk 3: "Either party may         sim: 0.94  │ ◀── TOP MATCH
  │           terminate this                       │
  │           agreement with 30                    │
  │           days written notice..."              │
  │ Chunk 4: "Intellectual property..."sim: 0.15  │
  │ Chunk 5: "In the event of          sim: 0.87  │ ◀── 2nd match
  │           breach, termination                  │
  │           is effective..."                     │
  └───────────────────────────────────────────────┘
       │
       ▼
  Return top 5 chunks sorted by similarity

Step 5: Prompt Building — Giving the LLM Context

Now we take the retrieved chunks and build a prompt that tells the LLM: "Answer this question using ONLY the provided context."

// prompt.js — Build the RAG prompt
function buildPrompt(question, retrievedChunks) {
  const context = retrievedChunks
    .map((chunk, i) => `[Source ${i + 1}] ${chunk.text}`)
    .join('\n\n');

  return `You are a helpful assistant that answers questions based on the provided document context.

IMPORTANT RULES:
- Only answer based on the context below
- If the answer is not in the context, say "I don't have enough information to answer that"
- Be specific and cite which source you're drawing from
- Keep answers concise but complete

CONTEXT:
${context}

QUESTION: ${question}

ANSWER:`;
}

Step 6: LLM Response — Generating the Answer

DocMind uses Ollama to run an LLM locally. No API keys, no cloud costs, complete privacy.

// llm.js — Query Ollama for the answer
async function queryLLM(prompt) {
  try {
    const response = await fetch('http://localhost:11434/api/generate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        model: 'qwen2:0.5b',   // Small, fast, runs on CPU
        prompt,
        stream: false,
        options: {
          temperature: 0.3,     // Low temp = more focused answers
          num_predict: 500,     // Max tokens in response
        },
      }),
    });

    const data = await response.json();
    return data.response;
  } catch (err) {
    // Fallback: extract answer using regex matching
    return fallbackExtract(prompt);
  }
}

// Fallback when Ollama is unavailable
function fallbackExtract(prompt) {
  const contextMatch = prompt.match(/CONTEXT:\n([\s\S]*?)\nQUESTION:/);
  if (!contextMatch) return "Unable to process your question.";

  const context = contextMatch[1];
  // Return the most relevant chunk as the answer
  const sentences = context.split(/[.!?]+/).filter(s => s.trim().length > 20);
  return sentences.slice(0, 3).join('. ') + '.';
}

Putting It All Together

// rag-pipeline.js — The complete RAG pipeline
const { extractText } = require('./extract');
const { chunkText } = require('./chunk');
const { embedText, embedChunks } = require('./embeddings');
const { VectorStore } = require('./vector-store');
const { buildPrompt } = require('./prompt');
const { queryLLM } = require('./llm');

const vectorStore = new VectorStore();

// UPLOAD: Process a new document
async function uploadDocument(filePath, docId, docName) {
  console.log(`Processing ${docName}...`);

  // Step 1: Extract text
  const text = await extractText(filePath);
  console.log(`Extracted ${text.length} characters`);

  // Step 2: Chunk text
  const chunks = chunkText(text, 500, 50);
  console.log(`Created ${chunks.length} chunks`);

  // Step 3: Generate embeddings
  const embeddedChunks = await embedChunks(chunks);
  console.log(`Generated ${embeddedChunks.length} embeddings`);

  // Step 4: Store in vector index
  vectorStore.addDocument(docId, docName, embeddedChunks);
  console.log(`Document "${docName}" indexed and ready for queries`);

  return { chunks: chunks.length, characters: text.length };
}

// QUERY: Ask a question about uploaded documents
async function askQuestion(question) {
  // Step 1: Embed the question
  const queryVector = await embedText(question);

  // Step 2: Find similar chunks
  const topChunks = vectorStore.search(queryVector, 5);

  if (topChunks.length === 0 || topChunks[0].similarity < 0.3) {
    return {
      answer: "I don't have enough relevant information to answer that question.",
      sources: [],
    };
  }

  // Step 3: Build prompt with context
  const prompt = buildPrompt(question, topChunks);

  // Step 4: Get LLM answer
  const answer = await queryLLM(prompt);

  return {
    answer,
    sources: topChunks.map(c => ({
      document: c.document,
      text: c.text.slice(0, 100) + '...',
      similarity: (c.similarity * 100).toFixed(1) + '%',
    })),
  };
}

Real-World Example

Here's what it looks like in practice:

  USER INTERACTION
  ════════════════

  > Upload: "annual-report-2025.pdf" (85 pages)
  ✓ Extracted 142,000 characters
  ✓ Created 284 chunks
  ✓ Generated 284 embeddings (took 12s)
  ✓ Document indexed and ready

  > Question: "What was the total revenue for Q4 2025?"

  Answer: Based on the financial summary in the report,
  total revenue for Q4 2025 was ₹1,247 crores, representing
  a 23% increase from Q4 2024. The primary drivers were
  the expansion of the digital services segment (up 34%)
  and increased enterprise contracts.

  Sources:
  ├── [96.2% match] Page 42: "Q4 2025 Financial Highlights..."
  ├── [91.8% match] Page 43: "Revenue breakdown by segment..."
  └── [87.4% match] Page 7: "Executive summary: record revenue..."

Common Mistakes in RAG Apps

1. Chunks Too Large or Too Small

  ┌──────────────────────────────────────────────────┐
  │ Chunk size │ Problem                              │
  ├────────────┼──────────────────────────────────────┤
  │ Too small  │ Loses context. "30 days" means       │
  │ (< 100)    │ nothing without surrounding text.    │
  ├────────────┼──────────────────────────────────────┤
  │ Too large  │ Embedding captures too many topics.  │
  │ (> 2000)   │ Search returns irrelevant content.   │
  ├────────────┼──────────────────────────────────────┤
  │ Sweet spot │ 300-800 characters with 50-100        │
  │            │ character overlap.                    │
  └────────────┴──────────────────────────────────────┘

2. No Overlap Between Chunks

If a key fact spans two chunks, both chunks miss it. Always overlap.

3. Using Different Embedding Models for Chunks and Queries

The question and the chunks MUST be embedded with the same model. Different models produce incompatible vector spaces.

4. Not Setting a Similarity Threshold

If the top result has 0.3 similarity, the document probably doesn't contain the answer. Return "I don't know" instead of hallucinating.

5. Sending the Entire Document to the LLM

LLMs have context limits. Even with large contexts, more text = slower + more expensive + less focused answers. Retrieve only the relevant chunks.

When to Use RAG / When Not to Use It

  USE RAG WHEN:                          DON'T USE WHEN:
  ══════════════                         ════════════════

  ✅ You have specific documents          ❌ The question is general knowledge
     to answer questions about               (just use the LLM directly)
  ✅ Documents change frequently          ❌ You need the LLM to reason
     (RAG = always up to date)               creatively (RAG constrains it)
  ✅ Accuracy matters more than           ❌ Documents are tiny (< 1 page)
     creativity                              (just put them in the prompt)
  ✅ You want source attribution          ❌ You need real-time data
     ("this came from page 42")              (RAG works on static docs)
  ✅ Privacy matters (local RAG           ❌ You need to synthesize across
     = data never leaves your machine)       hundreds of documents (use
                                             a search engine instead)

Key Takeaways

RAG = Retrieve + Augment + Generate — ground LLM answers in real document content
Chunking strategy matters — 300-800 chars with 50-100 char overlap is the sweet spot
Embeddings capture meaning — cosine similarity finds semantically similar text, not keyword matches
Use the same embedding model for both documents and queries
Set a similarity threshold — don't answer if the best match is weak
Local RAG is possible — Ollama + Xenova Transformers, zero API costs
Fallback gracefully — if the LLM is down, extract answers from the chunks directly

Connect With Me

If you found this useful, drop a reaction and follow — I post deep-dive AI engineering tutorials every week, always with full code and working examples.

GitHub: github.com/Robins163 — DocMind source code is in the ai-portfolio repo
Twitter/X: twitter.com/robinsingh — I post shorter breakdowns there too
LinkedIn: linkedin.com/in/robinsingh

Next in this series: "WebSocket vs SSE vs Long Polling — The Complete Visual Comparison"

Questions or feedback? Drop a comment below — I respond to every one.

The Node.js Event Loop — Finally Explained Right

Robins163 — Tue, 14 Apr 2026 07:20:34 +0000

TL;DR: The event loop is how Node.js handles thousands of concurrent operations with a single thread. It's not magic — it's a loop with 6 phases, each processing a specific type of callback. This post breaks down every phase with diagrams, shows you exactly what happens when you run async code, and explains the tricky timing bugs that bite even senior developers.

The Problem

Ask 10 Node.js developers how the event loop works, and you'll get 12 different answers. Most explanations are either:

Too simplified: "It's just a loop that checks for callbacks" (unhelpful)
Too academic: 50 pages of libuv source code (nobody reads this)
Flat-out wrong: "setTimeout(fn, 0) runs immediately" (it doesn't)

I've seen bugs in production — at Mstock, with 100K+ concurrent users — that were caused by developers misunderstanding the event loop. A misplaced process.nextTick() caused a recursive callback that starved I/O and froze the server during peak trading hours.

Let me explain it the way I wish someone had explained it to me.

The Big Picture: What Is the Event Loop?

Node.js is single-threaded for your JavaScript code. But it handles I/O (file reads, network requests, database queries) by offloading those operations to the OS kernel or a thread pool, and then processing the results via callbacks.

The event loop is the mechanism that coordinates this. It's literally a loop — an infinite while(true) that keeps running as long as there's work to do.

  THE EVENT LOOP — SIMPLIFIED VIEW
  ═════════════════════════════════

  ┌──────────────────────────────────────────────┐
  │                                              │
  │    Your JavaScript Code (Single Thread)      │
  │                                              │
  │    console.log('start');                      │
  │    fs.readFile('data.json', callback);        │
  │    setTimeout(fn, 100);                       │
  │    http.get(url, callback);                   │
  │    console.log('end');                        │
  │                                              │
  └──────────────────┬───────────────────────────┘
                     │
                     │  "Hey OS/thread pool,
                     │   handle these I/O tasks"
                     │
                     ▼
  ┌──────────────────────────────────────────────┐
  │                                              │
  │    OS Kernel / libuv Thread Pool              │
  │    (Does the actual I/O work)                │
  │                                              │
  │    ┌─────────┐ ┌──────────┐ ┌──────────┐    │
  │    │ Thread 1│ │ Thread 2 │ │ Thread 3 │    │
  │    │ fs.read │ │ dns.look │ │ crypto   │    │
  │    └─────────┘ └──────────┘ └──────────┘    │
  │                                              │
  └──────────────────┬───────────────────────────┘
                     │
                     │  "Done! Here are the results"
                     │
                     ▼
  ┌──────────────────────────────────────────────┐
  │                                              │
  │    EVENT LOOP                                │
  │    (Picks up completed I/O callbacks         │
  │     and runs them on the main thread)        │
  │                                              │
  │    while (hasWork()) {                       │
  │      processTimers();                        │
  │      processIOCallbacks();                   │
  │      processImmediates();                    │
  │      // ... more phases                      │
  │    }                                         │
  │                                              │
  └──────────────────────────────────────────────┘

The 6 Phases of the Event Loop

This is where most explanations go wrong. The event loop isn't one queue — it's 6 phases, each with its own queue. The loop visits each phase in order, processes all callbacks in that phase's queue, and then moves to the next phase.

  THE 6 PHASES (executed in order, every iteration)
  ═════════════════════════════════════════════════

   ┌───────────────────────────────────────────┐
   │                                           │
   │    ┌─────────────────────────────────┐    │
   │    │  1. TIMERS                      │    │
   │    │  setTimeout(), setInterval()    │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │    ┌──────────────▼──────────────────┐    │
   │    │  2. PENDING CALLBACKS           │    │
   │    │  System-level callbacks          │    │
   │    │  (TCP errors, etc.)             │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │    ┌──────────────▼──────────────────┐    │
   │    │  3. IDLE / PREPARE              │    │
   │    │  Internal use only              │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │    ┌──────────────▼──────────────────┐    │
   │    │  4. POLL                        │    │
   │    │  I/O callbacks: fs, net, etc.   │◀──── Most time spent here
   │    │  (This is where Node "waits")   │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │    ┌──────────────▼──────────────────┐    │
   │    │  5. CHECK                       │    │
   │    │  setImmediate() callbacks       │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │    ┌──────────────▼──────────────────┐    │
   │    │  6. CLOSE CALLBACKS             │    │
   │    │  socket.on('close'), etc.       │    │
   │    └──────────────┬──────────────────┘    │
   │                   │                       │
   │                   └───────────────────────┘
   │                   Loop back to Phase 1
   └───────────────────────────────────────────┘

flowchart TD
    A[Timers<br/>setTimeout, setInterval] --> B[Pending Callbacks<br/>System-level callbacks]
    B --> C[Idle / Prepare<br/>Internal use only]
    C --> D[Poll<br/>I/O callbacks - fs, net, http<br/>Node waits here if nothing to do]
    D --> E[Check<br/>setImmediate callbacks]
    E --> F[Close Callbacks<br/>socket.on close]
    F --> A

    style D fill:#3b82f6,color:#fff
    style A fill:#f59e0b,color:#fff
    style E fill:#22c55e,color:#fff

Phase 1: Timers

Executes callbacks scheduled by setTimeout() and setInterval().

Important: The timer specifies a minimum delay, not an exact delay. If the event loop is busy processing I/O callbacks, your timer callback runs later than expected.

// timer-demo.js — Timers don't run at exact times
const start = Date.now();

setTimeout(() => {
  console.log(`Timer fired after ${Date.now() - start}ms`);
  // Expected: ~100ms
  // Actual:   100-115ms (depending on event loop load)
}, 100);

// Simulate work that delays the timer
const blockFor50ms = Date.now() + 50;
while (Date.now() < blockFor50ms) {} // Blocking the loop

// Output: "Timer fired after ~150ms" (100ms delay + 50ms blocking)

Phase 2: Pending Callbacks

Executes I/O callbacks that were deferred from the previous loop iteration. This handles system-level errors like TCP ECONNREFUSED.

You rarely interact with this phase directly.

Phase 3: Idle / Prepare

Internal to Node.js and libuv. Not relevant to application code.

Phase 4: Poll (The Most Important Phase)

This is where Node.js spends most of its time. The poll phase does two things:

Calculates how long to block and wait for I/O events
Processes I/O callbacks in the poll queue (file reads, network responses, etc.)

  POLL PHASE BEHAVIOR
  ════════════════════

  Enter Poll Phase
        │
        ▼
  ┌─────────────────────┐
  │ Poll queue empty?    │
  └──────────┬──────────┘
             │
     ┌───────┼───────┐
     │ NO    │       │ YES
     ▼       │       ▼
  ┌──────────┐  ┌──────────────────────┐
  │ Execute  │  │ Any setImmediate()   │
  │ ALL I/O  │  │ callbacks scheduled? │
  │ callbacks│  └──────────┬───────────┘
  │ in queue │       ┌─────┼─────┐
  └──────────┘       │ YES │     │ NO
                     ▼     │     ▼
              ┌──────────┐ │ ┌──────────────┐
              │ Move to  │ │ │ WAIT here    │
              │ Check    │ │ │ for new I/O  │
              │ phase    │ │ │ events       │
              └──────────┘ │ │ (up to timer │
                           │ │  threshold)  │
                           │ └──────────────┘

Key insight: When the poll queue is empty and no setImmediate() is scheduled, Node.js blocks here, waiting for new I/O events. This is how Node.js achieves near-zero CPU usage when idle.

Phase 5: Check

Executes setImmediate() callbacks. This phase runs immediately after the poll phase completes.

// setImmediate runs after I/O, setTimeout runs at start of next loop
const fs = require('fs');

fs.readFile(__filename, () => {
  // Inside an I/O callback...

  setImmediate(() => {
    console.log('1. setImmediate');  // Runs FIRST (check phase is next)
  });

  setTimeout(() => {
    console.log('2. setTimeout');   // Runs SECOND (timers phase, next iteration)
  }, 0);
});

// Output (guaranteed order):
// 1. setImmediate
// 2. setTimeout

Phase 6: Close Callbacks

Handles close events like socket.on('close'). Cleanup phase.

Microtasks: The Queue That Jumps the Line

Here's what trips up most developers: between every phase (and even between callbacks within a phase), Node.js processes the microtask queues. These are:

process.nextTick() queue (highest priority)
Promise resolution queue

  MICROTASKS — THEY RUN BETWEEN EVERY PHASE
  ══════════════════════════════════════════

  ┌────────┐   ┌─────────┐   ┌────────┐   ┌─────────┐   ┌────────┐
  │ Timers │──▶│ Micro-  │──▶│Pending │──▶│ Micro-  │──▶│  Poll  │─ ─ ▶
  │        │   │ tasks   │   │Callback│   │ tasks   │   │        │
  └────────┘   └─────────┘   └────────┘   └─────────┘   └────────┘
                    │                          │
                    ▼                          ▼
              ┌──────────┐              ┌──────────┐
              │nextTick()│              │nextTick()│
              │Promise   │              │Promise   │
              └──────────┘              └──────────┘

  Microtask queues are drained COMPLETELY between phases.
  If you keep adding to them, they STARVE the event loop.

flowchart LR
    T[Timers] --> M1[Microtasks<br/>nextTick + Promises]
    M1 --> P[Pending]
    P --> M2[Microtasks]
    M2 --> I[Idle]
    I --> M3[Microtasks]
    M3 --> PO[Poll]
    PO --> M4[Microtasks]
    M4 --> C[Check]
    C --> M5[Microtasks]
    M5 --> CL[Close]
    CL --> M6[Microtasks]
    M6 --> T

    style M1 fill:#ef4444,color:#fff
    style M2 fill:#ef4444,color:#fff
    style M3 fill:#ef4444,color:#fff
    style M4 fill:#ef4444,color:#fff
    style M5 fill:#ef4444,color:#fff
    style M6 fill:#ef4444,color:#fff

process.nextTick() vs Promise vs setTimeout vs setImmediate

This is the single most confusing part of Node.js. Here's the definitive ordering:

// ordering-demo.js — Run this to see the exact execution order

console.log('1. Script start');

setTimeout(() => console.log('2. setTimeout'), 0);

setImmediate(() => console.log('3. setImmediate'));

Promise.resolve().then(() => console.log('4. Promise'));

process.nextTick(() => console.log('5. nextTick'));

console.log('6. Script end');

// OUTPUT:
// 1. Script start         ← Synchronous (runs immediately)
// 6. Script end           ← Synchronous (runs immediately)
// 5. nextTick             ← Microtask (highest priority, before promises)
// 4. Promise              ← Microtask (after nextTick, before macrotasks)
// 2. setTimeout           ← Macrotask (timers phase)*
// 3. setImmediate         ← Macrotask (check phase)*
// 
// * setTimeout vs setImmediate order is NON-DETERMINISTIC
//   when called from the main module (outside I/O callbacks)

  EXECUTION PRIORITY (highest → lowest)
  ══════════════════════════════════════

  ┌─────────────────────────────────────┐
  │  1. Synchronous code                │  ← Runs first, always
  ├─────────────────────────────────────┤
  │  2. process.nextTick()              │  ← Microtask, highest async priority
  ├─────────────────────────────────────┤
  │  3. Promise.then() / await          │  ← Microtask, after nextTick
  ├─────────────────────────────────────┤
  │  4. setTimeout(fn, 0)               │  ← Macrotask, timers phase
  ├─────────────────────────────────────┤
  │  5. setImmediate(fn)                │  ← Macrotask, check phase
  ├─────────────────────────────────────┤
  │  6. I/O callbacks                   │  ← Poll phase
  └─────────────────────────────────────┘

The Dangerous Recursive nextTick

This is the bug we hit at Mstock. process.nextTick() callbacks are processed before any I/O — if you call nextTick recursively, you starve the event loop:

// DANGEROUS — this starves all I/O
function badRecursion() {
  process.nextTick(() => {
    doSomeWork();
    badRecursion(); // nextTick queue never empties!
  });
}
badRecursion();
// Result: setTimeout, setImmediate, I/O callbacks NEVER execute
// The event loop is stuck processing nextTick forever

// SAFE — use setImmediate for recursive operations
function safeRecursion() {
  setImmediate(() => {
    doSomeWork();
    safeRecursion(); // Gives I/O a chance to run between iterations
  });
}

  RECURSIVE nextTick vs setImmediate
  ═══════════════════════════════════

  nextTick (DANGEROUS):
  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
  │ tick │→│ tick │→│ tick │→│ tick │→│ tick │→ ... forever
  └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
  I/O:   ❌ NEVER RUNS ❌

  setImmediate (SAFE):
  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
  │ imm  │→│ I/O  │→│ imm  │→│ I/O  │→│ imm  │→ ...
  └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
  I/O:   ✅ RUNS BETWEEN EACH ✅

Real-World Example: Event Loop in a Stock Trading Server

Here's how the event loop handles a typical moment during market hours at Mstock:

  ONE TICK OF THE EVENT LOOP (during market hours)
  ═══════════════════════════════════════════════════

  Phase 1: TIMERS
  ├── Price refresh timer (every 1s) → fetch latest prices from exchange
  ├── Session cleanup timer (every 30s) → remove expired sessions
  └── Health check timer (every 5s) → ping database and Redis

  Phase 2: PENDING CALLBACKS
  └── TCP error from disconnected client → log and clean up

  Phase 4: POLL (I/O)
  ├── WebSocket message: user subscribes to RELIANCE → add to channel
  ├── WebSocket message: user places BUY order → validate + submit
  ├── Redis response: price update for INFY → broadcast to subscribers
  ├── PostgreSQL response: order saved → send confirmation to user
  └── HTTP request: GET /api/portfolio → return cached data

  Phase 5: CHECK
  └── setImmediate: batch-write 50 price updates to TimescaleDB

  MICROTASKS (between each phase):
  ├── nextTick: emit 'order-confirmed' event
  └── Promise: resolve async order validation pipeline

// real-world-loop.js — How these phases interact in a trading server
const http = require('http');
const Redis = require('ioredis');
const { Pool } = require('pg');

const redis = new Redis();
const db = new Pool({ connectionString: process.env.DATABASE_URL });

const priceBatch = [];

// TIMERS PHASE — scheduled recurring tasks
setInterval(async () => {
  // Fetch latest prices from exchange feed
  const prices = await exchangeAPI.getLatestPrices();

  // Publish to Redis (triggers I/O callbacks for subscribers)
  for (const price of prices) {
    await redis.publish(`price:${price.symbol}`, JSON.stringify(price));
    priceBatch.push(price);
  }
}, 1000);

// CHECK PHASE — batch writes using setImmediate
setInterval(() => {
  if (priceBatch.length > 0) {
    const batch = priceBatch.splice(0, priceBatch.length);

    // setImmediate ensures I/O from poll phase is processed first
    setImmediate(async () => {
      await batchInsertPrices(db, batch);
    });
  }
}, 5000);

// POLL PHASE — HTTP server handles incoming requests
const server = http.createServer(async (req, res) => {
  if (req.url === '/api/portfolio') {
    // This callback runs in the poll phase
    const portfolio = await getPortfolio(req.userId);

    // nextTick ensures response is sent before any other I/O
    process.nextTick(() => {
      res.writeHead(200, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify(portfolio));
    });
  }
});

Common Mistakes (and How to Fix Them)

1. Blocking the Event Loop

The cardinal sin. If you block the main thread, nothing else runs — no timers, no I/O, no WebSocket messages.

// BAD — blocks the event loop for ~2 seconds
app.get('/api/report', (req, res) => {
  const data = fs.readFileSync('huge-file.csv');  // SYNCHRONOUS!
  const parsed = parseCSV(data);                  // CPU-intensive!
  res.json(parsed);
});

// GOOD — non-blocking with streaming
app.get('/api/report', (req, res) => {
  const stream = fs.createReadStream('huge-file.csv');
  stream.pipe(csvParser()).pipe(res);
});

// GOOD — offload CPU work to worker thread
const { Worker } = require('worker_threads');

app.get('/api/report', (req, res) => {
  const worker = new Worker('./csv-worker.js', {
    workerData: { file: 'huge-file.csv' },
  });
  worker.on('message', (result) => res.json(result));
  worker.on('error', (err) => res.status(500).json({ error: err.message }));
});

2. Assuming setTimeout(fn, 0) Is Instant

// This does NOT run "immediately"
setTimeout(() => {
  console.log('This runs after ALL microtasks + at least 1ms delay');
}, 0);

// If you need "as soon as possible after I/O":
setImmediate(() => {
  console.log('This runs right after the current poll phase');
});

// If you need "before anything else async":
process.nextTick(() => {
  console.log('This runs before setTimeout, setImmediate, and I/O');
});

3. Unhandled Promise Rejections

Unhandled rejections used to be silently swallowed. Since Node 15, they crash the process:

// BAD — unhandled rejection
async function fetchUser(id) {
  const res = await db.query('SELECT * FROM users WHERE id = $1', [id]);
  return res.rows[0]; // What if db.query throws?
}
fetchUser(123); // No .catch() → process crashes

// GOOD — always handle rejections
fetchUser(123).catch(err => console.error('Failed:', err));

// Or globally (as safety net, not primary handling):
process.on('unhandledRejection', (reason, promise) => {
  console.error('Unhandled Rejection at:', promise, 'reason:', reason);
  // Log and exit gracefully
});

4. Not Understanding async/await and the Event Loop

await doesn't block the event loop — it yields control back to it:

// What actually happens with async/await:

async function handleRequest() {
  console.log('A');                    // Synchronous
  const data = await fetchFromDB();    // Yields to event loop here
  console.log('B');                    // Runs when promise resolves
  return data;
}

// Is equivalent to:
function handleRequest() {
  console.log('A');
  return fetchFromDB().then(data => {
    console.log('B');
    return data;
  });
}

// The event loop is FREE to handle other requests while awaiting

  async/await UNDER THE HOOD
  ══════════════════════════

  handleRequest()
       │
       ▼
  console.log('A')          ← Synchronous, runs now
       │
       ▼
  await fetchFromDB()       ← Sends query, YIELDS to event loop
       │                       Event loop handles other work:
       │                       ├── WebSocket messages
       │                       ├── Other HTTP requests
       │                       ├── Timer callbacks
       │                       └── Price updates
       │
       ▼ (when DB responds)
  console.log('B')          ← Runs as microtask (promise resolution)
       │
       ▼
  return data

How to Monitor Your Event Loop

If your event loop is being blocked or starved, your server slows down for everyone. Here's how to detect it:

// event-loop-monitor.js — Detect event loop delays
let lastCheck = Date.now();

setInterval(() => {
  const now = Date.now();
  const delay = now - lastCheck - 1000; // Expected: 0ms
  lastCheck = now;

  if (delay > 50) {
    console.warn(`Event loop delayed by ${delay}ms — potential blocking operation!`);
  }
}, 1000);

// For production: use the built-in perf_hooks
const { monitorEventLoopDelay } = require('perf_hooks');

const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();

setInterval(() => {
  console.log({
    min: histogram.min / 1e6,    // Convert ns to ms
    max: histogram.max / 1e6,
    mean: histogram.mean / 1e6,
    p99: histogram.percentile(99) / 1e6,
  });
  histogram.reset();
}, 5000);

When to Use What

  ┌──────────────────────┬──────────────────────────────────────┐
  │ API                  │ When to use                          │
  ├──────────────────────┼──────────────────────────────────────┤
  │ process.nextTick()   │ Emit events after constructor        │
  │                      │ returns. Ensure callback runs        │
  │                      │ before ANY I/O.                      │
  ├──────────────────────┼──────────────────────────────────────┤
  │ Promise / await      │ Async operations. Standard flow      │
  │                      │ control. 99% of the time, use this. │
  ├──────────────────────┼──────────────────────────────────────┤
  │ setTimeout(fn, 0)    │ Defer work to next loop iteration.   │
  │                      │ Give I/O a chance to breathe.        │
  ├──────────────────────┼──────────────────────────────────────┤
  │ setImmediate()       │ Run callback after I/O events in     │
  │                      │ current loop. Best for recursive     │
  │                      │ operations.                          │
  ├──────────────────────┼──────────────────────────────────────┤
  │ Worker Threads       │ CPU-intensive work (parsing, crypto, │
  │                      │ compression). Keep the event loop    │
  │                      │ free.                                │
  └──────────────────────┴──────────────────────────────────────┘

Key Takeaways

The event loop has 6 phases: Timers → Pending → Idle → Poll → Check → Close
Microtasks (nextTick, Promises) run between every phase — they have the highest priority
Poll phase is where Node spends most time — it blocks here waiting for I/O when idle
process.nextTick() runs before Promises, which run before setTimeout(0), which runs before setImmediate()
Inside an I/O callback, setImmediate() always runs before setTimeout(0) (deterministic)
Outside I/O callbacks, setTimeout(0) vs setImmediate() order is non-deterministic
Never block the event loop — use streams for large files, worker threads for CPU work
Recursive nextTick() starves I/O — use setImmediate() for recursive async patterns
Monitor your event loop — use perf_hooks.monitorEventLoopDelay() in production

Connect with Me

If this helped you finally understand the event loop, follow along — I post visual explainers on Node.js, system design, and AI engineering every week.

Twitter/X: @robinsingh — short threads on engineering concepts
LinkedIn: Robin Singh — longer posts on system design and career
GitHub: robins163 — code from all my posts
Hashnode: unknowntoplay.hashnode.dev — full blog with diagrams

Next up: "I Built a RAG App That Chats With Any PDF — Here's How" — the full architecture, embedding pipeline, and code.

System Design: Real-Time Stock Trading Platform — From My Experience Building Mstock

Robins163 — Tue, 14 Apr 2026 07:20:32 +0000

TL;DR: I helped build Mstock, a stock trading platform handling 100K+ concurrent users during market hours. This post walks through the full system design — from WebSocket connections for live prices, to order matching, to the caching and database layers — with architecture diagrams, sequence flows, and the hard lessons we learned in production.

The Problem

Designing a stock trading platform isn't like building a typical web app. The constraints are brutal:

Latency: Order placement must complete in under 100ms. Users lose real money on delays.
Concurrency: 100K+ users online simultaneously during the 6-hour trading window (9:15 AM - 3:30 PM IST).
Data velocity: Stock prices update every second across 5,000+ symbols. That's 5,000 messages/second to broadcast.
Accuracy: A rounding error in price or quantity isn't a bug — it's a financial incident.
Availability: Downtime during market hours = regulatory scrutiny + angry traders + lost revenue.

I worked on Mstock as part of the frontend architecture team, but I had deep exposure to the full stack. Here's how a system like this is designed from the ground up.

High-Level Architecture

  STOCK TRADING PLATFORM — HIGH LEVEL ARCHITECTURE
  ═══════════════════════════════════════════════════

  ┌─────────────────────────────────────────────────────────────┐
  │                      CLIENT LAYER                           │
  │                                                             │
  │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌────────────┐ │
  │  │   Web    │  │ Mobile   │  │ Mobile   │  │  Desktop   │ │
  │  │  (React) │  │  (iOS)   │  │(Android) │  │  Terminal  │ │
  │  └────┬─────┘  └────┬─────┘  └────┬─────┘  └─────┬──────┘ │
  │       │              │              │              │        │
  │       └──────────────┴──────┬───────┴──────────────┘        │
  │                             │                               │
  └─────────────────────────────┼───────────────────────────────┘
                                │  HTTPS + WebSocket
                                ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                    GATEWAY LAYER                             │
  │                                                             │
  │  ┌──────────────────────────────────────────────────────┐   │
  │  │              API Gateway / Load Balancer              │   │
  │  │         (Nginx / AWS ALB / Kong / Traefik)           │   │
  │  │                                                      │   │
  │  │  - SSL termination                                   │   │
  │  │  - Rate limiting (1000 req/min per user)             │   │
  │  │  - WebSocket upgrade handling                        │   │
  │  │  - IP whitelisting for exchange connections          │   │
  │  └──────────────────────┬───────────────────────────────┘   │
  │                         │                                   │
  └─────────────────────────┼───────────────────────────────────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                   APPLICATION LAYER                         │
  │                                                             │
  │  ┌────────────┐  ┌────────────┐  ┌────────────────────┐    │
  │  │   Order    │  │  Market    │  │    Portfolio       │    │
  │  │  Service   │  │   Data     │  │    Service         │    │
  │  │            │  │  Service   │  │                    │    │
  │  │ - Place    │  │ - Live     │  │ - Holdings         │    │
  │  │ - Cancel   │  │   prices   │  │ - P&L calculation  │    │
  │  │ - Modify   │  │ - OHLCV    │  │ - Margin check     │    │
  │  │ - History  │  │ - Depth    │  │ - Fund transfer    │    │
  │  └──────┬─────┘  └──────┬─────┘  └──────┬─────────────┘    │
  │         │               │               │                   │
  │  ┌────────────┐  ┌────────────┐  ┌────────────────────┐    │
  │  │   Auth     │  │ WebSocket  │  │   Notification     │    │
  │  │  Service   │  │   Server   │  │    Service         │    │
  │  │            │  │            │  │                    │    │
  │  │ - JWT      │  │ - Price    │  │ - Order status     │    │
  │  │ - 2FA      │  │   feed     │  │ - Price alerts     │    │
  │  │ - Session  │  │ - Order    │  │ - Margin calls     │    │
  │  │            │  │   updates  │  │ - Push + Email     │    │
  │  └────────────┘  └────────────┘  └────────────────────┘    │
  │                                                             │
  └──────────────────────────┬──────────────────────────────────┘
                             │
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                     DATA LAYER                              │
  │                                                             │
  │  ┌────────────┐  ┌────────────┐  ┌────────────────────┐    │
  │  │   Redis    │  │ PostgreSQL │  │    TimescaleDB     │    │
  │  │  Cluster   │  │  (Primary) │  │  (Time-series)     │    │
  │  │            │  │            │  │                    │    │
  │  │ - Sessions │  │ - Users    │  │ - OHLCV candles    │    │
  │  │ - Prices   │  │ - Orders   │  │ - Tick data        │    │
  │  │ - Pub/Sub  │  │ - Trades   │  │ - Historical       │    │
  │  │ - Queues   │  │ - Holdings │  │   charts           │    │
  │  └────────────┘  └────────────┘  └────────────────────┘    │
  │                                                             │
  └─────────────────────────────────────────────────────────────┘
                             │
                             ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                  EXCHANGE LAYER                              │
  │                                                             │
  │  ┌────────────┐  ┌────────────┐  ┌────────────────────┐    │
  │  │    NSE     │  │    BSE     │  │    MCX             │    │
  │  │  Exchange  │  │  Exchange  │  │  (Commodities)     │    │
  │  └────────────┘  └────────────┘  └────────────────────┘    │
  │                                                             │
  │  Connected via: FIX Protocol / Exchange API                 │
  │  Latency requirement: < 5ms to exchange gateway             │
  │                                                             │
  └─────────────────────────────────────────────────────────────┘

Core Component Deep-Dives

1. WebSocket Server — The Real-Time Backbone

The WebSocket server is the heart of the real-time experience. Every user has a persistent connection that receives live price updates, order status changes, and alerts.

  WEBSOCKET CONNECTION LIFECYCLE
  ══════════════════════════════

  Client                          Server
    │                                │
    │──── HTTP Upgrade Request ─────▶│
    │     GET /ws                    │
    │     Upgrade: websocket         │
    │     Connection: Upgrade        │
    │     Sec-WebSocket-Key: ...     │
    │                                │
    │◀── 101 Switching Protocols ───│
    │     Upgrade: websocket         │
    │     Sec-WebSocket-Accept: ...  │
    │                                │
    │◀════ WebSocket Connected ═════▶│
    │                                │
    │──── Subscribe: RELIANCE ──────▶│
    │──── Subscribe: INFY ──────────▶│
    │──── Subscribe: TCS ───────────▶│
    │                                │
    │◀─── Price: RELIANCE 2450.50 ──│  (every ~1s)
    │◀─── Price: INFY 1623.75 ──────│
    │◀─── Price: TCS 3890.20 ───────│
    │◀─── Depth: RELIANCE {...} ────│
    │                                │
    │──── Place Order ──────────────▶│
    │◀─── Order Confirmed ──────────│
    │◀─── Order Executed ───────────│
    │                                │
    │──── Heartbeat (ping) ─────────▶│  (every 30s)
    │◀─── Heartbeat (pong) ─────────│
    │                                │

flowchart TD
    A[Client Connects via WS] --> B[Authenticate JWT]
    B -->|Valid| C[Add to Connection Pool]
    B -->|Invalid| D[Reject + Close]
    C --> E[Client Subscribes to Symbols]
    E --> F[Add to Symbol Channels]
    F --> G[Receive Price Updates via Redis Pub/Sub]
    G --> H[Broadcast to Subscribed Clients]
    H --> G

    style D fill:#ef4444,color:#fff
    style H fill:#22c55e,color:#fff

WebSocket Server Implementation

// ws-server.js — WebSocket server with Redis Pub/Sub for price distribution
const WebSocket = require('ws');
const Redis = require('ioredis');
const jwt = require('jsonwebtoken');

const wss = new WebSocket.Server({ noServer: true });
const redisSub = new Redis();  // Subscriber connection
const redisPub = new Redis();  // Publisher connection

// Track subscriptions: symbol → Set of WebSocket clients
const subscriptions = new Map();

// Track clients: ws → { userId, subscribedSymbols }
const clients = new Map();

// Handle new WebSocket connections
wss.on('connection', (ws, req) => {
  const userId = req.userId; // Set during upgrade authentication
  clients.set(ws, { userId, subscribedSymbols: new Set() });

  ws.on('message', (raw) => {
    const msg = JSON.parse(raw);

    switch (msg.type) {
      case 'subscribe':
        handleSubscribe(ws, msg.symbols);
        break;
      case 'unsubscribe':
        handleUnsubscribe(ws, msg.symbols);
        break;
      case 'order':
        handleOrder(ws, msg.data);
        break;
    }
  });

  ws.on('close', () => {
    // Clean up subscriptions
    const client = clients.get(ws);
    if (client) {
      for (const symbol of client.subscribedSymbols) {
        const subs = subscriptions.get(symbol);
        if (subs) {
          subs.delete(ws);
          if (subs.size === 0) {
            subscriptions.delete(symbol);
            redisSub.unsubscribe(`price:${symbol}`);
          }
        }
      }
      clients.delete(ws);
    }
  });
});

function handleSubscribe(ws, symbols) {
  const client = clients.get(ws);

  for (const symbol of symbols) {
    // Add to client's subscription set
    client.subscribedSymbols.add(symbol);

    // Add to symbol's subscriber set
    if (!subscriptions.has(symbol)) {
      subscriptions.set(symbol, new Set());
      // First subscriber for this symbol — subscribe to Redis channel
      redisSub.subscribe(`price:${symbol}`);
    }
    subscriptions.get(symbol).add(ws);
  }
}

// When Redis publishes a price update, broadcast to all subscribed clients
redisSub.on('message', (channel, message) => {
  const symbol = channel.replace('price:', '');
  const subscribers = subscriptions.get(symbol);

  if (subscribers) {
    const payload = JSON.stringify({
      type: 'price',
      symbol,
      data: JSON.parse(message),
      timestamp: Date.now(),
    });

    for (const ws of subscribers) {
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(payload);
      }
    }
  }
});

Scaling WebSockets to 100K+ Connections

A single Node.js process can handle ~10K-50K WebSocket connections (depending on message frequency). For 100K+, you need horizontal scaling:

  WEBSOCKET SCALING ARCHITECTURE
  ══════════════════════════════

                    ┌───────────────────┐
                    │   Load Balancer   │
                    │  (Sticky Sessions │
                    │   by IP hash)     │
                    └────────┬──────────┘
                             │
            ┌────────────────┼────────────────┐
            ▼                ▼                ▼
  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
  │  WS Server 1 │ │  WS Server 2 │ │  WS Server 3 │
  │  ~35K conns  │ │  ~35K conns  │ │  ~35K conns  │
  └──────┬───────┘ └──────┬───────┘ └──────┬───────┘
         │                │                │
         └────────────────┼────────────────┘
                          │
                 ┌────────▼────────┐
                 │  Redis Pub/Sub  │
                 │                 │
                 │  All 3 servers  │
                 │  subscribe to   │
                 │  same channels  │
                 └─────────────────┘

  Key insight: Redis Pub/Sub acts as the broadcast bus.
  When a price update comes in, ALL servers receive it
  and broadcast to their own connected clients.

Why sticky sessions? WebSocket connections are stateful. If a client disconnects and reconnects, they should hit the same server to resume their subscriptions. IP-hash based load balancing handles this without session state.

2. Order Flow — The Critical Path

The order flow is the most latency-sensitive part of the system. From the moment a user clicks "Buy" to the order reaching the exchange, every millisecond counts.

  ORDER FLOW — END TO END
  ═══════════════════════

  ┌────────┐     ┌──────────┐     ┌──────────────┐     ┌──────────┐     ┌──────────┐
  │ Client │────▶│   API    │────▶│    Order     │────▶│ Exchange │────▶│  Order   │
  │  (UI)  │     │ Gateway  │     │   Service    │     │ Gateway  │     │ Matching │
  └────────┘     └──────────┘     └──────────────┘     └──────────┘     └──────────┘
                                        │
                                        │ Validates:
                                        ├─ Sufficient margin?
                                        ├─ Valid symbol?
                                        ├─ Market hours?
                                        ├─ Price within circuit limits?
                                        ├─ Quantity within limits?
                                        └─ User not restricted?
                                        │
                                        ▼
                                  ┌──────────────┐
                                  │  Risk Check  │
                                  │   Engine     │
                                  └──────────────┘

  Timeline:
  ─────────────────────────────────────────────────────▶ time
  │ User clicks │ Validation │ Risk Check │ To Exchange │
  │   "Buy"     │   ~5ms     │   ~3ms     │   ~10ms     │
  │             │            │            │             │
  │             └────────────┴────────────┴─────────────│
  │                    Total: ~20-50ms                   │

sequenceDiagram
    participant Client
    participant API as API Gateway
    participant Order as Order Service
    participant Risk as Risk Engine
    participant Exchange
    participant WS as WebSocket Server
    participant DB as PostgreSQL

    Client->>API: Place Order (BUY RELIANCE x100 @ 2450)
    API->>Order: Validate & Forward
    Order->>Order: Check market hours, symbol, quantity
    Order->>Risk: Margin check
    Risk-->>Order: Margin OK (available: ₹5,00,000)
    Order->>DB: Save order (status: PENDING)
    Order->>Exchange: Submit via FIX Protocol
    Exchange-->>Order: Order ID: ORD123456
    Order->>DB: Update status: SUBMITTED
    Order->>WS: Notify client
    WS-->>Client: Order Submitted ✓

    Note over Exchange: Exchange matches order...

    Exchange-->>Order: Execution Report (FILLED @ 2449.50)
    Order->>DB: Update status: EXECUTED
    Order->>DB: Update holdings + positions
    Order->>WS: Notify client
    WS-->>Client: Order Executed ✓ (100 x 2449.50)

Order Validation Code

// order-service.js — Order validation and submission
const MARKET_OPEN = { hour: 9, minute: 15 };
const MARKET_CLOSE = { hour: 15, minute: 30 };

async function placeOrder(userId, orderData) {
  const { symbol, quantity, price, type, side } = orderData;

  // Step 1: Basic validation
  validateOrderParams(symbol, quantity, price, type, side);

  // Step 2: Check market hours (IST)
  const now = new Date();
  const istHour = now.getUTCHours() + 5;
  const istMin = now.getUTCMinutes() + 30;
  if (!isMarketOpen(istHour, istMin)) {
    throw new Error('Market is closed. Trading hours: 9:15 AM - 3:30 PM IST');
  }

  // Step 3: Margin check — can the user afford this order?
  const requiredMargin = calculateMargin(symbol, quantity, price, side);
  const availableMargin = await getAvailableMargin(userId);

  if (requiredMargin > availableMargin) {
    throw new Error(
      `Insufficient margin. Required: ₹${requiredMargin}, Available: ₹${availableMargin}`
    );
  }

  // Step 4: Block margin (prevent double-spending)
  await blockMargin(userId, requiredMargin);

  // Step 5: Save order to database
  const order = await db.query(
    `INSERT INTO orders (user_id, symbol, quantity, price, type, side, status, created_at)
     VALUES ($1, $2, $3, $4, $5, $6, 'PENDING', NOW())
     RETURNING *`,
    [userId, symbol, quantity, price, type, side]
  );

  // Step 6: Submit to exchange
  try {
    const exchangeOrderId = await exchangeGateway.submit({
      orderId: order.rows[0].id,
      symbol,
      quantity,
      price,
      type,  // LIMIT, MARKET, SL, SL-M
      side,  // BUY, SELL
    });

    // Update with exchange order ID
    await db.query(
      'UPDATE orders SET exchange_order_id = $1, status = $2 WHERE id = $3',
      [exchangeOrderId, 'SUBMITTED', order.rows[0].id]
    );

    // Notify client via WebSocket
    notifyClient(userId, {
      type: 'order_update',
      orderId: order.rows[0].id,
      status: 'SUBMITTED',
      exchangeOrderId,
    });

    return { orderId: order.rows[0].id, status: 'SUBMITTED' };
  } catch (err) {
    // Release blocked margin on failure
    await releaseMargin(userId, requiredMargin);
    await db.query(
      'UPDATE orders SET status = $1 WHERE id = $2',
      ['REJECTED', order.rows[0].id]
    );
    throw err;
  }
}

3. Market Data Pipeline — 5,000 Price Updates Per Second

Stock exchanges push price data via their feed APIs. We need to process, store, and broadcast this data to 100K+ users in real-time.

  MARKET DATA PIPELINE
  ════════════════════

  ┌──────────┐     ┌──────────────┐     ┌──────────┐     ┌──────────────┐
  │   NSE    │────▶│   Exchange   │────▶│  Price   │────▶│    Redis     │
  │ Exchange │     │  Feed Parser │     │ Processor│     │   Pub/Sub    │
  │  Feed    │     │              │     │          │     │              │
  │          │     │ - Binary     │     │ - LTP    │     │ price:INFY   │
  │ 5000     │     │   protocol   │     │ - OHLCV  │     │ price:TCS    │
  │ symbols  │     │ - Decode     │     │ - Change │     │ price:...    │
  │ ~1msg/s  │     │ - Validate   │     │ - Volume │     │              │
  │ each     │     │              │     │ - Depth  │     │ ~5000 msg/s  │
  └──────────┘     └──────────────┘     └──────────┘     └──────┬───────┘
                                                                │
                                                    ┌───────────┼───────────┐
                                                    ▼           ▼           ▼
                                              ┌──────────┐ ┌──────────┐ ┌──────────┐
                                              │  WS      │ │  WS      │ │  WS      │
                                              │ Server 1 │ │ Server 2 │ │ Server 3 │
                                              │ 35K conn │ │ 35K conn │ │ 35K conn │
                                              └──────────┘ └──────────┘ └──────────┘
                                                    │           │           │
                                                    ▼           ▼           ▼
                                                  Clients    Clients    Clients

// price-processor.js — Process exchange feed and publish to Redis
const Redis = require('ioredis');
const redis = new Redis();

class PriceProcessor {
  constructor() {
    this.lastPrices = new Map();
  }

  async processTickData(tick) {
    const {
      symbol,
      ltp,       // Last Traded Price
      open,
      high,
      low,
      close,
      volume,
      timestamp,
    } = tick;

    // Calculate change and % change
    const prevClose = this.lastPrices.get(symbol)?.close || close;
    const change = ltp - prevClose;
    const changePercent = ((change / prevClose) * 100).toFixed(2);

    const priceData = {
      symbol,
      ltp,
      open,
      high,
      low,
      close: prevClose,
      volume,
      change,
      changePercent: parseFloat(changePercent),
      timestamp,
    };

    // Update local cache
    this.lastPrices.set(symbol, priceData);

    // Publish to Redis — all WS servers will receive this
    await redis.publish(
      `price:${symbol}`,
      JSON.stringify(priceData)
    );

    // Store latest price in Redis hash (for API queries)
    await redis.hset(`stock:${symbol}`, {
      ltp: ltp.toString(),
      open: open.toString(),
      high: high.toString(),
      low: low.toString(),
      volume: volume.toString(),
      change: change.toString(),
      changePercent: changePercent,
      updatedAt: timestamp.toString(),
    });
  }
}

4. Database Design

  DATABASE SCHEMA (Simplified)
  ════════════════════════════

  ┌──────────────────┐       ┌──────────────────┐
  │      users       │       │     accounts     │
  ├──────────────────┤       ├──────────────────┤
  │ id          PK   │──┐    │ id          PK   │
  │ email            │  │    │ user_id     FK   │──┐
  │ phone            │  │    │ balance          │  │
  │ pan_number       │  │    │ blocked_margin   │  │
  │ kyc_status       │  │    │ available_margin │  │
  │ created_at       │  └───▶│ updated_at       │  │
  └──────────────────┘       └──────────────────┘  │
                                                    │
  ┌──────────────────┐       ┌──────────────────┐  │
  │      orders      │       │    holdings      │  │
  ├──────────────────┤       ├──────────────────┤  │
  │ id          PK   │       │ id          PK   │  │
  │ user_id     FK   │──┐    │ user_id     FK   │◀─┘
  │ symbol           │  │    │ symbol           │
  │ quantity         │  │    │ quantity         │
  │ price            │  │    │ avg_price        │
  │ type (LIMIT/..)  │  │    │ current_value    │
  │ side (BUY/SELL)  │  │    │ pnl             │
  │ status           │  │    │ updated_at       │
  │ exchange_order_id│  │    └──────────────────┘
  │ executed_price   │  │
  │ executed_at      │  │    ┌──────────────────┐
  │ created_at       │  │    │     trades       │
  └──────────────────┘  │    ├──────────────────┤
                        │    │ id          PK   │
                        └───▶│ order_id    FK   │
                             │ symbol           │
                             │ quantity         │
                             │ price            │
                             │ side             │
                             │ exchange_trade_id│
                             │ executed_at      │
                             └──────────────────┘

  ┌──────────────────────────────────────────────┐
  │         ohlcv_candles (TimescaleDB)          │
  ├──────────────────────────────────────────────┤
  │ time         TIMESTAMPTZ  (hypertable key)   │
  │ symbol       TEXT                             │
  │ open         DECIMAL(12,2)                    │
  │ high         DECIMAL(12,2)                    │
  │ low          DECIMAL(12,2)                    │
  │ close        DECIMAL(12,2)                    │
  │ volume       BIGINT                           │
  │ interval     TEXT  ('1m','5m','15m','1h','1d')│
  └──────────────────────────────────────────────┘

erDiagram
    USERS ||--o{ ACCOUNTS : has
    USERS ||--o{ ORDERS : places
    USERS ||--o{ HOLDINGS : owns
    ORDERS ||--o{ TRADES : generates

    USERS {
        int id PK
        string email
        string phone
        string pan_number
        string kyc_status
    }

    ORDERS {
        int id PK
        int user_id FK
        string symbol
        int quantity
        decimal price
        string type
        string side
        string status
    }

    HOLDINGS {
        int id PK
        int user_id FK
        string symbol
        int quantity
        decimal avg_price
    }

    TRADES {
        int id PK
        int order_id FK
        string symbol
        int quantity
        decimal price
        timestamp executed_at
    }

Why Two Databases?

  PostgreSQL vs TimescaleDB — DECISION MATRIX
  ═══════════════════════════════════════════

  ┌──────────────────┬──────────────────┬──────────────────────┐
  │                  │   PostgreSQL     │    TimescaleDB       │
  ├──────────────────┼──────────────────┼──────────────────────┤
  │ Used for         │ Users, Orders,   │ Price candles,       │
  │                  │ Holdings, Trades │ tick data, charts    │
  ├──────────────────┼──────────────────┼──────────────────────┤
  │ Query pattern    │ Random access    │ Time-range queries   │
  │                  │ by user/order ID │ "RELIANCE, last 30d" │
  ├──────────────────┼──────────────────┼──────────────────────┤
  │ Write pattern    │ Low-medium       │ Very high            │
  │                  │ (orders/trades)  │ (5000 writes/sec)    │
  ├──────────────────┼──────────────────┼──────────────────────┤
  │ Retention        │ Forever          │ 2 years, then        │
  │                  │                  │ aggregate + archive  │
  ├──────────────────┼──────────────────┼──────────────────────┤
  │ Why              │ ACID for money   │ Optimized for        │
  │                  │ transactions     │ time-series inserts  │
  │                  │                  │ + range scans        │
  └──────────────────┴──────────────────┴──────────────────────┘

5. Caching Strategy

We covered caching in detail in my Redis caching post, but here's how it fits into the trading platform specifically:

  WHAT WE CACHE (AND WHAT WE DON'T)
  ══════════════════════════════════

  CACHED (Redis):
  ┌──────────────────────────────────────────────────┐
  │  stock:RELIANCE    → { ltp, ohlcv, volume }     │  TTL: 2s
  │  depth:RELIANCE    → { bids[], asks[] }          │  TTL: 1s
  │  watchlist:user123 → [RELIANCE, INFY, TCS]       │  TTL: 5min
  │  session:abc123    → { userId, permissions }      │  TTL: 30min
  │  portfolio:user123 → { holdings, pnl }            │  TTL: 60s
  │  config:margins    → { symbol: margin% }          │  TTL: 1hr
  └──────────────────────────────────────────────────┘

  NOT CACHED (Direct DB):
  ┌──────────────────────────────────────────────────┐
  │  Order placement    → Must be 100% accurate      │
  │  Account balance    → Real-time for margin calc   │
  │  Trade execution    → Exchange is source of truth │
  │  Fund transfer      → Financial transaction       │
  └──────────────────────────────────────────────────┘

  Rule: If a stale value could cause financial loss,
        don't cache it. Go to the database.

6. Authentication & Security

  AUTH FLOW — JWT + TOTP 2FA
  ══════════════════════════

  ┌────────┐                    ┌──────────┐                 ┌───────┐
  │ Client │                    │   Auth   │                 │ Redis │
  └───┬────┘                    │ Service  │                 └───┬───┘
      │                         └────┬─────┘                     │
      │  1. POST /login              │                           │
      │  { email, password }         │                           │
      │─────────────────────────────▶│                           │
      │                              │  2. Verify password       │
      │                              │     (bcrypt compare)      │
      │                              │                           │
      │  3. Send TOTP code           │                           │
      │◀─────────────────────────────│                           │
      │                              │                           │
      │  4. POST /verify-2fa         │                           │
      │  { code: "482910" }          │                           │
      │─────────────────────────────▶│                           │
      │                              │  5. Verify TOTP           │
      │                              │                           │
      │                              │  6. Create session ──────▶│
      │                              │     SET session:uuid      │
      │                              │     { userId, perms }     │
      │                              │     TTL: 30 min           │
      │  7. Return JWT               │                           │
      │  { accessToken,              │                           │
      │    refreshToken }            │                           │
      │◀─────────────────────────────│                           │
      │                              │                           │
      │  8. Subsequent requests      │                           │
      │  Authorization: Bearer jwt   │                           │
      │─────────────────────────────▶│                           │
      │                              │  9. Verify JWT            │
      │                              │  10. Check session ──────▶│
      │                              │      in Redis             │
      │                              │◀──────────────────────────│
      │  11. Response                │                           │
      │◀─────────────────────────────│                           │

Handling Failure Scenarios

In production, things break. Here's how we designed for the common failure modes:

  FAILURE SCENARIOS & MITIGATIONS
  ═══════════════════════════════

  ┌─────────────────────────────────────────────────────────────┐
  │ SCENARIO              │ IMPACT        │ MITIGATION          │
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ Redis goes down       │ Slower reads  │ Fall back to DB     │
  │                       │               │ Serve stale prices  │
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ WS server crashes     │ Users         │ Client auto-        │
  │                       │ disconnected  │ reconnect with      │
  │                       │               │ exponential backoff │
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ Exchange feed drops   │ Stale prices  │ Show "last updated" │
  │                       │               │ timestamp, warn user│
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ Order rejected by     │ Order fails   │ Release margin,     │
  │ exchange              │               │ notify user, retry  │
  │                       │               │ if transient        │
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ DB write fails during │ Inconsistent  │ Use DB transaction  │
  │ order execution       │ state         │ + idempotency key   │
  ├───────────────────────┼───────────────┼─────────────────────┤
  │ Network partition     │ Split brain   │ Prefer consistency  │
  │ between services      │               │ (reject orders      │
  │                       │               │ rather than risk     │
  │                       │               │ double execution)   │
  └───────────────────────┴───────────────┴─────────────────────┘

Client Reconnection Strategy

// client-reconnect.js — WebSocket reconnection with exponential backoff
class TradingWebSocket {
  constructor(url) {
    this.url = url;
    this.retryCount = 0;
    this.maxRetries = 10;
    this.subscriptions = new Set();
    this.connect();
  }

  connect() {
    this.ws = new WebSocket(this.url);

    this.ws.onopen = () => {
      console.log('Connected to trading server');
      this.retryCount = 0;

      // Re-subscribe to all symbols after reconnection
      if (this.subscriptions.size > 0) {
        this.ws.send(JSON.stringify({
          type: 'subscribe',
          symbols: [...this.subscriptions],
        }));
      }
    };

    this.ws.onclose = () => {
      if (this.retryCount < this.maxRetries) {
        // Exponential backoff: 1s, 2s, 4s, 8s, 16s... capped at 30s
        const delay = Math.min(1000 * Math.pow(2, this.retryCount), 30000);
        console.log(`Reconnecting in ${delay}ms (attempt ${this.retryCount + 1})`);
        this.retryCount++;
        setTimeout(() => this.connect(), delay);
      } else {
        console.error('Max reconnection attempts reached');
      }
    };

    this.ws.onmessage = (event) => {
      const data = JSON.parse(event.data);
      this.handleMessage(data);
    };
  }

  subscribe(symbols) {
    symbols.forEach((s) => this.subscriptions.add(s));
    if (this.ws.readyState === WebSocket.OPEN) {
      this.ws.send(JSON.stringify({ type: 'subscribe', symbols }));
    }
  }

  handleMessage(data) {
    switch (data.type) {
      case 'price':
        this.onPriceUpdate?.(data);
        break;
      case 'order_update':
        this.onOrderUpdate?.(data);
        break;
      case 'alert':
        this.onAlert?.(data);
        break;
    }
  }
}

Scaling Considerations

  SCALING FROM 10K TO 100K+ USERS
  ════════════════════════════════

  10K Users (Single Instance)
  ┌──────────┐  ┌──────────┐  ┌──────────┐
  │ 1 Node.js│  │ 1 Redis  │  │ 1 PgSQL  │
  │ instance │  │ instance │  │ instance │
  └──────────┘  └──────────┘  └──────────┘

  50K Users (Horizontal + Read Replicas)
  ┌──────────┐  ┌──────────┐  ┌──────────┐
  │ 3 Node.js│  │ Redis    │  │ PgSQL    │
  │ instances│  │ Sentinel │  │ Primary  │
  │ + LB     │  │ (HA)     │  │ + 2 Read │
  └──────────┘  └──────────┘  │ Replicas │
                               └──────────┘

  100K+ Users (Full Cluster)
  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐
  │ 5+ Node  │  │ Redis    │  │ PgSQL    │  │Timescale │
  │ instances│  │ Cluster  │  │ Primary  │  │ DB for   │
  │ + ALB    │  │ (6 nodes │  │ + Read   │  │ candles  │
  │ + Auto-  │  │  3M+3S)  │  │ Replicas │  │ + ticks  │
  │  scale   │  │          │  │ + PgPool │  │          │
  └──────────┘  └──────────┘  └──────────┘  └──────────┘

  Key metrics to monitor:
  ├── WebSocket connections per server (target: <40K)
  ├── Redis memory usage and hit rate (target: >90%)
  ├── DB connection pool utilization (target: <80%)
  ├── Order processing latency P99 (target: <100ms)
  └── Price broadcast latency (target: <50ms from exchange)

Common Mistakes in Trading System Design

1. Using REST for Real-Time Data

Polling every second for 5,000 symbols = 5,000 HTTP requests/second per user. Use WebSockets.

2. Not Separating Read and Write Databases

Order writes and portfolio reads have very different patterns. Use read replicas for dashboards, primary for order execution.

3. Caching Account Balances

A stale balance means a user might place an order they can't afford. Always read balance from the primary database.

4. No Circuit Breaker for Exchange Connection

If the exchange gateway is slow, your order queue backs up. Use a circuit breaker pattern to fail fast and inform users.

5. Synchronous Order Processing

Order validation, margin check, and exchange submission can be pipelined. Validate first, then submit asynchronously and notify via WebSocket.

Key Takeaways

WebSocket + Redis Pub/Sub is the backbone for real-time price distribution across multiple servers
Separate your services: order execution, market data, portfolio, and auth have very different scaling and consistency requirements
Cache aggressively but carefully: stock prices (2s TTL) yes, account balances (never) no
Design for failure: exchanges go down, connections drop, databases lag — graceful degradation beats crashing
Use the right database for the job: PostgreSQL for transactional data, TimescaleDB for time-series, Redis for hot data
Horizontal scaling via sticky-session load balancing and Redis as the broadcast bus
Latency budget: know where every millisecond goes in your order flow

Connect with Me

If you found this useful, I write deep-dive system design posts and visual explainers every week. Follow along:

Twitter/X: @robinsingh — threads on system design, Node.js, and AI engineering
LinkedIn: Robin Singh — longer-form posts and career insights
GitHub: robins163 — code from all my posts
Hashnode: unknowntoplay.hashnode.dev — full blog with diagrams and animations

Next post: "The Node.js Event Loop — Finally Explained Right" — where I break down the event loop with animated diagrams and show exactly why setTimeout(fn, 0) doesn't mean what you think it does.

Drop a comment if you have questions about any part of this architecture — happy to go deeper on any component.

How Redis Caching Actually Works — A Visual Guide for Backend Developers

Robins163 — Fri, 10 Apr 2026 08:59:01 +0000

TL;DR: Redis caching sits between your application and database, storing frequently accessed data in memory for sub-millisecond reads. This guide walks through cache-aside, write-through, and write-behind patterns with diagrams, working Node.js code, and real-world lessons from building a stock trading platform handling 100K+ concurrent users.

The Problem

Your API is slow. Users are waiting 200-500ms for responses that hit the database every single time. Your PostgreSQL instance is sweating under load, and your cloud bill is climbing.

I hit this exact wall at Mstock — a stock trading platform with 100K+ concurrent users. Every user's dashboard needed real-time portfolio data, watchlist prices, and market depth. Hitting the database for every request? That's a recipe for a crashed production server during market hours.

The answer was Redis. Not as a silver bullet, but as a carefully designed caching layer. Here's how it actually works.

What Is Redis Caching?

Redis is an in-memory data store. "In-memory" is the key phrase — while your database reads from disk (even SSDs have latency), Redis reads from RAM.

┌─────────────────────────────────────────────────────────┐
│                    Response Times                        │
├──────────────────────┬──────────────────────────────────┤
│  RAM (Redis)         │  ~0.1 ms                         │
│  SSD (PostgreSQL)    │  ~1-10 ms                        │
│  Network DB call     │  ~10-100 ms                      │
│  Spinning Disk       │  ~10-20 ms                       │
└──────────────────────┴──────────────────────────────────┘

That's a 10x-1000x difference. When you're serving thousands of requests per second, this matters.

How Caching Fits in Your Architecture

Here's the big picture:

                    ┌──────────────┐
                    │    Client    │
                    │  (Browser)   │
                    └──────┬───────┘
                           │
                           ▼
                    ┌──────────────┐
                    │   API Server │
                    │  (Node.js)   │
                    └──────┬───────┘
                           │
              ┌────────────┼────────────┐
              │            │            │
              ▼            │            │
      ┌──────────────┐    │    ┌──────────────┐
      │    Redis      │    │    │   Database   │
      │   (Cache)     │    │    │ (PostgreSQL) │
      │               │    │    │              │
      │  ⚡ ~0.1ms    │    │    │  🐢 ~10ms   │
      └──────────────┘    │    └──────────────┘
              ▲            │            ▲
              │            │            │
              └────────────┴────────────┘
                    Read / Write

The API server checks Redis first. If the data is there (a cache hit), it returns instantly. If not (a cache miss), it queries the database, stores the result in Redis, and then returns.

Cache Hit vs Cache Miss — The Core Flow

This is the fundamental pattern. Every caching strategy is a variation of this:

flowchart TD
    A[Client Request] --> B{Check Redis Cache}
    B -->|Cache HIT| C[Return Cached Data]
    B -->|Cache MISS| D[Query Database]
    D --> E[Store Result in Redis]
    E --> F[Set TTL / Expiry]
    F --> G[Return Fresh Data]

    style C fill:#22c55e,color:#fff
    style D fill:#ef4444,color:#fff
    style E fill:#3b82f6,color:#fff

  Cache HIT Flow (fast path):                Cache MISS Flow (slow path):

  Client ──▶ Redis ──▶ Client               Client ──▶ Redis ──▶ DB
             ✅ Found!     ⚡ 0.5ms                     ❌ Not found  │
                                                                      ▼
                                              Client ◀── Redis ◀── DB
                                                        📝 Store    🐢 15ms

Working Code: Basic Cache-Aside in Node.js

// cache-aside.js — The most common caching pattern
const Redis = require('ioredis');
const { Pool } = require('pg');

const redis = new Redis({
  host: '127.0.0.1',
  port: 6379,
  maxRetriesPerRequest: 3,
});

const db = new Pool({
  connectionString: process.env.DATABASE_URL,
});

async function getUserPortfolio(userId) {
  const cacheKey = `portfolio:${userId}`;

  // Step 1: Check Redis first
  const cached = await redis.get(cacheKey);
  if (cached) {
    console.log('Cache HIT for', cacheKey);
    return JSON.parse(cached);
  }

  // Step 2: Cache MISS — query the database
  console.log('Cache MISS for', cacheKey);
  const result = await db.query(
    'SELECT * FROM portfolios WHERE user_id = $1',
    [userId]
  );

  const portfolio = result.rows[0];

  // Step 3: Store in Redis with 60-second TTL
  await redis.setex(cacheKey, 60, JSON.stringify(portfolio));

  return portfolio;
}

This is called the Cache-Aside pattern (also called "Lazy Loading"). The application manages the cache explicitly — it decides when to read and write to the cache.

The Three Caching Strategies You Need to Know

1. Cache-Aside (Lazy Loading)

The pattern shown above. The application is responsible for all cache operations.

  CACHE-ASIDE PATTERN
  ═══════════════════

  READ:
  ┌─────┐    ①check    ┌───────┐
  │ App │──────────────▶│ Redis │
  │     │◀──────────────│       │
  └──┬──┘    hit/miss   └───────┘
     │
     │ ②query (on miss only)
     ▼
  ┌──────┐
  │  DB  │
  └──┬───┘
     │
     │ ③store result in cache
     ▼
  ┌───────┐
  │ Redis │
  └───────┘

  WRITE:
  ┌─────┐   ①write     ┌──────┐
  │ App │──────────────▶│  DB  │
  │     │               └──────┘
  │     │   ②invalidate ┌───────┐
  │     │──────────────▶│ Redis │ DEL key
  └─────┘               └───────┘

When to use: Read-heavy workloads. Most web applications. This is your default choice.

Pros: Only caches data that's actually requested. Cache failures don't break the app.

Cons: First request is always slow (cache miss). Stale data possible between write and TTL expiry.

2. Write-Through

Every write goes to the cache AND the database simultaneously.

  WRITE-THROUGH PATTERN
  ═════════════════════

  WRITE:
  ┌─────┐              ┌───────┐              ┌──────┐
  │ App │──── ①write ──▶│ Redis │──── ②write ──▶│  DB  │
  └─────┘              └───────┘              └──────┘

  READ:
  ┌─────┐              ┌───────┐
  │ App │──── ①read ───▶│ Redis │  (always fresh!)
  └─────┘              └───────┘

sequenceDiagram
    participant App
    participant Redis
    participant DB

    App->>Redis: SET user:123 data
    Redis->>DB: INSERT/UPDATE user 123
    DB-->>Redis: OK
    Redis-->>App: OK

    Note over App,DB: Later, on read...
    App->>Redis: GET user:123
    Redis-->>App: data (always fresh)

When to use: When data consistency is critical and you can tolerate slightly slower writes.

// write-through.js
async function updatePortfolio(userId, data) {
  const cacheKey = `portfolio:${userId}`;

  // Write to both cache and DB
  const pipeline = redis.pipeline();
  pipeline.setex(cacheKey, 300, JSON.stringify(data));

  await Promise.all([
    pipeline.exec(),
    db.query(
      'UPDATE portfolios SET data = $1 WHERE user_id = $2',
      [JSON.stringify(data), userId]
    ),
  ]);
}

Pros: Cache is always consistent with DB. Reads are always fast.

Cons: Write latency increases (writing to two places). Cache may fill with data that's never read.

3. Write-Behind (Write-Back)

Writes go to the cache immediately, and the database is updated asynchronously in the background.

  WRITE-BEHIND PATTERN
  ════════════════════

  WRITE:
  ┌─────┐   ①write    ┌───────┐         ┌──────┐
  │ App │─────────────▶│ Redis │ ──③──── │  DB  │
  └─────┘   (instant)  └───────┘  async  └──────┘
                           │      batch
                           │      write
                        ②queue
                        ┌───────┐
                        │ Queue │
                        └───────┘

  Timeline:
  ──────────────────────────────────────────────▶ time
  │ App writes │         │ DB updated │
  │ to Redis   │         │ (later)    │
  │ ⚡ instant │         │ 🐢 async   │

// write-behind.js — Using Bull queue for async DB writes
const Queue = require('bull');
const writeQueue = new Queue('db-writes', 'redis://127.0.0.1:6379');

async function updateWatchlist(userId, stocks) {
  const cacheKey = `watchlist:${userId}`;

  // Step 1: Write to Redis immediately (user sees instant response)
  await redis.setex(cacheKey, 600, JSON.stringify(stocks));

  // Step 2: Queue the database write (happens in background)
  await writeQueue.add('update-watchlist', {
    userId,
    stocks,
    timestamp: Date.now(),
  });
}

// Background worker processes the queue
writeQueue.process('update-watchlist', async (job) => {
  const { userId, stocks } = job.data;
  await db.query(
    'UPDATE watchlists SET stocks = $1 WHERE user_id = $2',
    [JSON.stringify(stocks), userId]
  );
});

When to use: Write-heavy workloads where you can tolerate eventual consistency. Gaming leaderboards, analytics counters, activity feeds.

Pros: Fastest writes (only hitting RAM). Can batch DB writes for efficiency.

Cons: Risk of data loss if Redis crashes before DB write. More complex architecture.

Strategy Comparison

┌─────────────────┬───────────────┬────────────────┬───────────────┐
│                 │  Cache-Aside  │ Write-Through  │ Write-Behind  │
├─────────────────┼───────────────┼────────────────┼───────────────┤
│ Read Speed      │  ⚡ Fast*     │  ⚡ Fast       │  ⚡ Fast      │
│ Write Speed     │  Normal       │  🐢 Slower    │  ⚡ Fastest   │
│ Consistency     │  Eventual     │  ✅ Strong    │  ❌ Eventual  │
│ Data Loss Risk  │  None         │  None          │  ⚠️  Possible │
│ Complexity      │  Low          │  Medium        │  High         │
│ Best For        │  Read-heavy   │  Read+Write    │  Write-heavy  │
├─────────────────┼───────────────┼────────────────┼───────────────┤
│ * After first   │               │                │               │
│   cache miss    │               │                │               │
└─────────────────┴───────────────┴────────────────┴───────────────┘

TTL Strategies — When Should Cache Expire?

TTL (Time-To-Live) is how long data stays in the cache before Redis automatically deletes it. Getting this right is critical.

flowchart LR
    A[Data Written to Cache] --> B[TTL Timer Starts]
    B --> C{TTL Expired?}
    C -->|No| D[Serve from Cache]
    C -->|Yes| E[Key Deleted]
    E --> F[Next Request = Cache Miss]
    F --> G[Fetch from DB & Re-cache]

    style D fill:#22c55e,color:#fff
    style E fill:#ef4444,color:#fff

TTL Decision Guide

  How to Choose TTL
  ═════════════════

  Changes every second? (stock prices, live scores)
  └──▶ TTL: 1-5 seconds  OR  don't cache at all

  Changes every few minutes? (user sessions, dashboards)
  └──▶ TTL: 30-300 seconds

  Changes rarely? (user profiles, product catalog)
  └──▶ TTL: 1-24 hours

  Never changes? (country lists, configs)
  └──▶ TTL: 24+ hours  OR  no TTL (manual invalidation)

// ttl-strategies.js — Different TTL for different data types
const TTL = {
  STOCK_PRICE: 2,         // 2 seconds — changes constantly
  MARKET_DEPTH: 5,        // 5 seconds — near real-time
  USER_SESSION: 1800,     // 30 minutes
  PORTFOLIO: 60,          // 1 minute — balance between freshness and speed
  USER_PROFILE: 3600,     // 1 hour — rarely changes
  STATIC_CONFIG: 86400,   // 24 hours
};

async function cacheWithTTL(key, fetchFn, ttl) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const data = await fetchFn();
  await redis.setex(key, ttl, JSON.stringify(data));
  return data;
}

// Usage
const portfolio = await cacheWithTTL(
  `portfolio:${userId}`,
  () => db.query('SELECT * FROM portfolios WHERE user_id = $1', [userId]).then(r => r.rows[0]),
  TTL.PORTFOLIO
);

Cache Invalidation — The Hard Part

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

When the underlying data changes, you need to remove or update the cached version. Here are the strategies:

Strategy 1: Delete on Write (Most Common)

// invalidation.js — Delete cache when data changes
async function updateUserProfile(userId, newData) {
  // Update the database
  await db.query(
    'UPDATE users SET name = $1, email = $2 WHERE id = $3',
    [newData.name, newData.email, userId]
  );

  // Delete the cached version — next read will re-populate
  await redis.del(`user:${userId}`);
}

Strategy 2: Pattern-Based Invalidation

When one change affects multiple cache keys:

// When a user changes their profile, invalidate all related caches
async function invalidateUserCaches(userId) {
  // Find all keys matching the pattern
  const keys = await redis.keys(`user:${userId}:*`);

  if (keys.length > 0) {
    await redis.del(...keys);
  }
}

Warning: redis.keys() scans the entire keyspace and blocks Redis. Use SCAN in production:

// Production-safe pattern invalidation using SCAN
async function safeInvalidate(pattern) {
  let cursor = '0';
  do {
    const [nextCursor, keys] = await redis.scan(
      cursor, 'MATCH', pattern, 'COUNT', 100
    );
    cursor = nextCursor;
    if (keys.length > 0) {
      await redis.del(...keys);
    }
  } while (cursor !== '0');
}

Strategy 3: Pub/Sub Invalidation (Multi-Server)

When you have multiple app servers, each with their own connection to Redis:

  PUB/SUB INVALIDATION
  ════════════════════

  ┌──────────┐  publish   ┌──────────┐
  │ Server 1 │───────────▶│  Redis   │
  │ (writer) │            │ Channel: │
  └──────────┘            │ "cache-  │
                          │ invalid" │
                          └────┬─────┘
                     subscribe │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        ┌──────────┐    ┌──────────┐    ┌──────────┐
        │ Server 1 │    │ Server 2 │    │ Server 3 │
        │ DEL key  │    │ DEL key  │    │ DEL key  │
        └──────────┘    └──────────┘    └──────────┘

Real-World Example: Caching at Mstock (Stock Trading Platform)

At Mstock, we had 100K+ concurrent users during market hours (9:15 AM - 3:30 PM IST). Here's how we structured our caching:

The Architecture

  MSTOCK CACHING ARCHITECTURE
  ════════════════════════════

  ┌──────────────────────────────────────────────────┐
  │                  Client Layer                     │
  │  100K+ WebSocket connections                      │
  └──────────────────────┬───────────────────────────┘
                         │
                         ▼
  ┌──────────────────────────────────────────────────┐
  │              API Gateway (Nginx)                  │
  │         Rate limiting, load balancing             │
  └──────────────────────┬───────────────────────────┘
                         │
            ┌────────────┼────────────┐
            ▼            ▼            ▼
  ┌────────────┐ ┌────────────┐ ┌────────────┐
  │  Node.js   │ │  Node.js   │ │  Node.js   │
  │ Instance 1 │ │ Instance 2 │ │ Instance 3 │
  └──────┬─────┘ └──────┬─────┘ └──────┬─────┘
         │              │              │
         └──────────────┼──────────────┘
                        │
           ┌────────────┼────────────┐
           ▼                         ▼
  ┌─────────────────┐      ┌─────────────────┐
  │   Redis Cluster  │      │   PostgreSQL    │
  │                  │      │    (Primary)    │
  │  ┌────┐ ┌────┐  │      │                 │
  │  │ M1 │ │ M2 │  │      │  Portfolios     │
  │  └────┘ └────┘  │      │  Orders         │
  │  ┌────┐ ┌────┐  │      │  Users          │
  │  │ M3 │ │ M4 │  │      │                 │
  │  └────┘ └────┘  │      └─────────────────┘
  │                  │
  │  Hot data:       │
  │  - Stock prices  │
  │  - Watchlists    │
  │  - User sessions │
  │  - Market depth  │
  └─────────────────┘

What We Cached (and What We Didn't)

// mstock-cache-config.js — Real caching strategy

const CacheConfig = {
  // CACHED: High-read, tolerates slight staleness
  stockPrice: {
    pattern: 'stock:{symbol}:price',
    ttl: 2,  // 2 seconds — exchange pushes updates every ~1s
    strategy: 'write-through',
    example: 'stock:RELIANCE:price',
  },

  watchlist: {
    pattern: 'watchlist:{userId}',
    ttl: 300,  // 5 minutes
    strategy: 'cache-aside',
    invalidateOn: 'watchlist-update',
  },

  userSession: {
    pattern: 'session:{sessionId}',
    ttl: 1800,  // 30 minutes
    strategy: 'write-through',
  },

  marketDepth: {
    pattern: 'depth:{symbol}',
    ttl: 1,  // 1 second — must be near real-time
    strategy: 'write-through',
  },

  // NOT CACHED: Needs to be 100% accurate, no tolerance for stale data
  // - Order execution (goes directly to exchange)
  // - Account balance (must be real-time for margin checks)
  // - Trade history (read from DB, low frequency)
};

The Results

  BEFORE REDIS                          AFTER REDIS
  ════════════                          ═══════════

  Avg response time: 180ms              Avg response time: 12ms
  DB queries/sec:    15,000             DB queries/sec:    2,000
  P99 latency:      800ms              P99 latency:       45ms
  DB CPU usage:     85%                DB CPU usage:      25%

  ┌─────────────────────────────────────────────────┐
  │  Response Time (ms)                              │
  │                                                  │
  │  200 ┤ ████                                      │
  │  180 ┤ ████                                      │
  │  160 ┤ ████                                      │
  │  140 ┤ ████                                      │
  │  120 ┤ ████                                      │
  │  100 ┤ ████                                      │
  │   80 ┤ ████                                      │
  │   60 ┤ ████                                      │
  │   40 ┤ ████                                      │
  │   20 ┤ ████ ░░░░                                 │
  │    0 ┤ ████ ░░░░                                 │
  │      └──────────                                 │
  │       Before After                               │
  │       ████   ░░░░                                │
  └─────────────────────────────────────────────────┘

  Cache hit rate: 94% during market hours
  That's 94 out of 100 requests served from memory.

Common Mistakes

1. Caching Everything

Not everything needs caching. If data is rarely read or changes constantly, caching adds complexity without benefit.

2. No TTL (Keys Live Forever)

// BAD — key lives forever, stale data accumulates
await redis.set('user:123', JSON.stringify(data));

// GOOD — always set a TTL
await redis.setex('user:123', 3600, JSON.stringify(data));

3. Cache Stampede

When a popular key expires, hundreds of requests simultaneously hit the database.

  CACHE STAMPEDE
  ═════════════

  Popular key expires at T=0

  T=0.001s: Request 1 → Cache MISS → Query DB
  T=0.002s: Request 2 → Cache MISS → Query DB
  T=0.003s: Request 3 → Cache MISS → Query DB
  ...
  T=0.050s: Request 200 → Cache MISS → Query DB

  200 identical queries hit the database simultaneously!
  ──▶ DB CPU spikes ──▶ Cascading failures

Fix: Mutex lock (only one request rebuilds the cache)

// cache-stampede-fix.js
async function getWithMutex(key, fetchFn, ttl) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const lockKey = `lock:${key}`;
  // Try to acquire lock (NX = only if not exists, EX = 5 second timeout)
  const acquired = await redis.set(lockKey, '1', 'NX', 'EX', 5);

  if (acquired) {
    // This request rebuilds the cache
    const data = await fetchFn();
    await redis.setex(key, ttl, JSON.stringify(data));
    await redis.del(lockKey);
    return data;
  } else {
    // Another request is rebuilding — wait and retry
    await new Promise((resolve) => setTimeout(resolve, 100));
    return getWithMutex(key, fetchFn, ttl);
  }
}

4. Not Handling Redis Failures Gracefully

Redis going down shouldn't crash your app. Fall back to the database:

async function resilientGet(key, fetchFn, ttl) {
  try {
    const cached = await redis.get(key);
    if (cached) return JSON.parse(cached);
  } catch (err) {
    console.error('Redis error, falling back to DB:', err.message);
    // Don't throw — fall through to database
  }

  const data = await fetchFn();

  try {
    await redis.setex(key, ttl, JSON.stringify(data));
  } catch (err) {
    console.error('Redis write failed:', err.message);
    // Data is still returned from DB — cache will recover
  }

  return data;
}

5. Serialization Overhead

Storing large JSON blobs in Redis negates the speed benefit.

// BAD — caching the entire user object with all relations
await redis.set('user:123', JSON.stringify({
  ...user,
  orders: [...thousandsOfOrders],
  activityLog: [...yearsOfActivity],
}));

// GOOD — cache only what's needed for the hot path
await redis.set('user:123:summary', JSON.stringify({
  id: user.id,
  name: user.name,
  portfolioValue: user.portfolioValue,
}));

When to Use Redis Caching / When Not to Use It

  USE REDIS CACHING WHEN:                    DON'T USE WHEN:
  ════════════════════════                    ════════════════

  ✅ Read-heavy workloads (>80% reads)       ❌ Write-heavy with strong
  ✅ Data can tolerate slight staleness          consistency requirements
  ✅ Same data requested repeatedly           ❌ Data is unique per request
  ✅ Database is a bottleneck                 ❌ Data changes every second
  ✅ Response time is critical                ❌ Payload is very large (>1MB)
  ✅ You can handle cache invalidation        ❌ You can't afford the
                                                 operational overhead

Key Takeaways

Cache-Aside is your default strategy — simple, effective, handles failures gracefully
Write-Through when you need consistency — slower writes but cache is always fresh
Write-Behind for write-heavy workloads — fastest writes but risk of data loss
Always set a TTL — never let keys live forever
Guard against cache stampedes — use mutex locks for popular keys
Don't cache everything — only cache data that's read frequently and can tolerate staleness
Handle Redis failures — your app should work (slower) without the cache
Monitor your hit rate — if it's below 80%, rethink your strategy

Connect with Me

If you found this useful, I write deep-dive visual guides on system design, Node.js, and AI engineering — 2 posts every week (Monday + Thursday).

Twitter/X: @robinsingh — daily tips, threads, and quick takes
LinkedIn: Robin Singh — long-form posts and career updates
GitHub: robins163 — source code from all my articles
Hashnode: unknowntoplay.hashnode.dev — canonical home for all my posts
Substack: Newsletter — subscribe for weekly system design breakdowns delivered to your inbox

This post is part of the **System Design Visual Guides* series. Next up: WebSocket vs SSE vs Long Polling — the complete visual comparison.*

[Boost]

Robins163 — Tue, 22 Apr 2025 02:45:59 +0000

Soma

Apr 19 '25

The Software Developer Interview Prep Roadmap (With Resources)

#systemdesign #coding #programming #softwareengineering

139

Comments 12

6 min read