Forem: shubham khatik

Escaping the Buffer: The Advanced Guide to Streams in Node.js & Next.js

shubham khatik — Sat, 14 Mar 2026 12:17:40 +0000

Escaping the Buffer: The Advanced Guide to Streams in Node.js & Next.js

If you have been writing JavaScript for a few years, you are probably intimately familiar with async/await. It makes asynchronous code look synchronous, keeps the event loop unblocked, and is universally loved.

But async/await hides a massive system design bottleneck: Buffering.

If you are building data-intensive applications — proxying AWS S3 uploads, generating massive CSVs from MongoDB, or rendering complex Next.js Server Components — relying solely on buffered data will inevitably lead to Out of Memory (OOM) crashes and abysmal Time to First Byte (TTFB).

Let's break down the real-world problems buffering causes, understand the mental model of Streams, and look at how to properly architect streaming solutions across Node.js, Next.js, and databases.

1. The Real Problem: Buffering vs. Streaming

When you use standard await to read a file or fetch an API, the underlying engine loads 100% of that data into RAM before your code executes the next line.

The Backend Failure Mode (OOM Crashes)

Imagine an Express route that downloads a 2GB video file for a user:

// 🚨 THE WRONG WAY: Buffering
app.get('/download', async (req, res) => {
  // Node reads the ENTIRE 2GB file into RAM right now.
  const videoData = await fs.promises.readFile('./massive-video.mp4');
  res.send(videoData);
});

If your AWS EC2 instance has 1GB of RAM, this code instantly crashes your Node process with Fatal Error: heap out of memory. If you have 8GB of RAM, it only takes four concurrent users to kill your server.

The Frontend Failure Mode (High TTFB)

If your React frontend fetches a massive JSON payload, the browser downloads the entire payload, holds it in memory, and waits for the connection to close before parsing it. The user stares at a blank screen or a spinner until the very last byte arrives.

The intended outcome of Streams is to fix this by breaking data into manageable "chunks." You process chunk 1 while chunk 2 is downloading — keeping your server's memory footprint flat and delivering data to the frontend instantly.

2. The Mental Model: Two Colliding Worlds

The most confusing part of modern JavaScript architecture is that there are two entirely different Stream APIs. With the rise of the Next.js App Router and Edge computing, these two worlds are crashing into each other.

Node.js Streams (node:stream) — The backend heavyweights. These belong strictly to the Node runtime. They use .pipe() and .pipeline(). You use these for filesystem operations, heavy local data processing, and traditional Express req/res objects.
The Web Streams API (ReadableStream) — The modern standard. Originally built for the browser (like the fetch API response body), but now natively used by Next.js Edge Functions, Route Handlers, and Cloudflare Workers. They use .pipeTo() and .pipeThrough().

⚠️ Mixing these up is the number one cause of bugs when migrating an Express app to a Next.js App Router API.

3. Practical Workflows & Real-World Execution

Let's look at how Streams are actually used in real environments, evaluating the constraints and correct implementations.

Scenario A: Massive Database Exports (MongoDB / PostgreSQL)

The Problem: You need to export 1 million user records to a CSV file.

The Constraint: Running await User.find({}) will load 1 million objects into your Node server's RAM, crashing it instantly.

The Solution (Node Streams):

Both MongoDB and PostgreSQL offer native stream cursors. Instead of fetching an array, you stream documents one by one, format them, and pipe them directly to the client.

// Express.js + MongoDB Example
import { pipeline } from 'node:stream/promises';
import { Transform } from 'node:stream';

app.get('/export-users', async (req, res) => {
  res.setHeader('Content-Type', 'text/csv');
  res.setHeader('Content-Disposition', 'attachment; filename="users.csv"');

  // 1. Create a database read stream
  const cursorStream = User.find().cursor();

  // 2. Transform each JSON document into a CSV row on the fly
  const toCsvTransform = new Transform({
    objectMode: true,
    transform(doc, encoding, callback) {
      const csvLine = `${doc.name},${doc.email}\n`;
      callback(null, csvLine);
    }
  });

  // 3. pipeline() handles backpressure and cleans up memory if the user disconnects
  try {
    await pipeline(cursorStream, toCsvTransform, res);
  } catch (err) {
    console.error('Stream pipeline failed', err);
  }
});

Scenario B: AI Text Generation (The Web Streams API)

The Problem: LLMs take several seconds to generate a response. Waiting for the full string ruins the UX.

The Solution (Web Streams): Read the ReadableStream from the fetch API chunk-by-chunk to create a real-time "typing" effect.

// React Client Component
const handleAskAI = async () => {
  const response = await fetch('/api/chat');

  // Get the Web Stream reader
  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    // Decode the raw bytes into text and update React state immediately
    const chunkText = decoder.decode(value);
    setChatText((prev) => prev + chunkText);
  }
};

Scenario C: Next.js App Router (Bridging the Gap)

The Problem: You want to stream a large file from your server disk to the user, but Next.js Route Handlers expect a Web ReadableStream as the response — not a Node fs.ReadStream.

The Solution: Convert the Node Stream to a Web Stream. In modern Node (v16+), there is a built-in utility for this.

// src/app/api/download/route.ts
import { createReadStream } from 'node:fs';
import { Readable } from 'node:stream';

export async function GET() {
  const nodeStream = createReadStream('./heavy-asset.zip');

  // Convert Node.js Stream → Web Streams API ReadableStream
  const webStream = Readable.toWeb(nodeStream);

  return new Response(webStream, {
    headers: {
      'Content-Type': 'application/zip',
      'Content-Disposition': 'attachment; filename="asset.zip"',
    },
  });
}

4. When NOT to Use Streams (Failure Modes)

Streams add massive architectural complexity. Error handling is notoriously difficult because a stream can fail halfway through — for example, the user loses internet connection while downloading.

If you are fetching a standard JSON list of 50 items or doing simple CRUD operations, do not use streams. Stick to standard async/await and buffering.

✅ Use streams strictly as an accelerator when you hit the physical limits of your server's RAM, or when network latency demands immediate partial rendering.

5. The "Halfway" Failure Mode: Preventing Memory Leaks

Streams are not atomic. A 1GB download might fail at 500MB because the user closed their laptop, the browser tab crashed, or the Wi-Fi dropped.

If you do not handle this properly, the source stream (like a database cursor or a file read stream) will stay open forever, waiting to send the rest of the data. This creates a silent memory leak that will eventually take down your Node.js instance.

Here is how to handle halfway failures across different parts of the stack.

1. Node.js Backend: Never Use Raw `.pipe()`

In older Express tutorials, you will constantly see this:

// 🚨 DANGEROUS: If 'res' closes early, 'fileStream' stays open forever.
const fileStream = fs.createReadStream('./massive.mp4');
fileStream.pipe(res);

The Fix: Always use stream.pipeline (specifically the Promise-based version). If the user disconnects or the network fails, pipeline automatically sends a destroy signal to every stream in the chain, safely freeing up your server's RAM.

import { pipeline } from 'node:stream/promises';
import fs from 'node:fs';

app.get('/download', async (req, res) => {
  const fileStream = fs.createReadStream('./massive.mp4');

  try {
    // ✅ SAFE: pipeline monitors the connection.
    // If the user drops, it destroys fileStream automatically.
    await pipeline(fileStream, res);
  } catch (err) {
    if (err.code === 'ERR_STREAM_PREMATURE_CLOSE') {
      console.warn('User canceled the download halfway through.');
    } else {
      console.error('Pipeline failed:', err);
    }
  }
});

2. Next.js App Router: Catching the `AbortSignal`

In Next.js 14+ Route Handlers, the user's browser connection is tied to the standard Web Request object.

If a user navigates away while your API is streaming a heavy database query, you need to listen to req.signal — an AbortSignal that fires the moment the client drops.

// src/app/api/heavy-export/route.ts
export async function GET(req: Request) {
  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      try {
        for (let i = 0; i < 10000; i++) {
          // 🛑 CRITICAL CHECK: Did the user close the browser tab?
          if (req.signal.aborted) {
            console.log('Client disconnected. Halting DB query.');
            break;
          }

          // Simulate heavy DB fetch
          const data = await fetchNextDatabaseRow(i);
          controller.enqueue(encoder.encode(data + '\n'));
        }
        controller.close();
      } catch (error) {
        controller.error(error);
      }
    }
  });

  return new Response(stream);
}

3. React Frontend: Canceling the Stream Reader

If you are consuming a stream in the browser (like the AI "typing" effect) and the user clicks a "Stop Generating" button or navigates away, you must actively cancel the reader.

If you unmount the component without canceling, the browser will keep downloading chunks in the background — wasting the user's bandwidth.

'use client';
import { useEffect, useRef } from 'react';

export function AIChat() {
  const readerRef = useRef(null);

  const startStream = async () => {
    const res = await fetch('/api/ai');
    readerRef.current = res.body.getReader();

    // ... loop through reader.read() ...
  };

  // 🧹 Cleanup: runs when the component unmounts
  useEffect(() => {
    return () => {
      if (readerRef.current) {
        // Instantly kills the active download stream
        readerRef.current.cancel('User navigated away');
      }
    };
  }, []);

  return <button onClick={startStream}>Generate</button>;
}

The DSA Dependency Graph: A Logical Roadmap from Arrays to DP

shubham khatik — Thu, 25 Dec 2025 10:22:14 +0000

The Problem
We often treat Data Structures and Algorithms (DSA) like a grocery list. We pick "Heaps" today and "Graphs" tomorrow. But computer science topics have dependencies. You cannot understand Heaps if you don't understand Arrays (how they are stored) and Trees (how they are visualized).

I created a 7-Phase Roadmap tailored to build a mental model step-by-step. This order ensures that every new topic uses tools you mastered in the previous phase.

🏗️ Phase 1: Basic Linear Structures & Their Patterns Goal: Mastering sequential data and index manipulation. Before touching complex nodes, master the line.
Time & Space Complexity (Big-O): The measuring stick for all code.Arrays: Memory models (Static vs Dynamic).
Searching and sorting Algorithms: Essential pre-requisite for Two Pointers.
Recursion (Pattern): Just the basics (Base case vs Recursive case).Two Pointers (Pattern): The most common array optimization.
Binary Search (Algorithm): $O(\log N)$ logic.
Sliding Window (Pattern): Subarrays and substrings.
Prefix Sum (Pattern): 1D and 2D range queries.Intervals (Pattern): Merging and inserting ranges.
Matrix Traversal (Pattern): Basics of iterating 2D grids.

🔍 Phase 2: The Power of Lookup (Hashing)Goal: Trading space for speed.
Hash Table / Hash Map: The $O(1)$ lookup king.
Strings: Immutability and manipulation techniques.

🔗 Phase 3: Pointers & Recursion Goal: Moving from Indexes to References.
Linked List: Single and Doubly linked.
Recursion (Advanced): Multiple recursive calls (Memoization prep).
Stack & Queue: LIFO and FIFO mechanics.
Monotonic Stack (Pattern): Finding the "next greater element.
"Fast & Slow Pointers (Pattern): Cycle detection.

🌳 Phase 4: Non-Linear Structures (Hierarchical)Goal: Parent-Child relationships.
Trees: Height, depth, diameter properties.
Binary Trees: The standard interview structure.
Tree Traversals: Preorder, Inorder, Postorder, Level-order.
Binary Search Tree (BST): Sorted hierarchical data.
Heaps / Priority Queue: Organizing by priority.
Top K Elements (Pattern): Using Heaps efficiently.
Divide and Conquer (Pattern): Merge Sort logic applied to trees.

🕵️ Phase 5: Advanced Search & BacktrackingGoal: Exploring all possibilities.
Backtracking (Pattern): Solving puzzles (Sudoku, N-Queens).
Tries: Prefix trees for autocomplete.

🌐 Phase 6: Connectivity (Graphs)Goal: Many-to-Many relationships.
Graphs: Adjacency Matrix vs. Adjacency List.
Matrix Traversal (Advanced): Treating Grids as Graphs.
Graph Traversals: BFS (Shortest path in unweighted) & DFS.
Union Find: Disjoint Set Union (DSU).
Shortest Path: Dijkstra's Algorithm.

🚀 Phase 7: Optimization (The Hardest Part)Goal: Solving overlapping sub-problems.
Bit Manipulation: Binary logic.
Dynamic Programming (DP):
The pipeline: Recursion → Memoization → Tabulation → Space Optimization.
Why this order?
Recursion is Split: You need basic recursion for sorting, but advanced recursion for Trees and DP.
Matrix is Split: You need to know how to loop through a matrix (Phase 1) before you can run BFS on it (Phase 6).
Patterns First: We learn "Two Pointers" immediately after Arrays because that is how you actually use arrays in the real world.

Save this roadmap and stop getting lost in the noise! 🚀

==> Data Structures
=> Linear
. Array
. Strings
. Linked List
. Stack
. Queue
. Deque
. Hash Table / Hash Map

=> Non-Linear
. Trees
. Binary Search Tree (BST)
. Heaps / Priority Queue
. Segment Tree / Fenwick Tree (BIT)
. Graphs
. Tries
. Union Find (Disjoint Set)

==> Algorithms
. Linear Search
. Basic Sorting (Bubble, Selection, Insertion)
. Merge Sort / Quick Sort
. Cyclic Sort
. Binary Search
. Modified Binary Search
. Bit Manipulation
. Tree Traversals
. Graph Traversals
. Topological Sort
. Shortest Path (Dijkstra / Bellman-Ford / Floyd-Warshall)
. Minimum Spanning Tree (Kruskal's / Prim's)

==> Patterns
. Two Pointers
. Fast & Slow Pointers
. Sliding Window
. Prefix Sum
. Intervals
. Cyclic Sort ← borderline pattern/algorithm, fits both
. Matrix Traversal
. Monotonic Stack/Queue
. Divide & Conquer
. Greedy
. Recursion
. Backtracking
. Top K Elements (Heap)
. K-way Merge

. Dynamic Programming

====> order of learning
=> Phase 1: Basic Linear Structures & Their Patterns
. Time & Space Complexity (Big-O)
. array
. strings
. Recursion (pattern) [base case, recursive case, call stack visualization]
. searching (Algorithm) [Linear Search]
. Basic Sorting (Bubble, Selection, Insertion)
. two pointers (pattern)
. binary search (algorithm)
. Modified Binary Search
. sliding window (pattern)
. Cyclic Sort
. prefix sum (pattern) [(1D and 2D)]
. Intervals (Pattern)
. Matrix Traversal (Pattern) [indexing, 2D prefix sum]

=> Phase 2: The Power of Lookup (Hashing)
. hash table /hash map
. Greedy (Pattern)

=> Phase 3: Pointers & Recursion
. linked list
. Recursion (pattern) [multiple recursive calls, memoization prep, linked list recursion]
. sorting (Merge sort, Quick sort)
. Divide and Conquer (Pattern)
. stack
. queue
. Deque (Double-ended Queue)
. Monotonic Stack/Queue (pattern)
. fast and slow pointers (pattern)

=> Phase 4: Non-Linear Structures (Hierarchical)
. Trees and properties (height, depth, diameter)
. binary trees
. Tree Traversals (Algorithm) [(preorder, inorder, postorder, level-order)]
. Binary Search Tree (BST)
. Heaps / Priority Queue
. Segment Tree / Fenwick Tree (BIT)
. Top K Elements (Pattern)
. K-way Merge

=> Phase 5: Advanced Search & Backtracking
. Backtracking (Pattern)
. Tries

=> Phase 6: Connectivity (Graphs)
. Graphs and Graph representations (Adjacency List / Matrix)
. Matrix Traversal (Pattern) [(DFS/BFS on grids)]
. Graph Traversals (Algorithm)
. Topological Sort
. Union Find (Disjoint Set)
. Shortest Path (Algorithm) (Dijkstra / Bellman-Ford / Floyd-Warshall)
. Minimum Spanning Tree (Kruskal's / Prim's)

=> Phase 7: Optimization (The Hardest Part)
. Bit Manipulation (algorithm)
. Dynamic Programming (pattern) [Recursion → Memoization. Tabulation. Space Optimization]

Understanding OAuth Login Flow with GitHub: Why Your Backend is Essential

shubham khatik — Fri, 17 Oct 2025 11:31:21 +0000

Implementing social login using OAuth with providers like GitHub can be a powerful way to improve user experience and security. However, the OAuth flow involves several important steps that developers sometimes misunderstand, especially around how tokens are handled between frontend and backend.

In this post, you’ll walk through the typical OAuth login process using GitHub as an example, understand the roles of the frontend and backend, and highlight critical security considerations.

What is OAuth and Why Use It?

OAuth is an open authorization standard that grants third-party applications limited access to user accounts on services like Google, GitHub, and Facebook—without exposing user credentials. It enables users to log in using their existing accounts without creating new passwords.

The GitHub OAuth Login Flow in a Nutshell

When integrating GitHub OAuth in a React app, you often hear about getting a "code" after the user authorizes your app, then exchanging that code for an access token to fetch user data. But what exactly happens here, and why can’t the frontend do everything?

Step-by-Step Flow

1. User Authorizes Your App on GitHub

Users click “Login with GitHub” on your React app.
They are redirected to GitHub’s consent page to grant permission.
After authorization, GitHub redirects users back with a temporary authorization code.

2. Frontend Receives the Authorization Code

The React frontend receives this short-lived authorization code (let’s call it code1).
This code doesn’t have user info but signals the user’s consent.

3. Frontend Sends Code to Backend

fetch('/auth/github/callback', {
  method: 'POST',
  body: JSON.stringify({ code: code1 }),
  headers: { 'Content-Type': 'application/json' }
});

4. Backend Exchanges Code for Access Token

Backend uses the code, client_id, and client_secret to request an access token from GitHub.

const tokenResponse = await axios.post('https://github.com/login/oauth/access_token', {
  client_id: process.env.GITHUB_CLIENT_ID,
  client_secret: process.env.GITHUB_CLIENT_SECRET,
  code: receivedCode1,
}, {
  headers: { Accept: 'application/json' }
});

const accessToken = tokenResponse.data.access_token;

5. Backend Requests User Info from GitHub

const userResponse = await axios.get('https://api.github.com/user', {
  headers: { Authorization: `Bearer ${accessToken}` }
});

const userData = userResponse.data;
// Process and store user data or create session

6. Backend Creates Session or Issues JWT

Backend creates a session or issues a signed JWT token encoding user info.
Sends this token or session ID back to the frontend.
Frontend uses this on subsequent requests to authenticate the user.

Why Offload Token Exchange to Backend?

Client Secret Security: Never expose client_secret on the frontend—it’s a private credential.
Token Privacy: Backend keeps access tokens hidden from clients, limiting risk from frontend breaches.
Control & Revocation: Backend manages session lifecycle, allowing token/session invalidation.
CORS & Rate Limits: Backend avoids cross-origin and rate limiting issues with third-party APIs.

Summary of Data Flow

Party	What it Receives/Uses	Purpose
Frontend React App	Authorization Code (`code1`)	Signals user permission to backend
Backend Server	code1 → Access Token	Gets authorized API access
Backend Server	Access Token	Fetches user data
Backend Server	User data & session/JWT	Creates session or issues JWT
Frontend React App	Session cookie or JWT	Authenticates future requests

The Complete Guide to AI-SEO in 2026

shubham khatik — Tue, 26 Aug 2025 10:49:56 +0000

🔥 The Complete Guide to AI-SEO in 2026: 6 Factors That Matter Most

AI search engines like Google SGE (Search Generative Experience), Perplexity, and ChatGPT Browse are changing the way people consume information.
Instead of clicking links, users often read AI summaries directly.

So, how do you make sure your site is the one AI pulls data from?
The answer → AI-SEO.

Here are the 6 major factors that decide whether search engines and AI models trust your content.

Technical SEO (Crawlability & Indexability)

Before ranking or summarizing your content, Google bots and AI scrapers need to find and read your pages.

Key elements:

Sitemap.xml → Lists all pages & last updated date.

<url> <loc>https://example.com/ai-seo</loc> <lastmod>2025-08-24</lastmod> </url>

Use next-sitemap in Next.js to auto-generate.

Robots.txt → Controls what crawlers can access.
User-agent: * Allow: / Disallow: /admin

Page speed / Core Web Vitals → Optimize for LCP, CLS, FID.
Tools: PageSpeed Insights
.

Mobile-friendliness → Responsive UI is mandatory.

HTTPS → Secure browsing (SSL certificate).

Canonical tags → Prevent duplicate content issues.
<link rel="canonical" href="https://example.com/ai-seo" />

On-Page SEO (Content Optimization)

Once crawlers can access your site, they need to understand it.

Key elements:

Title tag & Meta description
<title>AI SEO Guide 2025 - Next.js Optimization</title> <meta name="description" content="Learn how to optimize websites for AI SEO in 2025 with Next.js.">

Headings (H1, H2, H3) → Maintain hierarchy with keywords.
Example:
`
H1: AI SEO Guide 2025

H2: Technical SEO Best Practices

H3: Sitemap.xml Example`

Keyword placement → Natural use in intro, body, headings, alt text.

Internal linking → Pass authority to deeper pages.
<a href="/ai-seo/technical">Technical SEO Guide</a>

Image alt text → Helps search engines & accessibility.
<img src="seo.png" alt="AI SEO best practices infographic" />

Schema markup (Structured Data) → Enables rich snippets.

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>AI SEO Guide</title> <script type="application/ld+json"> { "@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [{ "question": "What is AI SEO?", "acceptedAnswer": { "text": "AI SEO optimizes content for AI-powered search engines." } }] } </script> </head> <body> <h1>AI SEO Guide</h1> <p>AI SEO optimizes content for AI-powered search engines...</p> </body> </html>

Content Quality (E-E-A-T Framework)

Google and AI models now score content based on Expertise, Experience, Authoritativeness, and Trustworthiness (E-E-A-T).

Key elements:

Expertise → Written by subject-matter experts.

Experience → Real-world examples, case studies.

Authoritativeness → Backlinks from trusted sites.

Trustworthiness → Accuracy, updated sources, transparency.

📌 Example:

Add an author bio → “Written by Shubham Khatik, Full-Stack Developer (4 YOE in React & Next.js).”

Add citations & references to studies or docs.

Keep a “last updated” date on blog posts.

Off-Page SEO (Backlinks & Authority)

Search engines & AI models rely heavily on external signals.

Key elements:

Backlinks → Guest posts, niche collaborations.

Mentions → HARO (Help a Reporter Out), PR campaigns.

Social signals → Shares on LinkedIn, Twitter/X.

Community engagement → Reddit, StackOverflow answers with links.

📌 Example:
If you run a React SEO blog, write a guest article for Smashing Magazine → link back to your site.

User Experience (Engagement Metrics)

Search engines track how users behave on your site.

Key elements:

CTR (Click-through rate) → Titles & meta descriptions should attract clicks.

Dwell time → Keep users engaged with visuals, videos, TOCs.

Bounce rate → Reduce drop-offs with clean navigation & summaries.

Mobile UX → Ensure smooth layouts.

📌 Example:

Add Table of Contents in long blogs (reduces bounce).

Use sticky navigation for mobile.

Content Freshness

AI prefers up-to-date content.

Key elements:

Update old blogs regularly (e.g., “SEO 2022” → “SEO 2025”).

Sitemap → Helps Google & AI detect new updates.

Dynamic content → Job boards, latest stats, tools.

Seasonal updates → Example: “Black Friday Deals 2025.”

Schema dateModified → Mark updated content.

📌 Example:

{ "@context": "https://schema.org", "@type": "Article", "headline": "AI SEO Guide 2025", "dateModified": "2025-08-24" }

🎯 Final Takeaway

In 2025, SEO isn’t just about keywords anymore.
AI-driven search engines rank content using a combination of:

Technical SEO – Can bots access your site?

On-Page SEO – Is your content structured for machines?

Content Quality (E-E-A-T) – Is it expert & trustworthy?

Off-Page SEO – Do others vouch for you?

User Experience – Do users stay engaged?

Content Freshness – Is it up to date?

👉 AI will always summarize. Your job is to be the source AI trusts.

Sitemaps for SEO

shubham khatik — Tue, 26 Aug 2025 09:18:14 +0000

🚀 Just discovered something new about SEO & AI!

We all know tags help describe a single page, but here’s the twist:

👉 Sitemaps (sitemap.xml) tell Google & AI about your entire website and when each page was last updated.

Without it, crawlers guess your structure. With it, you hand them a clean map + freshness signals 📅.

Example (inside sitemap.xml):

<url>

  <loc>https://example.com/ai-seo</loc>

  <lastmod>2025-08-24</lastmod>

</url>

= your page URL

= when the page was last updated

This helps search engines & AI summaries pick the freshest content, not outdated info.

💡 In React apps → put sitemap.xml in /public/ folder.

💡 In Next.js (App Router) → create app/sitemap.ts and let Next auto-generate lastmod.

🔑 Takeaway:

SEO is shifting from just meta tags → to structured data + sitemaps + freshness.

If you want your content to show up in AI summaries, freshness is no longer optional 🚀.

🚀 Next.js App Router : CSR SSR SSG ISR

shubham khatik — Sun, 24 Aug 2025 10:17:57 +0000

🚀 Next.js App Router: Forget getStaticProps, Think Caching

When I first switched from Pages Router to the new App Router in Next.js, I noticed something strange:

👉 There’s no getStaticProps or getServerSideProps.

At first it felt like SSG & SSR were “gone”… but in reality, they’ve just been reimagined.

In App Router, rendering strategy = caching strategy.

Here’s how it works ⬇️

🔹 CSR (Client-Side Rendering)

`'use client';

useEffect(() => {

fetch('/api/data').then(r => r.json());

}, []);`

Happens by default when you fetch data in a Client Component.

🔹 SSR (Server-Side Rendering)

await fetch(url, { cache: 'no-store' })

This ensures fresh data on every request.

🔹 SSG (Static Site Generation)

await fetch(url, { cache: 'force-cache' }) // default

Data is fetched at build time, HTML is reused for all users.

🔹 ISR (Incremental Static Regeneration)

await fetch(url, { next: { revalidate: 60 } })

Page is pre-rendered like SSG, but revalidated every 60s in the background.

⚖️ Old mental model:

getStaticProps = SSG

getServerSideProps = SSR

⚡ New mental model:

fetch + caching = CSR, SSR, SSG, ISR

💡 Key takeaway:

Next.js App Router unifies rendering modes under one API. Instead of special lifecycle methods, you just decide caching rules → much more flexible (and powerful).

JavaScript Question Part 2

shubham khatik — Wed, 06 Aug 2025 10:59:11 +0000

🚀 Frontend Performance & React Tips Every Developer Should Know 🚀

Ever wondered what really happens when you hit a URL in your browser? It’s a fascinating journey through DNS lookup, HTTP requests, and the Critical Rendering Path (CRP)—the secret sauce that transforms code into pixels on your screen. Optimizing the CRP means faster load times and happier users!

⚡️ Quick Rundown on Key Frontend Concepts:

Repaint vs. Reflow:

Repaint updates visual styles (color, visibility) and costs less;

Reflow recalculates layouts (size, position) and is more expensive.

Canceling Old API Calls:

Use AbortController to cancel outdated fetch requests so your app only processes the latest data and stays performant.

Parallel API Calls:

Use Promise.all() when all API calls must succeed, as it fails fast on the first rejection. Use Promise.allSettled() when you need to run all calls in parallel and handle both successes and failures individually.

github

JavaScript Question Part 1

Frontend #WebDevelopment #ReactJS #JavaScript #Performance #WebPerf #CodingTips

JavaScript Question Part 1

shubham khatik — Tue, 05 Aug 2025 08:26:51 +0000

🚀 Boost Your Web Performance & User Experience!

Understanding frontend essentials can transform your website’s speed and efficiency:

🔹 Preload, Prefetch, Reconnect, Prerender: Techniques to load resources smarter—fetch critical files early, set up server connections in advance, anticipate user navigation, and even render pages before they’re requested!

🔹 Caching & HTTP Headers: Use Cache-Control and ETag headers to control how browsers store and validate content, reducing load times and server strain.

🔹 Asset Optimization & Image Compression: Minimize file sizes with compression techniques and serve modern formats like WebP for faster loads without sacrificing quality. Know when to use PNG, JPEG, or WebP.

🔹 Memory Leaks: Avoid hidden performance killers by managing memory carefully—release unused DOM nodes and event listeners to keep your app smooth and responsive.

Master these strategies and watch your frontend performance soar! ⚡

github

WebDevelopment #Frontend #WebPerformance #Caching #ImageOptimization #JavaScript #MemoryManagement

System Design

shubham khatik — Wed, 30 Jul 2025 06:30:33 +0000

PDF [in depth]

Summary: Backend System Design

This conversation covered foundational and advanced topics in backend system design, focusing on how to build reliable, scalable, performant, and maintainable systems. Here is a structured summary by category:

Core Concepts and Patterns

Consistency & Availability Patterns: Distributed systems balance data consistency (strong, eventual, causal, read-your-writes, monotonic reads/writes) with availability (failover, replication, load balancing, redundancy, auto-scaling, circuit breakers, bulkhead, leader election).
Horizontal Scaling: Adding more servers/instances improves throughput, uptime, and elasticity. Stateless design, load balancers, service discovery, and distributed storage enable scaling out effectively, while stateful components must use replication/sync strategies.

Architecture & Key Components

Databases:
- Wide Column Stores (e.g., Cassandra, HBase): Flexible, scalable for semi-structured big data.
- Denormalization: Boosts read performance but increases redundancy and consistency risk.
- Indexes: Speed up queries via auxiliary structures (B-trees, hash, bitmap) to avoid full scans, trading off extra storage and slower writes.
Caching:
- Strategies: Cache-aside, read-through, write-through, write-back, write-around, refresh-ahead.
- Placement: Client-side, server/db-side, distributed (e.g., Redis), edge/CDN.
- Balances speed, consistency, and complexity—chosen based on workload and latency needs.
System Design Patterns:
- Reliability Patterns: Bulkhead, circuit breaker, retry, saga, health checks, load leveling, leader election, replication; all used for fault tolerance and service continuity.
- Cloud Patterns: Bulkhead, circuit breaker, retry, throttling, cache aside, external configuration, strangler, etc., supporting resilience, scalability, and cost optimization.

Operations & Performance

Load Balancers & Algorithms: Hardware/software/cloud-based, L4 (transport) vs. L7 (application), and algorithms (round robin, least connections, source IP hash, etc.) optimize traffic distribution, uptime, and scalability.
- Load balancer and reverse proxy differences/overlaps highlighted.
Performance Antipatterns: Issues like chatty I/O, lack of caching, synchronous/blocking calls, retry storms, and single points of failure degrade speed and stability. These are avoided through batching, async communication, caching, and robust error handling.

Observability & Maintenance

Monitoring: Systems are instrumented using metrics, logging, tracing, dashboards, and alerting tools (Prometheus, Grafana, Datadog). Monitoring is vital for detecting/regressing faults, optimizing performance, and capacity planning.

Data & API Management

Service Discovery: Microservices use service registries (e.g., Consul, Eureka), client/server-side discovery, and mesh/gateways for dynamic routing as instances scale or change.
Event-Driven & Schedule-Driven Jobs: Systems offload background tasks, triggered by events (real-time response) or schedules (cron jobs), improving performance and automation flexibility.
GraphQL: An efficient, flexible, strongly-typed API query language enabling clients to fetch exactly the data needed in a single call, compared to REST.

Reliability, Security, and Resilience

Reliability Patterns: Patterns like replication, circuit breakers, bulkhead, and automatic failover are central to ensuring high availability and resilient, self-healing systems. Security is reinforced through patterns like rate limiting and external configuration.

Best Practices & Tradeoffs

Statelessness enables fast scaling; stateful systems require more carefully managed replication and synchronization.
Caching and indexing accelerate reads but bring risks of staleness and overhead.
Cloud design patterns must balance resilience, complexity, and cost.
Monitoring and observability are essential for continuous improvement and stability.
Idempotent operations and asynchronous design are key to building robust APIs and distributed systems resilient to retries and failures.

This collection of patterns and strategies forms a toolkit for building highly available, scalable, secure, and observable backend architectures suitable for modern cloud-native, data-intensive applications.

Forem: shubham khatik

Escaping the Buffer: The Advanced Guide to Streams in Node.js & Next.js

Escaping the Buffer: The Advanced Guide to Streams in Node.js & Next.js

1. The Real Problem: Buffering vs. Streaming

The Backend Failure Mode (OOM Crashes)

The Frontend Failure Mode (High TTFB)

2. The Mental Model: Two Colliding Worlds

3. Practical Workflows & Real-World Execution

Scenario A: Massive Database Exports (MongoDB / PostgreSQL)

Scenario B: AI Text Generation (The Web Streams API)

Scenario C: Next.js App Router (Bridging the Gap)

4. When NOT to Use Streams (Failure Modes)

5. The "Halfway" Failure Mode: Preventing Memory Leaks

1. Node.js Backend: Never Use Raw .pipe()

2. Next.js App Router: Catching the AbortSignal

3. React Frontend: Canceling the Stream Reader

The DSA Dependency Graph: A Logical Roadmap from Arrays to DP

. Dynamic Programming

Understanding OAuth Login Flow with GitHub: Why Your Backend is Essential

What is OAuth and Why Use It?

The GitHub OAuth Login Flow in a Nutshell

Step-by-Step Flow

1. User Authorizes Your App on GitHub

2. Frontend Receives the Authorization Code

3. Frontend Sends Code to Backend

4. Backend Exchanges Code for Access Token

5. Backend Requests User Info from GitHub

6. Backend Creates Session or Issues JWT

Why Offload Token Exchange to Backend?

Summary of Data Flow

The Complete Guide to AI-SEO in 2026

Sitemaps for SEO

🚀 Next.js App Router : CSR SSR SSG ISR

JavaScript Question Part 2

Frontend #WebDevelopment #ReactJS #JavaScript #Performance #WebPerf #CodingTips

JavaScript Question Part 1

WebDevelopment #Frontend #WebPerformance #Caching #ImageOptimization #JavaScript #MemoryManagement

System Design

Summary: Backend System Design

Core Concepts and Patterns

Architecture & Key Components

Operations & Performance

Observability & Maintenance

Data & API Management

Reliability, Security, and Resilience

Best Practices & Tradeoffs

1. Node.js Backend: Never Use Raw `.pipe()`

2. Next.js App Router: Catching the `AbortSignal`