Forem: HK Lee

7 Hidden Production Bugs AI Coding Agents Create (And How to Catch Them Before They Crash)

HK Lee — Wed, 08 Apr 2026 14:02:00 +0000

Your AI coding agent just built a feature in 45 seconds. The code compiles. The tests pass. The PR looks clean. You ship it to production on a Friday evening feeling confident.

By Saturday morning, your database is at 100% CPU, your Redis cluster is unresponsive, and three customers have reported seeing each other's data. The on-call engineer is staring at AI-generated code they've never seen before, and nothing in the error logs makes sense because the AI didn't add proper error logging.

This is not a hypothetical scenario. It's happening daily across the industry. A 2025 Endor Labs study found that 62% of AI-generated code contains security weaknesses or design flaws. But the more insidious problem isn't the obvious bugs — it's the hidden ones. The code that looks correct, passes unit tests, survives code review, and then detonates under real-world conditions that no one thought to test.

This guide dissects the 7 most dangerous hidden bug patterns that AI coding agents consistently produce. For each pattern, we'll cover why AI generates it, how to detect it before production, and the battle-tested fix. These aren't theoretical risks — they're patterns extracted from hundreds of production incidents at companies running AI-assisted development workflows in 2025-2026.

Pattern 1: The Cache Stampede

What AI Generates

When you ask an AI agent to add caching, it produces textbook cache-aside logic:

async function getProduct(id: string): Promise<Product> {
  const cached = await redis.get(`product:${id}`);
  if (cached) {
    return JSON.parse(cached);
  }

  const product = await db.query('SELECT * FROM products WHERE id = $1', [id]);
  await redis.set(`product:${id}`, JSON.stringify(product), 'EX', 3600);
  return product;
}

This code is correct in isolation. Every test passes. The code review looks clean.

Why It Detonates

Under production load, when a popular cache key expires, hundreds of concurrent requests all miss the cache simultaneously. Every single one hits the database with the same query. Your database CPU spikes to 100%, queries start timing out, and the cascade of failures brings down unrelated services.

This is a cache stampede (also called thundering herd). AI agents produce it because their training data is dominated by tutorial-style caching examples that assume single-threaded, low-traffic environments.

The Detection Strategy

// Add this to your load test suite
it('should handle concurrent cache misses without stampede', async () => {
  await redis.del('product:popular-item');

  // Simulate 100 concurrent requests for the same key
  const requests = Array.from({ length: 100 }, () =>
    getProduct('popular-item')
  );

  const dbQuerySpy = vi.spyOn(db, 'query');
  await Promise.all(requests);

  // If more than 1 query hits the DB, you have a stampede
  expect(dbQuerySpy).toHaveBeenCalledTimes(1);
});

The Production Fix

import { Mutex } from 'async-mutex';

const locks = new Map<string, Mutex>();

async function getProduct(id: string): Promise<Product> {
  const cacheKey = `product:${id}`;
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }

  // Acquire a per-key lock to prevent stampede
  if (!locks.has(cacheKey)) {
    locks.set(cacheKey, new Mutex());
  }
  const mutex = locks.get(cacheKey)!;

  return mutex.runExclusive(async () => {
    // Double-check after acquiring lock
    const rechecked = await redis.get(cacheKey);
    if (rechecked) {
      return JSON.parse(rechecked);
    }

    const product = await db.query(
      'SELECT * FROM products WHERE id = $1', [id]
    );
    await redis.set(cacheKey, JSON.stringify(product), 'EX', 3600);
    return product;
  });
}

The key insight: double-checked locking. The first request acquires the lock, fetches from DB, and populates the cache. Every other concurrent request waits on the lock, then finds the cache populated, and returns immediately without hitting the database.

Pattern 2: Connection Pool Exhaustion

What AI Generates

AI agents love async/await but consistently produce code that leaks database connections:

async function processOrder(orderId: string) {
  const client = await pool.connect();

  const order = await client.query(
    'SELECT * FROM orders WHERE id = $1', [orderId]
  );

  // AI generates business logic here...
  const inventory = await client.query(
    'SELECT * FROM inventory WHERE product_id = $1',
    [order.rows[0].product_id]
  );

  await client.query(
    'UPDATE inventory SET quantity = quantity - $1 WHERE product_id = $2',
    [order.rows[0].quantity, order.rows[0].product_id]
  );

  client.release();
  return { success: true };
}

Why It Detonates

If any query throws an error — a constraint violation, a timeout, a deadlock — client.release() never executes. The connection is leaked. After enough leaked connections, pool.connect() blocks indefinitely because all connections are occupied by abandoned handlers. Your entire application freezes with zero error logs because the hang happens at the connection acquisition level, not the query level.

AI generates this because its training data overwhelmingly shows the "happy path." Error handling in connection management is rarely demonstrated in tutorials.

The Detection Strategy

// Monitor pool metrics in production
setInterval(() => {
  const { totalCount, idleCount, waitingCount } = pool;
  logger.info('pool_metrics', {
    total: totalCount,
    idle: idleCount,
    waiting: waitingCount,
    active: totalCount - idleCount,
  });

  if (waitingCount > 5) {
    logger.warn('pool_pressure', {
      message: 'Connection pool under pressure',
      waiting: waitingCount,
    });
  }
}, 10_000);

The Production Fix

async function processOrder(orderId: string) {
  const client = await pool.connect();
  try {
    await client.query('BEGIN');

    const order = await client.query(
      'SELECT * FROM orders WHERE id = $1', [orderId]
    );

    const inventory = await client.query(
      'SELECT * FROM inventory WHERE product_id = $1',
      [order.rows[0].product_id]
    );

    await client.query(
      'UPDATE inventory SET quantity = quantity - $1 WHERE product_id = $2',
      [order.rows[0].quantity, order.rows[0].product_id]
    );

    await client.query('COMMIT');
    return { success: true };
  } catch (error) {
    await client.query('ROLLBACK');
    throw error;
  } finally {
    client.release(); // ALWAYS releases, even on error
  }
}

The try/catch/finally pattern guarantees connection release regardless of what happens. This is the single most important pattern that AI agents consistently omit.

Pattern 3: Silent Data Corruption

What AI Generates

When asked to handle API data, AI agents trust external input implicitly:

app.post('/api/users/:id/profile', async (req, res) => {
  const { name, email, role } = req.body;

  await db.query(
    'UPDATE users SET name = $1, email = $2, role = $3 WHERE id = $4',
    [name, email, role, req.params.id]
  );

  res.json({ success: true });
});

Why It Detonates

This endpoint allows any authenticated user to set their own role field to admin. The AI destructured role from the request body because the database schema has a role column, and the AI's pattern matching connected the two without considering authorization implications.

More subtly, the name and email fields accept any string. A user can set their name to a 10MB string, or their email to not-an-email. The data is "saved successfully" but corrupts downstream systems: email sending breaks, CSV exports crash, search indices bloat.

This is silent data corruption — the write succeeds, but invalid data spreads through your system like a slow poison.

The Detection Strategy

// Schema validation at the boundary — BEFORE business logic
import { z } from 'zod';

const UpdateProfileSchema = z.object({
  name: z.string().min(1).max(100).trim(),
  email: z.string().email().max(254).toLowerCase(),
  // Notice: 'role' is NOT in this schema
});

// Integration test: verify forbidden fields are rejected
it('should not allow role escalation via profile update', async () => {
  const res = await request(app)
    .post('/api/users/user-1/profile')
    .send({ name: 'Hacker', email: 'hacker@test.com', role: 'admin' })
    .set('Authorization', `Bearer ${userToken}`);

  const user = await db.query('SELECT role FROM users WHERE id = $1', ['user-1']);
  expect(user.rows[0].role).toBe('member'); // NOT 'admin'
});

The Production Fix

app.post('/api/users/:id/profile', async (req, res) => {
  // 1. Validate input — strip unknown fields
  const parsed = UpdateProfileSchema.safeParse(req.body);
  if (!parsed.success) {
    return res.status(400).json({
      error: 'Validation failed',
      issues: parsed.error.issues,
    });
  }

  // 2. Only use validated data (role is impossible to inject)
  const { name, email } = parsed.data;

  // 3. Verify the user can only update their own profile
  if (req.params.id !== req.user.id) {
    return res.status(403).json({ error: 'Forbidden' });
  }

  await db.query(
    'UPDATE users SET name = $1, email = $2 WHERE id = $3',
    [name, email, req.params.id]
  );

  res.json({ success: true });
});

The fix has three layers: schema validation (strips role), authorization check (own profile only), and constrained data types (email format, name length). AI-generated code almost never implements all three.

Pattern 4: The Unhandled Race Condition

What AI Generates

When implementing "like" or "upvote" functionality:

async function toggleLike(userId: string, postId: string) {
  const existing = await db.query(
    'SELECT id FROM likes WHERE user_id = $1 AND post_id = $2',
    [userId, postId]
  );

  if (existing.rows.length > 0) {
    await db.query('DELETE FROM likes WHERE id = $1', [existing.rows[0].id]);
    await db.query(
      'UPDATE posts SET like_count = like_count - 1 WHERE id = $1', [postId]
    );
    return { liked: false };
  } else {
    await db.query(
      'INSERT INTO likes (user_id, post_id) VALUES ($1, $2)',
      [userId, postId]
    );
    await db.query(
      'UPDATE posts SET like_count = like_count + 1 WHERE id = $1', [postId]
    );
    return { liked: true };
  }
}

Why It Detonates

Double-tap a like button on a phone with poor connectivity. The first request checks and finds no existing like. The second request, arriving milliseconds later, also checks and finds no existing like (because the first INSERT hasn't committed yet). Both requests insert a like. like_count increments by 2. The user now has two like records, and the count is permanently desynchronized.

This race condition is invisible in testing because tests run sequentially. It only manifests under concurrent production traffic.

The Detection Strategy

it('should handle concurrent like toggles correctly', async () => {
  // Simulate double-tap race condition
  const results = await Promise.all([
    toggleLike('user-1', 'post-1'),
    toggleLike('user-1', 'post-1'),
  ]);

  const likes = await db.query(
    'SELECT COUNT(*) FROM likes WHERE user_id = $1 AND post_id = $2',
    ['user-1', 'post-1']
  );

  // Should be exactly 0 or 1, never 2
  expect(Number(likes.rows[0].count)).toBeLessThanOrEqual(1);
});

The Production Fix

async function toggleLike(userId: string, postId: string) {
  return await db.transaction(async (tx) => {
    // Lock the row to prevent concurrent modifications
    const existing = await tx.query(
      `SELECT id FROM likes 
       WHERE user_id = $1 AND post_id = $2 
       FOR UPDATE`,                     // ← Row-level lock
      [userId, postId]
    );

    if (existing.rows.length > 0) {
      await tx.query('DELETE FROM likes WHERE id = $1', [existing.rows[0].id]);
      await tx.query(
        'UPDATE posts SET like_count = like_count - 1 WHERE id = $1',
        [postId]
      );
      return { liked: false };
    } else {
      await tx.query(
        `INSERT INTO likes (user_id, post_id) VALUES ($1, $2)
         ON CONFLICT (user_id, post_id) DO NOTHING`, // ← Idempotent
        [userId, postId]
      );
      await tx.query(
        'UPDATE posts SET like_count = like_count + 1 WHERE id = $1',
        [postId]
      );
      return { liked: true };
    }
  });
}

Two critical additions: FOR UPDATE creates a row-level lock that serializes concurrent operations, and ON CONFLICT DO NOTHING provides a safety net if the lock timing allows a duplicate. AI agents almost never generate FOR UPDATE because it barely exists in tutorial code.

Pattern 5: The Retry Storm

What AI Generates

When asked to "make this API call more resilient":

async function callPaymentAPI(data: PaymentRequest): Promise<PaymentResult> {
  const MAX_RETRIES = 3;

  for (let attempt = 0; attempt < MAX_RETRIES; attempt++) {
    try {
      const response = await fetch('https://api.payment.com/charge', {
        method: 'POST',
        body: JSON.stringify(data),
        headers: { 'Content-Type': 'application/json' },
      });

      if (!response.ok) {
        throw new Error(`Payment API error: ${response.status}`);
      }

      return await response.json();
    } catch (error) {
      if (attempt === MAX_RETRIES - 1) throw error;
      // Retry immediately
    }
  }

  throw new Error('Unreachable');
}

Why It Detonates

When the payment API is experiencing degraded performance (responding in 5 seconds instead of 200ms), every incoming request spawns up to 3 rapid-fire calls to the already-struggling service. If you have 1,000 concurrent users, the struggling API receives 3,000 requests instead of 1,000. This amplifies the failure, creates a feedback loop, and can cascade into a full outage of both your service and the downstream API.

Worse: the AI retried on a POST /charge endpoint. If the first request actually succeeded but the response was slow, the retry creates a duplicate charge. The customer gets billed twice.

The Detection Strategy

it('should implement exponential backoff, not immediate retry', async () => {
  let callTimestamps: number[] = [];
  vi.spyOn(global, 'fetch').mockImplementation(async () => {
    callTimestamps.push(Date.now());
    throw new Error('Service unavailable');
  });

  await expect(callPaymentAPI(mockData)).rejects.toThrow();

  // Verify exponential delays between retries
  for (let i = 1; i < callTimestamps.length; i++) {
    const delay = callTimestamps[i] - callTimestamps[i - 1];
    expect(delay).toBeGreaterThan(500 * Math.pow(2, i - 1));
  }
});

it('should not retry non-idempotent requests on ambiguous failures', async () => {
  vi.spyOn(global, 'fetch').mockRejectedValueOnce(new Error('timeout'));

  // For POST (non-idempotent), should NOT retry on timeout
  // because the request may have already been processed
  await expect(callPaymentAPI(mockData)).rejects.toThrow();
  expect(fetch).toHaveBeenCalledTimes(1);
});

The Production Fix

async function callPaymentAPI(data: PaymentRequest): Promise<PaymentResult> {
  // 1. Add idempotency key to prevent duplicate charges
  const idempotencyKey = crypto.randomUUID();

  // 2. Use exponential backoff with jitter
  const MAX_RETRIES = 3;
  for (let attempt = 0; attempt < MAX_RETRIES; attempt++) {
    try {
      const controller = new AbortController();
      const timeout = setTimeout(() => controller.abort(), 10_000);

      const response = await fetch('https://api.payment.com/charge', {
        method: 'POST',
        body: JSON.stringify(data),
        headers: {
          'Content-Type': 'application/json',
          'Idempotency-Key': idempotencyKey, // ← Prevents duplicate charges
        },
        signal: controller.signal,
      });

      clearTimeout(timeout);

      // 3. Only retry on retryable status codes
      if (response.status === 429 || response.status >= 500) {
        throw new RetryableError(response.status);
      }

      if (!response.ok) {
        // 4xx errors are NOT retryable (bad input, auth failure)
        throw new NonRetryableError(response.status);
      }

      return await response.json();
    } catch (error) {
      if (error instanceof NonRetryableError) throw error;
      if (attempt === MAX_RETRIES - 1) throw error;

      // 4. Exponential backoff with jitter
      const baseDelay = 1000 * Math.pow(2, attempt);
      const jitter = Math.random() * 500;
      await new Promise(resolve =>
        setTimeout(resolve, baseDelay + jitter)
      );
    }
  }

  throw new Error('Unreachable');
}

Four critical additions: idempotency keys (prevent duplicate charges), exponential backoff with jitter (prevent synchronized retry storms), retryable vs. non-retryable error classification (don't retry 400 errors), and explicit timeouts (prevent hanging connections). AI agents typically implement none of these.

Pattern 6: The Slow Memory Leak

What AI Generates

When building event processing or WebSocket handlers:

class NotificationService {
  private listeners: Map<string, Set<(data: any) => void>> = new Map();

  subscribe(userId: string, callback: (data: any) => void) {
    if (!this.listeners.has(userId)) {
      this.listeners.set(userId, new Set());
    }
    this.listeners.get(userId)!.add(callback);
  }

  notify(userId: string, data: any) {
    this.listeners.get(userId)?.forEach(cb => cb(data));
  }
}

// In the WebSocket handler
wss.on('connection', (ws, req) => {
  const userId = extractUserId(req);

  const callback = (data: any) => {
    ws.send(JSON.stringify(data));
  };

  notificationService.subscribe(userId, callback);

  ws.on('message', (msg) => { /* handle messages */ });
});

Why It Detonates

There's no unsubscribe call when the WebSocket disconnects. Every connection adds a callback to the Set, but no connection ever removes one. After a week of production traffic, the listeners Map holds millions of dead callbacks pointing to closed WebSocket connections. Memory usage grows linearly until the Node.js process crashes with an OOM error.

The leak is invisible in development because the process restarts frequently. In production, it takes days or weeks to manifest, making it extremely difficult to correlate with the original code change.

The Detection Strategy

it('should clean up listeners on disconnect', async () => {
  const ws = new MockWebSocket();
  simulateConnection(ws, 'user-1');

  const listenersBefore = notificationService.getListenerCount('user-1');
  expect(listenersBefore).toBe(1);

  // Simulate disconnect
  ws.emit('close');

  const listenersAfter = notificationService.getListenerCount('user-1');
  expect(listenersAfter).toBe(0);
});

// Production monitoring
setInterval(() => {
  let total = 0;
  notificationService.listeners.forEach((set) => { total += set.size; });
  logger.info('listener_count', { total });
  // Alert if total > expected active connections * 1.5
}, 60_000);

The Production Fix

wss.on('connection', (ws, req) => {
  const userId = extractUserId(req);

  const callback = (data: any) => {
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(JSON.stringify(data));
    }
  };

  notificationService.subscribe(userId, callback);

  // Clean up on disconnect — the critical missing piece
  ws.on('close', () => {
    notificationService.unsubscribe(userId, callback);
  });

  ws.on('error', () => {
    notificationService.unsubscribe(userId, callback);
  });

  ws.on('message', (msg) => { /* handle messages */ });
});

// Add the missing unsubscribe method
class NotificationService {
  // ... existing code ...

  unsubscribe(userId: string, callback: (data: any) => void) {
    const userListeners = this.listeners.get(userId);
    if (userListeners) {
      userListeners.delete(callback);
      if (userListeners.size === 0) {
        this.listeners.delete(userId); // Clean up empty Sets too
      }
    }
  }

  getListenerCount(userId: string): number {
    return this.listeners.get(userId)?.size ?? 0;
  }
}

The fix adds cleanup handlers for both close and error events, a readyState check before sending (prevents "write after end" errors), and cleanup of empty Sets in the Map. AI agents consistently implement subscribe but forget unsubscribe because the training data rarely shows cleanup code.

Pattern 7: The Auth Context Leak

What AI Generates

When building middleware or service layers with authentication:

// Auth middleware
function authMiddleware(req: Request, res: Response, next: NextFunction) {
  const token = req.headers.authorization?.split(' ')[1];
  const decoded = jwt.verify(token!, process.env.JWT_SECRET!);
  req.user = decoded as AuthUser;
  next();
}

// Service using the auth context
class OrderService {
  async getOrders(userId: string) {
    return db.query('SELECT * FROM orders WHERE user_id = $1', [userId]);
  }
}

// Controller
app.get('/api/orders', authMiddleware, async (req, res) => {
  const orders = await orderService.getOrders(req.query.userId as string);
  res.json(orders);
});

Why It Detonates

The auth middleware correctly validates the JWT and sets req.user. But the controller passes req.query.userId (from the URL query string) to the service instead of req.user.id (from the verified token). Any authenticated user can access any other user's orders by simply changing the userId query parameter.

This is an IDOR vulnerability (Insecure Direct Object Reference). The AI generated all three components correctly in isolation, but the wiring between them created a security hole. The AI's pattern-matching connected the userId parameter to the query string because the variable name appeared there, not because it was the secure choice.

The Detection Strategy

it('should not allow accessing other users orders', async () => {
  // User A's token
  const tokenA = generateToken({ id: 'user-a', role: 'member' });

  // Try to access User B's orders
  const res = await request(app)
    .get('/api/orders?userId=user-b')
    .set('Authorization', `Bearer ${tokenA}`);

  // Should either return 403 or only return User A's orders
  const orders = res.body;
  orders.forEach((order: any) => {
    expect(order.user_id).toBe('user-a');
  });
});

The Production Fix

// The controller MUST use the verified auth context, not query params
app.get('/api/orders', authMiddleware, async (req, res) => {
  // ✅ Uses verified identity from JWT, ignores query params
  const orders = await orderService.getOrders(req.user.id);
  res.json(orders);
});

// For admin endpoints that need cross-user access:
app.get('/api/admin/orders/:userId', authMiddleware, requireRole('admin'),
  async (req, res) => {
    const orders = await orderService.getOrders(req.params.userId);
    res.json(orders);
  }
);

The rule is simple: never use user-supplied identity when you have a verified identity. If the user is authenticated, their identity comes from the JWT. The query string is for filtering and pagination, never for identification.

The Meta-Pattern: Why AI Agents Produce These Bugs

All seven patterns share a common root cause: AI agents optimize for the happy path.

Their training data is overwhelmingly composed of tutorials, documentation examples, and Stack Overflow answers that demonstrate how things work when everything goes right. Real production code spends 80% of its complexity budget on handling what happens when things go wrong.

Training Data Distribution vs. Production Code Reality:

  AI Training Data:           Production Code:
  ─────────────────           ─────────────────
  Happy path:     85%         Happy path:     20%
  Error handling: 10%         Error handling: 35%
  Edge cases:      3%         Edge cases:     25%
  Concurrency:     1%         Concurrency:    10%
  Cleanup:         1%         Cleanup:        10%

This creates a systematic blind spot. The AI generates the 20% of code that handles normal operations beautifully, then either ignores or superficially handles the 80% that makes code production-ready.

The Detection Checklist

Before merging any AI-generated code, run through this checklist:

Category	Question	If "No" →
Concurrency	Does this code work with 1000 simultaneous requests?	Add locking / `FOR UPDATE` / idempotency
Connection Management	Are all connections released in a `finally` block?	Add try/finally or use connection helpers
Input Validation	Is every external input validated and constrained?	Add Zod schemas at API boundaries
Auth Context	Does the code use verified identity, not user-supplied?	Replace `req.query` / `req.params` with `req.user`
Retry Logic	Do retries use exponential backoff + idempotency keys?	Replace naive retry loops
Resource Cleanup	Are event listeners, timers, and subscriptions cleaned up?	Add close/error/unsubscribe handlers
Error Propagation	Do errors include enough context for debugging?	Add structured logging with correlation IDs
State Consistency	Can a crash between two writes leave data inconsistent?	Wrap in a database transaction

Building Your Defensive Layer

The most effective defense isn't reviewing every line of AI-generated code. It's building infrastructure that catches these patterns automatically.

1. Mandatory Integration Tests for Concurrency

Don't just test that the code works. Test that it works under concurrent load. Every write operation should have a Promise.all() test that verifies correctness under race conditions.

2. Connection Pool Monitoring

Add real-time pool metrics (active connections, waiting connections, idle connections) to your observability stack. Alert when waiting > 5. This catches connection leaks before they become outages.

3. Schema Validation at Every Boundary

Use a validation library (Zod, Valibot, ArkType) at every point where data crosses a trust boundary: API endpoints, queue consumers, webhook handlers, third-party API responses. Never trust that the shape of the data matches what you expect.

4. Auth-by-Default Architecture

Structure your codebase so that authentication and authorization are impossible to accidentally bypass. Services should receive a verified AuthContext parameter, not raw user IDs. If a service method doesn't receive an AuthContext, it should be a compile-time error.

5. Static Analysis Rules

Create custom ESLint or Biome rules that flag known anti-patterns:

pool.connect() without a corresponding release() in a finally block
fetch() retry loops without delay
addEventListener() / subscribe() without corresponding cleanup
Database queries that use req.query or req.params for identity

The Bottom Line

AI coding agents are productivity multipliers when used with discipline. They are liability multipliers when used without it.

The 7 patterns in this guide aren't edge cases. They are the most common, most damaging, and most predictable failure modes of AI-generated code. Every one of them is preventable with infrastructure: concurrency tests, connection monitoring, schema validation, auth-by-default architecture, and static analysis.

Your AI agent writes the first draft. Your engineering discipline writes the final one. The companies that master this distinction will ship faster and crash less. The ones that don't will keep debugging Saturday morning incidents caused by Friday evening deployments of AI code that "looked fine in the PR."

The code that looks correct is the most dangerous code of all.

⚡ Speed Tip: Read the original post on the Pockit Blog.

Tired of slow cloud tools? Pockit.tools runs entirely in your browser. Get the Extension now for instant, zero-latency access to essential dev tools.

Specification-Driven Development: How to Stop Vibe Coding and Actually Ship Production-Ready AI-Generated Code

HK Lee — Tue, 07 Apr 2026 13:39:00 +0000

You've seen the demos. An engineer types "build me a SaaS dashboard" into an AI coding agent, and in three minutes, a fully styled React app appears with auth, a database, and a payment flow. The crowd applauds. The tweet goes viral.

Then you try the same thing on your actual project — the one with 200 files, a custom auth layer, three external API integrations, and a monorepo structure — and the agent confidently rewrites your database schema, deletes a critical middleware, and introduces four security vulnerabilities before you can hit Ctrl+C.

This is the vibe coding gap. The distance between "works in a demo" and "works in production" isn't a tooling problem. It's a methodology problem. And the methodology that closes this gap has a name: Specification-Driven Development (SDD).

SDD isn't about writing more documentation. It's about giving AI agents exactly the constraints they need to operate reliably — through spec files, test-first loops, and a four-phase workflow that replaces "generate and pray" with "specify, design, task, implement." Teams adopting SDD consistently report 60-80% fewer AI-generated regressions and dramatically less time spent debugging code they didn't write.

This guide covers the full methodology: why vibe coding fails at scale, the four-phase SDD loop, how to structure spec files (CLAUDE.md, cursor rules, AGENTS.md), integrating TDD with AI agents, and production-grade patterns for orchestrating agents on real-world codebases.

Why Vibe Coding Breaks Down

Let's be precise about what fails and why. Vibe coding — the practice of prompting an AI with natural language and immediately deploying the output — works surprisingly well for greenfield prototypes. But it systematically fails on production codebases for four specific reasons:

1. The Context Collapse Problem

AI coding agents are stateless. Every new session starts from zero. Your 200-file project has implicit knowledge embedded everywhere: naming conventions, error handling patterns, the fact that userId in your auth layer is a UUID but user_id in your legacy API is a sequential integer. None of this exists in the agent's context unless you explicitly provide it.

Vibe coding provides no mechanism for this. You type a prompt, the agent generates code based on its training data, and the output follows the agent's default patterns — not yours. The result is code that "works" in isolation but creates subtle inconsistencies that compound into architectural drift.

Context Collapse in Practice:

  Your project:          What the agent generates:
  ─────────────          ──────────────────────────
  camelCase everywhere   snake_case in new files
  Zod for validation     Manual if-checks
  Custom error classes   Generic throw new Error()
  Repository pattern     Direct database calls
  UUID primary keys      Auto-increment integers

2. The Happy Path Bias

LLMs are trained predominantly on tutorial code, documentation examples, and Stack Overflow answers. This training data overwhelmingly demonstrates the "happy path" — when everything goes right. The result is AI-generated code that handles the primary flow gracefully but crumbles at boundaries:

Network timeouts? Not handled.
Concurrent requests to the same resource? Race conditions.
Database connection pool exhaustion? Crashes.
Malformed user input? Unvalidated.
Rate limits from external APIs? Ignored.

A 2025 study from Endor Labs found that 62% of AI-generated code contained security weaknesses or design flaws, with 44-49% of AI-suggested dependencies carrying known vulnerabilities. The Verizon DBIR 2025 report found third-party involvement in breaches doubled to 30%, underscoring the growing risk of unvetted AI-generated code entering software supply chains.

3. The Ownership Void

Vibe coding produces code that its supposed owner doesn't understand. The developer shipped it, but they can't explain the control flow, the error handling strategy, or why a particular library was chosen. When bugs appear in production at 2 AM, they face code written by an alien intelligence with different assumptions about how error states propagate.

This isn't a theoretical concern — it's the single most common complaint from engineering managers about AI-assisted development in 2026. The code works until it doesn't, and when it doesn't, nobody can fix it quickly because nobody understands it deeply.

4. The Compounding Technical Debt

Each vibe-coded session adds code that follows the agent's implicit patterns rather than the project's established ones. After ten sessions, you don't have one codebase — you have an accretion of ten slightly different coding styles, error handling approaches, and architectural assumptions. This technical debt compounds exponentially because each new AI session inherits the confusion of the previous ones.

The Four-Phase SDD Loop

Specification-Driven Development replaces the single "prompt → code" step with a four-phase loop that builds context before generating code. Each phase produces a document artifact that the AI agent can reference in subsequent phases.

The SDD Loop:

  Phase 1: REQUIREMENTS  ──→  requirements.md
       ↓                         "What are we building?"
  Phase 2: DESIGN        ──→  design.md
       ↓                         "How will we build it?"
  Phase 3: TASKS         ──→  tasks.md
       ↓                         "What exact steps in what order?"
  Phase 4: IMPLEMENTATION ──→  code + tests
       ↓                         "Build it, test it, verify it."
       └──── FEEDBACK ────→  Update specs, repeat.

Phase 1: Requirements

Start every feature by making the AI help you clarify what you're building. Don't ask for code — ask for a requirements analysis.

Prompt to your AI agent:

"I need to add a team invitation system to our SaaS app.
Before writing any code, create a requirements.md that covers:
- User stories for the invitation flow
- Edge cases (expired invites, duplicate emails, role conflicts)
- Security requirements (rate limiting, token validation)
- Integration points with our existing auth system
- What we are NOT building in this phase"

The "what we are NOT building" constraint is critical. Without explicit scope boundaries, AI agents tend to over-build — adding features that weren't requested, introducing premature abstractions, and expanding scope until the change touches half the codebase.

The output is a requirements.md file that both the human and the AI agent can reference. This document becomes the source of truth for the feature's scope.

Phase 2: Design

With requirements locked, translate them into technical decisions. Again, no code yet — just architecture.

Prompt to your AI agent:

"Read requirements.md. Now create a design.md that covers:
- Database schema changes (what tables/columns, with migration strategy)
- API endpoints (method, path, request/response shapes)
- Service layer architecture (what functions, what dependencies)
- Error handling strategy (what errors, what HTTP codes, what messages)
- State machine for invitation lifecycle (pending → accepted/expired/revoked)

Reference our existing patterns in src/services/ and src/api/.
Do NOT write implementation code."

This phase catches architectural mistakes before any code exists. The AI might propose a design that conflicts with your existing patterns. It's infinitely cheaper to fix a design document than to debug a half-implemented feature.

Phase 3: Tasks

Break the design into discrete, dependency-ordered implementation steps. Each task should be small enough that the AI agent can complete it in a single focused session.

Prompt to your AI agent:

"Read requirements.md and design.md. Create a tasks.md with:
- Numbered, ordered implementation steps
- Dependencies between steps (what must be done before what)
- Expected test coverage for each step
- Estimated complexity (S/M/L) for each step

Group tasks by: database → service layer → API → integration tests"

Example tasks.md output:

## Team Invitation System — Implementation Tasks

### Database Layer
- [x] Task 1 (S): Create `team_invitations` migration
  - Columns: id, team_id, email, role, token, status, expires_at
  - Tests: migration up/down, constraint validation

### Service Layer
- [ ] Task 2 (M): Implement `InvitationService.create()`
  - Depends on: Task 1
  - Validates: email format, duplicate check, team member limit
  - Tests: happy path, duplicate rejection, limit enforcement

- [ ] Task 3 (M): Implement `InvitationService.accept()`
  - Depends on: Task 2
  - Validates: token exists, not expired, not already accepted
  - Tests: valid acceptance, expired token, already-used token

### API Layer
- [ ] Task 4 (M): POST /api/teams/:id/invitations
  - Depends on: Task 2
  - Auth: team admin role required
  - Tests: 201 success, 403 non-admin, 409 duplicate, 429 rate limit

- [ ] Task 5 (M): POST /api/invitations/:token/accept
  - Depends on: Task 3
  - Auth: must be logged in, email must match invitation
  - Tests: 200 success, 404 invalid token, 410 expired

Phase 4: Implementation

Now — and only now — the AI writes code. But instead of a single massive prompt, you feed it one task at a time, with full context:

Prompt to your AI agent:

"Read requirements.md, design.md, and tasks.md.
Implement Task 2: InvitationService.create()

Requirements:
- Follow our existing service pattern in src/services/TeamService.ts
- Use Zod for input validation
- Use our custom AppError class for error handling
- Write tests first (test file, then implementation)
- Run tests after implementation to verify

Do NOT modify any existing files except to add imports.
Do NOT implement tasks 3-5 yet."

The constraints in this prompt are doing the heavy lifting. "Follow our existing pattern" prevents context collapse. "Write tests first" enforces TDD. "Do NOT modify existing files except imports" prevents the agent from "helpfully" refactoring unrelated code. "Do NOT implement tasks 3-5" prevents scope creep.

Structuring Your Spec Files

The SDD loop depends on persistent context — files that survive across sessions and tell every AI agent how to behave in your project. Here's how to structure them:

CLAUDE.md / Cursor Rules / AGENTS.md

These files serve the same purpose across different tools: they're the project's "briefing document" that the AI reads at the start of every session.

# CLAUDE.md (or .cursor/rules, or AGENTS.md)

## Project Overview
E-commerce SaaS platform built with Next.js 16, TypeScript, 
Drizzle ORM, PostgreSQL. Monorepo managed with Turborepo.

## Tech Stack
- Framework: Next.js 16 (App Router, Server Components)
- Language: TypeScript 6.0 (strict mode)
- Database: PostgreSQL 17 + Drizzle ORM
- Validation: Zod v3
- Styling: Tailwind CSS v4
- Testing: Vitest + Playwright
- Auth: Custom JWT + refresh token rotation

## Architecture Rules
1. All database access goes through repository classes in src/repositories/
2. Business logic lives in service classes in src/services/
3. API routes are thin controllers that call services
4. All inputs are validated with Zod schemas defined in src/schemas/
5. Errors use custom AppError class (see src/lib/errors.ts)
6. All IDs are UUIDs generated with crypto.randomUUID()

## Coding Conventions
- Use named exports, not default exports
- Use explicit return types on all public functions
- Error messages follow the pattern: "[Entity].[action] failed: [reason]"
- File naming: kebab-case for files, PascalCase for classes
- Imports: group by external → internal → types

## Common Commands
- `pnpm test` — Run all tests
- `pnpm test:watch` — Watch mode
- `pnpm db:migrate` — Run pending migrations
- `pnpm lint` — Biome lint + format check

## Critical Warnings
- NEVER use `any` type. Use `unknown` with type narrowing.
- NEVER use default exports. Named exports only.
- NEVER modify migration files. Create new ones for changes.
- NEVER commit .env files. Use .env.example for documentation.

Key Principle: Route, Don't Dump

Keep your root spec file under 200-300 lines. It should tell the agent what to do and where to find more information, not contain every rule in the project.

## Reference Documents
- Architecture decisions: /docs/architecture.md
- API design conventions: /docs/api-conventions.md
- Database schema: /docs/schema.md
- Feature specs: /docs/specs/[feature-name].md

The AI agent reads the root file, then reads referenced documents on-demand when working on relevant areas. This is progressive disclosure — the same UX principle that makes good software, applied to AI context management.

Per-Directory Rules

Many AI tools support directory-scoped rules. Use these for domain-specific constraints:

# src/services/.rules (or src/services/.cursorrules)

## Service Layer Rules
- Every service method must be async
- Services receive dependencies through constructor injection
- Services must not import from src/api/ (no circular deps)
- Every public method must have a corresponding test
- Use transactions for multi-table operations
- Log entry and exit of critical operations using logger.info()

TDD with AI Agents: The Red-Green-Refactor Loop

Test-Driven Development isn't just compatible with AI agents — it's the ideal workflow for them. Tests provide the one thing AI agents desperately need: an objective, verifiable definition of "correct."

Why TDD Works Better With AI Than Without

Without tests, you ask the AI to generate code and then manually review it for correctness. This is cognitively exhausting and error-prone — you're reading code written by an alien intelligence and trying to spot bugs in unfamiliar patterns.

With TDD, you define correctness first, then let the AI generate code until the tests pass. You're not reviewing implementation details — you're reviewing outcomes. The test suite is your automated verifier.

Traditional Flow (fragile):
  Prompt → Code → Manual Review → "Looks right?" → Ship → Bug

TDD Flow (reliable):
  Spec → Test (failing) → Prompt AI → Code → Run Tests → Pass? → Ship
                                          ↓
                                       Fail → AI iterates automatically

The Spec → Test → Implement Pattern

Here's the practical workflow for TDD with an AI agent:

Step 1: Write the test spec (human-driven)

// __tests__/services/invitation-service.test.ts
import { describe, it, expect, beforeEach } from 'vitest';
import { InvitationService } from '@/services/invitation-service';

describe('InvitationService.create', () => {
  it('should create an invitation with a valid email and role', async () => {
    const result = await service.create({
      teamId: 'team-uuid-1',
      email: 'new@example.com',
      role: 'member',
      invitedBy: 'admin-uuid-1',
    });

    expect(result.id).toBeDefined();
    expect(result.status).toBe('pending');
    expect(result.token).toHaveLength(64);
    expect(result.expiresAt).toBeInstanceOf(Date);
  });

  it('should reject duplicate invitations for the same email', async () => {
    await service.create({ teamId: 'team-uuid-1', email: 'dup@example.com', role: 'member', invitedBy: 'admin-uuid-1' });

    await expect(
      service.create({ teamId: 'team-uuid-1', email: 'dup@example.com', role: 'member', invitedBy: 'admin-uuid-1' })
    ).rejects.toThrow('Invitation.create failed: duplicate invitation');
  });

  it('should enforce team member limit', async () => {
    // Assume team already has max members
    await expect(
      service.create({ teamId: 'full-team', email: 'extra@example.com', role: 'member', invitedBy: 'admin-uuid-1' })
    ).rejects.toThrow('Invitation.create failed: team member limit reached');
  });

  it('should set expiration to 7 days from creation', async () => {
    const before = new Date();
    const result = await service.create({
      teamId: 'team-uuid-1',
      email: 'timed@example.com',
      role: 'member',
      invitedBy: 'admin-uuid-1',
    });
    const after = new Date();

    const sevenDaysMs = 7 * 24 * 60 * 60 * 1000;
    expect(result.expiresAt.getTime()).toBeGreaterThanOrEqual(before.getTime() + sevenDaysMs);
    expect(result.expiresAt.getTime()).toBeLessThanOrEqual(after.getTime() + sevenDaysMs);
  });
});

Step 2: Let the AI implement (agent-driven)

Prompt:

"The test file at __tests__/services/invitation-service.test.ts
defines the expected behavior. Implement InvitationService.create()
in src/services/invitation-service.ts to make all tests pass.

Follow the service pattern established in src/services/team-service.ts.
Use Zod for input validation (schema in src/schemas/invitation.ts).
Run `pnpm test __tests__/services/invitation-service.test.ts` after
implementation. Iterate until all tests pass."

The agent generates code, runs the tests, sees failures, and iterates — automatically. This is the Red-Green loop, but the AI is the one cycling through it. You defined the "what" (tests), and the AI figures out the "how" (implementation).

Step 3: Human review (outcome-focused)

Once tests pass, review the implementation for:

Does it follow project patterns? (Check against CLAUDE.md)
Are there performance concerns? (N+1 queries, missing indexes)
Are there security concerns? (Input validation, auth checks)

You're reviewing with purpose, not scanning line-by-line through unfamiliar code hoping to spot a bug.

Failure Patterns and How SDD Prevents Them

Let's map the most common AI coding failures to specific SDD mechanisms that prevent them:

Failure Pattern	Root Cause	SDD Prevention
Context collapse	Agent doesn't know project conventions	CLAUDE.md with architecture rules and coding conventions
Happy path bias	Agent doesn't generate error handling	Tests explicitly define error scenarios (Step 1 of TDD)
Scope creep	Agent over-builds or touches unrelated code	Task breakdown with explicit "do NOT" constraints
Architectural drift	Agent uses its own patterns, not yours	Design document + per-directory rules
Regression	New code breaks existing functionality	Test-first workflow catches regressions immediately
Security holes	Agent doesn't consider threat model	Security requirements in Phase 1 + security-focused test cases
Ownership void	Developer ships code they don't understand	Design review in Phase 2 forces understanding before implementation
Technical debt	Inconsistent patterns across sessions	CLAUDE.md ensures consistency across every session

Production Workflow: Putting It All Together

Here's a complete, real-world SDD workflow from feature request to merge:

1. Feature Brief

## Feature: Team Role Management

Users need the ability to change team member roles (admin → member,
member → admin) and remove members from teams. Only team admins
should be able to perform these actions.

2. AI-Assisted Requirements

Prompt: "Read the feature brief above and our existing auth system 
in src/services/auth-service.ts. Generate requirements.md covering 
user stories, edge cases, security requirements, and what we're NOT 
building. Consider: what happens when the last admin tries to 
change their own role?"

3. AI-Assisted Design

Prompt: "Read requirements.md. Create design.md with database schema 
changes, API endpoints, service methods, and a state diagram for role 
transitions. Follow patterns in our existing codebase. Flag any 
design decisions that need human review."

4. Human Review Checkpoint

This is the critical human-in-the-loop moment. Review the design document for:

Does the database schema make sense?
Are the API endpoints RESTful and consistent with our existing API?
Did the AI catch the "last admin" edge case?
Are there security implications the AI missed?

Make corrections to the design document before any code is written.

5. AI-Assisted Task Breakdown

Prompt: "Read requirements.md and design.md. Create tasks.md with 
ordered, dependency-aware implementation steps. Each task should 
include its test coverage requirements."

6. Iterative Implementation

Prompt: "Implement Task 1 from tasks.md. Write tests first, then 
implementation. Run tests to verify. Mark the task as complete in 
tasks.md when done. Do NOT proceed to Task 2."

Repeat for each task. After each task, the agent updates tasks.md to reflect progress. You can review each task independently, making code review manageable rather than facing a single massive PR.

7. Integration Verification

Prompt: "All tasks in tasks.md are complete. Run the full test suite 
with `pnpm test`. If there are failures, fix them. Then run 
`pnpm lint` and fix any issues."

Scaling SDD: Team Patterns

Shared Spec Repository

For teams, maintain spec files in version control alongside code:

project/
├── .claude/
│   └── rules/
│       ├── general.md         # Project-wide rules
│       ├── api-conventions.md # API-specific rules
│       └── testing.md         # Testing standards
├── docs/
│   └── specs/
│       ├── team-invitations/
│       │   ├── requirements.md
│       │   ├── design.md
│       │   └── tasks.md
│       └── role-management/
│           ├── requirements.md
│           ├── design.md
│           └── tasks.md
├── CLAUDE.md                  # Root briefing (routes to .claude/rules/)
├── AGENTS.md                  # Universal rules for all AI tools
└── src/

PR Template for AI-Assisted Work

Require PRs that include AI-generated code to reference their spec:

## PR Checklist (AI-Assisted)

- [ ] Feature spec exists in /docs/specs/[feature-name]/
- [ ] requirements.md reviewed and approved by tech lead
- [ ] design.md reviewed and approved by tech lead
- [ ] All tasks in tasks.md are completed and checked off
- [ ] Test coverage: all paths from requirements are tested
- [ ] No modifications to files outside the feature scope
- [ ] `pnpm test` passes
- [ ] `pnpm lint` passes

Metrics That Matter

Track these metrics to measure the effectiveness of SDD adoption:

Metric	Before SDD	After SDD	Why It Matters
AI-generated regressions per sprint	8-15	1-3	Direct measure of code quality
Time to debug AI code	2-4 hours per bug	15-30 min per bug	Ownership void reduction
PR review time	45-60 min (reading unfamiliar code)	15-20 min (checking against spec)	Review efficiency
Feature delivery time	Same (faster coding, slower debugging)	30-40% faster net	Real productivity gain
Security issues caught in review	20-30% catch rate	80-90% catch rate (tests + spec)	Security improvement

Common Objections (and Rebuttals)

"SDD is just more documentation overhead"

No. SDD documents are generated by the AI, reviewed by you, and consumed by the AI. The total documentation effort is 10-15 minutes per feature for the human (review time). The alternative is 2-4 hours per bug debugging code you don't understand.

"My project is too small for this"

If your project is small enough that you can hold the entire codebase in your head, vibe coding might work fine. SDD becomes essential when the codebase exceeds your working memory — roughly 10-20 files with interconnected logic.

"Tests slow down development"

Tests slow down the first hour. They accelerate every subsequent hour. With AI agents, this trade-off is even more favorable: the agent writes the implementation to match your tests, often getting it right on the first or second attempt. The test-writing phase takes 5-10 minutes for humans; the implementation phase takes the AI 30-60 seconds.

"I just need to write better prompts"

Prompt quality matters, but prompts are ephemeral. They disappear when the session ends. SDD spec files persist, improve over time, and work across different AI tools. Your CLAUDE.md doesn't just help today's session — it helps every session, every team member, and every AI tool.

The Maturity Model

Teams adopting SDD typically progress through three stages:

Stage 1: Reactive (Most teams today)

Use AI for code generation with ad-hoc prompts
Debug frequently, review painfully
No persistent context files
Each session starts from zero

Stage 2: Structured (SDD adoption)

CLAUDE.md / AGENTS.md in place
Four-phase loop for complex features
TDD integrated with AI workflows
Per-directory rules for domain-specific constraints
Design review before implementation

Stage 3: Systematic (Full SDD maturity)

Spec repository maintained alongside code
AI agents update spec documents as they work
Metrics tracked and optimized
Onboarding new team members via spec docs
Cross-tool consistency (Cursor, Claude Code, Copilot all follow same specs)

Most teams can reach Stage 2 in a single sprint. Stage 3 develops naturally over 2-3 months as spec files accumulate and patterns stabilize.

The Engineering Reality

AI coding agents are the most powerful tools we've ever had for producing code fast. They're also the most powerful tools we've ever had for producing bugs fast. The difference between the two outcomes isn't the model, the prompt, or the tool — it's the methodology.

Vibe coding treats AI as a replacement for engineering discipline. SDD treats AI as an amplifier for engineering discipline. The four-phase loop — requirements, design, tasks, implementation — isn't bureaucracy. It's the same engineering process that senior developers have always followed, now made explicit so that AI agents can follow it too.

The spec files take 15 minutes to set up. The TDD loop takes zero additional time (the AI writes the implementation). The design review takes 10 minutes per feature. In exchange, you get code that follows your patterns, handles your edge cases, passes your tests, and can be maintained by your team.

Your AI agent is the most productive junior developer you've ever hired. SDD is how you onboard them properly — with clear requirements, documented patterns, well-defined tasks, and verifiable acceptance criteria. Skip the onboarding, and you get the chaos that everyone complains about. Invest in the onboarding, and you get the productivity multiplier that everyone dreams about.

The choice isn't whether to use AI coding agents. The choice is whether to use them with engineering discipline or without it. SDD makes that choice obvious.

💡 Note: This article was originally published on the Pockit Blog.

Check out Pockit.tools for 60+ free developer utilities. For faster access, add it to Chrome and use JSON Formatter & Diff Checker directly from your toolbar.

DPoP Deep Dive: The Complete Guide to Making Stolen OAuth Tokens Useless

HK Lee — Mon, 06 Apr 2026 14:18:00 +0000

Your access tokens are bearer tokens. That means anyone who has the token string — whether they stole it from a log file, a compromised CDN, an XSS vulnerability, or a man-in-the-middle attack — can use it exactly as if they were your legitimate user. The token doesn't know who's holding it. It doesn't care.

This is the foundational security weakness of modern OAuth deployments, and it's been an open secret for a decade. Every security audit flags it. Every threat model acknowledges it. And until recently, the practical mitigations were either too complex (mTLS for browser clients?) or too limited (short token lifetimes just reduce the blast radius, they don't prevent the blast).

DPoP — Demonstrating Proof-of-Possession — changes the equation. Defined in RFC 9449 and now one of the two approved mechanisms (alongside mTLS) for the mandatory sender-constrained tokens in FAPI 2.0, DPoP cryptographically binds tokens to a client-held private key. Possession of the token alone is no longer sufficient. Every API request must also include a fresh cryptographic proof that the caller holds the original private key. Stolen tokens become inert strings.

This guide covers everything: why bearer tokens are fundamentally broken, how DPoP's cryptography works, full TypeScript implementations for both client and server, nonce handling for replay protection, key storage strategies across platforms, and a practical migration path from bearer to sender-constrained tokens.

The Bearer Token Problem

Bearer tokens work like cash. Whoever holds the bill can spend it. There's no ID check, no PIN, no biometric. This was a deliberate design choice — RFC 6750 explicitly defines a bearer token as one where "any party in possession of a bearer token can use it to get access to the associated resources (without demonstrating possession of a cryptographic key)."

That simplicity made OAuth 2.0 adoption fast. It also made token theft devastatingly effective.

How Tokens Get Stolen

The attack surface is wide:

Token Leak Vectors:
  1. XSS → document.cookie or localStorage read
  2. Log aggregation → tokens in URL params or headers appear in logs
  3. CDN/Proxy → intermediate services cache or log Authorization headers
  4. Browser extensions → malicious extensions read request headers
  5. Man-in-the-middle → compromised TLS termination
  6. Dependency supply chain → compromised npm package exfiltrates tokens

The 2025 Verizon DBIR reports that token theft accounts for 31% of MFA bypass techniques in enterprise environments, while stolen credentials remain present in 22% of all breaches. Short-lived tokens reduce the window, but a 15-minute access token is still 15 minutes of full API access for an attacker. Refresh token rotation helps, but if the refresh token itself is stolen before rotation, the attacker has long-term access.

Why Existing Mitigations Fall Short

Mitigation	Limitation
Short token lifetimes	Reduces window but doesn't prevent theft. 5-minute tokens still give 5 minutes of access.
Refresh token rotation	Race condition: attacker uses token before rotation. Also fails if refresh token is stolen at issuance.
Token binding (RFC 8471)	Never achieved broad browser support. Effectively dead.
mTLS (RFC 8705)	Excellent security but impractical for browser clients. Certificate management overhead is massive.
HttpOnly cookies	Protects against XSS but introduces CSRF risk and doesn't work for cross-origin APIs.

DPoP occupies the sweet spot: application-layer security that works in browsers, mobile apps, and server-to-server flows without requiring TLS client certificates.

How DPoP Works: The Cryptography

DPoP is conceptually simple: the client generates a key pair, proves it holds the private key with every request, and the server binds the token to that key. Here's the flow:

DPoP Flow:
  1. Client generates asymmetric key pair (e.g., EC P-256)
  2. Client requests token from Authorization Server
     → includes DPoP proof JWT signed with private key
     → proof contains: HTTP method, target URL, unique ID, timestamp
  3. Authorization Server validates proof, issues token
     → token contains 'cnf' claim with public key thumbprint
  4. Client calls Resource Server
     → sends token + NEW DPoP proof (signed, fresh, with token hash)
  5. Resource Server validates:
     → proof signature matches the key bound to token
     → HTTP method and URL match the actual request
     → proof is fresh (timestamp + optional nonce)
     → token hash matches the presented token

The critical insight: even if an attacker steals the access token, they cannot generate valid DPoP proofs without the private key. The token is cryptographically useless to anyone except the original client.

The DPoP Proof JWT

Every request includes a DPoP proof — a signed JWT with a specific structure:

DPoP Proof JWT Structure:

HEADER:
{
  "typ": "dpop+jwt",        // MUST be exactly this
  "alg": "ES256",           // Asymmetric algorithm (ES256, RS256, etc.)
  "jwk": {                  // Public key (for token requests)
    "kty": "EC",
    "crv": "P-256",
    "x": "...",
    "y": "..."
  }
}

PAYLOAD:
{
  "htm": "POST",            // HTTP method of the request
  "htu": "https://auth.example.com/token",  // Target URL
  "iat": 1712400000,        // Issued at (Unix timestamp)
  "jti": "unique-id-abc123", // Unique ID (prevents replay)
  "ath": "fUHyO2r2Z3DZ..."  // Access token hash (for resource requests)
  "nonce": "server-nonce"   // Server-provided nonce (if required)
}

Two critical claims differentiate DPoP from regular JWTs:

ath (Access Token Hash): Present only when calling resource servers. It's the base64url-encoded SHA-256 hash of the access token. This binds the proof to a specific token, preventing proof reuse across different tokens.
nonce: An opaque value provided by the server in a DPoP-Nonce response header. Enables server-side replay protection beyond the jti uniqueness check.

Full Implementation: Client Side

Let's build a complete DPoP client in TypeScript. We'll use the Web Crypto API for key generation and the jose library for JWT operations.

Key Pair Generation

// dpop-client.ts
import { SignJWT, exportJWK, calculateJwkThumbprint } from 'jose';
import { v4 as uuidv4 } from 'uuid';

interface DPoPKeyPair {
  privateKey: CryptoKey;
  publicKey: CryptoKey;
  publicJwk: JsonWebKey;
  thumbprint: string;
}

async function generateDPoPKeyPair(): Promise<DPoPKeyPair> {
  // Generate a non-extractable EC P-256 key pair
  const keyPair = await crypto.subtle.generateKey(
    {
      name: 'ECDSA',
      namedCurve: 'P-256',
    },
    false, // non-extractable: private key cannot be exported
    ['sign', 'verify']
  );

  const publicJwk = await exportJWK(keyPair.publicKey);
  const thumbprint = await calculateJwkThumbprint(
    publicJwk as Parameters<typeof calculateJwkThumbprint>[0],
    'sha256'
  );

  return {
    privateKey: keyPair.privateKey,
    publicKey: keyPair.publicKey,
    publicJwk,
    thumbprint,
  };
}

The false parameter in generateKey is critical — it marks the private key as non-extractable. Even JavaScript code running in the same context cannot read the raw key material. The key exists only inside the browser's crypto engine.

Creating DPoP Proofs

interface DPoPProofOptions {
  keyPair: DPoPKeyPair;
  method: string;
  url: string;
  accessToken?: string;  // Required for resource server requests
  nonce?: string;        // Server-provided nonce
}

async function createDPoPProof(options: DPoPProofOptions): Promise<string> {
  const { keyPair, method, url, accessToken, nonce } = options;

  // Build payload
  const payload: Record<string, unknown> = {
    htm: method.toUpperCase(),
    htu: url,
    jti: uuidv4(),
    iat: Math.floor(Date.now() / 1000),
  };

  // Add access token hash for resource server requests
  if (accessToken) {
    const encoder = new TextEncoder();
    const tokenBytes = encoder.encode(accessToken);
    const hashBuffer = await crypto.subtle.digest('SHA-256', tokenBytes);
    const hashArray = new Uint8Array(hashBuffer);
    payload.ath = base64urlEncode(hashArray);
  }

  // Add server-provided nonce if available
  if (nonce) {
    payload.nonce = nonce;
  }

  // Sign the proof
  const proof = await new SignJWT(payload)
    .setProtectedHeader({
      typ: 'dpop+jwt',
      alg: 'ES256',
      jwk: keyPair.publicJwk,
    })
    .sign(keyPair.privateKey);

  return proof;
}

function base64urlEncode(buffer: Uint8Array): string {
  const binary = String.fromCharCode(...buffer);
  return btoa(binary)
    .replace(/\+/g, '-')
    .replace(/\//g, '_')
    .replace(/=+$/, '');
}

The DPoP-Aware HTTP Client

Here's the crucial piece — an HTTP client wrapper that handles the full DPoP lifecycle including automatic nonce retry:

class DPoPClient {
  private keyPair: DPoPKeyPair | null = null;
  private nonces: Map<string, string> = new Map(); // origin → nonce

  async initialize(): Promise<void> {
    this.keyPair = await generateDPoPKeyPair();
  }

  async fetch(
    url: string,
    options: RequestInit & { accessToken?: string } = {}
  ): Promise<Response> {
    if (!this.keyPair) throw new Error('DPoP client not initialized');

    const method = options.method || 'GET';
    const origin = new URL(url).origin;

    // Generate DPoP proof
    const proof = await createDPoPProof({
      keyPair: this.keyPair,
      method,
      url,
      accessToken: options.accessToken,
      nonce: this.nonces.get(origin),
    });

    // Build headers
    const headers = new Headers(options.headers);
    headers.set('DPoP', proof);

    if (options.accessToken) {
      headers.set('Authorization', `DPoP ${options.accessToken}`);
    }

    const response = await fetch(url, { ...options, headers });

    // Handle nonce challenges
    const newNonce = response.headers.get('DPoP-Nonce');
    if (newNonce) {
      this.nonces.set(origin, newNonce);

      // If we got a 401 with use_dpop_nonce error, retry with the new nonce
      if (response.status === 401 || response.status === 400) {
        const wwwAuth = response.headers.get('WWW-Authenticate') || '';
        if (wwwAuth.includes('use_dpop_nonce')) {
          return this.fetch(url, options); // Retry with stored nonce
        }
      }
    }

    return response;
  }
}

Token Request with DPoP

async function requestToken(
  client: DPoPClient,
  tokenEndpoint: string,
  authCode: string,
  codeVerifier: string
): Promise<{ access_token: string; refresh_token: string }> {
  const response = await client.fetch(tokenEndpoint, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'authorization_code',
      code: authCode,
      code_verifier: codeVerifier,
      client_id: 'my-spa-client',
      redirect_uri: 'https://app.example.com/callback',
    }).toString(),
  });

  if (!response.ok) {
    throw new Error(`Token request failed: ${response.status}`);
  }

  return response.json();
}

Notice the Authorization scheme changes from Bearer to DPoP — this signals the resource server to expect and validate a DPoP proof.

Full Implementation: Server Side

The server side has two responsibilities: binding tokens to keys during issuance, and validating proofs during resource access.

Token Issuance (Authorization Server)

import { jwtVerify, importJWK, calculateJwkThumbprint } from 'jose';

interface DPoPProofPayload {
  htm: string;
  htu: string;
  jti: string;
  iat: number;
  ath?: string;
  nonce?: string;
}

async function validateDPoPProof(
  proofJwt: string,
  expectedMethod: string,
  expectedUrl: string,
  expectedNonce?: string,
  accessToken?: string
): Promise<{ thumbprint: string }> {
  // 1. Decode header WITHOUT verifying yet (to extract the public key)
  const [headerB64] = proofJwt.split('.');
  const header = JSON.parse(atob(headerB64));

  // 2. Validate header
  if (header.typ !== 'dpop+jwt') {
    throw new Error('Invalid typ: must be dpop+jwt');
  }
  if (!header.jwk) {
    throw new Error('Missing jwk in header');
  }
  if (header.alg === 'none' || header.alg.startsWith('HS')) {
    throw new Error('Symmetric algorithms not allowed for DPoP');
  }

  // 3. Import the public key and verify signature
  const publicKey = await importJWK(header.jwk, header.alg);
  const { payload } = await jwtVerify(proofJwt, publicKey, {
    typ: 'dpop+jwt',
    maxTokenAge: '60s', // Reject proofs older than 60 seconds
  });

  const claims = payload as unknown as DPoPProofPayload;

  // 4. Validate HTTP method and URL
  if (claims.htm !== expectedMethod) {
    throw new Error(`htm mismatch: expected ${expectedMethod}, got ${claims.htm}`);
  }
  if (claims.htu !== expectedUrl) {
    throw new Error(`htu mismatch: expected ${expectedUrl}, got ${claims.htu}`);
  }

  // 5. Validate jti uniqueness (check against a replay cache)
  const isReplay = await checkAndStoreJti(claims.jti, 300); // 5-min window
  if (isReplay) {
    throw new Error('DPoP proof replay detected');
  }

  // 6. Validate nonce if required
  if (expectedNonce && claims.nonce !== expectedNonce) {
    throw new Error('Invalid or missing DPoP nonce');
  }

  // 7. Validate access token hash (for resource server requests)
  if (accessToken) {
    const expectedAth = await computeAth(accessToken);
    if (claims.ath !== expectedAth) {
      throw new Error('Access token hash mismatch');
    }
  }

  // 8. Compute JWK thumbprint for token binding
  const thumbprint = await calculateJwkThumbprint(header.jwk, 'sha256');

  return { thumbprint };
}

async function computeAth(accessToken: string): Promise<string> {
  const encoder = new TextEncoder();
  const hashBuffer = await crypto.subtle.digest(
    'SHA-256',
    encoder.encode(accessToken)
  );
  return base64urlEncode(new Uint8Array(hashBuffer));
}

// Simple Redis-based JTI replay cache
async function checkAndStoreJti(
  jti: string,
  windowSeconds: number
): Promise<boolean> {
  const key = `dpop:jti:${jti}`;
  const exists = await redis.exists(key);
  if (exists) return true;
  await redis.setex(key, windowSeconds, '1');
  return false;
}

Token with Confirmation Claim

When the authorization server issues a DPoP-bound token, it includes a cnf (confirmation) claim containing the JWK thumbprint:

async function issueToken(
  userId: string,
  dpopThumbprint: string,
  scopes: string[]
): Promise<string> {
  const token = await new SignJWT({
    sub: userId,
    scope: scopes.join(' '),
    cnf: {
      jkt: dpopThumbprint, // JWK Thumbprint confirmation
    },
    token_type: 'DPoP',
  })
    .setProtectedHeader({ alg: 'RS256' })
    .setIssuedAt()
    .setExpirationTime('15m')
    .setIssuer('https://auth.example.com')
    .setAudience('https://api.example.com')
    .sign(serverPrivateKey);

  return token;
}

Resource Server Validation

// middleware/dpop-validator.ts
import { jwtVerify, decodeJwt } from 'jose';

async function validateDPoPRequest(req: Request): Promise<void> {
  // 1. Extract the DPoP proof from headers
  const dpopProof = req.headers.get('DPoP');
  if (!dpopProof) {
    throw new DPoPError(401, 'Missing DPoP proof header');
  }

  // 2. Extract the access token
  const authHeader = req.headers.get('Authorization');
  if (!authHeader?.startsWith('DPoP ')) {
    throw new DPoPError(401, 'Invalid authorization scheme, expected DPoP');
  }
  const accessToken = authHeader.slice(5);

  // 3. Decode the access token to get the bound thumbprint
  const tokenClaims = decodeJwt(accessToken);
  const boundThumbprint = (tokenClaims.cnf as { jkt: string })?.jkt;
  if (!boundThumbprint) {
    throw new DPoPError(401, 'Token is not DPoP-bound (missing cnf.jkt)');
  }

  // 4. Validate the DPoP proof
  const requestUrl = new URL(req.url).origin + new URL(req.url).pathname;
  const { thumbprint } = await validateDPoPProof(
    dpopProof,
    req.method,
    requestUrl,
    getCurrentNonce(),  // Server's current nonce
    accessToken
  );

  // 5. Verify the proof was signed by the key bound to the token
  if (thumbprint !== boundThumbprint) {
    throw new DPoPError(401, 'DPoP proof key does not match token binding');
  }
}

class DPoPError extends Error {
  constructor(
    public status: number,
    message: string
  ) {
    super(message);
  }
}

Nonce Management for Replay Protection

The jti claim provides client-side uniqueness, but a determined attacker with access to the proof in transit could replay it within the time window. Server-provided nonces add a second layer:

How Nonces Work

Nonce Challenge Flow:

  Client → Resource Server: DPoP proof (no nonce)
  Resource Server → Client: 401 + DPoP-Nonce: "abc123"
  Client → Resource Server: DPoP proof (nonce: "abc123")
  Resource Server → Client: 200 OK + DPoP-Nonce: "def456"
  Client → Resource Server: DPoP proof (nonce: "def456")
  ...continues, nonce rotates with each response...

Server-Side Nonce Implementation

class NonceManager {
  private currentNonce: string;
  private previousNonce: string | null = null;
  private rotationInterval: NodeJS.Timeout;

  constructor(rotationSeconds: number = 30) {
    this.currentNonce = this.generate();

    // Rotate nonces periodically
    this.rotationInterval = setInterval(() => {
      this.previousNonce = this.currentNonce;
      this.currentNonce = this.generate();
    }, rotationSeconds * 1000);
  }

  private generate(): string {
    const bytes = new Uint8Array(32);
    crypto.getRandomValues(bytes);
    return base64urlEncode(bytes);
  }

  getCurrent(): string {
    return this.currentNonce;
  }

  isValid(nonce: string): boolean {
    // Accept current or previous nonce (grace period during rotation)
    return nonce === this.currentNonce || 
           nonce === this.previousNonce;
  }
}

// Express middleware
function dpopNonceMiddleware(nonceManager: NonceManager) {
  return (req: Request, res: Response, next: NextFunction) => {
    // Always include the current nonce in responses
    res.setHeader('DPoP-Nonce', nonceManager.getCurrent());
    next();
  };
}

Important Nonce Rules

Nonces are per-server. Authorization server nonces and resource server nonces are completely separate. Never reuse a nonce from one server when talking to another.
Nonces are opaque. Clients must not parse, decode, or interpret nonce values. Treat them as opaque strings.
Accept the previous nonce. During rotation, accept both the current and the immediately previous nonce to avoid breaking in-flight requests.
Always send the nonce header. Include DPoP-Nonce in every response, even successful ones. This lets clients pre-populate the nonce for the next request without a failed roundtrip.

Key Storage Strategies

The security of DPoP depends entirely on the private key remaining un-extractable. Different platforms require different strategies:

Browser (SPA)

// Use IndexedDB for persistent key storage
async function persistKeyPair(keyPair: CryptoKeyPair): Promise<void> {
  const db = await openDB('dpop-keys', 1, {
    upgrade(db) {
      db.createObjectStore('keys');
    },
  });

  // CryptoKey objects can be stored directly in IndexedDB
  // They remain non-extractable even when persisted
  await db.put('keys', keyPair, 'dpop-keypair');
}

async function loadKeyPair(): Promise<CryptoKeyPair | null> {
  const db = await openDB('dpop-keys', 1);
  return db.get('keys', 'dpop-keypair');
}

Key property: CryptoKey objects stored in IndexedDB retain their extractable: false property. The raw key material is never exposed to JavaScript, even across page reloads.

Mobile (React Native / Native)

Platform	Storage	Security Level
iOS	Secure Enclave (Keychain)	Hardware-backed, tamper-proof
Android	Android Keystore (StrongBox if available)	Hardware-backed on supported devices
React Native	`react-native-keychain` + platform-specific backing	Depends on underlying platform

Server-to-Server

For backend services, the key pair can be loaded from environment variables or a key management service (KMS):

// Use environment variable or KMS
const privateKey = await importPKCS8(
  process.env.DPOP_PRIVATE_KEY!,
  'ES256'
);

Migration: Bearer to DPoP

You cannot flip a switch and require DPoP from all clients overnight. Here's a phased approach:

Phase 1: Dual Support

Accept both bearer and DPoP tokens. New clients use DPoP; existing clients continue with bearer tokens.

async function validateRequest(req: Request): Promise<TokenClaims> {
  const authHeader = req.headers.get('Authorization');

  if (authHeader?.startsWith('DPoP ')) {
    // DPoP flow: validate proof + token binding
    await validateDPoPRequest(req);
    return decodeAndVerifyToken(authHeader.slice(5));
  }

  if (authHeader?.startsWith('Bearer ')) {
    // Legacy bearer flow: still accepted
    metrics.increment('auth.bearer.used'); // Track for deprecation
    return decodeAndVerifyToken(authHeader.slice(7));
  }

  throw new Error('Missing or invalid Authorization header');
}

Phase 2: DPoP Preferred

Issue DPoP-bound tokens by default. Log bearer token usage for monitoring.

// Authorization server token endpoint
app.post('/token', async (req, res) => {
  const dpopProof = req.headers.get('DPoP');

  if (dpopProof) {
    // Client supports DPoP: issue sender-constrained token
    const { thumbprint } = await validateDPoPProof(dpopProof, 'POST', tokenUrl);
    const token = await issueToken(userId, thumbprint, scopes);
    res.json({ access_token: token, token_type: 'DPoP' });
  } else {
    // Legacy client: issue bearer token with deprecation warning
    const token = await issueBearerToken(userId, scopes);
    res.setHeader('Deprecation', 'true');
    res.json({ access_token: token, token_type: 'bearer' });
  }
});

Phase 3: DPoP Required

After monitoring confirms low bearer usage, enforce DPoP for all clients.

app.post('/token', async (req, res) => {
  const dpopProof = req.headers.get('DPoP');

  if (!dpopProof) {
    return res.status(400).json({
      error: 'invalid_dpop_proof',
      error_description: 'DPoP proof required. Bearer tokens are no longer accepted.',
    });
  }

  const { thumbprint } = await validateDPoPProof(dpopProof, 'POST', tokenUrl);
  const token = await issueToken(userId, thumbprint, scopes);
  res.json({ access_token: token, token_type: 'DPoP' });
});

Migration Metrics

Metric	Target	What It Tells You
DPoP adoption rate	>90% before Phase 3	Overall migration progress
Bearer token usage	Declining to <5%	Whether legacy clients are updating
Nonce retry rate	<10% of requests	If nonce rotation is too aggressive
Proof validation failures	<0.1%	Clock skew or implementation bugs
Key rotation events	Tracked per client	Key lifecycle management health

DPoP vs. Other Token Binding Approaches

Feature	Bearer Token	DPoP (RFC 9449)	mTLS (RFC 8705)	Token Binding (RFC 8471)
Token theft protection	❌ None	✅ Cryptographic binding	✅ TLS binding	✅ TLS binding
Browser support	✅ Universal	✅ Web Crypto API	❌ No client cert UI	❌ Abandoned by browsers
Mobile support	✅ Universal	✅ Platform crypto	⚠️ Complex cert mgmt	❌ Not implemented
Implementation complexity	⭐ Simple	⭐⭐ Moderate	⭐⭐⭐ High	N/A (Dead)
Performance overhead	None	~2ms per request (signing)	TLS handshake cost	N/A
Works with CDN/proxy	✅ Yes	✅ Yes (app layer)	⚠️ TLS termination issues	❌ TLS termination breaks it
Standard maturity	RFC 6750 (2012)	RFC 9449 (2023)	RFC 8705 (2020)	Abandoned

DPoP wins for web and mobile because it operates at the application layer. No special TLS configuration, no client certificates, no browser UI changes. Just cryptography in JavaScript.

Production Security Checklist

Client-Side

Use non-extractable keys. Always set extractable: false in crypto.subtle.generateKey(). This prevents any JavaScript code — including injected XSS payloads — from reading the raw private key.
Rotate keys periodically. Generate a new key pair when users re-authenticate or sessions start. Old token bindings become invalid automatically.
Use IndexedDB, not localStorage. CryptoKey objects can only be stored in IndexedDB. localStorage can only store strings, which would require extracting the key — defeating the purpose.
Handle clock skew. The iat claim must be within the server's tolerance window. If your client's clock is off, proofs will be rejected. Use Date.now() and accept that server-side validation should allow ±60 seconds of skew.

Server-Side

Maintain a JTI replay cache. Use Redis or a similar store with TTL matching your proof acceptance window. Without this, proofs can be replayed within the time window.
Validate everything. Check typ, alg (reject symmetric algorithms), htm, htu, iat, jti, ath, and nonce. Missing any validation step creates a bypass.
Use nonces in high-security contexts. For financial APIs or FAPI 2.0 compliance, server-provided nonces are mandatory. For general APIs, they add security but increase latency by one round-trip on first request.
Return nonces on every response. Don't require clients to fail before learning the nonce. Include the DPoP-Nonce header on all responses so clients can pre-populate it.
Reject alg: none and symmetric algorithms. The DPoP proof MUST use an asymmetric algorithm. Accepting HS256 defeats the entire purpose — it would mean the server and client share a secret, which is exactly what DPoP is designed to avoid.

Performance and Cost Model

DPoP adds cryptographic operations to every request. Here's the real-world impact:

Operation	Time (P50)	Time (P99)	Notes
Key generation (EC P-256)	0.5ms	2ms	Once per session
Proof signing (ECDSA)	0.8ms	2.5ms	Every request
Proof verification (server)	0.3ms	1ms	Every request
JTI cache lookup (Redis)	0.1ms	0.5ms	Every request
Thumbprint calculation	0.1ms	0.3ms	Token issuance

Total overhead per request: ~1.3ms client-side, ~0.5ms server-side. For context, a typical database query takes 5-50ms. DPoP's overhead is noise level.

The trade-off is clear: ~2ms of additional latency per request eliminates the entire class of token theft attacks.

The 2026 Reality

DPoP is not a future standard — it's a present requirement. RFC 9449 was published in September 2023 and has become one of the two required sender-constraint mechanisms for FAPI 2.0 compliance in financial services. Auth0 and Okta provide first-class DPoP support, and Microsoft Entra ID offers its own Proof-of-Possession token binding. The WebCrypto API is available in every major browser.

The real barrier is inertia. Teams that have built entire authentication systems around bearer tokens hesitate to add the complexity of cryptographic proofs. But the implementation is not complex — it's a key pair, a JWT per request, and validation middleware. The jose library handles the heavy lifting. The DPoP-aware fetch wrapper shown above is under 50 lines of code.

Bearer tokens were designed for simplicity in an era when XSS was less prevalent, supply chain attacks were rare, and API architectures were simpler. That era is over. Every token you issue as a plain bearer token is a token that can be stolen and replayed with zero friction.

Start with your highest-value API endpoints — authentication, billing, admin operations. Add DPoP support alongside bearer tokens. Monitor which clients upgrade. When coverage is high enough, deprecate bearer tokens entirely.

Your tokens should prove who's holding them. DPoP makes that possible with a few hundred lines of code and negligible latency. The only cost of not doing it is waiting for the breach.

🔒 Privacy First: This article was originally published on the Pockit Blog.

Stop sending your data to random servers. Use Pockit.tools for secure utilities, or install the Chrome Extension to keep your files 100% private and offline.

Passkeys and WebAuthn: The Complete Guide to Killing Passwords in Your Web App

HK Lee — Fri, 03 Apr 2026 13:40:00 +0000

Your users are still typing passwords. In 2026. Despite every major platform — Apple, Google, Microsoft — shipping passkey support, most web applications are still stuck on the same authentication architecture from 2005: hash a password, store it in a database, pray nobody finds it.

The reason isn't that passkeys are hard to understand. It's that the implementation path is littered with undocumented pitfalls, confusing specifications, and a WebAuthn API that looks simple in demos but breaks in production. What does allowCredentials actually do? Why does navigator.credentials.get() silently fail on some browsers? How do you migrate 500,000 existing users without breaking their sessions?

This guide covers everything you need to ship passkeys in production — from the cryptographic fundamentals to complete TypeScript implementations, Conditional UI integration, database schema design, multi-device sync, and the phased migration strategy that lets you transition without forcing users to change their behavior overnight.

Why Passwords Are Still a Liability

The numbers make the case. According to the 2025 Verizon DBIR report, over 80% of breaches involving web applications trace back to stolen or weak credentials. Phishing attacks succeed because passwords are a shared secret — the server knows the secret, the user knows the secret, and anyone who intercepts it in transit or at rest has full access.

What Passkeys Actually Solve

Passkeys eliminate the shared secret entirely. Here's the fundamental difference:

Password Authentication:
  User → sends password → Server compares hash
  Attack surface: phishing, credential stuffing, database breach

Passkey Authentication:
  User → signs challenge with private key → Server verifies with public key
  Attack surface: physical device theft (requires biometric)

The private key never leaves the user's device. The server only stores the public key. Even if your entire database leaks, attackers get nothing useful — public keys can't be used to authenticate.

The Three Properties That Matter

Phishing-resistant. Passkeys are cryptographically bound to your domain (the "Relying Party ID"). A fake login page on evil-example.com physically cannot trigger a passkey created for example.com. The browser enforces this at the protocol level — no amount of social engineering bypasses it.
No shared secrets. Your server never sees, stores, or transmits any secret material. There's nothing to hash, nothing to salt, nothing to leak. Credential stuffing attacks become irrelevant because there are no credentials to stuff.
Built-in multi-factor. A passkey authentication combines "something you have" (the device) with "something you are" (biometric) or "something you know" (device PIN). You get MFA strength without asking users to install an authenticator app.

Understanding the WebAuthn Flow

The Web Authentication API (WebAuthn) defines two core ceremonies: Registration (creating a credential) and Authentication (using it to log in).

Registration Flow

┌──────────┐         ┌──────────┐         ┌──────────────┐
│  Browser │         │  Server  │         │ Authenticator │
└────┬─────┘         └────┬─────┘         └──────┬───────┘
     │  1. Begin Registration  │                  │
     │ ──────────────────────> │                  │
     │                        │                  │
     │  2. Challenge + Options │                  │
     │ <────────────────────── │                  │
     │                        │                  │
     │  3. Create Credential  │                  │
     │ ──────────────────────────────────────────>│
     │                        │                  │
     │  4. Biometric Prompt   │                  │
     │ <──────────────────────────────────────────│
     │                        │                  │
     │  5. Public Key + Attestation              │
     │ ──────────────────────>│                  │
     │                        │                  │
     │  6. Verify & Store     │                  │
     │ <────────────────────── │                  │

Authentication Flow

┌──────────┐         ┌──────────┐         ┌──────────────┐
│  Browser │         │  Server  │         │ Authenticator │
└────┬─────┘         └────┬─────┘         └──────┬───────┘
     │  1. Begin Login     │                      │
     │ ──────────────────> │                      │
     │                     │                      │
     │  2. Challenge       │                      │
     │ <────────────────── │                      │
     │                     │                      │
     │  3. Sign Challenge  │                      │
     │ ──────────────────────────────────────────>│
     │                     │                      │
     │  4. Biometric Prompt│                      │
     │ <──────────────────────────────────────────│
     │                     │                      │
     │  5. Signed Assertion│                      │
     │ ──────────────────> │                      │
     │                     │                      │
     │  6. Verify Signature│                      │
     │ <────────────────── │                      │

The crucial detail: in step 5, the authenticator signs the server's challenge with the private key. The server then verifies that signature against the stored public key. No secret is transmitted — only a proof that the user possesses the private key.

Full Implementation with SimpleWebAuthn

Rolling your own WebAuthn implementation from scratch is a recipe for subtle security bugs. The CBOR parsing, attestation verification, and challenge management are complex enough that battle-tested libraries are the only sane path. SimpleWebAuthn is the most widely adopted TypeScript library for this.

Project Setup

# Server-side
npm install @simplewebauthn/server

# Client-side
npm install @simplewebauthn/browser

Database Schema

Before writing any auth code, design the storage layer. You need two tables:

-- Users table
CREATE TABLE users (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  username VARCHAR(255) UNIQUE NOT NULL,
  display_name VARCHAR(255),
  -- Legacy password fields (keep during migration)
  password_hash VARCHAR(255),
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Passkey credentials table (one user can have multiple passkeys)
CREATE TABLE passkey_credentials (
  id VARCHAR(512) PRIMARY KEY,             -- credential ID (base64url)
  user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
  public_key BYTEA NOT NULL,               -- stored as raw bytes
  counter BIGINT NOT NULL DEFAULT 0,       -- signature counter
  device_type VARCHAR(50) NOT NULL,        -- 'singleDevice' or 'multiDevice'
  backed_up BOOLEAN NOT NULL DEFAULT false, -- synced to cloud?
  transports TEXT[],                       -- ['internal', 'hybrid', etc.]
  display_name VARCHAR(255),               -- "MacBook Pro Touch ID"
  created_at TIMESTAMPTZ DEFAULT NOW(),
  last_used_at TIMESTAMPTZ
);

CREATE INDEX idx_credentials_user_id ON passkey_credentials(user_id);

Key design decisions:

One-to-many relationship. A user can register multiple passkeys (phone, laptop, security key). Never limit to one.
counter field. Authenticators increment a signature counter with each use. If a signature arrives with a counter lower than stored, it indicates a cloned credential — reject and flag it immediately.
device_type and backed_up. These tell you whether the passkey is synced across devices (iCloud Keychain, Google Password Manager) or bound to a single device (hardware security key).
transports array. Stores how the authenticator communicates ('internal' for platform biometric, 'hybrid' for cross-device via QR code, 'usb' for security keys). This speeds up subsequent authentications by hinting to the browser which transport to try first.

Server: Registration Endpoint

import {
  generateRegistrationOptions,
  verifyRegistrationResponse,
  type VerifiedRegistrationResponse,
} from '@simplewebauthn/server';

const RP_NAME = 'My App';
const RP_ID = 'example.com';
const ORIGIN = 'https://example.com';

// Step 1: Generate options
app.post('/api/auth/register/begin', async (req, res) => {
  const user = await getUserFromSession(req);

  // Fetch existing credentials to exclude them
  const existingCredentials = await db.query(
    'SELECT id, transports FROM passkey_credentials WHERE user_id = $1',
    [user.id]
  );

  const options = await generateRegistrationOptions({
    rpName: RP_NAME,
    rpID: RP_ID,
    userName: user.username,
    userDisplayName: user.display_name || user.username,
    // Prevent re-registering the same authenticator
    excludeCredentials: existingCredentials.rows.map(cred => ({
      id: cred.id,
      transports: cred.transports,
    })),
    authenticatorSelection: {
      residentKey: 'required',      // CRITICAL: makes it a passkey
      userVerification: 'preferred', // Biometric when available
    },
    attestationType: 'none',  // Skip attestation for consumer apps
  });

  // Store challenge in session for verification
  await setSessionChallenge(req, options.challenge);

  res.json(options);
});

// Step 2: Verify response
app.post('/api/auth/register/complete', async (req, res) => {
  const user = await getUserFromSession(req);
  const expectedChallenge = await getSessionChallenge(req);

  try {
    const verification: VerifiedRegistrationResponse = 
      await verifyRegistrationResponse({
        response: req.body,
        expectedChallenge,
        expectedOrigin: ORIGIN,
        expectedRPID: RP_ID,
      });

    if (!verification.verified || !verification.registrationInfo) {
      return res.status(400).json({ error: 'Verification failed' });
    }

    const { credential, credentialDeviceType, credentialBackedUp } = 
      verification.registrationInfo;

    // Save to database
    await db.query(
      `INSERT INTO passkey_credentials 
       (id, user_id, public_key, counter, device_type, backed_up, transports)
       VALUES ($1, $2, $3, $4, $5, $6, $7)`,
      [
        credential.id,
        user.id,
        Buffer.from(credential.publicKey),
        credential.counter,
        credentialDeviceType,
        credentialBackedUp,
        credential.transports ?? [],
      ]
    );

    res.json({ verified: true });
  } catch (error) {
    console.error('Registration verification failed:', error);
    res.status(400).json({ error: 'Registration failed' });
  }
});

Server: Authentication Endpoint

import {
  generateAuthenticationOptions,
  verifyAuthenticationResponse,
} from '@simplewebauthn/server';

// Step 1: Generate challenge
app.post('/api/auth/login/begin', async (req, res) => {
  const options = await generateAuthenticationOptions({
    rpID: RP_ID,
    userVerification: 'preferred',
    // Empty allowCredentials = discoverable credential flow
    // The authenticator will show all credentials for this RP
    allowCredentials: [],
  });

  // Store challenge — use a short-lived store (Redis, signed cookie)
  await setChallengeStore(options.challenge, { expiresIn: 300 }); // 5 min

  res.json(options);
});

// Step 2: Verify assertion
app.post('/api/auth/login/complete', async (req, res) => {
  const { id: credentialId } = req.body;

  // Look up the credential
  const credRow = await db.query(
    `SELECT pc.*, u.id as uid, u.username 
     FROM passkey_credentials pc 
     JOIN users u ON pc.user_id = u.id 
     WHERE pc.id = $1`,
    [credentialId]
  );

  if (credRow.rows.length === 0) {
    return res.status(401).json({ error: 'Unknown credential' });
  }

  const cred = credRow.rows[0];
  const expectedChallenge = await getChallengeStore(req.body.response.clientDataJSON);

  try {
    const verification = await verifyAuthenticationResponse({
      response: req.body,
      expectedChallenge,
      expectedOrigin: ORIGIN,
      expectedRPID: RP_ID,
      credential: {
        id: cred.id,
        publicKey: new Uint8Array(cred.public_key),
        counter: Number(cred.counter),
        transports: cred.transports,
      },
    });

    if (!verification.verified) {
      return res.status(401).json({ error: 'Verification failed' });
    }

    // CRITICAL: Update the counter to detect cloned credentials
    const { newCounter } = verification.authenticationInfo;
    await db.query(
      `UPDATE passkey_credentials 
       SET counter = $1, last_used_at = NOW() 
       WHERE id = $2`,
      [newCounter, credentialId]
    );

    // Issue session / JWT
    const token = await createSession(cred.uid);
    res.json({ verified: true, token });
  } catch (error) {
    console.error('Authentication failed:', error);
    res.status(401).json({ error: 'Authentication failed' });
  }
});

Client: Browser Integration

import { 
  startRegistration, 
  startAuthentication 
} from '@simplewebauthn/browser';

// Registration
async function registerPasskey(): Promise<void> {
  // 1. Get options from server
  const optionsRes = await fetch('/api/auth/register/begin', { 
    method: 'POST' 
  });
  const options = await optionsRes.json();

  // 2. Create credential (triggers biometric prompt)
  const credential = await startRegistration({ optionsJSON: options });

  // 3. Send to server for verification
  const verifyRes = await fetch('/api/auth/register/complete', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(credential),
  });

  const result = await verifyRes.json();
  if (result.verified) {
    console.log('Passkey registered successfully');
  }
}

// Authentication
async function loginWithPasskey(): Promise<void> {
  const optionsRes = await fetch('/api/auth/login/begin', { 
    method: 'POST' 
  });
  const options = await optionsRes.json();

  const assertion = await startAuthentication({ optionsJSON: options });

  const verifyRes = await fetch('/api/auth/login/complete', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(assertion),
  });

  const result = await verifyRes.json();
  if (result.verified) {
    window.location.href = '/dashboard';
  }
}

Conditional UI: Passkeys in the Autofill Menu

The biggest UX improvement for passkeys isn't the registration flow — it's Conditional UI. Instead of requiring users to click a "Sign in with passkey" button, the browser shows available passkeys directly in the username field's autofill dropdown, right next to saved passwords.

How It Works

<!-- The magic is in the autocomplete attribute -->
<form>
  <input 
    type="text" 
    id="username"
    name="username"
    autocomplete="username webauthn"
    placeholder="Email or username"
  />
  <input type="password" name="password" autocomplete="current-password" />
  <button type="submit">Sign In</button>
</form>

The webauthn token in the autocomplete attribute tells the browser to include passkey options in the autofill dropdown. It must be the last token in the attribute value.

Implementation

import { startAuthentication, browserSupportsWebAuthnAutofill } from '@simplewebauthn/browser';

async function initConditionalUI(): Promise<void> {
  // Feature detection — always check first
  const supported = await browserSupportsWebAuthnAutofill();
  if (!supported) {
    console.log('Conditional UI not supported, falling back to button');
    return;
  }

  // Get authentication options from server
  const optionsRes = await fetch('/api/auth/login/begin', { method: 'POST' });
  const options = await optionsRes.json();

  try {
    // This call will "wait" until the user selects a passkey from autofill
    const assertion = await startAuthentication({
      optionsJSON: options,
      useBrowserAutofill: true,  // Enables Conditional UI
    });

    // User selected a passkey from the dropdown — verify it
    const verifyRes = await fetch('/api/auth/login/complete', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(assertion),
    });

    const result = await verifyRes.json();
    if (result.verified) {
      window.location.href = '/dashboard';
    }
  } catch (error) {
    // User dismissed the autofill or chose a password instead
    if ((error as Error).name !== 'AbortError') {
      console.error('Conditional UI error:', error);
    }
  }
}

// Call on page load — NOT on button click
document.addEventListener('DOMContentLoaded', initConditionalUI);

Critical Implementation Details

1. Call on page load, not on click. Conditional UI must be initiated when the page loads. The browser continuously listens for the user to interact with the autofill dropdown. If you only call it on button click, you lose the seamless autofill experience.

2. Use an AbortController. If the user switches to a different login method (social login, magic link), abort the pending conditional UI request:

const abortController = new AbortController();

async function initConditionalUI(): Promise<void> {
  // ... feature detection ...

  const assertion = await startAuthentication({
    optionsJSON: options,
    useBrowserAutofill: true,
  });
  // ... verify ...
}

// When user clicks "Sign in with Google" or similar
function switchToAlternativeLogin(): void {
  abortController.abort();
  // Proceed with alternative login flow
}

3. Fallback strategy. Always provide a visible "Sign in with passkey" button for browsers that don't support Conditional UI. The button uses the standard modal flow as a fallback.

Phased Migration Strategy

You can't flip a switch and delete all passwords overnight. Here's the three-phase approach that works in production:

Phase 1: Introduction (Passive Enrollment)

After successful password login, prompt users to register a passkey. Don't force it — just make the option visible and attractive.

// After successful password login
async function postLoginPasskeyPrompt(user: User): Promise<void> {
  const hasPasskey = await userHasPasskey(user.id);
  if (hasPasskey) return;

  const dismissedCount = await getPromptDismissals(user.id);
  if (dismissedCount >= 3) return; // Don't be annoying

  // Show a non-blocking UI prompt
  showPasskeyEnrollmentBanner({
    title: 'Enable faster sign-in',
    description: 'Use Face ID or fingerprint instead of your password',
    onAccept: () => registerPasskey(),
    onDismiss: () => incrementPromptDismissals(user.id),
  });
}

Phase 2: Default (New Users Start Passwordless)

New user registration defaults to passkey. Password is available but secondary.

async function registerNewUser(userData: NewUser): Promise<void> {
  const user = await createUser(userData);

  // Try passkey first
  const webauthnSupported = await isWebAuthnSupported();

  if (webauthnSupported) {
    await registerPasskey(); // Primary — passkey
    // Optionally still collect a password as backup
  } else {
    await setPassword(user.id, userData.password); // Fallback
  }
}

Phase 3: Transition (Passwordless-Only Option)

Users with passkeys registered can opt to disable their password entirely.

app.post('/api/auth/disable-password', async (req, res) => {
  const user = await getUserFromSession(req);

  // Safety checks before allowing password deletion
  const credentials = await getUserCredentials(user.id);

  if (credentials.length < 2) {
    return res.status(400).json({ 
      error: 'Register at least 2 passkeys before disabling password' 
    });
  }

  // Check that at least one credential is backed up (synced)
  const hasSyncedPasskey = credentials.some(c => c.backed_up);
  if (!hasSyncedPasskey) {
    return res.status(400).json({
      error: 'Register at least one synced passkey (iCloud/Google) for recovery'
    });
  }

  // Soft-delete the password
  await db.query(
    'UPDATE users SET password_hash = NULL WHERE id = $1',
    [user.id]
  );

  res.json({ success: true });
});

Migration Metrics Dashboard

Track these metrics to measure migration health:

Metric	Target	What It Tells You
Passkey adoption rate	>30% within 6 months	Overall migration velocity
Registration completion	>80% who start	UX friction during enrollment
Auth success rate	>99%	Reliability of passkey flow
Fallback usage	Declining trend	Users successfully moving off passwords
Prompt dismiss rate	<60%	Whether enrollment prompts are too aggressive

Security Pitfalls in Production

1. Challenge Replay Attacks

Every challenge must be single-use and time-limited. If you store challenges in a stateless JWT, an attacker can replay the same challenge.

// BAD: Challenge in a signed cookie (replayable)
const challenge = signJWT({ challenge: randomBytes(32) });

// GOOD: Challenge in server-side store with TTL
const challenge = randomBytes(32).toString('base64url');
await redis.setex(`webauthn:challenge:${sessionId}`, 300, challenge);

// After verification, immediately delete
await redis.del(`webauthn:challenge:${sessionId}`);

2. Counter Validation

The signature counter is your defense against credential cloning. If an authenticator's counter goes backwards, something is very wrong.

async function validateCounter(
  credentialId: string, 
  newCounter: number
): Promise<void> {
  const stored = await getStoredCounter(credentialId);

  if (newCounter > 0 && newCounter <= stored) {
    // POTENTIAL CLONED CREDENTIAL
    await flagCredentialAsCompromised(credentialId);
    await notifySecurityTeam({
      event: 'counter_regression',
      credentialId,
      storedCounter: stored,
      receivedCounter: newCounter,
    });
    throw new Error('Credential counter regression detected');
  }
}

Caveat: Some platform authenticators (especially synced passkeys) always report a counter of 0. In this case, counter validation is effectively disabled — the sync mechanism provides security through other means. Only flag regressions when the counter is actually being used (i.e., when it's greater than 0).

3. Origin Validation

Always validate the origin strictly. A misconfigured origin check can defeat the phishing resistance that makes passkeys valuable.

// BAD: Substring match (vulnerable to subdomain attacks)
if (origin.includes('example.com')) { ... }

// GOOD: Exact match against allowed origins
const ALLOWED_ORIGINS = [
  'https://example.com',
  'https://app.example.com',
];

if (!ALLOWED_ORIGINS.includes(origin)) {
  throw new Error('Origin not allowed');
}

4. Account Recovery

The hardest problem in passwordless authentication. What happens when a user loses all their devices?

interface RecoveryStrategy {
  // Recommended: Require multiple passkeys
  minimumPasskeys: 2;

  // Recommend at least one synced passkey
  requireSyncedCredential: true;

  // Backup methods (in order of preference)
  fallbacks: [
    'recovery_codes',       // One-time codes generated at enrollment
    'email_magic_link',     // Time-limited login link
    'identity_verification' // Manual process with ID verification
  ];
}

Generate recovery codes during initial passkey registration and instruct users to store them securely. This is the same pattern security keys have used for years — it works.

Cross-Platform Considerations

Device Sync Behavior

Platform	Sync Mechanism	Scope
Apple	iCloud Keychain	All Apple devices with same Apple ID
Google	Google Password Manager	All Chrome/Android with same Google account
Microsoft	Microsoft Authenticator	Windows devices with same Microsoft account
1Password / Bitwarden	Third-party vault	Cross-platform, all devices with vault access

The Cross-Device Flow (Hybrid Transport)

When a user wants to log in on a device that doesn't have their passkey, the "hybrid" transport kicks in:

Desktop browser shows a QR code
User scans with their phone (which has the passkey)
Phone prompts for biometric
Authentication completes on the desktop

This works automatically — you don't need to implement anything special. The browser and authenticator handle the entire flow when transports includes 'hybrid'.

Performance and Cost Model

Passkey authentication is significantly cheaper at scale than password authentication:

Factor	Passwords	Passkeys
Server compute	bcrypt/Argon2 hashing (CPU intensive)	Ed25519 signature verification (fast)
Storage per user	~128 bytes (hash + salt)	~256 bytes (public key + metadata)
Support tickets	"I forgot my password" (~40% of support volume)	Near zero after enrollment
Breach cost	Average $4.44M per incident	No usable credentials to steal
SMS 2FA cost	$0.01-0.05 per message	$0 (built-in verification)

For a service with 100,000 users, eliminating password reset flows alone can save 15-20 hours of support time per month.

The Reality in 2026

Passkeys aren't a future technology — they're a present one. Every major browser supports them. Every major platform syncs them. The WebAuthn spec is stable. Libraries like SimpleWebAuthn handle the dangerous cryptographic details.

The actual blockers are organizational, not technical. Teams hesitate because "our users are used to passwords" or "we can't break the existing login flow." The phased migration strategy solves this. You don't need to deprecate passwords on day one. You just need to start offering something better alongside them.

The teams shipping passkeys today are seeing 50-70% voluntary adoption rates within six months when the enrollment UX is smooth. Users genuinely prefer touching their fingerprint to typing a password. The conversion friction is lower, the support load drops, and the security posture improves dramatically.

Start with Conditional UI. It requires minimal UI changes — just an autocomplete attribute and a page-load script. Users who have passkeys get the fast path automatically. Users who don't see the same login form they've always seen. Zero disruption, maximum upside.

The password is a 60-year-old technology. It's time to let it retire.

🚀 Explore More: This article is from the Pockit Blog.

If you found this helpful, check out Pockit.tools. It’s a curated collection of offline-capable dev utilities. Available on Chrome Web Store for free.

AI Code Review in Your CI/CD Pipeline: Automating PR Reviews, Test Generation, and Bug Detection with LLMs

HK Lee — Thu, 02 Apr 2026 14:09:00 +0000

Your team just shipped an AI coding assistant. Developers are writing code 40% faster. PRs are flooding in at twice the previous rate. And your two senior engineers — the ones who actually catch the subtle bugs — are now drowning in a review backlog that grows faster than they can read.

This is the paradox of AI-assisted development in 2026. The same tools that accelerate code generation create an unsustainable review bottleneck. More code, generated faster, with less human oversight per line. The math doesn't work unless you automate the review side too.

But here's where most teams get it wrong: they bolt on an AI reviewer, get flooded with low-quality noise ("consider renaming this variable"), lose trust in the tool within a week, and rip it out. The problem isn't the AI — it's the architecture. A good AI code review system isn't a chatbot reading diffs. It's a layered pipeline that understands your codebase, enforces your team's standards, and knows when to shut up.

This guide covers how to build that pipeline — from evaluating existing tools to building custom review bots, integrating AI test generation, and architecting a system that developers actually trust.

The Review Bottleneck Problem

Let's quantify it. Before AI coding assistants, a typical team of 8 developers produced 15-20 PRs per week. Senior engineers could review them within a day. Now that same team produces 30-40 PRs per week, and the review queue is perpetually behind.

Why Traditional Code Review Doesn't Scale with AI-Generated Code

The fundamental issue isn't volume alone — it's the nature of AI-generated code:

Surface-level correctness. AI-generated code usually passes linting, compiles fine, and handles the happy path. The bugs are in the edge cases, the missing error handling, the subtle race condition that only appears under load.
Pattern repetition at scale. AI assistants often generate structurally similar code across different parts of the codebase. A human reviewer sees "this looks right" because the pattern is familiar, but the same architectural flaw is replicated 15 times before anyone notices.
Context gap. The AI that wrote the code had a conversation context (the prompt, the file it was editing). The human reviewer doesn't see that context — they see a diff that "looks reasonable" but was generated from an incomplete understanding of the system.
Review fatigue multiplied. Studies consistently show that review quality drops after 200-400 lines of code. AI-generated PRs regularly exceed this threshold, making human-only review increasingly unreliable.

The solution isn't "hire more seniors." It's building an AI review layer that catches the mechanical issues (security flaws, missing tests, API misuse, style violations) so humans can focus on what they're actually good at: architecture decisions, business logic correctness, and system design trade-offs.

The AI Code Review Landscape in 2026

The market has matured significantly. Here's an honest breakdown of the major tools:

Tool Comparison

Tool	Approach	Best For	Key Strength	Key Weakness
CodeRabbit	Diff-aware PR review	Broad platform support	Cross-platform (GitHub, GitLab, Azure, Bitbucket), learns team patterns	Can be noisy on large PRs without tuning
Qodo (ex PR-Agent)	Full codebase indexing	Enterprise, multi-repo	Understands dependency graphs, architectural context	Heavier setup, enterprise pricing
Ellipsis	Fix-generating reviewer	Fast turnaround teams	Generates committable fixes, not just comments	Narrower platform support
Greptile	Deep codebase understanding	Complex architectural review	Full-repo indexing for cross-file impact analysis	Newer, smaller ecosystem
Codacy	Security + quality gates	Compliance-heavy teams	Strong security scanning, policy enforcement	Less AI-native, more traditional SAST

What to Actually Look For

Ignore the marketing. These are the three criteria that matter:

1. Context depth. Does the tool understand just the diff, or the entire repository? Diff-only tools produce surface-level comments ("this function is long") but miss cross-file impacts ("this change breaks the contract with module X"). Full-codebase indexing is significantly more useful but costs more in compute and setup time.

2. Signal-to-noise ratio. The #1 reason teams abandon AI reviewers is noise. If 80% of comments are trivial style suggestions, developers will stop reading any of them — including the 20% that catch real bugs. The best tools let you configure severity thresholds and suppress low-value feedback.

3. Actionability. "Consider improving the error handling here" is useless. "This catch block swallows the error silently. Here's the fix: [code suggestion]" is valuable. Tools that generate patches or one-click fixes get adopted; tools that generate vague suggestions get ignored.

Building a Custom AI Review Bot

Sometimes existing tools don't fit your needs. Maybe you have proprietary coding standards, domain-specific patterns, or a codebase structure that confuses generic tools. Here's how to build a lightweight, effective AI reviewer that runs in your CI/CD pipeline.

Architecture Overview

┌─────────────────────────────────────────────┐
│                  GitHub PR                   │
│   (push event / pull_request event)          │
└──────────────────┬──────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────┐
│           CI/CD Pipeline (GitHub Actions)     │
│                                              │
│  1. Fetch diff (changed files only)          │
│  2. Load context (related files, types)      │
│  3. Build review prompt                      │
│  4. Call LLM API                             │
│  5. Parse structured response                │
│  6. Post inline comments on PR               │
└─────────────────────────────────────────────┘

Step 1: Fetching the Right Context

The biggest mistake in custom review bots is sending only the diff. Without surrounding context, the LLM can't understand what the code does or whether the change is correct.

interface ReviewContext {
  diff: string;              // The actual changes
  fullFiles: string[];       // Complete files that were modified
  relatedFiles: string[];    // Files that import/reference the changed files
  projectRules: string;      // Team coding standards
  recentCommits: string[];   // Last 5 commit messages for intent
}

async function buildReviewContext(pr: PullRequest): Promise<ReviewContext> {
  const diff = await github.pulls.getDiff(pr);
  const changedFiles = parseDiffForFiles(diff);

  // Get full content of changed files (not just the diff)
  const fullFiles = await Promise.all(
    changedFiles.map(f => github.repos.getContent(f))
  );

  // Find files that import or reference the changed files
  const relatedFiles = await findRelatedFiles(changedFiles, {
    maxDepth: 2,        // Follow imports 2 levels deep
    maxFiles: 10,       // Cap to prevent context explosion
  });

  // Load team-specific rules
  const projectRules = await loadFile('.github/review-rules.md');

  // Recent commits for intent context
  const recentCommits = await github.pulls.listCommits(pr, { per_page: 5 });

  return { diff, fullFiles, relatedFiles, projectRules, recentCommits };
}

Step 2: The Review Prompt

This is where most custom bots fail. A generic "review this code" prompt produces generic results. You need to be extremely specific about what to look for and what to ignore.

function buildReviewPrompt(context: ReviewContext): string {
  return `You are a senior software engineer reviewing a pull request.

## Team Coding Standards
${context.projectRules}

## Context: Related Files
${context.relatedFiles.map(f => `### ${f.path}\n${f.content}`).join('\n\n')}

## Recent Commit Messages (for understanding intent)
${context.recentCommits.map(c => `- ${c.message}`).join('\n')}

## The Diff to Review
${context.diff}

## Review Instructions
Analyze this diff and provide feedback. Follow these rules strictly:

1. **DO NOT** comment on style, formatting, or naming unless it violates the team standards above.
2. **DO NOT** suggest changes that are purely cosmetic.
3. **DO** identify: security vulnerabilities, missing error handling, potential null/undefined issues, race conditions, broken API contracts, missing input validation, and logic errors.
4. **DO** check if new functions have corresponding tests in the diff.
5. For each issue found, provide:
   - The exact file and line number
   - Severity: CRITICAL, WARNING, or INFO
   - A specific code suggestion to fix the issue

## Response Format
Respond with a JSON array of review comments:
[
  {
    "file": "src/api/users.ts",
    "line": 42,
    "severity": "CRITICAL",
    "issue": "SQL injection vulnerability",
    "suggestion": "Use parameterized query instead of string interpolation",
    "code": "db.query('SELECT * FROM users WHERE id = $1', [userId])"
  }
]

If the code looks good and you have no substantive feedback, return an empty array [].`;
}

Step 3: Calling the LLM and Posting Comments

async function reviewPR(pr: PullRequest): Promise<void> {
  const context = await buildReviewContext(pr);
  const prompt = buildReviewPrompt(context);

  // Use a strong model for code review — accuracy matters more than speed
  const response = await openai.chat.completions.create({
    model: 'gpt-4.1',
    messages: [{ role: 'user', content: prompt }],
    response_format: { type: 'json_object' },
    temperature: 0.1,  // Low temperature for consistent, focused reviews
  });

  const comments: ReviewComment[] = JSON.parse(response.choices[0].message.content);

  // Filter out INFO-level comments if the PR is small (reduce noise)
  const filtered = pr.changedLines < 100
    ? comments.filter(c => c.severity !== 'INFO')
    : comments;

  // Post as inline PR comments
  for (const comment of filtered) {
    await github.pulls.createReviewComment({
      owner: pr.repo.owner,
      repo: pr.repo.name,
      pull_number: pr.number,
      body: formatComment(comment),
      path: comment.file,
      line: comment.line,
    });
  }

  // Post summary as a PR review
  await github.pulls.createReview({
    owner: pr.repo.owner,
    repo: pr.repo.name,
    pull_number: pr.number,
    body: generateSummary(comments),
    event: comments.some(c => c.severity === 'CRITICAL') ? 'REQUEST_CHANGES' : 'COMMENT',
  });
}

function formatComment(comment: ReviewComment): string {
  const icon = { CRITICAL: '🚨', WARNING: '⚠️', INFO: 'ℹ️' }[comment.severity];
  return `${icon} **${comment.severity}**: ${comment.issue}\n\n${comment.suggestion}\n\n\`\`\`suggestion\n${comment.code}\n\`\`\``;
}

Step 4: GitHub Actions Integration

# .github/workflows/ai-review.yml
name: AI Code Review
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  ai-review:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for context

      - name: Run AI Review
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: npx ts-node scripts/ai-review.ts

AI-Powered Test Generation

Code review catches bugs after they're written. Test generation prevents them from shipping. The two work together as complementary layers of your quality pipeline.

The State of AI Test Generation in 2026

Tool	Focus	Approach
Mabl	End-to-end testing	Agentic: takes requirements in plain English, generates and maintains test suites
Diffblue Cover	Java unit tests	Static analysis + AI: generates JUnit tests for every method
Codium/Qodo	Multi-language unit tests	Context-aware: analyzes function signatures, types, and usage patterns
Playwright + LLM	E2E test generation	Custom: LLM generates Playwright test scripts from user flows

Building a Lightweight Test Generator

For teams that want test generation without adding another SaaS dependency, here's a practical approach using your existing LLM provider:

async function generateTestsForChangedFiles(
  changedFiles: string[]
): Promise<Map<string, string>> {
  const tests = new Map<string, string>();

  for (const filePath of changedFiles) {
    // Skip non-source files, test files, and config files
    if (isTestFile(filePath) || isConfigFile(filePath)) continue;

    const sourceCode = await readFile(filePath);
    const existingTests = await findExistingTests(filePath);
    const imports = await resolveImports(filePath);

    const prompt = buildTestGenPrompt(sourceCode, existingTests, imports, filePath);

    const response = await openai.chat.completions.create({
      model: 'gpt-4.1',
      messages: [{ role: 'user', content: prompt }],
      temperature: 0.2,
    });

    const generatedTest = response.choices[0].message.content;

    // Validate: try to parse the generated test to catch syntax errors
    if (await isValidTypeScript(generatedTest)) {
      const testPath = toTestPath(filePath);
      tests.set(testPath, generatedTest);
    }
  }

  return tests;
}

function buildTestGenPrompt(
  source: string,
  existingTests: string | null,
  imports: ImportInfo[],
  filePath: string
): string {
  return `Generate unit tests for the following TypeScript file.

## Source File: ${filePath}
\`\`\`typescript
${source}
\`\`\`

## Available Imports and Types
${imports.map(i => `- ${i.name}: ${i.type}`).join('\n')}

${existingTests ? `## Existing Tests (extend, don't duplicate)\n\`\`\`typescript\n${existingTests}\n\`\`\`` : '## No existing tests found. Create a new test file.'}

## Requirements
1. Use Vitest as the test framework.
2. Test all exported functions.
3. Include tests for: happy path, edge cases (null, undefined, empty), error cases.
4. Mock external dependencies (database, API calls, file system).
5. Do NOT test private/internal implementation details.
6. Each test should have a clear, descriptive name that explains the expected behavior.
7. Aim for 80%+ branch coverage.

Output ONLY the test file content, no explanations.`;
}

The Test Validation Loop

Generated tests are useless if they don't actually run. Add a validation step:

async function validateAndCommitTests(
  tests: Map<string, string>
): Promise<TestValidationResult> {
  const results: TestValidationResult = { passed: [], failed: [], skipped: [] };

  for (const [testPath, testContent] of tests) {
    // Write the test file temporarily
    await writeFile(testPath, testContent);

    try {
      // Run just this test file
      const { exitCode, stdout, stderr } = await exec(
        `npx vitest run ${testPath} --reporter=json`
      );

      if (exitCode === 0) {
        results.passed.push(testPath);
      } else {
        // Test failed — try one self-repair cycle
        const repaired = await repairTest(testPath, testContent, stderr);
        if (repaired) {
          results.passed.push(testPath);
        } else {
          await removeFile(testPath);  // Don't commit broken tests
          results.failed.push({ path: testPath, error: stderr });
        }
      }
    } catch (error) {
      await removeFile(testPath);
      results.skipped.push(testPath);
    }
  }

  return results;
}

The Noise Problem: Why AI Reviews Get Ignored

This is the single most important section of this guide. Every team that has tried AI code review has hit the same wall: too much noise, not enough signal. Here's how to fix it.

The Trust Equation

Developer Trust = (Bugs Caught) / (Total Comments)

If your AI reviewer posts 20 comments and 18 of them are trivial style suggestions, developers will start ignoring all 20 — including the 2 that catch real bugs. You need a ratio of at least 50% actionable findings to maintain trust.

Strategies for Reducing Noise

1. Severity-based filtering. Never show INFO-level comments by default. Let developers opt in to verbose mode if they want it.

const minSeverity: Record<string, Severity> = {
  'hotfix/*':  'CRITICAL',    // Hotfixes: only block on critical issues
  'feature/*': 'WARNING',     // Features: show warnings and above
  'refactor/*': 'WARNING',    // Refactors: focus on regressions
  'default':   'INFO',        // Default: show everything
};

2. Suppression rules. Learn from your team's override patterns. If developers consistently dismiss a certain type of comment (e.g., "consider using optional chaining"), suppress it globally.

interface SuppressionRule {
  pattern: RegExp;           // Match against comment text
  dismissCount: number;      // How many times dismissed
  threshold: number;         // Auto-suppress after this many dismissals
  suppressedAt?: Date;
}

// Track dismissals
async function onCommentDismissed(comment: ReviewComment): Promise<void> {
  const rule = findMatchingRule(comment);
  rule.dismissCount++;

  if (rule.dismissCount >= rule.threshold) {
    rule.suppressedAt = new Date();
    await saveSuppressionRules();
    console.log(`Auto-suppressed: "${rule.pattern}" after ${rule.dismissCount} dismissals`);
  }
}

3. Incremental review. Don't re-review the entire PR on every push. Only review the new commits since the last review. This prevents the same comments from appearing multiple times.

4. Self-correction loop. Before posting, run a second LLM pass that evaluates each comment against the criteria: "Would a senior engineer actually care about this? Would this comment help catch a bug or prevent an outage?" Discard anything that scores below the threshold.

Architecture: The Full AI Review Pipeline

Here's the complete architecture that combines review, testing, and noise management into a production-ready system:

PR Opened / Updated
        │
        ▼
┌───────────────────┐
│  1. Context Build  │  ← Fetch diff + full files + related files + rules
└───────┬───────────┘
        │
        ▼
┌───────────────────┐
│  2. AI Review      │  ← LLM analyzes for bugs, security, missing tests
└───────┬───────────┘
        │
        ▼
┌───────────────────┐
│  3. Noise Filter   │  ← Severity gate + suppression rules + self-check
└───────┬───────────┘
        │
        ▼
┌───────────────────┐
│  4. Test Generation │  ← Generate tests for uncovered new code
└───────┬───────────┘
        │
        ▼
┌───────────────────┐
│  5. Test Validation │  ← Run generated tests, self-repair if needed
└───────┬───────────┘
        │
        ▼
┌───────────────────┐
│  6. PR Feedback     │  ← Post inline comments + summary + test PR
└───────────────────┘

Cost Management

AI review at scale gets expensive fast. Here's a cost model for a team of 10 developers:

Metric	Value
PRs per week	40
Average diff size	300 lines
Context per review	~8,000 tokens
LLM output per review	~2,000 tokens
Cost per review (GPT-4.1)	~$0.03
Weekly cost	~$1.20
Monthly cost	~$5.00

At $5/month for a team of 10, this is absurdly cheap compared to the engineering hours saved. Even at 10x the volume, the cost is under $50/month — less than a single hour of an engineer's time.

When NOT to Use AI Review

AI review is not a silver bullet. Skip it for:

Infrastructure-as-code changes (Terraform, CloudFormation) — too much proprietary context needed
Generated code (protobuf, OpenAPI) — reviewing auto-generated code with AI is pointless
Trivial changes (README updates, dependency bumps) — not worth the compute
Security-critical code — AI review should supplement, never replace, human security review

Measuring Success

You need data to know if your AI review pipeline is working. Track these metrics:

Review Quality Metrics

Metric	Target	What It Tells You
Bug catch rate	Increases over time	Whether AI actually finds bugs humans miss
False positive rate	< 30%	Whether developers trust the tool
Time to first review	< 5 minutes	Speed of automated feedback
Human review time saved	> 30%	ROI for the pipeline
Comment dismiss rate	< 50%	Whether feedback is actionable

The Feedback Loop

The most important pattern: when a human reviewer catches a bug that the AI missed, feed it back into the system. Add it to your .github/review-rules.md as a specific pattern to check for. Over time, your custom rules accumulate the institutional knowledge of your team.

<!-- .github/review-rules.md -->
# Team Review Standards

## Hard Rules (Always Flag)
- Never use `any` type in TypeScript (except in test files)
- All API endpoints must validate input with Zod schemas
- Database queries must use parameterized statements
- All async functions must have error handling (no unhandled promise rejections)

## Patterns We've Been Burned By
- Using `Date.now()` in tests (use fake timers instead)
- Forgetting to close database connections in error paths
- Not checking for null before accessing nested object properties from API responses
- Missing `await` on async operations inside loops (causes race conditions)

The 2026 Reality

AI code review isn't replacing human reviewers. It's becoming the first pass that handles the mechanical checks, freeing humans to focus on what matters: "Should we build this feature this way?" and "Does this change align with where we want the architecture to go?"

The teams that get this right are treating AI review as infrastructure, not a feature. It runs on every PR, it learns from dismissals, it generates tests for uncovered code, and it stays quiet when it has nothing useful to say.

The teams that get it wrong bolt on a chatbot, get a flood of "helpful suggestions," and go back to manual review within a month.

The difference isn't the AI. It's the pipeline.

Start with the noise filter. Everything else is optional until you solve that.

🛠️ Developer Toolkit: This post first appeared on the Pockit Blog.

Need a Regex Tester, JWT Decoder, or Image Converter? Use them on Pockit.tools or install the Extension to avoid switching tabs. No signup required.

7 Patterns That Stop Your AI Agent From Going Rogue in Production

HK Lee — Wed, 01 Apr 2026 13:48:00 +0000

Your AI agent works flawlessly in development. It passes every test, handles your demo scenarios perfectly, and impresses stakeholders in the sprint review. Then you deploy it. Within 48 hours, it burns $400 in API costs processing a recursive loop, emails a customer their neighbor's personal data, and confidently generates a SQL query that drops an index on your production database.

This isn't hypothetical. It's a pattern playing out across the industry in 2026. The gap between "demo-ready" and "production-ready" AI agents is wider than most teams realize, and the failure modes are fundamentally different from traditional software. Your REST API doesn't decide to answer a different question than the one it was asked. Your database driver doesn't hallucinate a table name. But your AI agent does both, and it does them with absolute confidence.

This guide covers seven battle-tested patterns for keeping AI agents reliable in production. These aren't theoretical frameworks — they're extracted from real incident post-mortems, production outages, and hard-won lessons from teams running agents at scale.

Pattern 1: The Circuit Breaker

Traditional software uses circuit breakers to prevent cascading failures when downstream services go down. AI agents need them too, but with a twist: you're not just protecting against HTTP 500s. You're protecting against a model that starts returning garbage.

Why Agents Need Circuit Breakers

An AI agent that calls a failing tool doesn't crash. It retries. And retries. And since it's "intelligent," it might try slightly different approaches each time — all of which fail, all of which cost tokens. Without a circuit breaker, a single broken tool can burn your entire daily API budget in minutes.

Implementation

class AgentCircuitBreaker {
  private failures: Map<string, { count: number; lastFailure: number }> = new Map();
  private readonly threshold = 5;        // failures before opening
  private readonly resetTimeout = 60000; // 1 minute cooldown

  async callTool(toolName: string, fn: () => Promise<any>): Promise<any> {
    const state = this.failures.get(toolName) || { count: 0, lastFailure: 0 };

    // Check if circuit is open
    if (state.count >= this.threshold) {
      const elapsed = Date.now() - state.lastFailure;
      if (elapsed < this.resetTimeout) {
        throw new CircuitOpenError(
          `Tool "${toolName}" is temporarily disabled. ` +
          `${Math.ceil((this.resetTimeout - elapsed) / 1000)}s until retry.`
        );
      }
      // Half-open: allow one attempt
      state.count = this.threshold - 1;
    }

    try {
      const result = await fn();
      // Success: reset failures
      this.failures.set(toolName, { count: 0, lastFailure: 0 });
      return result;
    } catch (error) {
      state.count++;
      state.lastFailure = Date.now();
      this.failures.set(toolName, state);
      throw error;
    }
  }
}

The Key Insight

When the circuit opens, feed the error back to the agent as context. Don't just throw an exception — tell the model that the tool is unavailable and suggest alternatives:

if (error instanceof CircuitOpenError) {
  return {
    role: 'tool',
    content: `The ${toolName} service is temporarily unavailable (circuit breaker open). ` +
             `Please inform the user that this feature is temporarily down, ` +
             `or try an alternative approach that doesn't require this tool.`
  };
}

This turns a hard failure into a graceful degradation. The agent can apologize to the user, suggest a workaround, or skip that step entirely — instead of silently looping.

Pattern 2: Retry-Classify (Don't Retry Blindly)

The naive retry pattern — "if it fails, try the exact same thing again" — is actively harmful with AI agents. If the model generated a malformed API call, retrying the same prompt will likely generate the same malformed call. You're paying double for the same failure.

The Retry-Classify Pattern

Instead of blind retries, classify the error first and route to the appropriate recovery strategy:

class RetryClassifier:
    def classify(self, error: Exception, tool_name: str) -> RetryStrategy:
        if isinstance(error, RateLimitError):
            return RetryStrategy.BACKOFF      # Wait and retry same request

        if isinstance(error, ValidationError):
            return RetryStrategy.REPAIR       # Feed error to LLM, ask it to fix

        if isinstance(error, AuthenticationError):
            return RetryStrategy.FAIL_FAST    # Don't retry, escalate immediately

        if isinstance(error, TimeoutError):
            return RetryStrategy.BACKOFF      # Likely transient

        if isinstance(error, ToolNotFoundError):
            return RetryStrategy.FALLBACK     # Try alternative tool

        return RetryStrategy.FAIL_FAST        # Unknown errors: don't retry


async def execute_with_retry(agent, action, max_retries=3):
    classifier = RetryClassifier()

    for attempt in range(max_retries):
        try:
            return await agent.execute(action)
        except Exception as e:
            strategy = classifier.classify(e, action.tool_name)

            if strategy == RetryStrategy.FAIL_FAST:
                raise  # Don't waste tokens

            if strategy == RetryStrategy.BACKOFF:
                wait = (2 ** attempt) + random.uniform(0, 1)  # Exponential + jitter
                await asyncio.sleep(wait)
                continue

            if strategy == RetryStrategy.REPAIR:
                # Feed error to LLM and ask it to fix
                action = await agent.repair_action(action, error=str(e))
                continue

            if strategy == RetryStrategy.FALLBACK:
                action = agent.get_fallback_action(action)
                continue

    raise MaxRetriesExceeded(f"Failed after {max_retries} attempts")

The Repair Strategy in Detail

The REPAIR strategy is where things get interesting. Instead of retrying the same prompt, you feed the error message back to the model as additional context:

async def repair_action(self, failed_action, error: str):
    repair_prompt = f"""Your previous tool call failed with this error:

Tool: {failed_action.tool_name}
Input: {json.dumps(failed_action.input)}
Error: {error}

Analyze the error and generate a corrected tool call.
Do NOT repeat the exact same input that caused the failure."""

    corrected = await self.llm.generate(repair_prompt)
    return corrected

This pattern resolves a significant share of validation errors on the first repair attempt. Wrong date formats, missing required fields, out-of-range values — these are exactly the kind of structured errors that models can self-correct when shown the specific error message. In practice, teams report repair success rates well above 50% for schema-level failures.

Pattern 3: Budget Governors

The scariest AI agent failure isn't a crash — it's a runaway cost spiral. An agent stuck in a reasoning loop can burn through hundreds of dollars in API costs before anyone notices. Budget governors are hard limits that prevent this.

Three Layers of Budget Control

interface BudgetConfig {
  maxTokensPerRequest: number;      // Single LLM call limit
  maxTokensPerSession: number;      // Entire conversation limit
  maxToolCallsPerSession: number;   // Prevent infinite tool loops
  maxCostPerSession: number;        // Dollar amount ceiling
  maxDurationSeconds: number;       // Wall-clock timeout
}

class BudgetGovernor {
  private usage = { tokens: 0, toolCalls: 0, cost: 0, startTime: Date.now() };

  check(config: BudgetConfig): void {
    if (this.usage.tokens > config.maxTokensPerSession) {
      throw new BudgetExceededError('Token budget exceeded');
    }
    if (this.usage.toolCalls > config.maxToolCallsPerSession) {
      throw new BudgetExceededError('Tool call limit exceeded — possible infinite loop');
    }
    if (this.usage.cost > config.maxCostPerSession) {
      throw new BudgetExceededError(`Cost ceiling hit: $${this.usage.cost.toFixed(2)}`);
    }
    const elapsed = (Date.now() - this.usage.startTime) / 1000;
    if (elapsed > config.maxDurationSeconds) {
      throw new BudgetExceededError(`Session timeout: ${elapsed.toFixed(0)}s`);
    }
  }

  recordUsage(tokens: number, cost: number, isToolCall: boolean): void {
    this.usage.tokens += tokens;
    this.usage.cost += cost;
    if (isToolCall) this.usage.toolCalls++;
  }
}

Setting the Right Limits

Limits that are too tight will break legitimate workflows. Limits that are too loose won't prevent real damage. Here's how to calibrate:

Budget Type	Development	Staging	Production
Tokens per session	50,000	30,000	20,000
Tool calls per session	50	25	15
Cost per session	$5.00	$2.00	$0.50
Timeout	5 min	3 min	2 min

Start restrictive in production and loosen based on actual usage data. It's far easier to increase limits than to explain a $2,000 surprise bill.

The "Stuck Detection" Pattern

Budget limits catch runaway agents, but you can detect the problem earlier by looking for repetitive behavior:

def detect_stuck_agent(tool_call_history: list[str], window: int = 5) -> bool:
    """Detect if agent is repeatedly calling the same tool without progress."""
    if len(tool_call_history) < window:
        return False

    recent = tool_call_history[-window:]
    # If >80% of recent calls are the same tool, agent is likely stuck
    most_common = max(set(recent), key=recent.count)
    return recent.count(most_common) / len(recent) >= 0.8

When stuck behavior is detected, inject a meta-prompt:

You appear to be repeating the same action without making progress. 
Stop and reconsider your approach. 
Either try a completely different strategy or inform the user 
that you cannot complete this specific task.

Pattern 4: Output Guardrails

The model will eventually generate something it shouldn't. PII in a customer-facing response. An SQL statement in a webhook payload. A hallucinated URL that leads to a phishing site. Output guardrails are your last line of defense before the agent's output reaches the user or an external system.

The Guardrail Pipeline

Run every agent output through a validation pipeline before it leaves your system:

interface Guardrail {
  name: string;
  check(output: string, context: AgentContext): GuardrailResult;
}

class GuardrailPipeline {
  private guardrails: Guardrail[] = [];

  async validate(output: string, context: AgentContext): Promise<string> {
    for (const guardrail of this.guardrails) {
      const result = guardrail.check(output, context);

      if (result.action === 'BLOCK') {
        throw new GuardrailViolation(guardrail.name, result.reason);
      }
      if (result.action === 'REDACT') {
        output = result.redactedOutput;  // Replace sensitive content
      }
      if (result.action === 'FLAG') {
        await this.alertOncall(guardrail.name, output, result.reason);
        // Continue but notify the team
      }
    }
    return output;
  }
}

Essential Guardrails for Production

1. PII Detection

const piiGuardrail: Guardrail = {
  name: 'pii-detector',
  check(output: string): GuardrailResult {
    const patterns = {
      ssn: /\b\d{3}-\d{2}-\d{4}\b/,
      email: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/,
      phone: /\b(\+\d{1,3}[-.]?)?\(?\d{3}\)?[-.]?\d{3}[-.]?\d{4}\b/,
      creditCard: /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
    };

    for (const [type, pattern] of Object.entries(patterns)) {
      if (pattern.test(output)) {
        return {
          action: 'REDACT',
          reason: `Detected ${type} in output`,
          redactedOutput: output.replace(pattern, `[REDACTED_${type.toUpperCase()}]`)
        };
      }
    }
    return { action: 'PASS' };
  }
};

2. Code Injection Prevention

const codeInjectionGuardrail: Guardrail = {
  name: 'code-injection',
  check(output: string, context: AgentContext): GuardrailResult {
    // Block if agent tries to return executable code in a text response
    const dangerousPatterns = [
      /DROP\s+TABLE/i, /DELETE\s+FROM/i, /UPDATE\s+.*SET/i,
      /<script\b[^>]*>/i,
      /eval\s*\(/i, /exec\s*\(/i,
      /rm\s+-rf/i
    ];

    if (context.responseType === 'user-facing') {
      for (const pattern of dangerousPatterns) {
        if (pattern.test(output)) {
          return { action: 'BLOCK', reason: `Dangerous pattern detected: ${pattern}` };
        }
      }
    }
    return { action: 'PASS' };
  }
};

3. Hallucination Anchor

const groundednessGuardrail: Guardrail = {
  name: 'groundedness',
  check(output: string, context: AgentContext): GuardrailResult {
    // If the agent references URLs, verify they exist in the source context
    const urls = output.match(/https?:\/\/[^\s)]+/g) || [];
    const sourceUrls = context.retrievedDocuments.flatMap(d => 
      d.content.match(/https?:\/\/[^\s)]+/g) || []
    );

    const fabricatedUrls = urls.filter(url => !sourceUrls.includes(url));
    if (fabricatedUrls.length > 0) {
      return {
        action: 'FLAG',
        reason: `Potentially fabricated URLs: ${fabricatedUrls.join(', ')}`
      };
    }
    return { action: 'PASS' };
  }
};

Pattern 5: The Kill Switch

Every production AI agent needs an emergency stop mechanism. Not "gracefully wind down over the next few minutes" — an immediate, hard stop that halts all agent activity across all instances.

Why You Need It

Kill switches aren't for normal error handling. They're for scenarios like:

The agent starts sending inappropriate content to customers
A prompt injection attack is actively being exploited
The agent is making unauthorized changes to production data
Cost is spiraling and budget governors aren't catching it (misconfigured limits)

Implementation: Feature Flag + Remote Config

The simplest and most reliable kill switch is a feature flag:

class AgentKillSwitch {
  // Check before EVERY agent action
  async checkBeforeAction(agentId: string): Promise<void> {
    // Remote config check (cached with 5s TTL)
    const config = await this.getRemoteConfig();

    if (config.globalKillSwitch) {
      throw new AgentHaltedError('All agents halted by global kill switch');
    }

    if (config.disabledAgents.includes(agentId)) {
      throw new AgentHaltedError(`Agent ${agentId} halted by targeted kill switch`);
    }

    // Check against real-time abuse signals
    if (await this.abuseDetector.isCompromised(agentId)) {
      await this.activateKillSwitch(agentId, 'Automated: abuse detected');
      throw new AgentHaltedError('Agent halted: abuse pattern detected');
    }
  }

  async activateKillSwitch(agentId: string, reason: string): Promise<void> {
    await this.remoteConfig.set(`agents.${agentId}.killed`, true);
    await this.alerting.sendPagerDutyAlert({
      severity: 'critical',
      summary: `Agent ${agentId} kill switch activated: ${reason}`,
    });
    await this.auditLog.record('KILL_SWITCH_ACTIVATED', { agentId, reason });
  }
}

The Critical Rule

The kill switch check must happen before every LLM call and every tool execution — not just at the start of a session. An agent session that started before the kill switch was activated must still be stopped mid-execution.

// In the main agent loop
while (hasMoreSteps) {
  await killSwitch.checkBeforeAction(this.agentId);  // <-- EVERY iteration

  const response = await llm.chat(messages);

  await killSwitch.checkBeforeAction(this.agentId);  // <-- After LLM, before tool

  if (response.toolCalls) {
    for (const call of response.toolCalls) {
      await killSwitch.checkBeforeAction(this.agentId);  // <-- Before each tool
      await executeTool(call);
    }
  }
}

Pattern 6: Observability and Tracing

You can't fix what you can't see. And AI agents are notoriously opaque — the same input can produce different reasoning chains, different tool call sequences, and different outputs. Traditional application monitoring (response times, error rates) tells you almost nothing about why an agent failed.

What to Trace

Every agent execution should produce a structured trace:

interface AgentTrace {
  traceId: string;
  sessionId: string;
  timestamp: string;

  // The full chain of reasoning
  steps: AgentStep[];

  // Aggregated metrics
  metrics: {
    totalTokens: number;
    totalCost: number;
    totalDuration: number;
    toolCallCount: number;
    retryCount: number;
    guardrailTriggered: boolean;
  };

  // Final outcome
  outcome: 'success' | 'failure' | 'timeout' | 'killed' | 'budget_exceeded';
  error?: string;
}

interface AgentStep {
  stepIndex: number;
  type: 'llm_call' | 'tool_call' | 'guardrail_check';

  // For LLM calls
  inputTokens?: number;
  outputTokens?: number;
  model?: string;

  // For tool calls
  toolName?: string;
  toolInput?: Record<string, any>;
  toolOutput?: string;
  toolDuration?: number;

  // For guardrails
  guardrailName?: string;
  guardrailAction?: 'PASS' | 'BLOCK' | 'REDACT' | 'FLAG';

  duration: number;
  error?: string;
}

The Three Dashboards You Need

1. Real-time Operations Dashboard

Metric	What It Tells You
Active sessions	How many agents are running right now
Error rate (5 min window)	Whether something just broke
P95 latency	User experience degradation
Cost per minute	Budget burn rate
Circuit breaker status	Which tools are failing

2. Quality Dashboard (Daily)

Metric	What It Tells You
Task completion rate	Are agents actually solving problems
Guardrail trigger rate	How often the model misbehaves
Retry rate per tool	Which integrations are flaky
Average steps per task	Whether prompts need optimization
User satisfaction (if available)	The only metric that ultimately matters

3. Incident Investigation View

When something goes wrong, you need to replay the exact sequence: Every message, every LLM response, every tool call input/output, every guardrail check. Store traces for at least 30 days. When an incident happens, this trace is your forensic evidence.

Practical Tip: Log the Prompt, Not Just the Response

Most teams log LLM responses but not the full prompt that was sent. This makes debugging impossible. Log the complete prompt (system message + conversation history + tool definitions) for every LLM call. Yes, it's verbose. Yes, it costs storage. It will save you hours of debugging when things go wrong.

Pattern 7: Human-in-the-Loop Approval Gates

Full autonomy is a goal, not a starting point. The most reliable production agents use tiered authorization — the agent can do low-risk things autonomously, but high-risk actions require human approval.

Defining Risk Tiers

enum RiskTier {
  LOW = 'low',       // Autonomous: read data, search, generate text
  MEDIUM = 'medium', // Notify: send emails, update records, modify configs
  HIGH = 'high',     // Approve: delete data, financial transactions, external API writes
  CRITICAL = 'critical', // Multi-approve: schema changes, access control, bulk operations
}

const toolRiskMap: Record<string, RiskTier> = {
  'search_documents': RiskTier.LOW,
  'generate_summary': RiskTier.LOW,
  'send_email': RiskTier.MEDIUM,
  'update_customer_record': RiskTier.MEDIUM,
  'delete_records': RiskTier.HIGH,
  'execute_sql': RiskTier.HIGH,
  'modify_billing': RiskTier.CRITICAL,
  'update_permissions': RiskTier.CRITICAL,
};

The Approval Flow

async function executeWithApproval(
  agent: Agent, 
  toolCall: ToolCall, 
  context: AgentContext
): Promise<ToolResult> {
  const risk = toolRiskMap[toolCall.name] || RiskTier.HIGH; // Default to HIGH

  switch (risk) {
    case RiskTier.LOW:
      return await executeTool(toolCall);

    case RiskTier.MEDIUM:
      // Execute but notify
      const result = await executeTool(toolCall);
      await notifyTeam(toolCall, result, context);
      return result;

    case RiskTier.HIGH:
      // Pause and wait for approval
      const approval = await requestApproval({
        toolCall,
        context,
        timeout: 300_000, // 5 minute timeout
      });

      if (approval.approved) {
        return await executeTool(toolCall);
      } else {
        return {
          role: 'tool',
          content: `Action was denied by reviewer: ${approval.reason}. ` +
                   `Please inform the user and suggest an alternative.`
        };
      }

    case RiskTier.CRITICAL:
      // Requires two independent approvals
      const approvals = await requestMultiApproval({
        toolCall,
        context,
        requiredApprovals: 2,
        timeout: 600_000, // 10 minute timeout
      });

      if (approvals.every(a => a.approved)) {
        return await executeTool(toolCall);
      } else {
        return { role: 'tool', content: 'Action requires additional approval.' };
      }
  }
}

The Practical Reality

Human-in-the-loop creates latency. A senior engineer reviewing an approval request takes 2-5 minutes. During that time, the agent is paused, the user is waiting, and resources are held open.

Mitigate this by:

Pre-approving common patterns. If the same tool call with similar parameters gets approved 20 times, auto-approve it going forward
Batching approvals. Group related high-risk actions into a single review ("The agent wants to update 3 customer records and send 2 emails — approve all?")
Async workflows. For non-urgent tasks, let the agent queue the action and notify the user when it's approved and completed
Progressive trust. Start with HITL for everything, then systematically lower the risk tier for specific tools as you gain confidence in the agent's reliability

Putting It All Together: The Reliability Stack

These seven patterns form layers of defense. No single pattern is sufficient; reliability comes from the combination:

┌─────────────────────────────────────────┐
│          Human-in-the-Loop              │  ← High-risk actions gated
├─────────────────────────────────────────┤
│          Output Guardrails              │  ← PII, injection, hallucination
├─────────────────────────────────────────┤
│          Budget Governors               │  ← Cost, tokens, time, tool calls
├─────────────────────────────────────────┤
│          Kill Switch                    │  ← Emergency stop
├─────────────────────────────────────────┤
│          Circuit Breakers               │  ← Tool failure isolation
├─────────────────────────────────────────┤
│          Retry-Classify                 │  ← Intelligent error recovery
├─────────────────────────────────────────┤
│          Observability                  │  ← Full trace of every decision
└─────────────────────────────────────────┘

The Implementation Order

Don't try to ship all seven at once. Implement in this order based on risk-to-effort ratio:

Budget Governors (Day 1) — Prevents financial damage immediately
Kill Switch (Day 1) — Your emergency brake, even if you never use it
Observability (Week 1) — You can't improve what you can't measure
Output Guardrails (Week 1-2) — Stop bad content from reaching users
Circuit Breakers (Week 2) — Isolate tool failures
Retry-Classify (Week 2-3) — Improve success rates
Human-in-the-Loop (Week 3-4) — Adds trust for high-stakes actions

The 2026 Reality

The AI agent ecosystem is maturing fast. Frameworks like LangGraph, CrewAI, and the Agents SDKs from OpenAI and Google are adding more built-in reliability primitives. But they're not enough on their own. Framework defaults are permissive — they're designed to make demos easy, not to keep production systems safe.

Your agent will eventually do something unexpected. The question isn't "if" but "when," and whether your reliability stack catches it before it reaches a user, a database, or a billing system.

The best AI agents aren't the smartest ones. They're the ones that fail gracefully.

⚡ Speed Tip: Read the original post on the Pockit Blog.

Tired of slow cloud tools? Pockit.tools runs entirely in your browser. Get the Extension now for instant, zero-latency access to essential dev tools.

htmx in 2026: When You Don't Need React (And When You Absolutely Do)

HK Lee — Tue, 31 Mar 2026 14:03:00 +0000

The frontend community has been arguing about htmx vs React for two years now, and most of the arguments are wrong. Not because either technology is bad, but because the debate frames them as competitors solving the same problem. They're not.

htmx crossed 47,000 GitHub stars. React still powers millions of production applications. The 2025 State of JS survey (published February 2026) shows htmx maintaining its "most admired" status among developers who've tried it, while React's satisfaction score dropped to its lowest point despite 83.6% usage. Something is clearly shifting, but what exactly?

This isn't another "htmx good, React bad" article. This is a decision framework. By the end, you'll know exactly when htmx is the right call, when React is still irreplaceable, and when the answer is "use both."

The Core Architectural Divide

Before comparing features, you need to understand the fundamental difference. htmx and React don't just use different APIs — they represent different philosophies about where application state should live.

React: The Client-Side Application Model

React treats the browser as an application runtime. Your server is an API that returns JSON. Your client is a full application that manages state, handles routing, renders UI, and orchestrates side effects.

[Server] → JSON → [React App] → DOM
                      ↑
              State Management
              Routing
              Side Effects
              Error Boundaries

This model is powerful. It enables offline-first applications, optimistic updates, complex animations, real-time collaboration, and rich interactive experiences. It's also complex. A typical React application in 2026 involves:

A meta-framework (Next.js, Remix, or similar)
A state management approach (React Context, Zustand, Jotai, or Server Components)
A data fetching layer (TanStack Query, SWR, or framework-native loaders)
A build pipeline (Vite, Turbopack, or Webpack)
TypeScript configuration
Testing infrastructure (Vitest, Playwright, Testing Library)

That's six layers of tooling before you write a single line of business logic.

htmx: The Hypermedia Model

htmx treats the browser as a document viewer that can swap parts of the document. Your server returns HTML fragments. The client has no application state, no routing logic, no build step.

[Server] → HTML → [Browser + htmx] → DOM (swap target)

The server is the application. The browser is the renderer. htmx is the bridge that lets you update parts of the page without full reloads.

<!-- A search form that updates results without a page reload -->
<input type="search" 
       name="q" 
       hx-get="/search" 
       hx-trigger="input changed delay:300ms" 
       hx-target="#results">

<div id="results">
  <!-- Server returns HTML fragments here -->
</div>

That's it. No JavaScript file. No build step. No state management. The server handles the search query, renders the results as HTML, and htmx swaps the #results div. The entire interaction is defined in HTML attributes.

Where htmx Wins: The 80% of Web Applications

Here's the uncomfortable truth that React advocates don't want to hear: most web applications are CRUD interfaces with forms, tables, filters, and navigation. They don't need a client-side application runtime. They need a server that renders HTML and a way to update parts of the page without reloading.

1. Admin Dashboards and Internal Tools

The single biggest category where htmx dominates. An admin panel for managing users, orders, or content is fundamentally a collection of:

Tables with sorting, filtering, and pagination
Forms for creating and editing records
Modal dialogs for confirmations
Toast notifications for feedback

With htmx, each of these is a server round-trip that returns a HTML fragment:

<!-- Sortable table header -->
<th hx-get="/users?sort=name&dir=asc" 
    hx-target="#user-table-body"
    hx-indicator="#loading">
  Name ↕
</th>

<!-- Delete with confirmation -->
<button hx-delete="/users/42" 
        hx-confirm="Delete this user?" 
        hx-target="closest tr" 
        hx-swap="outerHTML swap:500ms">
  Delete
</button>

<!-- Inline editing -->
<td hx-get="/users/42/edit-name" 
    hx-trigger="dblclick"
    hx-swap="innerHTML">
  John Doe
</td>

With React, the same functionality requires:

// State management for sort
const [sortConfig, setSortConfig] = useState({ key: 'name', dir: 'asc' });

// Data fetching with TanStack Query
const { data, isLoading, refetch } = useQuery({
  queryKey: ['users', sortConfig, filters, page],
  queryFn: () => fetchUsers({ ...sortConfig, ...filters, page }),
});

// Delete mutation with optimistic update
const deleteMutation = useMutation({
  mutationFn: deleteUser,
  onMutate: async (userId) => {
    await queryClient.cancelQueries(['users']);
    const previous = queryClient.getQueryData(['users']);
    queryClient.setQueryData(['users'], (old) => 
      old.filter(u => u.id !== userId)
    );
    return { previous };
  },
  onError: (err, userId, context) => {
    queryClient.setQueryData(['users'], context.previous);
  },
  onSettled: () => queryClient.invalidateQueries(['users']),
});

// Confirmation dialog state
const [deleteTarget, setDeleteTarget] = useState(null);
const [showConfirm, setShowConfirm] = useState(false);

The htmx version is 8 lines of HTML attributes. The React version is 25+ lines of JavaScript before you even touch the JSX. And the React version has more failure modes: stale cache, optimistic rollback bugs, race conditions between queries, and memory leaks from subscriptions.

2. Multi-Step Forms and Wizards

htmx handles partial form validation and multi-step flows naturally because the server controls the state:

<!-- Step 1: Server validates and returns step 2 -->
<form hx-post="/onboarding/step-1" 
      hx-target="#wizard-container"
      hx-swap="innerHTML">
  <input name="email" type="email" required>
  <input name="company" required>
  <button type="submit">Next</button>
</form>

The server validates step 1, stores progress in the session, and returns the HTML for step 2. No client-side form state management. No Formik, no React Hook Form, no Zod schemas wired to client components.

3. Content-Heavy Sites with Dynamic Sections

Blog platforms, documentation sites, e-commerce product pages — anywhere the core content is server-rendered but you need dynamic interactions (add to cart, live search, comment sections).

htmx lets you keep server-side rendering for SEO while adding targeted interactivity:

<!-- Add to cart without page reload -->
<button hx-post="/cart/add" 
        hx-vals='{"productId": "abc123", "qty": 1}'
        hx-target="#cart-count"
        hx-swap="innerHTML">
  Add to Cart
</button>

<!-- Live comment section -->
<div hx-get="/posts/42/comments" 
     hx-trigger="load, every 30s"
     hx-swap="innerHTML">
  <!-- Comments load here -->
</div>

The Performance Argument

For server-driven applications, htmx wins on every performance metric that matters:

Metric	htmx	React SPA
JavaScript bundle	~14 KB (htmx itself)	150-400 KB (framework + app)
Time to Interactive	Near-instant (no hydration)	Delayed (hydration required)
INP score	Excellent (minimal JS execution)	Variable (depends on component tree)
Server load	Higher (renders HTML)	Lower (returns JSON)
CDN cacheability	Excellent (HTML fragments cache well)	Poor (dynamic JSON responses)

The one area where React wins is server load. Rendering HTML server-side costs more CPU than serializing JSON. But in 2026, with edge computing and streaming HTML, this tradeoff increasingly favors htmx for most applications.

Where React is Still Irreplaceable

Now for the other side. There are applications where htmx genuinely cannot compete, and pretending otherwise is dishonest.

1. Rich Text Editors and Complex Interactive Components

Building something like Notion, Google Docs, or a collaborative whiteboard requires managing complex local state at microsecond granularity. Every keystroke, cursor position, selection range, and formatting operation must be handled client-side. A server round-trip for each character typed would create unbearable latency.

React (or its peers like Svelte/Vue) provides:

Virtual DOM for efficient batch updates
Controlled component patterns for input management
Fine-grained re-rendering for performance
Integration with collaboration protocols (CRDTs, OT)

htmx can't do this. A 200ms round-trip to the server for every keystroke isn't viable. This is client-side application territory.

2. Offline-First Applications

If your app needs to work without an internet connection — like a field service app, a mobile note-taking app, or a POS system — htmx is architecturally incompatible. htmx requires a server for every interaction. No server, no interaction.

React with a service worker, IndexedDB, and sync logic can handle days of offline operation and reconcile state when connectivity returns.

3. Real-Time Collaborative Features

Google Docs-style simultaneous editing, multiplayer games, collaborative design tools — anything where multiple users modify shared state at high frequency. These require:

Bidirectional WebSocket connections
Client-side conflict resolution
Optimistic local state updates
Complex undo/redo stacks

htmx's request-response model doesn't fit this pattern. You need a client application that maintains local state and synchronizes with peers.

4. High-Frequency UI Interactions

Drag-and-drop interfaces, interactive data visualizations (D3.js), animation-heavy UIs, canvas-based editors, and game-like interfaces require client-side frame-rate responsiveness. A server round-trip per drag frame isn't feasible.

The Decision Matrix

Use Case	htmx	React	Both
Admin dashboards	✅ Best	⚠️ Overkill	—
CRUD applications	✅ Best	⚠️ Overkill	—
Multi-step forms	✅ Good	✅ Good	—
E-commerce	✅ Good	✅ Good	✅ Best
Content sites with interactions	✅ Best	⚠️ Overkill	—
Rich text editors	❌ Can't	✅ Required	—
Offline-first apps	❌ Can't	✅ Required	—
Real-time collaboration	❌ Poor	✅ Required	—
Data visualization dashboards	⚠️ Limited	✅ Best	✅ Good
Drag-and-drop builders	❌ Can't	✅ Required	—

The "Use Both" Pattern: Islands of Interactivity

The most pragmatic answer in 2026 is often "use both." The architecture is called Islands of Interactivity — htmx handles the page structure, navigation, and standard CRUD, while isolated React (or Preact/Svelte) components handle the complex interactive bits.

How It Works

<!-- htmx-driven page shell -->
<main>
  <!-- htmx handles navigation -->
  <nav hx-boost="true">
    <a href="/dashboard">Dashboard</a>
    <a href="/analytics">Analytics</a>
    <a href="/settings">Settings</a>
  </nav>

  <!-- htmx handles the table/CRUD -->
  <section hx-get="/dashboard/table" hx-trigger="load" hx-target="this">
    Loading...
  </section>

  <!-- React island for the interactive chart -->
  <div id="analytics-chart" 
       data-config='{"range": "30d", "metrics": ["revenue", "users"]}'>
  </div>

  <script type="module">
    import { renderChart } from './chart-island.js';
    renderChart(document.getElementById('analytics-chart'));
  </script>
</main>

This pattern gives you:

Fast page loads — no hydration for the page shell
SEO-friendly — server-rendered HTML throughout
Rich interactions — React for the chart, editor, or complex widget
Simple state — htmx manages 90% of the page, React manages 10%

Companies like GitHub have used this pattern for years — the bulk of GitHub is server-rendered HTML with targeted JavaScript for interactive components like the code editor, file tree, and Actions workflow builder.

Migration: Extracting React from Your SPA

If you're considering moving from a React SPA to htmx, don't do a Big Bang rewrite. Use the Strangler Fig pattern: replace pages one at a time, starting with the simplest ones.

Step 1: Identify Low-Interaction Pages

Audit your React app. Find pages that are essentially:

Data tables with filters
Settings forms
Detail views
List/grid views

These are your migration candidates. They probably represent 60-80% of your page count but less than 20% of your interactive complexity.

Step 2: Server-Side Routes

For each migration candidate, create a server-side route that returns full HTML:

# Django example
def user_list(request):
    users = User.objects.filter(active=True)
    sort = request.GET.get('sort', 'name')

    if request.headers.get('HX-Request'):
        # htmx request: return just the table body
        return render(request, 'users/_table_body.html', {'users': users})

    # Full page request: return complete page
    return render(request, 'users/list.html', {'users': users})

The HX-Request header check is the key pattern. Same route, same logic, but htmx gets an HTML fragment while direct navigation gets the full page.

Step 3: Keep React Where It Matters

Don't migrate your interactive components. Keep the React rich text editor, the drag-and-drop kanban board, the real-time chat widget. Wrap them as self-contained islands that mount into server-rendered pages.

The ROI of Migration

Teams that have migrated CRUD-heavy React SPAs to htmx consistently report:

40-60% reduction in frontend code (measured in lines of JS/TS removed)
Faster feature delivery for standard CRUD screens (no client state management to wire up)
Improved Core Web Vitals (LCP and INP improvements from eliminated hydration)
Simplified debugging (server logs show the full story; no need to reproduce client state)

The tradeoff is increased server rendering load and slightly higher latency for interactions that were previously client-side. For internal tools and admin panels, this tradeoff is almost always worth it. For consumer-facing SPAs with complex interactions, it rarely is.

The Backend Factor: htmx Shines with Certain Stacks

One underappreciated aspect of htmx is how it unlocks backend teams. If your team is primarily Go, Python (Django/Flask), Ruby (Rails), Rust, PHP (Laravel), or Java (Spring) developers, htmx lets them build interactive web applications without learning React.

This isn't a "backend devs can't learn frontend" argument. It's a productivity argument. A Django developer can ship an interactive admin panel with htmx in a day. The same developer would need a week to set up Next.js, learn React patterns, configure TanStack Query, and wire up API routes.

htmx + Backend Language Ecosystem

Backend	htmx Integration	Template Engine	Production Examples
Go	Excellent (templ, standard library)	templ, html/template	Infrastructure dashboards
Python/Django	Excellent (django-htmx)	Django templates	Admin panels, CMS
Ruby/Rails	Excellent (Turbo + htmx)	ERB, Haml	SaaS admin interfaces
Rust/Axum	Growing (askama, maud)	askama, maud	High-performance dashboards
PHP/Laravel	Excellent (native Blade)	Blade	E-commerce admin, CRM
Java/Spring	Good (Thymeleaf)	Thymeleaf	Enterprise CRUD apps

The common pattern: htmx adoption is highest in ecosystems where server-side rendering is already the default and adding client-side interactivity was previously the painful part.

Common htmx Mistakes (And How to Avoid Them)

1. Treating htmx Like a Client-Side Framework

The biggest mistake is managing state on the client. If you find yourself writing JavaScript to track which tab is active, which filters are applied, or which items are selected, you're fighting htmx's architecture.

Wrong:

// Don't do this
let activeFilters = [];
document.querySelectorAll('.filter-checkbox').forEach(cb => {
  cb.addEventListener('change', () => {
    if (cb.checked) activeFilters.push(cb.value);
    else activeFilters = activeFilters.filter(f => f !== cb.value);
    htmx.ajax('GET', `/items?filters=${activeFilters.join(',')}`, '#results');
  });
});

Right:

<!-- Let the server manage filter state -->
<form hx-get="/items" hx-target="#results" hx-trigger="change">
  <label><input type="checkbox" name="filter" value="active"> Active</label>
  <label><input type="checkbox" name="filter" value="premium"> Premium</label>
  <label><input type="checkbox" name="filter" value="new"> New</label>
</form>

The form serializes automatically. The server reads the filter parameters, queries the database, and returns HTML. No client-side state management needed.

2. Making Too Many Requests

htmx makes server requests easy, which can lead to chatty applications. Use hx-trigger modifiers to debounce and throttle:

<!-- Debounce search input -->
<input hx-get="/search" 
       hx-trigger="input changed delay:300ms" 
       hx-target="#results"
       name="q">

<!-- Throttle scroll-based loading -->
<div hx-get="/feed/next" 
     hx-trigger="revealed throttle:500ms"
     hx-swap="afterend">
</div>

3. Forgetting About Loading States

htmx provides the hx-indicator pattern for loading states. Use it consistently:

<button hx-post="/process" hx-indicator="#spinner">
  Submit
  <img id="spinner" class="htmx-indicator" src="/spinner.svg">
</button>

.htmx-indicator { display: none; }
.htmx-request .htmx-indicator { display: inline; }
.htmx-request.htmx-indicator { display: inline; }

The 2026 Reality Check

The industry is converging on a pragmatic middle ground:

SPAs are not dead. Complex, interactive applications still need client-side frameworks. React, Vue, and Svelte aren't going anywhere.
SPAs are overused. Too many applications that are fundamentally server-driven CRUD were built as client-side SPAs because "that's how we do things now." htmx exposes this overengineering.
The best architecture is the one that matches your problem. A Notion clone needs React. An admin dashboard needs htmx. A SaaS product might need both.
htmx is not "going back." It's not PHP-era development. It's a modern approach that leverages HTTP semantics, modern browsers, and the fact that most interactions are request-response patterns. The server-rendering infrastructure of 2026 (edge functions, streaming HTML, CDN-cached fragments) makes this approach faster and more scalable than ever.
The JavaScript ecosystem fatigue is real. htmx's growth isn't just about technical merit. It's about developers who are tired of maintaining 200MB node_modules directories, debugging webpack configurations, and learning a new state management library every six months.

Make the architectural choice based on what your application actually needs, not what Twitter says is cool. For most web applications, that means less JavaScript, not more.

💡 Note: This article was originally published on the Pockit Blog.

Check out Pockit.tools for 60+ free developer utilities. For faster access, add it to Chrome and use JSON Formatter & Diff Checker directly from your toolbar.

CSS Anchor Positioning: The End of JavaScript Tooltip Libraries (Complete Guide)

HK Lee — Mon, 30 Mar 2026 13:44:00 +0000

Every frontend developer has fought the tooltip positioning battle. You need a dropdown menu attached to a button, a tooltip that follows its trigger, or a popover that stays anchored to a specific element. The solution has always been the same: install a JavaScript library, calculate coordinates on every scroll and resize event, handle overflow detection, and pray that the z-index stacking context doesn't break everything.

Floating UI (the successor to Popper.js) has over 23 million weekly npm downloads for its core package alone. That's millions of projects compensating for something CSS couldn't do natively — until now.

The CSS Anchor Positioning API has hit Baseline 2026 with full support across Chrome 125+, Firefox 147+, and Safari 26. This isn't an experimental feature behind a flag. It's production-ready, and it fundamentally changes how we build positioned UI elements on the web.

This guide covers everything: the core API, advanced fallback strategies, integration with the Popover API, real-world migration patterns from JavaScript libraries, and the edge cases that will bite you if you're not prepared.

The Problem CSS Anchor Positioning Solves

Before this API existed, positioning an element relative to another element required one of three approaches:

1. Absolute Positioning with Manual Offsets — Fragile, breaks on scroll, doesn't handle overflow.

/* The classic hack */
.tooltip-wrapper {
  position: relative;
}
.tooltip {
  position: absolute;
  bottom: 100%;
  left: 50%;
  transform: translateX(-50%);
}

This works for static layouts but falls apart when the trigger element is near the viewport edge, the page scrolls, or the container has overflow: hidden.

2. JavaScript Position Calculation — The industry standard for a decade.

// What Floating UI does under the hood (simplified)
function updatePosition(anchor, floating) {
  const anchorRect = anchor.getBoundingClientRect();
  const floatingRect = floating.getBoundingClientRect();

  let top = anchorRect.bottom + 8;
  let left = anchorRect.left + (anchorRect.width - floatingRect.width) / 2;

  // Overflow detection
  if (top + floatingRect.height > window.innerHeight) {
    top = anchorRect.top - floatingRect.height - 8; // flip to top
  }
  if (left < 0) left = 8; // shift right
  if (left + floatingRect.width > window.innerWidth) {
    left = window.innerWidth - floatingRect.width - 8; // shift left
  }

  floating.style.top = `${top}px`;
  floating.style.left = `${left}px`;
}

// Must run on scroll, resize, and mutation
window.addEventListener('scroll', () => updatePosition(anchor, floating), { passive: true });
window.addEventListener('resize', () => updatePosition(anchor, floating));
const observer = new ResizeObserver(() => updatePosition(anchor, floating));
observer.observe(anchor);

This works, but you're running layout calculations in JavaScript on every scroll frame. You're fighting the browser's rendering pipeline instead of working with it.

3. CSS Anchor Positioning — The native solution.

.trigger {
  anchor-name: --my-trigger;
}

.tooltip {
  position: fixed;
  position-anchor: --my-trigger;
  bottom: anchor(top);
  left: anchor(center);
  translate: -50% 0;

  /* Automatic overflow handling */
  position-try-fallbacks: flip-block;
}

No JavaScript. No scroll listeners. No resize observers. No getBoundingClientRect(). The browser handles everything in the rendering pipeline, where it belongs.

Core API: The Three Building Blocks

CSS Anchor Positioning is built on three concepts: declaring anchors, positioning against anchors, and handling overflow.

1. Declaring an Anchor

Any element can become an anchor by giving it a name with the anchor-name property:

.profile-avatar {
  anchor-name: --avatar;
}

.settings-gear {
  anchor-name: --settings-btn;
}

Rules:

Names must use the CSS dashed-ident syntax (-- prefix).
An element can only have one anchor name at a time.
Anchor names exist in the same scope as the positioned element — they follow containing block rules.

You can also declare anchors inline with HTML:

<button style="anchor-name: --menu-trigger">Menu</button>

2. Positioning Against an Anchor

To position an element relative to an anchor, you need three things:

The positioned element must have position: absolute or position: fixed.
Link it to an anchor via position-anchor.
Use the anchor() function in inset properties (top, right, bottom, left).

.status-badge {
  position: fixed;
  position-anchor: --avatar;

  /* Position at the bottom-right corner of the avatar */
  top: anchor(bottom);
  left: anchor(right);
}

The anchor() function accepts a side keyword that refers to the anchor's geometry:

`anchor()` value	What it means
`anchor(top)`	The anchor's top edge
`anchor(bottom)`	The anchor's bottom edge
`anchor(left)`	The anchor's left edge
`anchor(right)`	The anchor's right edge
`anchor(center)`	The anchor's center point (on the relevant axis)
`anchor(start)`	Logical start (respects writing direction)
`anchor(end)`	Logical end

You can also use anchor() with a percentage:

.tooltip {
  position: fixed;
  position-anchor: --trigger;

  /* Position at 25% from the anchor's left edge */
  left: anchor(25%);
  bottom: anchor(top);
}

The `position-area` Shorthand

For common placements, position-area provides a simpler mental model. Instead of thinking about individual sides, you think about a 3×3 grid around the anchor:

┌──────────┬──────────┬──────────┐
│ top left │ top      │ top right│
├──────────┼──────────┼──────────┤
│ left     │ center   │ right    │
├──────────┼──────────┼──────────┤
│ bottom   │ bottom   │ bottom   │
│ left     │          │ right    │
└──────────┴──────────┴──────────┘

/* Tooltip below the anchor, centered */
.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;
  margin-top: 8px;
}

/* Sidebar to the right of the anchor */
.sidebar {
  position: fixed;
  position-anchor: --panel;
  position-area: right;
}

/* Notification badge at top-right corner */
.badge {
  position: absolute;
  position-anchor: --icon;
  position-area: top right;
}

position-area is usually what you want. Use raw anchor() functions only when you need sub-element precision (like "20% from the left edge of the anchor").

3. Overflow Handling with `position-try-fallbacks`

This is where CSS Anchor Positioning truly beats JavaScript libraries. The position-try-fallbacks property defines what happens when the positioned element would overflow its containing block:

.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;
  margin-top: 8px;

  /* If it overflows the bottom, flip to the top */
  position-try-fallbacks: flip-block;
}

Built-in strategies:

Strategy	Behavior
`flip-block`	Flips the element to the opposite side on the block axis (top ↔ bottom)
`flip-inline`	Flips on the inline axis (left ↔ right)
`flip-block flip-inline`	Tries flipping on both axes

For more control, define custom fallback positions with @position-try:

@position-try --above {
  position-area: top center;
  margin-bottom: 8px;
}

@position-try --left-side {
  position-area: left;
  margin-right: 8px;
}

@position-try --right-side {
  position-area: right;
  margin-left: 8px;
}

.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;
  margin-top: 8px;

  /* Try each fallback in order until one fits */
  position-try-fallbacks: --above, --left-side, --right-side;
}

The browser evaluates each fallback in order and picks the first one that doesn't cause overflow. This happens automatically during layout — no JavaScript, no requestAnimationFrame, no resize observers.

Integration with the Popover API

CSS Anchor Positioning becomes even more powerful when combined with the HTML Popover API (popover attribute). Together, they provide the complete tooltip/dropdown solution:

<button popovertarget="user-menu" style="anchor-name: --menu-btn">
  Settings ⚙️
</button>

<div id="user-menu" popover anchor="menu-btn">
  <nav>
    <a href="/profile">Profile</a>
    <a href="/settings">Settings</a>
    <a href="/logout">Log out</a>
  </nav>
</div>

[popover] {
  position: fixed;
  position-anchor: --menu-btn;
  position-area: bottom left;
  margin-top: 4px;

  position-try-fallbacks: flip-block;
}

What you get for free:

Top layer rendering — No z-index wars. Popovers render in the browser's top layer.
Light dismiss — Clicking outside automatically closes the popover.
Accessible by default — Keyboard navigation and focus management handled by the browser.
Anchor positioning — The menu stays attached to its trigger, handles overflow, and adjusts on scroll.

Zero JavaScript. All of it.

Tooltip pattern with Popover

<button popovertarget="tip" popovertargetaction="toggle"
        style="anchor-name: --help-btn">
  Help?
</button>

<div id="tip" popover="hint">
  This action cannot be undone.
</div>

#tip {
  position: fixed;
  position-anchor: --help-btn;
  position-area: top center;
  margin-bottom: 8px;

  position-try-fallbacks: flip-block, flip-inline;

  /* Style it like a tooltip */
  background: var(--surface-inverse);
  color: var(--text-inverse);
  padding: 6px 12px;
  border-radius: 6px;
  font-size: 0.85rem;
  max-width: 240px;
}

The popover="hint" variant is non-modal and doesn't steal focus — perfect for tooltips.

Note: popover="hint" is part of the Interop 2026 focus area and browser support is still growing. If cross-browser compatibility is critical, use popover="manual" with JavaScript show/hide logic as a fallback.

Real-World Patterns

Pattern 1: Dropdown Menu with Submenus

/* Primary menu trigger */
.nav-item {
  anchor-name: --nav-item;
}

/* Dropdown */
.dropdown {
  position: fixed;
  position-anchor: --nav-item;
  position-area: bottom span-right;
  margin-top: 4px;

  position-try-fallbacks: flip-block;
}

/* Submenu items act as anchors for their submenus */
.dropdown-item {
  anchor-name: --sub-trigger;
}

.submenu {
  position: fixed;
  position-anchor: --sub-trigger;
  position-area: right;
  margin-left: 2px;

  position-try-fallbacks: flip-inline;
}

Note the span-right keyword in position-area — it stretches the positioned element to span from the anchor's position toward the right, which is exactly how dropdown menus should behave. Submenus use flip-inline to flip from right to left when they'd overflow the viewport.

Pattern 2: Form Field Validation Tooltip

<div class="field">
  <label for="email">Email</label>
  <input id="email" type="email" style="anchor-name: --email-input"
         required placeholder="you@example.com">
  <div class="validation-msg" popover="hint" id="email-error">
    Please enter a valid email address
  </div>
</div>

#email-error {
  position: fixed;
  position-anchor: --email-input;
  position-area: right;
  margin-left: 12px;

  position-try-fallbacks: --below-field;

  /* Danger styling */
  background: var(--color-danger-surface);
  color: var(--color-danger-text);
  border: 1px solid var(--color-danger-border);
  padding: 8px 12px;
  border-radius: 6px;
  font-size: 0.85rem;
  max-width: 200px;
}

@position-try --below-field {
  position-area: bottom span-right;
  margin-top: 4px;
  margin-left: 0;
}

On desktop, the validation message appears to the right of the input. On mobile or narrow viewports where there's no room, it automatically falls back to below the field. No media queries needed.

Pattern 3: Contextual Annotations (Google Docs Style)

/* Each comment marker is an anchor */
.comment-marker {
  anchor-name: --comment;
  background: var(--highlight-yellow);
}

.comment-thread {
  position: fixed;
  position-anchor: --comment;
  position-area: right;
  margin-left: 24px;
  width: 280px;

  position-try-fallbacks: --left-side;
}

@position-try --left-side {
  position-area: left;
  margin-right: 24px;
}

Migrating from JavaScript Libraries

If you're using Floating UI, Tippy.js, or a similar library, here's how the concepts map:

Floating UI → CSS Anchor Positioning

Floating UI concept	CSS equivalent
`computePosition()`	`position-anchor` + `anchor()`
`placement: 'bottom'`	`position-area: bottom center`
`flip()` middleware	`position-try-fallbacks: flip-block`
`shift()` middleware	`position-try-fallbacks` with custom `@position-try`
`offset(8)`	`margin-top: 8px` (or relevant margin)
`autoUpdate()`	Built-in (automatic)
`arrow` middleware	CSS `::before` pseudo-element with `anchor()`

Migration Example

Before (Floating UI):

import { computePosition, flip, shift, offset } from '@floating-ui/dom';

const button = document.querySelector('#trigger');
const tooltip = document.querySelector('#tooltip');

function update() {
  computePosition(button, tooltip, {
    placement: 'bottom',
    middleware: [offset(8), flip(), shift({ padding: 5 })],
  }).then(({ x, y }) => {
    Object.assign(tooltip.style, {
      left: `${x}px`,
      top: `${y}px`,
    });
  });
}

// Must be called on every scroll, resize, and DOM mutation
const cleanup = autoUpdate(button, tooltip, update);

After (CSS Anchor Positioning):

<button style="anchor-name: --trigger">Hover me</button>
<div class="tooltip">Tooltip content</div>

.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;
  margin-top: 8px;

  position-try-fallbacks: flip-block, flip-inline;
}

The JavaScript version requires importing a library (~3KB min+gzip for Floating UI core), setting up auto-update listeners, and cleaning them up on unmount. The CSS version is three properties and zero JavaScript.

What About the Arrow?

JavaScript tooltip libraries typically provide arrow middleware. With CSS Anchor Positioning, you use a pseudo-element:

.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;
  margin-top: 12px;

  position-try-fallbacks: flip-block;
}

.tooltip::before {
  content: '';
  position: absolute;
  bottom: 100%;
  left: anchor(--trigger center);
  translate: -50% 0;

  border: 6px solid transparent;
  border-bottom-color: var(--tooltip-bg);
}

The key insight: the arrow pseudo-element uses anchor() to stay aligned with the trigger's center, even if the tooltip shifts position due to overflow handling. This is something that JavaScript libraries handle with dedicated middleware — here it's one CSS property.

Edge Cases and Gotchas

1. Anchor Scope and Visibility

Anchors must be in the same containing block scope as the positioned element. If your anchor is inside a position: relative container with overflow: hidden, the positioned element won't be able to "see" outside that container.

Solution: Use position: fixed for the positioned element and ensure the anchor is accessible from the fixed positioning context.

2. Multiple Elements with the Same Anchor Name

If multiple elements have the same anchor-name, only the last one in DOM order becomes the active anchor. This is by design but can cause confusion.

/* DON'T: Both buttons have the same anchor name */
.btn { anchor-name: --btn; }

/* The tooltip will anchor to the LAST .btn in the DOM */
.tooltip { position-anchor: --btn; }

Solution: Use unique anchor names, or use the anchor-name property dynamically with CSS custom properties.

3. Implicit Anchoring with the `anchor` Attribute

The HTML anchor attribute provides implicit anchoring without needing position-anchor in CSS:

<button id="my-btn">Click</button>
<div anchor="my-btn" class="popup">Content</div>

.popup {
  position: fixed;
  /* No position-anchor needed — the HTML anchor attribute does it */
  position-area: bottom center;
}

Note: The anchor HTML attribute uses the element's ID (without -- prefix), while the CSS anchor-name property uses the CSS dashed-ident (with -- prefix). Don't mix them up.

4. Animation Considerations

When animating anchor-positioned elements, use CSS transitions on the positioned element itself, not on the anchor position properties:

.tooltip {
  position: fixed;
  position-anchor: --trigger;
  position-area: bottom center;

  /* Animate the tooltip, not the position */
  opacity: 0;
  transform: translateY(-4px);
  transition: opacity 0.2s, transform 0.2s;
}

.tooltip:popover-open {
  opacity: 1;
  transform: translateY(0);
}

/* Entry animation from display: none */
@starting-style {
  .tooltip:popover-open {
    opacity: 0;
    transform: translateY(-4px);
  }
}

The @starting-style rule is critical here — it defines the starting state for CSS transitions when an element goes from display: none to being visible (which is what happens when a popover opens).

5. Dynamic Anchor Switching

You can change which anchor an element is positioned against dynamically:

.contextual-toolbar {
  position: fixed;
  position-anchor: --active-selection;
  position-area: top center;
  margin-bottom: 8px;
}

// When user selects text, update the anchor
document.addEventListener('selectionchange', () => {
  const selection = window.getSelection();
  if (selection.rangeCount > 0) {
    const range = selection.getRangeAt(0);
    const marker = document.getElementById('selection-marker');
    const rect = range.getBoundingClientRect();
    marker.style.cssText = `
      anchor-name: --active-selection;
      position: fixed;
      top: ${rect.top}px;
      left: ${rect.left}px;
      width: ${rect.width}px;
      height: ${rect.height}px;
      pointer-events: none;
    `;
  }
});

This pattern is useful for contextual toolbars (like Medium's floating formatting toolbar) where the anchor position changes based on user interaction.

Performance: Why Native Beats JavaScript

The performance difference between CSS Anchor Positioning and JavaScript-based positioning isn't marginal — it's architectural.

JavaScript Positioning Pipeline

User scrolls
  → scroll event fires
    → JavaScript runs getBoundingClientRect() (forces layout)
      → JavaScript calculates new position
        → JavaScript updates element.style (triggers layout)
          → Browser re-renders

Every scroll frame: Layout → JS → Layout → Paint → Composite

This creates a layout thrashing loop. getBoundingClientRect() forces the browser to calculate layout, then setting style.top triggers another layout pass. In the worst case, this happens 60+ times per second.

CSS Anchor Positioning Pipeline

User scrolls
  → Browser updates positions during normal layout pass
    → Paint → Composite

Every scroll frame: Layout (includes positioning) → Paint → Composite

The browser handles anchor positioning during its normal layout phase. There's no JavaScript interruption, no forced layout calculations, no double layout passes. Anchor positions are resolved in the same pass as regular CSS layout.

What This Means in Practice

For a page with 20 tooltips/popovers:

JavaScript approach: 20 scroll listeners, each forcing layout recalculation. Total cost: measurable frame drops during fast scrolling.
CSS approach: Zero scroll listeners. All 20 positions resolved in a single layout pass. Zero JavaScript execution during scroll.

On mobile devices where CPU is constrained and every millisecond of main thread time matters, this difference is the gap between smooth 60fps scrolling and noticeable jank.

Browser Support and Progressive Enhancement

As of March 2026, CSS Anchor Positioning is fully supported in:

Chrome/Edge 125+ (since June 2024)
Firefox 147+ (since early 2026)
Safari 26+ (since early 2026)

For older browsers, use @supports to provide a fallback:

/* Fallback: simple absolute positioning */
.tooltip {
  position: absolute;
  bottom: 100%;
  left: 50%;
  transform: translateX(-50%);
}

/* Progressive enhancement: use anchor positioning if supported */
@supports (anchor-name: --a) {
  .tooltip {
    position: fixed;
    position-anchor: --trigger;
    position-area: top center;
    margin-bottom: 8px;
    translate: none;
    inset: auto;

    position-try-fallbacks: flip-block;
  }
}

If your users are on modern browsers (which is increasingly likely in 2026), you can use anchor positioning without fallbacks.

Conclusion

CSS Anchor Positioning is the most significant layout primitive added to CSS since Flexbox and Grid. It solves a problem that has required JavaScript for over a decade — positioning elements relative to other elements with overflow awareness.

The API is deceptively simple. Three properties (anchor-name, position-anchor, position-area) handle 80% of use cases. @position-try fallbacks handle the remaining 20%. Combined with the Popover API, you get complete tooltip, dropdown, and popover behavior with zero JavaScript.

For new projects in 2026, there is no reason to install Floating UI, Popper.js, or Tippy.js for basic anchored positioning. The browser does it natively, more performantly, and with less code.

For existing projects, the migration path is straightforward: replace computePosition() calls with CSS properties, remove scroll/resize listeners, and delete the positioning library from your dependencies. Your users get smoother scrolling, your bundle gets smaller, and your code gets simpler.

The tooltip positioning war is over. CSS won.

🔒 Privacy First: This article was originally published on the Pockit Blog.

Stop sending your data to random servers. Use Pockit.tools for secure utilities, or install the Chrome Extension to keep your files 100% private and offline.

Signals en JavaScript: Por Qué Todos los Frameworks los Están Adoptando (Y Qué Significa para React)

HK Lee — Fri, 27 Mar 2026 13:40:00 +0000

Algo inusual está pasando en el mundo frontend. Angular adoptó Signals. Svelte reemplazó su modelo de reactividad con Runes. Solid.js fue construido sobre ellos desde el primer día. El sistema de reactividad de Vue siempre fue similar a signals por debajo. Qwik, Preact, e incluso frameworks legacy como Ember — todos convergen hacia la misma primitiva.

Mientras tanto, React — el framework que domina el mercado — está yendo deliberadamente en la dirección opuesta, apostando todo a un compilador para solucionar los problemas de rendimiento que los signals resuelven por diseño.

La propuesta TC39 Signals, que ya avanza en el proceso de estandarización de JavaScript, busca incorporar esta primitiva reactiva directamente en el lenguaje. Si tiene éxito, los signals no serán solo una funcionalidad de framework — serán parte de JavaScript, como Promise o Array.

Este es el cambio arquitectónico más significativo en el desarrollo frontend desde que se introdujo el Virtual DOM hace más de una década. Vamos a analizar qué está pasando realmente, por qué importa y qué significa para el código que escribes hoy.

¿Qué Son los Signals, Exactamente?

Si quitamos las APIs específicas de cada framework, un signal es un concepto engañosamente simple: un contenedor reactivo para un valor que notifica automáticamente a sus dependientes cuando cambia.

Pensalo como una variable observable con tracking automático de dependencias. Cuando leés un signal dentro de una computación, el runtime recuerda esa dependencia. Cuando el valor del signal cambia, solo las computaciones que realmente dependen de él se re-ejecutan.

// Crear un valor reactivo
const count = signal(0);

// Computación derivada — trackea dependencias automáticamente
const doubled = computed(() => count.value * 2);

// Efecto secundario — se re-ejecuta cuando las dependencias cambian
effect(() => {
  console.log(`Count: ${count.value}, doubled: ${doubled.value}`);
});

// Solo se re-ejecutan las computaciones que dependen de `count`
count.value = 5;
// Consola: "Count: 5, doubled: 10"

No hay array de dependencias. No hay gestión manual de suscripciones. No hay algoritmo de diffing. El runtime sabe exactamente qué depende de qué porque observó el grafo de dependencias en tiempo de ejecución.

Las Tres Primitivas

Toda implementación de signals, sin importar el framework, está construida sobre tres primitivas:

1. Signal (Estado) — Un contenedor reactivo que mantiene un único valor.

const name = signal("Alice");
console.log(name.value); // "Alice"
name.value = "Bob";      // Notifica a los dependientes

2. Computed (Estado Derivado) — Un valor derivado de uno o más signals. Es lazy: solo recalcula cuando se lee, y solo si una dependencia cambió.

const firstName = signal("Alice");
const lastName = signal("Smith");

const fullName = computed(() => `${firstName.value} ${lastName.value}`);
// fullName no recalcula hasta que lo leas Y una dependencia haya cambiado

3. Effect (Efectos Secundarios) — Una función que se ejecuta cuando sus dependencias trackeadas cambian. Acá es donde hacés updates del DOM, requests de red o logging.

effect(() => {
  document.title = fullName.value; // Se re-ejecuta solo cuando fullName cambia
});

Este modelo de tres primitivas es la base. Ahora veamos qué tan profundo llega esto.

Cómo Funcionan los Signals Internamente

La magia de los signals no está en la API — está en el tracking automático de dependencias. Vamos a construir un runtime simplificado de signals desde cero para entender la mecánica interna.

El Grafo de Dependencias

En su núcleo, un runtime de signals mantiene un grafo acíclico dirigido (DAG) de dependencias:

┌─────────┐     ┌─────────┐
│ signal A │────▶│computed C│────▶ effect E
└─────────┘     └─────────┘
┌─────────┐        ▲
│ signal B │────────┘
└─────────┘

Cuando Signal A cambia, el runtime recorre el grafo y solo re-ejecuta Computed C y Effect E. Los otros dependientes de Signal B (si existen) quedan intactos.

Una Implementación Mínima

Acá hay una implementación funcional de signals en ~50 líneas de JavaScript:

let currentObserver = null;

function signal(initialValue) {
  let value = initialValue;
  const subscribers = new Set();

  return {
    get value() {
      // Track: si alguien está observando, registrar este signal
      if (currentObserver) {
        subscribers.add(currentObserver);
      }
      return value;
    },
    set value(newValue) {
      if (newValue === value) return; // Skip si no cambió
      value = newValue;
      // Notificar: re-ejecutar todos los suscriptores
      for (const subscriber of subscribers) {
        subscriber();
      }
    }
  };
}

function computed(fn) {
  let cachedValue;
  let dirty = true;

  const computation = () => { dirty = true; };

  return {
    get value() {
      if (dirty) {
        const prevObserver = currentObserver;
        currentObserver = computation;
        cachedValue = fn();
        currentObserver = prevObserver;
        dirty = false;
      }
      return cachedValue;
    }
  };
}

function effect(fn) {
  const execute = () => {
    const prevObserver = currentObserver;
    currentObserver = execute;
    fn();
    currentObserver = prevObserver;
  };
  execute(); // Ejecutar inmediatamente para establecer dependencias
}

La técnica clave es el stack global de observers (currentObserver). Cuando un computed o effect se ejecuta, se establece como el observer actual. Cualquier signal que se lea durante esa ejecución agrega automáticamente al observer a su conjunto de suscriptores. Por eso nunca necesitás declarar dependencias manualmente.

Optimizaciones de Producción

Las implementaciones reales agregan varias optimizaciones críticas:

1. Evaluación Push-Pull — En vez de re-ejecutar todos los suscriptores cuando un signal cambia, las implementaciones modernas marcan las computaciones downstream como "dirty" (push) y solo recalculan cuando se lee su valor (pull).

2. Ejecución Libre de Glitches — Si Signal A y Signal B cambian en la misma microtarea, un computed que depende de ambos debería ejecutarse una sola vez, no dos.

const a = signal(1);
const b = signal(2);
const sum = computed(() => a.value + b.value);

batch(() => {
  a.value = 10;
  b.value = 20;
});
// sum solo se ejecuta una vez con (10, 20)

3. Limpieza Automática — Cuando un effect se re-ejecuta, sus suscripciones previas se limpian automáticamente. Adiós memory leaks.

4. Checks de Igualdad — Los signals omiten notificaciones si el nuevo valor es idéntico al anterior (usando Object.is por defecto).

La Propuesta TC39 Signals

Lo más emocionante no está pasando dentro de ningún framework — está pasando a nivel del lenguaje. La propuesta TC39 Signals busca estandarizar la primitiva reactiva directamente en JavaScript.

¿Por Qué Estandarizar?

Cada framework tiene su propia implementación de signals. Los Angular Signals no pueden interoperar con createSignal de Solid. Una librería de date picker hecha con Preact Signals no funciona en una app Vue sin wrappers.

La propuesta TC39 resuelve esto con una API estándar Signal sobre la que todos los frameworks pueden construir:

// API de la Propuesta TC39 (Stage 1, sujeto a cambios)
const counter = new Signal.State(0);
const isEven = new Signal.Computed(() => (counter.get() & 1) === 0);

// Los frameworks envuelven esto con sus propias APIs ergonómicas
// pero el grafo reactivo subyacente es compartido

Lo Que Proporciona la Propuesta

La propuesta se enfoca en el algoritmo del grafo reactivo, no en la capa de renderizado:

Signal.State — Valor reactivo de lectura/escritura
Signal.Computed — Valor reactivo derivado (lazy, cacheado)
Signal.subtle.Watcher — API de bajo nivel para integración con frameworks

Dato clave: la propuesta no incluye effect(). Los efectos son específicos del framework — cómo actualizás el DOM, programás renders o agrupás cambios queda en manos de cada framework. El estándar solo provee las primitivas del grafo reactivo.

La Visión de Interoperabilidad

Imaginá un futuro donde:

Una librería de gráficos usa Signal.State internamente
La usás en una app Angular con Angular Signals (construido sobre Signal.State)
Tu colega usa la misma librería en una app Solid
La reactividad fluye sin problemas porque el grafo es compartido

Ese es el sueño — reactividad de estado compartida en todo el ecosistema JavaScript.

El Panorama de los Frameworks: Quién Usa Signals y Cómo

Angular Signals (v17+)

La adopción de signals por parte de Angular fue un evento sísmico. El framework famoso por RxJS, Zones y change detection reescribió su modelo de reactividad:

import { signal, computed, effect } from '@angular/core';

@Component({
  template: `
    <h1>{{ fullName() }}</h1>
    <button (click)="updateName()">Cambiar Nombre</button>
  `
})
export class UserComponent {
  firstName = signal('Alice');
  lastName = signal('Smith');

  fullName = computed(() => `${this.firstName()} ${this.lastName()}`);

  logger = effect(() => {
    console.log(`Nombre cambió a: ${this.fullName()}`);
  });

  updateName() {
    this.firstName.set('Bob');
  }
}

El cambio arquitectónico clave: Angular ahora puede saltarse Zone.js completamente para change detection. En vez de hacer dirty-checking del árbol completo de componentes en cada evento, Angular solo actualiza los nodos DOM específicos vinculados a signals que cambiaron. Resultado: renderizado 30-50% más rápido en benchmarks reales.

Solid.js — Signals Desde el Día Uno

Solid.js demostró que los signals podían impulsar un framework UI de nivel producción:

import { createSignal, createMemo, createEffect } from "solid-js";

function Counter() {
  const [count, setCount] = createSignal(0);
  const doubled = createMemo(() => count() * 2);

  createEffect(() => {
    console.log(`Count: ${count()}, Doubled: ${doubled()}`);
  });

  return (
    <button onClick={() => setCount(c => c + 1)}>
      {count()} × 2 = {doubled()}
    </button>
  );
}

La diferencia crítica con React: Solid no re-ejecuta componentes. La función Counter se ejecuta exactamente una vez. El JSX se compila a instrucciones de actualización DOM granulares. Cuando count cambia, solo los nodos de texto que muestran count() y doubled() se actualizan — la función del componente nunca se re-ejecuta.

Sin diffing de Virtual DOM, sin re-renders, sin necesidad de memoización. Nunca.

Svelte Runes (v5)

Svelte 5 reemplazó su reactividad anterior (la sintaxis de etiqueta $:) con Runes — esencialmente signals en tiempo de compilación:

<script>
  let count = $state(0);
  let doubled = $derived(count * 2);

  $effect(() => {
    console.log(`Count: ${count}, Doubled: ${doubled}`);
  });
</script>

<button onclick={() => count++}>
  {count} × 2 = {doubled}
</button>

La elegancia de Runes está en que parecen variables normales. El compilador transforma $state, $derived y $effect en primitivas signal internas, pero la experiencia del desarrollador se siente como escribir JavaScript plano.

La Reactividad de Vue (Composition API)

Vue ha usado un sistema de reactividad similar a signals desde la Composition API de Vue 3:

<script setup>
import { ref, computed, watchEffect } from 'vue';

const count = ref(0);
const doubled = computed(() => count.value * 2);

watchEffect(() => {
  console.log(`Count: ${count.value}, Doubled: ${doubled.value}`);
});
</script>

<template>
  <button @click="count++">
    {{ count }} × 2 = {{ doubled }}
  </button>
</template>

El ref de Vue es Signal. computed es Computed. watchEffect es Effect. Los nombres son diferentes, pero el grafo reactivo subyacente es arquitectónicamente idéntico. Vue estaba haciendo signals antes de que los signals fueran cool.

Preact Signals

Preact tomó un enfoque único al agregar signals como una librería compañera que se integra con el Virtual DOM:

import { signal, computed } from "@preact/signals";

const count = signal(0);
const doubled = computed(() => count.value * 2);

function Counter() {
  return (
    <button onClick={() => count.value++}>
      {count} × 2 = {doubled}
    </button>
  );
}

Lo especial de Preact Signals: podés pasar un signal directamente al JSX ({count} en vez de {count.value}), y Preact lo suscribe a nivel DOM, salteando el diff del Virtual DOM para ese nodo. Rendimiento cercano a Solid manteniendo el modelo familiar de componentes tipo React.

El Camino Divergente de React: Hooks + Compilador vs. Signals

Acá es donde la cosa se pone controversial. Todos los frameworks principales convergen en signals, pero React — el más usado — eligió explícitamente no adoptarlos. Entender por qué revela diferencias filosóficas fundamentales.

El Argumento de React Contra los Signals

1. Flujo de Datos Top-Down — React está diseñado sobre la idea de que los componentes son funciones de sus props y estado. Signals rompen este modelo porque actualizan nodos DOM directamente, sin pasar por la función del componente.

2. Facilidad de Debugging — En el modelo de React, podés poner un breakpoint en cualquier componente y ver el render completo en cada cambio de estado. Con signals, las actualizaciones pasan de forma granular y no hay "ciclo de render" que inspeccionar.

3. La Apuesta por el Compilador — La posición de React es que el compilador puede entregar rendimiento nivel signals sin cambiar el modelo de programación.

Los Contra-Argumentos

1. Overhead Fundamental — Incluso con memoización perfecta, React sigue re-ejecutando funciones de componente y haciendo diff de árboles Virtual DOM. Signals se saltean ambos pasos.

React (con compilador):
  Cambio de estado → Re-ejecutar función → Diff vDOM → Patch DOM

Signals:
  Cambio de estado → Patch DOM directamente

2. Costo de Runtime — La memoización del React Compiler agrega overhead de runtime (lookups de cache, checks de igualdad). Los signals solo trabajan cuando los valores realmente cambian.

3. Las "Reglas de React" — El compilador requiere que el código siga las "Reglas de React" (componentes puros, sin mutaciones durante render). Las violaciones causan bugs de correctitud silenciosos. Los signals no tienen esas restricciones.

4. Fragmentación del Ecosistema — Si TC39 estandariza signals, todos los frameworks excepto React compartirán una primitiva reactiva común.

Comparación de Rendimiento

Miremos números aproximados basados en los resultados públicos del js-framework-benchmark para una tabla de 10,000 filas:

Operación	React 19 + Compiler	Solid.js (Signals)	Angular (Signals)	Svelte 5 (Runes)
Crear 10k filas	~420ms	~190ms	~230ms	~200ms
Actualizar cada 10 filas	~80ms	~18ms	~25ms	~20ms
Intercambiar dos filas	~45ms	~12ms	~15ms	~14ms
Seleccionar fila	~8ms	~2ms	~3ms	~2ms
Eliminar fila	~38ms	~6ms	~9ms	~7ms
Memoria (post-create)	~9 MB	~4 MB	~4.5 MB	~3.5 MB

Nota: Estos son valores aproximados basados en tendencias de benchmarks públicos. Los números reales varían según hardware, navegador y versión del framework. Consultá js-framework-benchmark para resultados actualizados.

El patrón es consistente: los frameworks basados en signals superan a la abordaje del compilador de React por aproximadamente 2-4x en la mayoría de las operaciones. La diferencia de memoria es aún más dramática porque los signals no mantienen un árbol Virtual DOM.

Migración Práctica: Agregando Signals a Tu Stack

Si Estás en Angular

Ya estás ahí. Angular 17+ signals están listos para producción. Empezá a migrar desde patrones pesados en RxJS:

// Antes: RxJS Observables
@Component({...})
export class UserComponent implements OnInit, OnDestroy {
  user$!: Observable<User>;
  private destroy$ = new Subject<void>();

  ngOnInit() {
    this.user$ = this.userService.getUser().pipe(
      takeUntil(this.destroy$)
    );
  }

  ngOnDestroy() {
    this.destroy$.next();
    this.destroy$.complete();
  }
}

// Después: Angular Signals + resource API
@Component({...})
export class UserComponent {
  userId = input.required<string>();
  user = resource({
    request: () => this.userId(),
    loader: ({ request: id }) => this.userService.getUser(id)
  });
}

Sin gestionar suscripciones. Sin patrones takeUntil. Sin cleanup en OnDestroy.

Si Estás en React (Usando Preact Signals)

Podés usar signals en React hoy con @preact/signals-react:

import { signal, computed } from "@preact/signals-react";

const count = signal(0);
const doubled = computed(() => count.value * 2);

function Counter() {
  return (
    <div>
      <p>{count.value} × 2 = {doubled.value}</p>
      <button onClick={() => count.value++}>Incrementar</button>
    </div>
  );
}

Advertencia: es una integración de terceros. Funciona enganchándose al ciclo de render de React, así que no obtenés los beneficios completos de performance de signals nativos. Pero sí obtenés la ergonomía del tracking automático de dependencias sin useMemo/useCallback manual.

Si Empezás de Cero

Si estás eligiendo un framework para un proyecto nuevo en 2026 y el rendimiento es crítico:

Solid.js — Máximo rendimiento, bundle más pequeño, experiencia signals más "pura"
Svelte 5 — Mejor experiencia de desarrollo (Runes parecen JS normal), excelente rendimiento
Angular — Mejor opción para equipos enterprise grandes (TypeScript nativo, tooling integral)
Vue — Gran balance entre rendimiento y madurez del ecosistema

Ejemplo Real: Validación Reactiva de Formularios

Construyamos algo práctico para ver los signals en acción.

Vanilla Signals (Estilo Propuesta TC39)

const email = new Signal.State("");
const password = new Signal.State("");

const emailError = new Signal.Computed(() => {
  const value = email.get();
  if (!value) return "El email es requerido";
  if (!value.includes("@")) return "Formato de email inválido";
  return null;
});

const passwordStrength = new Signal.Computed(() => {
  const value = password.get();
  if (value.length === 0) return "empty";
  if (value.length < 8) return "weak";
  if (/(?=.*[A-Z])(?=.*[0-9])(?=.*[!@#$%])/.test(value)) return "strong";
  return "medium";
});

const isFormValid = new Signal.Computed(() => {
  return emailError.get() === null
    && passwordStrength.get() !== "weak"
    && passwordStrength.get() !== "empty";
});

Implementación con Solid.js

import { createSignal, createMemo, Show } from "solid-js";

function SignupForm() {
  const [email, setEmail] = createSignal("");
  const [password, setPassword] = createSignal("");

  const emailError = createMemo(() => {
    const value = email();
    if (!value) return "El email es requerido";
    if (!value.includes("@")) return "Formato de email inválido";
    return null;
  });

  const passwordStrength = createMemo(() => {
    const value = password();
    if (value.length === 0) return "empty";
    if (value.length < 8) return "weak";
    if (/(?=.*[A-Z])(?=.*[0-9])(?=.*[!@#$%])/.test(value)) return "strong";
    return "medium";
  });

  const isValid = createMemo(() =>
    emailError() === null && !["weak", "empty"].includes(passwordStrength())
  );

  return (
    <form>
      <input
        type="email"
        value={email()}
        onInput={(e) => setEmail(e.target.value)}
        classList={{ error: !!emailError() }}
      />
      <Show when={emailError()}>
        <span class="error">{emailError()}</span>
      </Show>

      <input
        type="password"
        value={password()}
        onInput={(e) => setPassword(e.target.value)}
      />
      <div class={`strength-${passwordStrength()}`}>
        Fortaleza: {passwordStrength()}
      </div>

      <button disabled={!isValid()}>Registrarse</button>
    </form>
  );
}

Cuando el usuario escribe en el campo de email:

Solo cambia el signal email
Solo recalcula emailError (no passwordStrength)
Solo recalcula isValid
Solo se actualizan los nodos DOM vinculados a emailError() e isValid()

El input de password, el indicador de fortaleza y todos los demás nodos DOM quedan completamente intactos. En React, toda la función del componente se re-ejecutaría, todas las expresiones JSX se re-evaluarían, y React haría diff del subárbol Virtual DOM completo.

Hacia Dónde Va el Frontend

La convergencia hacia signals no es coincidencia. Varias fuerzas convergen:

1. El Techo de Rendimiento del Virtual DOM

El diffing del Virtual DOM fue una idea brillante en 2013 cuando los motores JavaScript eran lentos. En 2026, los motores son extraordinariamente rápidos, y el overhead de "crear árbol virtual → diff → patch" se convirtió en el cuello de botella.

Los signals eliminan al intermediario. Cambio de estado → Actualización DOM. Sin paso de diffing.

2. El Auge de la Arquitectura Islands

Astro, Qwik, e incluso Next.js (con RSC) se mueven hacia "islas" de interactividad en un mar de HTML estático. Los signals encajan naturalmente porque cada isla puede tener su propio grafo reactivo local sin afectar al resto de la página.

3. El Endgame de TC39

Si Signal.State y Signal.Computed se vuelven parte del estándar JavaScript:

Todos los frameworks construidos sobre ellos interoperan automáticamente
Los motores del navegador pueden optimizar el grafo reactivo a nivel nativo
Librerías de terceros usan una primitiva reactiva universal

El fin de la era del "vendor lock-in" para manejo de estado.

Conclusión

El mundo frontend está atravesando una revolución de reactividad. Los signals han demostrado ser un modelo fundamentalmente más eficiente para el manejo de estado UI que el enfoque de diffing del Virtual DOM que React popularizó.

Todos los frameworks principales excepto React adoptaron signals. La propuesta TC39 trabaja en estandarizarlos en JavaScript mismo. Angular vio mejoras de renderizado del 30-50% post-migración. Solid.js entrega 2-4x el rendimiento de React en benchmarks estandarizados.

La apuesta de React por el compilador es audaz y puede reducir la brecha, pero no puede eliminar el overhead arquitectónico fundamental de re-ejecutar componentes y hacer diff de árboles virtuales. Ya sea que React eventualmente adopte signals, o logre demostrar que el enfoque del compilador es superior, la competencia está haciendo que todo el ecosistema sea mejor.

Sin importar qué framework uses hoy, entender signals ya no es opcional — es conocimiento fundamental. El grafo reactivo es el futuro del manejo de estado frontend, y ya está acá.

🚀 Explore More: This article is from the Pockit Blog.

If you found this helpful, check out Pockit.tools. It’s a curated collection of offline-capable dev utilities. Available on Chrome Web Store for free.

Fine-Tuning Open-Source LLMs with QLoRA and Unsloth: The Complete 2026 Guide

HK Lee — Thu, 26 Mar 2026 13:56:00 +0000

You've built a prototype with GPT-4 or Claude. It works great. Then the invoice arrives: $12,000 for last month's API calls. And it's growing 40% month-over-month.

This is the moment every AI engineer hits. The prototype-to-production cliff where managed API costs become unsustainable, latency requirements tighten, and you realize you need a model that actually understands your domain — not the entire internet's worth of knowledge.

Fine-tuning an open-source LLM is the answer. And thanks to QLoRA (Quantized Low-Rank Adaptation) and tools like Unsloth, you no longer need a cluster of A100 GPUs or a PhD in machine learning to do it. A single consumer GPU with 24GB of VRAM — an RTX 4090 or even a free Google Colab T4 — is enough to fine-tune a model with billions of active parameters to outperform GPT-4 on your specific task.

This guide covers everything from zero to production: why fine-tuning works, how QLoRA makes it feasible on consumer hardware, how to prepare your dataset, the exact training code, evaluation strategies, and deployment. Every code example is production-tested.

Why Fine-Tune Instead of Prompting?

Before diving into the how, let's be precise about when fine-tuning is the right choice. It's not always the answer.

Use prompting / RAG when:

Your task is general-purpose (summarization, translation, Q&A over documents)
Your data changes frequently (knowledge bases, support tickets)
You're still exploring what the model should do
You need to ship in days, not weeks

Use fine-tuning when:

The model needs to learn a specific style, format, or behavior that prompting can't reliably produce
You have a well-defined task with consistent input/output patterns
Latency and cost at scale matter (a fine-tuned 7B model is 10–50x cheaper per token than GPT-4)
You need the model to deeply understand domain-specific terminology
You want to reduce hallucinations on domain-specific facts

The most common fine-tuning use cases in production:

Use Case	Why Prompting Falls Short	Fine-Tuning Advantage
Code generation for internal APIs	Model doesn't know your SDK	Learns your specific patterns and conventions
Medical/Legal document analysis	Generic models hedge too much	Confident, domain-specific outputs
Structured data extraction	Prompt-based formatting is brittle	Consistent schema adherence
Customer support tone matching	System prompts drift over long conversations	Baked-in voice and personality
SQL generation for custom schemas	Schema in context eats tokens	Internalized schema knowledge

The key insight: fine-tuning doesn't teach the model new knowledge per se. It teaches the model new behaviors. A fine-tuned model doesn't memorize your database — it learns how to reason about your domain's patterns, produce outputs in your specific format, and apply your organization's conventions consistently.

Understanding LoRA and QLoRA

The Problem: Full Fine-Tuning Is Expensive

Traditional full fine-tuning updates every parameter in the model. For a 7B parameter model, this means:

Memory: ~28GB just for the model weights in FP32, plus ~28GB for optimizer states, plus ~28GB for gradients. Total: ~84GB of VRAM minimum.
Hardware: Multiple A100 80GB GPUs.
Cost: $10–50/hour on cloud GPU instances, training runs lasting hours to days.
Risk: Catastrophic forgetting — the model loses its general capabilities while learning your specific task.

LoRA: The Breakthrough

LoRA (Low-Rank Adaptation), introduced by Hu et al. in 2021, had a key insight: you don't need to update all parameters. When fine-tuning a pre-trained model, the weight changes tend to have a low intrinsic rank. This means the update matrix can be decomposed into two much smaller matrices.

Instead of updating a weight matrix W of dimensions d × k, LoRA freezes W and trains two small matrices A (d × r) and B (r × k), where r (the rank) is much smaller than both d and k:

Original:  W (4096 × 4096) → 16.7M parameters to update
LoRA:      A (4096 × 16) + B (16 × 4096) → 131K parameters to update

Reduction: 99.2% fewer trainable parameters

The forward pass becomes: output = W·x + α·B·A·x, where α is a scaling factor. During inference, you can merge B·A back into W, so there's zero additional latency compared to the original model.

# Conceptual illustration of LoRA
import torch
import torch.nn as nn

class LoRALayer(nn.Module):
    def __init__(self, original_layer: nn.Linear, rank: int = 16, alpha: float = 32):
        super().__init__()
        self.original = original_layer
        self.original.weight.requires_grad = False  # Freeze original weights

        d_in = original_layer.in_features
        d_out = original_layer.out_features

        # Low-rank decomposition matrices
        self.lora_A = nn.Parameter(torch.randn(d_in, rank) * 0.01)
        self.lora_B = nn.Parameter(torch.zeros(rank, d_out))

        self.scale = alpha / rank

    def forward(self, x):
        # Original computation (frozen) + low-rank update
        original_output = self.original(x)
        lora_output = (x @ self.lora_A @ self.lora_B) * self.scale
        return original_output + lora_output

    def merge(self):
        """Merge LoRA weights into original for zero-cost inference."""
        self.original.weight.data += (self.lora_A @ self.lora_B).T * self.scale

QLoRA: Making It Accessible

QLoRA (Quantized LoRA), introduced by Dettmers et al. in 2023, added three innovations that made fine-tuning accessible to consumer hardware:

4-bit NormalFloat (NF4) Quantization: The base model is quantized to 4 bits using a distribution-aware quantization scheme. This reduces a 7B model from ~14GB (FP16) to ~3.5GB.
Double Quantization: The quantization constants themselves are quantized, saving an additional 0.37 bits per parameter (~325MB on a 7B model).
Paged Optimizers: Optimizer states are offloaded to CPU RAM when GPU memory runs low, using NVIDIA unified memory. This prevents OOM crashes during training spikes.

The result: fine-tune a 7B model on a single 24GB GPU, or a 13B model on a 48GB GPU. Here's the memory breakdown:

Full Fine-Tuning (7B model):
  Model weights (FP32):    ~28 GB
  Optimizer states:        ~28 GB
  Gradients:               ~28 GB
  Total:                   ~84 GB → Needs 2x A100 80GB

QLoRA Fine-Tuning (7B model):
  Model weights (NF4):     ~3.5 GB
  LoRA adapters (FP16):    ~0.1 GB
  Optimizer states:        ~0.4 GB
  Gradients + activations: ~4.0 GB
  Total:                   ~8.0 GB → Fits on RTX 4090 (24GB) with room to spare

The quality difference between full fine-tuning and QLoRA? In most benchmarks, it's within 1–2% — a negligible tradeoff for a 10x reduction in hardware requirements.

Setting Up Your Environment

Hardware Requirements

GPU	VRAM	Maximum Model Size	Training Speed
T4 (Colab Free)	16GB	7B (tight)	~1.5 hours/epoch on 10K samples
RTX 3090/4090	24GB	7B (comfortable), 13B (tight)	~45 min/epoch on 10K samples
A100 40GB	40GB	13B (comfortable), 34B (tight)	~20 min/epoch on 10K samples
A100 80GB	80GB	70B with aggressive quantization	~15 min/epoch on 10K samples

Installation

Using Unsloth (recommended for 2–5x speedup over standard HuggingFace training):

# Create a fresh environment
conda create -n finetune python=3.11 -y
conda activate finetune

# Install PyTorch with CUDA support
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Install Unsloth (handles bitsandbytes, transformers, peft, trl automatically)
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps xformers trl peft accelerate bitsandbytes

# For evaluation
pip install rouge-score nltk scikit-learn

For a standard HuggingFace setup (without Unsloth):

pip install transformers peft trl bitsandbytes accelerate datasets
pip install flash-attn --no-build-isolation  # Optional but recommended

Choosing a Base Model (March 2026)

The choice of base model matters more than most people realize. Here's the current landscape:

Model	Parameters	Context Length	Best For	License
Llama 4 Scout	109B total / 17B active (MoE)	10M	Latest Meta flagship, massive context, needs H100	Llama 4 Community
Llama 3.1 8B	8B	128K	Best quality-to-size ratio for beginners	Llama 3.1 Community
Mistral Small 4	32B	128K	Multilingual, strong reasoning, Apache licensed	Apache 2.0
Gemma 3 27B	27B	128K	Multimodal, strong coding, Google ecosystem	Gemma License
Qwen 2.5 7B	7B	128K	Best for Chinese + English bilingual	Apache 2.0
Phi-4 14B	14B	16K	Compact, strong reasoning	MIT

Recommendation for first-timers: Start with Llama 3.1 8B. It has the largest community, the most tutorials, and the best quality-to-resources ratio. Once your pipeline works, swap in larger models like Llama 4 Scout or Gemma 3 27B. Note that Llama 4 Scout uses a Mixture-of-Experts (MoE) architecture — only 17B parameters are active per token, making it surprisingly efficient despite its 109B total parameter count.

Preparing Your Dataset

This is where most fine-tuning projects succeed or fail. The model is only as good as the data you feed it. A 500-example dataset of exceptional quality will outperform a 50,000-example dataset of mediocre quality.

Data Format: The Chat Template

Modern fine-tuning uses the conversational format (also called the "chat template" or "instruction format"). Here's the standard structure:

{
  "conversations": [
    {"role": "system", "content": "You are a senior backend engineer specializing in PostgreSQL optimization."},
    {"role": "user", "content": "This query is taking 30 seconds on a table with 50M rows:\n\nSELECT * FROM orders WHERE customer_id = 12345 AND status = 'pending' ORDER BY created_at DESC LIMIT 10;"},
    {"role": "assistant", "content": "The issue is likely a missing composite index. Here's the analysis:\n\n**Problem:** Without an index on `(customer_id, status, created_at)`, PostgreSQL performs a sequential scan on 50M rows, then sorts the results.\n\n**Solution:**\n```

sql\nCREATE INDEX CONCURRENTLY idx_orders_customer_status_created\nON orders (customer_id, status, created_at DESC);\n

```\n\n**Why this works:**\n1. The index covers all three columns in the WHERE and ORDER BY clauses\n2. `created_at DESC` in the index matches your ORDER BY, eliminating a sort operation\n3. PostgreSQL can use an Index Only Scan if you replace `SELECT *` with specific columns\n4. `CONCURRENTLY` prevents locking the table during creation\n\n**Expected improvement:** From ~30s to <10ms for this query pattern."}
  ]
}

Data Quality Checklist

Before training, audit your dataset against these criteria:

def audit_dataset(dataset):
    issues = []

    for i, example in enumerate(dataset):
        conversations = example['conversations']

        # Check 1: Minimum conversation length
        if len(conversations) < 2:
            issues.append(f"Example {i}: Less than 2 turns")

        # Check 2: Response quality (length proxy)
        assistant_msgs = [c for c in conversations if c['role'] == 'assistant']
        for msg in assistant_msgs:
            if len(msg['content']) < 50:
                issues.append(f"Example {i}: Very short assistant response ({len(msg['content'])} chars)")
            if len(msg['content']) > 8000:
                issues.append(f"Example {i}: Very long assistant response ({len(msg['content'])} chars)")

        # Check 3: No empty messages
        for c in conversations:
            if not c['content'].strip():
                issues.append(f"Example {i}: Empty message from {c['role']}")

        # Check 4: Proper role alternation
        roles = [c['role'] for c in conversations if c['role'] != 'system']
        for j in range(1, len(roles)):
            if roles[j] == roles[j-1]:
                issues.append(f"Example {i}: Consecutive {roles[j]} messages")

        # Check 5: No data leakage (model shouldn't reference being fine-tuned)
        for c in conversations:
            if any(phrase in c['content'].lower() for phrase in ['as an ai', 'i am an ai', 'language model']):
                issues.append(f"Example {i}: Potential identity leakage in {c['role']} message")

    return issues

How Many Examples Do You Need?

The answer depends on your task:

Task Type	Minimum	Sweet Spot	Diminishing Returns
Style/tone adaptation	50–100	200–500	>1,000
Domain-specific Q&A	200–500	1,000–3,000	>10,000
Code generation (specific SDK)	500–1,000	2,000–5,000	>15,000
Complex reasoning chains	1,000–2,000	5,000–10,000	>20,000

The 80/10/10 rule for data splitting:

80% for training
10% for validation (monitored during training to prevent overfitting)
10% for final evaluation (never seen during training)

from datasets import load_dataset, DatasetDict

def prepare_splits(dataset_path):
    dataset = load_dataset("json", data_files=dataset_path, split="train")
    dataset = dataset.shuffle(seed=42)

    # 80/10/10 split
    train_test = dataset.train_test_split(test_size=0.2, seed=42)
    val_test = train_test['test'].train_test_split(test_size=0.5, seed=42)

    return DatasetDict({
        'train': train_test['train'],
        'validation': val_test['train'],
        'test': val_test['test'],
    })

Generating Synthetic Training Data

If you don't have enough examples, you can bootstrap your dataset using a strong model (GPT-4, Claude) to generate training data for a smaller model. This technique, called knowledge distillation via synthetic data, is used extensively in production.

import openai
import json

SYSTEM_PROMPT = """You are generating training data for a fine-tuned model
that will act as a PostgreSQL optimization expert.

Generate realistic user questions about PostgreSQL performance issues and
provide expert-level responses. Include:
- Specific SQL queries with realistic table names and sizes
- EXPLAIN ANALYZE output interpretation
- Concrete index recommendations with CREATE INDEX statements
- Performance improvement estimates

Each response should be 200-500 words with code examples."""

async def generate_training_examples(n_examples: int = 500):
    client = openai.AsyncOpenAI()
    examples = []

    topics = [
        "slow JOIN queries on large tables",
        "N+1 query problems in ORMs",
        "full table scans on indexed columns",
        "lock contention in high-write scenarios",
        "query plan regression after VACUUM",
        "connection pool exhaustion",
        "index bloat detection and remediation",
        # ... more topics
    ]

    for i in range(n_examples):
        topic = topics[i % len(topics)]

        response = await client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": f"Generate a training example about: {topic}. "
                                             f"Vary the complexity and table schemas."}
            ],
            temperature=0.9,  # Higher temperature for diversity
            response_format={"type": "json_object"},
        )

        example = json.loads(response.choices[0].message.content)
        examples.append(example)

    return examples

Critical warning: Always manually review a sample of your synthetic data. LLMs can generate plausible-sounding but incorrect technical advice. Budget time for human review of at least 10–20% of synthetic examples.

Training with Unsloth

Now for the main event. Here's the complete training script:

from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments
from datasets import load_dataset

# ─────────────────────────────────────────
# 1. Load Model with 4-bit Quantization
# ─────────────────────────────────────────
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-Instruct",
    max_seq_length=4096,      # Maximum sequence length for training
    dtype=None,                # Auto-detect: float16 for older GPUs, bfloat16 for Ampere+
    load_in_4bit=True,         # QLoRA: load base model in 4-bit NF4
    # token="hf_...",          # Uncomment if using gated models
)

# ─────────────────────────────────────────
# 2. Configure LoRA Adapters
# ─────────────────────────────────────────
model = FastLanguageModel.get_peft_model(
    model,
    r=32,                       # LoRA rank — higher = more capacity, more VRAM
    lora_alpha=64,              # Scaling factor — typically 2x rank
    target_modules=[            # Which layers to adapt
        "q_proj", "k_proj", "v_proj", "o_proj",  # Attention layers
        "gate_proj", "up_proj", "down_proj",       # MLP layers
    ],
    lora_dropout=0.05,          # Slight dropout for regularization
    bias="none",                # Don't train bias terms
    use_gradient_checkpointing="unsloth",  # 60% less VRAM for long contexts
    random_state=42,
)

# Verify trainable parameters
model.print_trainable_parameters()
# Output: trainable params: 83,886,080 || all params: 8,113,831,936 || trainable%: 1.034%

# ─────────────────────────────────────────
# 3. Load and Format Dataset
# ─────────────────────────────────────────
dataset = load_dataset("json", data_files="training_data.jsonl", split="train")

def format_chat(example):
    """Format conversations into the Llama 3 chat template."""
    messages = example['conversations']
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False,
    )
    return {"text": text}

dataset = dataset.map(format_chat, remove_columns=dataset.column_names)

# ─────────────────────────────────────────
# 4. Configure Training
# ─────────────────────────────────────────
training_args = TrainingArguments(
    output_dir="./outputs",
    per_device_train_batch_size=4,      # Adjust based on VRAM
    gradient_accumulation_steps=4,       # Effective batch size = 4 * 4 = 16
    num_train_epochs=3,                  # 2-4 epochs is typical
    learning_rate=2e-4,                  # Standard for QLoRA
    lr_scheduler_type="cosine",          # Cosine decay with warmup
    warmup_ratio=0.05,                   # 5% of steps for warmup
    weight_decay=0.01,                   # Mild regularization
    logging_steps=10,
    save_strategy="steps",
    save_steps=100,
    eval_strategy="steps",
    eval_steps=100,
    fp16=not torch.cuda.is_bf16_supported(),
    bf16=torch.cuda.is_bf16_supported(),
    optim="adamw_8bit",                  # 8-bit Adam saves VRAM
    seed=42,
    max_grad_norm=0.3,                   # Gradient clipping for stability
    report_to="wandb",                   # Optional: Weights & Biases logging
)

# ─────────────────────────────────────────
# 5. Initialize Trainer and Start
# ─────────────────────────────────────────
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    args=training_args,
    max_seq_length=4096,
    dataset_text_field="text",
    packing=True,              # Pack short examples together for efficiency
)

# Start training
print("Starting fine-tuning...")
stats = trainer.train()
print(f"Training completed in {stats.metrics['train_runtime']:.0f} seconds")
print(f"Final loss: {stats.metrics['train_loss']:.4f}")

Hyperparameter Tuning Guide

The defaults above work well for most cases, but here's how to tune them:

LoRA Rank (r):

r=8: Minimal adaptation. Good for simple style changes.
r=16: Default. Works for most tasks.
r=32: Higher capacity. Good for complex domain adaptation.
r=64+: Approaching full fine-tuning capacity. Rarely needed.

Rule of thumb: start with r=16. If validation loss plateaus early, increase to 32. If it overfits quickly, decrease to 8.

Learning Rate:

2e-4: Standard QLoRA learning rate. Start here.
1e-4: More conservative. Use if training is unstable.
5e-5: Very conservative. Use for very small datasets (<200 examples).

Number of Epochs:

1–2: Large datasets (>10K examples)
2–4: Medium datasets (1K–10K examples)
4–8: Small datasets (<1K examples)
Watch for overfitting: If validation loss starts increasing while training loss keeps decreasing, you're overfitting. Stop training.

# Monitor overfitting during training
from transformers import EarlyStoppingCallback

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,        # Critical: provide validation set
    args=training_args,
    callbacks=[
        EarlyStoppingCallback(
            early_stopping_patience=3,    # Stop after 3 eval steps without improvement
            early_stopping_threshold=0.01 # Minimum improvement threshold
        ),
    ],
    max_seq_length=4096,
    dataset_text_field="text",
    packing=True,
)

What the Training Loss Curve Should Look Like

A healthy training run looks like this:

Loss
 4.0 |X
     |  X
 3.0 |    X
     |      X
 2.0 |        X  X
     |            X  X  X
 1.0 |                    X  X  X  X  X  X  ← plateau (good, model converged)
     |
 0.0 +─────────────────────────────────────
     0    200   400   600   800   1000
                    Steps

Red flags:
- Loss doesn't decrease → Learning rate too low, or data has issues
- Loss drops to near 0 → Overfitting badly, reduce epochs or increase data
- Loss is very noisy → Batch size too small or learning rate too high
- Loss spikes suddenly → Gradient explosion, reduce learning rate

Evaluating Your Model

Training is only half the battle. Evaluation tells you whether your fine-tuned model actually improves on the base model for your specific task.

Automated Evaluation

import torch
from rouge_score import rouge_scorer
from sklearn.metrics import accuracy_score
import json

def evaluate_model(model, tokenizer, test_dataset, max_samples=100):
    """Comprehensive evaluation of fine-tuned model."""
    model.eval()
    scorer = rouge_scorer.RougeScorer(['rouge1', 'rougeL'], use_stemmer=True)

    results = {
        'rouge1_scores': [],
        'rougeL_scores': [],
        'format_compliance': [],
        'avg_response_length': [],
        'examples': [],
    }

    for i, example in enumerate(test_dataset.select(range(min(max_samples, len(test_dataset))))):
        conversations = example['conversations']

        # Build prompt from all messages except the last assistant response
        prompt_messages = []
        expected_response = ""
        for msg in conversations:
            if msg['role'] == 'assistant' and msg == conversations[-1]:
                expected_response = msg['content']
            else:
                prompt_messages.append(msg)

        # Generate response
        inputs = tokenizer.apply_chat_template(
            prompt_messages,
            tokenize=True,
            add_generation_prompt=True,
            return_tensors="pt"
        ).to(model.device)

        with torch.no_grad():
            outputs = model.generate(
                inputs,
                max_new_tokens=1024,
                temperature=0.1,       # Low temperature for evaluation
                do_sample=False,        # Greedy decoding for reproducibility
            )

        generated = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)

        # Score
        rouge_scores = scorer.score(expected_response, generated)
        results['rouge1_scores'].append(rouge_scores['rouge1'].fmeasure)
        results['rougeL_scores'].append(rouge_scores['rougeL'].fmeasure)
        results['avg_response_length'].append(len(generated))

        # Check format compliance (e.g., does it include code blocks when expected?)
        expected_has_code = '```

' in expected_response
        generated_has_code = '

```' in generated
        results['format_compliance'].append(expected_has_code == generated_has_code)

        # Store examples for manual review
        if i < 10:
            results['examples'].append({
                'prompt': prompt_messages[-1]['content'][:200],
                'expected': expected_response[:300],
                'generated': generated[:300],
                'rouge1': rouge_scores['rouge1'].fmeasure,
            })

    # Aggregate results
    summary = {
        'avg_rouge1': sum(results['rouge1_scores']) / len(results['rouge1_scores']),
        'avg_rougeL': sum(results['rougeL_scores']) / len(results['rougeL_scores']),
        'format_compliance_rate': sum(results['format_compliance']) / len(results['format_compliance']),
        'avg_response_length': sum(results['avg_response_length']) / len(results['avg_response_length']),
        'examples': results['examples'],
    }

    return summary

A/B Comparison: Base vs. Fine-Tuned

The most informative evaluation compares your fine-tuned model against the base model on the same prompts:

async def ab_comparison(base_model, finetuned_model, tokenizer, test_prompts):
    """Side-by-side comparison of base vs fine-tuned model responses."""
    results = []

    for prompt in test_prompts:
        messages = [
            {"role": "system", "content": "You are a PostgreSQL optimization expert."},
            {"role": "user", "content": prompt},
        ]

        # Generate from both models
        inputs = tokenizer.apply_chat_template(messages, tokenize=True,
                                                add_generation_prompt=True,
                                                return_tensors="pt")

        base_output = base_model.generate(inputs.to(base_model.device),
                                           max_new_tokens=512, temperature=0.1)
        ft_output = finetuned_model.generate(inputs.to(finetuned_model.device),
                                              max_new_tokens=512, temperature=0.1)

        base_text = tokenizer.decode(base_output[0][inputs.shape[1]:], skip_special_tokens=True)
        ft_text = tokenizer.decode(ft_output[0][inputs.shape[1]:], skip_special_tokens=True)

        results.append({
            'prompt': prompt,
            'base_response': base_text,
            'finetuned_response': ft_text,
        })

    return results

LLM-as-Judge Evaluation

For subjective quality assessment, use a stronger model to judge:

async def llm_judge_evaluation(examples, judge_model="gpt-4o"):
    """Use a strong LLM to evaluate response quality."""
    client = openai.AsyncOpenAI()
    scores = []

    for ex in examples:
        response = await client.chat.completions.create(
            model=judge_model,
            messages=[{
                "role": "system",
                "content": """You are evaluating the quality of a fine-tuned model's response.
                Rate each response on these dimensions (1-5 scale):
                1. Technical Accuracy: Are the technical claims correct?
                2. Completeness: Does it address all aspects of the question?
                3. Format Compliance: Does it follow the expected output format?
                4. Actionability: Can the user directly apply this advice?

                Respond as JSON: {"accuracy": N, "completeness": N, "format": N, "actionability": N, "reasoning": "..."}"""
            }, {
                "role": "user",
                "content": f"Question: {ex['prompt']}\n\nResponse: {ex['finetuned_response']}"
            }],
            response_format={"type": "json_object"},
        )

        score = json.loads(response.choices[0].message.content)
        scores.append(score)

    return scores

Saving and Deploying Your Model

Saving Options

After training, you have three saving options:

# Option 1: Save LoRA adapters only (~100-300MB)
# Best for: Version control, quick swapping between adapters
model.save_pretrained("./my-lora-adapter")
tokenizer.save_pretrained("./my-lora-adapter")

# Option 2: Merge and save full model in 16-bit (~14GB for 7B)
# Best for: Standard deployment with vLLM or TGI
model.save_pretrained_merged("./my-model-merged", tokenizer, save_method="merged_16bit")

# Option 3: Export as GGUF for llama.cpp / Ollama deployment
# Best for: CPU inference, edge deployment, local development
model.save_pretrained_gguf("./my-model-gguf", tokenizer, quantization_method="q4_k_m")

# Option 4: Push to Hugging Face Hub
model.push_to_hub("your-username/my-fine-tuned-model", token="hf_...")
tokenizer.push_to_hub("your-username/my-fine-tuned-model", token="hf_...")

Deployment with vLLM (Production Recommended)

vLLM is the standard for production LLM serving. It supports continuous batching, PagedAttention, and speculative decoding for maximum throughput:

# Install vLLM
# pip install vllm

# Serve the merged model
# vllm serve ./my-model-merged --port 8000 --max-model-len 4096

# Or serve the base model with LoRA adapter (hot-swappable!)
# vllm serve unsloth/Meta-Llama-3.1-8B-Instruct \
#   --enable-lora \
#   --lora-modules my-adapter=./my-lora-adapter \
#   --port 8000

# Client code — uses OpenAI-compatible API
import openai

client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

response = client.chat.completions.create(
    model="./my-model-merged",    # Or "my-adapter" for LoRA
    messages=[
        {"role": "system", "content": "You are a PostgreSQL optimization expert."},
        {"role": "user", "content": "My query is doing a sequential scan on 100M rows..."},
    ],
    temperature=0.3,
    max_tokens=1024,
)

print(response.choices[0].message.content)

Deployment with Ollama (Local/Edge)

For local development or edge deployment, export to GGUF and use Ollama:

# After exporting to GGUF
# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./my-model-gguf/unsloth.Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""

PARAMETER temperature 0.3
PARAMETER num_ctx 4096
SYSTEM "You are a PostgreSQL optimization expert."
EOF

# Create and run the Ollama model
ollama create my-pg-expert -f Modelfile
ollama run my-pg-expert "How do I optimize a slow GROUP BY query?"

Production Checklist

Before shipping your fine-tuned model to production, run through this checklist:

Pre-Deployment

[ ] Evaluation scores exceed baseline: Fine-tuned model outperforms base model on your test set by a meaningful margin
[ ] No catastrophic forgetting: Test on general tasks to ensure the model hasn't lost basic capabilities
[ ] Guardrails in place: Test for harmful, biased, or out-of-scope outputs and implement content filtering
[ ] Latency benchmarks: Measure P50, P95, and P99 latency under realistic load
[ ] Cost projection: Calculate per-request cost including GPU compute, and compare to API-based alternatives

Infrastructure

# Health check endpoint for production deployment
from fastapi import FastAPI
import time

app = FastAPI()

@app.get("/health")
async def health():
    start = time.time()
    # Quick inference test
    response = model.generate(
        tokenizer("test", return_tensors="pt")["input_ids"].to(model.device),
        max_new_tokens=10,
    )
    latency_ms = (time.time() - start) * 1000

    return {
        "status": "healthy",
        "model": "my-finetuned-model-v1",
        "inference_latency_ms": round(latency_ms, 2),
        "gpu_memory_used_gb": round(torch.cuda.memory_allocated() / 1e9, 2),
        "gpu_memory_total_gb": round(torch.cuda.get_device_properties(0).total_mem / 1e9, 2),
    }

Post-Deployment Monitoring

[ ] Track output quality over time: Set up automated evaluation on a random sample of production requests
[ ] Monitor for distribution drift: If user queries start differing from training data, model quality will degrade
[ ] Version your models: Use semantic versioning (v1.0.0, v1.1.0) and maintain rollback capability
[ ] Retrain cadence: Plan for periodic retraining as you accumulate more production data

Common Pitfalls and How to Avoid Them

Pitfall 1: Training on Bad Data

Symptom: Model generates plausible-sounding but factually incorrect responses.
Cause: Synthetic training data was generated without human verification.
Fix: Always have domain experts review at least 10% of your training data. One incorrect example can poison hundreds of related outputs.

Pitfall 2: Overfitting on Small Datasets

Symptom: Training loss approaches 0, but model outputs are formulaic and seem to memorize training examples verbatim.
Cause: Too many epochs on too few examples.
Fix: Reduce epochs, increase LoRA dropout to 0.1, use a lower LoRA rank, or augment your dataset.

Pitfall 3: Chat Template Mismatch

Symptom: Model outputs garbage, repeated tokens, or ignores the system prompt.
Cause: Using a different chat template during training vs. inference.
Fix: Always use tokenizer.apply_chat_template() for both training data formatting and inference prompt construction. Never manually construct chat prompts.

Pitfall 4: Catastrophic Forgetting

Symptom: Model performs well on your specific task but can't handle basic tasks it could do before fine-tuning.
Cause: Aggressive fine-tuning that overwrites general knowledge.
Fix: Use a lower learning rate (5e-5 instead of 2e-4), fewer epochs, or mix in general-purpose data (10–20% of your training set should be general-purpose examples).

Pitfall 5: Ignoring Quantization Effects

Symptom: Fine-tuned model performs well in FP16 but degrades significantly after GGUF quantization for deployment.
Cause: Aggressive quantization (Q2_K, Q3_K) on models that weren't designed for it.
Fix: Use Q4_K_M or Q5_K_M for deployment quantization. Always benchmark after quantization to verify quality is maintained. If quality drops significantly, use Q6_K or Q8_0 at the cost of higher memory usage.

Cost Comparison: Fine-Tuning vs. API

Let's do the math on a realistic production scenario:

Scenario: 100,000 requests/month, average 500 input tokens + 500 output tokens per request.

Approach	Monthly Cost	Latency (P50)	Control
GPT-4o API	~$1,500	800ms	Low (OpenAI controls the model)
Claude 3.5 Sonnet API	~$1,800	600ms	Low
GPT-4o-mini API	~$30	400ms	Low
Self-hosted Llama 3.1 8B (A10G)	~$350	120ms	Full
Self-hosted Fine-Tuned 8B (A10G)	~$350	120ms	Full + Domain expertise

The fine-tuned model costs the same to run as the base model — but produces higher-quality domain-specific outputs. On the A10G instance, you're paying ~$0.75/hour for dedicated GPU compute versus per-token API pricing that scales linearly with usage.

Break-even point: For most teams, self-hosting a fine-tuned model becomes cheaper than API calls at around 30,000–50,000 requests/month, depending on the API model and response length.

What's Next: Emerging Techniques in 2026

Fine-tuning is evolving fast. Here's what's on the horizon:

Unsloth Studio (March 2026): Unsloth just released an open-source, local, no-code interface that handles the entire fine-tuning lifecycle — data preparation, training, and deployment — in a single GUI. It claims 70% less VRAM and 2x faster training. If you're not comfortable with the Python scripts above, Studio is a game-changer for accessibility.

DoRA (Weight-Decomposed Low-Rank Adaptation): A 2024 innovation that separates magnitude and direction in weight updates, consistently outperforming LoRA by 1–3% with negligible additional overhead. Already integrated into the PEFT library.

GaLore (Gradient Low-Rank Projection): Promises full fine-tuning quality at LoRA-level memory costs by projecting gradients into a low-rank space. Still experimental but promising for users who want to avoid the LoRA quality ceiling.

LoRA Merging and Model Soups: Combining multiple LoRA adapters (each trained on different tasks) into a single model through weight averaging. Enables multi-task specialization without separate model deployments.

GRPO (Group Relative Policy Optimization): An emerging technique for training "reasoning AI" models that can perform multi-step logic and chain-of-thought. Unsloth supports GRPO with as little as 5GB of VRAM, making it accessible on local hardware. This is how the next generation of reasoning models (like DeepSeek-R1) are being trained.

Reward Model Fine-Tuning (RLHF/DPO): After supervised fine-tuning (SFT), a second training pass using Direct Preference Optimization (DPO) aligns model outputs with human preferences. This is how production models are trained to be helpful, harmless, and honest — and the tooling for applying it to custom models is now accessible.

# DPO training after SFT (sketch)
from trl import DPOTrainer, DPOConfig

dpo_config = DPOConfig(
    output_dir="./dpo-output",
    beta=0.1,                    # KL-divergence penalty strength
    learning_rate=5e-6,          # Much lower than SFT
    per_device_train_batch_size=2,
    num_train_epochs=1,
)

# DPO dataset requires chosen/rejected pairs
# {"prompt": "...", "chosen": "good response", "rejected": "bad response"}
dpo_trainer = DPOTrainer(
    model=sft_model,
    ref_model=None,              # Uses implicit reference with PEFT
    args=dpo_config,
    train_dataset=preference_dataset,
    tokenizer=tokenizer,
)

dpo_trainer.train()

Final Summary

Fine-tuning open-source LLMs has gone from a research-lab activity to a standard engineering workflow. With QLoRA and Unsloth, you can fine-tune a model that outperforms GPT-4 on your specific task — on a single consumer GPU, in under an hour.

The key principles:

Data quality over quantity. 500 excellent examples beat 50,000 mediocre ones.
Start small. Use Llama 3.1 8B with QLoRA on your smallest viable dataset. Get the pipeline working before scaling.
Evaluate rigorously. Automated metrics (ROUGE, format compliance) plus LLM-as-judge plus manual review. All three.
Monitor in production. The model's performance will drift as user queries evolve. Plan for periodic retraining.
Know when NOT to fine-tune. If RAG or better prompting solves your problem, it's cheaper and faster than fine-tuning.

The gap between closed-source and open-source models shrinks every month. With the techniques in this guide, you can build production AI systems that are cheaper, faster, more private, and more specialized than anything an API can give you.

🛠️ Developer Toolkit: This post first appeared on the Pockit Blog.

Need a Regex Tester, JWT Decoder, or Image Converter? Use them on Pockit.tools or install the Extension to avoid switching tabs. No signup required.

OAuth 2.1 Is Here: What Changed, What's Deprecated, and How to Migrate Your App

HK Lee — Wed, 25 Mar 2026 13:28:00 +0000

If you shipped a single-page application before 2024, your OAuth implementation is probably insecure. Not "theoretically vulnerable" insecure — actually exploitable insecure.

The Implicit Grant flow that every React tutorial taught you to use? Removed in OAuth 2.1. The Resource Owner Password Credentials (ROPC) flow your mobile app relies on? Also removed. Bearer tokens in URL query strings? Banned.

OAuth 2.1 isn't a minor version bump. It's a decade of security lessons codified into spec, and it breaks real production code. Libraries you depend on are already shipping OAuth 2.1 defaults. Identity providers are deprecating legacy endpoints. If you haven't migrated yet, you're running on borrowed time.

This guide covers every breaking change in OAuth 2.1, explains why each decision was made (so you know it's not arbitrary bureaucracy), and provides production-ready TypeScript code to migrate your existing implementations.

What is OAuth 2.1, exactly?

OAuth 2.1 is not a new protocol. It's a consolidation of OAuth 2.0 (RFC 6749) plus every security best practice RFC published since 2012. Think of it as OAuth 2.0 with fourteen years of errata, security advisories, and "you should really be doing this" recommendations baked directly into the core spec.

The key RFCs it absorbs:

RFC	What It Covers	OAuth 2.1 Impact
RFC 7636	PKCE (Proof Key for Code Exchange)	Now mandatory for all clients
RFC 7009	Token Revocation	Integrated as core feature
RFC 8252	Native App Best Practices	Loopback redirects standardized
RFC 9207	Authorization Server Issuer Identification	Issuer verification required
RFC 9126	Pushed Authorization Requests (PAR)	Recommended for high-security flows
RFC 9449	DPoP (Demonstration of Proof-of-Possession)	Recommended over bearer tokens

The practical effect: one spec to read instead of six. But the migration cost is real, because OAuth 2.1 removes flows that millions of applications still use.

The four breaking changes

1. Implicit Grant is dead

What's removed: The entire response_type=token flow.

In OAuth 2.0, the Implicit Grant was designed for browser-based apps that couldn't safely store a client secret. The authorization server returned an access token directly in the URL fragment (#access_token=...). It was simple. It was also a security nightmare.

Why it's insecure:

// The Implicit Flow vulnerability chain

1. User clicks "Login with Google"
2. Redirect to: https://auth.example.com/authorize?
     response_type=token&
     client_id=my-spa&
     redirect_uri=https://app.example.com/callback

3. After authentication, redirect back:
   https://app.example.com/callback#access_token=eyJhbGciOi...

// 🚨 Problem 1: Token in URL fragment
// - Visible in browser history
// - Logged by any proxy/CDN between user and server
// - Accessible to any JavaScript on the page (XSS = game over)

// 🚨 Problem 2: No way to verify the token receiver
// - An attacker can intercept the redirect and steal the token
// - No PKCE, no code exchange, no verification step

// 🚨 Problem 3: No refresh tokens
// - Short-lived tokens mean constant re-authentication
// - Users get frustrated, developers extend token lifetimes
// - Long-lived tokens in URL fragments = worse security

The OAuth 2.1 replacement: Authorization Code flow with PKCE. Every SPA, every mobile app, every client that used Implicit must switch.

// ❌ Old: Implicit Grant (REMOVED in OAuth 2.1)
const authUrl = new URL('https://auth.example.com/authorize');
authUrl.searchParams.set('response_type', 'token'); // ← Banned
authUrl.searchParams.set('client_id', CLIENT_ID);
authUrl.searchParams.set('redirect_uri', REDIRECT_URI);
window.location.href = authUrl.toString();

// ✅ New: Authorization Code + PKCE
const codeVerifier = generateCodeVerifier();
const codeChallenge = await generateCodeChallenge(codeVerifier);

const authUrl = new URL('https://auth.example.com/authorize');
authUrl.searchParams.set('response_type', 'code'); // ← Code, not token
authUrl.searchParams.set('client_id', CLIENT_ID);
authUrl.searchParams.set('redirect_uri', REDIRECT_URI);
authUrl.searchParams.set('code_challenge', codeChallenge);
authUrl.searchParams.set('code_challenge_method', 'S256');
window.location.href = authUrl.toString();

2. PKCE is mandatory for ALL clients

What changed: PKCE was optional in OAuth 2.0, recommended only for public clients (SPAs, mobile apps). In OAuth 2.1, it's mandatory for every client type — including confidential server-side applications that already have a client secret.

Why even confidential clients need PKCE:

Even with a client secret, the authorization code can still be intercepted during the redirect. PKCE prevents authorization code injection attacks where an attacker substitutes their own authorization code into the victim's session. The client secret protects the token endpoint; PKCE protects the authorization flow.

How PKCE works:

import { createHash, randomBytes } from 'crypto';

// Step 1: Generate a cryptographically random code verifier
function generateCodeVerifier(): string {
  // 32 bytes = 43 characters in base64url
  return randomBytes(32)
    .toString('base64url');
}

// Step 2: Create the code challenge from the verifier
async function generateCodeChallenge(verifier: string): Promise<string> {
  // SHA-256 hash, then base64url encode
  const hash = createHash('sha256')
    .update(verifier)
    .digest();
  return Buffer.from(hash).toString('base64url');
}

// Step 3: Include in authorization request
const verifier = generateCodeVerifier();
const challenge = await generateCodeChallenge(verifier);

// Send challenge to authorization server
// Store verifier securely (session storage, not localStorage)
sessionStorage.setItem('pkce_verifier', verifier);

// Step 4: Include verifier in token exchange
async function exchangeCode(code: string): Promise<TokenResponse> {
  const verifier = sessionStorage.getItem('pkce_verifier');

  const response = await fetch('https://auth.example.com/token', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'authorization_code',
      code,
      redirect_uri: REDIRECT_URI,
      client_id: CLIENT_ID,
      code_verifier: verifier!, // ← Proves we made the original request
    }),
  });

  return response.json();
}

The verification flow:

Client                          Auth Server
  |                                  |
  |-- code_challenge=SHA256(v) ----> |  Authorization Request
  |                                  |  (stores challenge)
  |<-------- code=abc123 ----------- |  Authorization Response
  |                                  |
  |-- code=abc123 -----------------> |  Token Request
  |   code_verifier=v                |  (computes SHA256(v),
  |                                  |   compares to stored challenge)
  |<-------- access_token ---------- |  Token Response
  |                                  |

// If an attacker intercepts the code, they can't exchange it
// because they don't have the code_verifier that matches
// the code_challenge sent in the original request.

3. ROPC (Resource Owner Password Credentials) is dead

What's removed: The grant_type=password flow.

ROPC let applications collect the user's username and password directly and exchange them for tokens. It was designed as a "migration path" for legacy apps that couldn't redirect to an authorization server. In practice, it became a crutch that eliminated every security benefit of OAuth.

Why it was removed:

// ❌ ROPC: Your app handles raw credentials
const response = await fetch('https://auth.example.com/token', {
  method: 'POST',
  body: new URLSearchParams({
    grant_type: 'password',       // ← Removed in OAuth 2.1
    username: 'user@example.com', // ← App sees the password
    password: 'hunter2',          // ← Phishing risk, credential stuffing
    client_id: CLIENT_ID,
  }),
});

// 🚨 Problems:
// 1. App has user's raw password — violates the entire point of OAuth
// 2. No MFA support — can't do 2FA through a password grant
// 3. No consent screen — user can't control what permissions they grant
// 4. Trains users to type passwords into third-party apps
// 5. If your app is compromised, all user passwords are exposed

The OAuth 2.1 replacement depends on your use case:

Scenario	Old Flow	New Flow
User login (web/mobile)	ROPC	Authorization Code + PKCE
Machine-to-machine auth	ROPC (abused)	Client Credentials
CLI tool authentication	ROPC	Device Authorization (RFC 8628)
Legacy system migration	ROPC	Token Exchange (RFC 8693)

Device Authorization flow for CLIs:

// ✅ Device Authorization Flow (replaces ROPC for CLI tools)
async function deviceLogin(): Promise<void> {
  // Step 1: Request a device code
  const deviceResponse = await fetch('https://auth.example.com/device', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      client_id: CLI_CLIENT_ID,
      scope: 'read write',
    }),
  });

  const { device_code, user_code, verification_uri, interval } =
    await deviceResponse.json();

  // Step 2: Display code to user
  console.log(`Open ${verification_uri} and enter code: ${user_code}`);

  // Step 3: Poll for completion
  while (true) {
    await sleep(interval * 1000);

    const tokenResponse = await fetch('https://auth.example.com/token', {
      method: 'POST',
      body: new URLSearchParams({
        grant_type: 'urn:ietf:params:oauth:grant-type:device_code',
        device_code,
        client_id: CLI_CLIENT_ID,
      }),
    });

    const result = await tokenResponse.json();

    if (result.access_token) {
      saveTokens(result);
      console.log('Authenticated successfully!');
      return;
    }

    if (result.error === 'expired_token') {
      throw new Error('Login expired. Please try again.');
    }
    // 'authorization_pending' or 'slow_down' → keep polling
  }
}

4. Strict redirect URI matching

What changed: OAuth 2.0 allowed "loose" redirect URI matching — prefix matching, wildcard subdomains, and other flexible patterns. OAuth 2.1 requires exact string matching.

// ❌ OAuth 2.0: These "loose" patterns were allowed
// Registered: https://app.example.com/callback
// Match: https://app.example.com/callback?foo=bar     ← OK
// Match: https://app.example.com/callback/extra        ← OK (prefix match)
// Match: https://*.example.com/callback                ← OK (wildcard)

// ✅ OAuth 2.1: Exact string matching only
// Registered: https://app.example.com/callback
// Match: https://app.example.com/callback              ← OK
// No match: https://app.example.com/callback?foo=bar   ← REJECTED
// No match: https://app.example.com/callback/           ← REJECTED (trailing slash)
// No match: https://sub.example.com/callback            ← REJECTED

Why this matters more than you think:

Open redirect vulnerabilities were one of the most common OAuth attacks. An attacker could register a redirect URI like https://evil.com/steal and, if the authorization server used prefix matching with a lax comparison, redirect the authorization code to their server.

Migration checklist:

// Audit your redirect URI registrations
const redirectUris = {
  // ❌ Problems to fix:
  bad: [
    'https://app.example.com/*',              // Wildcard — not allowed
    'https://app.example.com/auth/callback/',  // Trailing slash mismatch
    'http://localhost:3000/callback',          // HTTP in production
  ],
  // ✅ Correct registrations:
  good: [
    'https://app.example.com/auth/callback',   // Exact match, no trailing slash
    'https://staging.example.com/auth/callback', // Separate entry for staging
    'http://127.0.0.1:3000/callback',           // Loopback for dev (RFC 8252)
    'http://[::1]:3000/callback',               // IPv6 loopback for dev
  ],
};

// Note: For development, RFC 8252 allows HTTP on loopback addresses
// (127.0.0.1 and [::1]), but NOT on "localhost" (DNS resolution issue)

Additional security requirements

Beyond the four breaking changes, OAuth 2.1 tightens several other security practices:

No bearer tokens in URLs

// ❌ OAuth 2.0 allowed this
fetch('https://api.example.com/data?access_token=eyJhbGciOi...');
// Token in URL = logged in server access logs, proxy logs, CDN logs, browser history

// ✅ OAuth 2.1: Authorization header only
fetch('https://api.example.com/data', {
  headers: {
    'Authorization': 'Bearer eyJhbGciOi...',
  },
});

// ✅ Or in POST body for form submissions
fetch('https://api.example.com/data', {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: new URLSearchParams({
    access_token: 'eyJhbGciOi...',
  }),
});

Refresh token rotation

OAuth 2.1 strongly recommends (effectively requires) refresh token rotation. Each time a refresh token is used, the old one is invalidated and a new one is issued.

interface TokenStore {
  accessToken: string;
  refreshToken: string;
  expiresAt: number;
}

async function refreshAccessToken(store: TokenStore): Promise<TokenStore> {
  const response = await fetch('https://auth.example.com/token', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'refresh_token',
      refresh_token: store.refreshToken,
      client_id: CLIENT_ID,
    }),
  });

  if (!response.ok) {
    // Refresh token was already used or revoked
    // Force re-authentication
    throw new AuthenticationRequiredError('Session expired');
  }

  const data = await response.json();

  return {
    accessToken: data.access_token,
    refreshToken: data.refresh_token,  // ← New refresh token!
    expiresAt: Date.now() + data.expires_in * 1000,
  };
}

// 🚨 Critical: Handle race conditions
// If two tabs try to refresh at the same time, one will get a new
// refresh token and the other will fail because the old token was
// invalidated. Use a mutex or leader election pattern:

class TokenRefresher {
  private refreshPromise: Promise<TokenStore> | null = null;

  async getValidToken(store: TokenStore): Promise<TokenStore> {
    if (store.expiresAt > Date.now() + 30_000) {
      return store; // Still valid with 30s buffer
    }

    // Deduplicate concurrent refresh attempts
    if (!this.refreshPromise) {
      this.refreshPromise = refreshAccessToken(store).finally(() => {
        this.refreshPromise = null;
      });
    }

    return this.refreshPromise;
  }
}

Complete migration: React SPA example

Here's a complete before/after migration for a typical React SPA that was using the Implicit Grant:

// === auth.ts — OAuth 2.1 compliant authentication module ===

const AUTH_CONFIG = {
  authority: 'https://auth.example.com',
  clientId: 'my-spa-client',
  redirectUri: 'https://app.example.com/auth/callback', // Exact match
  scope: 'openid profile email',
  tokenEndpoint: 'https://auth.example.com/token',
  authorizeEndpoint: 'https://auth.example.com/authorize',
};

// --- PKCE Utilities ---

function generateRandomString(length: number): string {
  const array = new Uint8Array(length);
  crypto.getRandomValues(array);
  return btoa(String.fromCharCode(...array))
    .replace(/\+/g, '-')
    .replace(/\//g, '_')
    .replace(/=+$/, '');
}

async function sha256(plain: string): Promise<ArrayBuffer> {
  const encoder = new TextEncoder();
  return crypto.subtle.digest('SHA-256', encoder.encode(plain));
}

async function generatePKCE(): Promise<{
  verifier: string;
  challenge: string;
}> {
  const verifier = generateRandomString(32);
  const hashed = await sha256(verifier);
  const challenge = btoa(String.fromCharCode(...new Uint8Array(hashed)))
    .replace(/\+/g, '-')
    .replace(/\//g, '_')
    .replace(/=+$/, '');

  return { verifier, challenge };
}

// --- Login Flow ---

export async function login(): Promise<void> {
  const { verifier, challenge } = await generatePKCE();
  const state = generateRandomString(16);

  // Store PKCE verifier and state in sessionStorage
  // (survives the redirect, clears when tab closes)
  sessionStorage.setItem('oauth_code_verifier', verifier);
  sessionStorage.setItem('oauth_state', state);

  const params = new URLSearchParams({
    response_type: 'code',
    client_id: AUTH_CONFIG.clientId,
    redirect_uri: AUTH_CONFIG.redirectUri,
    scope: AUTH_CONFIG.scope,
    state,
    code_challenge: challenge,
    code_challenge_method: 'S256',
  });

  window.location.href =
    `${AUTH_CONFIG.authorizeEndpoint}?${params.toString()}`;
}

// --- Callback Handler ---

export async function handleCallback(): Promise<{
  accessToken: string;
  refreshToken: string;
  idToken: string;
}> {
  const params = new URLSearchParams(window.location.search);
  const code = params.get('code');
  const state = params.get('state');

  // Verify state to prevent CSRF
  const savedState = sessionStorage.getItem('oauth_state');
  if (!state || state !== savedState) {
    throw new Error('Invalid state parameter — possible CSRF attack');
  }

  const verifier = sessionStorage.getItem('oauth_code_verifier');
  if (!verifier) {
    throw new Error('Missing PKCE verifier — restart login flow');
  }

  // Exchange code for tokens
  const response = await fetch(AUTH_CONFIG.tokenEndpoint, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'authorization_code',
      code: code!,
      redirect_uri: AUTH_CONFIG.redirectUri,
      client_id: AUTH_CONFIG.clientId,
      code_verifier: verifier,
    }),
  });

  if (!response.ok) {
    const error = await response.json();
    throw new Error(`Token exchange failed: ${error.error_description}`);
  }

  // Clean up
  sessionStorage.removeItem('oauth_code_verifier');
  sessionStorage.removeItem('oauth_state');

  // Clean URL
  window.history.replaceState({}, '', window.location.pathname);

  return response.json();
}

// --- Token Management ---

export async function refreshToken(
  currentRefreshToken: string
): Promise<{
  accessToken: string;
  refreshToken: string;
}> {
  const response = await fetch(AUTH_CONFIG.tokenEndpoint, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'refresh_token',
      refresh_token: currentRefreshToken,
      client_id: AUTH_CONFIG.clientId,
    }),
  });

  if (!response.ok) {
    // Refresh token rotated and already used, or revoked
    throw new Error('SESSION_EXPIRED');
  }

  return response.json();
}

// --- Logout ---

export async function logout(idToken: string): Promise<void> {
  // Revoke tokens server-side
  await fetch('https://auth.example.com/revoke', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      token: idToken,
      token_type_hint: 'access_token',
      client_id: AUTH_CONFIG.clientId,
    }),
  });

  // Clear local state and redirect
  sessionStorage.clear();
  window.location.href = 'https://auth.example.com/logout?' +
    new URLSearchParams({
      id_token_hint: idToken,
      post_logout_redirect_uri: 'https://app.example.com',
    }).toString();
}

Migration guide by identity provider

Each major identity provider has different timelines for enforcing OAuth 2.1 semantics:

Auth0 / Okta

// Auth0 SDK v2+ already defaults to Authorization Code + PKCE
// Migration: Update the SDK and remove legacy config

// ❌ Old Auth0 configuration
const auth0 = new Auth0Client({
  domain: 'your-tenant.auth0.com',
  clientId: 'YOUR_CLIENT_ID',
  useRefreshTokens: false,     // ← Was common in Implicit setups
  cacheLocation: 'localstorage', // ← Was needed without refresh tokens
});

// ✅ New Auth0 configuration (OAuth 2.1 compliant)
const auth0 = new Auth0Client({
  domain: 'your-tenant.auth0.com',
  clientId: 'YOUR_CLIENT_ID',
  authorizationParams: {
    redirect_uri: 'https://app.example.com/callback', // Exact match
  },
  useRefreshTokens: true,      // ← Enable refresh token rotation
  cacheLocation: 'memory',     // ← In-memory is more secure
});

Google OAuth

// Google deprecated the Implicit flow for new apps in 2022
// Existing apps: migration deadline varies

// ❌ Old: Google Sign-In (Implicit)
// <script src="https://apis.google.com/js/platform.js"></script>
// gapi.auth2.init({ client_id: '...' }) — deprecated

// ✅ New: Google Identity Services (Authorization Code + PKCE)
// Uses the new GIS library with code flow
google.accounts.oauth2.initCodeClient({
  client_id: GOOGLE_CLIENT_ID,
  scope: 'openid profile email',
  ux_mode: 'redirect',
  redirect_uri: 'https://app.example.com/auth/google/callback',
  state: generateState(),
});

Microsoft Entra ID (Azure AD)

// MSAL.js v2+ uses Authorization Code + PKCE by default
import { PublicClientApplication } from '@azure/msal-browser';

const msalConfig = {
  auth: {
    clientId: 'YOUR_CLIENT_ID',
    authority: 'https://login.microsoftonline.com/YOUR_TENANT_ID',
    redirectUri: 'https://app.example.com/auth/callback', // Exact match
  },
  cache: {
    cacheLocation: 'sessionStorage', // ← More secure than localStorage
    storeAuthStateInCookie: false,
  },
};

const msalInstance = new PublicClientApplication(msalConfig);

// Login — PKCE is automatic with MSAL v2+
await msalInstance.loginRedirect({
  scopes: ['openid', 'profile', 'User.Read'],
});

Security hardening beyond OAuth 2.1

OAuth 2.1 sets the floor, not the ceiling. For production apps handling sensitive data, consider these additional measures:

DPoP (Demonstration of Proof-of-Possession)

DPoP binds access tokens to a specific client, preventing token theft and replay attacks. Instead of bearer tokens (which anyone can use if stolen), DPoP tokens are cryptographically bound to the client's key pair.

// DPoP: Proof-of-Possession tokens
async function createDPoPProof(
  url: string,
  method: string,
  accessToken?: string
): Promise<string> {
  // Generate or retrieve your DPoP key pair
  const keyPair = await crypto.subtle.generateKey(
    { name: 'ECDSA', namedCurve: 'P-256' },
    false,
    ['sign', 'verify']
  );

  const header = {
    alg: 'ES256',
    typ: 'dpop+jwt',
    jwk: await crypto.subtle.exportKey('jwk', keyPair.publicKey),
  };

  const payload = {
    jti: crypto.randomUUID(),
    htm: method,
    htu: url,
    iat: Math.floor(Date.now() / 1000),
    // Include access token hash for token binding
    ...(accessToken && {
      ath: await sha256Base64url(accessToken),
    }),
  };

  return signJWT(header, payload, keyPair.privateKey);
}

// Usage: Attach DPoP proof to every API request
const dpopProof = await createDPoPProof(
  'https://api.example.com/data',
  'GET',
  accessToken
);

fetch('https://api.example.com/data', {
  headers: {
    'Authorization': `DPoP ${accessToken}`,
    'DPoP': dpopProof,
  },
});

Pushed Authorization Requests (PAR)

PAR prevents authorization request tampering by sending authorization parameters directly to the server before the redirect:

// PAR: Push authorization parameters server-side first
async function initiateLoginWithPAR(): Promise<void> {
  const { verifier, challenge } = await generatePKCE();
  const state = generateRandomString(16);

  // Step 1: Push authorization request to the server
  const parResponse = await fetch('https://auth.example.com/par', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      client_id: AUTH_CONFIG.clientId,
      redirect_uri: AUTH_CONFIG.redirectUri,
      scope: AUTH_CONFIG.scope,
      response_type: 'code',
      state,
      code_challenge: challenge,
      code_challenge_method: 'S256',
    }),
  });

  const { request_uri } = await parResponse.json();

  // Step 2: Redirect with only the request_uri
  // (All parameters are stored server-side, not in the URL)
  sessionStorage.setItem('oauth_code_verifier', verifier);
  sessionStorage.setItem('oauth_state', state);

  window.location.href =
    `${AUTH_CONFIG.authorizeEndpoint}?` +
    `client_id=${AUTH_CONFIG.clientId}&` +
    `request_uri=${encodeURIComponent(request_uri)}`;
}

OAuth 2.1 and AI agents

One of the less-discussed implications of OAuth 2.1 is its role in the emerging AI agent ecosystem. As Model Context Protocol (MCP) servers and Agent-to-Agent (A2A) protocols mature, OAuth 2.1 provides the security foundation for delegated AI access.

// AI Agent OAuth: Scoped, time-limited access
// An AI agent should NEVER have full user permissions

const agentTokenRequest = {
  grant_type: 'authorization_code',
  code: authorizationCode,
  code_verifier: pkceVerifier,
  client_id: 'ai-agent-client',
  redirect_uri: 'https://agent.example.com/oauth/callback',

  // ← Narrow scopes for AI agents
  scope: 'read:emails read:calendar', // NOT 'write:*'

  // ← Request short-lived tokens
  // (Agent tasks shouldn't need multi-day access)
};

// For MCP server authentication:
// The MCP spec recommends OAuth 2.1 with PKCE
// for third-party tool access via AI agents.
// This ensures users explicitly consent to what
// data the AI agent can access on their behalf.

Why this matters: An AI agent that accesses your GitHub repos, Slack channels, and email inbox needs cryptographically verifiable, narrowly scoped, time-limited authorization — not a static API key in an environment variable. OAuth 2.1 with PKCE + DPoP provides exactly this.

Common migration mistakes

Mistake 1: Storing PKCE verifier in localStorage

// ❌ Bad: localStorage persists across sessions
localStorage.setItem('pkce_verifier', verifier);
// An XSS attack can read this and complete the code exchange

// ✅ Good: sessionStorage clears when tab closes
sessionStorage.setItem('pkce_verifier', verifier);
// Still vulnerable to XSS, but shorter exposure window

// ✅ Best: Store server-side in an HTTP-only session cookie
// (for BFF / Backend-for-Frontend patterns)

Mistake 2: Not handling refresh token race conditions

// ❌ Bad: Two concurrent API calls trigger two refresh attempts
// Tab 1: refresh_token=abc → gets new refresh_token=def
// Tab 2: refresh_token=abc → FAILS (abc was already rotated)
// Tab 2 logs the user out unnecessarily

// ✅ Good: Centralize token refresh with a mutex
class TokenManager {
  private refreshLock: Promise<void> | null = null;

  async getAccessToken(): Promise<string> {
    const tokens = this.getStoredTokens();

    if (this.isExpired(tokens)) {
      if (!this.refreshLock) {
        this.refreshLock = this.doRefresh().finally(() => {
          this.refreshLock = null;
        });
      }
      await this.refreshLock;
    }

    return this.getStoredTokens().accessToken;
  }
}

Mistake 3: Ignoring the state parameter

// ❌ Bad: No state parameter = no CSRF protection
const authUrl = `${authEndpoint}?response_type=code&client_id=${clientId}`;
// Attacker can craft a URL that logs the victim into the attacker's account

// ✅ Good: Always generate and validate state
const state = crypto.randomUUID();
sessionStorage.setItem('oauth_state', state);
const authUrl = `${authEndpoint}?response_type=code&client_id=${clientId}&state=${state}`;

// In callback: verify state matches before exchanging code

Mistake 4: Using "localhost" for development redirects

// ❌ Bad: "localhost" resolves via DNS (can be hijacked)
const devRedirect = 'http://localhost:3000/callback';

// ✅ Good: Use loopback IP addresses (RFC 8252)
const devRedirect = 'http://127.0.0.1:3000/callback';
// Or IPv6: 'http://[::1]:3000/callback'
// These resolve locally without DNS, preventing redirect hijacking

The migration checklist

Use this checklist to audit your existing OAuth implementation:

## OAuth 2.1 Migration Checklist

### Critical (Must fix before IdP enforcement)
- [ ] Remove all `response_type=token` usage (Implicit Grant)
- [ ] Remove all `grant_type=password` usage (ROPC)
- [ ] Add PKCE to all authorization code flows
- [ ] Switch to exact redirect URI matching
- [ ] Remove bearer tokens from URL query strings

### High Priority
- [ ] Implement refresh token rotation
- [ ] Handle refresh token race conditions (mutex pattern)
- [ ] Store PKCE verifiers in sessionStorage, not localStorage
- [ ] Migrate development redirects from localhost to 127.0.0.1
- [ ] Add state parameter to all authorization requests

### Recommended
- [ ] Implement DPoP for high-security token binding
- [ ] Use PAR (Pushed Authorization Requests) for sensitive flows
- [ ] Scope AI agent tokens narrowly (read-only, time-limited)
- [ ] Audit third-party libraries for OAuth 2.1 compliance
- [ ] Set up token revocation on logout

Conclusion

OAuth 2.1 isn't about adding complexity — it's about removing the sharp edges that caused a decade of security incidents. The Implicit Grant was a shortcut that created real vulnerabilities. ROPC was a migration path that became permanent. Wildcard redirects were convenient until they weren't.

The migration is straightforward for most applications:

Replace Implicit Grant with Authorization Code + PKCE. This is the single most impactful change. If you're using a modern SDK (Auth0, MSAL, Firebase), updating the SDK version often handles this automatically.
Remove ROPC flows. Switch to the appropriate replacement: Authorization Code for user-facing apps, Client Credentials for M2M, and Device Authorization for CLIs.
Audit your redirect URIs. Register every environment (production, staging, development) as an exact-match URI. Use 127.0.0.1 instead of localhost for local development.
Enable refresh token rotation. Implement the mutex pattern to handle concurrent refresh attempts across tabs.
Stop putting tokens in URLs. Use the Authorization header exclusively.

The identity providers are already moving — Auth0, Okta, Google, and Microsoft have either deprecated legacy flows or set enforcement timelines. Migrating now, on your own schedule, is significantly less painful than migrating under deadline pressure when your authentication starts returning 400 errors.

⚡ Speed Tip: Read the original post on the Pockit Blog.

Tired of slow cloud tools? Pockit.tools runs entirely in your browser. Get the Extension now for instant, zero-latency access to essential dev tools.

Running LLMs in the Browser: WebGPU, Transformers.js, and Chrome's Built-in AI Explained

HK Lee — Tue, 24 Mar 2026 14:06:00 +0000

Every AI feature you ship today makes an API call. User types a prompt, your server forwards it to OpenAI, waits 800ms-3s, pays $0.01-0.50, and sends the response back. For a chat feature with 10,000 DAU, that's $3,000-15,000/month in API costs alone — before you even count server infrastructure.

But what if the model ran on the user's device? No API call. No latency beyond computation. No cost per inference. No user data ever leaving the browser.

This isn't hypothetical anymore. In 2026, the browser has become a legitimate AI inference runtime. WebGPU provides near-native GPU access. Quantized models under 2GB run at interactive speeds on consumer hardware. Chrome ships Gemini Nano on-device. And libraries like Transformers.js have made the developer experience surprisingly smooth.

This guide covers everything you need to run LLMs in the browser today: the technology stack, model selection and quantization, performance benchmarks on real hardware, Chrome's Built-in AI APIs, and production patterns for shipping client-side AI features. All with working TypeScript code.

Why run AI in the browser?

Before diving into implementation, let's be clear about when client-side AI makes sense — and when it doesn't.

The case for browser-based AI

Zero marginal cost per inference. Once the model downloads, every subsequent inference is free. For features with high per-user query volume (autocomplete, grammar checking, code suggestions), the unit economics are dramatically better than API calls.

Privacy by architecture. User data never leaves the device. No privacy policy gymnastics, no GDPR concerns about data transfer, no risk of training data leakage. For sensitive domains (healthcare, legal, personal journals), this isn't a nice-to-have — it's a requirement.

Latency under 100ms. No network round-trip means responses can be near-instant for small models. Autocomplete, inline suggestions, and real-time classification feel instantaneous.

Offline capability. Once the model is cached, it works without network connectivity. PWAs with AI features that work on a plane — that's a real differentiator.

No rate limits. No API quotas, no throttling, no "429 Too Many Requests" at 3 AM when your feature launches on Hacker News.

When server-side AI is still better

Large model capability. GPT-4-class reasoning still requires 100B+ parameter models that don't fit in a browser. For complex reasoning, multi-step agents, or large context windows, API calls remain necessary.

First-load experience. Model downloads (500MB-2GB) create a significant first-use delay. Users on slow connections will wait minutes before their first inference.

Mobile battery constraints. Running GPU inference on mobile devices drains battery aggressively. Heavy inference workloads need server-side handling for mobile users.

Consistency guarantees. Different GPUs, drivers, and quantization produce slightly different outputs. If you need reproducible, deterministic results, server-side inference offers more control.

The sweet spot in 2026: use browser AI for high-frequency, low-complexity tasks (autocomplete, classification, summarization of short text, embeddings) and server AI for heavy reasoning (multi-step agents, long-form generation, complex analysis).

The technology stack

Three pillars make browser-based AI possible in 2026:

1. WebGPU — The performance backbone

WebGPU replaces WebGL as the modern GPU API for the web. Unlike WebGL (designed for graphics), WebGPU was built for compute workloads — exactly what neural network inference needs.

// Check WebGPU support
async function checkWebGPU(): Promise<GPUDevice | null> {
  if (!navigator.gpu) {
    console.warn('WebGPU not supported in this browser');
    return null;
  }

  const adapter = await navigator.gpu.requestAdapter();
  if (!adapter) {
    console.warn('No GPU adapter found');
    return null;
  }

  const device = await adapter.requestDevice();

  // Log GPU info
  const info = await adapter.requestAdapterInfo();
  console.log(`GPU: ${info.vendor} ${info.device}`);
  console.log(`Max buffer size: ${device.limits.maxBufferSize / 1024 / 1024}MB`);
  console.log(`Max compute workgroup size: ${device.limits.maxComputeWorkgroupSizeX}`);

  return device;
}

Browser support (March 2026):

Browser	WebGPU Status	Notes
Chrome 113+	✅ Stable	Full support since April 2023
Edge 113+	✅ Stable	Chromium-based, same as Chrome
Firefox 147+	✅ Stable	Enabled by default since Jan 2026 (Win/macOS)
Safari 26+	✅ Stable	Full WebGPU support on macOS/iOS/iPadOS
Mobile Chrome	⚠️ Android only	Requires flagship GPU (Adreno 730+)
iOS Safari 26+	✅ Supported	WebGPU available on iOS 26+

WebGPU vs WebGL performance for matrix multiplication (critical for transformers):

Operation	WebGL	WebGPU	Speedup
MatMul 1024×1024	45ms	8ms	5.6×
MatMul 4096×4096	890ms	95ms	9.4×
Batch attention (8 heads)	120ms	18ms	6.7×
Full forward pass (125M params)	340ms	52ms	6.5×

The jump is massive. WebGPU's compute shaders, shared memory, and workgroup synchronization unlock performance that makes real-time LLM inference viable in the browser.

2. Transformers.js — The developer-friendly path

Transformers.js (by Hugging Face) brings the familiar Transformers Python API to JavaScript. Under the hood, it uses ONNX Runtime Web, which delegates to WebGPU for acceleration.

import { pipeline, env } from '@huggingface/transformers';

// Configure for browser
env.allowLocalModels = false;
env.useBrowserCache = true;

// Text generation — runs entirely client-side
const generator = await pipeline('text-generation', 
  'onnx-community/Qwen2.5-0.5B-Instruct', {
    device: 'webgpu',
    dtype: 'q4',  // 4-bit quantization
  }
);

const output = await generator('Explain WebGPU in one paragraph:', {
  max_new_tokens: 150,
  temperature: 0.7,
  do_sample: true,
});

console.log(output[0].generated_text);

Key Transformers.js v3 features:

WebGPU device targeting (device: 'webgpu')
Built-in quantization support (dtype: 'q4', 'q4f16', 'fp16')
Streaming token generation for chat UIs
1,200+ pre-converted ONNX models on Hugging Face
Model caching in browser Cache API (persists across sessions)
Web Worker support for non-blocking inference

3. ONNX Runtime Web — The inference engine

ONNX Runtime Web is the engine beneath Transformers.js. If you need lower-level control or have custom ONNX models, you can use it directly:

import * as ort from 'onnxruntime-web/webgpu';

async function runInference(modelPath: string, inputText: string) {
  // Create session with WebGPU execution provider
  const session = await ort.InferenceSession.create(modelPath, {
    executionProviders: ['webgpu'],
    graphOptimizationLevel: 'all',
  });

  // Prepare input tensor
  const inputIds = tokenize(inputText); // Your tokenizer
  const tensor = new ort.Tensor('int64', 
    BigInt64Array.from(inputIds.map(BigInt)), 
    [1, inputIds.length]
  );

  // Run inference
  const results = await session.run({ input_ids: tensor });

  return results.logits;
}

When to use ONNX Runtime directly vs Transformers.js:

Scenario	Use Transformers.js	Use ONNX Runtime directly
Standard NLP tasks	✅ High-level API	Overkill
Custom fine-tuned models	If already ONNX	✅ Full control
Non-text modalities (audio, vision)	✅ Supported pipelines	For custom pipelines
Maximum performance tuning	Limited control	✅ Session options, graph optimization
Prototype speed	✅ 3 lines of code	More boilerplate

Model selection: What actually runs in a browser?

This is the critical question. A 70B parameter model in fp16 needs 140GB of VRAM — obviously not happening in a browser tab. But with aggressive quantization, you have more options than you'd expect.

Models that work well (March 2026)

Model	Params	Quantized Size	tok/s (RTX 4070)	tok/s (M3 MacBook)	Best for
Qwen2.5-0.5B-Instruct	0.5B	350MB (Q4)	85	45	Classification, extraction
Qwen2.5-1.5B-Instruct	1.5B	900MB (Q4)	42	22	Short text generation
SmolLM2-1.7B-Instruct	1.7B	1.0GB (Q4)	38	20	General chat
Phi-3.5-mini-instruct	3.8B	2.1GB (Q4)	18	9	Reasoning tasks
Gemma-2-2B-Instruct	2.0B	1.2GB (Q4)	28	14	Instruction following
Llama-3.2-1B-Instruct	1.2B	750MB (Q4)	52	28	Fast general purpose

Rule of thumb: For interactive browser UIs, you want >20 tokens/second. This limits you to models ≤2B parameters on mainstream hardware. 3B+ models work but feel sluggish for real-time chat.

Quantization: Trading size for speed

Quantization reduces model precision from 32-bit floats to smaller representations. Here's what the options mean:

fp32 (32-bit) → fp16 (16-bit) → int8 (8-bit) → int4 (4-bit)
   Full size    →    Half     →   Quarter    →   Eighth
   Best quality →              →              → Fastest/smallest

Impact on quality (measured on MMLU benchmark for Qwen2.5-1.5B):

Precision	Model Size	MMLU Score	Tokens/sec	Memory Usage
fp16	3.0 GB	61.8	12	3.4 GB
int8	1.5 GB	61.2	28	1.8 GB
int4 (Q4)	900 MB	59.1	42	1.2 GB
int4 (Q4_K_M)	950 MB	60.3	40	1.3 GB

The Q4_K_M mixed quantization is the sweet spot — it keeps attention layers at higher precision while aggressively quantizing feed-forward layers, preserving 97% of the quality at 1/3 the size.

Loading models with progress tracking

Users need to see download progress. Here's a production-ready model loader:

import { AutoTokenizer, AutoModelForCausalLM, TextStreamer } from '@huggingface/transformers';

interface LoadingProgress {
  status: 'downloading' | 'loading' | 'ready';
  file?: string;
  progress?: number;
  loaded?: number;
  total?: number;
}

async function loadModel(
  modelId: string,
  onProgress: (progress: LoadingProgress) => void
): Promise<{ model: any; tokenizer: any }> {
  onProgress({ status: 'downloading' });

  const tokenizer = await AutoTokenizer.from_pretrained(modelId, {
    progress_callback: (data: any) => {
      if (data.status === 'progress') {
        onProgress({
          status: 'downloading',
          file: data.file,
          progress: data.progress,
          loaded: data.loaded,
          total: data.total,
        });
      }
    },
  });

  const model = await AutoModelForCausalLM.from_pretrained(modelId, {
    device: 'webgpu',
    dtype: 'q4',
    progress_callback: (data: any) => {
      if (data.status === 'progress') {
        onProgress({
          status: 'downloading',
          file: data.file,
          progress: data.progress,
          loaded: data.loaded,
          total: data.total,
        });
      }
    },
  });

  onProgress({ status: 'ready' });

  return { model, tokenizer };
}

Chrome's Built-in AI APIs

Chrome 131+ introduced experimental Built-in AI APIs that let you use Gemini Nano (a small on-device model) through browser-native APIs. No model downloads. No libraries. The model ships with Chrome itself.

The Prompt API

// Check availability
const capabilities = await self.ai.languageModel.capabilities();
console.log(capabilities.available); // 'readily', 'after-download', 'no'

if (capabilities.available !== 'no') {
  // Create a session
  const session = await self.ai.languageModel.create({
    systemPrompt: 'You are a helpful coding assistant. Be concise.',
    temperature: 0.7,
    topK: 40,
  });

  // Simple prompt
  const result = await session.prompt('What is a closure in JavaScript?');
  console.log(result);

  // Streaming
  const stream = session.promptStreaming('Explain WebGPU briefly.');
  for await (const chunk of stream) {
    process.stdout.write(chunk);
  }

  // Session maintains conversation context
  const followUp = await session.prompt('Give me a code example.');

  // Cleanup
  session.destroy();
}

The Summarization API

const summarizer = await self.ai.summarizer.create({
  type: 'tl;dr',        // 'tl;dr', 'key-points', 'teaser', 'headline'
  length: 'medium',      // 'short', 'medium', 'long'
  format: 'plain-text',  // 'plain-text', 'markdown'
});

const summary = await summarizer.summarize(longArticleText);
console.log(summary);

The Translation API

const translator = await self.ai.translator.create({
  sourceLanguage: 'en',
  targetLanguage: 'ja',
});

const translated = await translator.translate('Hello, world!');
console.log(translated); // こんにちは、世界！

Built-in AI vs Transformers.js: When to use which

Factor	Chrome Built-in AI	Transformers.js
Model download	None (ships with Chrome)	350MB-2GB first load
Setup complexity	3 lines of code	npm install + config
Model choice	Gemini Nano only	1,200+ models
Browser support	Chrome only	All WebGPU browsers
Quality (vs GPT-4)	~60%	Varies by model (50-75%)
Task flexibility	Text, image, audio (multimodal)	Text, vision, audio, embeddings
Fine-tuning	Not possible	Custom ONNX models
Offline	✅ After Chrome install	✅ After model cache

Recommendation: Use Chrome Built-in AI for quick prototypes and Chrome-only features. Use Transformers.js when you need cross-browser support, specific models, or non-text modalities.

Production patterns

Pattern 1: Web Worker isolation

Never run inference on the main thread. GPU compute blocks the event loop and freezes your UI.

// ai-worker.ts — run in a Web Worker
import { pipeline } from '@huggingface/transformers';

let generator: any = null;

self.onmessage = async (e: MessageEvent) => {
  const { type, payload } = e.data;

  switch (type) {
    case 'LOAD': {
      self.postMessage({ type: 'STATUS', status: 'loading' });

      generator = await pipeline('text-generation', payload.model, {
        device: 'webgpu',
        dtype: 'q4',
        progress_callback: (progress: any) => {
          self.postMessage({ type: 'PROGRESS', progress });
        },
      });

      self.postMessage({ type: 'STATUS', status: 'ready' });
      break;
    }

    case 'GENERATE': {
      if (!generator) {
        self.postMessage({ type: 'ERROR', error: 'Model not loaded' });
        return;
      }

      const result = await generator(payload.prompt, {
        max_new_tokens: payload.maxTokens ?? 256,
        temperature: payload.temperature ?? 0.7,
        do_sample: true,
      });

      self.postMessage({ 
        type: 'RESULT', 
        text: result[0].generated_text,
      });
      break;
    }
  }
};

// main.ts — use from your app
class BrowserAI {
  private worker: Worker;
  private pending = new Map<string, (value: any) => void>();

  constructor() {
    this.worker = new Worker(
      new URL('./ai-worker.ts', import.meta.url),
      { type: 'module' }
    );

    this.worker.onmessage = (e) => {
      // Handle responses
    };
  }

  async load(model: string): Promise<void> {
    this.worker.postMessage({ type: 'LOAD', payload: { model } });
    // Wait for 'ready' status...
  }

  async generate(prompt: string, options = {}): Promise<string> {
    this.worker.postMessage({ 
      type: 'GENERATE', 
      payload: { prompt, ...options },
    });
    // Wait for result...
    return '';
  }
}

Pattern 2: Streaming token generation

For chat UIs, stream tokens as they're generated:

import { 
  AutoTokenizer, 
  AutoModelForCausalLM, 
  TextStreamer 
} from '@huggingface/transformers';

async function* streamGenerate(
  model: any,
  tokenizer: any,
  prompt: string,
  maxTokens: number = 256,
): AsyncGenerator<string> {
  const inputs = tokenizer(prompt, { return_tensors: 'pt' });

  // Custom streamer that yields tokens
  const tokens: string[] = [];
  let resolveNext: ((value: string) => void) | null = null;

  const streamer = new TextStreamer(tokenizer, {
    skip_prompt: true,
    callback_function: (text: string) => {
      if (resolveNext) {
        resolveNext(text);
        resolveNext = null;
      } else {
        tokens.push(text);
      }
    },
  });

  // Start generation (runs in background)
  const generatePromise = model.generate({
    ...inputs,
    max_new_tokens: maxTokens,
    temperature: 0.7,
    do_sample: true,
    streamer,
  });

  // Yield tokens as they arrive
  while (true) {
    if (tokens.length > 0) {
      yield tokens.shift()!;
    } else {
      const token = await new Promise<string>((resolve) => {
        resolveNext = resolve;
      });
      yield token;
    }

    // Check if generation is complete
    // (simplified — real implementation needs done signal)
  }

  await generatePromise;
}

// Usage in a React component
function ChatMessage({ prompt }: { prompt: string }) {
  const [text, setText] = useState('');

  useEffect(() => {
    (async () => {
      for await (const token of streamGenerate(model, tokenizer, prompt)) {
        setText(prev => prev + token);
      }
    })();
  }, [prompt]);

  return <p>{text}</p>;
}

Pattern 3: Graceful degradation with server fallback

Not all users have WebGPU. Build a fallback chain:

type AIBackend = 'webgpu' | 'wasm' | 'server';

async function detectBestBackend(): Promise<AIBackend> {
  // 1. Try WebGPU
  if (navigator.gpu) {
    const adapter = await navigator.gpu.requestAdapter();
    if (adapter) {
      const info = await adapter.requestAdapterInfo();
      // Check for minimum GPU capability
      const device = await adapter.requestDevice();
      if (device.limits.maxBufferSize >= 256 * 1024 * 1024) {
        return 'webgpu';
      }
    }
  }

  // 2. Fall back to WASM (CPU-only, slower but universal)
  if (typeof WebAssembly !== 'undefined') {
    return 'wasm';
  }

  // 3. Last resort: server-side
  return 'server';
}

async function createAIClient(): Promise<AIClient> {
  const backend = await detectBestBackend();

  switch (backend) {
    case 'webgpu':
      return new BrowserAIClient({ 
        device: 'webgpu', 
        model: 'onnx-community/Qwen2.5-0.5B-Instruct' 
      });

    case 'wasm':
      return new BrowserAIClient({ 
        device: 'wasm',
        model: 'onnx-community/Qwen2.5-0.5B-Instruct',
        // WASM is 5-10x slower but works everywhere
      });

    case 'server':
      return new ServerAIClient({ 
        endpoint: '/api/ai/generate' 
      });
  }
}

Pattern 4: Smart model caching

Models are large. Cache them properly to avoid re-downloads:

class ModelCache {
  private cacheName = 'ai-models-v1';

  async getCacheInfo(): Promise<{
    models: string[];
    totalSize: number;
  }> {
    const cache = await caches.open(this.cacheName);
    const keys = await cache.keys();

    let totalSize = 0;
    const models: string[] = [];

    for (const request of keys) {
      const response = await cache.match(request);
      if (response) {
        const blob = await response.blob();
        totalSize += blob.size;
        models.push(new URL(request.url).pathname);
      }
    }

    return { models, totalSize };
  }

  async clearOldModels(maxCacheSizeMB: number = 2048): Promise<void> {
    const { totalSize } = await this.getCacheInfo();

    if (totalSize > maxCacheSizeMB * 1024 * 1024) {
      // Clear cache and re-download active model
      await caches.delete(this.cacheName);
      console.log(`Cleared model cache (was ${(totalSize / 1024 / 1024).toFixed(0)}MB)`);
    }
  }

  async isModelCached(modelId: string): Promise<boolean> {
    const cache = await caches.open(this.cacheName);
    const keys = await cache.keys();
    return keys.some(k => k.url.includes(modelId));
  }
}

// Show UI based on cache status
async function initAI() {
  const cache = new ModelCache();
  const isCached = await cache.isModelCached('Qwen2.5-0.5B-Instruct');

  if (isCached) {
    // Instant load — model already downloaded
    showStatus('Loading AI model from cache...');
    // Loads in 2-5 seconds from cache vs 30-60s download
  } else {
    // First-time download needed
    showStatus('Downloading AI model (350MB)...');
    showProgressBar();
  }
}

Practical use cases that work today

Not every AI use case works in the browser. Here are the ones that do:

1. Smart autocomplete

// Fast, local autocomplete for text inputs
const completer = await pipeline('text-generation', 
  'onnx-community/Qwen2.5-0.5B-Instruct',
  { device: 'webgpu', dtype: 'q4' }
);

async function autocomplete(partial: string): Promise<string[]> {
  const prompt = `Complete this sentence naturally: "${partial}"`;

  const results = await completer(prompt, {
    max_new_tokens: 30,
    num_return_sequences: 3,
    temperature: 0.8,
    do_sample: true,
  });

  return results.map((r: any) => 
    r.generated_text.replace(prompt, '').trim()
  );
}

2. Client-side text classification

// Spam detection, sentiment analysis, content moderation — no API calls
const classifier = await pipeline('zero-shot-classification',
  'Xenova/mobilebert-uncased-mnli',
  { device: 'webgpu' }
);

async function classifyContent(text: string): Promise<{
  label: string;
  score: number;
}> {
  const result = await classifier(text, [
    'spam', 'legitimate',
    'positive', 'negative', 'neutral',
    'question', 'statement',
  ]);

  return {
    label: result.labels[0],
    score: result.scores[0],
  };
}

3. Local embeddings for search

// Generate embeddings entirely client-side — great for local search
const embedder = await pipeline('feature-extraction',
  'Xenova/all-MiniLM-L6-v2',
  { device: 'webgpu' }
);

async function embed(text: string): Promise<number[]> {
  const result = await embedder(text, {
    pooling: 'mean',
    normalize: true,
  });

  return Array.from(result.data);
}

// Build a local search index without any API calls
async function localSearch(
  query: string, 
  documents: string[]
): Promise<{ doc: string; score: number }[]> {
  const queryEmbedding = await embed(query);
  const docEmbeddings = await Promise.all(documents.map(embed));

  return docEmbeddings
    .map((docEmb, i) => ({
      doc: documents[i],
      score: cosineSimilarity(queryEmbedding, docEmb),
    }))
    .sort((a, b) => b.score - a.score);
}

4. Real-time translation

// Translation without API calls — perfect for chat apps
const translator = await pipeline('translation',
  'Xenova/nllb-200-distilled-600M',
  { device: 'webgpu', dtype: 'q4' }
);

async function translate(
  text: string, 
  from: string, 
  to: string
): Promise<string> {
  const result = await translator(text, {
    src_lang: from,
    tgt_lang: to,
    max_length: 512,
  });

  return result[0].translation_text;
}

Performance optimization

Warm-up inference

The first inference after model load is always slowest (WebGPU pipeline compilation). Run a warm-up:

async function warmUp(model: any, tokenizer: any): Promise<void> {
  const dummyInput = tokenizer('warmup', { return_tensors: 'pt' });
  await model.generate({
    ...dummyInput,
    max_new_tokens: 1,
  });
  // First real inference will now be 2-3x faster
}

KV cache management

For multi-turn conversations, manage the KV cache to avoid recomputing previous tokens:

interface ConversationState {
  pastKeyValues: any;
  tokenCount: number;
}

async function continueConversation(
  model: any,
  tokenizer: any,
  newMessage: string,
  state: ConversationState | null,
): Promise<{ response: string; newState: ConversationState }> {
  const inputs = tokenizer(newMessage, { return_tensors: 'pt' });

  const generation = await model.generate({
    ...inputs,
    max_new_tokens: 256,
    past_key_values: state?.pastKeyValues ?? null,
    // Reuses cached computation from previous turns
  });

  return {
    response: tokenizer.decode(generation[0], { skip_special_tokens: true }),
    newState: {
      pastKeyValues: generation.past_key_values,
      tokenCount: (state?.tokenCount ?? 0) + inputs.input_ids.length,
    },
  };
}

Memory pressure monitoring

Browsers kill tabs that use too much memory. Monitor and respond:

function monitorMemory(thresholdMB: number = 1500): void {
  if ('memory' in performance) {
    const memInfo = (performance as any).memory;
    const usedMB = memInfo.usedJSHeapSize / 1024 / 1024;
    const limitMB = memInfo.jsHeapSizeLimit / 1024 / 1024;

    console.log(`Memory: ${usedMB.toFixed(0)}MB / ${limitMB.toFixed(0)}MB`);

    if (usedMB > thresholdMB) {
      console.warn('High memory usage — consider unloading model');
      // Trigger model unload or reduce batch size
    }
  }
}

// Check periodically
setInterval(() => monitorMemory(), 10000);

Common pitfalls

Pitfall 1: Blocking the main thread

The most common mistake. Even with WebGPU, model loading and tokenization happen on the CPU and can freeze your UI for seconds.

// ❌ Bad: loading on main thread
const model = await pipeline('text-generation', 'model-id');
// UI is frozen during download + initialization

// ✅ Good: Web Worker + progress UI
const worker = new Worker(new URL('./ai-worker.ts', import.meta.url));
worker.postMessage({ type: 'LOAD', model: 'model-id' });
// Show loading spinner while worker initializes

Pitfall 2: Ignoring model warm-up

First inference is always 2-5x slower due to WebGPU pipeline compilation. Users blame your app.

// ❌ Bad: first user prompt gets slow response
// User types, waits 3 seconds → bad UX

// ✅ Good: warm up immediately after load
await loadModel();
await warmUp(model, tokenizer); // Pre-compile GPU pipelines
// First user prompt is consistently fast

Pitfall 3: No fallback for unsupported browsers

~15% of web users still lack WebGPU support (older browsers, some Android versions, Linux without updated drivers).

// ❌ Bad: assume WebGPU is available
const model = await pipeline('text-generation', 'model', { device: 'webgpu' });
// Crashes on unsupported browsers

// ✅ Good: progressive enhancement
const backend = await detectBestBackend();
if (backend === 'server') {
  showMessage('AI features use our server for your browser. Upgrade to Chrome for faster, private AI.');
}

Pitfall 4: Downloading models on page load

A 900MB download that the user didn't ask for is hostile UX.

// ❌ Bad: auto-download on page load
window.onload = () => loadModel('900MB-model');
// User's bandwidth destroyed, mobile data plan drained

// ✅ Good: load on demand with explicit user action
document.getElementById('ai-btn')!.onclick = async () => {
  showConfirmation('Download AI model (900MB)? It will be cached for future visits.');
  // Only download after user confirms
};

The decision framework

Do you need AI in the browser?
    ↓
Is it high-frequency, low-complexity?
    ↓ Yes                    ↓ No
    ↓                        → Use server API
Is privacy critical?
    ↓ Yes              ↓ No
    ↓                   → Consider server API
    ↓                     (simpler, more capable)
    ↓
Can you tolerate 500MB-2GB first-load download?
    ↓ Yes              ↓ No
    ↓                   → Use Chrome Built-in AI
    ↓                     (zero download, Chrome only)
    ↓
Use Transformers.js + WebGPU
    ↓
Model ≤ 2B params for interactive speed
    ↓
Deploy with Web Worker + server fallback

Conclusion

Browser-based AI in 2026 is real, practical, and ready for production — with caveats. It's not a replacement for server-side AI; it's a complementary layer that excels at specific use cases.

The sweet spot: high-frequency, privacy-sensitive, latency-critical tasks where the cost of API calls doesn't make sense. Autocomplete, classification, local search, content moderation, real-time translation — these all work beautifully with sub-2B parameter models running on WebGPU.

Here's what to do:

Start with a specific use case, not "let's put AI in the browser." Pick the one feature where local inference solves a real problem (cost, privacy, latency).
Default to Qwen2.5-0.5B or Llama-3.2-1B as your first model. Both are fast, capable enough for most tasks, and fit comfortable in browser memory at Q4 quantization.
Always use Web Workers. No exceptions. Main thread inference is an instant path to janky UI.
Build the fallback chain. WebGPU → WASM → Server. Never assume the user's browser supports WebGPU.
Don't download models without asking. An explicit opt-in with size indication is basic UX respect.

The gap between "AI that requires a data center" and "AI that runs in a browser tab" is closing fast. The models are getting smaller and smarter. The runtime (WebGPU) is getting faster. The tooling (Transformers.js) is getting smoother. For the right use cases, client-side AI isn't the future — it's already the best option.

💡 Note: This article was originally published on the Pockit Blog.

Check out Pockit.tools for 60+ free developer utilities. For faster access, add it to Chrome and use JSON Formatter & Diff Checker directly from your toolbar.