Forem: Shane Ho

How I Built a RAG-Powered AI Chatbot Builder with Next.js, Supabase, and pgvector

Shane Ho — Tue, 24 Mar 2026 12:25:30 +0000

Every website has docs, FAQs, and knowledge buried across dozens of pages. I wanted to turn all of that into an AI chatbot that actually knows the website — not a generic GPT wrapper, but something grounded in real content.

So I built a RAG (Retrieval-Augmented Generation) pipeline: scrape a website, chunk the text, generate vector embeddings, store them in Postgres with pgvector, and retrieve relevant context at query time. Here's how the whole thing works under the hood.

The Architecture

The stack:

Next.js 15 (App Router) — frontend + API routes
Supabase — Postgres database with pgvector extension
OpenAI — embeddings (text-embedding-3-small) and chat completions (gpt-4o-mini)
Cheerio — server-side HTML parsing for the scraper

The flow looks like this:

User enters URL → Scrape website → Chunk text → Generate embeddings → Store in pgvector
                                                                          ↓
Visitor asks question → Embed query → Vector similarity search → Build context → LLM generates answer

Step 1: The Web Scraper

The scraper starts at the user's URL and crawls internal links using a BFS approach, up to 20 pages. Cheerio handles HTML parsing — no headless browser needed.

export async function scrapeWebsite(url: string): Promise<ScrapedPage[]> {
  const origin = new URL(url).origin;
  const visited = new Set<string>();
  const queue: string[] = [normalizeUrl(url)];
  const results: ScrapedPage[] = [];

  while (queue.length > 0 && results.length < MAX_PAGES) {
    const current = queue.shift()!;
    if (visited.has(current)) continue;
    visited.add(current);

    const page = await fetchAndParse(current);
    if (!page) continue;

    results.push({ url: current, title: page.title, content: page.content });

    for (const link of page.links) {
      const resolved = resolveLink(link, current, origin);
      if (resolved && !visited.has(resolved)) queue.push(resolved);
    }
  }
  return results;
}

A few important decisions here:

Same-origin only — we skip external links to avoid scraping the entire internet
Content extraction — we strip <script>, <style>, <nav>, <footer>, and [aria-hidden] elements, then prefer <main> or <article> content over the full <body>
URL normalization — trailing slashes, tracking params (utm_*), and hashes are stripped to avoid duplicate pages
Minimum content threshold — pages with less than 50 characters of extracted text are skipped (they're usually empty shells)

Step 2: Chunking for Embeddings

You can't just throw an entire page into an embedding model. The text needs to be split into chunks that are small enough to be meaningful but large enough to carry context.

export function chunkText(
  text: string,
  maxTokens: number = 500,
  overlap: number = 50
): string[] {
  const maxChars = maxTokens * CHARS_PER_TOKEN; // ~4 chars per token
  const overlapChars = overlap * CHARS_PER_TOKEN;

  const paragraphs = text.split(/\n\s*\n/);
  const chunks: string[] = [];
  let currentChunk = '';

  for (const para of paragraphs) {
    if (currentChunk.length + para.length > maxChars && currentChunk.length > 0) {
      chunks.push(currentChunk.trim());
      // Overlap: carry the tail of the previous chunk into the next
      currentChunk = currentChunk.slice(-overlapChars) + ' ' + para;
    } else {
      currentChunk += (currentChunk ? '\n\n' : '') + para;
    }
  }

  if (currentChunk.trim()) chunks.push(currentChunk.trim());
  return chunks.filter(c => c.length > 20);
}

The overlap is key. Without it, if a relevant fact spans a chunk boundary, you lose it entirely. A 50-token overlap ensures continuity. We split on paragraph boundaries first (natural breakpoints), then fall back to sentence splitting for oversized paragraphs.

Step 3: Generating and Storing Embeddings

We use OpenAI's text-embedding-3-small model (1536 dimensions). Texts are batched to stay within token limits:

const EMBEDDING_MODEL = 'text-embedding-3-small';
const EMBEDDING_DIMENSIONS = 1536;

export async function generateEmbeddings(texts: string[]): Promise<number[][]> {
  const batches = createBatches(texts, MAX_TOKENS_PER_BATCH);
  const allEmbeddings: number[][] = [];

  for (const batch of batches) {
    const response = await getOpenAI().embeddings.create({
      model: EMBEDDING_MODEL,
      dimensions: EMBEDDING_DIMENSIONS,
      input: batch,
    });
    const sorted = response.data.sort((a, b) => a.index - b.index);
    for (const item of sorted) allEmbeddings.push(item.embedding);
  }
  return allEmbeddings;
}

On the Supabase side, we store documents with their embeddings and use a match_documents RPC function that performs cosine similarity search via pgvector:

CREATE OR REPLACE FUNCTION match_documents(
  query_embedding vector(1536),
  match_chatbot_id uuid,
  match_count int DEFAULT 5
) RETURNS TABLE (
  id uuid, content text, title text, source_url text, similarity float
) AS $$
  SELECT id, content, title, source_url,
    1 - (embedding <=> query_embedding) AS similarity
  FROM documents
  WHERE chatbot_id = match_chatbot_id
  ORDER BY embedding <=> query_embedding
  LIMIT match_count;
$$ LANGUAGE sql;

The <=> operator is pgvector's cosine distance. We return 1 - distance as similarity so higher = more relevant.

Step 4: The Chat Endpoint

This is where it all comes together. When a visitor asks a question, we need to:

Embed their question
Find the most relevant chunks
Build a prompt with that context
Stream the LLM's response

The critical insight was parallelizing the first phase:

const parallelResults = await Promise.all([
  // Verify chatbot exists and is active
  admin.from('chatbots').select('...').eq('id', chatbot_id).single(),

  // Generate query embedding (~970ms)
  generateEmbeddings([trimmedMessage]),

  // Fetch conversation history
  conversation_id
    ? admin.from('messages').select('role, content')
        .eq('conversation_id', conversation_id).limit(10)
    : Promise.resolve({ data: null, error: null }),
]);

The embedding call takes ~970ms. The database queries take ~50ms each. By running them in parallel instead of sequentially, we save almost a full second on every request.

The system prompt keeps the model grounded:

function buildSystemPrompt(chatbotName: string, context: string): string {
  return `You are "${chatbotName}", a helpful AI assistant that answers
questions based on the website content provided below.

INSTRUCTIONS:
- Answer questions accurately using ONLY the provided context.
- If the context does not contain enough information, say so honestly.
- Do not make up information that is not in the context.

CONTEXT FROM WEBSITE:
${context || 'No relevant content found.'}`;
}

Performance: From 9.5s to 3s

The first version was painfully slow. Here's what the timing breakdown looked like:

Phase	Before	After
Embedding	970ms	970ms (parallel)
DB lookups	150ms	50ms (parallel)
Vector search	120ms	120ms
LLM completion	2-4s	2-4s (streaming)
DB writes	500ms	0ms (deferred)
Total (perceived)	~9.5s	~3s

Three optimizations made the difference:

1. Parallel I/O

Instead of sequential await calls, we use Promise.all() for independent operations. Embedding generation, chatbot verification, and conversation history all run simultaneously.

2. Streaming Responses

We switched from waiting for the full completion to Server-Sent Events (SSE) streaming. The user sees the first token within ~1s, even though the full response takes 3-4s to generate.

const stream = new ReadableStream({
  async start(controller) {
    const openaiStream = await getOpenAI().chat.completions.create({
      model: chatbot.model || 'gpt-4o-mini',
      messages: openaiMessages,
      stream: true,
    });

    for await (const chunk of openaiStream) {
      const content = chunk.choices[0]?.delta?.content || '';
      if (content) {
        controller.enqueue(
          encoder.encode(`data: ${JSON.stringify({ content })}\n\n`)
        );
      }
    }
    controller.close();
  },
});

3. Deferred Database Writes

All database writes (message storage, usage stats, performance metrics) happen after the response is sent to the user, using Next.js's after() API:

after(async () => {
  await deferDbWrites({ admin, activeConversationId, ... });
});

This is critical on Vercel's serverless platform. The user gets their answer immediately; bookkeeping happens in the background.

Lessons Learned

1. Chunk overlap matters more than chunk size. I experimented with 200, 500, and 1000 token chunks. 500 with 50-token overlap gave the best retrieval quality. Too small and you lose context; too large and the embeddings become too generic.

2. pgvector is surprisingly fast. For datasets under 100K vectors, cosine similarity search over pgvector in Supabase completes in under 200ms without any indexing tricks. You don't need Pinecone or Weaviate for most use cases.

3. Fire-and-forget doesn't work on serverless. My first attempt used void saveMetrics() — a fire-and-forget pattern. On Vercel, the function shuts down the moment the response is sent, killing any pending promises. Next.js's after() solves this by extending the function's lifetime for background work.

4. Scraping is the hardest part. Not the AI, not the vector math — scraping. Every website has different HTML structure, JavaScript-rendered content, rate limits, and anti-bot measures. Cheerio handles static HTML well, but SPAs need a different approach.

5. The 'context window' UX problem. When the LLM can't find relevant context, it should say "I don't know" — but users find that frustrating. The trick is to acknowledge what they asked and suggest what the chatbot can help with, based on the available document titles.

The Full Training Pipeline

Putting it all together, the training flow is:

User provides a URL
Scraper crawls up to 20 pages (BFS, same-origin)
Content is chunked (500 tokens, 50 overlap)
Embeddings generated in batches via OpenAI
Documents + embeddings stored in Supabase (pgvector)
Chatbot status set to 'active'

The whole process takes 15-45 seconds depending on the website size.

At query time:

Visitor's question is embedded
pgvector finds the top 5 most similar chunks
Chunks are injected into the system prompt as context
gpt-4o-mini generates a grounded response
Response is streamed back via SSE

If you want to see this in action, I built it into DocuChat — you can paste any website URL and have a working AI chatbot in under a minute.

The full codebase uses Next.js 15 App Router, Supabase with pgvector, and OpenAI's API. If you're building something similar, the key takeaway is: parallelize everything, stream responses, and defer writes. Your users will thank you.

5 Patterns That Make Claude Code Actually Follow Your Rules

Shane Ho — Sun, 15 Mar 2026 06:13:23 +0000

You wrote 150 lines of rules in your CLAUDE.md file. Every line exists because Claude did something wrong and you swore it would never happen again.

Then Claude does it again.

You are not alone. GitHub issues, Reddit threads, and blog posts are full of developers hitting the same wall: Claude Code reads your instructions, acknowledges them, and then quietly ignores them when generating code. It is the single most frustrating part of AI-assisted development.

The good news: this is a solvable problem. After months of daily Claude Code usage and a lot of trial and error, I found 5 patterns that dramatically improve instruction compliance. None of them require special prompts or hacks — just a better understanding of how large language models process instructions.

Why Claude Ignores Your Rules in the First Place

Before the fixes, you need to understand the root cause. Large language models do not "remember" your instructions the way a human would. Every time Claude responds, it reads the entire conversation — system prompt, CLAUDE.md, your messages, its own previous replies — as one long text document.

Two things happen as that document grows:

Primacy/recency bias. Instructions at the very beginning and very end of the context get more attention. Everything in the middle fades.
Cognitive load. Research shows frontier models reliably follow about 150-200 discrete instructions. Claude Code's own system prompt uses roughly 50 of those slots. That leaves you 100-150 slots for your rules.

If your CLAUDE.md has 200 lines of detailed instructions, you have already blown past the budget. Claude is not being stubborn — it is overloaded.

Pattern 1: The 30-Line Rule

Before: A 200-line CLAUDE.md covering every edge case you have ever encountered.

After: A 30-line CLAUDE.md covering only the things Claude cannot infer from your code.

# Project: acme-dashboard

Next.js 14 app with Tailwind, Prisma (PostgreSQL), NextAuth.
Deploy target: Vercel.

## Commands
- Dev: `pnpm dev`
- Test: `pnpm vitest run`
- Lint: `pnpm biome check --write .`
- DB migrate: `pnpm prisma migrate dev`

## Code Style
- camelCase variables, PascalCase components, UPPER_SNAKE_CASE constants
- kebab-case file names
- Pure functions preferred. No classes unless wrapping a third-party SDK.
- All API routes return `{ data, error }` shape.
- Never use `any`. If the type is unknown, use `unknown` and narrow.

## Conventions
- Tests go in `__tests__/` next to the source file.
- Every PR needs at least one test for the changed behavior.
- Error messages must include the function name: `[functionName] what went wrong`
- No default exports. Named exports only.

That is 25 lines. For each line, apply this test: "If I remove this, will Claude make a mistake?" If the answer is no, delete it.

Why this works: fewer instructions means each one gets more attention weight. A focused 30-line file outperforms a comprehensive 200-line file every time.

Pattern 2: Positive Instructions Over Negative Ones

This is the single highest-leverage change most people miss.

Language models struggle with negation. When you write "Do NOT use semicolons," the model has to process the concept of semicolons and then apply a negation to it. The concept itself gets activated either way.

Before (negative):

- Do NOT use default exports
- Do NOT use `any` type
- Do NOT put tests in a separate `tests/` folder
- Never use console.log in production code
- Don't use relative imports from parent directories

After (positive):

- Use named exports exclusively
- Type everything explicitly; use `unknown` and narrow when unsure
- Place tests in `__tests__/` next to the source file
- Use the logger utility (`src/lib/logger.ts`) for all logging
- Use path aliases (`@/`) for all imports

Each rule now tells Claude what TO do instead of what to avoid. The model does not need to resolve a negation — it just follows the positive instruction directly.

In my testing, flipping 10 negative rules to positive equivalents cut rule violations by roughly half.

Pattern 3: Anchor Critical Rules at the Top and Bottom

Remember the primacy/recency bias? Use it deliberately.

Structure your CLAUDE.md so that the rules Claude violates most often are at the very top (first 5 lines) and the very bottom (last 5 lines). Put the less critical rules in the middle.

# CRITICAL — Read these first
- Named exports only. No default exports anywhere.
- Every function must have explicit return types.
- Run `pnpm vitest run` before considering any task complete.

# Project context
Next.js 14, Tailwind, Prisma, PostgreSQL, Vercel.

# Commands
...

# Code style
...

# CRITICAL — Read these last
- Named exports only. No default exports anywhere.
- Every function must have explicit return types.
- Run `pnpm vitest run` before considering any task complete.

Yes, you are duplicating 3 rules. That is intentional. The rules that Claude violates most often deserve two placement slots — one at the beginning where primacy bias is strongest, and one at the end where recency bias kicks in.

This feels redundant, but it works. The most important rules get double the attention weight.

Pattern 4: Use Hooks for Hard Enforcement

Here is the uncomfortable truth: prompt-level instructions are suggestions. No matter how you phrase them, there is always a probability that Claude will skip a rule. If a rule absolutely must be followed every time, do not rely on the CLAUDE.md — enforce it with code.

Claude Code supports hooks that run shell commands at specific lifecycle points. For example, you can block any commit that contains console.log:

// .claude/settings.json
{
  "hooks": {
    "PreCommit": [
      {
        "command": "grep -r 'console\\.log' src/ && echo 'ERROR: Remove console.log before committing' && exit 1 || exit 0"
      }
    ],
    "PostWrite": [
      {
        "command": "pnpm biome check --write $CLAUDE_FILE_PATH"
      }
    ]
  }
}

The PostWrite hook runs your formatter every time Claude writes a file. The PreCommit hook blocks commits that violate your rules. Claude cannot skip these — they are enforced by the system, not by the prompt.

Rule of thumb: If you have told Claude not to do something 3 times and it keeps doing it, move that rule from CLAUDE.md to a hook. Prompt instructions are for preferences. Hooks are for requirements.

Pattern 5: Scope Rules to Subdirectories

Most people put everything in the root CLAUDE.md. But Claude Code actually supports CLAUDE.md files at multiple levels:

~/.claude/CLAUDE.md          # Global (all projects)
project/CLAUDE.md             # Project-wide
project/src/CLAUDE.md         # Only for src/ files
project/src/api/CLAUDE.md     # Only for API route files

When Claude works on a file in src/api/, it loads all four files. When it works on a root config file, it only loads the first two.

This is powerful because it lets you keep each file short while still being specific:

# project/src/api/CLAUDE.md

API route rules:
- Always validate request body with zod schema before processing
- Return { data: T } on success, { error: string } on failure
- Log every request: logger.info(`[${method}] ${path}`, { userId })
- Rate limit: use `rateLimit()` middleware on all public endpoints

These 4 rules only matter for API routes. Putting them in the root CLAUDE.md would add noise for every other file Claude touches. Scoping them to src/api/ keeps the root file lean and gives Claude precise, relevant instructions exactly when they apply.

Putting It All Together

Here is the checklist:

Pattern	Action	Time to Implement
30-Line Rule	Audit your CLAUDE.md. Delete anything Claude can infer.	15 minutes
Positive Instructions	Rewrite every "don't" as a "do"	10 minutes
Anchor Critical Rules	Move your top 3 violated rules to line 1 and the last line	5 minutes
Hooks	Move hard requirements to PreCommit/PostWrite hooks	20 minutes
Subdirectory Scoping	Split domain-specific rules into scoped CLAUDE.md files	15 minutes

Total time: about an hour. The payoff is immediate — fewer corrections, less frustration, and a coding agent that actually does what you tell it.

The Bigger Lesson

The instinct when Claude ignores a rule is to add more rules. Longer explanations, more examples, bolder formatting. But the problem is almost never that Claude did not understand your rule — it is that your rule got lost in a sea of other rules.

Less rules, better structured, enforced by code where it matters. That is the entire strategy.

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.

5 Instruction File Patterns I Wish I Knew From Day 1

Shane Ho — Sat, 14 Mar 2026 13:17:53 +0000

You asked your AI to add an API route. It generated pages/api/users.ts with getServerSideProps. Your project uses Next.js 14 App Router. There is no pages/ directory.

You asked it to query the database. It wrote Prisma code. You migrated to Drizzle two months ago.

Sound familiar? The AI isn't broken. It just has zero context about your project — so it guesses based on the most common patterns in its training data.

The fix is deceptively simple: a project instruction file. In Claude Code, it's called CLAUDE.md. Cursor uses .cursorrules. GitHub Copilot has .github/copilot-instructions.md. The name doesn't matter -- the patterns do.

I've been refining my instruction files across a dozen projects over the past few months. Here are the 5 patterns that made the biggest difference -- each one with a before/after so you can steal them today.

Pattern 1: Project Conventions (Stop the Style Roulette)

Without explicit conventions, every AI response is a coin flip. camelCase or snake_case? Tabs or spaces? Named exports or default exports? The AI will happily switch between all of them within the same file.

The pattern: Declare your naming, file structure, and formatting rules upfront.

## Conventions
- Naming: kebab-case files, camelCase variables, PascalCase types
- Exports: named exports only, no default exports
- Formatting: Biome (not Prettier, not ESLint)
- Imports: absolute paths via @/ alias, no relative imports above 2 levels
- File size: max 200 lines per file, split if larger

Why it works: The AI treats these as hard constraints, not suggestions. Instead of guessing your style from the 3 files it can see, it follows explicit rules consistently. This alone eliminates ~70% of "fix the formatting" follow-up prompts.

Pro tip: Add a "Forbidden" section for things that keep creeping back in:

## Forbidden
- No `any` type (use `unknown` + type guard)
- No default exports
- No barrel files (index.ts re-exports)
- No console.log in committed code (use logger util)

The "never do X" framing is surprisingly effective. LLMs respond well to explicit prohibitions because it narrows the generation space -- fewer valid options means fewer wrong ones.

Pattern 2: Role Definition (Shape the AI's Decision-Making)

This is the pattern most people skip, and it's arguably the most powerful. You're not just telling the AI what to write -- you're telling it who to be when making decisions.

The pattern: Define the kind of developer you want the AI to emulate.

## Role
You are a senior backend engineer who:
- Prioritizes readability over cleverness
- Writes code that a junior dev can understand in 6 months
- Prefers boring, proven solutions over cutting-edge experiments
- Always handles errors explicitly (no silent catches)
- Treats tests as documentation, not afterthoughts

Before (no role):

// AI writes "clever" code
const getUser = (id: string) =>
  pipe(id, validateUUID, chain(fetchFromCache), alt(() => fetchFromDB), map(sanitize));

After (with role):

// AI writes readable code
async function getUser(id: string): Promise<User> {
  if (!isValidUUID(id)) {
    throw new InvalidIdError(id);
  }

  const cached = await cache.get<User>(`user:${id}`);
  if (cached) {
    return cached;
  }

  const user = await db.users.findById(id);
  if (!user) {
    throw new NotFoundError("User", id);
  }

  return user;
}

Same functionality. One version is a puzzle. The other one is code your team can actually maintain.

When to customize: Adjust the role based on the project. A quick prototype might want "move fast, skip edge cases." A payment system wants "paranoid about data integrity, validate everything twice." The role shapes every decision downstream.

Pattern 3: Anti-Patterns List (Prevent Recurring Mistakes)

Every codebase has landmines that AI assistants love stepping on. Maybe it's importing from an internal package that's being deprecated. Maybe it's using an ORM method that has a known N+1 problem. Maybe it's that one API endpoint that silently returns 200 even on failure.

The pattern: Document the specific mistakes you've seen the AI make (or that your team keeps making), and explicitly ban them.

## Anti-Patterns (NEVER do these)

### Database
- Never use `findAll()` without pagination — will OOM on large tables
- Never raw SQL without parameterized queries
- Never cascade delete without explicit confirmation in the code

### API Design
- Never return 200 for errors — use proper HTTP status codes
- Never expose internal IDs in public APIs — use UUIDs
- Never accept unbounded arrays in request bodies — always set maxItems

### State Management
- Never store derived state — compute it from source of truth
- Never mutate function arguments — clone first
- Never use global mutable state — pass dependencies explicitly

### Security
- Never log request bodies (may contain PII/tokens)
- Never hardcode secrets, even in tests — use env vars
- Never trust client-side validation alone — always validate server-side

Why this works better than you'd expect: LLMs are trained on millions of code examples, including a lot of bad ones. Without anti-pattern rules, the AI might generate code that's technically correct but uses patterns you've specifically moved away from. The anti-pattern list acts as a negative filter on its training data.

Real-world example: I had an AI repeatedly generate try { ... } catch (e) { return null; } -- silently swallowing errors. Adding Never silently catch errors — always log or re-throw with context to the anti-patterns section killed that behavior immediately.

Pattern 4: Stack-Specific Instructions (Framework-Aware Generation)

Generic AI coding advice is fine. But every framework has idioms, best practices, and sharp edges that only matter if you're actually using that framework. A React 18 project has completely different rules than a Svelte 5 project, even if they're both "frontend."

The pattern: Add framework-specific rules with version numbers.

## Tech Stack
- Runtime: Node.js 22 (use native fetch, no node-fetch)
- Framework: Next.js 14 App Router (NOT Pages Router)
- ORM: Drizzle (NOT Prisma — we migrated in Q3)
- Auth: Better Auth v1 (NOT NextAuth)
- Styling: Tailwind CSS v4 (use @theme, not tailwind.config)
- Testing: Vitest + React Testing Library

## Framework Rules
### Next.js 14
- Use Server Components by default, add "use client" only when needed
- Data fetching in Server Components, not useEffect
- Use `next/navigation` not `next/router`
- Route handlers go in app/api/[route]/route.ts
- Loading states via loading.tsx, not manual Suspense

### Drizzle ORM
- Schema defined in src/db/schema.ts
- Migrations via `drizzle-kit generate` then `drizzle-kit migrate`
- Always use `.prepare()` for frequently-run queries
- Relations defined with `relations()` helper, not raw joins

Why version numbers matter: AI models are trained on code from multiple framework versions. Without Next.js 14 App Router, you'll get a mix of Pages Router (v12) and App Router (v13/14) patterns in the same file. Specifying the version eliminates an entire class of hallucination.

The "NOT X" pattern is gold for recent migrations. If you switched from Prisma to Drizzle last quarter, the AI will still reach for Prisma unless you explicitly block it. (NOT Prisma — we migrated in Q3) gives it both the rule and the reason.

Pattern 5: Testing & Deployment Constraints (Guard the Pipeline)

The AI can write beautiful code that breaks your CI pipeline, fails your linter, or deploys to the wrong environment. This pattern makes the AI aware of your operational constraints.

The pattern: Document your testing expectations and deployment boundaries.

## Testing Requirements
- Every new function needs at least one test
- Test file location: colocated (Button.test.tsx next to Button.tsx)
- Test naming: `describe("functionName")` → `it("should [expected behavior] when [condition]")`
- Mocking: use vi.mock() for external services, never mock internal utils
- No snapshot tests — they break on every UI change and nobody reviews the diff
- Integration tests for API routes, unit tests for utils/helpers

## Deployment
- Platform: Cloudflare Workers (no Node.js APIs: no fs, no path, no process)
- Max bundle size: 1MB (Workers limit)
- Environment variables: accessed via env parameter, NOT process.env
- Database: D1 (SQLite-compatible, no PostgreSQL features)
- Secrets: `wrangler secret put`, never commit .env files

## CI Pipeline
- Biome check runs first — fix formatting before pushing
- Type check: `tsc --noEmit` must pass
- Tests: `vitest run` must pass, no `.skip` or `.only` in committed code
- PR titles: Conventional Commits format (feat:, fix:, refactor:, etc.)

The Cloudflare Workers example is instructive. Without these constraints, an AI will happily write import fs from 'node:fs' in a Workers project. It'll compile locally and explode in production. The constraint no Node.js APIs: no fs, no path, no process prevents this at generation time instead of debug time.

The "no snapshot tests" rule is a taste decision, not a universal truth. That's fine -- your instruction file should encode your team's decisions, not generic best practices. The AI doesn't need to agree. It needs to comply.

Putting It All Together

Here's a minimal but complete instruction file combining all 5 patterns:

# Project Instructions

## Role
Senior TypeScript developer. Prioritize readability and maintainability.
Prefer explicit over clever. Handle all errors.

## Conventions
- Files: kebab-case | Variables: camelCase | Types: PascalCase
- Named exports only. No barrel files.
- Max 200 lines per file.

## Stack
- Next.js 14 App Router + TypeScript strict mode
- Drizzle ORM + D1 database
- Tailwind CSS v4
- Vitest + React Testing Library

## Anti-Patterns
- No `any` type — use `unknown` with type guards
- No `try/catch` that silently returns null
- No `useEffect` for data fetching — use Server Components
- No `process.env` — use Workers env parameter

## Testing
- Colocate test files. Name: `describe → it("should X when Y")`
- Unit tests for logic, integration tests for API routes
- No `.skip` or `.only` in committed code

## Deployment
- Cloudflare Workers — no Node.js APIs (fs, path, process)
- Max 1MB bundle. Secrets via `wrangler secret put`.

That's ~30 lines. It takes 10 minutes to write. And it fundamentally changes the quality of every AI-generated code block in your project.

Your Next Steps

Create the file right now. Open your project, add CLAUDE.md (or .cursorrules or whatever your tool uses), and write just the Conventions section. Even 5 lines make a difference.
Build it iteratively. Every time the AI generates something wrong, don't just fix it -- add a rule to prevent it. Your instruction file is a living document that gets smarter over time.
Share it with your team. Commit it to the repo. Now every developer gets the same AI behavior, not just whoever happens to know the right prompt.

Your instruction file should be personal — built from your own mistakes and preferences. Start with 5 lines today, and add a rule every time the AI gets something wrong. In a week, you'll have a file that feels like a cheat code.

Want a head start? I maintain a collection of 12 starter templates for different project types (Next.js, FastAPI, Rust, Go, etc). Steal what's useful, delete the rest.

What's the ONE rule in your instruction file that saved you the most debugging time? Mine is Never silently catch errors — always log or re-throw with context. That single line eliminated a whole category of "why is this returning null?" bugs.

Drop yours in the comments — I reply to every one.

Follow @docat0209 for weekly deep-dives on AI-assisted development.

3 Patterns That Fix LLM API Calling — Stop Getting Hallucinated Parameters

Shane Ho — Sat, 14 Mar 2026 06:49:32 +0000

You gave your LLM a tool. It called it wrong. Again. Maybe it hallucinated a parameter name, nested an object three levels deep when the API expected a flat string, or cheerfully returned a 200 OK summary of a response that was actually a 422 validation error.

If you've wired up any LLM to a real API -- OpenAI function calling, Anthropic tool use, MCP servers, LangChain agents -- you've hit this wall. The good news: most failures follow predictable patterns, and there are concrete fixes.

Here are three patterns I use to take LLM tool calling from "works 60% of the time" to "works 95%+ of the time."

Why LLMs Fumble API Calls

Before the patterns, a quick mental model of why this happens.

LLMs generate tokens left-to-right. When your tool schema looks like this:

{
  "body": {
    "type": "object",
    "properties": {
      "user": {
        "type": "object",
        "properties": {
          "address": {
            "type": "object",
            "properties": {
              "street": { "type": "string" },
              "city": { "type": "string" },
              "geo": {
                "type": "object",
                "properties": {
                  "lat": { "type": "number" },
                  "lng": { "type": "number" }
                }
              }
            }
          }
        }
      }
    }
  }
}

The model has to hold 4 levels of nesting in its attention window while deciding what to generate next. Each nested brace is a point where it can lose track of which object it's inside. It's like asking someone to write valid JSON by hand, blindfolded, one character at a time.

Three things go wrong most often:

Hallucinated keys -- the model invents parameter names that sound right but don't exist in the schema
Wrong nesting -- values end up at the wrong depth (city inside geo instead of address)
Dropped required fields -- deep-nested required params get silently skipped

Let's fix each one.

Pattern 1: Flatten Your Parameter Schemas

This is the single highest-impact change you can make. Instead of handing the LLM a nested object tree, flatten it into a single-depth key-value map using dot-notation or underscore-delimited keys.

Before (nested):

{
  "body": {
    "type": "object",
    "properties": {
      "user": {
        "type": "object",
        "properties": {
          "name": { "type": "string" },
          "address": {
            "type": "object",
            "properties": {
              "city": { "type": "string" },
              "zip": { "type": "string" }
            }
          }
        }
      }
    }
  }
}

After (flat):

{
  "body__user__name": { "type": "string" },
  "body__user__address__city": { "type": "string" },
  "body__user__address__zip": { "type": "string" }
}

The LLM now sees a simple list of key-value pairs. No nesting to track, no braces to match. Your middleware reconstructs the nested structure before sending to the actual API.

Here's the transform in Python:

def flatten_schema(schema, prefix="", separator="__"):
    """Flatten a nested JSON schema into dot-notation keys."""
    flat = {}

    if schema.get("type") == "object" and "properties" in schema:
        for key, value in schema["properties"].items():
            new_prefix = f"{prefix}{separator}{key}" if prefix else key
            if value.get("type") == "object" and "properties" in value:
                flat.update(flatten_schema(value, new_prefix, separator))
            else:
                flat[new_prefix] = value
    else:
        flat[prefix] = schema

    return flat


def unflatten_params(flat_params, separator="__"):
    """Reconstruct nested dict from flat keys before sending to API."""
    nested = {}

    for key, value in flat_params.items():
        parts = key.split(separator)
        current = nested
        for part in parts[:-1]:
            current = current.setdefault(part, {})
        current[parts[-1]] = value

    return nested

Usage in your tool-calling middleware:

# 1. Flatten the schema before registering the tool
original_schema = load_openapi_spec("petstore.yaml")
flat_schema = flatten_schema(original_schema["requestBody"])

# 2. Register tool with flat schema
register_tool("create_user", parameters=flat_schema)

# 3. When the LLM calls the tool, unflatten before forwarding
def handle_tool_call(name, flat_args):
    nested_args = unflatten_params(flat_args)
    return call_api(name, nested_args)

Why this works: every parameter is now a single decision point. The model picks a key, picks a value, moves on. No state tracking across nesting levels. In my testing, this alone cuts parameter hallucination by roughly 40-60% on complex APIs.

Pattern 2: Truncate API Responses Intelligently

The second failure mode isn't about calling the API -- it's about what happens after. Your LLM calls a list endpoint and gets back 500 objects, each with 30 fields, nested 4 levels deep. That's easily 50,000+ tokens of raw JSON crammed into the context window.

The LLM either:

Chokes and produces garbage
Summarizes incorrectly ("The API returned 3 users" when it returned 500)
Blows your token budget in one call

Smart truncation solves this by applying two rules:

Rule 1: Slice arrays to a reasonable size

If the response is an array of 500 items, the LLM almost never needs all 500. Slice it and tell the model what happened.

def truncate_arrays(data, max_items=20):
    """Slice arrays and add a count hint for the LLM."""
    if isinstance(data, list):
        total = len(data)
        sliced = [truncate_arrays(item, max_items) for item in data[:max_items]]
        if total > max_items:
            sliced.append(f"... ({total - max_items} more items, {total} total)")
        return sliced

    if isinstance(data, dict):
        return {k: truncate_arrays(v, max_items) for k, v in data.items()}

    return data

Rule 2: Limit nesting depth

Deep nesting past 4-5 levels rarely carries information the LLM needs for its next decision. Collapse it.

def limit_depth(data, max_depth=5, current_depth=0):
    """Replace deeply nested structures with a type hint."""
    if current_depth >= max_depth:
        if isinstance(data, dict):
            return f"{{...}} ({len(data)} keys)"
        if isinstance(data, list):
            return f"[...] ({len(data)} items)"
        return data

    if isinstance(data, dict):
        return {
            k: limit_depth(v, max_depth, current_depth + 1)
            for k, v in data.items()
        }

    if isinstance(data, list):
        return [limit_depth(item, max_depth, current_depth + 1) for item in data]

    return data

Combine both:

def smart_truncate(response_data, max_items=20, max_depth=5):
    """Apply both truncation strategies."""
    truncated = truncate_arrays(response_data, max_items)
    truncated = limit_depth(truncated, max_depth)
    return truncated

The key insight: you're not losing information. You're giving the LLM a useful summary instead of a data dump. The model can still see the structure, the first N items, and the total count. If it needs item #47, it can ask for a filtered query.

Pattern 3: Generate Tool Definitions from API Specs

The most error-prone step in LLM tool calling is one most people do by hand: writing the tool definition itself.

You read the API docs, you write a JSON schema, you describe each parameter. And you get it subtly wrong -- a typo in a field name, a missing enum value, a required field marked as optional. Now the LLM is working from a broken map, and no amount of prompt engineering will save it.

The fix: don't write tool definitions by hand. Generate them directly from the API's OpenAPI (Swagger) spec.

Most APIs already have one. If they don't, you can usually generate a rough spec from their docs in minutes.

import yaml

def generate_tools_from_spec(spec_path):
    """Generate flat tool definitions from an OpenAPI spec."""
    with open(spec_path) as f:
        spec = yaml.safe_load(f)

    tools = []
    for path, methods in spec.get("paths", {}).items():
        for method, operation in methods.items():
            if method not in ("get", "post", "put", "patch", "delete"):
                continue

            # Collect parameters from path, query, header, and body
            params = {}
            for param in operation.get("parameters", []):
                params[param["name"]] = {
                    "type": param.get("schema", {}).get("type", "string"),
                    "description": param.get("description", ""),
                    "required": param.get("required", False),
                }

            # Flatten request body schema if present
            body_schema = (
                operation.get("requestBody", {})
                .get("content", {})
                .get("application/json", {})
                .get("schema", {})
            )
            if body_schema:
                flat_body = flatten_schema(body_schema, prefix="body")
                params.update(flat_body)

            tools.append({
                "name": operation.get("operationId", f"{method}_{path}"),
                "description": operation.get("summary", ""),
                "parameters": params,
            })

    return tools

This approach gives you:

Accurate field names -- straight from the spec, no typos
Complete enum values -- the LLM sees every valid option
Correct required/optional markers -- no more dropped fields
Automatic updates -- re-generate when the API version bumps

I automated this exact pipeline into an open-source tool called mcp-openapi that converts any OpenAPI spec into working MCP tools with flattening and truncation built in -- but the pattern works regardless of what framework you're using.

Your Next Steps

You don't need to implement all three patterns at once. Here's a prioritized action plan:

Start with flattening (Pattern 1). Pick your most-used API integration, flatten its schema, and measure the before/after accuracy. You'll likely see an immediate improvement.
Add response truncation (Pattern 2) for any endpoint that returns lists or deeply nested objects. Start with max_items=20 and max_depth=5 -- you can tune from there.
Automate tool generation (Pattern 3) once you're integrating more than 2-3 APIs. The upfront cost pays for itself fast in fewer bugs and easier maintenance.

The underlying principle behind all three patterns is the same: reduce ambiguity for the model. Flat schemas reduce structural ambiguity. Truncation reduces information overload. Spec-driven generation reduces human error. Stack all three and your LLM tool calling becomes boring and reliable -- which is exactly what you want in production.

What's the weirdest LLM tool-calling failure you've seen? I once watched GPT-4 confidently pass "latitude": "yes" to a geocoding API. Drop your best horror stories in the comments -- I reply to every one.

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.

I Built 13 Free Browser Tools in a Weekend — Here's What I Learned

Shane Ho — Fri, 13 Mar 2026 11:52:52 +0000

Last month I launched FreeTools — a collection of free, browser-based utilities. It started with 6 tools. Now it's 13 and growing.

The twist? Zero backend. Zero frameworks. Zero build step. Just HTML, Tailwind CDN, and vanilla JavaScript.

Here's what I built and what I learned.

The Tool Suite

Tool	What it does
Password Generator	Cryptographically secure passwords via `crypto.getRandomValues()`
Invoice Generator	Professional invoices with live preview + PDF download
Privacy Policy Generator	GDPR-compliant policies from a questionnaire
JSON Formatter	Format, validate, minify with syntax highlighting
QR Code Generator	URLs, text, WiFi, email, phone, SMS
Markdown Preview	Live GFM editor with side-by-side preview
Color Palette Generator	Harmonious schemes from any base color
Base64 Encoder/Decoder	Text and file encoding/decoding
Word Counter	Words, characters, sentences, reading time
CSV to JSON	Auto-detect delimiters, instant conversion
Lorem Ipsum Generator	Paragraphs, sentences, or words — classic start option
Hash Generator	MD5, SHA-1, SHA-256, SHA-512 with hash comparison
UUID Generator	v4 (random) and v7 (time-ordered) with bulk generation

Every tool is a single index.html file. No dependencies except Tailwind CDN.

Why No Framework?

I know this sounds controversial in 2026, but here's my reasoning:

1. Zero build = zero maintenance

No node_modules. No package.json. No Webpack/Vite/Turbopack config. No dependency updates. No security advisories for 847 transitive dependencies.

Each tool is a self-contained HTML file. Deploy to GitHub Pages and forget it.

2. Instant load times

Each tool page is under 30KB. No JavaScript bundle to parse. No hydration step. The page is interactive the moment HTML parsing completes.

3. Truly private

"100% client-side" isn't just a tagline — there's literally no server to send data to. GitHub Pages serves static files. That's it.

Technical Highlights

Hash Generator: Web Crypto API + Pure JS MD5

The SHA family (SHA-1, SHA-256, SHA-512) uses the Web Crypto API:

async function sha(algo, text) {
  const data = new TextEncoder().encode(text);
  const hashBuffer = await crypto.subtle.digest(algo, data);
  return Array.from(new Uint8Array(hashBuffer))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

But Web Crypto doesn't support MD5 (it's considered insecure). So I included a pure JS MD5 implementation for checksum use cases. All hashing happens in real-time as you type.

The tool also includes a hash comparison feature — paste an expected hash and it instantly tells you which algorithm matches.

UUID Generator: v4 and v7

UUID v4 is straightforward — fill 16 random bytes, set the version and variant bits:

function generateUuidV4() {
  const bytes = new Uint8Array(16);
  crypto.getRandomValues(bytes);
  bytes[6] = (bytes[6] & 0x0f) | 0x40; // version 4
  bytes[8] = (bytes[8] & 0x3f) | 0x80; // variant 1
  return formatBytes(bytes);
}

UUID v7 (RFC 9562) is more interesting — it embeds a Unix timestamp in the first 48 bits, making UUIDs naturally sortable by creation time. This is huge for database primary keys:

function generateUuidV7() {
  const bytes = new Uint8Array(16);
  crypto.getRandomValues(bytes);
  const now = Date.now();
  // Embed timestamp in first 6 bytes
  bytes[0] = (now / 2**40) & 0xff;
  bytes[1] = (now / 2**32) & 0xff;
  // ... remaining timestamp bytes
  bytes[6] = (bytes[6] & 0x0f) | 0x70; // version 7
  bytes[8] = (bytes[8] & 0x3f) | 0x80; // variant 1
  return formatBytes(bytes);
}

The Pattern: One Directory, One File

Every tool follows the exact same structure:

tool-name/
└── index.html    ← everything lives here

Nav with breadcrumb → Main content → SEO section → Footer → <script> tag.

No shared components. No imports. Each tool is completely independent. You can literally copy one index.html to create a new tool.

Is it DRY? No. Is it maintainable? Absolutely. Each tool is ~150-300 lines. If I need to update the footer, a quick find-and-replace across 13 files takes 10 seconds.

What I'd Do Differently

Add <meta> structured data — I should add JSON-LD markup for each tool to improve search appearance.

Use Web Components — For the shared nav/footer, a lightweight custom element would reduce duplication without adding a build step.

Add offline support — A service worker would make these tools work without internet. Since there's no backend, it's trivial.

The SEO Play

Each tool targets a specific search query:

"free password generator online"
"uuid generator v7"
"sha256 hash online"
"lorem ipsum generator"
"csv to json converter"

These are high-volume, low-competition queries. The tools are genuinely useful (not just SEO bait), so they should rank well over time.

Try It

👉 FreeTools — 13 Free Browser Tools

All source code is on GitHub. MIT licensed.

Building AI-powered tools? Check out mcp-openapi (turn any REST API into AI-callable tools) and graphql-to-mcp (same for GraphQL).

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.

I Made Claude Talk to Any GraphQL API — Zero Code, One Command

Shane Ho — Thu, 12 Mar 2026 11:16:04 +0000

You have a GraphQL API. You want Claude (or any AI) to use it. The usual path: write wrapper functions, map types, handle auth, deal with pagination...

Or: one command.

npx graphql-to-mcp https://countries.trevorblades.com/graphql

That's it. Every query and mutation becomes an MCP tool. Claude can now search countries, filter by continent, look up languages — all from the schema it just introspected.

The Problem

GraphQL is great for developers. Typed schemas, introspection, nested queries — powerful stuff.

But LLMs hate nested input objects. Give Claude a mutation like this:

mutation CreateUser($input: CreateUserInput!) {
  createUser(input: $input) { id name }
}

Where CreateUserInput is:

input CreateUserInput {
  name: String!
  email: String!
  address: AddressInput
}

input AddressInput {
  street: String
  city: String
  country: String
}

The LLM has to construct:

{
  "input": {
    "name": "Alice",
    "email": "alice@example.com",
    "address": {
      "street": "123 Main St",
      "city": "Portland",
      "country": "US"
    }
  }
}

This fails constantly. LLMs hallucinate extra nesting, miss required fields, or get the structure wrong. Deeply nested JSON is the #1 cause of tool-calling failures.

The Solution: Flat Schemas

graphql-to-mcp flattens every InputObject into simple key-value parameters:

Parameter	Type	Required
`input_name`	string	yes
`input_email`	string	yes
`input_address_street`	string	no
`input_address_city`	string	no
`input_address_country`	string	no

The LLM fills flat fields. The tool reconstructs the nested GraphQL variables automatically.

Result: near-perfect accuracy, even with complex mutations.

Setup in 30 Seconds

Claude Desktop / Cursor

Add to your MCP config:

{
  "mcpServers": {
    "my-api": {
      "command": "npx",
      "args": ["-y", "graphql-to-mcp", "https://your-api.com/graphql"]
    }
  }
}

With Auth

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": [
        "-y", "graphql-to-mcp",
        "https://api.github.com/graphql",
        "--bearer", "ghp_your_token",
        "--prefix", "github"
      ]
    }
  }
}

Filter Operations

Only expose what you need:

{
  "args": [
    "-y", "graphql-to-mcp",
    "https://api.example.com/graphql",
    "--include", "get*",
    "--exclude", "internal*"
  ]
}

What Happens Under the Hood

Introspect — Fetches the full GraphQL schema via introspection query
Flatten — Every InputObject type gets flattened into simple parameters
Generate — Each query/mutation becomes an MCP tool with a JSON Schema
Execute — When the LLM calls a tool, flat args → nested GraphQL variables → POST to your endpoint
Truncate — Large responses get smart-truncated (arrays sliced to 20, depth pruned to 5 levels)

Smart Truncation

GraphQL can return massive payloads. A listUsers with 10K results would blow up any LLM context window.

graphql-to-mcp handles this automatically:

Arrays → sliced to 20 items, with _meta: "… 9980 more items (10000 total)"
Deep nesting → pruned at 5 levels, with "[Object(12 keys)]" summaries
Hard safety net at 50K characters

No configuration needed. The LLM gets enough data to understand the response without drowning in tokens.

Built-in Resilience

Retry logic — automatic retries on 429/5xx with exponential backoff
Timeout — configurable request timeout (default 30s)
Partial errors — GraphQL partial errors are surfaced alongside data

Why Not Just Use [existing-tool]?

There are a few GraphQL MCP tools out there. Here's why this one is different:

Feature	graphql-to-mcp	Others
Flat schemas	Yes — nested inputs flattened	No — raw nested JSON
Auto-introspection	Yes	Some
Smart truncation	Yes — array + depth pruning	No — raw or hard cut
Include/exclude filters	Yes — glob patterns	Rare
Retry on 429/5xx	Yes — exponential backoff	No
Zero config	Yes — just an endpoint URL	Most need config

Try It Now

npx graphql-to-mcp https://countries.trevorblades.com/graphql

Then ask Claude: "What countries are in Europe?"

It just works.

Links:

npm: graphql-to-mcp
GitHub: Docat0209/mcp-graphql
Also: mcp-openapi — same approach for REST/OpenAPI APIs

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.

Why LLMs Suck at Calling APIs — And How Flat Schemas Fix It

Shane Ho — Wed, 11 Mar 2026 23:23:17 +0000

Every AI tool-calling framework has the same dirty secret: LLMs are terrible at constructing nested JSON.

If you've built MCP servers, OpenAI function calls, or any tool-use integration, you've seen it. The model hallucinates field names, forgets closing braces, puts arrays where objects should be, and confidently generates invalid payloads.

The fix is surprisingly simple: flatten your schemas.

The Problem

Consider a typical REST API endpoint:

POST /api/orders
Content-Type: application/json

{
  "customer": {
    "name": "Alice",
    "address": {
      "street": "123 Main St",
      "city": "Portland",
      "state": "OR",
      "zip": "97201"
    }
  },
  "items": [
    { "sku": "WIDGET-1", "quantity": 2 }
  ],
  "shipping": {
    "method": "express",
    "instructions": "Leave at door"
  }
}

When you expose this as an MCP tool with the full nested schema, you're asking the LLM to:

Remember a 3-level deep JSON structure
Keep track of which braces belong to which object
Correctly nest address inside customer, not at the root
Generate a valid array of objects for items
Get all of this right in a single generation pass with no backtracking

The success rate drops with every level of nesting. In our testing, a 3-level nested schema produces malformed tool calls 15-25% of the time. At 4+ levels, it's worse.

Why Nesting Fails

LLMs generate tokens left-to-right. They don't have a "syntax checker" running in parallel — they predict the next token based on context.

When generating nested JSON:

{
  "customer": {
    "name": "Alice",
    "address": {
      "street": "123 Main St",

By this point, the model is tracking three open braces and needs to remember the exact structure to close them correctly. Each additional level of nesting adds cognitive load to the generation.

Common failure modes:

Premature closing: "street": "123 Main St" } } — closes address and customer too early
Wrong level: Puts city at the customer level instead of inside address
Missing objects: Omits the address wrapper entirely, putting street/city at the customer level
Array confusion: Generates "items": { "sku": "..." } instead of an array

These aren't bugs in the model. They're a natural consequence of autoregressive generation on complex structures.

The Solution: Flat Schemas

Instead of exposing the nested schema, flatten it:

{
  "customer_name": { "type": "string", "description": "Customer name" },
  "customer_address_street": { "type": "string" },
  "customer_address_city": { "type": "string" },
  "customer_address_state": { "type": "string" },
  "customer_address_zip": { "type": "string" },
  "items_0_sku": { "type": "string", "description": "First item SKU" },
  "items_0_quantity": { "type": "integer" },
  "shipping_method": { "type": "string", "enum": ["standard", "express"] },
  "shipping_instructions": { "type": "string" }
}

The LLM now generates a simple key-value map:

{
  "customer_name": "Alice",
  "customer_address_street": "123 Main St",
  "customer_address_city": "Portland",
  "customer_address_state": "OR",
  "customer_address_zip": "97201",
  "items_0_sku": "WIDGET-1",
  "items_0_quantity": 2,
  "shipping_method": "express",
  "shipping_instructions": "Leave at door"
}

No nesting. No ambiguity. The tool's executor reconstructs the proper nested JSON before sending the HTTP request.

Results:

Tool-call accuracy: ~95%+ (up from ~75-85% with nested schemas)
Average retries per call: 0.1 (down from 0.5-1.0)
Token usage: roughly the same (flat keys are longer, but fewer retries)

When NOT to Flatten

Flattening isn't always the right choice:

Very simple schemas (1 level deep, < 5 fields) — nesting is fine, no accuracy impact
User-facing tools where the schema IS the API — users expect the real structure
Dynamic/recursive schemas — trees, linked lists, etc. can't be statically flattened

For the 80% of cases where you're wrapping a REST API for LLM consumption, flatten everything.

Implementing This in MCP

If you're building MCP tools by hand, here's a minimal flattener:

function flattenSchema(
  schema: Record<string, any>,
  prefix = "",
  result: Record<string, any> = {}
): Record<string, any> {
  for (const [key, value] of Object.entries(schema.properties ?? {})) {
    const fullKey = prefix ? `${prefix}_${key}` : key;

    if (value.type === "object" && value.properties) {
      flattenSchema(value, fullKey, result);
    } else {
      result[fullKey] = { ...value };
    }
  }
  return { type: "object", properties: result };
}

Then in your tool executor, unflatten the args back into the nested structure before making the API call.

Or skip all of this and use mcp-openapi, which does it automatically:

npx mcp-openapi --spec https://your-api.com/openapi.json

It reads your OpenAPI spec, flattens all parameter schemas, generates MCP tools, and handles the unflatten→HTTP request mapping internally. Zero config.

Response Side: Smart Truncation

The other half of the problem is responses. Large API responses (paginated lists, deeply nested objects) can blow up the LLM's context window or get hard-truncated mid-JSON, leaving the model confused.

Smart truncation strategies:

1. Array slicing — Show first N items + metadata:

[
  { "id": 1, "name": "Widget A" },
  { "id": 2, "name": "Widget B" },
  { "_meta": "showing 2 of 847 items. Use offset/limit to paginate." }
]

2. Depth pruning — Summarize beyond a certain depth:

{
  "user": {
    "name": "Alice",
    "orders": "[array(23)]",
    "preferences": "[object(8 keys)]"
  }
}

3. Field filtering — Use JMESPath to extract only relevant fields:

items[].{id: id, name: name, price: price}

These techniques preserve the shape of the data while keeping it within context limits. The LLM can still understand the structure and ask for more details if needed.

Key Takeaways

LLMs fail on nested JSON — 15-25% error rate at 3+ levels of nesting
Flat schemas fix this — simple key-value pairs reduce errors to < 5%
The tool executor handles reconstruction — flatten for the LLM, unflatten for the API
Smart truncation > hard truncation — preserve data structure, not just characters
For OpenAPI specs, automate it — tools like mcp-openapi handle this end-to-end

If you're building MCP servers or any LLM tool integration, try flattening your schemas. The accuracy improvement is immediate and dramatic.

Star the repo: github.com/Docat0209/mcp-openapi

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.

I Made Claude Call Any REST API in 30 Seconds — Zero Code, Just One Command

Shane Ho — Wed, 11 Mar 2026 13:24:01 +0000

You have an OpenAPI spec. You want Claude to call your API. The "standard" approach? Write a custom MCP server, define every tool by hand, wire up auth, handle errors, deploy it.

Or just run one command:

npx mcp-openapi --spec https://petstore3.swagger.io/api/v3/openapi.json

Done. Every endpoint in your spec is now an MCP tool. Claude can call them all.

No code generation. No config files. No boilerplate.

The Problem: MCP Servers Are Tedious to Build

Model Context Protocol (MCP) is the open standard that lets AI assistants like Claude call external tools. It's powerful — but building an MCP server for your existing REST API means:

Reading your OpenAPI spec manually
Writing a tool definition for each endpoint
Mapping parameters (path, query, headers, body) correctly
Handling authentication
Adding error handling, retries, response formatting
Deploying and maintaining it

For an API with 50 endpoints, that's days of repetitive glue code. And every time the API changes, you update the MCP server too.

What if the spec itself is the server?

How mcp-openapi Works

mcp-openapi reads your OpenAPI 3.x or Swagger 2.0 spec and generates MCP tools at runtime. No build step. No generated files.

OpenAPI Spec  →  mcp-openapi  →  MCP Tools  →  Claude Desktop
  (URL/file)      (runtime)      (auto-generated)   (calls your API)

Each endpoint becomes one tool:

Name is derived from operationId (e.g., listPets, createUser)
Description comes from the spec's summary/description
Parameters are flattened into a single schema (more on this below)
Responses are automatically truncated to fit LLM context windows

Add It to Claude Desktop

Edit your claude_desktop_config.json:

{
  "mcpServers": {
    "petstore": {
      "command": "npx",
      "args": [
        "mcp-openapi",
        "--spec", "https://petstore3.swagger.io/api/v3/openapi.json"
      ]
    }
  }
}

Restart Claude Desktop. Ask "List all available pets" — Claude calls find_pets_by_status automatically.

Why Flat Parameter Schemas Matter

Here's something most OpenAPI-to-MCP converters get wrong: they pass nested JSON objects to the LLM.

Consider an endpoint like POST /users with a JSON body. A naive converter gives the LLM this input schema:

{
  "body": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "email": { "type": "string" },
      "address": {
        "type": "object",
        "properties": {
          "street": { "type": "string" },
          "city": { "type": "string" }
        }
      }
    }
  }
}

LLMs frequently mess up nested structures — missing braces, wrong nesting levels, hallucinated fields. The deeper the nesting, the worse the accuracy.

mcp-openapi flattens everything into a single level:

{
  "name": { "type": "string" },
  "email": { "type": "string" },
  "address_street": { "type": "string" },
  "address_city": { "type": "string" }
}

Path parameters, query parameters, headers, and body fields all live in one flat object. The tool reconstructs the proper HTTP request internally. The LLM only sees simple key-value pairs.

This isn't just a nice-to-have. Flat schemas dramatically improve tool-calling accuracy — fewer malformed calls, fewer retries, better user experience.

Authentication Built In

Most real APIs require auth. mcp-openapi supports three methods out of the box:

Bearer Token

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": [
        "mcp-openapi",
        "--spec", "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
        "--auth-type", "bearer",
        "--auth-token", "$GITHUB_TOKEN",
        "--prefix", "github",
        "--include", "listReposForAuthenticatedUser,getRepo,listIssues"
      ],
      "env": {
        "GITHUB_TOKEN": "ghp_your_token_here"
      }
    }
  }
}

API Key

npx mcp-openapi \
  --spec https://api.example.com/openapi.json \
  --auth-type api-key \
  --auth-name X-API-Key \
  --auth-value '$MY_API_KEY' \
  --auth-in header

OAuth2 Client Credentials

npx mcp-openapi \
  --spec https://api.example.com/openapi.json \
  --auth-type oauth2 \
  --auth-client-id $CLIENT_ID \
  --auth-client-secret $CLIENT_SECRET \
  --auth-token-url https://auth.example.com/token

Note the $ENV_VAR syntax — tokens are never hardcoded.

Filter Endpoints

Large APIs can have hundreds of endpoints. You don't want Claude to see all of them. Use --include or --exclude to control which endpoints become tools:

# Only expose specific operations
npx mcp-openapi --spec ./api.json --include 'listUsers,getUser,createUser'

# Exclude destructive operations
npx mcp-openapi --spec ./api.json --exclude 'deleteUser,dropDatabase'

# Add a prefix to avoid naming conflicts
npx mcp-openapi --spec ./api.json --prefix myapi

With --prefix myapi, tools become myapi_list_users, myapi_get_user, etc. — useful when connecting multiple APIs to the same Claude session.

Real-World Example: GitHub API

Let's connect Claude to the GitHub API with read-only access:

1. Get a GitHub token with repo scope from github.com/settings/tokens

2. Add to Claude Desktop config:

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": [
        "mcp-openapi",
        "--spec", "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
        "--auth-type", "bearer",
        "--auth-token", "$GITHUB_TOKEN",
        "--prefix", "gh",
        "--include", "listReposForAuthenticatedUser,getRepo,listIssues,createIssue,listPullRequestsForRepo"
      ],
      "env": {
        "GITHUB_TOKEN": "ghp_your_token"
      }
    }
  }
}

3. Ask Claude:

"List my GitHub repos sorted by recent activity, then show open issues for the most active one."

Claude calls gh_list_repos_for_authenticated_user, picks the top repo, then calls gh_list_issues — all from the same conversation.

Comparison with Alternatives

Feature	mcp-openapi	openapi-mcp-generator	Hand-written servers
Zero config (no build step)	Yes	No (generates code)	No
Swagger 2.0 support	Yes	No	Manual
Flat parameter schemas	Yes	No	Manual
Built-in auth (Bearer/API key/OAuth2)	Yes	Partial	DIY
Auto-retry with backoff	Yes	No	DIY
Response truncation	Yes	No	DIY
Endpoint filtering	Yes	Yes	Manual
Runtime overhead	Minimal	None (pre-generated)	None

When to use code generation instead: If you need to customize tool behavior per-endpoint (custom validation, response transforms, side effects), a code-generated approach gives you more control. mcp-openapi is for the 80% case where you just want your API callable.

Getting Started

Install and run:

npx mcp-openapi --spec <your-spec-url-or-path>

Or add to any MCP client config:

{
  "mcpServers": {
    "my-api": {
      "command": "npx",
      "args": ["mcp-openapi", "--spec", "https://your-api.com/openapi.json"]
    }
  }
}

Supports:

OpenAPI 3.0.x, 3.1.x
Swagger 2.0
JSON and YAML specs
Remote URLs and local files
Node.js 18+

What's Next

We're working on Pro features for teams and enterprise use cases:

Multi-API composition — connect multiple specs into one MCP session with intelligent routing
Context window management — smart pagination and response streaming for large API responses
Usage analytics — track which tools Claude calls most, optimize your API accordingly
Custom response transforms — shape API responses before they reach the LLM

Star the repo if this is useful: github.com/Docat0209/mcp-openapi

mcp-openapi is open source (MIT). Contributions welcome.

If this helped, follow @docat0209 — I write weekly about AI-assisted development and developer tools.