Forem: yunbow

Why AI Doesn't Code What You Designed: The Structural Gap Between Specs and Implementation

yunbow — Tue, 05 May 2026 06:29:37 +0000

You write a detailed design doc. You paste it into your AI assistant. You wait.

The output compiles. Tests pass. And yet — it's not quite what you designed. The auth middleware is in the wrong layer. The error handling pattern differs from the rest of the codebase. The field names don't match the schema.

You fix it. Next task, same thing.

This happens constantly, and it's not a model capability problem. It's a structural problem in how we communicate design intent to AI. And "write better prompts" is not the solution — it's a band-aid.

Why "Better Prompts" Doesn't Scale

The instinct when AI misses the mark: add more detail to the prompt. Describe the pattern more explicitly. Give an example.

This works — sometimes, for that one task.

The problem: design intent isn't a single instruction. It's institutional knowledge accumulated over months. It's the reason two experienced engineers on your team write recognizably similar code without coordinating on every PR. It's the unspoken answer to "what does good look like here?"

You can't fit that into a prompt. And even if you could, you'd need to paste it every single time.

Think about onboarding a new engineer. You don't brief them on your entire codebase in the first conversation. You give them documentation, you pair with them, you let them absorb context over weeks. The conventions transfer gradually because there's a persistent learning mechanism.

With AI, there's no persistent learning mechanism by default. Every conversation starts from scratch. Your prompt is one-time context. Your design intent is institutional knowledge that needs to live somewhere the AI can always access.

Three Structural Gaps

Gap 1: Implicit Conventions Stay Implicit

Every team has them. Rules that are enforced in code review but never written down because "everyone knows that."

"We use ActionResult<T> for Server Actions"
"Zod schemas live in lib/schemas/, not colocated with components"
"Don't use useEffect for data fetching — we have Server Components for that"
"Always call auth() before any database access in an API route"

These live in your engineers' heads. When a new hire submits a PR that violates them, a reviewer catches it and leaves a comment. The new hire learns. The knowledge transfers.

AI doesn't get those review comments. It generates code, the review happens, but the AI's context doesn't update. Next task, same violation.

The cost: every AI output needs a manual convention-correction pass. The faster AI can generate code, the more convention-correction work piles up.

Gap 2: Specs Describe WHAT, Not HOW or WHY

Your design doc says "implement user authentication with session management." Good.

What it doesn't say:

Auth middleware belongs in the route handler, not a global Next.js middleware
Sessions are stored in the database, not JWT (because you had a security incident)
Use NextAuth's auth() helper, not direct session inspection
All session-dependent routes return 401, not redirect to login

These "how" and "why" constraints are the engineering culture. They're the decisions your team made, the tradeoffs you've accepted, the patterns you've settled on. A spec that describes the what gives AI enough rope to generate something plausible that violates all of the above.

An AI-effective spec would include the constraints that shape implementation, not just the behavior.

Gap 3: No Feedback Loop for Rule Violations

Human code review creates a feedback loop. Author submits code → reviewer catches violations → author adjusts → over time, author internalizes the rules.

AI-generated code breaks this loop in two ways.

First, the feedback doesn't persist. You can tell Claude "in this project, Server Actions always return ActionResult<T>" in one conversation. Next conversation, you tell it again.

Second, the volume overwhelms the feedback loop. One engineer might submit 3 PRs a week. With AI assistance, they might submit 20. Reviewers who were previously keeping up now face a backlog. Review quality degrades under volume. Violations start slipping through.

At scale, violations accumulate faster than review can catch them. Plausible-but-wrong patterns spread across the codebase. Six months later, you have a refactoring problem that looks like technical debt but is actually accumulated convention drift.

The Compounding Effect

A single convention violation in one file: minor, easy to fix.

10 AI-generated files with the same violation: a pattern.

50 AI-generated files with diverse convention violations: an architectural consistency problem. Different error handling here, different auth patterns there, schemas mixed into component files, raw Prisma objects returned from APIs.

This is the state many teams are in now — not because AI is bad, but because they're using AI without the infrastructure to give it their institutional knowledge.

From retrospectives I've reviewed and conversations with teams adopting AI tools: teams without explicit guidelines consistently report spending significantly more time — often 2–4× — on "cleanup" and "consistency work" than teams with structured rule systems. A common pattern: the team celebrates 3× faster feature generation, then quietly absorbs the same hours back in review cycles, convention-fixing passes, and cross-file inconsistency cleanup. The productivity gains from AI generation are real; they're being partially eaten by convention-drift cleanup.

What "Design Intent" Actually Means

When I say "design intent," I mean more than the features described in a spec. I mean:

Naming conventions and why they exist. Not just "use camelCase" but "function names that fetch data start with get, mutations start with update or create, and no function is named handle because it tells you nothing."

Which patterns are preferred and which are deprecated. "We use Server Components for data fetching. useEffect for data fetching was how we did it two years ago and there's still some in the codebase — don't follow it."

Security invariants. "Every external-facing API route must validate the session. This is not a preference, it's a requirement. There are no exceptions."

The tradeoffs you've accepted. "We verbose-type everything even when TypeScript could infer it. We know this is more code. We've decided explicit types are worth it for this team."

These don't belong in a design doc — they're too foundational, too stable. They belong in a persistent context that every AI interaction can access.

Rethinking What a Spec Is For

A traditional specification document is written for humans. It describes desired behavior, UI flows, data models, and edge cases. The reader is assumed to already know your team's conventions.

An AI-effective specification is different. It still describes behavior. But it also includes:

The constraints that shape implementation ("use NextAuth, not custom JWT")
The non-options ("do not introduce a new state management library")
The patterns that must be followed ("Server Actions must follow the ActionResult<T> pattern")
The why behind significant decisions ("we use Prisma's typed client because we've been burned by SQL injection in the past")

Consider the difference between these two spec sections:

Traditional:

Implement an API endpoint that allows authenticated users to update their profile information.

AI-effective:

Implement an API route at app/api/profile/route.ts that allows authenticated users to update their profile.

Implementation constraints:

Use auth() from @/lib/auth to verify the session

Accept only name and image fields — no other fields should be updatable via this endpoint

Validate input with a Zod schema before any database operation

Return { data: user } on success with only safe fields selected (no passwordHash, emailVerified, or internal IDs)

Return { error: "..." } with appropriate HTTP status on failure

The second version leaves less room for AI to infer its own patterns. It specifies the how alongside the what.

The Solution Direction: Externalizing Tacit Knowledge

The fix isn't writing better prompts. It's building an infrastructure that makes your team's tacit knowledge persistent and accessible.

The practical shape of this:

1. A persistent context file (CLAUDE.md or equivalent)

Keep it to 3 directive lines — a project header, "follow these guidelines," and "upper sections take priority." The rest is a list of file references pointing to your actual rule files. This loads on every interaction, but because it's just an index, it doesn't dilute Claude's attention on your task.

2. Guideline files for L3 specifics

Naming conventions, security invariants, error handling patterns — these live in dedicated files (docs/ai-dev-os/03_guidelines/common/security.md, etc.) and are loaded contextually when relevant. A rule submodule like ai-dev-os-rules-typescript provides these for TypeScript/Next.js projects out of the box.

3. AI-effective specs for larger tasks

For non-trivial features, include implementation constraints in the spec alongside behavioral requirements — which auth helper to use, which patterns to follow, which approaches to avoid. Write the spec like you're briefing a developer who knows nothing about your codebase's conventions.

This is a systems-design problem, not a prompting problem. You design the system once. The AI operates within it on every task.

The next article in this series walks through building this infrastructure for a Next.js project — step by step, with before/after code.

The structural gap between specs and implementation isn't inherent to AI. It's a gap we created by giving AI our "what" without our "how."

Closing it requires treating your team's conventions as infrastructure, not tribal knowledge.

What's the most persistent spec-drift you've hit with AI coding tools? Drop it in the comments — I'm cataloging these to improve guideline templates.

Next in this series: Codifying Tacit Knowledge: The Missing Layer Between Your Team's Conventions and Your AI Assistant (link will be added on publish)

Under 30-Minute Setup: AI Coding Guidelines for Your Next.js Project (Before/After Code)

yunbow — Tue, 05 May 2026 01:49:14 +0000

Your Next.js project has an AI assistant. The question isn't whether to use AI — it's whether the code it generates follows your project's conventions.

This is a hands-on walkthrough. In under 30 minutes (10 via CLI, 20 manually), you'll install AI Dev OS into an existing Next.js project using Claude Code. By the end, your AI will consistently apply your naming conventions, security patterns, and architectural rules — without you reminding it every prompt.

All examples show before/after code so you can see exactly what changes.

What You're Installing (5 min)

AI Dev OS uses a 4-layer model. For a Next.js project, here's what goes where:

CLAUDE.md                     ← L1 + L2: Always loaded, project philosophy
.claude/guidelines/
  nextjs.md                   ← L3: App Router, Server Actions, Middleware
  typescript.md               ← L3: Type conventions, naming rules
  security.md                 ← L3: Auth patterns, input validation
.claude/commands/             ← L4: Reusable Skills for common tasks

The rules you care about most day-to-day live in guidelines/. They're loaded when relevant — not every single prompt — so they don't dilute Claude's attention on your actual task.

Prerequisites

Node.js >= 18
Next.js >= 14 (App Router)
Claude Code CLI installed
TypeScript (strict mode strongly recommended)

Already have a running Next.js project? Good — work from there. Starting fresh also works.

Option A: CLI Setup (10 min)

The fastest path:

npx ai-dev-os init --rules typescript --plugin claude-code

This generates:

CLAUDE.md — guidelines index (3 directive lines + file references)
docs/ai-dev-os/ — rule submodule with L1–L3 guidelines for TypeScript/Next.js
.claude/plugins/ai-dev-os/ — Skills, Agents, and Hooks (L4)
Pre-commit hooks for automated rule verification

After it runs, open CLAUDE.md and add your project-specific guideline files to the ## Project-Specific Guidelines section (3 files is the recommended starting point).

Verify the setup:

npx ai-dev-os doctor

Output should show all guideline files detected and no configuration conflicts.

Option B: Manual Setup (20 min)

Manual setup takes longer but gives you a clear mental model of what each file does. Recommended if you want to adapt the rules heavily.

Step 1: Add the rule submodule and copy the CLAUDE.md template

# Add L1–L3 rules for TypeScript/Next.js
git submodule add https://github.com/yunbow/ai-dev-os-rules-typescript.git docs/ai-dev-os
git submodule update --init

# Add Claude Code plugin (Skills, Agents, Hooks)
git submodule add https://github.com/yunbow/ai-dev-os-plugin-claude-code.git .claude/plugins/ai-dev-os

# Copy the CLAUDE.md template
cp docs/ai-dev-os/templates/nextjs/CLAUDE.md.template ./CLAUDE.md

CLAUDE.md is an index file — 3 directive lines followed by references to guideline files in the submodule. Open it and fill in your project name, then add project-specific guideline files in the ## Project-Specific Guidelines section:

## Project-Specific Guidelines
- docs/guidelines/your-business-rule.md
- docs/guidelines/your-external-api.md
- docs/guidelines/your-compliance.md

3 files is the recommended starting point. Create these files in docs/guidelines/ and document the rules specific to your project (auth provider choices, external service integrations, business constraints).

Step 2: Understand what the submodule provides

The docs/ai-dev-os/ submodule already contains comprehensive L3 guidelines. Key files for a Next.js project:

docs/ai-dev-os/03_guidelines/
  frameworks/nextjs/
    overview.md         ← App Router conventions, tech stack overview
    api.md              ← API route patterns (auth check, field selection)
    server-actions.md   ← ActionResult<T> pattern, Zod validation
    auth.md             ← NextAuth integration
    database.md         ← Prisma usage, no raw SQL
  common/
    security.md         ← Auth invariants, data exposure prevention
    validation.md       ← Zod schema placement and usage
    error-handling.md   ← Error patterns
    naming.md           ← Naming conventions

These files contain the concrete rules with examples — including the patterns shown in the Before/After section below. You don't write these; they come with the submodule.

Step 3: Create the ActionResult type

If you don't have it already:

// types/action.ts
export type ActionResult<T = void> =
  | (T extends void ? { success: true } : { success: true; data: T })
  | { success: false; error: string }

This conditional type handles the common void case — Server Actions that return no data can use return { success: true } without a data field, while actions that return data get a fully typed { success: true; data: T }.

Verification: Test Before/After

The best way to confirm your setup is working: ask Claude the same task twice — before applying guidelines, after.

Test prompt:

Create an API route at app/api/profile/route.ts that returns 
the current user's profile data from the database.

Before (no guidelines, typical output):

import { prisma } from '@/lib/prisma'

export async function GET(request: Request) {
  const userId = request.headers.get('x-user-id')
  const user = await prisma.user.findUnique({ where: { id: userId } })
  return Response.json(user)
}

Problems: takes user ID from a header (trivially spoofable), returns the entire user object including password hash and tokens, no error handling if user not found. Also note the import uses prisma from @/lib/prisma — in the conventions we're setting up, the db client is exposed as db from @/lib/db for consistency.

After (with guidelines loaded):

import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

export async function GET(): Promise<Response> {
  const session = await auth()
  if (!session) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 })
  }

  const user = await db.user.findUnique({
    where: { id: session.user.id },
    select: {
      id: true,
      name: true,
      email: true,
      image: true,
      createdAt: true,
    },
  })

  if (!user) {
    return Response.json({ error: 'Not found' }, { status: 404 })
  }

  return Response.json({ data: user })
}

Auth from session (not headers). Explicit field selection. Error cases handled. Consistent response shape.

Before / After: Server Action

Test prompt:

Create a Server Action to update the current user's display name.

Before:

'use server'
import { prisma } from '@/lib/prisma'

export async function updateDisplayName(name: string) {
  await prisma.user.update({
    where: { id: 'user_id_here' },
    data: { name },
  })
}

Problems: hardcoded placeholder ID, no validation, no auth, no return value.

After:

'use server'
import { z } from 'zod'
import type { ActionResult } from '@/types/action'
import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

const UpdateNameSchema = z.object({
  name: z.string().min(1, 'Name is required').max(100, 'Name too long'),
})

export async function updateDisplayName(
  formData: FormData
): Promise<ActionResult<void>> {
  const session = await auth()
  if (!session) return { success: false, error: 'Unauthorized' }

  const parsed = UpdateNameSchema.safeParse({ name: formData.get('name') })
  if (!parsed.success) return { success: false, error: parsed.error.message }

  await db.user.update({
    where: { id: session.user.id },
    data: { name: parsed.data.name },
  })

  return { success: true }
}

Loading Guidelines in Practice

The Skills bundled with ai-dev-os-plugin-claude-code handle contextual loading automatically. For example, the /api skill loads the relevant framework and security guidelines before generating code.

You can also reference guidelines directly in your prompt:

@docs/ai-dev-os/03_guidelines/frameworks/nextjs/server-actions.md
Create a Server Action to update the current user's display name.

Or add a project-specific skill in .claude/commands/api.md that loads your project-specific rules alongside the framework guidelines:

Load the API and security guidelines, then complete the task:

@docs/ai-dev-os/03_guidelines/frameworks/nextjs/api.md
@docs/ai-dev-os/03_guidelines/common/security.md
@docs/guidelines/your-business-rule.md

Task: $ARGUMENTS

Now in Claude Code: /api Create a route for updating user preferences

Next Steps

Add your project-specific rules. The submodule provides the framework-level rules. Create docs/guidelines/ files for your project's business logic, external API integrations, and compliance requirements — these go in the ## Project-Specific Guidelines section of CLAUDE.md.

Commit the submodule reference to git. Team members who clone the repo run git submodule update --init --recursive and get the full guideline set automatically.

Keep the submodule updated. Run git submodule update --remote docs/ai-dev-os to pull the latest rules, or pin to a specific version with git checkout v1.x.x inside the submodule.

The full ruleset (with Python/FastAPI support) is at github.com/yunbow/ai-dev-os.

What Next.js patterns does your team enforce that keep slipping through AI-generated code? I'm collecting edge cases to improve the default ruleset.

200 Lines in CLAUDE.md Dropped My Code Quality to 79% — Splitting into 3 Files Got It to 96.9%

yunbow — Mon, 04 May 2026 00:29:54 +0000

More rules should mean better output. That's the intuition.

I spent weeks building a comprehensive CLAUDE.md — 200 lines covering naming conventions, security rules, error handling, architectural patterns, import ordering, type safety requirements, and more. I was proud of it. I'd thought through every scenario.

Then I scored the output.

79.0 / 100.

My carefully crafted documentation was actively making AI performance worse. Here's the experiment, what I found, and how restructuring it to 4 files (the original CLAUDE.md trimmed to ~35 lines, plus 3 topic-specific guideline files) pushed the score to 96.9.

The Experiment

Project: Next.js 14 + TypeScript + Prisma + NextAuth

What I measured: Code quality across 5 dimensions, scored by a structured rubric applied to 20 generated files (mix of API routes, Server Actions, components, utility functions). I applied the rubric manually — each file scored 0–100 per dimension, then averaged across all files. All generation tasks used the same prompts to isolate the config variable:

Dimension	What was scored
Naming	Variable names, function names, file naming — match project conventions?
Security	Auth checks present, input validation, no sensitive data exposure
Error handling	Consistent pattern, proper types, no silent failures
Type safety	No `any`, proper inference, Zod schemas where needed
Test coverage	Unit tests generated alongside implementation files

Baseline: 200-line monolithic CLAUDE.md containing all rules

Why More Rules Made Things Worse

I expected a positive correlation: more rules → better adherence. I got the opposite. Here's why.

Position sensitivity

Claude reads the entire CLAUDE.md on every interaction. A 200-line file isn't huge in absolute terms, but attention is not uniformly distributed across a long document. Rules near the end — in my case, security invariants starting at line 160 — received demonstrably less weight than rules at the top.

In my tests, moving the security section from line 160 to line 20 pushed security scores from 68% to 81% — without changing a single rule. The content was identical. Position was the only variable.

Why: the user's prompt arrives at the end of the context window. Rules at the bottom of CLAUDE.md are immediately adjacent to the prompt and compete with everything above them for the model's attention during generation. The further up a rule is, the less it competes.

Rule conflicts

When L1 philosophy rules and L3 specific rules live in the same file, conflicts appear. My L1 section said "prefer explicitness." My L3 section said "keep implementations concise." When generating a Server Action, the AI had to arbitrate between these — and sometimes picked the wrong tradeoff for the context.

The result by dimension

Dimension	200-line CLAUDE.md
Naming	81%
Security	68%
Error handling	77%
Type safety	82%
Test coverage	86%
Overall	79.0%

Security being the lowest made sense in retrospect. Security rules are naturally verbose and were buried at the bottom.

The Fix: Two-Tier Context

The insight: CLAUDE.md should contain only what the AI needs to know for every single interaction. Everything else should be loaded on demand.

CLAUDE.md                         ← Always loaded (3 directive lines + file refs)
docs/ai-dev-os/03_guidelines/
  frameworks/nextjs/              ← Next.js App Router, Server Actions, etc.
  common/security.md              ← Auth patterns, input validation
  common/validation.md            ← Zod usage
  ...                             ← Referenced from CLAUDE.md
docs/guidelines/                  ← Your project-specific rules (3 files recommended)

The rule of thumb: CLAUDE.md is an index file, not a rules file. Keep it to 3 lines of directives and a list of file references. The actual rules live in the referenced files.

What CLAUDE.md looks like

# Project: [Your App Name] - Claude Code Guidelines

Please write code following the guidelines below for this project.
In case of conflicts, upper sections take priority.

## Development Guidelines
- docs/ai-dev-os/03_guidelines/frameworks/nextjs/overview.md - Overview & tech stack
- docs/ai-dev-os/03_guidelines/frameworks/nextjs/server-actions.md - Server Actions (ActionResult pattern)
- docs/ai-dev-os/03_guidelines/common/security.md - Security
- docs/ai-dev-os/03_guidelines/common/validation.md - Validation
- docs/ai-dev-os/03_guidelines/common/error-handling.md - Error handling
...

## Project-Specific Guidelines
- docs/guidelines/your-business-rule.md
- docs/guidelines/your-external-api.md
- docs/guidelines/your-compliance.md

That's it — 3 directive lines, then file references. The philosophy, coding conventions, and security invariants are all in the referenced files in docs/ai-dev-os/, which come from the rule submodule.

What the topic files contain

The submodule (ai-dev-os-rules-typescript) provides:

docs/ai-dev-os/03_guidelines/common/security.md:

Every auth pattern with examples
Field selection rules for Prisma queries
Input validation requirements
Rate limiting requirements for auth endpoints
Logging requirements for security events

docs/ai-dev-os/03_guidelines/frameworks/nextjs/server-actions.md:

Server Action patterns with full examples (ActionResult<T>)
Middleware structure
Data fetching patterns (no useEffect for data)

docs/ai-dev-os/03_guidelines/common/naming.md:

Import ordering
Type naming conventions
Error type definitions

Your project-specific rules go into docs/guidelines/ — 3 files is the recommended starting point.

Loading on demand

In Claude Code, the Skills bundled with the plugin (ai-dev-os-plugin-claude-code) handle contextual loading automatically. For manual invocation:

Load the Next.js guidelines, then complete the task:

@docs/ai-dev-os/03_guidelines/frameworks/nextjs/overview.md
@docs/ai-dev-os/03_guidelines/common/security.md

Task: $ARGUMENTS

Usage: /nextjs Create an API route for updating user preferences

For Cursor: the Cursor plugin (.cursor/rules/) provides scoped .mdc files with globs patterns — no manual setup needed after running the CLI.

For Kiro: the Kiro plugin copies steering rules to .kiro/steering/ with appropriate include conditions — no manual configuration needed.

After the Split

Dimension	200-line CLAUDE.md	File-reference split
Naming	81%	98%
Security	68%	95%
Error handling	77%	97%
Type safety	82%	99%
Test coverage	86%	95%
Overall	79.0%	96.9%

Security improved the most: from 68% to 95%. Moving security rules to a dedicated file and explicitly loading them for auth/API work meant they were in context when they mattered and not diluting attention when they didn't.

How to Refactor Your Own CLAUDE.md

If yours has grown to 100+ lines, here's the audit process I used:

Step 1: Categorize every rule

Go through your CLAUDE.md line by line and tag each rule:

[always] — needs to apply to every single generated line
[task:X] — only relevant when working on task type X
[rarely] — edge case that doesn't need to be in persistent context

Step 2: Move all rules to topic files

Rules in [task:X] and [rarely] move to dedicated files (security.md, testing.md, etc.) in docs/guidelines/ or a rule submodule. Replace the inline content in CLAUDE.md with file references.

Step 3: Reduce CLAUDE.md to 3 directive lines + references

CLAUDE.md should contain:

A project header
"Please write code following the guidelines below."
"In case of conflicts, upper sections take priority."
A list of file references (not inline rules)

If CLAUDE.md still has inline rules rather than file references, you haven't finished the refactor.

Step 4: Set up loading mechanisms

Claude Code: The ai-dev-os-plugin-claude-code plugin includes Skills and Agents that load the relevant guideline files contextually
Cursor: The ai-dev-os-plugin-cursor provides scoped .mdc files with globs patterns
Kiro: The ai-dev-os-plugin-kiro installs steering rules into .kiro/steering/

Step 5: Score before and after

Run the same 5–10 generation tasks with your old config and your new config. If you don't have a formal rubric, just note which rules are adhered to. You'll see the pattern.

The Counter-Intuitive Takeaway

The impulse to "add more to your CLAUDE.md" when you notice rule violations is natural — and usually wrong.

If the AI isn't following a rule, the first question isn't "should I add more detail?" It's "is this rule in the right place?" If it's a convention only relevant to one type of task, moving it to a topic file and loading it contextually will do more than elaborating on it in the always-on context.

Less in CLAUDE.md. More in the right place at the right time.

Next in the series: 30-Minute Setup: AI Coding Guidelines for Your Next.js Project (link will be added on publish)

Why High-Context Culture Makes AI Coding Harder — A Japanese Developer's Perspective

yunbow — Sun, 03 May 2026 12:10:41 +0000

I'm a software engineer in Japan. I've been using AI coding assistants — Claude Code, Cursor, Copilot — for about one years now.

At some point I started keeping informal notes on how many prompt revisions it took to get production-quality output. After a few months, a pattern was hard to ignore.

For tasks I described in Japanese: 4–6 revisions on average.
Watching colleagues who worked primarily in English: 1–3.

Same AI. Same model. Roughly similar task complexity. Different language — and apparently, different results.

I started asking why.

The Most Invisible Problem in a High-Context Language

Japanese ranks among the world's most high-context languages. What that means practically:

Subject omission is grammatically normal. "Did it" is a complete sentence. Who did what to whom lives in context.
Vague expressions are socially preferred over explicit ones. "Somehow it feels off" is a complete critique in a code review.
Politeness layers add indirection at every level. Direct requests sound rude. Softened requests carry meaning only if you know what's being softened.

None of this is a flaw. It's a communication system that works — between humans who share context.

AI assistants do not share your context.

When I write "please handle this appropriately," I know what "appropriately" means in my codebase. My AI assistant does not. It makes its best guess, which is informed by the global distribution of code it was trained on — not by the six months of implicit decisions my team has accumulated.

The result is plausible-looking code that misses the point in ways that take hours to diagnose.

The 6 Tacit Knowledge Zones

After talking to engineers across Japanese dev teams, I found the same blind spots appearing in almost every organization:

1. Documentation that doesn't document
Formats vary by author. Files go months without updates. The real spec lives in Slack DMs and someone's memory.

2. Team rules that exist only in senior developers' heads
Coding conventions exist in practice — enforced silently in code review, never written down because "everyone knows."

3. Security as an afterthought
Security is something you consider before release, not during development. The AI doesn't know this is your team's operating assumption. It will skip security considerations unless explicitly prompted — which you won't do if you don't think about it mid-task.

4. Task management via word of mouth
The real priority queue is the one your team lead carries in their head and communicates in standup. The ticket system is a formality.

5. Testing without test cases
You test the happy path manually before shipping. Edge cases are discovered in production. The AI will generate code that matches this standard — because you didn't specify otherwise.

6. Operations knowledge locked in one person
When something breaks at 2am, one person knows what to do. That person's knowledge doesn't exist anywhere else.

Notice what these have in common: they're not the absence of knowledge. They're knowledge that lives in exactly one place — a person's head — and will disappear when that person leaves.

Why AI Makes This Worse, Not Better

The instinctive argument: "AI will fix this — it generates more documentation, writes more consistent code."

The reality is the opposite.

AI is excellent at transforming explicit knowledge into more explicit knowledge. Give it a well-defined spec and it produces well-defined code. Give it a type definition and it generates a correctly-typed implementation.

What AI cannot do is extract tacit knowledge from the environment. It cannot interview your senior engineer. It cannot observe your code review culture. It cannot infer that your team decided six months ago that useEffect for data fetching was banned after a painful production incident — because that decision exists only in the memory of the three people who were there.

When you give an AI assistant a vague prompt, it fills the gaps. It uses the global distribution of code patterns it learned during training. That distribution doesn't include your team's specific decisions.

The higher your tacit knowledge density — and high-context organizations tend to have higher tacit knowledge density — the larger the gap between what you intend and what the AI produces.

A Scenario That's More Common Than It Should Be

The following is a fictional but composite scenario based on failure patterns I've seen and read about. The system and people are made up. The mechanisms are real.

A team builds a medical appointment system. Development moves fast — AI-assisted, one month to MVP.

What gets missed in the first week: domain knowledge. The engineers don't know that insurance category codes have a specific format. They don't know that doctor license numbers have validation rules. The AI doesn't ask. No one thinks to tell it.

By month one: the same appointment conflict logic has been implemented in three different places, each slightly differently, because three developers each asked the AI to "handle booking conflicts" without knowing the others had done it. The AI gave each of them a reasonable implementation.

Security holes accumulate invisibly. Patient records are accessible without proper authorization checks because the prompt said "fetch appointments for this user" and didn't say "verify the requesting user has access to this patient's records." The AI did exactly what was asked.

The system ships. In the first week, a researcher discovers they can access any patient's records by modifying the URL parameter. The system goes offline.

When the team tries to fix it, they discover that the implicit business logic — the stuff that existed only in the product manager's head — was never written down. No one knows what the correct behavior is for edge cases anymore.

The CTO spends the next three months rebuilding from scratch, this time writing the spec before writing the code.

The lesson isn't "don't use AI." The lesson is: the bottleneck in AI-assisted development isn't writing code. It's making implicit knowledge explicit before the AI touches anything.

What Actually Shifts the Outcome

I've been working on an open-source framework called AI Dev OS that's essentially a structured answer to this problem. But the framework is less important than the underlying principle.

1. Prompt in the language that minimizes ambiguity for the task

This is uncomfortable to write as a Japanese developer, but: for technical specification, English or highly explicit Japanese outperforms natural conversational Japanese. Not because English is better, but because technical communication benefits from low-context precision.

❌  「いい感じに処理してください」
✅  「ユーザーがPOSTした画像ファイルを、
    サーバーサイドでWebP形式に変換して
    S3バケット {env.BUCKET_NAME} に保存してください。
    ファイルサイズ上限は5MB。超過はValidationErrorを返すこと。」

2. Write down the thing that "everyone knows" before you prompt

The most valuable five minutes before any significant AI-assisted task: write one paragraph of the rules that your senior developer would enforce in code review. Not a full spec — just the implicit decisions that would otherwise live in their head and filter silently through review.

Put it in a CLAUDE.md file or .cursorrules. Make it persistent. Now the AI has access to your context.

3. Spec before code, always

Ask the AI to write the specification first: input/output types, error cases, security assumptions, business rules. Review it. Then ask for the implementation.

The friction this adds is real. The bugs it prevents are also real — and much more expensive.

4. Treat tacit knowledge as an organizational risk, not a personal style

When the person who holds the knowledge leaves, it's gone. An AI tool that depends on that person's mental model in every prompt is inheriting that fragility.

The act of writing down team conventions isn't documentation overhead. It's risk reduction.

The Question Worth Asking Your Team

What's the most important unwritten rule in your codebase right now?

Not the stuff in your linting config or your CONTRIBUTING.md. The real rule — the one a senior engineer would catch in review but that exists nowhere in text.

If you can name it, you can write it down. If you can write it down, your AI assistant can follow it.

That one rule, made explicit, is worth more than any prompt engineering technique I've found.

For what it's worth: the thing I wrote down that made the biggest difference was two sentences. "Server Actions always return ActionResult<T>. Never throw from a Server Action." Six months of drift, fixed by two sentences in a markdown file.

Start there.

I built AI Dev OS to systematize this process for Next.js + TypeScript projects. It's open source — the framework, the templates, the reasoning. If this resonated, that's where the implementation lives.

What's the unwritten rule in your codebase? I'm genuinely curious — drop it in the comments.

AI Dev OS

yunbow — Sun, 03 May 2026 08:07:56 +0000

Articles

I Built an OSS to Stop AI from Writing Inconsistent Code Every Time (Claude Code / Cursor / Kiro)

yunbow — Sun, 03 May 2026 04:07:06 +0000

"Write a function to fetch the list of users." — same prompt, same codebase.

Yesterday: getUsers(). Today: fetchUserList(). Tomorrow: loadAllUsers().

Six months of AI-assisted coding and I kept hitting this wall. My initial reaction was "maybe I need to write better prompts." I wrote better prompts. The functions got slightly better. New inconsistencies appeared elsewhere.

The problem wasn't the AI's capability. It was that I had never given it my team's unwritten rules.

That realization led me to build AI Dev OS — an open-source framework that converts implicit developer knowledge into explicit, enforceable rules for AI coding assistants. It supports Claude Code, Cursor, and Kiro (Amazon's AI-native IDE).

GitHub: github.com/yunbow/ai-dev-os

The Root Problem: "Almost Correct" Is the Most Expensive Output

When an AI produces obviously wrong code, you catch it immediately and discard it. Not a big deal.

When an AI produces almost correct code — code that compiles, passes basic tests, looks reasonable — you merge it. Six months later, your codebase has three error handling patterns, four naming styles, and no one is sure which is canonical.

That's the real cost. Not obvious errors. Plausible drift.

Here's why this happens structurally: your AI assistant has no access to your team's conventions by default. It doesn't know that you use ActionResult<T> for Server Actions, that Zod schemas live in lib/schemas/ and not colocated with components, that you've banned useEffect for data fetching. These conventions exist in your engineers' heads and in unwritten PR review culture — nowhere an AI can read.

Switching to a more capable model doesn't fix this. A more capable model without your conventions just produces more fluent drift.

The Solution: A 4-Layer Rule Framework

The core insight of AI Dev OS: rules have different lifespans, and mixing them into one file is what causes fragility.

┌─────────────────────────────────────────────────────────┐
│  L1: Design Philosophy          (2–5 year lifespan)     │
│      "We optimize for correctness and observability     │
│       over developer convenience"                       │
│                                                         │
│  L2: Technology Decisions       (1–3 years)             │
│      "Next.js App Router, Prisma, Zod, NextAuth"        │
│      "We use Tailwind — no CSS-in-JS"                   │
│                                                         │
│  L3: Concrete Coding Rules      (6–12 months)           │
│      Naming conventions, error handling patterns,       │
│      security invariants, test structure                │
│                                                         │
│  L4: Tool-Specific Config       (2–4 months)            │
│      CLAUDE.md, .cursor/.mdc files, Kiro Steering Rules │
└─────────────────────────────────────────────────────────┘

Why does this matter? When you update your AI tool's configuration (L4), you shouldn't risk destabilizing your design philosophy (L1). When you adopt a new library (L2), you shouldn't have to rewrite your security rules (L3). Separating by lifespan makes updates surgical.

The 7-Repository Structure

AI Dev OS is distributed across 7 repositories, each with a clear responsibility:

ai-dev-os                     ← Core specification and theory     [available]
ai-dev-os-rules-typescript    ← L1–L3 rules for TypeScript / Next.js  [available]
ai-dev-os-rules-python        ← L1–L3 rules for Python / FastAPI  [available]
ai-dev-os-plugin-claude-code  ← L4: Skills, Agents, Hooks for Claude Code  [available]
ai-dev-os-plugin-cursor       ← L4: .mdc rules for Cursor         [available]
ai-dev-os-plugin-kiro         ← L4: Steering Rules for Kiro       [available]
ai-dev-os-cli                 ← init / update / doctor commands   [available]

You don't need all of them. If you're using Claude Code with TypeScript, you need ai-dev-os-rules-typescript and ai-dev-os-plugin-claude-code. Pick what fits your stack.

How to Get Started

Option A: CLI (10 minutes)

npx ai-dev-os init --rules typescript --plugin claude-code

This creates:

CLAUDE.md — guidelines index referencing your rule files
docs/ai-dev-os/ — rule submodule (L1–L3)
.claude/plugins/ai-dev-os/ — Skills, Agents, Hooks (L4)
Pre-commit hooks for automated rule verification

Verify the setup:

npx ai-dev-os doctor

Option B: Manual (20 minutes, more control)

Step 1: Add the rule submodule and copy the CLAUDE.md template.

git submodule add https://github.com/yunbow/ai-dev-os-rules-typescript.git docs/ai-dev-os
cp docs/ai-dev-os/templates/nextjs/CLAUDE.md.template ./CLAUDE.md

CLAUDE.md is an index file — 3 lines of directives plus references to the guideline files in the submodule. Add your project-specific rules (3 files recommended) in the ## Project-Specific Guidelines section:

## Project-Specific Guidelines
- docs/guidelines/your-business-rule.md
- docs/guidelines/your-external-api.md
- docs/guidelines/your-compliance.md

Step 2: Add the Claude Code plugin for Skills, Agents, and Hooks:

git submodule add https://github.com/yunbow/ai-dev-os-plugin-claude-code.git .claude/plugins/ai-dev-os

Claude Code loads CLAUDE.md automatically on every interaction. The guideline files referenced within it are read on demand.

Before / After: What This Actually Changes

API Route

Before (no guidelines):

// app/api/users/route.ts
export async function GET(req: Request) {
  const users = await db.user.findMany()
  return Response.json(users)
}

Problems: no auth check, exposes all user fields including sensitive ones, no error handling.

After (with AI Dev OS):

// app/api/users/route.ts
import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

export async function GET(): Promise<Response> {
  const session = await auth()
  if (!session) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 })
  }

  const users = await db.user.findMany({
    select: { id: true, name: true, email: true, createdAt: true },
  })

  return Response.json({ data: users })
}

Auth check. Field selection. Consistent response shape. The AI generated this without being asked because the rules made it the default.

Server Action

Before:

'use server'
export async function updateUser(formData: FormData) {
  const name = formData.get('name')
  await db.user.update({ where: { id: '...' }, data: { name } })
}

Problems: no validation, no error handling, name is typed as FormDataEntryValue | null, hardcoded ID.

After:

'use server'
import { z } from 'zod'
import type { ActionResult } from '@/types/action'
import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

const UpdateUserSchema = z.object({
  name: z.string().min(1).max(100),
})

export async function updateUser(
  formData: FormData
): Promise<ActionResult<void>> {
  const session = await auth()
  if (!session) return { success: false, error: 'Unauthorized' }

  const parsed = UpdateUserSchema.safeParse({ name: formData.get('name') })
  if (!parsed.success) return { success: false, error: parsed.error.message }

  await db.user.update({
    where: { id: session.user.id },
    data: parsed.data,
  })

  return { success: true }
}

Every Server Action in the codebase now follows this pattern — not because individual developers remember to, but because the AI enforces it by default.

The Effect Over Time

The value compounds. After 2 weeks with AI Dev OS:

Every new API route has auth checks and field selection
Every Server Action validates input and returns ActionResult<T>
PR reviews stop being about convention enforcement and start being about logic

After 2 months, a new engineer joining the project reads the guideline files and understands the codebase's patterns — because those patterns are now consistently applied everywhere, including the AI-generated portions.

What This Isn't

AI Dev OS won't catch logical bugs. It won't validate your business rules. It won't replace code review.

What it does: ensure the AI produces code that matches your codebase's established patterns. The output is still yours to review — but you're reviewing logic, not convention drift.

Try It

# Check out the core spec
git clone https://github.com/yunbow/ai-dev-os

# Or start with the TypeScript rules
git clone https://github.com/yunbow/ai-dev-os-rules-typescript

If you've been hitting the inconsistency wall and want to talk through how to adapt the rules for your stack — drop a comment. What conventions does your team enforce that your AI assistant keeps ignoring?