Forem: Serhii Panchyshyn

Things You're Overengineering in Your AI Agent (The LLM Already Handles Them)

Serhii Panchyshyn — Tue, 14 Apr 2026 20:15:22 +0000

I've been building AI agents in production for the past two years. Not demos. Not weekend projects. Systems that real users talk to every day and get angry at when they break.

And the pattern I keep seeing? Engineers building elaborate machinery around the model. Custom orchestration layers. Hand-rolled retry logic. Massive tool routing systems. All to solve problems the LLM was already solving if you just let it.

Here's what I'd rip out if I could go back.

1. Custom Tool Selection Logic

You built a classifier that decides which tool the agent should use. Maybe a regex-based router. Maybe a whole separate model call just to pick the right function.

Stop.

Modern LLMs are shockingly good at tool selection when you give them well-named, well-described tools. The problem was never the model. It was your tool descriptions.

// Bad: vague tool name, model guesses wrong
{ name: "search", description: "Searches for things" }

// Good: specific name, clear scope, model nails it
{ name: "search_customer_orders", description: "Search customer order history by order ID, customer name, or date range. Returns order status, items, and tracking info." }

The fix isn't a smarter router. It's better tool design. Name your tools like you're writing an API for a junior dev who's never seen your codebase. Be embarrassingly specific.

Tool selection metrics can look great while the final answer is still garbage. I've seen this firsthand. The agent picks the right tool 95% of the time but still gives wrong answers because the tool descriptions don't explain what the returned data actually means.

2. Prompt Chains for Multi-Step Reasoning

I used to build 4-5 step prompt chains for anything complex. Break the problem down. Feed output A into prompt B. Parse the result. Feed it into prompt C.

Turns out a single well-structured system prompt with clear instructions handles most of this natively. The model already knows how to decompose problems. You just need to tell it what your constraints are and what good output looks like.

// Instead of chaining 3 prompts:
// 1. "Classify the user intent"
// 2. "Based on intent X, gather context"  
// 3. "Now generate the answer"

// Just do this:
const systemPrompt = `You are a support agent for a logistics platform.

When a user asks a question:
1. Identify whether they need order status, account help, or technical support
2. Use the appropriate tool to get the data
3. Answer in plain English with the specific details they asked for

If you're unsure about intent, ask one clarifying question. Never guess.`

The chain approach also creates a hidden problem. Each step is a failure point. And debugging a 4-step chain when something breaks on step 3 is miserable. A single prompt with clear instructions is easier to observe, easier to eval, and fails more gracefully.

3. Retrieval Complexity Before Retrieval Quality

This one hurts because I've done it myself.

You spend two weeks building a hybrid retrieval pipeline. BM25 plus vector search plus re-ranking. Beautiful architecture. Looks great in a diagram.

Then you realize the actual problem is that your knowledge base documents are written in a way the model can't parse. Or your chunking strategy splits the answer across two chunks and neither one makes sense alone.

The retrieval pipeline doesn't matter if the underlying data is messy.

Before you optimize the search algorithm, ask yourself:

If I showed this chunk to a human with no context, would they understand the answer?
Are my documents written for the model or for the original author's brain?
Am I chunking at logical boundaries or just every 500 tokens?

I've seen teams where retrieval "works" but answers are still wrong because the reference data itself contains outdated or incorrect information. That's not a retrieval problem. That's a data quality problem wearing a retrieval costume.

4. Custom Guardrails That Block Legitimate Use

You built a content filter. It catches bad inputs. Great.

Then users start complaining that normal questions get blocked. Someone asks about "terminating a contract" and the guardrail flags "terminating." Someone asks about shipping "explosive growth" and that trips another filter.

Rule-based guardrails at scale become a whack-a-mole game you can't win.

The LLM itself is already pretty good at understanding intent and context. Instead of building regex walls around the model, build guardrails INTO the model's instructions. Tell it what topics are off-limits. Tell it what information it should never reveal. Tell it to redirect gracefully instead of stonewalling.

// Instead of: regex filter that blocks "kill", "terminate", "destroy"
// Try this in your system prompt:

`If a user asks about topics outside your domain (logistics and order management), 
politely redirect them. Never share internal system details, API keys, 
or other customer data. You can decline requests, but always explain why 
and suggest what you CAN help with.`

Guardrails and permissions are product design, not just safety theater. Treat them that way.

5. Agent Memory as a Separate System

You have your agent's database over here. Its memory system over there. A vector store somewhere else. And glue code holding all of it together with prayers and setTimeout.

The real question is simpler than the architecture you built: what does the agent actually need to remember between sessions?

Most agents don't need a sophisticated memory system. They need a well-structured context window. The conversation history plus a few key facts about the user. That's it. The model handles the rest.

When you DO need persistent memory, keep it close to your data. Don't build a separate memory service that has to sync with your database. Store memory where your data lives. Query it with the same tools.

The moment your agent's memory can't see its own database, you've created an integration problem disguised as a feature.

6. Sub-Agent Orchestration for Everything

Multi-agent architectures are seductive. One agent plans. One retrieves. One generates. One validates. They talk to each other through a message bus. It looks amazing on a whiteboard.

In production it's a nightmare to debug. When the answer is wrong, which agent broke? The planner? The retriever? The generator? You end up building observability tooling just to trace what happened across four agents when one would have been fine.

Start with one agent. Push it until it genuinely can't handle the complexity. Only THEN split into specialized sub-agents with clear, narrow responsibilities.

The rule I use: a sub-agent should exist only when the parent agent's context window literally can't hold the information it needs. Not because "separation of concerns" sounds good in a design doc.

Specialized agents make sense for high-context tasks where the prompt would blow up the token budget. General agents handle 80% of use cases with less operational overhead. Know which one you're building and why.

7. Evaluations That Test Happy Paths

This is the one that bites hardest.

You write 50 eval cases. The agent passes 48 of them. Ship it.

Then users find the 200 edge cases you didn't think of. The model hallucinates a tracking number. It confidently answers a question it should have said "I don't know" to. It uses data from one customer to answer another customer's question.

Good evals don't test whether the agent CAN answer correctly. They test whether it WILL answer correctly under pressure.

Build evals that target failure modes:

What happens when the tool returns empty results?
What happens when two tools return conflicting information?
What happens when the user asks something slightly outside the agent's domain?
What happens when the context is ambiguous?

The eval suite is the real moat. Not the model. Not the prompts. Not the architecture. The team that can systematically find and fix failure modes ships better agents than the team with the fancier framework.

The Uncomfortable Truth

Most of the complexity in your agent isn't making it smarter. It's making it harder to debug, harder to eval, and harder to change.

The best agent architectures I've built are embarrassingly simple. One model. Clear system prompt. Well-named tools. Good data. Ruthless evals.

Everything else is either premature optimization or an expensive lesson waiting to happen.

What's the most over-engineered thing you've built into an agent that turned out to be unnecessary?

How to Make a Company AI-Native (Without Building Anything)

Serhii Panchyshyn — Tue, 14 Apr 2026 14:33:17 +0000

I've been helping companies add AI to their products since early 2023.

Two years doesn't sound like much. But in AI time, it's multiple generations. We've gone from "ChatGPT can write emails" to agents running workflows to AI systems coordinating with other AI systems.

Through all of that, the failure patterns stayed the same.

Most teams think becoming AI-native means building AI features. Ship a chatbot. Add a copilot. Sprinkle some RAG on the knowledge base.

That's not what AI-native means.

The teams that actually get there change how they operate first. How they learn. How they document. How they measure. How they build trust. The AI features come after.

Here's what I've learned helping B2B SaaS teams make that shift.

AI-native is about operating around ambiguity

Here's the thing nobody tells you: AI systems are probabilistic. They're not like traditional software where the same input gives the same output every time.

That breaks a lot of assumptions.

Traditional software: "If we ship this feature, it will work the same way for every user."

AI software: "If we ship this feature, it will probably work most of the time, and we need systems to catch when it doesn't."

The teams that become AI-native redesign their workflows around this reality. They build feedback loops. They measure obsessively. They treat failures as data, not embarrassments.

The teams that fail keep expecting AI to behave like traditional software. Then they're surprised when it doesn't.

Start with a maturity model, not a feature roadmap

Most companies treat AI adoption as a feature checklist. "We need a chatbot. We need RAG. We need agents."

That's backwards.

I use a five-stage model when I work with teams:

Stage	What it looks like
0	No AI usage. Maybe some ChatGPT for personal stuff.
1	Individual experimentation. People trying tools on their own.
2	Workflow integration. AI embedded in daily tools and processes.
3	Specialized AI for specific domains and jobs.
4	AI systems coordinating with other AI systems.

The goal isn't to jump to Stage 4. That's how you build impressive demos that break in production.

The goal is to help everyone move up one stage.

A support team at Stage 0 needs permission to experiment and a few quick wins. An engineering team at Stage 2 might be ready for domain-specific AI. A data team at Stage 3 might be ready for more automation.

When I start with a new client, I map where each team actually is. Not where they think they are. Not where leadership wishes they were. Where they actually are today.

Then we figure out what moving up one stage looks like for each group.

Make learning mandatory and social

The teams that succeed at AI adoption don't just "allow experimentation." They create structured learning environments.

One pattern I've seen work: dedicated AI learning weeks. No customer calls. No side meetings. Everyone learns together.

The details that make it work:

Everyone teaches something. Not just a few experts presenting to passive audiences. Each person finds one thing they've figured out and shares it with the team.

Sessions are leveled. Mark each session by audience level and prerequisites. A Stage 1 person shouldn't sit through an advanced automation deep-dive. They'll tune out and feel behind.

Context packs are provided. Don't just demo tools. Give people the actual prompts, templates, and access they need to use the tools during and after the session.

It's not optional. The forcing function matters. "Feel free to experiment" produces nothing. "We're all doing this together for a week" produces adoption.

I've seen teams go from 20% AI usage to 80%+ in a single quarter using this approach. The structure matters more than the specific tools.

Trust is an eval problem, not a hype problem

Most teams try to build AI trust through enthusiasm. "Look what it can do! It wrote a whole email!"

That backfires fast.

Someone uses it. Hits an obvious failure. Loses trust. Stops using it. Tells their team it doesn't work.

The teams that build real trust treat it differently. They treat AI like a probabilistic system that must be measured, not believed in.

What this looks like in practice:

Measure things separately. Don't aggregate everything into one "accuracy" number. Did it find the right information? Did it use that information correctly? Did the user actually accept the output? Each question has a different answer and a different fix.

Inspect failures, not just wins. When something breaks, look at what actually happened. What did the AI see? What did it do? Why? Share these learnings openly.

Categorize failure modes. Not all failures are the same. Missing information. Wrong information. Right information used incorrectly. Each has a different root cause.

Share failures openly. "It made something up because the information wasn't documented anywhere" builds more trust than hiding the failure. People understand that AI isn't magic once you show them the mechanics.

Trust increases through validation, verification, and visibility. Not hype.

Documentation becomes infrastructure

In most companies, documentation is something you write once and forget.

In AI-native companies, documentation is working infrastructure. It's not optional. It's a prerequisite for useful AI.

Here's why: when an AI assistant can't answer a question, the root cause is usually that the information isn't documented anywhere. Not that the AI is bad. The knowledge simply doesn't exist in a retrievable form.

I've seen systems where 30-40% of AI failures traced back to missing documentation. Three concepts weren't written down. Three docs got written. Failures dropped.

This changes how you think about it:

Missing docs are bugs. When AI fails because something isn't documented, treat it like any other bug. File it. Fix it.

Use AI failures as a feedback loop. Every failure surfaces a gap. Every fix improves the docs. The AI becomes a quality check on your documentation.

Documentation is no longer just for humans. You're writing for people and for AI systems. That changes what "good documentation" means.

The best teams I've worked with have a direct pipeline from AI failure analysis to documentation improvements. It's not a separate initiative. It's the same workflow.

Internal adoption comes before external features

I've seen teams ship AI features to customers in week one. Trust gets destroyed. The project gets shelved. Starting over is harder than starting slow.

The pattern that works:

Start with 3 internal users. Not 30. Three people who will use it for real work and tell you when it breaks.

Review what's actually happening. For the first week, look at every interaction. What worked? What didn't? Why?

Build your understanding from real failures. Every failure teaches you something. The failures from week one become the fixes for week two.

Expand slowly. 3 users → 10 users → one team → all internal teams → external customers.

Internal users are forgiving. They'll tell you what's broken. They'll help you fix it. External users will just churn.

The companies that become AI-native make their employees power users first. Then they build for customers.

Redesign rituals, not just tools

AI-native is not something you bolt onto existing workflows. It changes how people learn, plan, review, and improve.

Rituals I've seen work:

Quarterly AI learning weeks. Blocked time. Mandatory attendance. Everyone teaches something.

Weekly failure reviews. What broke? Why? What did we learn? No blame, just data.

Documentation sprints. Dedicated time to fill gaps surfaced by AI failures.

Continuous improvement loops. AI quality isn't a launch milestone. It's an ongoing process that gets better every week.

The teams that succeed treat AI adoption as an operating system change. Not a feature installation.

What this looks like when it works

Teams that follow this playbook:

Have 80%+ AI tool adoption across functions
Catch problems before customers do
Improve AI quality continuously through feedback loops
Build trust through measurement, not hype
Ship AI features faster because the foundation is solid

The teams that skip straight to building features stay stuck. They launch impressive demos that don't survive real usage.

The pattern

Map where each team actually is on the maturity model
Help everyone move up one stage
Make AI learning mandatory, structured, and social
Build trust through measurement, not enthusiasm
Treat documentation as working infrastructure
Roll out internally before shipping externally
Redesign rituals to support continuous improvement

AI-native isn't about adding AI features. It's about changing how your team operates around ambiguity.

The tools are the easy part. The behavior change is the hard part. Start there.

How to Roll Out an Internal AI Product Without Lying to Yourself

Serhii Panchyshyn — Mon, 13 Apr 2026 23:53:55 +0000

I've helped teams roll out AI products for the past two years.

The same failure pattern shows up almost every time.

They build something that demos well. Leadership gets excited. They ship it to 50 users in week one. Within two weeks, trust is destroyed and the project gets shelved 😅

The teams that succeed do something different. This is the playbook I walk clients through now.

The problem I see everywhere

Most teams measure AI rollouts wrong.

They track one number. "Accuracy" or "user satisfaction" or something equally vague. The number looks good. They ship broadly. Then real users hit edge cases, the agent hallucinates, and suddenly everyone thinks "AI doesn't work for us."

The issue isn't the AI. The issue is they never built the infrastructure to see what was actually happening.

You can't improve what you can't observe. And most teams can't observe anything.

The rollout framework that works

Here's what I advise now. Nine steps, usually 6-8 weeks before external users.

Step 1: Start with 3 users, not 30

Every team wants to move fast. "Let's get feedback from the whole department!"

I push back hard on this.

More users means more noise. You can't inspect every trace. You start pattern-matching on vibes instead of data.

The right first cohort:

3 people who actually need the tool for real work
Different roles (support, ops, sales)
Direct channel to the eng team

One client started with 30 users. Couldn't keep up. Rolled back to 5. Found more bugs in one week than the previous month.

// What I recommend tracking for each early user
interface EarlyUserContext {
  userId: string;
  role: string;           // "support", "ops", "sales"
  primaryUseCase: string; // "answer customer questions"
  feedbackChannel: string; // direct line to eng team
}

Step 2: Instrument everything before anyone touches it

This is where most teams cut corners. They want to ship. Observability feels like overhead.

It's not optional.

Before the first user session, you need to answer these questions from your traces:

What query did the user send?
What tools did the agent consider?
Which tool did it pick and why?
What context was in the window?
What was the final response?
Did the user accept, edit, or reject it?

I've seen teams ship without trace logging. They have no idea why things fail. They guess. They tweak prompts randomly. Nothing improves.

// Minimum viable trace structure
interface AgentTrace {
  runId: string;
  userId: string;
  query: string;
  toolsConsidered: string[];
  toolSelected: string;
  contextSummary: string;
  response: string;
  userFeedback: "accepted" | "edited" | "rejected" | null;
  latencyMs: number;
}

LangSmith, Langfuse, whatever. The tool matters less than having something.

Step 3: Review every trace for the first week

Yes, every single one.

This is where you learn what's actually broken. Not what you assumed was broken.

I sit with clients and review traces together. Same patterns show up:

Wrong tool selection: Agent picked searchOrders when it should have picked searchShipments
Missing context: Agent couldn't answer because the right doc wasn't retrieved
Hallucinations: Agent made up data that doesn't exist
Premature stopping: Agent gave up too early
Slow responses: Anything over 10 seconds feels broken

Create a simple spreadsheet. Log every failure. Categorize them.

Run ID	Failure Type	Root Cause	Fix
abc123	Wrong tool	Vague tool name	Renamed function
def456	Hallucination	No source doc	Added missing doc
ghi789	Slow response	Too much context	Scoped retrieval

After one week, you'll have a clear picture. This spreadsheet becomes your roadmap.

Step 4: Fix perception before prompts

Here's the insight that saves teams weeks of wasted effort:

90% of early failures come from three sources:

Bad tool names and descriptions
Missing or wrong context
Retrieval pulling irrelevant docs

These aren't prompt problems. They're perception problems.

I tell clients: the agent can only do the right thing if it can see the right things.

// Before: I see this constantly
const tool = {
  name: "handleData",
  description: "Handles data operations"
}

// After: Clear enough for the model to reason about
const tool = {
  name: "createShipmentFromOrder",  
  description: "Creates a new shipment record from an existing order. Requires orderId. Returns shipmentId and tracking number."
}

One client renamed 12 tools in week one. Tool selection accuracy went from 60% to 87%. No prompt changes.

Step 5: Build evals from your failures

Don't build generic evals. Build evals from the specific failures you observed.

Every row in that failure spreadsheet becomes a test case.

// Example eval case from a real client failure
const evalCase = {
  id: "shipment-status-check",
  query: "What's the status of order 12345?",
  expectedTool: "getShipmentByOrderId",
  expectedBehavior: "Return actual status from database",
  failureWeObserved: "Agent said 'delivered' without checking",
  groundTruth: "in_transit"
}

One team I worked with had 47 eval cases after two weeks. All from actual user sessions. All testing things that actually broke.

Generic benchmarks tell you nothing. Failure-driven evals tell you everything.

Step 6: Measure the right things separately

This is where most teams lie to themselves.

They compute one accuracy number. "We're at 85%!" Leadership is happy. But 85% of what?

I push clients to measure these separately:

interface AgentMetrics {
  // Did we pick the right tool?
  toolSelectionAccuracy: number;

  // Did we retrieve relevant docs?
  retrievalRecall: number;

  // Did the final answer match ground truth?
  answerCorrectness: number;

  // Did we cite the right sources?
  groundingAccuracy: number;

  // Did the user accept the response?
  userAcceptanceRate: number;
}

You can have 95% tool selection and 40% answer correctness. That means retrieval or synthesis is broken.

You can have 90% answer correctness and 60% user acceptance. That means the answer is technically right but useless in practice.

Separate metrics tell you where to focus. One number tells you nothing.

Step 7: Expand slowly with permission gates

After 2 weeks with 3 users, you might be ready for 10.

Don't flip a switch. Add gates.

const canUseAgent = (user: User): boolean => {
  // Phase 1: Named early adopters
  if (ROLLOUT_PHASE === 1) {
    return earlyAdopters.includes(user.id);
  }

  // Phase 2: Specific teams
  if (ROLLOUT_PHASE === 2) {
    return user.team === "support" || user.team === "ops";
  }

  // Phase 3: Everyone
  return true;
}

Each phase should last at least a week. Each phase needs its own baseline metrics.

If metrics drop when you expand, you've found a gap. That's good. That's the system working.

Step 8: Watch for drift

The first week is not representative.

Early users are curious. They ask simple questions. They're forgiving.

By week 4, they're using it for real work. Queries get harder. Edge cases appear. Patience drops.

I tell clients to track metrics weekly, not just at launch:

Week 1: 87% tool accuracy, 72% answer correctness
Week 2: 85% tool accuracy, 75% answer correctness  
Week 3: 83% tool accuracy, 71% answer correctness
Week 4: 79% tool accuracy, 68% answer correctness  ← investigate

If metrics drift down, dig into traces. Usually it's new use cases, missing docs, or users learning to ask harder questions.

Step 9: Know when you're actually ready

I've seen teams ship too early and destroy trust. I've also seen teams wait forever and never ship.

Here's what ready looks like:

✅ Tool selection accuracy > 90%

✅ Answer correctness > 80%

✅ User acceptance rate > 75%

✅ p95 latency < 8 seconds

✅ No hallucinations in last 100 traces

✅ You've handled the top 10 failure modes

Not ready:

❌ Still finding new failure categories weekly

❌ Metrics vary wildly day to day

❌ Users work around the agent instead of using it

❌ You can't explain why it fails when it fails

The outcome when this works

Teams that follow this playbook:

Ship with confidence, not hope
Have real data to show leadership
Know exactly where to focus engineering effort
Build user trust instead of destroying it

The teams that skip steps end up with shelved projects and skeptical users. I've seen it enough times to know.

The checklist

Week 0: Instrument everything
Week 1: 3 users, review every trace, build failure spreadsheet
Week 2: Fix perception issues (tools, context, retrieval)
Week 3: Build evals from failures, establish baselines
Week 4: Expand to 10 users, new roles, new use cases
Week 5: Fix new failures, update evals
Week 6: Expand to full internal team
Week 7+: Monitor drift, harden edge cases
When metrics stabilize: Consider external rollout

The boring work is the real work. Instrument first. Start small. Review everything. Fix perception before prompts. Measure the right things separately. Expand slowly.

Your agent is only as good as your willingness to watch it fail and fix what you find.

If you're rolling out an AI product and want a second set of eyes on your approach, I help teams get this right. DM me on X or LinkedIn.

Stop Prompting. Start Engineering Perception.

Serhii Panchyshyn — Mon, 13 Apr 2026 23:43:02 +0000

I've watched teams spend weeks rewriting the same system prompt.

Different phrasings. More examples. Clearer instructions. The agent still picks the wrong tool. Still hallucinates. Still feels broken.

Then they rename six functions and accuracy jumps 30%.

This pattern shows up constantly. The model doesn't care how clever your prompt is. It cares about what it can see.

The problem I see everywhere

Teams treat prompts like magic spells. Say the right words, get the right output.

But agents aren't following instructions. They're making predictions based on everything in context. The tool names. The API responses. The error messages. The structure of your data.

That's perception. And it matters way more than your system prompt.

Most teams optimize the wrong layer. They iterate on prompts for weeks while their tool names are handleData and processRequest. The model has no chance.

Here are 10 patterns I've seen work across the past two years of helping teams build production agents 💪

1. Tool names are the real prompt

Bad tool names are invisible to the model.

I audit client codebases and find this constantly:

// ❌ The model has no idea what this does
async function handleRequest(data: unknown) { }

// ✅ Now it knows exactly when to use this
async function createShipmentFromOrder(orderId: string) { }

One client had 47 tools. Half had names like processData or executeAction. The model was guessing.

We renamed 12 functions. Tool selection accuracy went from 60% to 87%. No prompt changes.

2. Tool descriptions matter more than you think

The model reads descriptions to decide which tool to pick.

I tell clients: write descriptions like you're onboarding a new developer. Because you are.

// ❌ Vague description
const tool = {
  name: "searchRecords",
  description: "Search for records in the system"
}

// ✅ Specific description with constraints
const tool = {
  name: "searchShipments",
  description: "Search shipments by tracking number, origin, destination, or date range. Returns max 50 results. Use filters to narrow results before searching."
}

Specific descriptions reduce wrong tool selection by 30-40% in my experience.

3. Passing everything into context is lazy

I've reviewed architectures where teams dump entire conversation histories into context. 20 turns. 50 tool results. Everything.

The model drowns.

What works:

Last 3 turns by default
Relevant retrieved docs only
Structured summaries instead of raw data

Less context. Better decisions. Faster responses.

One team cut their context by 60% and saw answer quality improve. Counter-intuitive until you realize the model was distracted by noise.

4. Scoped retrieval beats broad retrieval

Early RAG implementations pull from everywhere. The whole knowledge base. 200+ docs. The model has no idea which ones matter.

I push clients toward module-level filtering. If someone asks about shipments, only retrieve shipment docs.

// ❌ Retrieve from everything
const docs = await retriever.search(query);

// ✅ Scope to relevant module
const docs = await retriever.search(query, { 
  module: detectModule(query),
  maxResults: 5 
});

Recall goes up. Hallucinations go down. Should be the default from day one.

5. Structured outputs prevent downstream chaos

If another agent or system consumes your output, structure it.

// ❌ Free text response
"I found 3 shipments that match. The first one is #12345 going to Chicago..."

// ✅ Structured response
{
  "shipments": [
    { "id": "12345", "destination": "Chicago", "status": "in_transit" },
    { "id": "12346", "destination": "Denver", "status": "delivered" }
  ],
  "total": 3,
  "hasMore": true
}

Unstructured responses compound errors. Each downstream consumer has to parse and guess. I've seen entire pipelines break because one agent returned prose instead of JSON.

6. Silent failures are invisible failures

The model can't fix what it can't see.

I audit error handling in every client codebase. Same pattern:

// ❌ Silent failure
if (!hasPermission) {
  return null
}

// ✅ Loud failure
if (!hasPermission) {
  return {
    error: "PERMISSION_DENIED",
    message: "User lacks 'shipments.create' permission",
    requiredPermission: "shipments.create",
    suggestedAction: "Request access from workspace admin"
  }
}

Explicit errors let the agent reason about what went wrong. And let you debug faster.

7. Real system state beats assumed state

I watched an agent confidently tell a user their shipment was delivered.

It wasn't. The agent assumed based on typical timelines. It never checked the actual record.

This happens when teams don't pass real state:

// ❌ Agent has to guess
const context = {
  orderId: "12345"
}

// ✅ Agent knows the truth
const context = {
  shipment: {
    id: "12345",
    status: "in_transit",  // actual current status
    lastUpdate: "2024-01-15T10:30:00Z",
    currentLocation: "Memphis hub"
  }
}

Agents will make up state if you don't give them real state. Always.

8. Specialized agents beat one generalist

I've seen teams try to build one agent that handles everything. Customer questions. Data entry. Workflow automation. Reports.

It's mediocre at all of them.

The pattern that works:

One agent for Q&A using org context
One agent for record operations with strict schemas
One agent for document extraction with specialized prompts

Each one is easier to eval. Easier to constrain. Easier to improve.

Generalist agents are harder to debug and harder to trust. I push clients toward decomposition early.

9. Guardrails should block bad things, not useful things

I've seen guardrails so aggressive they blocked legitimate business operations.

"Can you help me set up a webhook?" → BLOCKED (mentions code execution)

"What's the API endpoint for shipments?" → BLOCKED (mentions API)

The users stopped trusting the product. Not because the AI was bad. Because the guardrails were dumb.

Narrow guardrails work better. Be specific about what's actually dangerous. Allow everything else.

10. Audit perception before rewriting prompts

When a client tells me their agent is underperforming, I ask these questions first:

Can it see the right tools? Are names and descriptions clear?
Can it see the right context? Or is it drowning in noise?
Can it see real state? Or is it guessing?
Can it see errors? Or do failures happen silently?

Nine times out of ten, the problem is perception. Not the prompt.

The outcome when you get this right

Teams that engineer perception instead of prompts:

Stop the endless prompt iteration cycle
Get measurable accuracy improvements in days, not months
Build agents that actually work in production
Have clear debugging paths when things break

The teams that keep tweaking prompts stay stuck. I've seen it enough times to know.

The mental model shift

Prompt engineering asks: "How do I word this better?"

Perception engineering asks: "What does the agent need to see to make a good decision?"

One has diminishing returns after a few iterations.

The other compounds as your system improves.

Stop rewriting prompts. Start auditing what your agent can perceive.

Rename tools for clarity
Scope your context
Pass real state
Make errors loud
Use specialized agents

Your agent is only as good as what it can see 👀

If you're building agents and want a second set of eyes on your architecture, I help teams get this right. DM me on X or LinkedIn.

My First RAG System Had No Evals. 40% of Answers Were Wrong.

Serhii Panchyshyn — Mon, 13 Apr 2026 20:58:06 +0000

When I started building production RAG systems, I noticed something: nobody was measuring retrieval quality.

Teams would ship a system, ask users if it "felt good," and move on. No metrics. No baseline. No way to know if changes actually helped.

So I started measuring everything. And the first thing I discovered: most RAG failures aren't LLM failures. They're retrieval failures.

The documents that could answer the question aren't making it into the context window. The LLM is being asked to answer questions without the information it needs. No wonder it hallucinates.

Here's what I've learned about measuring and fixing RAG systems after building them for B2B SaaS companies.

The metric that actually matters: Recall@k

Before I measure anything else on a new RAG system, I measure Recall@k.

Recall@k answers a simple question: "Of all the documents that should have been retrieved, what percentage actually made it into the top k results?"

def recall_at_k(retrieved_ids: list, relevant_ids: list, k: int) -> float:
    """What % of relevant docs are in the top k results?"""
    top_k = set(retrieved_ids[:k])
    relevant = set(relevant_ids)

    if not relevant:
        return 1.0

    return len(top_k & relevant) / len(relevant)

On systems I've audited, Recall@10 is often around 60%. That means 40% of the time, the document that could answer the question isn't even in the context. The LLM never had a chance.

Here's the math that drives everything:

P(correct answer) ≈ P(correct context retrieved)

If the right chunks aren't retrieved, the LLM can't answer correctly. This is why I always measure retrieval separately from answer quality. Otherwise you're debugging the wrong layer.

You can start measuring today

You don't need production traffic to build evals. Generate synthetic test data from your corpus:

def generate_synthetic_evals(chunks: list) -> list:
    """Generate question-answer pairs from your chunks."""
    eval_pairs = []

    for chunk in chunks:
        response = llm.generate(f"""
Generate 3 questions that this text can answer.
Make them specific. "What is this about?" doesn't test retrieval.

Text:
{chunk.text}

Return JSON: [{{"question": "...", "chunk_id": "{chunk.id}"}}]
""")

        eval_pairs.extend(parse_json(response))

    return eval_pairs

50-100 questions is enough to establish a baseline. Run your retriever, measure Recall@10, write down the number. Now you can actually tell if changes help.

The two fixes that consistently move the needle

I've tried a lot of retrieval improvements. Most make marginal differences. Two consistently deliver results.

Fix 1: Hybrid search

Embeddings are great at semantic similarity. "How do I reset my password?" matches "Steps to recover account access" even though they share no keywords.

But embeddings are weak on:

Numbers: They don't understand that 49 is close to 50
Exact match: Product codes, IDs, ticker symbols
Rare terms: Domain jargon not in the training data

BM25 (keyword search) catches what embeddings miss. Combine them:

def hybrid_search(query: str, k: int = 10) -> list:
    """Combine embedding search and BM25 using RRF."""

    embedding_results = embedding_index.search(query, k=20)
    bm25_results = bm25_index.search(query, k=20)

    # Reciprocal Rank Fusion
    scores = {}
    rrf_k = 60

    for rank, doc_id in enumerate(embedding_results):
        scores[doc_id] = scores.get(doc_id, 0) + 1 / (rrf_k + rank + 1)

    for rank, doc_id in enumerate(bm25_results):
        scores[doc_id] = scores.get(doc_id, 0) + 1 / (rrf_k + rank + 1)

    ranked = sorted(scores.keys(), key=lambda x: scores[x], reverse=True)
    return ranked[:k]

Typical improvement: 5-15% recall boost depending on query mix.

Fix 2: Add a reranker

Embedding models are bi-encoders. They encode query and documents separately, then compare. Fast, but imprecise.

Cross-encoders (rerankers) look at the query and document together. Slower, but much more accurate. Use them as a second pass:

def search_with_rerank(query: str, k: int = 5) -> list:
    """Retrieve broadly, then rerank precisely."""

    # Cast a wide net
    candidates = hybrid_search(query, k=20)

    # Rerank with cross-encoder
    pairs = [(query, get_content(doc_id)) for doc_id in candidates]
    scores = reranker.score(pairs)

    # Return top k after reranking
    ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
    return [doc_id for doc_id, score in ranked[:k]]

Typical improvement: another 5-10% on top of hybrid search.

Combined, these two fixes often take a system from 60% to 80% recall. That's the difference between "works sometimes" and "works reliably."

Chunking decisions that make or break retrieval

Your chunking strategy matters more than your embedding model choice. A few things I always check:

The "it" problem

Chunks that start with "It also supports..." or "This feature allows..." are useless on their own. The word "it" has no meaning without the previous chunk.

Fix: Prepend context to every chunk.

def chunk_with_context(doc) -> list:
    chunks = []

    for section in doc.sections:
        # Prepend document and section info
        context = f"Document: {doc.title}\nSection: {section.header}\n\n"

        for chunk_text in split_section(section.content):
            chunks.append({
                "content": context + chunk_text,
                "metadata": {
                    "doc_title": doc.title,
                    "section": section.header
                }
            })

    return chunks

Other chunking rules I follow

Never split mid-table. A row without headers is meaningless.
10-20% overlap between consecutive chunks.
Test multiple chunk sizes (256, 512, 1024 tokens). Optimal depends on your queries.

The workflow I use on every RAG project

Week 1-2: Establish baseline

Parse documents (test multiple parsers for PDFs)
Chunk with context headers
Generate 50-100 synthetic eval questions
Build basic retriever
Measure Recall@10
Write down the number

Week 2-4: Apply standard fixes

Add hybrid search (BM25 + embeddings)
Add reranker
Measure again
Compare to baseline

Week 4+: Debug specific failures

Break down recall by query type
Find worst-performing segment
Fix that segment
Measure again

The key: measure after every change. If you can't see improvement in numbers, you're guessing.

When to measure answer quality

Only after retrieval is solid.

Once Recall@10 is above 80%, start measuring end-to-end:

def eval_answer(question: str, answer: str, context: list) -> dict:
    """Use LLM-as-judge for answer evaluation."""

    result = llm.generate(f"""
Evaluate this answer. Return JSON:
- correct: true/false (factually accurate)
- grounded: true/false (supported by the context)
- complete: true/false (addresses the full question)

Context: {format_context(context)}
Question: {question}
Answer: {answer}
""")

    return parse_json(result)

But if retrieval is broken, this eval is noise. You're just measuring how well your LLM fills in gaps it shouldn't have to fill.

The takeaway

RAG quality is retrieval quality.

Before you touch your prompts:

Generate synthetic evals from your corpus
Measure Recall@10
Add hybrid search
Add a reranker
Fix your chunking
Measure again

The fixes are straightforward. The impact is not.

This is Part 1 of a series on production AI systems. Next: how to know when to fix your prompts vs. build an evaluator.

About me

I help B2B SaaS companies ship production AI in 6 weeks.

If you're building RAG and want a second set of eyes, I do free AI Teardowns — a 30-45 min video showing exactly where your pipeline is breaking and how to fix it.

No pitch. Just clarity.

AI Implementation for B2B SaaS | AnimaNova Labs | AnimaNova Labs

Ship production AI features in 6 weeks. For B2B SaaS companies who need AI but can't hire fast enough. No $300K engineer. No 6-month timeline.

animanovalabs.com