Forem: capman

Capman: Stop Routing Every Intent Through an LLM

capman — Sun, 12 Apr 2026 10:31:26 +0000

Most AI apps send every user message to a language model — even when the intent is completely predictable. capman intercepts the 80% you already know how to answer, resolves them in under 2 ms at zero token cost, and lets the LLM handle only the queries that genuinely need it.

The Problem

When a user types "what time is it?" into your AI app, this is what happens:

User input
  → serialize message
  → HTTP request to LLM API     ← 800–1500 ms
  → token cost (~$0.001–$0.01)
  → parse function call JSON
  → invoke time handler

You paid an LLM to decode "what time is it" into a clock call. It knew the answer in 1 ms. You waited 1,200 ms anyway.

Standard function calling routes every query through the model — by design. For ambiguous, open-ended input, that's the right call. For well-defined intents like "check availability", "go to settings", or "get my orders", it's pure overhead.

What is capman?

capman is a TypeScript library that sits in front of your LLM. It reads a capability manifest — a machine-readable list of everything your app can do — and matches user queries against it using weighted keyword scoring.

If it finds a confident match: it resolves the query directly. No LLM. No network. No tokens.
If it doesn't: it escalates to your LLM automatically.

import { CapmanEngine, readManifest } from 'capman'

const engine = new CapmanEngine({
  manifest: readManifest(),
  mode: 'balanced',          // keyword-first, LLM only when needed
  llm: async (prompt) => callYourLLM(prompt),
})

const result = await engine.ask('Check availability for blue jacket')

console.log(result.match.capability?.id)  // 'check_product_availability'
console.log(result.resolvedVia)           // 'keyword' | 'llm' | 'cache'
console.log(result.trace.totalMs)         // 1.6

How It Works

Every engine.ask() call passes through six steps:

1. Cache check — normalized query key lookup. Hit → return in <1 ms.

2. Keyword scorer — every capability in your manifest is scored 0–100 across three weighted sources:

Source	Weight	Signal
Example sentences	60 pts	Strongest — direct intent overlap
Capability description	30 pts	Secondary — synonym coverage
Capability name	10 pts	Weakest — tiebreaker only

3. Branch decision — confidence ≥ 50%? Direct resolve. Below threshold? LLM escalation (rate-limited + circuit-breaker protected).

4. Privacy enforcement — public, user_owned, or admin checked before any API call fires.

5. Resolver — api (fetch + retry + timeout), nav (URL template), or hybrid (both in parallel).

6. Learning + trace — full ExecutionTrace returned with step-by-step timing. Pre-boost result recorded to prevent feedback loops.

Three Modes, One Decision

cheap      → keyword only, <2 ms, $0 — for high-volume known intents
balanced   → keyword first, LLM if <50% confidence — default
accurate   → LLM first, keyword fallback — for ambiguous open input

The mode is set per-engine instance. Switch by swapping the constructor option.

capman vs Standard Function Calling

The fundamental difference is when the LLM is involved:

	LLM Function Calling	capman
Latency	800–1,500 ms	<2 ms (direct)
Token cost	~$0.001–$0.01/call	$0 on direct match
Failure rate	~25% (hallucinations, timeouts)	0% on direct match
Privacy enforcement	Manual	Built-in per capability
Caching	External setup	Memory / File / Combo built-in
Tracing	Custom logging needed	Automatic on every call

capman doesn't replace your LLM — it protects it. The model gets fewer calls, better-framed prompts, and none of the deterministic work it was doing before.

Demo: The Weather Query

The demo runs a battle test: the same query submitted three times in a row, both engines running in parallel. Here's what came back.

The Query

"what is the weather"

Results

Round 1 — cold start:

LLM: 1,151.17 ms ✓ Success → correct tool
capman: 1.61 ms — matched "weather" exactly — 715× faster

Round 2 — learning kicks in (+1 pt boost):

LLM: 851.51 ms
capman: 0.59 ms — boost applied from prior hit — 1,443× faster

Round 3 — boost grows (+2 pts):

LLM: 931.02 ms
capman: 0.78 ms — 1,193× faster

Why Does the Latency Drop on Rounds 2 and 3?

The learning store tracks keyword → capability → hitCount. On each successful match, it builds a boost index. On the next call:

boost = min(15, log₂(hitCount + 1) × 2)

More hits → logarithmically larger boost → capability scores higher faster → match resolves sooner in the scorer loop. The cap is +15 points total. The pre-boost result is what gets recorded (not the post-boost result) — this prevents a feedback loop where boosted winners accumulate hits and permanently displace keyword matches.

A Capability Definition

Everything capman does flows from a structured manifest. Here's a full capability:

// capman.config.js
module.exports = {
  app: 'my-store',
  baseUrl: 'https://api.my-store.com',
  capabilities: [
    {
      id: 'check_product_availability',
      name: 'Check product availability',
      description: 'Check stock and pricing for a product by name or ID.',
      examples: [
        'Is the blue jacket in stock?',
        'Check availability for product 42',
        'Do you have size M?',
      ],
      params: [
        {
          name: 'product',
          description: 'Product name or ID',
          required: true,
          source: 'user_query',   // extracted from the query
        },
      ],
      returns: ['stock', 'price', 'variants'],
      resolver: {
        type: 'api',
        endpoints: [{ method: 'GET', path: '/products/{product}/availability' }],
      },
      privacy: { level: 'public' },
    },
  ],
}

Then:

npx capman generate   # → manifest.json
npx capman validate   # → check all capabilities
npx capman demo       # → live test with sample queries

What capman Does Not Replace

capman handles the routing layer. It doesn't:

Generate creative responses
Summarize or reason over content
Handle open-ended questions with no predictable shape
Replace conversational context management

For any of those, the LLM still runs — capman just makes sure it runs only when it needs to.

Install

npm install capman

npx capman init       # create capman.config.js
npx capman generate   # generate manifest.json
npx capman run "your query" --debug

GitHub: github.com/Hobbydefiningdoctory/capman

Summary

If your app has any predictable intents — commands, retrievals, navigations — you are currently paying LLM latency and token cost for work that a keyword scorer can do in 1 ms. capman is the layer that captures that saving without changing your LLM integration.

The LLM becomes a fallback, not a default. That's the shift.

capman v0.4.5 — TypeScript, MIT licence, dual CJS/ESM, zero runtime dependencies beyond zod.

I built a tool that lets AI agents interact with your app without navigating the UI

capman — Mon, 30 Mar 2026 19:43:21 +0000

When I started thinking about how AI agents interact with applications, something bothered me.

Today, when an AI agent needs to answer "are there seats available for Friday?" — it navigates your app like a tourist with no map:

AI clicks → Home → Explore → Events → Category → Availability

That's slow, wasteful, and it exposes parts of your app the AI was never supposed to see.

So I built capman — a Capability Manifest Engine that gives AI agents a structured map of what your app can do, and shows you exactly why it made every decision.

The core idea

Your app publishes a capability manifest — a machine-readable list of everything it can do, what API to call, and what data scope is allowed.

// capman.config.js
module.exports = {
  app: 'my-app',
  baseUrl: 'https://api.my-app.com',
  capabilities: [
    {
      id: 'check_availability',
      name: 'Check availability',
      description: "'Check if a product or slot is available on a given date.',"
      examples: [
        'Are there seats available Friday?',
        'Check availability for blue jacket',
      ],
      params: [
        { name: 'item',    description: "'Item name or ID', required: true,  source: 'user_query' },"
        { name: 'date',    description: "'Date to check',   required: false, source: 'user_query' },"
      ],
      returns: ['available', 'count', 'price'],
      resolver: {
        type: 'api',
        endpoints: [{ method: 'GET', path: '/availability/{item}' }],
      },
      privacy: { level: 'public' },
    },
  ],
}

The AI reads this manifest and goes directly to the answer — no navigation, no guessing.

The part I'm most proud of: the execution trace

Every query returns a full execution trace. No black box.

const engine = new CapmanEngine({ manifest, baseUrl: 'https://api.my-app.com' })
const result = await engine.ask('Are there seats available Friday?')

console.log(result.trace)
// {
//   query: 'Are there seats available Friday?',
//   candidates: [
//     { capabilityId: 'check_availability', score: 100, matched: true  },
//     { capabilityId: 'get_orders',         score: 12,  matched: false },
//     { capabilityId: 'navigate_to_screen', score: 0,   matched: false },
//   ],
//   reasoning: [
//     'Matched "check_availability" with 100% confidence',
//     'Rejected: get_orders (12%)',
//     'Resolved via: keyword',
//     'Extracted params: item=seats, date=Friday',
//   ],
//   steps: [
//     { type: 'cache_check',   status: 'miss', durationMs: 0 },
//     { type: 'keyword_match', status: 'pass', durationMs: 1, detail: 'confidence: 100%' },
//     { type: 'privacy_check', status: 'pass', durationMs: 0, detail: 'level: public'    },
//     { type: 'resolve',       status: 'pass', durationMs: 2, detail: 'via api'          },
//   ],
//   totalMs: 4,
// }

This answers the question every AI developer eventually asks:

"Why did it pick this capability?"
"Why wrong parameters?"
"Why did it fail?"

Today with LangChain, OpenAI function calling, or custom agents — there's no clear answer. Logs are scattered, debugging is painful.

capman makes AI execution debuggable like backend code.

Three matching modes — control cost vs accuracy

// cheap — keyword only, free, fast
const engine = new CapmanEngine({ manifest, mode: 'cheap' })

// balanced — keyword first, LLM fallback if confidence < 50% (default)
const engine = new CapmanEngine({ manifest, mode: 'balanced', llm: myLLM })

// accurate — LLM first, always correct
const engine = new CapmanEngine({ manifest, mode: 'accurate', llm: myLLM })

The LLM function is plug-and-play — works with Anthropic, OpenAI, or any model:

const engine = new CapmanEngine({
  manifest,
  mode: 'balanced',
  llm: async (prompt) => {
    const res = await anthropic.messages.create({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 500,
      messages: [{ role: 'user', content: prompt }],
    })
    return res.content[0].text
  },
})

Privacy enforcement — per capability, not per request

{
  id: 'get_my_orders',
  privacy: { level: 'user_owned' },  // blocked without auth
}

{
  id: 'get_admin_report',
  privacy: { level: 'admin' },       // blocked without admin role
}

Pass auth context once — capman enforces it before resolution:

const engine = new CapmanEngine({
  manifest,
  auth: {
    isAuthenticated: true,
    role: 'user',
    userId: 'user-123',  // auto-injected into session params
  },
})

See it live in 30 seconds

npx capman demo

That runs a live demo against a sample e-commerce app — no config needed. You'll see matched capabilities, execution traces, candidate scores, and API calls constructed in real time.

Or run a query against your own manifest:

npx capman init          # creates capman.config.js
npx capman generate      # generates manifest.json
npx capman run "are there seats available Friday?" --debug

Get started

npm install capman

GitHub: github.com/Hobbydefiningdoctory/capman
npm: npmjs.com/package/capman

What I'm working on next

The v0.5.0 roadmap includes:

Wiring the learning index back into the keyword matcher — usage data improves matching over time
CapmanEngine.explain(query) — understand why a query would match without executing
Redis adapter for multi-instance deployments
mode: 'adaptive' — uses learning data to boost confidence on proven patterns

If you're building AI agents and you've ever stared at logs wondering why the AI did what it did — capman is for you.

Would love feedback. What would make this useful for your stack?