Forem: Arnold Cartagena

npm Trusted Publishing with GitHub Actions OIDC — What the Docs Don't Tell You (Scoped Packages)

Arnold Cartagena — Thu, 26 Mar 2026 22:29:18 +0000

After the recent npm supply chain attacks, long-lived tokens are out. Trusted publishing via OIDC is the way forward. But if you maintain scoped packages (@org/package), you're going to hit some walls the docs don't warn you about.

I spent a full day getting @edictum/openclaw to publish via trusted publishing from GitHub Actions. Here's everything I learned so you don't have to.

The Setup

I maintain Edictum, a runtime contract enforcement library for AI agents. We just shipped a native OpenClaw plugin and needed to publish @edictum/openclaw to npm from GitHub Actions using trusted publishing — no tokens, no secrets.

What Worked

Here's the final working workflow:

name: Publish to npm

on:
  release:
    types: [published]

permissions:
  id-token: write
  contents: read

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5

      - uses: pnpm/action-setup@v4

      - uses: actions/setup-node@v5
        with:
          node-version: 22
          registry-url: https://registry.npmjs.org
          cache: pnpm

      - run: pnpm install --frozen-lockfile

      # THIS IS CRITICAL — bundled npm doesn't support OIDC for scoped packages
      - run: npm install -g npm@latest

      - run: pnpm build
      - run: pnpm test

      - run: npm publish --provenance --access public

And in package.json:

{
  "publishConfig": {
    "access": "public",
    "provenance": true,
    "registry": "https://registry.npmjs.org/"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/edictum-ai/edictum-openclaw.git"
  }
}

The Problems (and Fixes)

1. Bundled npm doesn't support OIDC for scoped packages

This is a known issue. The npm version bundled with Node.js 22 on GitHub-hosted runners is too old. You must upgrade:

- run: npm install -g npm@latest

The npm docs say you need npm 11.5.1+. Without this, you get:

npm error code E404
npm error 404 Not Found - PUT https://registry.npmjs.org/@scope%2fpackage - Not found

This E404 is misleading — the package exists, but the old npm can't do the OIDC token exchange for scoped packages.

2. `actions/setup-node` injects a default token that breaks OIDC

When you use registry-url with actions/setup-node, it automatically sets NODE_AUTH_TOKEN to ${{ github.token }}. This GitHub token overrides the OIDC flow, and npm tries to authenticate with it instead of doing the OIDC exchange.

Some people clear it with NODE_AUTH_TOKEN: "", but that breaks authentication entirely (ENEEDAUTH). Others remove registry-url, but then npm doesn't know where to publish.

The fix that actually works: upgrade npm (step 1 above). The upgraded npm correctly handles the OIDC flow even with the injected token present. The --provenance flag triggers the OIDC path explicitly.

3. The environment field on npmjs.com must match (or be empty)

When configuring the trusted publisher on npmjs.com, there's an "Environment name" field. If you set this to npm or release, the GitHub Actions job must run in a matching GitHub environment.

What tripped me up: I set the environment name on npmjs.com to npm and added environment: npm to my workflow job. It still failed with E404. Removing the environment name from npmjs.com (leaving it blank) fixed it immediately.

If you need environment-based protection (approval gates, etc.), make sure the names match exactly — case-sensitive, no trailing spaces. But if you don't need it, leave it blank.

4. `--provenance` is NOT automatic (despite what docs say)

The npm docs state:

When you publish using trusted publishing, npm automatically generates and publishes provenance attestations. You don't need to add the --provenance flag.

This was not my experience. Publishing without --provenance resulted in ENEEDAUTH. Adding it fixed the issue. You can also set it in package.json:

{
  "publishConfig": {
    "provenance": true
  }
}

5. `repository.url` must match your GitHub repo exactly

The trusted publisher config on npmjs.com requires your org/repo. Your package.json must agree:

{
  "repository": {
    "type": "git",
    "url": "git+https://github.com/your-org/your-repo.git"
  }
}

If these don't match, npm silently rejects the publish with a 404.

How Big Projects Handle This

Curious how others do it, I checked several major open-source projects:

Project	Method
Vercel AI SDK	`NPM_TOKEN` secret (token-based)
LangChain.js	`NPM_TOKEN` secret + manual `.npmrc`
OpenClaw	OIDC trusted publishing (non-scoped package)
shadcn/ui	`NPM_TOKEN` secret

Most major projects still use token-based publishing. The ones using OIDC successfully tend to be non-scoped packages. Scoped packages with OIDC are still rough — npm/cli#8976 is open as of today.

The Checklist

If you're setting up trusted publishing for a scoped npm package from GitHub Actions:

[ ] npm 11.5.1+ (npm install -g npm@latest in your workflow)
[ ] permissions: id-token: write in your workflow
[ ] registry-url: https://registry.npmjs.org in actions/setup-node
[ ] --provenance flag on npm publish
[ ] repository.url in package.json matches your GitHub repo
[ ] publishConfig.access: "public" in package.json (for scoped packages)
[ ] Trusted publisher configured on npmjs.com → package → Settings
[ ] Environment name on npmjs.com: leave blank unless you specifically need it
[ ] No NODE_AUTH_TOKEN secret set in your repo (it would override OIDC)

The Result

@edictum/openclaw is now published with OIDC trusted publishing and provenance attestations, no long-lived secrets anywhere:

npm i @edictum/openclaw

Or if you're an OpenClaw user:

openclaw plugins install @edictum/openclaw

One command, zero code changes, 25 security contracts active. Check it out: github.com/edictum-ai/edictum-openclaw

Edictum is runtime contract enforcement for AI agent tool calls. Deterministic YAML contracts that execute outside the model — the LLM can't talk its way past them. Available in Python, TypeScript, and Go.

AI Monoculture: When Every Engineer Has the Same Architect

Arnold Cartagena — Mon, 16 Mar 2026 17:40:38 +0000

Vibe coding is technical debt at scale. The rewrite is coming.

I was asking myself this question for quite a while, it's like we all have the same Architect, so I ran an experiment across six frontier AI models to see what happens when you ask them to design software architecture versus when you ask them to help build the project.

144 prompts. Four different systems. Three prompt styles.

When asked to design architecture, the models recommended Rust, Aerospike, ClickHouse, event streaming systems.

When asked to help build the same systems, those choices vanished.

Rust dropped from 43 recommendations to zero. SQLite appeared 12 times. TypeScript jumped to 25.

Same models. Same problems.

The only difference was the prompt: "design it" versus "help me build it."

And that difference could explain why so many AI-built apps end up with the exact same stack.

This is already happening and most people don't see it

You already know what this looks like. You've seen it. You might be building one right now.

Next.js + Supabase. Auth? Supabase. Database? Supabase. Real-time? Supabase. File storage? Supabase. The DX is smooth, the deploy is one click, and Vercel has done genuinely excellent work making it frictionless. Credit where it's due.

But when you look at a thousand AI-assisted projects and they all have the same skeleton, that's not a thousand engineers independently arriving at the same conclusion. That's one model's opinion, amplified a thousand times. It's an AI monoculture — and it's everywhere.

The thing is, most people building with AI right now don't realize they're producing the same architecture as everyone else. Each project feels like a fresh start. The assistant gives you a clean scaffold, the code works, the structure looks thoughtful. It doesn't feel like a default. It feels like a decision.

That's the core problem. Not that the default stack is bad — it's often perfectly fine. But that thousands of engineers are adopting it without realizing it was never chosen. The AI chose it for them, based on what it generates most fluently, and presented it with enough confidence that nobody questioned it.

The scaffold picks your architecture

Most people don't sit down and say "design me an architecture for 500,000 requests per second." Not because they lack the skill — but because they don't have those numbers yet. They're figuring out the product. So they say:

"I want to build a bidding platform, help me get started."
"I need an IoT dashboard. Set up the project and write the ingestion endpoint."
"Help me build a crypto exchange. Start with the order placement API."

Feature by feature. Conversationally. And at each step, the assistant picks the stack it can scaffold fastest — TypeScript, PostgreSQL, React — because that's what it generates most fluently. Each individual feature is small enough that the default stack is "fine." The model never encounters a moment where it has to say "wait, this won't work."

The architecture gets locked in at the first npx create-next-app and never gets revisited. Not because anyone chose it. Because nobody stopped to choose.

The models know better. They just don't act on it.

This is the part that surprised me. The crucial split wasn't between models or even between problems. It was between architecture mode and build mode.

Across architecture-mode prompts, the models gave genuinely good answers. Rust for the bid engine. Aerospike for feature lookup. Kafka for event streaming. They discussed lock-free data structures and kernel-bypass networking. Across Variants A and B, Rust was the primary backend language in 45% of recommendations. PostgreSQL was the primary database in 53% — but many of those designs still paired it with specialized stores.

Then I switched to build mode. Same problems, same models, same underlying domains — but now the prompt was "help me build this."

That's where everything collapsed. Across the benchmark, Rust fell from 43 architecture-mode recommendations to zero in build mode. TypeScript jumped from 2 to 25. Python went from roughly absent to 15. SQLite went from 0 to 12. ClickHouse, Aerospike, and ScyllaDB disappeared entirely. The stack novelty score — a measure of how many components fall outside the default web-app stack — dropped from 11.2 to 4.0.

The models know the right answer. They proved it thirty minutes earlier. They just don't use it when you say "help me build."

The monoculture is invisible from the inside

This is what makes it different from previous waves of stack convergence. When Rails was everywhere in 2010, people knew they were choosing Rails. They could name their stack, defend it, argue about it. The choice was visible.

With AI-assisted development, the choice is invisible. You didn't pick TypeScript + PostgreSQL. You asked for help building something and that's what appeared. It feels bespoke because the AI generated it specifically for your project. But the architecture underneath is the same one it generates for every project, because it's optimizing for scaffold fluency, not for your problem.

That's why the monoculture grows without anyone noticing. Each developer thinks they're building something custom. Nobody looks around and realizes the foundation is identical across thousands of products.

The AI didn't invent this. It industrialized it.

I should be fair — I use AI assistants daily and they've made me dramatically more productive. This isn't an argument against AI-assisted development. It's a field note from inside the machine.

A lot of this convergence existed before AI. Hiring availability, library ecosystems, managed cloud support, tutorials, startup pressure to ship — all of these already pushed teams toward the boring stack. Even without AI, many teams would choose Python + Postgres + React.

But before AI, that choice involved friction. You had to read docs, compare options, talk to other engineers, live with uncertainty. That friction was annoying, but it was productive. It forced architecture to be a thinking exercise.

AI removes the friction. And what gets lost isn't speed — it's the argument. The internal debate that used to happen before a stack was chosen now never occurs, because a complete, working scaffold appears before you have time to doubt.

The boring stack is often the right stack. The danger is when it's chosen by the scaffold instead of by the engineer.

I'm not immune

I saw this in my own work while building Edictum, my open-source runtime governance framework for AI agents. I'm a platform engineer. I run Kafka clusters and Kubernetes infrastructure professionally. I hold a Kubestronaut certification. I know what production event-driven systems look like.

And still, when I sat down to build with AI assistance, I shipped FastAPI + React + PostgreSQL. The exact default. Not because I chose it after evaluation, but because the AI made it effortless and I was optimizing for speed.

A prospective client asked me how I was thinking about scale. And I started talking about Kafka, event ingestion pipelines, ClickHouse for analytics. That's what I do professionally. Then I looked at what I'd actually built and realized none of that knowledge was in the product. The AI had scaffolded a perfectly functional FastAPI monolith, and I'd let it.

The model doesn't make you worse. It makes you generic. And you don't notice until someone asks a question that forces you to look at what you actually built versus what you know.

What this means

Right now, thousands of products are being built on the same invisible foundation. The same databases for the wrong workloads. The same frameworks for problems they weren't designed for. The same patterns, the same dependencies, the same scaling ceilings — all chosen by AI assistants optimizing for fluency rather than fitness.

When those products hit scale, the rewrites will come. And the fix is never incremental. You don't "add Kafka" to a Next.js app. You don't bolt ClickHouse onto Supabase. You don't retrofit event sourcing into a CRUD scaffold. You rewrite.

The time saved by not having the architecture argument up front gets repaid with interest. And the interest rate is brutal, because by then you have users, data, integrations, and a team that learned the wrong patterns.

The AI monoculture is real. It's growing. And most people building inside it have no idea they're there.

The question every founder and tech lead should ask themselves:

Would I have built it this way if I had to justify every decision to a staff engineer?

If the answer is no, you don't have an architecture. You have a default.

The data

I tested GPT-5.4, Claude Opus 4.6, Claude Sonnet 4.6, Kimi K2.5 (Moonshot AI), GLM-5 (Zhipu AI), and MiniMax M2.5 across four problem briefs: real-time bidding platform (AdTech), IoT telemetry platform (Industrial IoT), government benefits portal (GovTech — the control, where a boring stack is actually correct), and cryptocurrency exchange (FinTech).

Each problem was prompted three ways:

Variant A — "Design the architecture." Open-ended.
Variant B — "Design the architecture for these hard requirements." With specific numbers.
Variant C — "Help me build this. Set up the project, pick the stack, write the scaffolding." The vibe coding prompt.

Six models × four briefs × three variants × two temperatures = 144 completions. Responses were parsed using Claude Sonnet 4.6 as a structured extraction layer, validated manually on a 10% sample.

The numbers

Variants A and B (architecture mode, 96 responses):

PostgreSQL as primary DB: 53%
Rust as primary language: 45%
React correctly omitted: 64%
Models chose Aerospike, ScyllaDB, TimescaleDB, ClickHouse for the right problems
Mean stack novelty score: 11.2

Architecture mode (A+B) → Build mode (C):

Rust: 43 → 0
TypeScript: 2 → 25
Python: 1 → 15
SQLite: 0 → 12
ClickHouse, Aerospike, ScyllaDB: all → 0
Mean stack novelty score: 11.2 → 4.0

Raw data, runner code, and extraction pipeline: github.com/acartag7/ai-monoculture

My AI agent pushed directly to main. The system prompt said don't.

Arnold Cartagena — Sun, 08 Feb 2026 15:52:41 +0000

I was demoing my AI agent to colleagues. The agent had access to Git tooling, and my carefully crafted system prompt was clear: create a branch, open a PR, never push directly to the repo.

The agent pushed directly to main.

I tried rewording the prompt. I tried being more explicit. I tried few-shot examples. The agent pushed to main again — because when an LLM decides something is "the fastest way to help," your prompt is a suggestion it can override.

I had no way to block that tool call. No mechanism between "the LLM decided to do this" and "the tool executed." I needed something at that boundary — deterministic, not probabilistic. Something the LLM couldn't talk its way past.

My first attempt was hardcoded Python — regex patterns matching against bash command strings, wired into the SDK's hook system. It worked, but the patterns were buried in code, untestable without spinning up the agent, and impossible for anyone outside my team to review or modify.

So I built Edictum to turn that approach into declarative, testable, framework-agnostic contracts.

What it does

Edictum sits between your agent and its tools. When your agent decides to call a tool, Edictum evaluates the call against YAML contracts before it executes. If the contract says deny, the call never happens. The LLM never gets a chance to argue.

apiVersion: edictum/v1
kind: ContractBundle
metadata:
  name: git-safety-policy
defaults:
  mode: enforce
contracts:
  - id: block-push-to-main
    type: pre
    tool: Bash
    when:
      args.command: { matches: 'git\s+push\s+.*main' }
    then:
      effect: deny
      message: "Direct push to main blocked. Use a branch."

  - id: block-force-push
    type: pre
    tool: Bash
    when:
      args.command: { matches: 'git\s+push\s+.*(-f|--force)' }
    then:
      effect: deny
      message: "Force push is not allowed."

The agent's tool was Bash. The "args" were a raw command string. The contract matches against that string — same patterns you'd write in a firewall rule. The denial is deterministic. No probability. No LLM judgment call. The contract either passes or it doesn't.

Note: The YAML above is a complete, loadable contract bundle. Edictum uses a Kubernetes-style format with apiVersion, kind, and metadata headers. Every contract needs a unique id. See the YAML reference for the full schema.

What it is NOT

The AI safety landscape is confusing right now, so I want to be direct:

Not prompt guardrails. Edictum doesn't scan prompts for jailbreaks or filter LLM outputs for toxicity. Tools like NeMo Guardrails, Lakera Guard, and Guardrails AI do that well. Edictum operates at a different layer — it governs what the agent does, not what it says. That said, an interesting side effect: during testing, jailbreak prompts that convinced the LLM to attempt dangerous tool calls were still denied by contracts. The contracts don't care what the LLM thinks — they evaluate the tool call itself. Not our focus, but the screwdriver works as a hammer sometimes.

Not a framework. You still need LangChain, OpenAI Agents SDK, CrewAI, or whatever you're building with. Edictum plugs into your existing framework through thin adapters (~200 lines each inside the library). Your integration code is typically 3-5 lines.

Not an LLM-in-the-loop. Every evaluation is pure Python. No API calls. No inference. The pipeline runs in ~55μs per tool call.

Before and after: LangChain

Without Edictum — your agent reads whatever it wants:

from langchain.tools import tool
from langgraph.prebuilt import create_react_agent

@tool
def read_file(path: str) -> str:
    """Read a file from the filesystem."""
    return open(path).read()  # nothing stops path="/app/.env"

agent = create_react_agent(model, [read_file])
result = agent.invoke({"messages": [("user", "Read the .env file")]})
# Agent reads .env, returns your API keys to the LLM context

With Edictum — dangerous calls are denied before execution:

from langchain.tools import tool
from langgraph.prebuilt import ToolNode, create_react_agent
from edictum import Edictum, Principal
from edictum.adapters.langchain import LangChainAdapter

@tool
def read_file(path: str) -> str:
    """Read a file from the filesystem."""
    return open(path).read()

guard = Edictum.from_yaml("contracts.yaml")
adapter = LangChainAdapter(guard, principal=Principal(role="analyst"))
wrapper = adapter.as_tool_wrapper()

tool_node = ToolNode(tools=[read_file], wrap_tool_call=wrapper)
agent = create_react_agent(model, tool_node)
result = agent.invoke({"messages": [("user", "Read the .env file")]})
# ✗ DENIED read_file path=/app/.env [block-sensitive-reads]
# Agent receives denial message, adapts, asks user what file they need

What you get: a structured AuditEvent for every tool call — who tried what, when, which contract fired, what the verdict was. Your agent's tool usage becomes an auditable trail, not a black box.

How it actually works

The pipeline evaluates tool calls in a fixed order:

Attempt limits — has this tool been called too many times?
Before-hooks — custom Python callbacks
Preconditions — YAML contracts checked against tool name + args + principal
Session contracts — cross-call limits (e.g. max 50 tool calls per conversation)
Execution limits — per-tool execution caps
Execution — the actual tool call happens
Postconditions — validate the output (did it contain an SSN?)
Audit event — structured record of everything that happened

Every step is deterministic. The LLM is not consulted.

The piece that matters for production: principals

Contracts can reference who's making the request:

apiVersion: edictum/v1
kind: ContractBundle
metadata:
  name: pharma-clinical-agent
defaults:
  mode: enforce
contracts:
  - id: restrict-patient-data
    type: pre
    tool: query_patients
    when:
      not:
        principal.role: { in: [pharmacovigilance, admin] }
    then:
      effect: deny
      message: "Role {principal.role} cannot access patient records"

Your application creates the principal:

from edictum import Principal

principal = Principal(
    role="researcher",
    claims={"department": "oncology"},
    ticket_ref="JIRA-456"
)

Today the library trusts what your application passes. There's an open question about whether identity verification should live in the library or stay in the application layer — both approaches have tradeoffs. For now, the design gives you principal-aware policies without prescribing how you verify identity. The roadmap includes server-side JWT/OIDC verification for teams that want the trust boundary inside Edictum rather than outside it.

Observe mode

This comes from my background in networking. When you configure new firewall rules, you don't apply them blindly to production. You put them in monitor mode, watch the traffic, verify the rules match what you expect, then flip to enforce.

Same idea:

guard = Edictum.from_yaml("contracts.yaml", mode="observe")

In observe mode, violations are logged but calls proceed normally. You see what would be denied without breaking anything. Run for a week, review the audit trail, fix false positives, flip to enforce. Zero-risk policy deployment.

CLI: contracts as testable artifacts

# Validate YAML syntax and schema
edictum validate contracts.yaml

# Run precondition test cases
edictum test contracts.yaml --cases tests.yaml

Put edictum test in CI. Your security policies become versioned, tested, reviewable artifacts — not buried in prompt templates.

Note: edictum test evaluates preconditions against your test cases. For full end-to-end testing including postconditions and session limits, use the Python API directly.

What it doesn't do (yet)

I want to be honest about where the edges are:

Single-process only. Session counters live in-memory. If you have multiple agent instances, each tracks its own counters independently. A central policy server is planned but not built.
No PII detection built-in. The protocol is defined (v0.6.0) — you can plug in your own detector. Built-in regex and Presidio-based detectors are coming.
No production sinks beyond file. Audit events go to stdout or .jsonl files. Webhook, Splunk, and Datadog sinks are planned.
OpenTelemetry is early. Span instrumentation exists but isn't battle-tested in production yet. It's opt-in and no-op if the OTel SDK isn't installed.
No hot-reload. Contracts are loaded at startup. Changing them requires a restart.

The roadmap shows what's planned and when.

The landscape — where Edictum fits

Tool	What it does	Layer
NeMo Guardrails	Programmable dialog flows, content safety, jailbreak detection	Prompt/response
Guardrails AI	Output validation, schema enforcement, hallucination detection	LLM output
Lakera Guard	Prompt injection detection, PII scanning	Input/output proxy
LlamaGuard	Safety classification of conversations	Content classification
Edictum	Contract enforcement on tool calls — preconditions, postconditions, session limits, audit	Tool execution boundary

These tools are complementary, not competing. You can run Lakera on prompts AND Edictum on tool calls. Different layers, different threats.

Performance

The governance pipeline adds ~55μs per tool call. That's measured, not estimated. For context, a typical LLM API call takes 500ms-3s. Edictum's overhead is invisible.

Zero runtime dependencies in core. YAML parsing, adapters, CLI, and OTel are optional extras — install only what you need.

Try it

pip install edictum

GitHub: github.com/acartag7/edictum
Docs: docs.edictum.dev

If you're deploying agents that touch production systems — files, databases, APIs, infrastructure — I'd genuinely like to hear how you're handling the gap between "the LLM decided to call a tool" and "the tool executed." That's the layer Edictum was built for.

Edictum is MIT licensed. Built during recovery from liver surgery because apparently I can't sit still. Feedback, issues, and PRs welcome.

Why Your LLM Returns “Sure! Here’s the JSON” and How to Fix It

Arnold Cartagena — Tue, 03 Feb 2026 15:39:26 +0000

You ask for JSON. The LLM returns:

Sure! Here's the JSON you requested:
{"name": "test", "value": 42}
Let me know if you need anything else!

Your parser crashes. Your RAG/Agentic pipeline fails (or worse: gets swallowed behind a generic infinite retry handler). You add more prompt engineering. It works 90% of the time. The other 10%? You're debugging infinitely wondering which of your 12 nodes broke. You didn't want those "Sure! Here's the JSON you requested" or the "Let me know if you need anything else!", I just wanted json.

I had this a lot when trying to get a consistent output of LLMs.

I thought this was just me.

The Pattern

Most teams shipping/testing LLM features run into some version of this:

You ask for JSON, you get "Sure! Here's the JSON you requested:"
The JSON has trailing commas, single quotes, or gets truncated
json.loads() fails with "line 1 column 47" — low-context at best
You retry, but the LLM makes the same mistake
You add prompt engineering. It works 90% of the time. The other 10%...

This prompt engineering part is really a pain to do, multiple versions of the prompt when the problem can actually be solved in other ways.

Search any LLM framework's issues for "JSON" or "ValidationError". The problem shows up across models and frameworks. The solutions are scattered across docs, GitHub issues, and custom workarounds.

There are really two failures here: parsing (turning text into JSON) and validation (ensuring the JSON matches what your pipeline expects). handoff-guard handles both, plus retries with feedback.

Why LLMs Do This

LLMs are trained to be helpful. When you ask for JSON, they want to:

Acknowledge your request ("Sure!")
Explain what they're giving you ("Here's the JSON:")
Format it nicely (markdown code blocks)
Offer follow-up help ("Let me know if...")

This is great for chat. It's terrible for parsing.

And it gets worse:

Truncation: Hit the token limit? Your JSON ends mid-string: {"draft": "This is a long article about...
Malformed syntax: Trailing commas, single quotes, unquoted keys. All common LLM outputs
Nested code blocks: JSON containing ``` characters breaks regex-based parsers

Common Approaches (and Their Tradeoffs)

"Just use JSON mode" — JSON/structured-output modes help when available, but they guarantee syntax, not schema. You still get validation errors, truncation, and no framework-level context like "which node failed."

"Use OutputFixingParser" — LangChain's output-fixing pattern repairs by calling the LLM again—adding latency and cost for every error. Its recommended usage has also shifted across LangChain versions.

"Use Instructor" — Powerful for structured generation across many providers. When it fixes errors, it usually does so by re-prompting the LLM. If you want fast, local repair without burning more tokens, you need a post-processor.

"Use Outlines" — Great for constrained decoding, but requires control over the inference server (e.g., vLLM). It doesn't help if you're calling a closed API like OpenAI or Anthropic.

"Add more prompt engineering" — You're playing whack-a-mole. Fix one edge case, another appears.

What I Built Instead

I needed something that:

Works with raw text output from any provider (post-hoc, not constrained generation)
Identifies which node failed (not just "validation error")
Retries with feedback (tells the LLM what went wrong)
Repairs common syntax issues locally (without calling the LLM again)
Stays lightweight (no embeddings, no ML, just parsing)

So I built handoff-guard.

Before


python
def writer_agent(state: dict) -> dict:
    response = call_llm("Return JSON with: draft, word_count, tone")

    # Hope it's valid JSON
    try:
        data = json.loads(response)
    except json.JSONDecodeError:
        # Which node? What failed? Can the agent retry?
        raise

    # Hope it matches the schema
    try:
        validated = WriterOutput(**data)
    except ValidationError:
        # "1 validation error for WriterOutput" — thanks for nothing
        raise

    return data

After


python
from handoff import guard, retry, parse_json  # PyPI: handoff-guard
from pydantic import BaseModel, Field

class WriterOutput(BaseModel):
    draft: str = Field(min_length=100)
    word_count: int = Field(ge=50)
    tone: str

@guard(output=WriterOutput, node_name="writer", max_attempts=3)
def writer_agent(state: dict) -> dict:
    prompt = "Return JSON with: draft, word_count, tone"

    if retry.is_retry:
        prompt += f"\n\nPrevious attempt failed:\n{retry.feedback()}"

    response = call_llm(prompt)
    return parse_json(response)  # Strips wrappers, repairs syntax

If it fails after 3 attempts:



HandoffViolation in 'writer':
  Contract: output
  Field: draft
  Expected: String should have at least 100 characters
  Received: 'Too short...' (str)
  Suggestion: Increase the length of 'draft'

For logs/telemetry, access e.total_attempts, e.history, or e.to_dict().

What `parse_json` Actually Does


python
from handoff import parse_json

# Strips conversational wrappers
obj = parse_json('Sure! Here\'s the JSON:\n{"key": "value"}\nLet me know!')
# -> Python dict/list (parsed JSON), not a JSON string

# Handles common syntax issues (via json-repair)
parse_json('{"a": 1,}')        # trailing comma → {"a": 1}
parse_json("{'a': 1}")         # single quotes → {"a": 1}
parse_json('{a: 1}')           # unquoted keys → {"a": 1}
parse_json('{"a": 1 // comment}')  # JS comments → {"a": 1}

# Detects truncation (v0.2.1)
result = parse_json('{"draft": "long text...', detailed=True)
# -> ParseResult with .data (dict), .truncated (bool), .repaired (bool)
result.truncated  # True — best-effort signal (unmatched braces detected)
result.repaired   # True — json-repair path was used successfully

No LLM calls. No embeddings. Deterministic parsing with best-effort repair. I haven't published benchmarks; this was built from real failure modes in my own graphs.

Why Not Instructor/Outlines?

	Instructor	Outlines	handoff-guard
Approach	Generation-time validation	Constrained generation	Post-hoc validation & repair
Works with	OpenAI, Anthropic, etc.	vLLM, Transformers	Any string output
LangGraph compatible	Yes (manual)	No	Yes (adapter: `guarded_node`)
Identifies failed node	No	N/A	Yes
Retries with feedback	Yes	N/A	Yes
Repairs malformed JSON	Yes (via re-prompt)	N/A	Yes (local, no tokens)
Dependencies	Pydantic + provider SDKs	Transformers/vLLM stack	Pydantic + json-repair

Instructor and Outlines are excellent tools. The difference is when and how they work:

Instructor validates at generation time and fixes errors by re-prompting—effective but costs tokens
Outlines constrains generation at the model level—powerful but requires inference server control
handoff-guard validates after the LLM responds and repairs locally—no extra tokens, works with any provider

The Problems This Actually Solves

handoff-guard doesn't fix framework bugs. It helps when you control the code that receives LLM output:

Problem	Example	How handoff-guard helps
LLM wraps JSON in conversation	`"Sure! Here's the JSON: {...}"`	`parse_json()` strips wrappers
Malformed JSON syntax	Trailing commas, single quotes, unquoted keys	`parse_json()` repairs common issues
Truncated output at token limit	`{"draft": "long text...`	`parse_json(detailed=True)` detects truncation
"ValidationError" with no context	`1 validation error for State`	`@guard(node_name="writer")` tells you which node
No retry on validation failure	Agent fails once, stays failed	`@guard(max_attempts=3)` retries automatically
LLM doesn't know why it failed	Retry happens but same error repeats	`retry.feedback()` tells the LLM what went wrong

Limits

What this won't magically fix:

Missing or hallucinated data — If the model omits required fields or invents values, deterministic repair can't invent correct data. Retries are still needed.
Ambiguous repairs — "Repair" is sometimes a best-effort guess (e.g., unquoted keys, stray punctuation). Always validate the result.
Severe truncation — You can detect it, but you can't recover missing content without another generation.
Adversarial or multi-JSON outputs — parse_json extracts the first JSON object/array boundary it finds. Complex tool traces or multiple embedded objects may need custom handling.

Security note: If you're parsing untrusted model output, treat "repaired JSON" as untrusted input. Validate types and ranges.

Get Started


bash
pip install handoff-guard

The package is handoff-guard, the import namespace is handoff:


python
from handoff import guard, retry, parse_json

That's it. No config files. No API keys. No Docker.

GitHub: github.com/acartag7/handoff-guard
PyPI: pypi.org/project/handoff-guard

What's Next

The library does what it set out to do. I'm not planning major features just bug fixes and edge cases as users report them, as it actually works for my current need.

If you hit something it doesn't handle, open an issue.

Built because "ValidationError: 1 validation error" tells you nothing useful.