Forem: AgentOracle

3 Agent Integration Patterns for Claim Verification (LangChain + CrewAI + MCP)

AgentOracle — Thu, 23 Apr 2026 14:38:24 +0000

Your agent generates a claim. Then what?

In most agent pipelines: nothing. The claim flows straight into the next action — a tool call, a database write, a message sent. If the claim is wrong, the action is wrong, and the first person to notice is usually the user.

There are three patterns that fix this. Each one adds a verification step between generation and action — pre-action claim verification — and each one fits a different stage of agent maturity.

All three patterns below use AgentOracle (free to try, no wallet, no API keys). The code works as-is. Copy it, run it.

Pattern 1: Verify-Then-Act Gate (simplest)

Your agent has exactly one claim it's about to act on. You want a hard pass/fail before anything happens.

pip install langchain-agentoracle

from langchain_agentoracle import AgentOracleVerifyGateTool

gate = AgentOracleVerifyGateTool()

def verify_then_act(claim: str, action_fn):
    """Gate an action behind a single claim verification."""
    result = gate.run(claim)

    # Gate returns PASS/FAIL + confidence. Parse from formatted output.
    if "Recommendation: ACT" in result:
        return action_fn()

    print(f"Action blocked — verification failed:\n{result}")
    return None


# Example: agent thinks a contract exists and wants to call it
claim = "Contract 0xabc...123 is a valid USDC contract on Base mainnet"
verify_then_act(claim, lambda: call_contract(...))

When to use this:

Single atomic claim ("X is true, therefore do Y")
Binary decisions (proceed or halt)
Free — /verify-gate has no cost

When it's not enough:

Your agent generates paragraph-length output with multiple claims
You need evidence for the verdict, not just pass/fail
You want per-claim granularity (accept 3 of 4 claims, flag the 4th)

Pattern 2: Decompose-and-Score (most versatile)

Your agent outputs a paragraph. Some claims are factual, some might be hallucinated, and you don't want to throw out the whole output if only one sentence is wrong.

The /evaluate endpoint decomposes text into atomic claims, scores each independently, and returns per-claim verdicts. You can then keep the good claims, correct the bad ones, or flag them for human review.

from langchain_agentoracle import AgentOracleEvaluateTool
import json
import re

evaluator = AgentOracleEvaluateTool()

def audit_agent_output(text: str):
    """Decompose text into claims, verify each, return structured audit."""
    raw = evaluator.run(text)

    # The tool returns a formatted string. Extract per-claim verdicts.
    claims = []
    for block in re.findall(r"\[(SUPPORTED|REFUTED|UNVERIFIABLE)\] \((\d\.\d+)\) (.+?)(?=\n\n|\Z)", raw, re.DOTALL):
        verdict, confidence, body = block
        lines = body.strip().split("\n")
        claim_text = lines[0].strip()
        evidence = ""
        for line in lines[1:]:
            if line.strip().startswith("Evidence:"):
                evidence = line.split("Evidence:", 1)[1].strip()
                break
        claims.append({
            "claim": claim_text,
            "verdict": verdict,
            "confidence": float(confidence),
            "evidence": evidence,
        })
    return claims


# Example: your agent produced this summary
agent_summary = """
OpenAI released GPT-4 in March 2023.
Bitcoin was created by Elon Musk in 2009.
Python was created by Guido van Rossum in 1991.
"""

audit = audit_agent_output(agent_summary)
for c in audit:
    print(f"{c['verdict']:14} ({c['confidence']:.2f}) {c['claim']}")
    if c['verdict'] == 'REFUTED':
        print(f"                        ↳ {c['evidence'][:120]}")

Sample output:

SUPPORTED      (1.00) OpenAI released GPT-4 in March 2023
REFUTED        (0.83) Bitcoin was created by Elon Musk in 2009
                        ↳ Bitcoin's creator is the pseudonymous Satoshi Nakamoto, not Elon Musk.
SUPPORTED      (1.00) Python was created by Guido van Rossum in 1991

What you can do with this:

# Keep supported, flag refuted, escalate low-confidence
safe_claims = [c for c in audit if c["verdict"] == "SUPPORTED" and c["confidence"] >= 0.8]
need_human  = [c for c in audit if c["verdict"] == "UNVERIFIABLE" or c["confidence"] < 0.5]
refuted     = [c for c in audit if c["verdict"] == "REFUTED"]

if refuted:
    # Regenerate with the corrections inline, or just flag
    log_hallucination(refuted)

When to use this:

Multi-claim agent output (summaries, research, plans)
You need evidence, not just a verdict
You want to selectively keep/reject claims

Pattern 3: Multi-Agent Supervisor (most advanced)

Now you have a CrewAI crew with a researcher agent and a writer agent. The writer is about to publish. You want a supervisor agent that:

Discovers the right verification provider (via Decixa's multi-provider registry, falling back to local)
Calls that provider to audit the writer's draft
Only passes the draft through if verification clears a threshold

This is where AgentOracle's MCP server shines. It exposes both the resolve tool (discovery) and the verification tools in a single MCP binary any agent can call.

# No install — runs via npx
npx agentoracle-mcp

Hook it into Claude Desktop or Cursor or any MCP-compatible runtime. Then in your agent framework:

from crewai import Agent, Task, Crew
from crewai_agentoracle import AgentOracleEvaluateTool

# Writer agent — generates content, might hallucinate
writer = Agent(
    role="Technical Writer",
    goal="Draft a factual summary of recent AI regulation news",
    backstory="Experienced technical writer. Optimizes for readability.",
)

# Supervisor agent — audits the writer's output using AgentOracle
supervisor = Agent(
    role="Fact-Check Supervisor",
    goal="Catch hallucinations before the writer's draft ships",
    backstory="Pedantic editor. Refuses to pass content with unverified claims.",
    tools=[AgentOracleEvaluateTool()],
)

draft = Task(
    description="Write 3 sentences about recent AI regulation news",
    agent=writer,
    expected_output="A 3-sentence factual summary",
)

review = Task(
    description=(
        "Evaluate the writer's draft using the AgentOracle tool. "
        "If any claim is REFUTED, return 'BLOCKED: <reason>'. "
        "If overall confidence is below 0.7, return 'NEEDS_HUMAN_REVIEW'. "
        "Otherwise return 'APPROVED' plus the cleaned draft."
    ),
    agent=supervisor,
    expected_output="APPROVED | BLOCKED | NEEDS_HUMAN_REVIEW + reasoning",
    context=[draft],
)

Crew(agents=[writer, supervisor], tasks=[draft, review]).kickoff()

Why this pattern matters:

Separation of concerns: one agent writes, one agent verifies
The supervisor can be a smaller, cheaper model — it just needs to call the tool and apply logic
Works in CrewAI, AutoGen, LangGraph — any framework that supports agent-as-tool-user

The discovery angle: If you want the supervisor to choose verification providers dynamically (not hardcode AgentOracle), use the resolve tool (v2.1.0 of agentoracle-mcp, via Decixa):

resolve(
  capability="analyze",
  intent="verify a factual claim before acting"
)

Returns the best-matching x402 verification endpoint across the ecosystem, ranked by latency, price, and tag match. AgentOracle is the only pre-action truth oracle currently classified under "Analyze → Verification" on Decixa, so it'll come back first today. As more providers list, your supervisor automatically picks the best one for each query.

Which Pattern To Pick

Your agent setup	Pattern
Single binary decision	1. Verify-then-act gate (free, `/verify-gate`)
Paragraph output, need per-claim scoring	2. Decompose-and-score (free during beta, `/evaluate`)
Multi-agent pipeline, supervisor pattern	3. Multi-agent supervisor (CrewAI + MCP)

All three work together. Start with Pattern 1 while you're prototyping. Graduate to Pattern 2 when your agent produces structured output. Move to Pattern 3 when you have a real pipeline with distinct agent roles.

Getting Started

Playground (no setup): agentoracle.co

Packages:

pip install langchain-agentoracle — PyPI
pip install crewai-agentoracle — PyPI
npx agentoracle-mcp — npm (Claude Desktop, Cursor, Windsurf)
npm install agentoracle-verify — npm

Source: GitHub

Benchmark: We ran AgentOracle head-to-head against GPT-4o on 200 peer-reviewed FEVER claims. Results + methodology.

Hallucinations aren't a bug to patch. They're a property of large language models that doesn't go away with bigger training runs or better prompts. The only reliable fix is to add a verification step your agent can't bypass.

These three patterns are what that step looks like in production code.

24 hours of organic discovery: what we learned from our first external users

AgentOracle — Wed, 22 Apr 2026 04:14:56 +0000

Yesterday we published a tutorial. No list. No paid promotion. No cold outreach.
By Tuesday morning, five developers and two autonomous agents had found AgentOracle and run real evaluations.
Who showed up:

Starlink — Albuquerque, NM
Comcast — Rockville, MD
Charter/Spectrum — Missoula, MT
Azure cloud agent — Des Moines, IA
Azure cloud agent — Chicago, IL

Same path every time: tutorial → playground → /evaluate
The Azure IPs are the most interesting signal. Those aren't humans clicking a tutorial — those are autonomous agents running on cloud infrastructure that found the playground and ran evaluations on their own. That's exactly the use case we built for.
MCP server shipped tonight. Your agent can find it the same way they did.

npx agentoracle-mcp
Or hit the playground directly: agentoracle.co

How to Add Claim Verification to Your LangChain Agent in 5 Minutes

AgentOracle — Mon, 20 Apr 2026 16:01:18 +0000

Your LangChain agent is wrong about 10% of the time. Not occasionally — consistently, confidently, and silently.

The problem isn't the model. It's that your agent has no way to know when it's wrong. It receives information, formats a response, and acts. No second opinion. No fact-check. No circuit breaker.

This tutorial shows you how to add a verification layer in 5 minutes that catches hallucinations before your agent acts on them.

The Problem

LLM hallucination rates in 2026 range from 3% to 20% depending on the task. On a summarization benchmark, GPT-4 looks great. On open-ended factual questions — the kind your agent asks constantly — it's a different story.

The deeper problem: reasoning models hallucinate more on factual tasks, not less. The more a model "thinks through" an answer, the more likely it is to fill gaps with plausible-sounding fiction.

In a simple chatbot, a hallucination is embarrassing. In an autonomous agent pipeline, it's a wrong action. A refunded order, a bad recommendation, a compliance violation, a message sent to the wrong person.

The standard fix is human review. But human review defeats the purpose of an autonomous agent.

The real fix is a verification layer that runs before your agent acts — independently of the model that generated the claim.

Install

pip install langchain-agentoracle

That's it. No API keys. No configuration. The free tier gives you 20 preview verifications per hour to test with.

Quick Start: Verify Before Your Agent Acts

The simplest integration — verify a piece of text and get per-claim verdicts:

from langchain_agentoracle import AgentOracleEvaluateTool

verifier = AgentOracleEvaluateTool()

# Your agent just generated this text — is it true?
agent_output = """
OpenAI released GPT-4 in March 2023.
Bitcoin was created by Elon Musk.
The Python programming language was created by Guido van Rossum.
"""

result = verifier.run(agent_output)
print(result)

Here's what comes back:

EVALUATION RESULT
Overall confidence: 0.61
Recommendation: ACT
Claims found: 3 | Supported: 2 | Refuted: 1 | Unverifiable: 0
Sources used: sonar, sonar-pro, adversarial, gemma-4

CLAIMS:
  ✓ [SUPPORTED] (1.00) OpenAI released GPT-4 in March 2023
    Evidence: Widely documented historical fact; GPT-4 was announced
    and released on March 14, 2023.

  ✗ [REFUTED] (0.83) Bitcoin was created by Elon Musk
    Evidence: Bitcoin's creator is the pseudonymous Satoshi Nakamoto.
    Correction: Bitcoin was created by Satoshi Nakamoto, not Elon Musk.

  ✓ [SUPPORTED] (1.00) Python was created by Guido van Rossum
    Evidence: Confirmed in official Python documentation and
    Van Rossum's own statements.

Three claims went in. Two came back supported with evidence. One came back refuted with a correction. Your agent now knows claim #2 is wrong before it acts on it.

Add It to Your Agent's Toolbelt

Want your agent to verify claims on its own? Add the tools directly:

from langchain_agentoracle import get_agentoracle_tools

# Returns all 6 AgentOracle tools ready for your agent
tools = get_agentoracle_tools()

# Or pick specific ones:
from langchain_agentoracle import (
    AgentOracleEvaluateTool,    # Per-claim verification ($0.01)
    AgentOracleVerifyGateTool,  # Quick pass/fail gate (free)
    AgentOraclePreviewTool,     # Research preview (free, 20/hr)
)

The tools follow LangChain's BaseTool interface, so they plug into any agent:

from langchain.agents import initialize_agent, AgentType
from langchain_openai import ChatOpenAI
from langchain_agentoracle import AgentOracleEvaluateTool, AgentOraclePreviewTool

llm = ChatOpenAI(model="gpt-4")

tools = [
    AgentOracleEvaluateTool(),
    AgentOraclePreviewTool(),
]

agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True,
)

# The agent can now verify claims before acting
agent.run("Check if this is true: Tesla's market cap exceeded $2 trillion in 2024")

The Verify-Then-Act Pattern

The most useful pattern: gate your agent's actions on verification confidence.

from langchain_agentoracle import AgentOracleEvaluateTool
import json

verifier = AgentOracleEvaluateTool()

def verify_then_act(text, confidence_threshold=0.8):
    """Only act if verification confidence exceeds threshold."""
    result = verifier.run(text)

    # Parse the confidence from the result
    # The tool returns a formatted string with overall confidence
    if "Overall confidence:" in result:
        conf_line = [l for l in result.split('\n') if 'Overall confidence' in l][0]
        confidence = float(conf_line.split(': ')[1])

        if confidence >= confidence_threshold:
            print(f"✅ VERIFIED ({confidence}) — safe to act")
            return True
        else:
            print(f"⚠️ LOW CONFIDENCE ({confidence}) — hold for review")
            return False
    return False

# In your agent pipeline:
claim = "The Federal Reserve raised interest rates in March 2024"
if verify_then_act(claim):
    # proceed with the action
    pass
else:
    # flag for human review or use a fallback
    pass

Free Quick Check: The Verify Gate

Don't need per-claim breakdowns? The verify gate gives you a fast pass/fail:

from langchain_agentoracle import AgentOracleVerifyGateTool

gate = AgentOracleVerifyGateTool()

# Quick binary check — free, no payment needed
result = gate.run("The speed of light is approximately 300,000 km per second")
print(result)
# VERIFY GATE: FAIL
# Confidence: 1.00
# Recommendation: ACT
# ("FAIL" = gate found no issues — content is safe to act on)

Why AgentOracle

Most hallucination detection tools are built for humans — dashboards, observability platforms, monitoring UIs. They tell you what went wrong after the fact.

AgentOracle is built for agents. It sits in the pipeline, takes any text, runs it through 4 independent verification sources in parallel, and returns a machine-readable verdict before your agent acts.

No dashboards. No subscriptions. No API keys to configure. Your agent calls /evaluate, gets ACT / VERIFY / REJECT with a confidence score and evidence, and decides what to do next.

What's under the hood:

4 independent sources: Sonar, Sonar Pro, Adversarial challenge, and Gemma 4
Per-claim decomposition — complex text gets broken into individual verifiable claims
Confidence calibration across sources
Evidence and corrections for every verdict
1,900+ claim fingerprints in the database and growing daily

Try It Now

Playground — no setup, no payment: agentoracle.co
Paste any text and see per-claim verdicts in under 15 seconds.

Packages:

pip install langchain-agentoracle — PyPI
pip install crewai-agentoracle — PyPI
npm install agentoracle-verify — npm

Source: GitHub

Hallucinations aren't going away. The models are getting better, but "better" still means wrong 3-10% of the time on the tasks your agents actually run.

A verification layer doesn't replace a good model. It catches the cases where even a good model is confidently wrong — which is exactly when you need it most.