The latest articles on Forem by Miroslav Šotek (@anulum). https://forem.com/anulum https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3848490%2F36416691-0a81-4276-9be0-1a32852ed99a.jpeg https://forem.com/anulum en Miroslav Šotek Sun, 29 Mar 2026 01:17:47 +0000 https://forem.com/anulum/i-built-an-open-source-real-time-llm-hallucination-guardrail-here-are-the-benchmarks-22em https://forem.com/anulum/i-built-an-open-source-real-time-llm-hallucination-guardrail-here-are-the-benchmarks-22em <h2> What is Director-Class AI? </h2> <p>An open-source Python library that guards LLM output in real time. It watches tokens as they stream and halts generation the moment it detects a hallucination.</p> <p>It uses <strong>NLI</strong> (Natural Language Inference via DeBERTa/FactCG) and optional <strong>RAG knowledge grounding</strong> to score each claim against source documents.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>pip <span class="nb">install </span>director-ai </code></pre> </div> <p>Two-line integration:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight python"><code><span class="kn">from</span> <span class="n">director_ai</span> <span class="kn">import</span> <span class="n">guard</span> <span class="n">client</span> <span class="o">=</span> <span class="nf">guard</span><span class="p">(</span><span class="n">openai</span><span class="p">.</span><span class="nc">OpenAI</span><span class="p">())</span> <span class="c1"># wraps any OpenAI/Anthropic client </span></code></pre> </div> <h2> Benchmarks (measured, not aspirational) </h2> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Metric</th> <th>Value</th> <th>Conditions</th> </tr> </thead> <tbody> <tr> <td>Balanced accuracy</td> <td>75.8%</td> <td>FactCG on LLM-AggreFact (29,320 samples)</td> </tr> <tr> <td>GPU latency</td> <td>14.6ms/pair</td> <td>GTX 1060, ONNX, batch=16</td> </tr> <tr> <td>L40S latency</td> <td>0.5ms/pair</td> <td>FP16, batch=32</td> </tr> <tr> <td>E2E catch rate</td> <td>90.7%</td> <td>Hybrid mode, 600 HaluEval traces</td> </tr> <tr> <td>Rust BM25 speedup</td> <td>10.2x</td> <td>Over pure Python implementation</td> </tr> </tbody> </table></div> <h2> Framework Integrations </h2> <p>LangChain, LlamaIndex, LangGraph, CrewAI, Haystack, DSPy, Semantic Kernel, and SDK Guard (wraps OpenAI/Anthropic/Bedrock/Gemini/Cohere clients).</p> <h2> Honest Limitations </h2> <ul> <li>NLI-only scoring needs KB grounding for domain use (medical FPR=100% without KB)</li> <li>ONNX CPU is slow (383ms/pair) — GPU recommended</li> <li>Long documents need &gt;=16GB VRAM</li> <li>Summarisation accuracy weakest (AggreFact-CNN 68.8%)</li> </ul> <h2> Quality </h2> <ul> <li>3,545 tests, 91% coverage</li> <li>Sigstore-signed releases, SLSA provenance</li> <li>OpenSSF Best Practices: 100%</li> <li>19 badges of CI/security health</li> </ul> <h2> Links </h2> <ul> <li> <strong>GitHub</strong>: <a href="https://github.com/anulum/director-ai" rel="noopener noreferrer">github.com/anulum/director-ai</a> </li> <li> <strong>Docs</strong>: <a href="https://anulum.github.io/director-ai" rel="noopener noreferrer">anulum.github.io/director-ai</a> </li> <li> <strong>PyPI</strong>: <a href="https://pypi.org/project/director-ai/" rel="noopener noreferrer">pypi.org/project/director-ai</a> </li> </ul> <p>AGPL-3.0 with commercial licensing available.</p> <p>Would love feedback from anyone working on LLM reliability, RAG pipelines, or AI safety!</p> machinelearning