Forem: CloakLLM

What the Data Act Misses: The Last Mile Between Regulation and Adoption

CloakLLM — Thu, 09 Apr 2026 14:43:23 +0000

Every company using Large Language Models (LLMs) is sending data somewhere. Most of them don't have a clear answer for what happens to the personal information inside those API calls. That's not a future compliance problem, it's a right-now problem. And waiting for regulation to catch up isn't a strategy; it's a liability.

I founded CloakLLM because I kept seeing the same scenario play out across industries. Companies were ready to adopt LLMs for high-impact use cases-customer support automation, complex document processing, internal knowledge search- but they hit a wall. There was no clear path to move from a "cool demo" to "production-ready" without exposing sensitive customer data to a third-party API.

When that wall is hit, the project usually stalls. But "stalled" is a polite word for what actually happens.

In reality, the project often goes underground. When official AI initiatives are blocked by compliance concerns, employees don't stop needing the technology. They open ChatGPT in a personal browser tab and paste customer data, proprietary code, or sensitive legal summaries into a consumer interface. They do this with zero logging, zero PII protection, and zero audit trail. The compliance concern hasn't been solved; it's just become invisible to the C-suite. This "Shadow AI" is one of the largest unmanaged risks in the enterprise today.

The Vision vs. The Reality

This gap exists everywhere, but in the EU, the tension is particularly high. The regulatory framework is actually ahead of most jurisdictions. The Data Act went live on September 12, 2025. The AI Act mandates automatic record-keeping (Article 12) and transparency (Article 13). GDPR remains the bedrock, requiring data minimization and the right to erasure.

Europe got the vision right. I believe that a trust-based framework is a long-term competitive advantage. But there's a "last mile" problem: none of these high-level principles translates into tools that organizations can actually deploy today.

There is no off-the-shelf solution. No middleware says, "This request contains PII from an EU data subject; here is how to handle it before it leaves your infrastructure." That's why I founded CloakLLM - to create a transparent, open-source layer that sits between the application and the model, ensuring that the "legal yes" is as easy to achieve as the "technical yes."

Solving the Natural Language Problem

Protecting data in an LLM call is fundamentally different from protecting a database. Personal information in natural language isn't a solved problem. Simple pattern matching catches structured data, IBANs, credit cards, and phone numbers, but misses the nuance of human speech.

That's why CloakLLM uses a 3-pass detection pipeline:

Pattern Matching: For high-speed, high-confidence structured data.

Named Entity Recognition: To identify people, organizations, and locations in context.

Local AI Reasoning: This is the critical layer. A local model reasons about what is actually sensitive. It can distinguish between "123 Main St" as a generic example and "the house next to the bakery" as a specific, identifiable address.

All of this happens locally; the data never leaves the organization's infrastructure until it's already replaced with safe placeholder tokens. When inference happens on a cloud API, that's not a design preference; it's a requirement for GDPR compliance.

The Three Critical Gaps

From where I sit, three gaps are preventing the Data Act from becoming a catalyst for adoption:

The API-Layer Standard: The Data Act focuses heavily on IoT-generated data and portability. The AI Act focuses on risk classification. Neither addresses the primary data flow of the current AI boom: the API call. We need a standard for protecting the data in these requests and a recognized format for logging them. Without a standard, every legal department is forced to reinvent the wheel, leading to months of unnecessary friction.
The Documentation-Agile Conflict: The AI Act's Annex IV demands comprehensive records of design decisions and data lineage. But for companies using Retrieval-Augmented Generation (RAG) - where the AI pulls from a live knowledge base to answer questions - "data lineage" is a moving target. Every time the knowledge base updates with new documents or fresh data, the system's context changes.

Annex IV assumes a static audit trail; RAG is inherently dynamic. If you change how the system processes or retrieves information, your compliance snapshot is technically obsolete. Compliance documentation needs to be generated automatically as part of the development process, not maintained manually after the fact. And every month spent on manual compliance work is a month your competitor ships without you.

The Missing Implementation Layer: Regulators have published a flood of guidance. These are valuable for legal teams, but the people actually building AI products need practical tools and implementation blueprints. The gap between a regulatory PDF and a working product is where most innovation quietly dies.

Moving Toward "Compliance-as-Code"

What would actually change the equation? Standardized, open-source reference implementations.

If the EU were to fund standard open-source components, privacy detection layers, audit logging tools, and consent-tracking systems - in the same way it funds research, the impact would be massive. Standardization removes the excuse. When a compliance layer is one install away, "we couldn't figure out how to handle PII" stops being a valid reason to delay AI adoption.

For companies outside the EU, the lesson is the same: adopting this infrastructure now isn't just about avoiding a future fine. It's about building the trust necessary to move AI out of the sandbox and into the core of your business.

I'd like to hear from others navigating this. If you're leading AI adoption at your company, what's getting in the way? Is it the model capability, or is it the infrastructure of trust?

The code is open-source: cloakllm.dev | github.com/cloakllm/CloakLLM

EUAIAct #DataAct #opensource #privacy #LLM

Cryptographic Proof That Your LLM Never Saw Real Data

CloakLLM — Thu, 19 Mar 2026 05:50:02 +0000

Cryptographic Proof That Your LLM Never Saw Real Data

Every PII protection tool makes the same promise: "We sanitized it before sending." But promises aren't proof. When a regulator asks you to demonstrate that patient names never reached OpenAI's servers, "trust us" isn't an answer.

We just shipped CloakLLM v0.3.2 with cryptographic attestation - Ed25519-signed certificates that mathematically prove sanitization happened. Here's how it works, why it matters, and how to use it.

The Problem: Trust Without Verification

Most PII middleware operates on trust. You install it, it sanitizes prompts, and you believe it did its job. Your audit log says "sanitized 3 entities at 14:30:00." But that log is just text. Anyone with file access can edit it. Nothing ties the log entry to the actual data that was processed.

This creates three gaps:

Gap 1: No proof of execution. Your compliance team can show the tool is installed. They can't prove it was running when a specific prompt was sent.

Gap 2: No tamper evidence on individual operations. Hash-chained audit logs prove the chain wasn't modified after the fact. But they don't prove what was sanitized - only that something was logged.

Gap 3: No cross-system verification. If a downstream execution layer wants to confirm that PII was handled before it processes an action, there's no machine-readable artifact to check.

The EU AI Act (Article 12, enforcement begins August 2, 2026) requires audit logs that regulators can mathematically verify. Gap 2 is the one that will fail an audit.

The Solution: Sanitization Certificates

Starting in v0.3.2, every sanitize() call can produce an Ed25519-signed certificate. The certificate contains:

Input hash - SHA-256 of the original text (proves what was processed)
Output hash - SHA-256 of the sanitized text (proves what was produced)
Entity count - how many PII entities were detected
Categories - breakdown by type (EMAIL: 2, PERSON: 1, SSN: 1)
Detection passes - which detection methods ran (regex, NER, LLM)
Mode - tokenize (reversible) or redact (irreversible)
Timestamp - ISO 8601
Key ID - identifies which signing key produced the certificate
Nonce - UUID4 preventing replay attacks
Signature - Ed25519 signature over the canonical JSON of all fields above

The certificate is attached to the token map returned by sanitize(). It's also hashed into the audit log entry, binding the certificate to the chain.

How It Works

Step 1: Generate a Signing Key

from cloakllm import DeploymentKeyPair

keypair = DeploymentKeyPair.generate()
keypair.save("./keys/signing_key.json")

This produces an Ed25519 keypair. The private key signs certificates. The public key verifies them. You generate it once per deployment and keep the private key secure.

Step 2: Configure the Shield

from cloakllm import Shield, ShieldConfig

shield = Shield(ShieldConfig(
    attestation_key=keypair
))

Or load from a file:

shield = Shield(ShieldConfig(
    attestation_key_path="./keys/signing_key.json"
))

Or from an environment variable:

export CLOAKLLM_SIGNING_KEY_PATH=./keys/signing_key.json

Step 3: Sanitize and Get the Certificate

sanitized, token_map = shield.sanitize(
    "Email john@acme.com about Sarah Johnson's medical records"
)

cert = token_map.certificate

The certificate now contains a signed attestation of what happened. You can serialize it:

cert_dict = cert.to_dict()
# {
#   "version": "1.0",
#   "timestamp": "2026-03-15T10:30:00Z",
#   "input_hash": "sha256:9f86d081...",
#   "output_hash": "sha256:a3f2b1c4...",
#   "entity_count": 3,
#   "categories": {"EMAIL": 1, "PERSON": 1, "MEDICAL": 1},
#   "detection_passes": ["regex", "ner"],
#   "mode": "tokenize",
#   "key_id": "ed25519:7b2a...",
#   "nonce": "a1b2c3d4-e5f6-...",
#   "signature": "base64:..."
# }

Step 4: Verify

Anyone with the public key can verify:

assert cert.verify(keypair.public_key)
assert shield.verify_certificate(cert)

If a single byte of the certificate has been modified, verification fails. The signature covers a canonical JSON serialization of all fields, so field reordering doesn't break verification.

Batch Attestation with Merkle Trees

Single-text certificates are straightforward. But what about sanitize_batch() with 50 texts? You don't want 50 separate certificates - that's expensive and hard to manage.

CloakLLM uses Merkle trees for batch attestation. Instead of hashing each text individually into the certificate, we compute:

A Merkle root of input hashes - one hash representing all original texts
A Merkle root of output hashes - one hash representing all sanitized texts

The batch certificate contains these two roots instead of individual hashes. One signature covers the entire batch.

texts = [
    "Email john@acme.com",
    "SSN 123-45-6789",
    "Call Sarah at 555-0100"
]
sanitized_texts, token_map = shield.sanitize_batch(texts)

cert = token_map.certificate        # single certificate for entire batch
merkle_tree = token_map.merkle_tree  # Merkle tree with proofs

To verify that a specific text was part of the batch:

import hashlib
from cloakllm import MerkleTree

leaf = hashlib.sha256(texts[0].encode()).hexdigest()
proof = merkle_tree["input"].proof(0)
assert MerkleTree.verify_proof(leaf, proof, merkle_tree["input"].root)

This is the same data structure used in blockchain systems and certificate transparency logs. It lets you prove the inclusion of a single item without revealing the others.

Cross-Language Verification

Certificates are fully portable between Python and JavaScript. The canonical JSON serialization and Ed25519 signature format are identical in both SDKs.

from cloakllm import Shield, ShieldConfig, DeploymentKeyPair

keypair = DeploymentKeyPair.generate()
shield = Shield(ShieldConfig(attestation_key=keypair))
sanitized, token_map = shield.sanitize("Email john@acme.com")
cert_dict = token_map.certificate.to_dict()
public_key_hex = keypair.public_key_hex

Verify in JavaScript:

const { SanitizationCertificate, DeploymentKeyPair } = require('cloakllm');

const cert = SanitizationCertificate.fromDict(certDict);
const publicKey = DeploymentKeyPair.fromPublicKeyHex(publicKeyHex);
assert(cert.verify(publicKey));

This matters for architectures where the sanitization layer and the verification layer run in different languages or services.

Why Ed25519?

We chose Ed25519 over RSA or ECDSA for three reasons:

Deterministic signatures. Same input always produces the same signature. No nonce-related vulnerabilities.
Small keys and signatures. 32-byte public key, 64-byte signature. Certificates stay compact.
Fast. Ed25519 verification is roughly 60x faster than RSA-2048 for single-signature verification.

The JavaScript SDK uses Node.js built-in crypto module - zero additional dependencies. The Python SDK supports either pynacl or cryptography as an optional dependency.

pip install cloakllm[attestation]  # installs pynacl

What This Means for EU AI Act Compliance

Article 12 of the EU AI Act requires that high-risk AI systems maintain logs enabling traceability of how the system operates. Enforcement begins August 2, 2026. Non-compliance can result in fines up to 7% of global annual revenue.

CloakLLM's audit chain already provides hash-linked logs. Attestation certificates add a stronger guarantee: each sanitization operation is individually signed, and the signature binds the operation's inputs, outputs, and metadata into a single tamper-evident artifact.

For an auditor, this means:

Existence proof - the certificate proves a sanitization operation occurred at a specific time
Integrity proof - the signature proves the certificate hasn't been modified
Content proof - the input/output hashes prove what was processed, without revealing the actual PII
Batch proof - Merkle proofs demonstrate individual text inclusion without exposing the full batch

What This Means for Execution Governance

There's an emerging architectural pattern in agentic AI systems: execution boundaries that decide whether a proposed action is allowed to proceed. These boundaries evaluate intent, authority, and system state before committing an action.

Sanitization certificates add a data-layer signal to this decision. An execution boundary can now ask: "Was the input to this reasoning step provably sanitized?" and verify the answer cryptographically - without trusting the sanitization layer itself.

This is the difference between a policy that says "PII must be removed" and an architecture that enforces it with mathematical proof.

Try It

pip install cloakllm[attestation]==0.3.2
npm install cloakllm@0.3.2

Generate a key, configure the shield, and every sanitize() call produces a signed certificate. The certificate is attached to the token map, hashed into the audit log, and verifiable in any language.

Full documentation: cloakllm.dev
Source: github.com/cloakllm/CloakLLM

CloakLLM is open source (MIT). Built by Ziv Chen.

Every LLM Prompt You Send Is Plaintext. Here's How to Fix That Before the EU Makes You.

CloakLLM — Mon, 02 Mar 2026 17:32:31 +0000

Your LLM calls are unencrypted confessions.

Every time you call litellm.completion() or openai.chat.completions.create(), the provider receives your prompt in full plaintext. Names, emails, SSNs, API keys, medical records - all of it sitting in someone else's logs.

That's been a privacy risk for years. In 5 months, it becomes illegal.

August 2, 2026

The EU AI Act enters enforcement. Article 12 mandates tamper-evident audit logs for AI systems - not console.log(), not a JSON file you append to. Logs that regulators can mathematically verify haven't been altered.

The penalty: up to 7% of global annual revenue.

If you use LLMs and handle EU data, you need:

PII never reaches the provider (or you need explicit consent per entity)
Every AI interaction logged in a verifiable audit trail

Most teams have neither. I built CloakLLM to fix both.

What CloakLLM Does

CloakLLM is open-source middleware that sits between your app and any LLM provider. Python, Node.js, and MCP for Claude Desktop.

It does three things:

Detects PII - three layers deep:

spaCy NER for names, orgs, locations (Python)
Regex for emails, SSNs, credit cards, API keys, IBANs, phones, IPs, JWTs (both SDKs)
Local LLM via Ollama (opt-in) - catches context-dependent PII that regex misses: addresses, medical terms, financial data, national IDs. Your data never leaves your machine.

Cloaks it with context-preserving tokens:

Your app:      "Email sarah.j@techcorp.io about Project Falcon"
Provider sees: "Email [EMAIL_0] about Project Falcon"
You receive:   Original email restored in the response

The LLM still understands the prompt. It just never sees real data.

Logs everything to a hash-chained audit trail:

{
  "seq": 42,
  "event_type": "sanitize",
  "entity_count": 3,
  "prompt_hash": "sha256:9f86d0...",
  "prev_hash": "sha256:7c4d2e...",
  "entry_hash": "sha256:b5e8f3..."
}

Each entry's SHA-256 hash includes the previous entry's hash. Tamper with one log entry, and every subsequent hash breaks. This is what Article 12 actually requires.

Python - One Line with LiteLLM

Works with 100+ providers (Anthropic, OpenAI, Azure, Bedrock, Ollama, etc.) via LiteLLM:

import cloakllm
cloakllm.enable()  # Done.

import litellm
response = litellm.completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{
        "role": "user",
        "content": "Help me email sarah.j@techcorp.io about the Q3 audit"
    }]
)
# Provider never saw the email. Response has it restored.

Node.js - OpenAI SDK

const cloakllm = require('cloakllm');
const OpenAI = require('openai');

const client = new OpenAI();
cloakllm.enable(client);  // Done.

const response = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{
    role: 'user',
    content: 'Write a reminder for sarah.j@techcorp.io about the Q3 audit'
  }]
});
// Provider never saw the email.

Node.js - Vercel AI SDK

const { createCloakLLMMiddleware } = require('cloakllm');
const { generateText, wrapLanguageModel } = require('ai');
const { openai } = require('@ai-sdk/openai');

const middleware = createCloakLLMMiddleware();
const model = wrapLanguageModel({ model: openai('gpt-4o'), middleware });

const { text } = await generateText({
  model,
  prompt: 'Write a reminder for sarah.j@techcorp.io about the Q3 audit'
});
// Streaming supported - handles tokens split across chunk boundaries

MCP Server - Claude Desktop

CloakLLM also ships as an MCP server with three tools: sanitize, desanitize, and analyze. Add it to your claude_desktop_config.json and every Claude Desktop conversation gets PII cloaking.

Or Use It Standalone

# Python
from cloakllm import Shield
shield = Shield()
cloaked, token_map = shield.sanitize("Send to john@acme.com, SSN 123-45-6789")
# cloaked: "Send to [EMAIL_0], SSN [SSN_0]"

// Node.js
const { Shield } = require('cloakllm');
const shield = new Shield();
const [cloaked, tokenMap] = shield.sanitize("Send to john@acme.com, SSN 123-45-6789");

Works with any LLM client, any provider, any framework.

Local LLM Detection

Regex catches structured PII. But "I live at 742 Evergreen Terrace" or "diagnosed with hypertension" - regex can't catch that.

CloakLLM has an opt-in LLM detection layer that runs through local Ollama:

shield = Shield(config=ShieldConfig(
    llm_detection=True,
    llm_model="llama3.2:3b",
))

It detects addresses, medical terms, financial data, national IDs, biometrics, usernames, and passwords - all without your data leaving your machine.

The LLM pass runs after regex, so already-detected entities are skipped. No double counting.

Design Decisions

Regex first, NER second, LLM third. Structured data (emails, SSNs, credit cards) is caught by regex with near-zero latency. spaCy NER runs second for names and orgs. The LLM pass is opt-in and catches everything else. Fast path stays fast.

Overlap detection. If regex catches john@acme.com and NER detects acme.com as ORG, the overlap is caught and the duplicate is skipped.

System prompt injection. Without it, LLMs see [PERSON_0] and ask "what's the real name?" CloakLLM injects a system message telling the model to treat tokens as real values. Only when tokens are present.

Token injection protection. If user input contains [PERSON_0], it gets escaped to fullwidth Unicode brackets before tokenization - preventing attackers from injecting fake tokens to extract other users' PII during desanitization.

try/finally cleanup. Even if the LLM API throws, the token map (which contains PII mappings) is always cleaned up. No PII lingers in process memory.

Vercel streaming. The Vercel AI SDK middleware buffers text-delta chunks and desanitizes on text-end, correctly handling tokens that span chunk boundaries like [EM + AIL_0].

Verify Your Audit Chain

$ cloakllm verify ./cloakllm_audit/
✅ Audit chain integrity verified - no tampering detected.

If someone edits a log entry:

Entry #40 ✅ → #41 ✅ → #42 ❌ TAMPERED → #43 ❌ BROKEN → ...

Hand this to an auditor.

Install

# Python
pip install cloakllm                  # standalone
pip install cloakllm[litellm]         # with LiteLLM middleware
python -m spacy download en_core_web_sm

# Node.js
npm install cloakllm

The Numbers

150 tests across Python (62), JS (79), and MCP (9). Security audited with 6 vulnerability classes found and fixed: backreference injection, fake token injection, ReDoS hardening, spaCy model validation, middleware memory cleanup, and custom pattern safety checks. All regression-tested.

Zero runtime dependencies on the JS side. Python depends on spaCy.

What's Next

The roadmap includes LangChain.js integration, OpenTelemetry span emission, RFC 3161 trusted timestamping, sensitivity-based routing (PII → local model, clean → cloud), and an admin dashboard.

The EU AI Act deadline is August 2, 2026. 5 months from today.

→ GitHub: github.com/cloakllm/CloakLLM
→ Python SDK: github.com/cloakllm/CloakLLM-PY | pip install cloakllm
→ Node.js SDK: github.com/cloakllm/CloakLLM-JS | npm install cloakllm
→ MCP Server: github.com/cloakllm/cloakllm-mcp