Forem: Pavel Gajvoronski

I Built 23 Pages in One Day With AI. Then One API Key Almost Killed Everything

Pavel Gajvoronski — Wed, 15 Apr 2026 10:12:48 +0000

This is a build-in-public update on Kepion — an AI platform that deploys companies from a text description. First post here.*

"This is a build-in-public update..."

Two days ago I shared the architecture. Today I want to share what actually happened when I started building — the wins, the disasters, and the numbers.

The disaster: 3 hours lost to a phantom API key

I sat down at 8am ready to build. Opened my terminal. Ran GSD-2 (my build orchestrator). Got this:

Error: All credentials for "anthropic" are in a cooldown window.

My Max plan showed 3% usage. The tool said I was rate-limited. For three hours I debugged, restarted, cleared caches, filed a support ticket. The fix?

unset ANTHROPIC_API_KEY

An old API key from a previous tool installation was silently overriding my subscription. One environment variable. Three hours gone.

The lesson: invisible defaults are the most dangerous bugs in AI tooling.

I'm sharing this because every developer building with AI agents will hit this. Your LLM provider's auth layer has more failure modes than your application code.

What GSD-2 actually built in one day

Once the auth was fixed, I pointed GSD-2 at Kepion and let it work. Here's the raw output from a single day:

Security hardening (10 items)

Deny-by-default auth middleware — every new route is blocked unless explicitly whitelisted
Path traversal fix in vault manager
WebSocket authentication (was anonymous before)
CORS whitelist replacing wildcard *
Password policy: 12+ chars, uppercase, digit, special char
Rate limiting by user email instead of IP
Upload validation: file extension whitelist, 5MB limit
Business ownership verification on all endpoints
Session scoping by user_id
Login attempt tracking with 30-minute lockout after 10 failures

Observability (shipped)

Every HTTP request gets a trace_id
Every agent call becomes a span linked to the trace
Slow trace detection (>5s)
Error trace listing
All persisted in SQLite

Cost intelligence (shipped)

Per-agent, per-model, per-business cost breakdown
Anomaly detection: flags agents with z-score > 2σ above mean
Cost circuit breaker: blocks requests at configurable limits

Team Memory (shipped)

Agents save learnings across sessions
Effectiveness scoring (0.0–1.0)
Auto context injection — relevant memories prepended to prompts
Categories: solution, pattern, mistake, optimization

Checkpoint & Replay (shipped)

Checkpoint after every chain step
Resume on failure with can_resume: true
Dead letter queue for chains that fail after all retries
Configurable retry policies: default, critical, fast_fail

Event-driven triggers (shipped)

5 trigger types: schedule, webhook, event_pattern, vault_change, threshold
4 action types: run_agent, run_chain, webhook_out, notify

Web UI: 23 pages (shipped)

Full Next.js 16 dashboard with collapsible sidebar
Dashboard, Chat, Agents, Pipelines, Businesses, Integrations
Vault, Research, Patterns, YouTube, Workflows, Gate
Costs, Traces, Triggers, Admin, Pricing, Account
Live support chat widget with typing indicators
Pricing page with 5 tiers and competitive comparison table

Telegram bot: fully functional (shipped)

/start with auto-registration and JWT token storage
/agents, /agent, /business, /status, /costs, /help
Free text → auto-routing to the right agent
Typing indicators while agents think
Auth headers on every API call

The numbers

Metric	Value
Services	30+
API endpoints	40+
Agent prompts (v3)	31 × 17 sections each
Tests	180+
Web UI pages	23
Telegram commands	15
Lines changed in one day	~3,000+

One person. One AI build tool. One day.

What I learned

1. Security is invisible until it isn't. Nobody sees path traversal protection. But without it, the first user with ../../etc/passwd in a vault search owns your server. I'm glad GSD-2 caught every item from the CONCERNS.md audit.

2. Observability changes everything. Before traces, debugging a 5-agent chain was guesswork. Now I can see: request → router (2ms) → researcher (4.3s) → sentinel (1.1s) → warden (0.8s) → response. The bottleneck is always the researcher.

3. Cost circuit breakers are non-negotiable. Without them, one hallucinating agent in a loop burns through your OpenRouter budget in minutes. Our circuit breaker has 4 levels: per-request ($2), per-agent-hourly ($10), per-business-daily ($50), platform-hourly ($100).

4. Team Memory is the moat. Every business Kepion creates makes the next one better. Agents save what worked and what failed. Business #5 benefits from patterns discovered in businesses #1-4. This compounds. Competitors can copy the code — they can't copy the accumulated knowledge.

What's next

Autonomous Operations — agents posting to Twitter, sending emails, running outreach. Every output goes through Sentinel (fact-check) and Warden (quality gate) before publishing. Quality over spam.
Full Deploy Pipeline — /deploy chess-school → buy domain → deploy frontend (Vercel) → deploy backend (Railway) → configure Paddle payments → live URL. One command.
Code Ownership — all generated code pushes to the user's GitHub. You own everything. Kepion is the builder, not the landlord.

Questions for you

I'm genuinely curious:

How do you handle AI agent costs in production? We built a 4-tier model routing system (Free → Budget → Performance → Premium) with auto-escalation on failure. Is anyone doing this differently?
Team Memory vs RAG — what's your experience? We went with vault-based memory with effectiveness scoring instead of pure vector search. The scoring means bad memories decay. Has anyone combined both approaches?
What's your threshold for "good enough" security in an MVP? We went aggressive (deny-by-default, path traversal, rate limiting) before launch. Some say ship fast, secure later. Curious where others draw the line.

Follow the build: GitHub | kepion.app

TraceHawk vs Datadog for AI Agent Monitoring in 2026

Pavel Gajvoronski — Tue, 14 Apr 2026 08:02:08 +0000

"I built TraceHawk after spending hours debugging why my AI agent was making 47 filesystem calls before a single GitHub call. Datadog showed me the waterfall. It didn't show me the why."

TraceHawk vs Datadog for AI Agent Monitoring in 2026

I built TraceHawk after spending hours debugging why my AI agent was making 47 filesystem calls before a single GitHub call. Datadog showed me the waterfall. It didn't show me the why.

This comparison covers what Datadog actually gives you for AI agent observability, where it falls short for MCP-heavy workloads, and why teams are switching to purpose-built tools like TraceHawk. I'm going to be honest about both sides — Datadog is genuinely good at some things, and acknowledging that matters more than cheerleading.

What Datadog gives you for AI agents

Datadog's LLM Observability module launched in 2024 and has matured significantly. The Python agent (v10.13.0, June 2025) added MCP client tracing — waterfall diagrams for MCP requests, automatic instrumentation for tool invocations, session correlation. If you're already a Datadog customer, this is zero additional setup.

The strongest argument for Datadog is the unified view. If an LLM latency spike is caused by a downstream database slowdown, Datadog shows you both in the same trace. Your AI layer, your infrastructure, your queues — one pane of glass. That's genuinely valuable and not something purpose-built LLM tools can replicate.

Datadog also has enterprise compliance sorted: SOC2 Type II, HIPAA, PCI DSS. If you're in a regulated industry, that matters.

Where Datadog genuinely wins: AI as one component of a complex system you already monitor. The correlation between LLM latency and infrastructure health is something no standalone LLM tool can match.

Where Datadog falls short

The cost gap is real

Datadog's LLM Observability is priced per event, stacked on top of existing APM costs. For teams running agents at scale — thousands of traces per day — the math gets uncomfortable fast. Enterprise contracts start at $50k/year. That's before the AI-specific add-ons.

TraceHawk is $99/month flat for unlimited spans, with a 50K span/month free tier. For a startup running agents as core product, this difference is existential.

MCP as an afterthought

Datadog added MCP support in June 2025 — 18 months after MCP launched. It traces MCP client sessions and tool invocations, but it's built on top of their generic APM span model. What you get: session ID, tool name, latency, error code. What you don't get:

✗ MCP server health dashboard with uptime and degradation detection
✗ Per-server p50/p95 latency trends (not just per-call)
✗ Error rate by server (which of your 12 MCP servers is flaky?)
✗ Tool call heatmap — when during the day does each server get hammered?
✗ Degraded server alerts — notify when error rate crosses a threshold

TraceHawk was built around MCP from day one. Every MCP tool call gets structured telemetry automatically:

{
  "span_kind": "MCP",
  "mcp.server_name": "filesystem",
  "mcp.tool_name": "read_file",
  "mcp.tool_input": { "path": "/workspace/src/auth.ts" },
  "mcp.output_size_bytes": 4280,
  "duration_ms": 12,
  "status": "ok",
  "trace_id": "3e4f5a6b...",
  "parent_span_id": "1a2b3c4d"
}

Agent decisions are invisible

Datadog shows you a trace waterfall — spans in chronological order. You can see what happened, but not why. When your agent calls the filesystem server 47 times before calling GitHub, a flat waterfall doesn't explain the decision path.

TraceHawk parses parent-child span relationships into a visual decision tree: root is the task, branches are LLM decisions, leaves are tool calls. You can see exactly why the agent chose one tool over another, and what context it had at each decision point.

No agent session replay

Datadog has no concept of agent session replay. TraceHawk shows a step-by-step session timeline — agent start, each LLM call with full prompt and response, each tool invocation, each MCP server response. Click any event to expand full detail. This is what you need when debugging why an agent got stuck in a loop or made an unexpected decision.

Cost attribution vs token tracking

Datadog tracks token usage. TraceHawk tracks token costs — with per-model pricing tables updated as models change, per-agent cost budgets, and alerts when a specific agent is trending toward budget overage before the month ends. That's a different product than a token counter.

Full feature comparison

Feature	TraceHawk	Datadog
Price	$99 / month	$50k+ / year (enterprise)
Free tier	50K spans/month	Limited trial
MCP-native tracing	✅ Day one	⚠️ Added June 2025
MCP server health dashboard	✅ Built-in	❌ Not available
Per-server error rates	✅	❌
Tool call heatmap	✅ Time × server	❌
p50 / p95 per MCP server	✅	❌
Degraded server alerts	✅ Slack / PagerDuty	❌
Agent decision tree	✅ Visual	❌
Agent session replay	✅ Step-by-step	❌
Prompt / response viewer	✅	✅
Token cost attribution	✅ Per span / budget	⚠️ Token count only
Budget alerts	✅	❌
Infra correlation (APM)	❌	✅ Core strength
APM + AI unified view	❌	✅
SOC2 / HIPAA	⚠️ Planned	✅
Self-hosted	✅ Open source	❌
Setup time	2 minutes	1–2 weeks
SDK install	pip install tracehawk	Datadog agent

When to choose Datadog

Be honest with yourself here. Datadog is the right choice if:

You already pay for Datadog and AI is a small part of your monitored system
You need to correlate LLM latency with infrastructure failures — the unified view is genuinely valuable
Enterprise compliance requirements today (HIPAA, PCI DSS) — TraceHawk doesn't have these yet
Your AI layer is one piece of a complex distributed system you monitor with Datadog
Your team has Datadog expertise and doesn't want to learn another tool

When to choose TraceHawk

Your product IS the AI agent — observability needs to be deep, not broad
You use MCP servers and need real visibility into per-server performance
You want to understand agent decisions, not just log them
Cost attribution at the span level with budget management matters
You're a startup or small team ($99/mo vs $50k/yr is a real constraint)
You need to be set up in 2 minutes, not 2 weeks
You want the open-source option — TraceHawk is self-hostable

Bottom line

Datadog is a great choice if you already use it and AI is a small part of your stack. The unified infrastructure + AI view is a real advantage that purpose-built tools can't replicate. But the cost structure is built for enterprises monitoring everything, not teams whose entire product is an AI agent.

If AI agents are your core product — especially if you use MCP servers — you need a tool built around them, not retrofitted for them. TraceHawk gives you MCP-native tracing, agent decision trees, session replay, and cost budgets in one place, at a fraction of the cost.

The 50K span free tier covers most development and early-stage production workloads. You can instrument your first agent in 2 minutes and see the difference yourself.

→ Try TraceHawk free — no credit card required.

Tags: #aiagents #observability #mcp #datadog

I Built a Voice AI GMAT Tutor with Long-Term Memory in 6 Weeks — Here's the Full Stack

Pavel Gajvoronski — Mon, 13 Apr 2026 11:34:43 +0000

samiwise.app — live now

GMAT prep tutors charge $150–200 per hour. For a 3-month prep period, that's $5,000–10,000. Most people preparing for an MBA simply can't afford that — or can't find a good tutor available at 11pm when they finally have time to study.

So I built SamiWISE — a voice AI GMAT tutor that remembers every session, adapts to your weak spots, and explains material in real time using RAG over official GMAT materials. This is the story of how it was built, what I learned, and the technical decisions that made it work.

The three things no competitor combines: voice + memory + real GMAT content

The Core Problem I Was Solving

Every GMAT prep tool I looked at had the same fundamental issue: they start from scratch every single session. You explain your weak spots again. You get generic explanations that don't account for what confused you last Tuesday. There's no continuity.

A good human tutor doesn't do this. They remember that you always mess up Data Sufficiency with inequalities. They know that analogies work better for you than abstract explanations. They track your trajectory over weeks.

I wanted to build that — but accessible to everyone, available 24/7, at $49/month.

The Architecture

The system has four main layers:

User (voice)
  → Deepgram STT (~1s)
  → Orchestrator Agent — Groq llama-3.3-70b (~200ms routing)
  → Specialist Agent — Claude Sonnet + RAG from Pinecone (~3-5s)
  → ElevenLabs TTS (~1s)
  → User hears response

Total latency: 5–8 seconds. Not perfect, but feels natural — like a real tutor pausing to think.

The Agent System

The most interesting architectural decision was the multi-agent routing system.

Instead of one monolithic AI tutor, there are five specialist agents and an invisible orchestrator:

Agent	Specialization
`quantitative`	Problem Solving + Data Sufficiency
`verbal`	Critical Reasoning + Reading Comprehension
`data_insights`	Table Analysis, MSR, Graphics Interpretation, TPA
`strategy`	Timing, exam psychology, study planning
`orchestrator`	Routes messages — user never sees this

The orchestrator runs on Groq (llama-3.3-70b) because it needs to be fast — 200ms routing decisions. Specialist agents run on Claude Sonnet because they need to be smart.

Routing prompt returns structured JSON:

{
  "route": "quantitative",
  "confidence": 0.94,
  "detected_topic": "data sufficiency with inequalities",
  "difficulty": "hard",
  "notes": "user has struggled with DS inequalities in past 3 sessions"
}

The user always hears the same voice — Sam. Transitions between agents are completely invisible.

Every other GMAT tool treats you like a stranger on every visit. Sam carries your entire learning history.

The Memory System — The Hard Part

This is where most AI tutors fail. Building long-term memory that actually improves tutoring quality took the most iteration.

After every session, a Memory Agent runs in the background. It reads the full session transcript and extracts a structured learner profile:

interface GmatLearnerProfile {
  weak_topics: string[]
  strong_topics: string[]
  effective_techniques: string[]      // what explanation styles worked
  ineffective_approaches: string[]    // what didn't land
  insight_moments: string[]           // "aha" phrases that clicked
  common_error_patterns: string[]     // e.g. "misreads DS question stem"
  learning_style: string
  next_session_plan: string
  score_trajectory: string
  time_pressure_notes: string
}

This profile gets stored in Supabase as a JSON field on the User model. At the start of every session, the full profile is injected into the specialist agent's system prompt.

The result: Sam says things like "Last week you struggled with probability in DS — let's approach this one differently than before" without you having to explain anything.

RAG — What I Indexed and Why

The knowledge base lives in Pinecone with 7 namespaces:

gmat-quant       → Quantitative problems and methods
gmat-verbal      → Verbal problems and methods
gmat-di          → Data Insights problems
gmat-strategy    → Strategies, timing, test psychology
gmat-focus       → GMAT Focus Edition specific content
gmat-errors      → Common error patterns

Free sources I used:

deepmind/aqua_rat — 97,467 GMAT/GRE algebra problems with rationales (Apache 2.0)
allenai/math_qa — Math word problems with annotated formulas (Apache 2.0)
mister-teddy/gmat-database — DS, PS, CR, SC questions in JSON (MIT)
ReClor paper — 17 CR question types with examples (research)
Manhattan Review free PDFs — Strategy guides openly distributed

The RAG pipeline uses @xenova/transformers for embeddings (runs locally, no API cost) and retrieves top-5 chunks with reranking before passing to the specialist agent.

The Tech Stack

Frontend:     Next.js 14 + TypeScript + Tailwind CSS
Auth:         Supabase Auth
Database:     Supabase PostgreSQL + Prisma 6
Vector DB:    Pinecone (7 namespaces)
LLM Router:   Groq (llama-3.3-70b) — fast, cheap
LLM Agents:   Anthropic Claude Sonnet — smart, consistent
STT:          Deepgram (Whisper)
TTS:          ElevenLabs
Memory:       Custom Memory Agent → Supabase JSON
Payments:     Paddle (Merchant of Record, handles US tax)
Deploy:       Vercel (frontend) + Railway (agents backend)

Why split Vercel + Railway?

Vercel has an 800-second serverless function limit. A 30-minute voice tutoring session would time out. Railway runs persistent containers — no limits, no cold starts for agents.

Practice Mode with FSRS

Beyond voice sessions, I built a visual practice mode where users can work through GMAT questions in exam format.

The interesting part: I implemented FSRS (Free Spaced Repetition Scheduler) — the same algorithm used by Anki. After each answer, the system records:

Was it correct?
How long did it take?
What was the difficulty?

Then it schedules the next review using an exponential forgetting curve. Questions you answered wrong come back sooner. Questions you mastered disappear for weeks.

This means the practice queue automatically prioritizes your weak spots without you having to manage anything.

The Study Journal

Every session automatically updates a daily journal entry:

interface StudyJournalEntry {
  date: Date
  totalMinutes: number
  questionsTotal: number
  accuracy: number
  topicsCovered: string[]
  errorTypes: Record<string, number>
  samInsight: string          // AI-generated daily summary
  milestones: string[]        // "100 questions solved", "5 hour week"
  streakDay: number
}

The streak counter turned out to be unexpectedly powerful for retention — users don't want to break their streak. Same psychology as Duolingo, but for GMAT prep.

What Doesn't Work Yet

Being honest about where things stand:

Voice pipeline not live yet — Deepgram + ElevenLabs keys configured but need production testing with real users
RAG not indexed — scripts are ready, Pinecone account set up, but haven't pushed the data yet
No real users — launching next week, zero feedback so far

The architecture is built. The UI works. The agents respond correctly. Next step is connecting all the APIs and getting real people to use it.

Lessons Learned

1. Multi-agent routing is worth the complexity.
A single "GMAT tutor" prompt produces mediocre results across all topics. Specialist agents with deep domain prompts are significantly better. The routing overhead is minimal.

2. Memory quality matters more than memory quantity.
I originally tried to store everything — full transcripts, every message. The prompts became too long and performance degraded. The Memory Agent that extracts structured insights (not raw content) works much better.

3. Split your infrastructure early.
I almost deployed everything to Vercel. The 800-second limit would have killed voice sessions. Railway for long-running processes saved the architecture.

4. Free datasets are better than I expected.
The deepmind/aqua_rat dataset has 97,000 high-quality GMAT-style problems with step-by-step rationales. Apache 2.0 license. This single dataset provides more practice material than most paid prep courses.

5. Paddle for payments if you're targeting the US market.
They handle sales tax across all 50 states automatically. As a Merchant of Record, they handle chargebacks and disputes. The 5% + $0.50 fee is worth it for the peace of mind.

What's Next

Get first 10 beta users from r/GMAT and GMAT Club
Connect production APIs (Deepgram, ElevenLabs, Pinecone)
Run the RAG indexing scripts
Collect feedback on voice experience quality

If you're interested in trying it or have feedback on the architecture, I'd love to hear from you. The product is live at samiwise.app — 7-day free trial, no credit card required.

Built with Next.js, Claude Sonnet, Groq, Pinecone, Deepgram, ElevenLabs, Supabase, Paddle, and Railway. Full stack TypeScript.

I'm Building a Platform That Deploys AI Companies From a Single Sentence

Pavel Gajvoronski — Mon, 13 Apr 2026 08:41:01 +0000

I'm building an AI Company Builder — a platform where you describe a business idea in plain text, and a team of 28 AI agents researches the market, validates viability, designs the product, writes the code, creates content, and runs marketing. All autonomously.

This is not a chatbot. This is not another wrapper around ChatGPT. This is a full-stack agent orchestration platform with a two-layer architecture, persistent knowledge vault, multi-model routing across 300+ models, and a built-in marketplace for buying and selling AI-powered businesses.

I want to share the architecture, the tech stack, and the decisions I made — because I haven't seen anyone build exactly this combination yet.

The problem

Right now, if you want to launch a business with AI, you're stitching together 5-10 tools manually: ChatGPT for strategy, Lovable for code, Jasper for content, Perplexity for research, Notion for knowledge, Zapier for automation. Each tool does one thing. None of them talk to each other. And none of them understand your business as a whole.

What if one platform did it all? Not by being mediocre at everything — but by orchestrating specialized agents, each expert in their domain, all sharing context through a persistent knowledge vault?

The architecture: two layers, not one

Most agent platforms put all agents on the same level. Marketing agent, coding agent, sales agent — flat list, no hierarchy. This works for simple automation but breaks down when you're building a complete business.

I went with a two-layer approach:

Business layer — 7 manager agents that understand your specific business. Product Manager (Max), Marketing Lead (Ivy), Sales Strategist (Sam), Financial Analyst (Finn), Customer Success (Joy), Legal Advisor (Lex), and a Business Generator (Chief) that creates the whole structure from your description. These agents know your niche, your competitors, your audience.

Tool layer — 21 universal agents that do the actual work. Architect (Atlas), Designer (Maya), Frontend Dev (Kai), Backend Dev (Dev), Security (Shield), Researcher (Nova), Writer (Sage), and 14 more. These don't know your business — they know their craft. The business layer delegates to them with full context.

The key insight: business agents are per-business instances, tool agents are shared. If you run 3 businesses simultaneously, each has its own Max and Ivy, but they all share the same Atlas and Kai. This scales without multiplying costs.

Model routing: 90% quality at 7% cost

Running everything on Claude Opus 4.6 would cost a fortune. Running everything on a cheap model would produce garbage. The answer is intelligent routing.

I use OpenRouter as a gateway to 300+ models, organized in 4 tiers:

Tier	Models	Cost/1M tokens	Use case
Free	Llama 3.3 70B	$0	Routing, classification
Budget	DeepSeek V3, Gemini Flash	$0.14-0.60	Content writing, planning
Performance	MiniMax M2.7	$0.30-1.20	Coding, testing, debugging
Premium	Claude Sonnet/Opus 4.6	$3-25	Architecture, security, design

MiniMax M2.7 is the secret weapon here. In real-world tests, it delivers 90% of Opus quality for 7% of the cost. It found all 6 bugs and all 10 security vulnerabilities that Opus found — the fixes were just slightly less thorough. For most coding tasks, that's more than enough.

The system also auto-escalates: if an agent fails 3 times on a cheaper model, it automatically upgrades to the next tier. And auto-downgrades: 10 consecutive successes on Sonnet? The system suggests trying M2.7 next time.

A full project milestone that costs $50-80 on all-Opus runs $10-12 with routing. That's 80-85% savings.

The research stack: not just chat, actual research

This is where I think most agent platforms fall short. They can write code and generate content — but they can't research. They don't know what's happening in the market right now.

My stack includes four self-hosted research tools:

Perplexica — open-source Perplexity alternative. AI-powered web search with cited sources. When Nova (researcher agent) needs to analyze a market, she searches the web through Perplexica and gets answers with real citations, not hallucinations.

SurfSense — open-source NotebookLM alternative. Upload documents, chat with them, get cited answers. Hybrid search (semantic + full text). Can even generate podcasts from documents.

AnythingLLM — RAG workspace for document analysis. Upload PDFs, DOCX, code files — agents query them with grounded answers.

Firecrawl — web scraping via MCP. Agents can scrape any URL into clean markdown, crawl entire websites, extract structured data.

The combination means agents can research a market, analyze competitors, scrape their pricing pages, summarize uploaded pitch decks, and cite every claim with a real source.

The gate system: think before you build

Here's what nobody else does. Before my system commits resources to building something, it analyzes whether it's worth building.

You write: "Build an online chess school for kids 6-14. Analyze viability first. Only proceed if rating is above 7/10."

The system runs a full analysis:

Criterion	Weight	Score
Market size	20%	8/10
Competition level	20%	7/10
Niche uniqueness	15%	9/10
Revenue potential	15%	8/10
Acquisition cost	15%	5/10
Channel accessibility	15%	8/10
Overall		7.4/10

If it passes the threshold — development begins. If not — the system explains why and suggests modifications. "Focus on children 6-10 instead of 6-14 — less competition, higher willingness to pay. Adjusted score: 8.1/10."

This saves thousands of dollars and weeks of development on ideas that won't work.

Persistent memory: the Obsidian Vault

Every research finding, every architectural decision, every bug fix, every content plan — saved as markdown notes in an Obsidian-compatible vault with git version control.

The vault isn't just storage. It's a living knowledge base:

Auto-indexing: Vault Librarian agent (Libra) maintains indexes, tags notes, creates links between related decisions
Git history: every change tracked, every note timestamped, full rollback capability
Memory consolidation: Libra periodically merges scattered notes into coherent knowledge structures
Cross-project learning: insights from one project automatically available in related projects

After 3 months of operation, the vault contains hundreds of notes — and the system is measurably smarter. Nova doesn't re-research topics she already investigated. Atlas references past ADRs when making new architecture decisions. The knowledge compounds.

Event-driven architecture: everything is observable

Every agent action emits an event to Redis pub/sub:

{
  "agent": "Atlas",
  "action": "created_adr",
  "model": "claude-opus-4.6",
  "tokens_in": 2400,
  "tokens_out": 5100,
  "cost_usd": 0.142,
  "duration_ms": 8300,
  "vault_note": "projects/chess/decisions/ADR-001.md"
}

Multiple services subscribe: the audit logger saves to immutable JSONL, the cost tracker aggregates spending, the vault manager auto-saves results, and the live activity stream pushes to the Web UI via WebSocket.

This gives you:

Full audit trail for compliance (EU AI Act, GDPR)
Real-time cost tracking with ROI calculation ("Your agents saved $28,000 in equivalent human labor this month")
Live activity feed — watch your agents work in real-time
Kill switch — instantly halt all agent activity if something goes wrong

A2A protocol readiness

Google's Agent-to-Agent protocol (A2A) is becoming the standard for inter-platform agent communication. 50+ partners including Salesforce, SAP, and PayPal are building on it.

I'm building A2A compatibility from day one. Every agent has an Agent Card — a JSON file describing its capabilities. External agents can discover our agents, send tasks, and receive results through standardized endpoints.

Why this matters: in 2027-2028, your business agents will negotiate with supplier agents, customer agents will talk to support agents across platforms, and marketing agents will coordinate campaigns with influencer agents — all machine-to-machine. Building the protocol layer now means we're ready when this arrives.

What's coming next

The full platform has 14 milestones. I'm currently on the build phase, deploying infrastructure on a Hetzner VPS with Claude Code + GSD-2 running the development process.

What I'm building toward:

YouTube content pipeline: from idea to published video, fully automated
Business Exchange: marketplace for buying and selling AI-powered businesses
Cross-business learning: anonymous patterns shared across all businesses on the platform
400+ integrations via Composio: Gmail, Slack, HubSpot, Notion, Jira — one MCP server
7 business templates: Online Education, SaaS, Agency, E-commerce, Content, Marketplace, Coaching

Tech stack summary

Layer	Technology
Server	Ubuntu 24.04, Hetzner CPX31
AI Engine	Claude Code CLI via OpenRouter
Orchestration	GSD-2 + Ruflo
API Gateway	FastAPI + Redis pub/sub
Interfaces	Telegram (aiogram), React Web UI, REST API
Research	Perplexica, SurfSense, AnythingLLM, Firecrawl
Integrations	Composio (400+ apps), MCP servers
Memory	Obsidian Vault + MCPVault + git
Payments	Stripe, Paddle (MoR)

Why I'm sharing this

Two reasons. First, I genuinely believe this is where software is heading — from tools to autonomous business operators. The predictions from Dario Amodei, Sam Altman, and every major AI lab point to agents handling multi-week projects autonomously by 2028. Building the platform for this now is a bet on the near future.

Second, building in public keeps me honest. If you see flaws in the architecture, I want to know. If you're building something similar, let's compare notes. If you want to be an early user — I'll be opening access soon.

Follow the build: I'll be posting weekly updates here on dev.to with technical deep dives into each component.

What would you build if you had 28 AI agents at your command?

TraceHawk vs LangSmith: AI Agent Observability in 2026

Pavel Gajvoronski — Fri, 10 Apr 2026 10:02:19 +0000

LangSmith is the default choice for LangChain teams. But if your stack has moved beyond LangChain — or you're using MCP servers — you're working around LangSmith, not with it.

Feature Comparison

Feature	TraceHawk	LangSmith
MCP server name captured	✅ Always	⚠️ Requires manual tagging
Per-server latency (p50/p95)	✅ Built-in	❌ Not tracked
MCP error details	✅ Full error + stack	❌ Not available
MCP server health dashboard	✅ Built-in	❌ Not available
OTEL-native ingest	✅ OTLP endpoint	⚠️ LangChain-first, OTEL adapter
LLM call tracing	✅	✅
Cost attribution	✅ Per agent/trace/org	✅ Per run
Prompt versioning / hub	⚠️ Roadmap	✅ LangSmith Hub
Agent replay timeline	✅ Step-by-step	✅ Run timeline
Dataset / eval harness	❌ Not in scope	✅ Built-in
Retry loop detection	✅ Automatic badge	❌ Not available
OTEL dual-write re-export	✅ Built-in fan-out	❌ Not available
Self-host option	✅ Open source core	❌ Cloud only (Enterprise)
Free tier	50K spans/month	Limited (Developer)
Pro tier	$99/month	$39/month (25 seats)
Framework support	Any (OTEL-compatible)	LangChain/LangGraph-first

The core difference

LangSmith was built to observe LangChain chains. Everything else is a wrapper around that mental model. TraceHawk was built around OpenTelemetry from day one — which means any framework, any language, and first-class support for Model Context Protocol.

This isn't a criticism of LangSmith. It's the right tool if your entire stack is LangChain/LangGraph and you want deep eval/dataset tooling. The question is whether that describes your stack in 2026.

MCP support: built-in vs bolted on

Model Context Protocol is now the dominant way AI agents use tools — Claude Code, LangGraph, CrewAI, OpenAI Agents SDK all support it natively. LangSmith doesn't have a concept of "MCP server" — you can log the spans manually, but there's no:

Per-server health dashboard (error rate, p95 latency, call frequency)
Automatic tool name extraction from mcp.tool_name attributes
Server degradation alerts
MCP-aware retry loop detection
Agent → server dependency graph

In TraceHawk, all of this is automatic. If you emit standard OTLP spans with mcp.server_name and mcp.tool_name attributes, the dashboard populates itself. No configuration required.

Framework independence

LangSmith works best with LangChain. The tracing callbacks are tightly coupled to the LangChain execution model — on_llm_start, on_tool_end, etc. If you switch to OpenAI Agents SDK, CrewAI, or write a custom agent, you're on your own.

TraceHawk uses OTLP as the ingest protocol. Any framework that emits OpenTelemetry spans works out of the box — including LangChain, LangGraph, CrewAI, OpenAI Agents SDK, Claude Code hooks, and custom agents. One endpoint, everything traces.

When LangSmith wins

LangSmith has capabilities TraceHawk doesn't aim to replicate:

Prompt Hub — version-controlled prompt management with deployment
Evaluation datasets — structured datasets for regression testing
LangChain-native callbacks — zero-config if your stack is 100% LangChain
LangGraph Studio integration — visual graph debugging

If your workflow is "build in LangGraph, test with eval datasets, iterate on prompts in Hub" — LangSmith is genuinely great. TraceHawk doesn't try to replace that.

When TraceHawk wins

Your stack uses MCP servers (Claude Code, custom MCP, any framework)
You want OTEL-native ingest without framework lock-in
You need cost attribution per agent/trace/organization
You want to self-host (open source core, Docker-deployable)
You need retry loop detection and server health alerts
You want to dual-write to Datadog/Grafana simultaneously

Pricing

LangSmith Developer tier is free with limited traces. Their paid plans start at $39/month for a team of 25. TraceHawk is $0 for 50K spans/month, $99/month for unlimited — no per-seat pricing, no surprise overages.

For production AI agent teams, the relevant comparison is: LangSmith Plus ($99–$499/month, per-seat) vs TraceHawk Pro ($99/month flat). If your team is 5+ people, TraceHawk is cheaper.

The bottom line

LangSmith is excellent if you're all-in on LangChain. TraceHawk is the right choice if you're using MCP, want framework independence, or need production-grade observability without per-seat pricing.

They're not direct competitors — LangSmith is a LangChain-native eval platform that includes tracing. TraceHawk is an OTEL-native observability platform that focuses on what matters for AI agent teams in 2026: MCP visibility, cost attribution, and production alerting.

Try TraceHawk free: 50K spans/month, no credit card. tracehawk.dev

EU AI Act Deadline August 2026: What SMBs Need to Do Now

Pavel Gajvoronski — Fri, 10 Apr 2026 09:55:39 +0000

Five months. That's roughly how long you have until the EU AI Act's high-risk compliance deadline on August 2, 2026.

If you're running a small or medium-sized SaaS company with AI features and European customers, this deadline should be on your radar. Even if you've heard rumors about a possible extension, the legal reality is more nuanced — and more urgent — than the headlines suggest.

This article gives you the full picture: what's already in force, what's coming, what the Digital Omnibus actually says, and a concrete 5-month plan to get your house in order.

The Full EU AI Act Timeline: What's Already Happened

The AI Act doesn't flip a single switch on one date. It phases in across multiple milestones:

August 1, 2024 — Entry Into Force

The regulation was published in the Official Journal. The clock started ticking, but no obligations applied yet.

February 2, 2025 — Prohibited Practices + AI Literacy

Two categories of rules became enforceable. First, prohibited AI practices under Article 5 — social scoring, subliminal manipulation, exploitation of vulnerabilities, certain biometric uses, and predictive policing — are now banned. Violations carry penalties of up to €35 million or 7% of global turnover.

Second, AI literacy obligations under Article 4 require organizations to ensure their staff have sufficient understanding of AI systems they develop or use. The European Commission has since proposed shifting primary responsibility for AI literacy to member states and the Commission itself under the Digital Omnibus, but until that's adopted, the original obligation stands.

August 2, 2025 — GPAI Rules + Governance Infrastructure

This was the second major milestone. General-purpose AI (GPAI) model obligations under Article 53 became applicable, covering technical documentation, training data summaries, and copyright compliance. The GPAI Code of Practice was published in July 2025, and 26 major providers signed it, including Amazon, Anthropic, Google, IBM, Microsoft, OpenAI, and Mistral AI. Notably, Meta declined to sign.

The EU governance infrastructure also came online: the AI Office, the AI Board, the Scientific Panel, and the Advisory Forum all became operational. Member states were required to designate national competent authorities.

The penalty regime also took effect — market surveillance authorities can now impose fines for non-compliance. However, enforcement powers specific to GPAI model providers don't kick in until August 2, 2026.

August 2, 2026 — The Big One

This is when the majority of the AI Act becomes enforceable. Key elements include full compliance requirements for high-risk AI systems listed in Annex III (covering hiring, credit scoring, biometrics, education, emergency services, and more), transparency obligations under Article 50 for limited-risk systems, innovation measures including AI regulatory sandboxes (at least one per member state), and full enforcement at both national and EU levels.

August 2, 2027 — High-Risk Products

Rules for high-risk AI systems that are safety components of regulated products (covered under Annex I EU harmonization legislation) apply from this date. This primarily affects manufacturers of physical products like medical devices, machinery, and vehicles.

The Digital Omnibus: What It Actually Says

On November 19, 2025, the European Commission published the Digital Omnibus — a sweeping proposal to simplify the EU's digital regulatory framework. For the AI Act specifically, the most significant proposal involves extending the compliance timeline for high-risk systems.

Here's what the Omnibus proposes for high-risk AI:

Conditional delay: High-risk obligations would not apply until the Commission confirms that adequate compliance support — harmonized standards, common specifications, or guidelines — is available. Once confirmed, Annex III systems (standalone high-risk uses like hiring and credit scoring) would have 6 months to comply. Annex I systems (product safety components) would have 12 months.

Backstop dates: Even if standards aren't ready, rules would apply no later than December 2, 2027 for Annex III systems and August 2, 2028 for Annex I systems.

Grace period for GPAI transparency: Providers of generative AI systems placed on the market before August 2026 would get until February 2, 2027 to meet content-marking transparency obligations.

SME-friendly simplifications: Simplified quality management system (QMS) requirements under Article 17 would be extended from microenterprises to all SMEs.

Why You Shouldn't Rely on the Omnibus

There are three critical reasons not to treat the Omnibus as a get-out-of-jail-free card.

First, it's not law yet. The Omnibus is a Commission proposal that must go through trilogue negotiations with the European Parliament and Council. This process could take months, and the final text may look significantly different from the current proposal.

Second, the timing is extremely tight. For the Omnibus to have any effect before August 2, 2026, it must be adopted before that date. If Parliament and Council don't agree in time, the original deadline applies as written. Multiple legal commentators have flagged this as a realistic risk.

Third, the core framework stays. Even under the Omnibus, the AI Act's risk classification, prohibited practices, and obligation structure remain intact. The delay is about timing, not about watering down requirements. Every obligation you'd need to meet in August 2026 you'll still need to meet by December 2027 at the latest.

The Commission itself has called this a "structural recalibration," not deregulation.

The Penalty Framework

For SMBs, the financial exposure is significant:

Up to €35 million or 7% of global annual turnover for violations involving prohibited AI practices. Up to €15 million or 3% of turnover for violations of high-risk system obligations or GPAI provider obligations. Up to €7.5 million or 1% of turnover for providing incorrect, incomplete, or misleading information to authorities.

The AI Act does include proportionality considerations for SMEs and startups (Article 99(6)), with penalties accounting for the size of the company. But "proportional" doesn't mean "zero" — it means you'll face a fine calibrated to your revenue rather than the maximum cap.

Beyond direct penalties, non-compliance carries other consequences. Market surveillance authorities can order you to withdraw your AI system from the EU market. Customers — especially enterprise buyers — are increasingly asking about AI Act compliance during procurement. And a public enforcement action can do lasting reputational damage.

Your 5-Month Action Plan: March to August 2026

Here's a concrete month-by-month plan for an SMB getting serious about compliance now.

Month 1 (March): Inventory and Classification

Start by creating a complete inventory of every AI system you develop, deploy, or use. For each system, document what it does, what AI model or method it uses, what data it processes, who is affected by its outputs, and what markets it serves.

Then classify each system by risk level. Map them against the 8 Annex III categories. Check Article 6(3) exceptions. Determine your role — provider, deployer, or both.

This step alone gives you clarity on what applies to you. If all your systems are minimal or limited risk, your path forward is much simpler.

Month 2 (April): Gap Analysis

For each high-risk system, assess your current compliance status against Articles 9 through 15. For each requirement, determine whether you already have it, partially have it, or are starting from zero.

Key questions to answer: Do you have a documented risk management system? How is your training data governed and documented? Do you have technical documentation that meets Annex IV's 9 sections? Are your systems logging decisions automatically? Can users understand how decisions are made? Is there meaningful human oversight? Have you tested for accuracy, robustness, and cybersecurity?

Calculate a compliance score. Prioritize critical gaps — especially Articles 9 (risk management), 10 (data governance), and 11 (technical documentation), as these are the most time-consuming to address.

Month 3 (May): Documentation Sprint

The Annex IV technical documentation is the most labor-intensive requirement. It demands 9 sections covering your system's general description, development process, monitoring and control mechanisms, performance metrics, risk management approach, lifecycle changes, applied standards, EU declaration of conformity, and post-market monitoring plan.

Start drafting. Use templates where available. If you have multiple high-risk systems, look for common elements you can reuse across documentation.

Also prepare your risk management system documentation (Article 9) and data governance records (Article 10).

Month 4 (June): Implementation and Testing

Implement any technical measures you're missing. Common gaps include automatic logging (Article 12) — make sure your system records key decisions, inputs, and outputs. Human oversight mechanisms (Article 14) — ensure humans can effectively monitor and intervene. Transparency information (Article 13) — create clear documentation for deployers about how your system works, its limitations, and correct use.

If you're a deployer of AI tools, review your vendor's documentation. Under the AI Act, deployers must verify that their high-risk AI providers have conducted conformity assessments and registered their systems.

Month 5 (July): Conformity and Registration

Complete your conformity assessment. For most Annex III high-risk systems, this is a self-assessment (the provider evaluates their own compliance). Only biometric identification systems used in law enforcement require third-party assessment.

Prepare your EU Declaration of Conformity (Article 47). Register your high-risk AI systems in the EU database (Article 49). The AI Office is expected to provide registration tools.

Run a final compliance review. Fix any remaining gaps. Brief your team.

What If the Omnibus Gets Adopted?

If the Digital Omnibus passes before August 2, 2026, you'll have more time — but all the work you've done still counts. You'll simply have a buffer to refine and finalize rather than rushing.

If it doesn't pass, you'll be compliant on time while competitors who gambled on the extension scramble to catch up.

Either way, early preparation is the winning strategy.

Special Considerations for SMBs

The AI Act and the Omnibus include several provisions specifically aimed at reducing the burden on smaller companies.

Simplified QMS: Under the proposed Omnibus, SMEs would access simplified quality management system requirements previously available only to microenterprises.

Regulatory sandboxes: Each member state must have at least one AI regulatory sandbox operational by August 2026. These provide a controlled environment where companies can test AI systems with regulatory guidance.

Proportional penalties: Fines are scaled to company size and revenue.

Free tools: The AI Office's Service Desk answers compliance questions. The GPAI Code of Practice provides templates. And platforms like Complyance offer self-serve classification and gap analysis at a fraction of what enterprise consultants charge.

The Bottom Line

The August 2, 2026 deadline is real until legislation says otherwise. The Digital Omnibus might give you extra time, but it might not pass in time. Either way, every action you take now directly reduces your compliance gap and your risk exposure.

The companies that start now will be the ones with a genuine competitive advantage — able to demonstrate AI compliance to enterprise buyers, avoid regulatory penalties, and build trust with customers who increasingly care about responsible AI.

Start today. Classify your AI systems for free at complyance.io. Get your risk classification, see your compliance gaps, and build your roadmap — all in one session, no sales calls required.

Disclaimer: This article is for informational purposes only and does not constitute legal advice. Compliance planning should be verified with a qualified legal professional specializing in AI regulation.