Forem: Varshith V Hegde

I Spent 3 Days Debugging Our LLM Setup. Turns Out We Needed an AI Gateway the Whole Time.

Varshith V Hegde — Wed, 15 Apr 2026 08:46:08 +0000

Let me tell you about a Friday afternoon I'd rather forget.

Three teams, four models, six API keys living in different .env files, one very angry compliance officer, and me just staring at a terminal trying to figure out why we got a $1,400 OpenAI bill for a feature that was supposed to cost fifty bucks.

That was my "okay something is genuinely broken here" moment.

Not some big insight. Just a $1,400 invoice and dead silence on a Slack thread for about ten minutes.

If you've felt even a small version of that, this post is for you.

So what actually is an AI Gateway?

Not the textbook answer. That one goes something like "middleware that abstracts your LLM provider calls." Technically fine, tells you nothing.

Here's how I actually think about it.

You know how bigger engineering orgs eventually build out a platform team? Before that team exists, every squad is doing their own thing. Their own CI setup, their own infra configs, their own credentials. It mostly works. Until it doesn't. And then it catastrophically doesn't all at once.

An AI Gateway is basically that platform layer, except it's for LLMs.

Every single request your app makes to any model (OpenAI, Anthropic, a self-hosted Llama, whatever you're running) goes through it. Because everything flows through one place, you finally get:

One set of credentials instead of keys scattered across five repos
Rate limits and budgets that are actually enforced
Cost tracking per team, per model, per request
Guardrails that catch PII before it leaves your infra
One place to look when something blows up

One control plane. Every team. Every model.

The architecture is simpler than it sounds

Here's what happens when you put a gateway in the middle:

Request comes in from your app, gateway catches it, validates auth, checks rate limits, applies input guardrails, picks the right provider, logs everything, checks the response output, sends it back. That's the whole flow.

Your application code doesn't change. You stop pointing at api.openai.com directly and point at your gateway instead. That's literally it from your team's perspective.

The control layer just sits there doing its job quietly.

"But I already have an API gateway. Isn't that enough?"

This is where most people get confused. Including me when I first looked into this.

Quick answer: no. Here's why.

Your API gateway (Kong, AWS API Gateway, Nginx, take your pick) understands traffic. It knows Team A sent 10,000 HTTP requests. It can enforce rate limits, handle auth tokens. That's useful.

Your AI gateway understands what's actually inside those requests. It knows Team A sent 4.2 million tokens to GPT-4o, it cost $84, average latency was 340ms, and 3 of those requests triggered the PII guardrail.

One sees requests. The other sees meaning. That's not a small difference.

For stateless REST APIs, a regular API gateway is totally fine. For LLM workloads where tokens equal money and every prompt is a potential compliance issue, you need something that actually speaks the language.

Do you actually need one right now though?

Let me skip the usual "it depends" and be direct.

You're probably fine without one if:

One team, one model, one use case
Nobody is asking about costs yet
Zero compliance requirements
It's a POC or side project

Don't add infrastructure you don't need. Raw SDK calls are fast to ship. Keep it simple when simple works.

You've outgrown the simple setup if:

Multiple teams are calling models independently with no visibility into what they're doing
Swapping providers requires actual code changes
Someone from legal or security or finance asked a question you couldn't answer
You've had an API key accidentally committed to a public repo (or almost did)
You can't answer "what did we spend on AI last month, by team?" without going on a scavenger hunt through billing dashboards

That last point is genuinely the biggest tell. If someone asks that question and you have to go digging, you already needed this.

What actually pushes teams over the edge

It's never one thing. It's always a pile of smaller things that suddenly feel heavy together.

DevOps realizes they can't track spend because keys are everywhere. Someone commits a key to a public repo. A team uses GPT-4 Turbo for tasks that GPT-4 Mini handles just fine, and you find out after they've burned $2K. Compliance asks for an audit trail and you have nothing.

Each of those individually, fine, you deal with it. All of them stacking up at the same time? That's when the "simple" setup reveals it was never actually simple. You were just deferring the complexity.

What a production gateway actually looks like

Okay enough talking around it. Here's what it gives you in practice, using TrueFoundry as the concrete example.

One API key across all providers

Your teams stop touching raw OpenAI or Anthropic keys entirely. One key, routed through the gateway, with access to every approved model. Rotate it in one place. Done.

Per-team budgets with real enforcement

Not "we log it and send you a Slack alert." Actual hard limits. Team hits their monthly budget, the next request gets rejected with a clear error. No surprise bills, no awkward retros about where the spend went.

Automatic failover

OpenAI goes down. It happens. Your app doesn't go down with it because requests automatically route to Anthropic or your self-hosted model. No code changes. No one gets paged. It just keeps working.

Full request tracing

Every prompt, every response, every token count, every cost attribution. Logged and queryable. Pull a request from six months ago and reconstruct exactly what happened. This feature alone has saved me more debugging time than I can measure.

Guardrails that actually run everywhere

PII filtering, prompt injection detection, custom output policies. You define the rule once and it applies across every team and every model. No per-team implementation, no "oops we forgot to add the check in this service."

Runs inside your own environment

VPC, on-prem, air-gapped. Data doesn't leave your infra. SOC 2, HIPAA, GDPR compliant. If your compliance team has ever asked "but where does the data actually go," this is finally a clean answer.

Performance-wise it handles 350+ RPS on a single vCPU with sub-3ms latency so you're not adding meaningful overhead to your request path.

TrueFoundry is in the 2026 Gartner Market Guide for AI Gateways and processes 10B+ requests per month for companies like Siemens Healthineers, NVIDIA, Resmed, and Automation Anywhere. Mentioning it not as a flex but as a sense of scale.

The question that actually helped me decide

Forget "do I need an AI gateway."

Ask this instead: when does the cost of NOT having one start to exceed the cost of setting one up?

For most teams that crossover happens way earlier than expected. For us it wasn't one event. It was the accumulation. The audit trail we didn't have. The $1,400 bill nobody could explain. The near-miss with a key in a public repo.

Setting up TrueFoundry honestly took less time than the post-mortem meeting for that billing incident.

Try TrueFoundry free at truefoundry.com (no credit card required, deploys on your cloud in under 10 minutes).

What does your current setup look like? Still on raw SDK calls or have you already hit the wall? Drop a comment, genuinely curious where people are when they start asking this question.

The Great Claude Code Leak of 2026: Accident, Incompetence, or the Best PR Stunt in AI History?

Varshith V Hegde — Wed, 01 Apr 2026 02:29:18 +0000

TL;DR: On March 31, 2026, Anthropic accidentally shipped the entire source code of Claude Code to the public npm registry via a single misconfigured debug file. 512,000 lines. 1,906 TypeScript files. 44 hidden feature flags. A Tamagotchi pet. And one very uncomfortable question: was it really an accident?

1. What Actually Happened

The Root Cause: One Missing Line in `.npmignore`

This is both the most embarrassing and most instructive part of the story. Let me walk through the technical chain of events.

When you publish a JavaScript/TypeScript package to npm, your build toolchain (Webpack, esbuild, Bun, etc.) optionally generates source map files, which have a .map extension. Their entire purpose is debugging: they bridge the gap between the minified, bundled production code and your original readable source. When a crash happens, a source map lets the stack trace point to your actual TypeScript file at line 47 rather than main.js:1:284729.

Source maps are strictly for internal debugging. They should never ship to users.

The way you exclude them from npm packages is with an .npmignore file, or a files field in package.json. Here's the mistake in plain English:

# What Claude Code's .npmignore should have had:
*.map
dist/*.map

# What it apparently had:
# (nothing about .map files)

That's it. That's the whole disaster.

But it gets worse. The source map didn't contain the source code directly. It referenced it, pointing to a URL of a .zip file hosted on Anthropic's own Cloudflare R2 storage bucket. A publicly accessible one, with no authentication required.

So the full chain looked like this:

npm install @anthropic-ai/claude-code
  → downloads package including main.js.map (59.8 MB)
    → .map file contains URL pointing to src.zip
      → src.zip is hosted publicly on Anthropic's R2 bucket
        → anyone can download and unzip 512,000 lines of TypeScript

Two separate configuration failures, stacked on top of each other.

As software engineer Gabriel Anhaia put it in his deep dive: "A single misconfigured .npmignore or files field in package.json can expose everything."

The Bun Factor

There's a third layer. Anthropic acquired the Bun JavaScript runtime at the end of 2025, and Claude Code is built on top of it. A known Bun bug (issue #28001, filed on March 11, 2026) reports that source maps are served in production builds even when the documentation says they shouldn't be.

The bug was open for 20 days before this happened. Nobody caught it. Anthropic's own acquired toolchain contributed to exposing Anthropic's own product.

2. The Timeline

00:21 UTC — March 31, 2026
Malicious axios versions (1.14.1 / 0.30.4) appear on npm
with an embedded Remote Access Trojan. Unrelated to Anthropic,
but catastrophically bad timing.

~04:00 UTC
Claude Code v2.1.88 is pushed to npm. The 59.8 MB source map
ships with it. The R2 bucket containing all source code is live
and publicly accessible.

04:23 UTC
Chaofan Shou (@Fried_rice), an intern at Solayer Labs,
tweets the discovery with a direct download link.
16 million people descend on the thread.

Next 2 hours
GitHub repositories spring up. The fastest repo in history
to hit 50,000 stars does it in under 2 hours.
41,500+ forks proliferate. DMCA requests begin.

~08:00 UTC
Anthropic pulls the npm package from the registry.
Issues the "human error, not a security breach" statement
to VentureBeat, The Register, CNBC, Fortune, Axios, Decrypt.

Same day
A Python clean-room rewrite appears, legally DMCA-proof.
Decentralized mirrors on Gitlawb go live with the message:
"Will never be taken down."
The code is permanently in the wild.

By the Numbers

Metric	Value
Lines of code exposed	512,000+
TypeScript files	1,906
Source map file size	59.8 MB
GitHub forks (peak)	41,500+
Stars on fastest repo	50,000 in 2 hours
Hidden feature flags	44
Claude Code ARR	$2.5 billion
Anthropic total ARR	$19 billion
Views on original tweet	16 million

3. SECURITY ALERT: The axios RAT

Stop. Read this before anything else if you updated Claude Code that morning.

Coinciding with the leak, but entirely unrelated to it, was a real supply chain attack on npm. Malicious versions of the widely-used axios HTTP library were published:

axios@1.14.1
axios@0.30.4

Both contain an embedded Remote Access Trojan (RAT). The malicious dependency is called plain-crypto-js.

If you ran npm install or updated Claude Code between 00:21 UTC and 03:29 UTC on March 31, 2026:

# Check your lockfiles immediately:
grep -r "1.14.1\|0.30.4\|plain-crypto-js" package-lock.json
grep -r "1.14.1\|0.30.4\|plain-crypto-js" yarn.lock
grep -r "1.14.1\|0.30.4\|plain-crypto-js" bun.lockb

If you find a match:

Treat the machine as fully compromised
Rotate all credentials, API keys, and secrets immediately
Perform a clean OS reinstallation
File incident reports for any organizational data

Going forward, Anthropic has designated the Native Installer as the recommended installation method:

curl -fsSL https://claude.ai/install.sh | bash

The native installer uses a standalone binary that doesn't rely on the npm dependency chain.

4. What Was Inside: The Full Breakdown

The leaked codebase is the src/ directory of Claude Code, the "agentic harness" that wraps the underlying Claude model and gives it the ability to use tools, manage files, run bash commands, and orchestrate multi-agent workflows. This is not the model weights (those weren't exposed), but in many ways this is more strategically valuable.

The Architecture

The Tool System (~40 tools, ~29,000 lines)

Claude Code isn't a chat wrapper. It's a plugin-style architecture where every capability is a discrete, permission-gated tool:

BashTool — shell command execution with safety guards
FileReadTool, FileWriteTool, FileEditTool
WebFetchTool — live web access
LSPTool — Language Server Protocol integration for IDE features
GlobTool, GrepTool — codebase search
NotebookReadTool, NotebookEditTool — Jupyter support
MultiEditTool — atomic multi-file edits
TodoReadTool, TodoWriteTool — task tracking

Each tool has its own permission model, validation logic, and output formatting. The base tool definition alone spans 29,000 lines.

The Query Engine (46,000 lines)

Labeled "the brain of the operation" in Gabriel Anhaia's analysis. It handles all LLM API calls and response streaming, token caching and context management, multi-agent orchestration, and retry logic.

The Memory Architecture

This is what competitors will study most carefully. Anthropic built a solution to "context entropy," the tendency for long-running AI sessions to degrade into hallucination as the context grows. Their answer is a three-layer memory system:

Layer 1: MEMORY.md
  → A lightweight index of pointers (~150 chars per entry)
  → Always loaded in context
  → Stores LOCATIONS, not data

Layer 2: Topic Files
  → Actual project knowledge, fetched on-demand
  → Never fully in context simultaneously

Layer 3: Raw Transcripts
  → Never re-read fully
  → Only grep'd for specific identifiers when needed

The key insight is what they call Strict Write Discipline. The agent can only update its memory index after a confirmed successful file write. This prevents the agent from polluting its context with failed attempts. The agent also treats its own memory as a "hint" and verifies facts against the actual codebase before acting, rather than trusting its stored beliefs.

5. Hidden Features Anthropic Never Meant to Ship

KAIROS: Always-On Autonomous Agent

KAIROS (from the Ancient Greek for "the right moment") is mentioned 150+ times in the source. It's an unreleased autonomous background daemon mode that runs background sessions while you're idle, executes a process called autoDream for nightly memory consolidation, merges disparate observations, removes logical contradictions, and converts vague insights into verified facts. It also has a special Brief output mode designed for a persistent assistant and access to tools regular Claude Code doesn't have.

Think of it as Claude Code actively maintaining its understanding of your project while you sleep, not just sitting there waiting.

ULTRAPLAN: 30-Minute Remote Planning Sessions

ULTRAPLAN offloads a complex planning task to a remote Cloud Container Runtime (CCR) session running Opus, gives it up to 30 minutes to think, and lets you approve the result from your phone or browser. When approved, a special sentinel value __ULTRAPLAN_TELEPORT_LOCAL__ brings the result back to your local terminal. Remote cloud-powered reasoning, delivered locally.

Coordinator Mode: Multi-Agent Orchestration

One Claude spawning and managing multiple worker Claude agents in parallel. The Coordinator handles task distribution, result aggregation, and conflicts between worker outputs. It's infrastructure for AI teams, not just AI assistants.

BUDDY: The Part Nobody Expected

The most talked-about find, not for its strategic implications but because it's genuinely fun.

buddy/companion.ts implements a full Tamagotchi-style AI pet that lives in a speech bubble next to your terminal input.

Species (18 total, hidden via String.fromCharCode() arrays):
duck, dragon, axolotl, capybara, mushroom, ghost, nebulynx...

Rarity tiers:
Common > Uncommon > Rare > Epic > Legendary
1% shiny chance, independent of rarity

Stats:
DEBUGGING / PATIENCE / CHAOS / WISDOM / SNARK

Determined by:
Mulberry32 PRNG seeded from your userId hash + salt 'friend-2026-401'
(Same user always gets the same buddy species -- deterministic)

Claude generates a custom name and personality description for your buddy on first hatch. There are sprite animations and a floating heart effect. The planned rollout window in the source code: April 1-7, 2026.

Someone at Anthropic is clearly having a very good time.

Anti-Distillation: Poisoning Competitor Training Data

In claude.ts (lines 301-313), a flag called ANTI_DISTILLATION_CC, when enabled, sends anti_distillation: ['fake_tools'] in API requests. This tells the server to inject decoy tool definitions into the system prompt. The idea: if a competitor is recording Claude Code's API traffic to train their own model, the fake tool definitions corrupt that training data.

There's a second mechanism in betas.ts (lines 279-298): server-side connector-text summarization. When enabled, the API buffers the assistant's reasoning between tool calls, returns only summaries, and cryptographically signs them. Competitors recording traffic get the summaries, not the full reasoning chain.

As Alex Kim notes in his analysis: "Anyone serious about distilling from Claude Code traffic would find the workarounds in about an hour of reading the source. The real protection is probably legal, not technical."

Frustration Detection via Regex

Found in userPromptKeywords.ts:

/\b(wtf|wth|ffs|omfg|shit(ty|tiest)?|dumbass|horrible|awful|
piss(ed|ing)? off|piece of (shit|crap|junk)|what the (fuck|hell)|
fucking? (broken|useless|terrible|awful|horrible)|fuck you|
screw (this|you)|so frustrating|this sucks|damn it)\b/

A multi-billion-dollar AI company is detecting user frustration with a regex. The Hacker News thread lost it. To be fair though, it's faster, cheaper, and more predictable than running an LLM inference every time to check if the user is angry at the tool.

250,000 Wasted API Calls Per Day

The most candid internal admission in the entire codebase. From autoCompact.ts (lines 68-70):

"BQ 2026-03-10: 1,279 sessions had 50+ consecutive failures 
(up to 3,272) in a single session, wasting ~250K API calls/day globally."

The fix was three lines: MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES = 3. After 3 consecutive compaction failures, it just stops trying. Sometimes good engineering is knowing when to give up.

6. The "Capybara" Model Confirmed

The leak didn't expose Claude's model weights, but it did expose multiple references to Anthropic's next major model family. Internal codenames: Capybara (also referred to as Mythos in a separate leaked document from the prior week).

The beta flags in the source reference specific API version strings for Capybara, suggesting it's well beyond concept stage. Security researcher Roy Paz from LayerX Security, who reviewed the code for Fortune, indicated it will likely ship in fast and slow variants with a significantly larger context window than anything currently on the market.

These references also confirmed the existence of undercover.ts, a module that actively instructs Claude Code to never mention internal codenames like "Capybara" or "Tengu" when used in external repositories. There's a hard-coded NO force-OFF — you can force Undercover Mode on, but you cannot force it off. In external builds, the function gets dead-code-eliminated entirely.

The implication raised in the Hacker News thread: AI-authored commits from Anthropic employees in open source repos will have no indication an AI wrote them. The tool actively conceals its own involvement.

7. Alternative Theory: Was This Anthropic's PR Play?

I'm not saying I believe this. I'm saying the circumstantial evidence is strange enough that it deserves to be stated clearly.

Anthropic is the self-proclaimed "safety-first AI lab." They're racing for developer mindshare against OpenAI (better brand) and Google (better distribution). Claude Code is their breakout product. They're preparing for an IPO. And they'd just made themselves unpopular with the developer community ten days earlier by sending legal threats to OpenCode for using their internal APIs.

So let's look at what this "leak" actually did for Anthropic.

Exhibit A: The April Fools' Timing

The leak occurred on March 31, the day before April 1st. The Buddy/companion system had a planned rollout window of April 1-7 coded directly into the source. The "leak" gave developers a sneak peek at what was about to launch anyway. Was this a controlled preview dressed up as an accident?

Exhibit B: The Bun Bug Nobody Fixed

Anthropic acquired Bun. They own the runtime. The bug causing source maps to ship in production was filed 20 days before the leak and was still open. If you own the runtime and its bug tracker, and that bug causes your own code to leak... why hadn't anyone internally marked it as critical?

Exhibit C: The Undercover Mode Irony

Claude Code has an entire subsystem called Undercover Mode, purpose-built to prevent internal codenames from leaking through AI-generated content. They built AI-powered leak prevention into the product. Then humans accidentally shipped the entire source code. The gap between their AI safety engineering and their human release engineering is either tragic or theatrical.

Exhibit D: The OpenCode Reputation Reversal

Ten days before the leak, Anthropic sent cease-and-desist letters to OpenCode, a popular third-party tool. The developer community was furious. The narrative was "Anthropic is acting like a gatekeeping megacorp."

Then a "leak" happens that shows Anthropic's impressive engineering to the world, makes them look like the underdog, generates three days of breathless coverage about KAIROS, BUDDY, and ULTRAPLAN, and completely reversed developer sentiment. Within 48 hours, developers went from "Anthropic sucks" to "holy shit look what Anthropic is building."

Exhibit E: The Permanent Mirror Problem

Anthropic filed DMCA takedowns. GitHub complied immediately. But the decentralized mirror at Gitlawb, with a public message saying "Will never be taken down," has been live since day one. Anthropic has a legal team, deep pockets, and relationships. A serious legal effort could make life difficult for every mirror operator. They chose not to go that hard.

Exhibit F: The "Second Leak in a Week" Pattern

This wasn't Anthropic's first incident that week. A draft blog post about the Capybara/Mythos model had "accidentally" been publicly accessible just days before, as Fortune reported on Thursday. Two high-profile "leaks" in five days, both generating enormous excitement about Anthropic's upcoming roadmap, both very conveniently timed.

The Counter-Arguments (Why It's Probably Just Incompetence)

To be fair:

Strategic roadmap exposure is genuinely damaging. Cursor, Copilot, and Windsurf now know exactly what Anthropic has already built and what's nearly ready to ship. That's real competitive intelligence permanently in the public domain.

The IPO narrative cuts both ways. "We shipped our source code to npm" is not a line you want in your S-1.

The axios RAT timing. Nobody would engineer a PR stunt to overlap with an active malware attack on npm. That part made a bad news day significantly worse for anyone who updated Claude Code that morning, and there's no upside to being associated with a supply chain attack.

The most likely answer is plain human error. A misconfigured .npmignore. A known Bun bug nobody had marked as critical. A public R2 bucket that should have been private. Three configuration failures that compounded into a disaster.

The PR outcome though? Undeniably good. The strategic damage? Real but survivable. The timing? Genuinely strange.

Draw your own conclusions.

8. Why DMCA Won't Fix This

DMCA takedowns work on centralized platforms. GitHub complied within hours. But the code spread to places that are harder to reach.

Gitlawb, with its explicit "Will never be taken down" message, operates outside the DMCA's practical reach. The Python port that appeared the same day was declared DMCA-proof by The Pragmatic Engineer's Gergely Orosz, who noted the rewrite is a new creative work that violates no copyright. There's also the AI copyright question: Anthropic's own CEO has implied that significant portions of Claude Code were written by Claude. The DC Circuit upheld in March 2025 that AI-generated work doesn't carry automatic copyright. If Anthropic's copyright claim over Claude-authored code is legally murky, the entire takedown strategy weakens.

And then there are torrents. Content once on the internet at scale doesn't come back.

The practical reality: 512,000 lines of Claude Code are permanently in the wild, regardless of what any court decides.

9. What This Means For You

If you're using Claude Code: Update immediately past v2.1.88 and use the native installer going forward (curl -fsSL https://claude.ai/install.sh | bash). If you updated via npm between 00:21 and 03:29 UTC on March 31, do the axios/RAT check above.

If you're building AI coding tools: The leaked source is now the most detailed public documentation of how to build a production-grade AI agent harness that exists. The three-layer memory architecture, the permission system, the tool plugin design, the multi-agent coordination patterns. It's all there, already analyzed by thousands of developers. The bar for what "production-grade" means just got documented in detail.

If you're at Anthropic: The code is out. KAIROS, ULTRAPLAN, and BUDDY are already built. Ship them. The community already knows they're coming. Turn the leak into a launch.

10. Lessons for Every Dev Team

This incident is a clear example of how release pipeline failures compound. Regardless of your opinion on Anthropic, every team should run through this checklist:

# 1. Audit your .npmignore / package.json "files" field
cat .npmignore
# Do you explicitly exclude *.map, dist/*.map, *.d.ts.map?

# 2. Check if source maps ship in your production build
ls dist/ | grep "\.map$"
# If you see anything: your bundler config needs review

# 3. Audit your cloud storage permissions
# Are any buckets referenced in your build artifacts publicly accessible?

# 4. Check your build toolchain for known bugs
# If you're on Bun, check issue #28001 status

# 5. Review your npm publish workflow
npm pack --dry-run
# Review EVERY file that would be published before actually publishing

The line that came out of the Hacker News thread: "Your .npmignore is load-bearing. Treat it like a security boundary."

Conclusion

Here's what we know for certain: a misconfigured .npmignore and a public cloud storage bucket exposed 512,000 lines of Claude Code, the code spread instantly and is now permanently in the wild, the leak revealed a technically impressive product with a compelling feature roadmap, and Anthropic's brand among developers bounced back remarkably fast.

What we'll probably never know: whether anyone inside Anthropic saw the Bun bug and made a judgment call, whether the April Fools' timing of the BUDDY rollout was coincidence, and whether Anthropic's relative restraint on DMCA enforcement is legal strategy or resource allocation.

What's not in question is that the engineering inside Claude Code is genuinely impressive. The memory architecture, the anti-distillation mechanisms, the multi-agent coordination, the DRM-at-the-HTTP-layer attestation. This is a serious piece of software doing things that are actually hard.

Accident or not, the world now knows what Anthropic is capable of building.

And maybe that was the point.

References

Source	Link
Alex Kim's technical deep-dive	alex000kim.com
VentureBeat — Full breakdown + axios RAT warning	venturebeat.com
The Register — Anthropic's official statement	theregister.com
Fortune — Strategic analysis + Capybara confirmation	fortune.com
Decrypt — DMCA analysis + permanent mirror situation	decrypt.co
CNBC — Revenue figures + company response	cnbc.com
Axios — Feature flag breakdown + roadmap analysis	axios.com
DEV.to (Gabriel Anhaia) — Architecture walkthrough	dev.to
Kuberwastaken/claude-code GitHub	github.com
Hacker News thread	news.ycombinator.com
Bun bug #28001	github.com/oven-sh/bun
CyberSecurityNews — Supply chain attack details	cybersecuritynews.com

If this was useful, drop a reaction. If you spot anything I got wrong, leave it in the comments.

Using Claude Code with Any LLM: Why a Gateway Changes Everything

Varshith V Hegde — Fri, 13 Mar 2026 03:30:00 +0000

I've been using Claude Code for a while now, and if you're a developer who has added it to your daily workflow, you probably know the feeling. It's genuinely good. It reads your codebase, runs commands, modifies files, and helps implement features right from your terminal without you having to context-switch constantly.

But at some point, most developers hit the same wall I did: what if I want to use a different model?

What if GPT-4o handles your specific codebase better? What if Gemini's larger context window is exactly what you need for that massive legacy project? What if you're spending more on API calls than you should be, and you know some of those simpler tasks could run on a cheaper model just fine?

Out of the box, Claude Code only talks to Anthropic. That's just how it works. And while Anthropic's models are genuinely strong, being locked into a single provider means you're trading flexibility for convenience. This guide is about getting both.

The Real Friction Points

Before jumping into the solution, it helps to be specific about what problems we're actually solving.

Model flexibility. Different models have different strengths. Claude Sonnet is excellent for most coding tasks, but you can't know it's the best tool for every job without being able to test alternatives. Without a gateway, you can't experiment without completely switching tools.

Cost management. Claude Code burns through tokens quickly during an active session. Complex architectural work and boilerplate generation are not the same job, and pricing them identically doesn't make much sense. Routing simpler requests to a more affordable model can cut costs significantly without affecting output quality where it matters.

Compliance and data routing. If you work in fintech, healthcare, or any regulated industry, you've likely dealt with requirements around where your data goes. Routing all API traffic through your own infrastructure before it reaches any external provider is often non-negotiable.

Observability. This one gets overlooked a lot. How many tokens does a typical Claude Code session consume? What's your actual cost per feature shipped? Without request logging, you're genuinely guessing.

Why Bifrost

Bifrost is an open-source LLM gateway built by Maxim AI to route, manage, and optimize requests between your application and multiple model providers. It's Apache 2.0 licensed, self-hostable, and supports 20+ providers including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, Mistral, Cohere, Groq, and more.

A few things that make it stand out technically:

Performance that doesn't get in the way. At 5,000 requests per second, Bifrost adds less than 15 microseconds of internal overhead per request. At production scale, that's essentially nothing.

Zero-config startup. A single npx command launches the gateway, and everything else is configurable through a web UI.

Built-in fallbacks and load balancing. If a provider fails or rate-limits you, Bifrost automatically routes to a backup. Traffic can also be distributed across multiple keys or providers using weighted rules.

Semantic caching. Repeated or semantically similar queries can be served from cache, which reduces both latency and cost for workflows with a lot of repetitive prompting.

Full observability out of the box. Prometheus metrics, request tracing, token usage, latency, and a built-in web dashboard are all included.

The architecture is straightforward:

Claude Code  -->  Bifrost (localhost:8080)  -->  Any LLM Provider

Claude Code uses an environment variable called ANTHROPIC_BASE_URL to know where to send API requests. Normally it points to https://api.anthropic.com. You point it at Bifrost instead. Bifrost accepts requests in Anthropic's Messages API format, translates them to whichever provider you've configured, and translates the response back. Claude Code never knows the difference.

No code changes. No patching. One environment variable.

What We'll Cover

Setting up and configuring Bifrost with multiple LLM providers
Integrating Claude Code with the gateway
Running Claude Code with any model
Configuring routing rules, fallbacks, and budgets
Integrating MCP tools
Using built-in observability and monitoring

Part 1: Setting Up Bifrost

Step 1: Install Bifrost

Create a project folder, open it in your editor, and run:

npx -y @maximhq/bifrost -app-dir ./my-bifrost-data

The -app-dir flag tells Bifrost where to store all its data. Bifrost will start listening on port 8080.

If you prefer Docker:

docker pull maximhq/bifrost
docker run -p 8080:8080 -v $(pwd)/data:/app/data maximhq/bifrost

The -v flag mounts a volume so your configuration persists across container restarts.

Step 2: Create Your Config File

Inside your ./my-bifrost-data folder, create a config.json file. This defines which providers Bifrost can route to, enables request logging, and sets up database persistence:

{
  "$schema": "https://www.getbifrost.ai/schema",
  "client": {
    "enable_logging": true,
    "disable_content_logging": false,
    "drop_excess_requests": false,
    "initial_pool_size": 300,
    "allow_direct_keys": false
  },
  "providers": {
    "openai": {
      "keys": [
        {
          "name": "openai-primary",
          "value": "env.OPENAI_API_KEY",
          "models": [],
          "weight": 1.0
        }
      ]
    },
    "anthropic": {
      "keys": [
        {
          "name": "anthropic-primary",
          "value": "env.ANTHROPIC_API_KEY",
          "models": [],
          "weight": 1.0
        }
      ]
    },
    "gemini": {
      "keys": [
        {
          "name": "gemini-primary",
          "value": "env.GEMINI_API_KEY",
          "models": [],
          "weight": 1.0
        }
      ]
    }
  },
  "config_store": {
    "enabled": true,
    "type": "sqlite",
    "config": {
      "path": "./config.db"
    }
  },
  "logs_store": {
    "enabled": true,
    "type": "sqlite",
    "config": {
      "path": "./logs.db"
    }
  }
}

The "value": "env.OPENAI_API_KEY" syntax tells Bifrost to read actual keys from environment variables rather than storing them in the file. Your secrets stay out of version control.

Step 3: Set Your API Keys

export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"
export GEMINI_API_KEY="your-gemini-api-key"

Step 4: Start the Gateway

Stop any previously running Bifrost instance, then start it again with the app directory flag:

npx -y @maximhq/bifrost -app-dir ./my-bifrost-data

Open http://localhost:8080 in your browser. You'll see the Bifrost dashboard where all configuration and monitoring lives.

Part 2: Connecting Claude Code to Bifrost

Step 1: Install Claude Code

npm install -g @anthropic-ai/claude-code

Step 2: Point Claude Code at Bifrost

Set these two environment variables in the same terminal session where you'll run Claude Code:

export ANTHROPIC_BASE_URL="http://localhost:8080/anthropic"
export ANTHROPIC_API_KEY="dummy-key"

The dummy-key part is a bit counterintuitive at first. Claude Code requires this variable to be set before it will run, but Bifrost handles actual authentication to providers using the keys you configured earlier. You can put any non-empty string here.

Step 3: Run Claude Code with Any Model

Start Claude Code and specify whichever model you want to use:

claude --model openai/gpt-4o

To route to other providers, use the provider prefix pattern:

openai/gpt-4o
openai/gpt-4o-mini
gemini/gemini-2.5-pro
groq/llama-3.1-70b-versatile
mistral/mistral-large-latest
anthropic/claude-sonnet-4-20250514
ollama/llama3

Run a quick sanity check by asking something simple like "Hello there" to confirm requests are flowing through correctly.

Part 3: Routing Rules, Fallbacks, and Budgets

Once Claude Code is connected, you can start using Bifrost's routing features to get more control over how requests are handled.

Weighted Routing Across Providers

Virtual Keys in Bifrost let you define routing logic that applies automatically. Navigate to Governance > Virtual Keys, create a key, and configure your routing weights:

{
  "name": "dev-routing",
  "budget": {
    "max_budget": 100,
    "budget_duration": "monthly"
  },
  "providers": [
    { "provider": "openai", "model": "gpt-4o", "weight": 0.7 },
    { "provider": "anthropic", "model": "claude-sonnet-4-20250514", "weight": 0.3 }
  ]
}

This routes 70% of requests to GPT-4o and 30% to Claude Sonnet, with a hard monthly cap of $100. Once the budget is exhausted, Bifrost stops routing automatically. For teams, this replaces a lot of manual cost monitoring.

Automatic Fallbacks

When a provider goes down or you hit a rate limit, Bifrost works down a fallback list until a request succeeds:

{
  "model": "openai/gpt-4o",
  "fallbacks": [
    { "provider": "anthropic", "model": "claude-sonnet-4-20250514" },
    { "provider": "gemini", "model": "gemini-2.5-pro" }
  ]
}

Your coding session continues without any manual intervention when a provider has issues.

Part 4: MCP Tool Integration

If you're using Model Context Protocol servers for filesystem access, web search, database queries, or custom integrations, Bifrost supports those too. Configure them once in Bifrost, and they become available to any model routing through it.

Step 1: Add MCP Configuration to Bifrost

Update your config.json to include MCP server definitions. Here's an example with filesystem access:

{
  "$schema": "https://www.getbifrost.ai/schema",
  "client": {
    "enable_logging": true,
    "disable_content_logging": true,
    "drop_excess_requests": false,
    "initial_pool_size": 300,
    "allow_direct_keys": false
  },

  "mcp": {
    "client_configs": [
      {
        "name": "filesystem",
        "connection_type": "stdio",
        "stdio_config": {
          "command": "npx",
          "args": ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
        },
        "tools_to_execute": ["*"],
        "tools_to_auto_execute": [
          "read_file",
          "list_directory",
          "create_file",
          "delete_file"
        ]
      }
    ],
    "tool_manager_config": {
      "max_agent_depth": 10,
      "tool_execution_timeout": 300000000000,
      "code_mode_binding_level": "server"
    }
  }
}

Restart Bifrost and navigate to the MCP catalog page in the web UI to confirm the filesystem server shows as connected.

Step 2: Add Bifrost as an MCP Server in Claude Code

claude mcp add --transport http bifrost http://localhost:8080/mcp

Step 3: Verify with a Real Task

Restart Claude Code and try a task that exercises the MCP tools. For example:

Create a simple calculator program in Python.

It should support addition, subtraction, multiplication, and division.
The user should input two numbers and an operation, and the program should print the result.

Then follow up with:

Analyze this repository and create a README.md explaining how the project works.
Include the project architecture and instructions for running it locally.

If the MCP integration is working, Claude Code will read your files, create new ones, and interact with your filesystem through Bifrost's tool injection.

Part 5: Observability and Monitoring

This is the part that surprised me most when I first set it up.

Every request that passes through Bifrost is logged with full detail: the input prompt, the response, which model handled it, latency, and cost. The web interface at http://localhost:8080/logs provides:

Real-time streaming of requests and responses
Token usage tracking per request
Latency measurements
Filtering by provider, model, or conversation content
Full request and response inspection

For individual developers, it's useful for understanding your actual usage patterns. For teams, it becomes a proper audit trail. You can see which models are being used most, where the expensive requests are coming from, and whether your routing rules are actually behaving as expected.

Bifrost also exposes Prometheus metrics for teams that want to integrate this data into existing monitoring pipelines.

Is This Worth Setting Up?

If you're a solo developer who uses Claude Code occasionally and doesn't have any compliance or cost concerns, the default setup is probably fine.

But if any of the following are true, a gateway is worth the time:

You want to test how different models perform on your specific workload
You're managing API costs across a team
Your organization has requirements around data routing or infrastructure control
You want actual visibility into your AI usage rather than end-of-month billing surprises
You use MCP tools and want them available across multiple model providers without reconfiguring each time

Bifrost being open source and self-hosted means your prompts and responses stay on your own infrastructure. For teams working on proprietary codebases, that's a meaningful difference from routing everything directly to a third-party API.

Get started:

ContractCompass: Your AI Contract Analyst That Actually Speaks Human

Varshith V Hegde — Sun, 08 Feb 2026 12:34:11 +0000

This is a submission for the Algolia Agent Studio Challenge: Consumer-Facing Conversational Experiences

What I Built

ContractCompass is an AI-powered contract analysis tool that turns legal jargon into plain English through natural conversation. Think of it as having a friendly lawyer friend who can review your contract over coffee, except this friend never gets tired, works 24/7, and doesn't charge $400/hour.

The Problem

Most people sign contracts they don't fully understand. Employment agreements, rental leases, SaaS terms—they're all written in dense legal language that assumes you went to law school. By the time you realize that "perpetual, irrevocable, worldwide license" means the company owns your weekend projects forever, you've already signed away your rights.

According to research, over 90% of people don't read terms and conditions before accepting them. It's not laziness. These documents are genuinely incomprehensible to the average person. A typical employment contract might be 15 pages of legal clauses that take hours to parse, assuming you even know what to look for.

The Solution

ContractCompass solves this through dialogue-based AI interaction. Instead of drowning you in legal analysis reports, it lets you have a natural conversation about your contract:

"What are the red flags here?"
"Can you explain this termination clause like I'm five?"
"Is this non-compete actually enforceable?"
"What should I negotiate before signing?"

The AI agent responds in real-time with contextual answers grounded in a curated database of contract clauses, powered by Algolia Agent Studio's semantic search capabilities.

Key Capabilities

Conversational AI Interface - Chat naturally with the agent. No forms, no checkboxes, just questions and answers.

Intelligent Risk Detection - Every clause gets analyzed and scored on a three-tier system (Low, Medium, High risk) with visual indicators.

Plain English Translations - Legal jargon becomes "here's what this actually means for you."

Industry Comparisons - The agent explains whether clauses are standard practice or unusual outliers worth negotiating.

Rich Visual Analysis - For deep dives, the agent generates structured analysis cards with prevalence bars, red flag lists, and detailed reasoning.

Demo

Live Demo: https://contractcompass.varshithvhegde.in/

No login required. No credit card. Just upload a contract (or try one of the built-in samples) and start asking questions.

How It Feels to Use

1. Upload is effortless

Drag and drop a PDF, paste text, or click one of the sample contracts. I've included four pre-loaded examples:

Friendly Startup Offer (Low Risk) - A well-balanced employment agreement with fair terms
Red Flag Employment Contract (High Risk) - Includes unilateral salary cuts, 24-month lock-in, overbroad IP assignment, 3-year non-compete, and $500K liquidated damages
Predatory Rental Agreement (High Risk) - Non-refundable deposits, tenant pays for ALL repairs, no-notice landlord entry, uncapped rent increases
Reasonable SaaS Agreement (Low Risk) - Standard business terms with mutual protections

2. The interface splits

Your contract appears on the left for reference, chat on the right. You can always scroll back to check what clause the AI is talking about.

3. Suggested prompts guide you

Six smart buttons help you get started:

Full Risk Analysis
Find red flags
Explain in plain English
What should I negotiate?
Compare to standards
Is this enforceable?

4. Streaming responses feel natural

The AI types back to you in real-time, token by token, like a real conversation. No waiting for a complete response to load. You see the analysis unfold naturally.

How I Used Algolia Agent Studio

Algolia Agent Studio is the intelligence engine that makes ContractCompass possible. Here's how it powers the entire conversational experience.

The Index: A Knowledge Base of Contract Clauses

I created an Algolia index called contract_clauses containing 50+ curated contract clauses across four contract types (employment, rental, SaaS, freelance). Each record includes:

clause_text - The full text of the clause
clause_type - Category (termination, compensation, non-compete, etc.)
contract_type - Employment, rental, SaaS, or freelance
industry - Tech, real estate, or general
prevalence_score - A 0-1 score indicating how common this clause is
risk_level - Low, medium, or high
plain_english - Simple explanation for non-lawyers
red_flags - List of concerning aspects
standard_version - What a fair version would look like
legal_implications - Real-world impact of accepting the clause

For example, a "predatory non-compete" clause record looks like this:

{
  "objectID": "emp-nc-003",
  "clause_text": "Employee agrees not to work for any competing business...",
  "clause_type": "non_compete",
  "contract_type": "employment",
  "industry": "tech",
  "prevalence_score": 0.2,
  "risk_level": "high",
  "plain_english": "You can't work for competitors for 3 years across all of North America",
  "red_flags": ["Unreasonably broad geographic scope", "Excessive duration"],
  "standard_version": "Typically 6-12 months within 50 miles of office",
  "legal_implications": "May prevent you from working in your field"
}

How Retrieval Powers the Conversation

When a user uploads a contract and starts asking questions, here's what happens behind the scenes:

1. Semantic Clause Matching

The Algolia agent retrieves semantically similar clauses from the index to provide context-aware responses. For example, if someone asks "Is this non-compete fair?", the agent:

Identifies the non-compete clause in the uploaded contract
Searches the index for similar non-compete clauses
Compares the uploaded clause against standard versions
Explains whether it's typical or unusually restrictive

2. Contract Type Detection

The agent automatically identifies the type of contract (employment, rental, SaaS, etc.) based on the language and clauses present, then adjusts its analysis accordingly. An employment contract gets compared against employment standards, not rental standards.

3. Prevalence-Based Risk Assessment

Using the prevalence scores from the indexed data, the agent can say things like:

"This termination clause is standard. About 95% of tech employment contracts include similar terms"
"This security deposit policy is unusual. Only 15% of rental agreements make deposits non-refundable"

4. Standard Version Recommendations

When a clause is problematic, the agent doesn't just say "this is bad." It shows what a fair version would look like:

"The current non-compete restricts you for 3 years across North America. A standard tech industry non-compete is typically 6-12 months within 50 miles of the office."

Making the Agent Conversational

The key to making ContractCompass feel natural was teaching the agent to think like a helpful friend, not a legal robot. I crafted prompts that guide it to:

Speak like a human - Use simple language. Avoid legal jargon unless explaining it. Be conversational but professional.

Be honest about risks - If a clause is predatory, say so clearly. Don't sugarcoat problematic terms.

Ground everything in data - Always search the contract_clauses index for similar examples. Compare the user's clause against standard versions and explain how it differs from typical industry practice.

Provide actionable advice - Don't just identify problems. Suggest what to negotiate and how to approach it.

This approach ensures every response is both friendly and useful, backed by real contract data rather than generic advice.

Why Fast Retrieval Matters

Algolia's speed and semantic search capabilities are critical to making ContractCompass feel like a real conversation rather than a clunky Q&A bot.

Speed Creates Natural Dialogue

When someone asks "What are the red flags in this contract?", they expect an answer within seconds, not minutes. Algolia's sub-50ms search latency means the agent can:

Retrieve relevant clause examples instantly
Stream responses token-by-token without lag
Handle follow-up questions in the same conversation thread
Maintain context across multiple queries

If retrieval took 5-10 seconds per query, users would lose patience. The conversation would feel broken. Fast retrieval makes the experience feel fluid and natural.

Contextual Retrieval Enables Nuanced Analysis

Algolia's semantic search doesn't just match keywords. It understands meaning. This is crucial for contract analysis because:

Legal language varies widely

A non-compete clause might say "Employee shall not engage in competitive activities" or "You agree not to work for rival companies." These are semantically similar but textually different. Algolia's vector-based search matches them both.

Users ask in natural language

Someone might ask "Can they really fire me for any reason?" which should match clauses about "at-will employment" or "termination without cause." Semantic search bridges this gap.

Context matters

A liability cap of $100K might be reasonable in a $10K/year SaaS contract but predatory in a $500K enterprise agreement. By retrieving similar contracts in the same industry and price range, the agent provides context-aware analysis.

Retrieval Grounds Responses in Real Data

One of the biggest risks with AI agents is hallucination. Making up plausible-sounding but incorrect information. By grounding every response in retrieved data from the curated index, ContractCompass avoids this problem.

When the agent says "This non-compete is unusually restrictive," it's not guessing. It's comparing the uploaded clause against the prevalence scores and standard versions in the index. When it explains what a fair clause looks like, it's showing you actual examples from the database.

This retrieval-augmented generation (RAG) approach makes the agent both reliable and trustworthy.

The Impact on User Experience

From a user perspective, fast contextual retrieval translates to:

Confidence in the analysis - "This isn't just an AI's opinion, it's based on real contract data"

Immediate answers - "I can get my questions answered in real-time without waiting"

Conversational flow - "It feels like talking to a human expert who knows contract law"

Actionable insights - "I now know exactly what to negotiate before signing"

Without Algolia's speed and semantic capabilities, ContractCompass would be a generic chatbot that gives vague, unhelpful advice. With them, it's a genuinely useful tool that empowers people to understand and negotiate their contracts.

Technical Architecture

Frontend (React + TypeScript)

The interface is built with React 18 and TypeScript for type safety. I chose a modern stack that prioritizes performance and developer experience:

UI Library: Tailwind CSS + shadcn/ui components for a clean, professional look
State Management: React hooks for local state (no complex state library needed)
Markdown Rendering: react-markdown for rich text in chat responses

AI Agent (Algolia Agent Studio)

The chat interface calls Algolia Agent Studio directly from the frontend. This direct integration means:

Real-time streaming responses that appear token-by-token
No backend proxy needed for chat, which reduces latency
Full conversation history sent with each request for contextual follow-ups

Search Index (Algolia)

The contract_clauses index contains 300+ curated clauses. Each clause is enriched with metadata (prevalence scores, risk levels, plain English explanations) that the agent uses to provide contextual analysis.

PDF Processing

When users upload PDFs, the text extraction happens server-side using Google Gemini 2.5 Flash. The flow is:

User uploads PDF via drag-and-drop
PDF converts to base64 on the client
Base64 data sent to serverless function
Function calls Gemini API for text extraction
Extracted text returns to the frontend
Text loads into chat interface for analysis

Backend (Serverless Functions)

Four serverless functions handle specific tasks:

extract-pdf - PDF text extraction using Gemini
analyze-contract - Clause parsing and analysis
search-clauses - Direct Algolia index queries
seed-algolia - Index population with curated data

Design Decisions

Split-Screen Layout

I chose a split-screen design (contract on left, chat on right) because users need to reference the original text while discussing it. It feels more collaborative, like reviewing a document with someone. Mobile users get a stacked layout that still works well.

Color-Coded Risk Levels

Risk levels use universal color psychology:

Green - Safe, standard terms
Amber - Caution, worth discussing
Red - Danger, likely problematic

These colors are consistent across risk badges, prevalence bars, and analysis cards. You can glance at a clause and immediately understand its risk level.

Suggested Prompts

Not everyone knows what questions to ask about a contract. The six suggested prompts serve as:

Onboarding - Showing users what's possible
Efficiency - Common questions answered with one click
Discovery - Revealing features users might not know about

Streaming Responses

Token-by-token streaming makes the AI feel more human and less like a loading bar. It also provides immediate feedback that the system is working. Users don't stare at a blank screen wondering if anything is happening.

Challenges and Learnings

Challenge 1: Balancing Legal Accuracy with Accessibility

Legal language exists for precision. Simplifying it risks losing important nuances. I solved this by:

Providing both the original clause text and plain English side-by-side
Including detailed "legal implications" sections for those who want depth
Being honest about limitations (the disclaimer reminds users this isn't legal advice)

Challenge 2: Handling Diverse Contract Formats

Contracts vary wildly in structure. Some are 2 pages, others are 50. Some use headers, others are wall-to-wall text. The PDF extraction with Gemini handles this by:

Preserving structure where possible
Extracting text even from scanned/image PDFs
Cleaning up formatting artifacts

Challenge 3: Preventing AI Hallucination

Early versions sometimes invented red flags that didn't exist. The solution was retrieval-augmented generation. Every analysis is now grounded in retrieved clause data from the index. The agent can only reference what it finds in the search results.

Challenge 4: Making Risk Scores Meaningful

A simple "high risk" label isn't actionable. I added:

Prevalence scores - "Only 20% of contracts include this"
Standard versions - "Here's what fair looks like"
Specific red flags - "This clause is concerning because..."

These additions turn a vague warning into specific, actionable information.

Export Analysis as PDF

Let users download a full risk report they can share with lawyers or keep for their records. Make it official and presentable.

What I Learned

Building ContractCompass taught me that the best AI tools don't feel like AI tools. They feel like helpful conversations with knowledgeable friends. The key is combining:

Fast, semantic search that finds the right information instantly
Thoughtful prompting that guides the AI to be helpful, not robotic
Real data that grounds responses in facts, not hallucinations
Clear design that makes complex information accessible

Algolia Agent Studio made the first part possible. The rest was about understanding what people actually need when facing a contract: clarity, confidence, and actionable advice.

Conclusion

ContractCompass demonstrates how conversational AI powered by fast, semantic search can democratize access to legal understanding. By combining Algolia Agent Studio's retrieval capabilities with a thoughtfully designed user experience, it transforms contract analysis from an intimidating expert task into an accessible conversation.

The key insight: people don't need to become lawyers to understand their contracts. They just need the right questions answered in language they can understand, backed by real data about what's standard and what's not.

Try it yourself: https://contractcompass.varshithvhegde.in/

Built With

Powered by:

Algolia Agent Studio - Conversational AI with semantic search
React + TypeScript - Frontend framework
Tailwind CSS + shadcn/ui - UI components
Google Gemini - PDF text extraction

GitHub:

Varshithvhegde / contract-compass

ContractCompass 🧭

AI-Powered Contract Analysis for Non-Lawyers

Chat with AI to understand your contract. Identify risks, get plain-English explanations, and learn what to negotiate — powered by Algolia Agent Studio.

📋 Table of Contents

Overview

ContractCompass is an intelligent contract analysis tool designed to help everyday people — not lawyers — understand legal documents before they sign. Users upload or paste a contract, then have a real-time conversation with an AI agent that identifies risky clauses, explains legal jargon in plain English, and compares terms against industry standards.

The AI agent is powered by Algolia Agent Studio, which provides semantic search and retrieval of similar contract clauses…

View on GitHub

ContractCompass is not a substitute for professional legal advice. Always consult a qualified attorney for legal matters.

Why Your AI Gateway Needs MCP Integration in 2026

Varshith V Hegde — Mon, 02 Feb 2026 10:19:30 +0000

You know that feeling when you've spent three hours debugging why your AI agent can't access your database for the third time this week?

I was there last month. Five different tool integrations, each with its own authentication flow, error handling, and connection management. Want to add Slack notifications? Write another integration. Need file system access? Another one. Every integration was basically the same boilerplate with different endpoints.

Then I found the Model Context Protocol and Bifrost. It sounded too good to be true one gateway, one protocol, unlimited tools. But it actually works, and it's probably the most practical shift in AI infrastructure you'll deal with this year.

What's an AI Gateway and Why Should You Care?

Think of an AI gateway as the central hub between your apps and multiple AI providers. Instead of writing separate code for OpenAI, Anthropic, Google, and others, you connect once to the gateway, and it handles the rest.

The benefits are immediate:

Automatic failover: If one AI provider goes down, requests switch to another
Load balancing: Distribute requests across multiple API keys to avoid rate limits
Caching: Reduce costs and improve response times
Unified monitoring: One place to track all your AI interactions

Bifrost is an AI gateway built in Go that adds only 11 microseconds of latency while handling 5,000 requests per second. When you're running production AI systems, those microseconds matter.

The Model Context Protocol: USB-C for AI

Anthropic introduced MCP in November 2024. Within a year, it became the industry standard. OpenAI adopted it in March 2025. Google DeepMind followed. By December 2025, it was donated to the Linux Foundation with backing from major tech companies.

Here's why it matters: Before MCP, connecting an AI model to a new tool meant writing custom integration code. Every. Single. Time.

AI needs to search files? Custom code.
Access a database? More custom code.
Connect to Slack? Yet another integration.

This created what Anthropic called the "N×M problem" N models needing M different integrations, resulting in exponentially growing complexity.

MCP solved this with a standardized protocol. Write an MCP server once for a tool, and any MCP-compatible AI client can use it. It's like USB-C for AI systems one standard connection instead of different cables for different devices.

The Problem with Direct MCP Connections

When you connect AI models directly to MCP servers, you run into scaling problems. Every request from the AI includes all available tool definitions in its context window. Connect to five MCP servers with 100 total tools, and every single request carries those 100 tool definitions even for simple queries that don't need tools.

This creates three issues:

1. Wasted tokens: Most of your context budget goes to tool catalogs instead of actual work. A six-turn conversation with 100 tools burns 600+ tokens just on definitions.

2. Security gaps: Tools can execute without validation or approval. No audit trail, no safety checks before destructive operations.

3. Coordination overhead: Each tool call requires a separate round trip to the AI model.

How Bifrost Solves This

Bifrost integrates MCP natively into the gateway itself. You get both AI provider management and tool orchestration through a single interface.

It supports four connection types:

In-process tools: Run directly in Bifrost's memory with zero network overhead
Local MCP servers via STDIO: For filesystem operations or database queries
HTTP connections: For remote microservices
Server-Sent Events: For real-time data streams

The killer feature is Code Mode. Instead of including hundreds of tool definitions in every request, Bifrost exposes just four meta-tools:

listToolFiles() - Discover available servers
readToolFile(fileName) - Get tool signatures
getToolDocs(server, tool) - Get detailed documentation
executeToolCode(code) - Run Starlark (Python-like) code

The AI writes Starlark code that orchestrates tools inside a sandboxed environment, and tool definitions load only when needed. This reduces token usage by 50%+ when using multiple MCP servers (3+). With 8-10 MCP servers (150+ tools), you avoid wasting context on massive tool catalogs.

Getting Started: A Real Example

Let me show you how this works in practice. I'll walk through building a simple MCP server and connecting it to Bifrost.

Step 1: Start Bifrost

npx -y @maximhq/bifrost

That's it. Bifrost starts with zero configuration and opens at localhost:8080.

Step 2: Build a Simple MCP Server

I created a Flask server with three tools: getting programming jokes, inspirational quotes, and basic calculations. Here's the core:

from flask import Flask, jsonify, request
from flask_cors import CORS
import json
import random

app = Flask(__name__)
CORS(app)

jokes = [
    "Why do programmers prefer dark mode? Because light attracts bugs!",
    "Why did the developer go broke? Because he used up all his cache!"
]

@app.route('/sse', methods=['POST'])
def handle_message():
    data = request.json
    method = data.get('method')

    if method == 'initialize':
        return jsonify({
            "jsonrpc": "2.0",
            "id": data.get('id'),
            "result": {
                "protocolVersion": "2024-11-05",
                "capabilities": {"tools": {}},
                "serverInfo": {"name": "example-server", "version": "1.0.0"}
            }
        })

    elif method == 'tools/list':
        return jsonify({
            "jsonrpc": "2.0",
            "id": data.get('id'),
            "result": {
                "tools": [
                    {
                        "name": "get_joke",
                        "description": "Returns a random programming joke",
                        "inputSchema": {"type": "object", "properties": {}}
                    },
                    {
                        "name": "calculate",
                        "description": "Performs basic arithmetic",
                        "inputSchema": {
                            "type": "object",
                            "properties": {
                                "operation": {"type": "string", "enum": ["add", "multiply"]},
                                "a": {"type": "number"},
                                "b": {"type": "number"}
                            }
                        }
                    }
                ]
            }
        })

    elif method == 'tools/call':
        tool_name = data['params']['name']
        args = data['params'].get('arguments', {})

        if tool_name == 'get_joke':
            result = {"content": [{"type": "text", "text": random.choice(jokes)}]}
        elif tool_name == 'calculate':
            a, b = args['a'], args['b']
            if args['operation'] == 'add':
                answer = a + b
            else:
                answer = a * b
            result = {"content": [{"type": "text", "text": f"Result: {answer}"}]}

        return jsonify({"jsonrpc": "2.0", "id": data.get('id'), "result": result})

if __name__ == '__main__':
    app.run(port=5000)

Run it with: python mcp_server.py

Step 3: Configure Model Providers and Connect to Bifrost

Setting Up Model Providers

In the Bifrost UI at localhost:8080, navigate to Model Providers in the left sidebar. You'll see a comprehensive list of supported providers including OpenAI, Anthropic, Google, AWS Bedrock, Azure, and many others.

Click on OpenAI from the list, then click "+ Add new key" in the top-right corner.

Fill in the key configuration:

Name: Give it a descriptive name like "Production Key"
API Key: Enter your actual API key (e.g., sk-proj-...) or use an environment variable like env.OPENAI_KEY
Models: Click to select which models this key can access (e.g., gpt-4o, gpt-4o-mini)
Weight: Set to 1 for load balancing (higher weights receive proportionally more traffic)
Use for Batch APIs: Toggle this on if you want to use this key for batch operations

Click Save to add the key. You'll see it appear in your configured keys list with its weight and enabled status.

Pro tip: For production setups, add multiple API keys for the same provider. Bifrost automatically distributes requests across them to avoid rate limits. You can also add keys from different providers (e.g., OpenAI and Google) for automatic failover.

Connecting Your MCP Server

Now go to MCP Gateway in the left sidebar and click "New MCP Server":

Configuration:

Name: localmcp
Connection Type: HTTP (Streamable)
Connection URL: http://localhost:5000/sse
Ping Available for Health Check: Enable this

Bifrost immediately connects, discovers your tools, and shows them in "Available Tools."

Step 4: Use It

Here's a Python client that uses everything together:

import requests
import json

BIFROST_URL = "http://localhost:8080"

def ask_ai(message, history=None):
    if history is None:
        history = []

    history.append({"role": "user", "content": message})
    print(f"\n👤 You: {message}")

    # Send to AI via Bifrost
    response = requests.post(
        f"{BIFROST_URL}/v1/chat/completions",
        json={"model": "openai/gpt-4o", "messages": history}
    ).json()

    assistant_msg = response["choices"][0]["message"]

    # Handle tool calls
    if "tool_calls" in assistant_msg:
        print(f"🔧 AI is using {len(assistant_msg['tool_calls'])} tools...")
        history.append(assistant_msg)

        for tool_call in assistant_msg["tool_calls"]:
            # Bifrost executes the tool on your MCP server
            result = requests.post(
                f"{BIFROST_URL}/v1/mcp/tool/execute",
                json={"tool_call": tool_call}
            ).json()

            history.append({
                "role": "tool",
                "tool_call_id": tool_call["id"],
                "name": tool_call["function"]["name"],
                "content": json.dumps(result)
            })

        # Get final response
        response = requests.post(
            f"{BIFROST_URL}/v1/chat/completions",
            json={"model": "openai/gpt-4o", "messages": history}
        ).json()

        assistant_msg = response["choices"][0]["message"]

    history.append(assistant_msg)
    print(f"🤖 AI: {assistant_msg['content']}\n")
    return assistant_msg["content"], history

# Try it
ask_ai("Tell me a programming joke")
ask_ai("What is 25 times 4?")

What Just Happened?

Your script sends "What is 25 times 4?" to Bifrost
Bifrost adds your MCP tools to the AI's context
GPT-4 decides to use the calculate tool
Your script calls Bifrost's tool execution endpoint
Bifrost sends a JSON-RPC request to your Flask server
Your server calculates 25 × 4 = 100 and returns it
The result goes back to GPT-4
GPT-4 responds: "25 times 4 equals 100"

The beautiful part? Clean separation of concerns:

Your client doesn't know MCP protocol details
Bifrost handles all MCP communication
The AI doesn't know your server implementation
Your MCP server doesn't know which AI is calling it

This is the power of standardization.

Security Matters

In April 2025, researchers identified MCP security issues: prompt injection, permission combinations that could exfiltrate data, and lookalike tools.

Bifrost addresses this with a "suggest, don't execute" model by default. When an AI proposes a tool call, nothing runs automatically. Your code reviews and approves each execution. You get full audit trails for compliance.

You can configure Agent Mode for specific tools. Safe operations like reading files can auto-execute, while destructive operations require approval.

For scenarios with many MCP servers (3+), you can enable Code Mode to reduce token usage.

This configuration tells Bifrost to expose the four meta-tools instead of all tool definitions directly.

Why This Matters Now

If you're building AI systems without MCP integration in 2026, you're solving yesterday's problems. The standardization is here. The ecosystem is mature. The question isn't whether to adopt MCP, but how quickly.

Bifrost makes adoption straightforward:

Setup takes less than a minute
Web UI makes configuration visual
Open-source means you can examine and customize
Native support for multiple connection types

This is infrastructure that matters. Not because it's flashy, but because it solves real problems every organization faces when building AI systems.

Resources

Get started with Bifrost:

GitHub: https://git.new/bifrost
Documentation: https://docs.getbifrost.ai
Quick Start: https://docs.getbifrost.ai/quickstart/gateway/setting-up
Code Mode: https://docs.getbifrost.ai/mcp/code-mode
Agent Mode: https://docs.getbifrost.ai/mcp/agent-mode
MCP Overview: https://docs.getbifrost.ai/mcp/overview

Top 5 LLM Gateways in 2026: A Deep-Dive Comparison for Production Teams

Varshith V Hegde — Thu, 22 Jan 2026 01:41:56 +0000

I spent the last few weeks researching LLM gateway solutions for production teams. Here's what I found after testing five different options, talking to engineering teams running them at scale, and breaking things in my staging environment.

I didn't test every edge case. We focused on REST APIs with streaming responses, didn't test batch processing extensively, and our traffic patterns might be different from yours. But here's what I learned.

Why Production Teams Need LLM Gateways

Here's what happened when we didn't use one:

Our application relied only on OpenAI. When they had an outage last month, our entire product went down. This created problems when we had customers waiting for support.

Then there's cost. We were using GPT-4 for simple tasks that Claude Haiku could handle for one-tenth the price. One weekend of refactoring our routing logic saved us $3,000 per month.

But managing multiple providers yourself creates its own problems. You end up writing custom code for each API, normalizing their different error formats, managing API keys, building retry logic from scratch, and spending hours debugging why Anthropic's rate limit response looks different from OpenAI's.

LLM gateways solve this. One API for all providers. Automatic fallbacks. Cost tracking that works. And your application won't crash because one provider is having issues.

Here are the five gateways that impressed me.

1. Bifrost (by Maxim AI)

What it is: A high-performance LLM gateway built in Go. It's designed for speed and reliability.

Best for: Customer-facing applications where latency matters. Real-time chat, high-traffic APIs, anything where users will notice if responses are slow.

The performance numbers caught my attention first. In our synthetic load tests, Bifrost added about 11 microseconds of latency at 5,000 requests per second. When I ran the same test with LiteLLM (which is Python-based), it added around 50 microseconds.

What really sold me was the P99 latency test. At 1,000 concurrent users, LiteLLM's slowest responses hit 28 seconds. Bifrost stayed under 50 milliseconds. If you're building a chatbot, that's the difference between users staying on your application and immediately leaving.

Now, I didn't test this with burst traffic or serverless deployments - our setup is traditional Kubernetes. Your results might differ depending on your infrastructure.

What makes it different:

Smart load balancing that actually works. Bifrost was the first gateway I found that automatically routes requests based on real-time performance. It monitors which providers are healthy, routes around failures, and prevents you from hitting rate limits. Most gateways claim to do this, but Bifrost's implementation is noticeably better.

It also has cluster mode built in, so you can run multiple instances without complicated setup. And here's what surprised me - it includes SSO, audit logs, team budgets, and role-based access control without adding latency. Most gateways make you choose between features and speed. Bifrost somehow does both.

Setup:

npx -y @maximhq/bifrost

In 30 seconds you have a gateway running with a web UI. Since it uses OpenAI's API format, integrating it is just changing your base URL. I had our staging environment switched over in under 10 minutes.

Bifrost covers all the major providers - OpenAI, Anthropic, Google Vertex AI, AWS Bedrock, Azure OpenAI, Cohere, Mistral, Groq, Together AI, and Replicate. Plus they added support for any OpenAI-compatible endpoint, which means you can actually use custom or self-hosted models too.

For most production use cases, you're using one of these major providers anyway. LiteLLM does have broader coverage and a more mature open-source community - they've been around longer with more contributors and community support. If that ecosystem and maximum provider choice matters more to you than raw performance, LiteLLM is a solid pick. But for our needs, Bifrost's speed and provider coverage were enough.

Why we chose it: For our use case (high-scale, customer-facing chat), the 11 microsecond overhead was too good to pass up. The enterprise features were a bonus we didn't expect at this performance level.

Pricing: Open-source and free to self-host. Enterprise support is available.

2. LiteLLM

This is probably the most popular open-source LLM gateway. Python-based, with both an SDK and proxy server.

If you're in a Python environment or need access to niche models, this is the default choice. The provider coverage is unmatched - over 100 providers including all the major ones (OpenAI, Anthropic, Google, Azure, AWS) plus specialized options like HuggingFace, Ollama, Replicate, Anyscale, and Perplexity.

For Python developers, setup is straightforward:

from litellm import completion

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}],
    api_key="your-key"
)

# Switch to Claude without changing code
response = completion(
    model="claude-4-sonnet",
    messages=[{"role": "user", "content": "Hello"}]
)

Configuration uses YAML. The documentation is thorough, and there's a strong community.

Where it breaks down: Performance at scale. LiteLLM is written in Python using FastAPI. At low to moderate traffic, it performs well. But in our load tests, the limitations showed clearly.

At 500 requests per second, P99 latency hit 28 seconds. At 1,000 requests per second, it crashed - ran out of memory and started failing requests. The Python GIL and async overhead become real bottlenecks when handling thousands of concurrent requests.

I saw this in our staging environment. At 200 requests per second, everything ran smoothly. When I simulated higher traffic (around 2,000 requests per second), LiteLLM started timing out. Memory usage increased to over 8GB, and we got cascading failures.

When to use it:

Development and testing environments
Prototyping and trying different models
Internal tools with moderate traffic (under 500 RPS)
When you need access to 100+ providers
Python-first teams where ecosystem fit matters

When to avoid it:

Customer-facing applications at scale
Real-time features where every millisecond counts
Production workloads requiring 99.9%+ uptime

The ecosystem is mature with active development, but if you're planning to handle thousands of requests per second in production, you'll likely hit performance issues.

Pricing: Fully open-source and free. You pay for hosting it yourself.

3. Portkey

Portkey is more than just a gateway - it's a full AI control plane with routing, observability, guardrails, and governance.

The observability depth is what sets it apart. Every request gets full traces showing you which user made the call, which models were tried, why they failed, which fallback was used, how long each step took, and the exact cost. This isn't just logging - it's distributed tracing for AI.

When our staging environment started using too many tokens, Portkey's traces showed us exactly which user and which prompt was causing it. That level of detail is valuable when debugging production issues.

from portkey_ai import Portkey

portkey = Portkey(
    api_key="your-portkey-key",
    virtual_key="your-provider-virtual-key"
)

response = portkey.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}],
    model="gpt-4"
)

Enterprise features:

PII detection, content filtering, prompt injection detection
SOC 2, HIPAA, GDPR compliance with full audit trails
SSO/SAML, team permissions, role-based access
Data residency controls

According to their team, they handle over 10 billion requests monthly with 99.9999% uptime. I couldn't independently verify this, but the platform felt stable during our testing.

The tradeoff: I measured latency overhead of 20-40 milliseconds when using advanced features like guardrails and detailed tracing. For a small team that just needs basic routing, Portkey is probably more than necessary. The learning curve is also steeper than simpler gateways.

Why we didn't choose it: For our use case, the added latency and complexity weren't worth the governance features we didn't need yet. But I talked to a healthcare company using Portkey specifically for PII detection. Every LLM request gets scanned for protected health information, logged with full audit trails, and only routed to HIPAA-compliant providers. For them, the compliance features justified the cost.

If you're in a regulated industry or managing AI across multiple teams with governance requirements, Portkey's observability is among the best available.

Pricing: Free tier for development | Starts at $49/month | Enterprise custom pricing

4. Kong AI Gateway

Kong's API Gateway with AI-specific features added. If you're already using Kong, this is worth looking at.

Kong brings decades of API gateway experience to LLM routing - authentication, rate limiting, security, and observability at large scale. All the infrastructure pieces that matter when running production workloads.

# Install AI Proxy plugin
curl -X POST http://localhost:8001/services/ai-service/plugins \
  --data "name=ai-proxy" \
  --data "config.route_type=llm/v1/chat" \
  --data "config.auth.header_name=Authorization" \
  --data "config.model.provider=openai" \
  --data "config.model.name=gpt-4"

AI-specific capabilities:

Unified API across OpenAI, Anthropic, AWS Bedrock, Azure AI, Google Vertex
RAG pipelines built in
PII removal across 12 languages
Content filtering and safety controls

Where this makes sense: You're already using Kong for API management. That's the primary reason to choose this. The integration with existing Kong infrastructure is seamless, and you get unified observability across all your APIs.

Where it doesn't: If you're not already on Kong, the learning curve is significant. It's built for large enterprises, not small teams needing quick deployment. We evaluated this briefly but decided it was more complexity than we needed.

Pricing: Available through Kong Konnect (managed) or self-hosted | Enterprise custom pricing

5. Helicone AI Gateway

Started as an observability platform, recently launched a Rust-based gateway. Lightweight and fast.

Built in Rust, Helicone achieves around 8ms P50 latency with sub-5ms overhead even under load, based on what their team shared with me. The gateway ships as a single 15MB binary that runs anywhere.

# Run with npx
npx @helicone/ai-gateway

# Or with Docker
docker run -p 8787:8787 helicone/ai-gateway

The observability is their core strength - request-level tracing, user tracking, cost forecasting, performance analytics, and real-time alerts. It's as comprehensive as Portkey's but with less complexity.

Flexible deployment:

Cloud-hosted (managed service)
Self-hosted (full control)
Hybrid (self-host gateway, use cloud observability)

The consideration: The gateway is newer (launched mid-2024). Core routing is solid, but some advanced enterprise features are still developing. For most teams this isn't a problem, but large enterprises might want to validate specific requirements first.

Pricing:

Gateway: Open-source and free to self-host
Observability: Starts free, then $20/month for 100,000 requests

The separation is smart - you can self-host for free and only pay for observability if you want it.

How to Choose

After evaluating these gateways, here's what I learned:

Choose Bifrost if: Performance is critical. You're handling 5,000+ requests per second, serving customer-facing features, or building real-time applications where latency matters. The 11 microsecond overhead is hard to beat.

Choose LiteLLM if: You're in a Python environment with moderate traffic (under 500 RPS). The provider coverage is unmatched - over 100 models including specialized ones. Great for development, prototyping, and internal tools.

Choose Portkey if: You're in a regulated industry needing compliance controls (HIPAA, SOC 2) or managing AI across multiple teams. The observability and governance features are excellent, but you'll pay for it in latency (20-40ms overhead).

Choose Kong if: You're already using Kong for API management. Otherwise, the learning curve probably isn't worth it unless you're a large enterprise needing infrastructure-level control.

Choose Helicone if: You want performance and observability without enterprise complexity. Good for teams with data residency requirements who want self-hosted infrastructure with cloud monitoring.

Questions?

Have you deployed LLM gateways in production? What did you choose and why? What surprised you?

Still evaluating options? I can help with specific questions about performance, integration, or cost modeling at your scale. Leave a comment below.

Daily Echo - Your Life in Motion 🎥

Varshith V Hegde — Sat, 03 Jan 2026 16:23:33 +0000

This is a submission for the DEV's Worldwide Show and Tell Challenge Presented by Mux

What I Built

Daily Echo is a private video journaling app where you record 1-minute daily video diaries. It's like having a conversation with your future self. The app helps you track your mood, reflect on your experiences, and create a visual archive of your life that you can revisit anytime.

My Pitch Video

Demo

Live App: https://dailyecho.varshithvhegde.in/
GitHub: https://github.com/Varshithvhegde/dailyecho
Varshithvhegde / dailyecho

DailyEcho - A beautiful, private video journaling app that lets you record daily video diary entries, track your mood over time, and relive your memories through immersive story modes and interactive visual walls.
🎥 Daily Echo

A beautiful, private video journaling app that lets you record daily video diary entries, track your mood over time, and relive your memories through immersive story modes and interactive visual walls.

✨ Features

🎬 Immersive Story Modes (New!)
- Memory Stories - Watch your entries in a sequential, story-like format similar to social media.
- Auto-Curated Playlists - Choose from "Recent Moments", "Moments of Joy" (happy/excited/grateful), or "Flashback" (random picks from the past).
- Smooth Navigation - Interactive progress bars, auto-advance, and gesture/keyboard support.
🧱 Echo Wall (Mosaic Mode)
- Living Visual History - A dynamic masonry grid of your life in motion.
- Living Video Tiles - Each tile plays a Mux-generated animated GIF preview simultaneously for a "Harry Potter" newspaper effect.
- Interactive Previews - Retro CRT scanline overlays and cinematic hover effects.
📹 Video Recording & Playback
- Mux-powered streaming - Professional-grade video processing and playback with adaptive streaming.
- Mux GIFs…
View on GitHub

Testing Credentials:

Email: test@gmail.com
Password: devtest

Detailed Explanation

The Story Behind It

I have a terrible memory. Seriously. Ask me what I did last Tuesday and I'll draw a blank. But I've always been fascinated by the idea of looking back at my life, especially when the end of the year rolls around and everyone's doing their "year in review" thing.

I wanted to create something that would help me remember the small moments - not just the big events, but the everyday stuff. What was I thinking about on a random Wednesday in March? How did I feel when that thing happened at work? What was going through my mind during that phase of my life?

The idea was simple: record a 1-minute video every day. Just sit down, talk to the camera like you're talking to a friend, and capture whatever's on your mind. But I didn't want it to feel like a chore. I wanted it to be something I'd actually look forward to doing.

So I built Daily Echo with features that make revisiting your memories feel magical. The Echo Wall shows all your entries as living video tiles playing simultaneously (like those moving newspapers in Harry Potter). Memory Stories let you watch your entries in sequence, almost like watching a documentary about your own life. And the Time Capsule feature shows you what you were up to exactly one month or one year ago.

It's been incredibly powerful for me personally. There's something about being able to go back and watch yourself from months ago, seeing how you've grown or changed, or just remembering moments you'd completely forgotten.

Technical Highlights

Daily Echo is built with React 18, TypeScript, and Vite on the frontend, with Tailwind CSS and shadcn/ui for the design. The backend runs on Supabase, handling PostgreSQL database, authentication, and edge functions.

What makes the app special technically:

1. Living Video Previews Everywhere

Every entry card in the timeline shows an animated GIF preview that plays automatically. When you hover over the Echo Wall (our mosaic view), you see all your memories playing at once. It creates this incredible "living history" effect that static thumbnails just can't match.

2. AI-Powered Insights

Using OpenAI's GPT-4o-mini, the app automatically analyzes your video transcripts to generate:

Two-sentence summaries of each entry
Emotional sentiment detection
Personalized daily advice based on what you talked about
Mood tracking over time

3. Immersive Story Modes

You can watch your entries in different ways:

Recent Moments: Your latest recordings in sequence
Moments of Joy: Auto-curated playlist of happy entries
Flashback: Random picks from your past

Each mode has interactive progress bars, auto-advance, and keyboard controls for a cinematic experience.

4. Gamification That Actually Matters

Achievement badges like "Zen Master" (recorded before 6 AM), "Night Owl" (recorded after 10 PM), and "Weekend Warrior" (weekend recordings) make the habit more engaging. You can track your recording streaks and see your mood variety over time.

Use of Mux

Mux is the heart and soul of Daily Echo. Here's how I'm using it:

1. Professional Video Infrastructure

When you record a video, it goes through Mux's direct upload API. No dealing with complicated encoding pipelines or storage headaches. Mux handles everything: transcoding, optimization, and adaptive streaming. The result? Your videos play smoothly on any device, any connection speed.

2. Automatic Transcription

This was a game-changer. By enabling Mux's transcription feature during upload, I get accurate text transcripts of every video entry automatically. These transcripts power the AI analysis, search functionality, and accessibility features. I didn't have to integrate a separate transcription service or worry about accuracy.

3. Animated GIF Previews

Instead of static thumbnails, every entry shows a living preview using Mux's GIF generation API. You can watch all your memories playing simultaneously in the Echo Wall view. It's like having a magical photo album where every picture moves. Mux generates these GIFs automatically from your video without any extra work on my end.

4. Reliable Streaming with Mux Player

The integrated Mux Player component handles playback with built-in caption support. It just works - no buffering issues, no format compatibility problems, no manual quality switching needed.

5. Webhook Integration

Mux's webhook system notifies my Supabase edge function when videos are ready, when transcripts are available, and if anything goes wrong. This lets me update the UI in real-time and handle the entire video lifecycle automatically.

The developer experience with Mux has been fantastic. The documentation is clear, the API is intuitive, and features like automatic transcription and GIF generation saved me weeks of development time. Instead of building video infrastructure, I could focus on making the journaling experience special.

What really impressed me: I initially thought I'd need separate services for video hosting, transcription, and preview generation. Mux does all of this out of the box, and it scales effortlessly. When a user records their 100th video, it works just as smoothly as their first.

I hope Daily Echo inspires others to start capturing their daily thoughts. Life moves fast, and our memories fade faster. Having a video archive of your own life is like having a superpower - you can literally go back in time and remember who you were and what mattered to you at any moment.

Give it a try with the test credentials above, and maybe start your own daily echo habit!

My 2025 wrap

Varshith V Hegde — Wed, 31 Dec 2025 15:16:32 +0000

2025 was a rollercoaster for me. Looking back, I can clearly divide it into two distinct halves. One that tested me, and another that transformed me.

The First Half: Climbing the Corporate Ladder

January started strong. I got promoted at work, which was amazing! Though it was mostly a position bump, I was actually leading a project as an Associate Engineer. The best part? We went from 5 days in the office to just 2 days a week. But honestly, I still chose to work from the office most days because I was so invested in the project.

Then came February, the "love month," and let's just say things didn't go as planned. I hit one of the lowest points in my life.

But you know what kept me going? I dove into spirituality and writing poems. These became my anchors during the tough times.

The Turning Point: When Everything Changed

And then came the moment that changed everything.

I had this massive challenge at work. Our tool was taking 8 minutes just to load an MF4 (MDF file). EIGHT MINUTES. And that's before any computation! We were using the asammdf package in Python, which is good, but painfully slow.

I became obsessed with solving this. I researched everything. Tried JIT compilation, which improved computation time but not the loading. Then I had this wild idea: what if I rewrote the entire package in Rust?

This wasn't just a work task anymore. This was MY mission. I worked my regular job during the day and coded this project at night. I went deep into understanding how MDF files work at the byte level, implementing a custom package specifically for our project. Countless all-nighters, endless debugging sessions, but when it finally worked? Pure magic.

10 seconds. That's all it took to load a 4GB file AND do the computation (which I also replaced with Rust). Only the UI remained in Python.

This earned me so much respect at work. Due to NDA, I can't share the code or methods (corporate life, you know), but the fact that I pulled this off still makes me proud. This was my turning point.

Reigniting the Developer Within

Huge shoutout to Jess and Ben for their weekly "What was your win this week?" posts. Reading those comments and seeing everyone's achievements? That reignited my inner developer. I wanted to be part of that energy again.

I restarted my dev.to journey, but I was rusty. I couldn't figure out what to write about.

Then I discovered DEV Challenges, and wow, what a gold mine! I could build, showcase, learn, and enjoy all at once. Every weekend became about tackling a new challenge. This was exactly what I needed to grow and fall in love with coding all over again.

Discovering New Heights (Literally!)

In the second half of the year, I found another passion. Trekking. I climbed Asia's 1st and 2nd largest monolithic rocks! Out of 9 total treks, 8 happened in the second half alone.

I also did my first solo travel to Hampi. I know it's not far, but for me, it was a huge achievement. Plus, I met some amazing people along the way!

Projects That Keep On Giving

Some of my older projects surprised me this year:

Varshithvhegde / FreeShare

FreeShare is a free online file sharing platform designed to simplify the process of sharing files without the need for any sign-up or verification.

FreeShare: File Sharing Platform

🗂️ Description

FreeShare is a file sharing platform built with React, Firebase, and Cloud Functions. This project allows users to share files easily and efficiently. It's designed for individuals and teams who need a simple and secure way to share files.

The platform provides a user-friendly interface for uploading, sharing, and managing files. With FreeShare, you can share files with others by generating a unique link, and recipients can access the files without needing to create an account.

✨ Key Features

File Sharing

Upload and share files with others via a unique link

Supports various file types, including documents, images, and videos

Security and Authentication

Secure file storage using Firebase Storage

Authentication and authorization using Firebase Authentication

User Interface

Responsive and user-friendly interface built with React and Material-UI

Easy navigation and file management

🗂️ Folder Structure

graph TD src-->components src-->App.test.js; src-->index.js; src-->reportWebVitals.js; src-->setupTests.js; public-->index.html; public-->manifest.json; public-->robots.txt;
…

View on GitHub

FreeShare is a free online file-sharing platform that needs no sign-up or verification. I built it 2 years ago in college when I was just starting out. I honestly thought it was dead, but when I checked Firebase recently... 10K total users! People are still using it, and my blog post about it still gets views. Sure, it has flaws, but hey, we all start somewhere.

Varshithvhegde / notepage

NotePage is a web application that allows you to easily share code, text, or any content using a unique link.

NotePage

NotePage is a web application that allows you to easily share code, text, or any content using a unique link. You can create new note pages by simply visiting https://notepage.vercel.app.

Features

Custom Pages: Create your own custom pages to share content with others. Just use https://notepage.vercel.app/<your-page-name> and start sharing.

Password Protection: Optionally protect your pages with a password, ensuring that only authorized users can access your content.

Real-time Collaboration: Collaborate with others in real-time. When multiple users access the same link, any changes made by one user are instantly visible to others, without requiring a page refresh.

Shareable Links: Share your pages with others by sending them the unique link.

Tech Stack

NotePage is built using the following technologies:

Angular: A powerful and popular front-end framework.

Firebase: A real-time cloud database, authentication, and hosting platform.

Angular Material: A UI component library…

View on GitHub

This was just a side project I built while learning Angular. The UI wasn't great, and it's full of bugs. But here's the thing: my office has a strict NO ChatGPT policy (we can only use a company AI), and copying/sharing text was difficult. When a friend needed to share something, I suggested notepage, and it blew up within the office! I tried improving the UI, but people love the old version, so I kept it. Now I'm almost hitting free tier limits, but I'll keep it free because projects like these taught me so much and drove me to write for the DEV community.

The DEV Community Love

And then came the end of the year. Oh wow.

I finally reached 10K followers! I remember celebrating my first 1K like it was yesterday (it was actually 2 years ago).

Varshith V Hegde

Feb 23 '23

Achieving 1K Followers on dev.to: My Journey to Success

#career #productivity #networking #mentorship

Comments 18

3 min read

I was also a Top Weekly Author twice! My DEV profile is practically part of my resume now. Whenever I'm in an interview, I proudly show it and talk about my journey. Why not? I've worked hard for this.

I participated in 5 DEV Challenges and won 2 of them. These challenges helped me grow immensely. A huge thank you to the entire DEV Team for creating such an amazing initiative!

Shoutouts

Some amazing devs/writers whose content I absolutely loved this year:

Nikoloz Turazashvili (@axrisi)Follow

Founder & CTO at Vexrail (www. vexrail.com), Axrisi (www.axrisi.com). Opened Chicos restaurant in Tbilisi, Georgia.

Dumebi OkoloFollow

Confident technical writer with frontend developer skills, marketing skills and developer relations skills. I am also a very fun person to hang around with.

Arindam Majumder Follow

Developer Advocate | Technical Writer | 600k+ Reads | Mail for Collabs

DivyaFollow

A curious lifelong learner, currently a full-time Masters student persuing Computer Science stream. Enthusiastic about development.

Looking Ahead to 2026

So yeah, the second half of 2025 was my redemption arc. I really loved this year, and my resolutions are crystal clear:

Get all my friends to participate in DEV Challenges
Level up my skills in AI and Agentic development
Travel more (maybe even an international trip!)
Blog about my journey more consistently
Start learning rock climbing (need to get in shape first 😅)
Finally get my driver's license 😭 (It was my 2025 resolution too, but I still haven't done it!)

That's all for this year, folks!

Thank you everyone for the support and love. Here's to an even better 2026!

Happy New Year to ALL! 🎉

I Built a Form Backend in a Weekend Because Paying $20/Month for Contact Forms is Stupid

Varshith V Hegde — Tue, 30 Dec 2025 15:59:39 +0000

So here's the thing - I was helping my friend set up his portfolio site last weekend. Everything was going smooth. Nice design, fast site, Vercel hosting on the free tier. Perfect.

Then he goes "I need a contact form."

Cool, I say. Just use one of those form backend services. Easy.

He checks the pricing. "$20 a month?! Just to save some text?"

And honestly? He's right. When did we all just accept this?

This got me thinking

We're paying Netflix money for what's basically a database insert and an email. That's it. Store some text, send a notification.

I spent more time being annoyed about this than I'd like to admit. Then I figured - you know what, I can probably build this myself. How hard can it be?

What I built

FormRelay is pretty straightforward. You point your HTML form at it, it saves the data, sends you an email, and shows everything in a dashboard. That's the whole thing.

The difference? You host it yourself. Your Supabase database. Your Vercel deployment. Your data.

And the best part? Supabase free tier gives you 50k users. Vercel hobby plan is free. Resend gives you 3k emails free per month.

So yeah. $0/month vs $20/month. You do the math.

"But isn't self-hosting complicated?"

This is what everyone says. And look, 10 years ago? Sure. You needed to know server management, deal with security updates, all that stuff.

But now? With Vercel and Supabase?

Fork the repo
Click deploy
Copy paste some environment variables
You're done

Took me longer to write the README than it takes to deploy this thing.

Compare that to creating yet another account, entering your credit card, dealing with their dashboard, hitting some arbitrary limit, and then having to migrate everything when they raise prices next year.

Which one sounds more complicated?

Here's where I might lose some of you

I think we've gotten too comfortable renting everything.

The whole indie web thing used to be about actually owning your stuff. Yeah, it was messier. Yeah, you had to learn things. But your website was yours.

Now? We just subscribe to everything. And sure, time is money, I get that. Not everyone wants to deal with infrastructure.

But there's this huge gap between "run your own server rack" and "pay someone $300/year to store contact form entries."

That's the gap I'm trying to fill here.

Random things I learned

Next.js 15 is actually good now. After all the App Router drama, it finally feels right. Server actions just work. No more fighting with it.

Supabase is wild. Real-time updates, auth, and actual good documentation? Sign me up.

The hardest part wasn't the code. It was making the setup instructions clear enough that anyone could follow them. Spent way too long on that.

About the email thing

I used Resend because my domain didn't come with email when I bought it, and I wasn't planning to buy email hosting separately. Resend's free tier (3k emails/month) was perfect for this.

But here's the thing - if you already have email with your domain, you can just swap Resend for regular SMTP. It's actually simpler in some ways. Just plug in your SMTP credentials and you're good to go.

So even that "dependency" isn't really a dependency. Use what you've got.

Tech stuff if you care

Next.js 15 (app router)
Supabase (postgres + realtime + auth)
Tailwind CSS
Radix UI
Resend for emails (or just use SMTP if you have it)
Lucide icons

Nothing fancy. Just stuff that works and doesn't break.

You can use it

Whole thing's on GitHub: github.com/Varshithvhegde/formrelay

Live demo: formrelay.varshithvhegde.in

It's MIT licensed. Do whatever you want with it. If you find bugs, let me know. If you want to add features, PRs are open.

The actual point

This isn't really about forms or saving money.

It's about remembering that we can actually build stuff ourselves. We don't need a SaaS product for every little thing. The tools are there. The platforms are free. We know how to code.

A lot of problems that cost $20/month are actually just weekend projects in disguise.

Quick note: Yes, I know SaaS companies provide value. Support, maintenance, features, etc. But for something as basic as form handling? Come on.

If you are creating any Multi Agent or AI apps you need to check this out!!!

Varshith V Hegde — Sun, 21 Dec 2025 15:47:12 +0000

Varshith V Hegde

Dec 18 '25

Bifrost: The LLM Gateway That's 40x Faster Than LiteLLM

#webdev #ai #programming #agents

Comments 2

10 min read

Bifrost: The LLM Gateway That's 40x Faster Than LiteLLM

Varshith V Hegde — Thu, 18 Dec 2025 16:26:42 +0000

A technical deep dive into Bifrost: an open-source, self-hostable Go LLM gateway

Gateway Overhead in Production LLM Systems

In most LLM systems, the gateway becomes a shared dependency: it affects tail latency, routing/failover behavior, retries, and cost attribution across providers. LiteLLM works well as a lightweight Python proxy, but in our production-like load tests we started seeing gateway overhead and operational complexity show up at higher concurrency. We moved to Bifrost for lower overhead and for first-class features like governance, cost semantics, and observability built into the gateway.

In our benchmark setup (with logging/retries enabled), LiteLLM added hundreds of microseconds of overhead per request. Results vary by deployment mode and configuration. When handling thousands of requests per second, this overhead compounds—infrastructure costs increase, tail latency suffers, and operational complexity grows.

Bifrost takes a different approach.

Enter Bifrost

Bifrost is an LLM gateway written in Go that adds approximately 11 microseconds of overhead per request in our test environment. That's roughly 40x faster than what we observed with LiteLLM in comparable configurations.

But the performance improvement is just one part of the story. Bifrost rethinks the control plane for LLM infrastructure—providing governance, cost attribution, and observability as first-class gateway features rather than requiring external tooling or application-level instrumentation.

Let me walk through the technical details.

maximhq / bifrost

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

Bifrost AI Gateway

The fastest way to build AI applications that never go down

Bifrost is a high-performance AI gateway that unifies access to 15+ providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, and more) through a single OpenAI-compatible API. Deploy in seconds with zero configuration and get automatic failover, load balancing, semantic caching, and enterprise-grade features.

Quick Start

Go from zero to production-ready AI gateway in under a minute.

Step 1: Start Bifrost Gateway

# Install and run locally
npx -y @maximhq/bifrost

# Or use Docker
docker run -p 8080:8080 maximhq/bifrost

Step 2: Configure via Web UI

# Open the built-in web interface
open http://localhost:8080

Step 3: Make your first API call

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello, Bifrost!"}]
  }'

That's it! Your AI gateway is running with a web interface for visual configuration…

View on GitHub

Setup and Deployment

Traditional gateway deployment often involves managing Python environments, dependency chains, and configuration files. Here's the Bifrost approach:

npx -y @maximhq/bifrost

This single command downloads a pre-compiled binary for your platform and starts a production-ready gateway on port 8080 with a web UI for configuration.

Compare this to typical Python gateway setup:

# Install Python (verify version compatibility)
pip install litellm
# Configure environment variables
# Set up configuration file
# Install additional dependencies for features
# Debug environment-specific issues

Bifrost uses NPX to download a pre-compiled binary for your platform. No Python interpreter required. No virtual environments. No dependency resolution. A single statically-linked executable that runs immediately.

Why Go for Gateway Infrastructure

The choice of Go over Python has measurable impacts on production systems, particularly around concurrency, memory efficiency, and operational simplicity.

Concurrency Model

Python gateways scale via async and multiple workers. At high concurrency, the tradeoffs show up as higher memory per instance, coordination overhead, and tail-latency under burst.

Go doesn't have these constraints. Go's goroutines are lightweight threads that can run truly in parallel across all available CPU cores. When a request arrives, Bifrost spawns a goroutine. When a thousand requests arrive simultaneously, Bifrost spawns a thousand goroutines—all running concurrently with minimal overhead.

Memory Efficiency

A Python process typically requires 30-50MB of memory at startup in most configurations. Add Flask or FastAPI, and baseline memory usage often reaches 100MB+ before handling any requests, though this varies based on the specific setup and dependencies.
The entire Bifrost binary is approximately 20MB. In memory, a single Bifrost instance uses roughly 50MB under sustained load while handling thousands of requests per second.

Startup Time

Python applications require time to initialize—import packages, start the interpreter, load configurations. Typical startup time is 2-3 seconds minimum.

Bifrost starts in milliseconds. This matters for autoscaling, development iteration, and serverless deployments where cold starts impact user experience.

Benchmark Results

Here are measurements from a sustained load test on a t3.xlarge EC2 instance at 5,000 requests per second:

Metric	LiteLLM	Bifrost	Improvement
Gateway Overhead	440 µs	11 µs	40x faster
Memory Usage	~500 MB	~50 MB	10x less
Gateway-level Failures	11%	0%	No failures observed
Queue Wait Time	47 µs	1.67 µs	28x faster
Total Latency (with provider)	2.12 s	1.61 s	24% faster

These measurements represent sustained load over multiple hours, not synthetic benchmarks.

Beyond Performance: Control-Plane Features That Matter in Production

The main reason to move from LiteLLM to Bifrost isn't language; it's control-plane features. Bifrost adds governance (virtual keys, budgets, rate limits), consistent cost attribution, and production-oriented observability at the gateway layer, not scattered across application code.

This architectural choice centralizes concerns that would otherwise require external services or application-level instrumentation:

Governance controls managed at the gateway rather than per-application
Cost attribution with per-request tracking and aggregation
Observability with structured logs, metrics, and request tracing built-in
Failure isolation with circuit breakers and automatic failover

Let's examine these features in detail.

Production Features

Automatic Failover

When your primary provider hits rate limits or experiences downtime, requests should seamlessly move to backup providers without manual intervention.

Bifrost configuration:

{
  "fallbacks": {
    "enabled": true,
    "order": [
      "openai/gpt-4o-mini",
      "anthropic/claude-sonnet-4",
      "mistral/mistral-large-latest"
    ]
  }
}

When OpenAI returns a rate limit error, Bifrost automatically retries with Anthropic. If that fails, it tries Mistral. Your application receives a successful response without implementing retry logic.

Load Balancing

Distributing load across multiple API keys prevents any single key from hitting rate limits:

{
  "providers": {
    "openai": {
      "keys": [
        {"name": "key-1", "value": "sk-...", "weight": 2.0},
        {"name": "key-2", "value": "sk-...", "weight": 1.0},
        {"name": "key-3", "value": "sk-...", "weight": 1.0}
      ]
    }
  }
}

The first key receives 50% of traffic, the other two receive 25% each. When one key approaches its rate limit, Bifrost automatically shifts load to healthy keys.

Semantic Caching

Semantic caching isn't a new concept; teams can build it externally, but Bifrost ships it as a first-class gateway feature, reducing moving parts.

Traditional caching requires exact string matches. But users rarely phrase questions identically:

"What's the weather like?"
"How's the weather today?"
"Tell me about current weather conditions"

These are semantically equivalent. Bifrost uses vector embeddings to understand semantic similarity:

Request arrives: "What is Python?"
Bifrost generates an embedding using a fast model
Checks vector store for similar embeddings
Finds previous request: "Explain Python to me"
Returns cached response (similarity score: 0.92)

Result: No LLM call required. Response in approximately 5 milliseconds instead of 2 seconds. Cost: $0.00 instead of $0.0001.

Savings depend on cache hit-rate and workload repetition. Over a million requests with 60% cache hit rate, this saves approximately $60.

Unified Interface

Every LLM provider has different API formats. OpenAI uses one schema. Anthropic uses another. Bedrock and Vertex AI each have their own specifications.

Bifrost provides a single API that works with all providers:

from openai import OpenAI

# Change only the base URL
client = OpenAI(
    api_key="not-needed",
    base_url="http://localhost:8080/openai"
)

# Use ANY provider with the same code
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4",  # Not an OpenAI model
    messages=[{"role": "user", "content": "Hello"}]
)

Your application code remains unchanged. Switch providers by modifying one line. No refactoring required. No rewriting integration tests.

Model Context Protocol (MCP)

MCP is Anthropic's protocol for letting AI models use external tools. Integration with web search, filesystem access, or database queries:

{
  "mcp": {
    "enabled": true,
    "servers": {
      "web-search": {
        "command": "npx",
        "args": ["-y", "@modelcontextprotocol/server-brave-search"]
      },
      "filesystem": {
        "command": "npx",
        "args": ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
      }
    }
  }
}

This enables AI models to perform actions rather than only generating text responses.

Web UI and Observability

Most gateways provide configuration files and command-line tools. Bifrost includes a comprehensive web interface at http://localhost:8080:

Dashboard: Real-time metrics showing request counts, error rates, and costs per provider

Providers: Visual configuration for all providers with click-based key management

Logs: Complete request/response history with token usage, searchable and filterable

Settings: Configure caching, governance, and plugins without editing configuration files

All configuration, monitoring, and debugging can be performed through the web interface without SSH access to servers or manual log analysis.

Architecture Details

Request Flow

Request arrives at Bifrost's HTTP server
Request validation happens in microseconds
Cache lookup checks semantic cache if enabled
Cache hit? Return immediately (approximately 5ms total)
Cache miss? Continue to provider selection
Load balancer selects API key based on weights and health
Concurrent request dispatched to provider (goroutine spawned)
Response streaming begins immediately if enabled
Cache storage happens asynchronously (non-blocking)
Response returns to client with metadata

All operations are non-blocking where possible. Cache lookup doesn't block provider calls in no-store mode. Cache storage doesn't delay response delivery.

Concurrency Implementation

Bifrost uses Go's goroutines for concurrency:

Traditional Python Threading:

Request 1 → Thread 1 → Process (limited parallelism)
Request 2 → Thread 2 → Wait/Process (coordination overhead)
Request 3 → Thread 3 → Wait/Process (memory per thread)

Bifrost Goroutines:

Request 1 → Goroutine 1 ⟍
Request 2 → Goroutine 2 ⟋→ All process in parallel → Responses
Request 3 → Goroutine 3 ⟋

Each goroutine uses approximately 2KB of memory. You can run millions concurrently.

Vector Store Integration

For semantic caching, Bifrost integrates with Weaviate:

Request arrives with cache key: "user-session-123"
Bifrost extracts message content
Generates embedding using fast model (text-embedding-3-small)
Searches Weaviate for similar embeddings (threshold: 0.8)
Finds match with similarity 0.92
Returns cached response with metadata

Embedding generation: approximately 50ms. Vector search: approximately 10ms. Total: 60ms compared to 2000ms for an actual LLM call.

Setting Up Semantic Caching

Step 1: Install Weaviate

Use Docker for local development:

docker run -d \
  --name weaviate \
  -p 8081:8080 \
  -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
  semitechnologies/weaviate:latest

Or use Weaviate Cloud (free tier available) at https://console.weaviate.cloud/

Step 2: Configure Bifrost

Update your config.json:

{
  "providers": {
    "openai": {
      "keys": [{
        "name": "main",
        "value": "env.OPENAI_API_KEY",
        "models": ["gpt-4o-mini"]
      }]
    }
  },
  "vector_store": {
    "enabled": true,
    "type": "weaviate",
    "config": {
      "host": "localhost:8081",
      "scheme": "http"
    }
  },
  "plugins": [{
    "enabled": true,
    "name": "semantic_cache",
    "config": {
      "provider": "openai",
      "embedding_model": "text-embedding-3-small",
      "ttl": "5m",
      "threshold": 0.8
    }
  }]
}

Step 3: Test the Cache

# First request (cache miss, calls LLM)
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-bf-cache-key: user-123" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is Docker?"}]
  }'

# Similar request (cache hit, fast response)
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-bf-cache-key: user-123" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Explain Docker to me"}]
  }'

The second request returns in approximately 60ms instead of 2000ms.

Cache Hit Response

{
  "choices": [...],
  "extra_fields": {
    "cache_debug": {
      "cache_hit": true,
      "hit_type": "semantic",
      "similarity": 0.94,
      "threshold": 0.8,
      "provider_used": "openai",
      "model_used": "text-embedding-3-small"
    }
  }
}

The response includes debugging information showing the similarity score and threshold. Adjust the threshold based on your accuracy requirements.

Drop-In Replacement

You can replace existing OpenAI or Anthropic SDK calls with Bifrost by changing one parameter.

Python Example

Before:

from openai import OpenAI

client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}]
)

After:

from openai import OpenAI

client = OpenAI(
    api_key="not-needed",
    base_url="http://localhost:8080/openai"  # Only change
)
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}]
)

Node.js Example

Before:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

After:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'not-needed',
  baseURL: 'http://localhost:8080/openai'  // Only change
});

Benefits

This approach enables:

Adding Bifrost to existing applications without refactoring
Testing in production with gradual rollout
Access to all Bifrost features (caching, fallbacks, monitoring) immediately
Easy rollback if needed

Monitoring and Observability

Bifrost exposes Prometheus metrics at /metrics:

# Request metrics
bifrost_requests_total{provider="openai",model="gpt-4o-mini"} 1543
bifrost_request_duration_seconds{provider="openai"} 1.234

# Cache metrics
bifrost_cache_hits_total{type="semantic"} 892
bifrost_cache_misses_total 651

# Error metrics
bifrost_errors_total{provider="openai",type="rate_limit"} 12

Grafana Dashboard

Connect Prometheus to Grafana for visualization:

Requests per second by provider
Latency percentiles (p50, p95, p99)
Cache hit rates over time
Cost tracking per provider
Error rates and types

Structured Logging

Bifrost logs to structured JSON:

{
  "level": "info",
  "time": "2024-01-15T10:30:00Z",
  "msg": "request completed",
  "provider": "openai",
  "model": "gpt-4o-mini",
  "duration_ms": 1234,
  "tokens": 456,
  "cache_hit": false
}

This format integrates with any log aggregation service (CloudWatch, Datadog, Elasticsearch).

Common Configuration Issues

Issue 1: Missing Cache Key

Semantic caching requires the x-bf-cache-key header. Without it, every request is a cache miss.

Incorrect:

curl -X POST http://localhost:8080/v1/chat/completions -d '{...}'

Correct:

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "x-bf-cache-key: user-session-123" \
  -d '{...}'

Issue 2: Threshold Configuration

Start with a threshold of 0.8 and adjust based on cache hit rate and accuracy:

{
  "threshold": 0.8  // Starting point
}

Monitor your cache hit rate. If below 30%, lower the threshold to 0.75. If you're getting incorrect cached results, raise it to 0.85.

Issue 3: Config Store Requirement

Some plugins require a config store:

{
  "config_store": {
    "enabled": true,
    "type": "sqlite",
    "config": {"path": "./config.db"}
  }
}

Issue 4: Weaviate Network Configuration

Ensure Weaviate is accessible from Bifrost. For Docker deployments:

{
  "vector_store": {
    "enabled": true,
    "type": "weaviate",
    "config": {
      "host": "weaviate-container:8080",  // Use correct hostname
      "scheme": "http"
    }
  }
}

For Weaviate Cloud:

{
  "vector_store": {
    "enabled": true,
    "type": "weaviate",
    "config": {
      "host": "<weaviate-host>.gcp.weaviate.cloud",
      "scheme": "https",
      "api_key": "<weaviate-api-key>"
    }
  }
}

When to Use Bifrost

Bifrost provides immediate value for:

Production systems handling more than 1,000 requests per day
Applications where tail latency impacts user experience
Teams that need automatic failover without complex orchestration
Organizations tracking LLM costs across multiple providers
Systems requiring governance controls (rate limits, budgets, virtual keys)
Deployments where operational simplicity reduces maintenance burden

Even for smaller projects, Bifrost's minimal overhead and built-in features provide a robust foundation that scales without requiring future refactoring.

Getting Started

Run npx -y @maximhq/bifrost
Open http://localhost:8080
Add your API keys in the UI
Point your application to http://localhost:8080/openai
Monitor performance and costs through the dashboard

Resources

GitHub: github.com/maximhq/bifrost
Website: getbifrost.ai
Documentation: docs.getbifrost.ai

Questions or feedback? Please leave a comment below. If you use Bifrost in production, I'd be interested to hear about your experience and any challenges you encounter.

HTML tags that NO ONE talks about

Varshith V Hegde — Sun, 14 Dec 2025 11:01:34 +0000

Look, I get it. HTML can feel boring sometimes. You throw in your divs and spans, maybe toss in a form here and there, and call it a day. But there are some genuinely cool HTML elements that most of us completely ignore, and honestly? They make things so much simpler.

Let me show you six HTML tags that actually solve real problems. No fancy jargon, just practical stuff you can start using today.

1. The `<template>` Element - Your HTML Blueprint

Okay, so you need to add stuff to your page dynamically. Maybe it's a new comment, a product card, whatever. You've probably been doing one of these:

The old way (don't do this):

// Using innerHTML - looks easy but it's dangerous
element.innerHTML = '<div class="card"><h3>' + userInput + '</h3></div>';
// If userInput has <script> tags? Yeah, you're in trouble.

Or maybe you went the "safe but annoying" route:

// createElement - safe but tedious
const div = document.createElement('div');
div.className = 'card';
const h3 = document.createElement('h3');
h3.textContent = userInput;
div.appendChild(h3);
// This gets old real fast with complex HTML

The better way with <template>:

<template id="card-template">
  <div class="card">
    <h3 class="card-title"></h3>
    <p class="card-description"></p>
  </div>
</template>

// Clone it and use it
const template = document.getElementById('card-template');
const clone = template.content.cloneNode(true);

// Fill in your data
clone.querySelector('.card-title').textContent = userInput;
clone.querySelector('.card-description').textContent = description;

// Add it to the page
document.body.appendChild(clone);

Why this rocks: The HTML stays in HTML where it belongs, it's safer than innerHTML, and it's faster because the browser only has to layout the new element instead of re-parsing everything.

2. Input Types You Didn't Know You Had

Stop reinventing the wheel with custom date pickers and validation libraries. The humble <input> tag is way more powerful than you think.

Email and phone validation built right in:

<input type="email" placeholder="your@email.com">
<input type="tel" placeholder="555-1234">

On mobile, these automatically show the right keyboard. Plus, you get free validation styling:

input:invalid {
  border-color: red;
}

Want pattern matching without JavaScript?

<input 
  type="text" 
  pattern="[0-9]{3}-[0-9]{3}-[0-9]{4}"
  placeholder="123-456-7890"
>

Date and time pickers that just work:

<input type="date">
<input type="datetime-local">
<input type="month">
<input type="week">
<input type="time">

No library needed. They work across all modern browsers and look native on each platform.

Other handy types:

<input type="search">  <!-- Gets a little X to clear on some browsers -->
<input type="range" min="0" max="100">  <!-- Slider! -->
<input type="color">  <!-- Color picker! -->

Autocomplete with <datalist>:

<input list="cities" placeholder="Choose a city">
<datalist id="cities">
  <option value="New York">
  <option value="Los Angeles">
  <option value="Chicago">
  <option value="Houston">
</datalist>

Type a few letters and get suggestions. Zero JavaScript required.

3. The `inert` Attribute - Focus Control Made Easy

Ever had a modal popup where users could still tab to stuff behind it? Super annoying, right?

Before (the hard way):
You'd have to manually track all focusable elements, disable them, remember their state, re-enable them later... ugh.

After (the easy way):

<div class="page-content" inert>
  <!-- Everything in here becomes unclickable and unfocusable -->
  <button>Can't click me now</button>
  <input type="text">  <!-- Can't focus this either -->
</div>

<div class="modal">
  <!-- Only stuff in here is interactive -->
  <button>Close</button>
</div>

That's it. One attribute. The browser handles everything, including making it work properly with screen readers.

4. The `<dialog>` Element - Modals That Don't Suck

Speaking of modals, stop building them from scratch with divs and a prayer.

Here's how simple it can be:

<dialog id="my-dialog">
  <h2>Hello!</h2>
  <p>This is a proper dialog.</p>
  <button onclick="this.closest('dialog').close()">Close</button>
</dialog>

<button onclick="document.getElementById('my-dialog').showModal()">
  Open Dialog
</button>

const dialog = document.getElementById('my-dialog');

// Show it as a modal (with backdrop, makes everything else inert)
dialog.showModal();

// Or show it without the modal behavior
dialog.show();

// Close it
dialog.close();

Control how it closes:

<!-- Only closes via JavaScript -->
<dialog closedby="none">...</dialog>

<!-- Closes with Escape key or JavaScript -->
<dialog closedby="closerequest">...</dialog>

<!-- Closes with Escape, clicking outside, or JavaScript -->
<dialog closedby="any">...</dialog>

The backdrop (that dark overlay) comes free, everything else on the page becomes inert automatically, and it's accessible by default.

5. The `<picture>` Element - Responsive Images Done Right

Stop loading massive desktop images on phones. Your users on data plans will thank you.

Basic responsive images:

<picture>
  <!-- Small screens get the mobile version -->
  <source media="(max-width: 500px)" srcset="small-image.jpg">

  <!-- Medium screens get the tablet version -->
  <source media="(max-width: 1000px)" srcset="medium-image.jpg">

  <!-- Fallback for everything else -->
  <img src="large-image.jpg" alt="Description">
</picture>

Handle high-DPI displays:

<picture>
  <source 
    media="(max-width: 500px)" 
    srcset="mobile.jpg 1x, mobile@2x.jpg 2x, mobile@3x.jpg 3x"
  >
  <img src="desktop.jpg" alt="Description">
</picture>

Different crops for different orientations:

<picture>
  <!-- Portrait orientation gets a vertical crop -->
  <source 
    media="(orientation: portrait)" 
    srcset="portrait-crop.jpg"
  >

  <!-- Landscape gets a horizontal crop -->
  <img src="landscape-crop.jpg" alt="Description">
</picture>

The browser picks the first matching source and only downloads that one image. Smart, right?

6. The `<output>` Element - Show Your Results

This one's simple but makes your forms way more accessible.

Instead of this:

<span id="total">$0.00</span>

Do this:

<label>
  Price: 
  <input type="number" id="price" value="0">
</label>

<label>
  Quantity: 
  <input type="number" id="quantity" value="0">
</label>

<output for="price quantity" id="total">$0.00</output>

function updateTotal() {
  const price = document.getElementById('price').value;
  const quantity = document.getElementById('quantity').value;
  document.getElementById('total').textContent = 
    '$' + (price * quantity).toFixed(2);
}

Why bother? Screen readers understand that this is a calculated result related to those specific inputs. They can announce the change without moving focus away from what the user is doing. It's just better UX.

Wrapping Up

HTML keeps getting better, but a lot of us are still writing it like it's 2010. These tags aren't exotic or experimental anymore. They work, they're supported, and they solve real problems.

Next time you reach for a JavaScript library or start building something from scratch, take a minute to check if HTML already has what you need. You might be surprised.

Now go forth and use some proper HTML tags. Your future self (and your users) will appreciate it.

Forem: Varshith V Hegde

I Spent 3 Days Debugging Our LLM Setup. Turns Out We Needed an AI Gateway the Whole Time.

So what actually is an AI Gateway?

The architecture is simpler than it sounds

"But I already have an API gateway. Isn't that enough?"

Do you actually need one right now though?

What actually pushes teams over the edge

What a production gateway actually looks like

The question that actually helped me decide

The Great Claude Code Leak of 2026: Accident, Incompetence, or the Best PR Stunt in AI History?

1. What Actually Happened

The Root Cause: One Missing Line in .npmignore

The Bun Factor

2. The Timeline

By the Numbers

3. SECURITY ALERT: The axios RAT

4. What Was Inside: The Full Breakdown

The Architecture

5. Hidden Features Anthropic Never Meant to Ship

KAIROS: Always-On Autonomous Agent

ULTRAPLAN: 30-Minute Remote Planning Sessions

Coordinator Mode: Multi-Agent Orchestration

BUDDY: The Part Nobody Expected

Anti-Distillation: Poisoning Competitor Training Data

Frustration Detection via Regex

250,000 Wasted API Calls Per Day

6. The "Capybara" Model Confirmed

7. Alternative Theory: Was This Anthropic's PR Play?

The Counter-Arguments (Why It's Probably Just Incompetence)

8. Why DMCA Won't Fix This

9. What This Means For You

10. Lessons for Every Dev Team

Conclusion

References

Using Claude Code with Any LLM: Why a Gateway Changes Everything

The Real Friction Points

Why Bifrost

What We'll Cover

Part 1: Setting Up Bifrost

Step 1: Install Bifrost

Step 2: Create Your Config File

Step 3: Set Your API Keys

Step 4: Start the Gateway

Part 2: Connecting Claude Code to Bifrost

Step 1: Install Claude Code

Step 2: Point Claude Code at Bifrost

Step 3: Run Claude Code with Any Model

Part 3: Routing Rules, Fallbacks, and Budgets

Weighted Routing Across Providers

Automatic Fallbacks

Part 4: MCP Tool Integration

Step 1: Add MCP Configuration to Bifrost

Step 2: Add Bifrost as an MCP Server in Claude Code

Step 3: Verify with a Real Task

Part 5: Observability and Monitoring

Is This Worth Setting Up?

ContractCompass: Your AI Contract Analyst That Actually Speaks Human

What I Built

The Problem

The Solution

Key Capabilities

Demo

How It Feels to Use

How I Used Algolia Agent Studio

The Index: A Knowledge Base of Contract Clauses

How Retrieval Powers the Conversation

Making the Agent Conversational

Why Fast Retrieval Matters

Speed Creates Natural Dialogue

Contextual Retrieval Enables Nuanced Analysis

Retrieval Grounds Responses in Real Data

The Impact on User Experience

Technical Architecture

Frontend (React + TypeScript)

AI Agent (Algolia Agent Studio)

Search Index (Algolia)

PDF Processing

Backend (Serverless Functions)

Design Decisions

Split-Screen Layout

The Root Cause: One Missing Line in `.npmignore`