Swap OpenAI for DeepSeek without rewriting a single line of code

TokenHub — Mon, 27 Apr 2026 03:55:51 +0000

Last month I added Claude to a project that was already using GPT-4o. Two SDKs, two error formats, two retry strategies. By the time I finished I had wrapped both in my own abstraction — a tiny LLM gateway, badly written, that I now had to maintain.

Then I noticed something I should have noticed earlier: most of the new providers expose an OpenAI-compatible endpoint. DeepSeek, Mistral, Together, Fireworks — they all speak the same wire format. You don't need a new SDK. You need a new base_url.

This post is the 5-minute version of that realization, with the tradeoffs I learned the hard way.

The "before" code

Standard OpenAI Python:

from openai import OpenAI

client = OpenAI(api_key="sk-...")

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize this PR diff..."}],
)
print(resp.choices[0].message.content)

The "after" code

from openai import OpenAI

client = OpenAI(
    api_key="th-...",
    base_url="https://jiatoken.com/v1",   # gateway
)

# Same call, different model
resp = client.chat.completions.create(
    model="deepseek-chat",                # DeepSeek-V3
    messages=[{"role": "user", "content": "Summarize this PR diff..."}],
)

That's it. Two lines changed. The rest of your code — streaming handlers, tool calls, retry logic — keeps working because the response shape is identical.

Why this works

The OpenAI Python SDK is just a typed HTTP client. It POSTs JSON to {base_url}/chat/completions. Anything that responds with the same JSON shape is, from the SDK's point of view, OpenAI.

Most gateways take advantage of this:

DeepSeek ships its own OpenAI-compatible endpoint at api.deepseek.com/v1. You can point the SDK there directly.
Anthropic does not — Claude has its own message format. You need a translator.
Gemini has both: a native API and a Vertex-side OpenAI shim.

A multi-model gateway (LiteLLM, OpenRouter, TokenHub, your own) collapses these into one endpoint. One key, one base_url, every model behind it.

What I actually save

For the workload I just migrated (~3M input tokens / 1M output per day, mostly summarization):

Model	Input $/1M	Output $/1M	Daily cost
GPT-4o	2.50	10.00	$17.50
Claude 3.5	3.00	15.00	$24.00
DeepSeek-V3	0.07	0.28	$0.49

DeepSeek isn't a drop-in quality replacement for everything — GPT-4o still wins on instruction following in my evals — but for the 80% of calls that are "summarize this", "extract these fields", "rewrite in tone X", it's fine and ~35× cheaper.

The annoying parts

A few things don't carry over cleanly through OpenAI compatibility:

Tool calling JSON shape. Most providers match it now, but older OSS models return tool calls inside the content string. Always test with your actual prompts before flipping production.
Vision. OpenAI uses image_url parts; some providers want base64. A gateway should normalize this for you — verify before you assume.
Streaming with usage stats. OpenAI added stream_options={"include_usage": True} to get token counts on the final SSE chunk. Not every backend forwards this.
Rate limits. You're now subject to the gateway's RPM, which may be lower than direct provider limits.

When NOT to use a gateway

You only ever call one provider. Direct SDK is one less moving part.
You need provider-specific features (Anthropic's prompt caching, OpenAI's Realtime API, Gemini's long context). Gateways usually lag behind native features by weeks.
You're in a regulated environment that requires data plane control. Most gateways are SaaS.

For everything else — especially side projects and prototypes where the model you "want" changes every two weeks — a gateway pays for itself in saved switching cost.

TL;DR

  client = OpenAI(
      api_key="...",
+     base_url="https://your-gateway/v1",
  )
  client.chat.completions.create(
-     model="gpt-4o",
+     model="deepseek-chat",
      ...
  )

If you want to skip running your own LiteLLM, TokenHub hosts a pre-configured gateway with 40+ models behind one key. Otherwise, LiteLLM self-hosted is the standard answer.

I Built an OpenAI-Compatible Gateway to 40+ AI Models (DeepSeek, MiniMax, Claude)

TokenHub — Sun, 26 Apr 2026 08:31:18 +0000

The Problem

I was paying for 5+ different AI subscriptions: OpenAI, Anthropic, Google, etc. Each with separate API keys, billing dashboards, and SDK quirks.

When DeepSeek-V3 dropped at ~$0.28 per million output tokens (vs GPT-4o at $10), I wanted to switch — but the friction of changing SDKs across multiple projects was a pain.

So I built TokenHub — an OpenAI-compatible gateway that routes to 40+ AI models with a single API key.

How It Works

It's a drop-in replacement for the OpenAI SDK. Just change base_url and api_key:

from openai import OpenAI

client = OpenAI(
    api_key="your-tokenhub-key",
    base_url="https://jiatoken.com/v1"
)

# Use any of 40+ models — DeepSeek, MiniMax, Claude, GPT, Gemini, Llama, etc.
response = client.chat.completions.create(
    model="deepseek-v3",
    messages=[{"role": "user", "content": "Explain async/await in Python"}]
)

print(response.choices[0].message.content)

That's it. The same code works with:

gpt-4o
claude-sonnet-4-6
gemini-2.5-pro
deepseek-v3 / deepseek-r1
minimax-text-01
llama-3.3-70b
...and more

Real Pricing Comparison

Per million tokens (input / output):

Model	Provider	Input	Output
GPT-4o	OpenAI	$2.50	$10.00
GPT-4o mini	OpenAI	$0.15	$0.60
DeepSeek-V3	TokenHub	$0.07	$0.28
DeepSeek-R1	TokenHub	$0.14	$0.55
MiniMax-Text-01	TokenHub	$0.10	$0.40

For high-volume workloads (RAG, agents, batch summarization), DeepSeek-V3 is ~35x cheaper than GPT-4o for output tokens.

When to Use Which Model

A quick mental model from my own usage:

Cheap & good enough → DeepSeek-V3 (most general tasks)
Reasoning → DeepSeek-R1 (CoT-style tasks)
Long context → MiniMax-Text-01 (200K+ tokens)
Frontier capability → GPT-4o or Claude (still worth it for hard problems)
Code → Claude Sonnet 4.6 or DeepSeek-V3

The win is being able to A/B test across models without rewriting code.

Why I Open-Sourced the Routing Logic

(Note: TokenHub itself is hosted, but the routing pattern is straightforward.)

The hardest part wasn't the proxy — it was:

Normalizing function-calling formats across providers
Handling streaming differences (SSE format quirks)
Token counting for accurate billing pre-request

If you're building something similar, the OpenAI spec is the de facto standard. Most providers either match it or have OpenAI-compatible endpoints already.

Try It

If you're tired of juggling AI subscriptions:

👉 https://jiatoken.com
Free credits to start
Pay-as-you-go, no monthly commitment
Compatible with OpenAI SDK out of the box

I'd love feedback — especially on which models you'd want added, or pricing pain points.

What's your current setup? Are you using a single provider or juggling multiple?

Forem: TokenHub