I Built an Open-Source Multi-Model API Gateway

csx0574 — Fri, 15 May 2026 16:26:55 +0000

The Problem

Managing multiple AI model providers is a mess. Each has its own API, pricing, and quirks.

I got tired of juggling keys and decided to build one gateway to rule them all.

What I Built

An OpenAI-compatible API gateway that routes requests across 43 models from 13 providers—transparently, with cost tracking and smart routing.

Live Demo (Cost Calculator): https://csx0574--calculator.modal.run

Features

43 models, 13 providers — OpenAI, Anthropic, Google, Meta, Mistral, Zhipu, DeepSeek, Minimax, Groq, Fireworks, Novita, Kampute, XAI
OpenAI-compatible endpoint — Drop-in replacement for your existing OpenAI code
Smart routing — Choose by cost, speed, or balance
Cost tracking — See exactly how much each request costs

Quick Start

git clone https://github.com/csx0574/ai-multi-model-gateway.git
cd ai-multi-model-gateway/gateway
pip install -r requirements.txt
export OPENAI_API_KEY=sk-...
python gateway.py

Use It Like OpenAI

import openai

openai.api_base = "https://csx0574--gateway.modal.run/v1"
openai.api_key = "user-api-key"

response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

Smart Routing

Pass mode in the request to let the gateway choose:

Mode	Behavior
`cost`	Cheapest model that gets the job done
`speed`	Fastest response
`balanced`	Best cost/quality trade-off

Why This Matters

Chinese AI providers (Zhipu GLM, DeepSeek, Minimax) offer surprisingly competitive pricing—sometimes 10x cheaper than OpenAI for comparable quality. But they're scattered and hard to integrate.

This gateway unifies them under one OpenAI-compatible API.

GitHub: https://github.com/csx0574/ai-multi-model-gateway

Forem: csx0574