DEV Community

Cover image for Prototyping AI Agents with GitHub Models (for Free!) 🤖💸
Boopathi
Boopathi

Posted on

3 2 2 2 2

Prototyping AI Agents with GitHub Models (for Free!) 🤖💸

I love when companies roll out generous free tiers. It feels like they’re saying, “Hey, go build your thing — we’ve got your back.” And if you’re a student, between jobs, or just tired of racking up charges for every API call (yep, been there too), free tiers can be a total game-changer.

That’s exactly why GitHub Models stood out to me — it’s like an AI candy shop, completely free to explore, as long as you have a GitHub account.

Here’s what’s on the shelf:

🔮 OpenAI models like gpt-4o, o3-mini

🧠 Research favorites like Phi and LLaMA

🌍 Multimodal models like llama-vision-instruct

📚 Embeddings from Cohere and OpenAI

⚡ Plus providers like Mistral, Jamba, Codestral and more

Oh, and the best part? Many of these models support function calling, making them perfect for agent-style apps.

Now here’s the real kicker:

GitHub Models speak OpenAI-compatible API — which means any Python framework that already works with OpenAI's ChatCompletion API... just works out of the box.

Example 1: Connecting openai SDK to GitHub Models

import openai
import os

client = openai.OpenAI(
  api_key=os.environ["GITHUB_TOKEN"],
  base_url="https://models.inference.ai.azure.com"
)
Enter fullscreen mode Exit fullscreen mode

Now go ahead and use it like you would with OpenAI:

client.chat.completions.create(...)
Enter fullscreen mode Exit fullscreen mode

Simple, clean, no surprises.

Example 2: Running AutoGen with GitHub Models

import autogen_ext.models.openai
import autogen_agentchat.agents
import os

client = autogen_ext.models.openai.OpenAIChatCompletionClient(
  model="gpt-4o",
  api_key=os.environ["GITHUB_TOKEN"],
  base_url="https://models.inference.ai.azure.com"
)

math_teacher = autogen_agentchat.agents.AssistantAgent(
    name="Math_teacher",
    model_client=client,
    system_message="You only teach maths."
)
Enter fullscreen mode Exit fullscreen mode

Just like that, your agent is ready to go.

You can plug GitHub Models into tons of other Python libraries too — LangGraph, PydanticAI, LlamaIndex, you name it.

Go build something fun. Happy tinkering!

Postmark Image

The email service that speaks your language

Whether you code in Ruby, PHP, Python, C#, or Rails, Postmark's robust API libraries make integration a breeze. Plus, bootstrapping your startup? Get 20% off your first three months!

Start free

Top comments (0)

ACI image

ACI.dev: Fully Open-source AI Agent Tool-Use Infra (Composio Alternative)

100% open-source tool-use platform (backend, dev portal, integration library, SDK/MCP) that connects your AI agents to 600+ tools with multi-tenant auth, granular permissions, and access through direct function calling or a unified MCP server.

Check out our GitHub!

👋 Kindness is contagious

If you found this article helpful, please give a ❤️ or share a friendly comment!

Got it