Forem: Anmol Baranwal

AIMock: One Mock Server For Your Entire AI Stack

Anmol Baranwal — Wed, 08 Apr 2026 17:14:01 +0000

TL;DR Our CI was flaky, our tests hit live APIs, and every run burned tokens unnecessarily. So the CopilotKit team built AIMock - a mock server that covers your entire agentic stack so your tests are fast, free, and actually reliable. Let's dive in!

Why we built AIMock

It started with LLM mocking. We built LLMock, open-sourced it, and thought we were done.

But a single agent request in 2026 can touch six or seven services before it returns a response: the LLM, an MCP tool server, a vector database, a reranker, a web search API, a moderation layer, and a sub-agent over A2A.

Most teams mock one of them. The other six are live, non-deterministic, and quietly making your test suite a lie. That is why LLMock became AIMock.

npm install @copilotkit/aimock

AI apps in 2026 call a lot more than just an LLM
Introducing AIMock: mock your entire agentic stack
LLMock: full streaming, tool calls, reasoning across every major provider
MCPMock: a mock MCP server for testing tool integrations
A2AMock: test multi-agent workflows without live agents
VectorMock: deterministic retrieval for RAG pipelines
Services: search, rerank, and moderation
Drift Detection: catch provider changes before your users do
Record and Replay: stop hand-writing fixtures
Chaos Testing: prove your app handles failure
What else is included
AG-UI uses AIMock in production
Migrating to AIMock

AIMock is open source and free.

AI apps in 2026 call a lot more than just an LLM

A realistic agent request looks like this:

User message
→ LLM decides to use a tool
→ Tool call via MCP (file system, database, calendar)
→ RAG retrieval from Pinecone or Qdrant
→ Web search via Tavily
→ Cohere reranker to sort results
→ Back to LLM with full context

Each of these is a live network call in your test environment. Everyone can fail, return something slightly different, or cost you tokens.

We looked at every tool out there. Some handled LLM mocking. Some handled one protocol. None covered the full picture. You would have to stitch together three or four libraries, each with its own config format, and still have gaps.

Here is how AIMock compares to other options out there.

Introducing AIMock: mock your entire agentic stack

It mocks everything your AI app talks to. Here is what is inside:

LLMock : 11 providers, full streaming, tool calls, reasoning models
MCPMock : local MCP server with full JSON-RPC 2.0, session management, tools, resources, prompts
A2AMock : agent card discovery, message routing, SSE streaming between agents
VectorMock : mock vector database for deterministic RAG retrieval
Services : search, rerank, and moderation. The APIs everyone forgets to mock.

On top of that, AIMock does three things no other mocking tool does:

Drift Detection : runs daily against real provider APIs and catches response format changes within 24 hours, before your users do
Record and Replay : proxies real API calls, saves them as fixtures, replays them forever in CI without touching live APIs again
Chaos Testing : inject 500s, malformed JSON, and mid-stream disconnects to prove your app handles failure

Run all of them on one port with a single config file:

{
  "llm": { "fixtures": "./fixtures/llm", "providers": ["openai", "claude", "gemini"] },
  "mcp": { "tools": "./fixtures/mcp/tools.json" },
  "a2a": { "agents": "./fixtures/a2a/agents.json" },
  "vector": { "path": "/vector", "collections": [] }
}

Get started with the AIMock CLI:

npx aimock --config aimock.json --port 4010

Let's go through each one in brief.

LLMock: full streaming, tool calls, reasoning across every major provider

LLMock runs a real HTTP server on a real port, not in-process patching. Any process on the machine can reach it: your Next.js app, agent workers, LangGraph processes, anything that speaks HTTP.

It has native support for 10 providers: OpenAI, Claude, Gemini, Bedrock, Azure, Vertex AI, Ollama, Cohere, OpenRouter, and Anthropic Azure. Reasoning models are supported across all of them.

Any OpenAI-compatible endpoint, such as Mistral, Groq, Together AI, or vLLM, works out of the box, too. Full streaming, tool calls, structured outputs, extended thinking, multi-turn conversations, and WebSocket APIs are all built in.

AIMock handles the translation internally, so one fixture format works across all providers.

Using the programmatic API with vitest, register a fixture and assert on the response. beforeAll starts the mock server once for the suite, afterAll tears it down, and mock.on() registers a fixture that maps a user message to a deterministic response.

import { LLMock } from "@copilotkit/aimock";
import { describe, it, expect, beforeAll, afterAll } from "vitest";

let mock: LLMock;

beforeAll(async () => {
  mock = new LLMock();
  await mock.start();
});

afterAll(async () => {
  await mock.stop();
});

it("non-streaming text response", async () => {
  mock.on({ userMessage: "hello" }, { content: "Hello! How can I help?" });

  const res = await fetch(`${mock.url}/v1/chat/completions`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "gpt-4",
      messages: [{ role: "user", content: "hello" }],
      stream: false,
    }),
  });

  const body = await res.json();
  expect(body.choices[0].message.content).toBe("Hello! How can I help?");
  expect(body.object).toBe("chat.completion");
  expect(body.id).toMatch(/^chatcmpl-/);
});

Point your existing OpenAI client at the mock URL. Nothing else in your code changes. No API keys, no network calls, the same response every time. Full provider docs here.

MCPMock: a mock MCP server for testing tool integrations

MCP is how agents call tools. If your agent uses MCP, every tool call in your test suite hits a live server with real latency and no control over the response.

MCPMock gives you a local server that speaks the full MCP protocol over JSON-RPC 2.0. Your agent connects to it exactly like a real MCP server. You control what comes back.

import { MCPMock } from "@copilotkit/aimock/mcp";

const mcp = new MCPMock();

mcp.addTool({
  name: "search",
  description: "Search the web",
  inputSchema: { type: "object", properties: { query: { type: "string" } } },
});

mcp.onToolCall("search", (args) => {
  const { query } = args as { query: string };
  return `Found 3 results for "${query}"`;
});

const url = await mcp.start();
// point your MCP client at `url`

In a real agent test, your LLM and MCP tool server run together. Mount MCPMock onto LLMock so they share one port:

import { LLMock, MCPMock } from "@copilotkit/aimock";

const llm = new LLMock({ port: 5555 });
const mcp = new MCPMock();

mcp.addTool({ name: "calc", description: "Calculator" });
mcp.onToolCall("calc", (args) => "42");

llm.mount("/mcp", mcp);
await llm.start();
// MCP available at http://127.0.0.1:5555/mcp

It supports:

Tools - register handlers that return strings or rich content
Resources - static files and data your agent can read via URI
Prompts - reusable prompt templates with arguments
Session management - full Mcp-Session-Id lifecycle per the Streamable HTTP spec

Full docs here.

A2AMock: test multi-agent workflows without live agents

A2A (Agent2Agent) is how agents discover and talk to each other. It handles agent cards, message routing, and streaming responses between agents.

Testing a multi-agent system is hard when every agent is live. One agent goes down, your whole test suite breaks.

A2AMock gives you a local A2A server with full agent card discovery, message routing, task management, and SSE streaming. Register your agents, define how they respond, and test your multi-agent workflows end-to-end without anything actually running.

import { A2AMock } from "@copilotkit/aimock/a2a";

const a2a = new A2AMock();

a2a.registerAgent({
  name: "translator",
  description: "Translates text between languages",
  skills: [{ id: "translate", name: "Translate" }],
});

a2a.onMessage("translator", "translate", [{ text: "Translated text" }]);

const url = await a2a.start();
// Agent card at: ${url}/.well-known/agent-card.json
// JSON-RPC at: ${url}/

Here is the pattern for long-running tasks with incremental updates:

a2a.onStreamingTask("agent", "long-task", [
  { type: "status", state: "TASK_STATE_WORKING" },
  { type: "artifact", parts: [{ text: "partial result" }], name: "output" },
  { type: "artifact", parts: [{ text: "final result" }], lastChunk: true, name: "output" },
], 50); // 50ms delay between events

For task management, agent cards, and JSON-RPC methods, see the full docs here.

VectorMock: deterministic retrieval for RAG pipelines

If your app does retrieval-augmented generation, your tests depend on what is in your vector database right now. Your dev index is messy, your staging index does not match prod, and results shift every time someone upserts new vectors.

VectorMock is a mock vector database server that supports Pinecone, Qdrant, and ChromaDB API formats with collection management, upsert, query, and delete operations.

import { VectorMock } from "@copilotkit/aimock/vector";

const vector = new VectorMock();

vector.addCollection("docs", { dimension: 1536 });
vector.onQuery("docs", [
  { id: "doc-1", score: 0.95, metadata: { title: "Getting Started" } },
  { id: "doc-2", score: 0.87, metadata: { title: "API Reference" } },
]);

const url = await vector.start();
// point your vector DB client at `url`

If you need results to vary based on the query, use a dynamic handler instead:

vector.onQuery("docs", (query) => {
  const topK = query.topK ?? 10;
  return Array.from({ length: topK }, (_, i) => ({
    id: `result-${i}`,
    score: 1 - i * 0.1,
  }));
});

Here are compatible endpoints for Pinecone, Qdrant, and ChromaDB APIs.

Consistent retrieval results. Every test run. Full docs here.

Services: search, rerank, and moderation

These are the APIs most people forget to mock, and the ones that quietly make your test suite non-deterministic.

Services within AIMock provides built-in mocks for web search, reranking, and content moderation. Register fixture patterns on your LLMock instance and requests are matched by query/input text. No separate server needed.

Tavily search - mock web search results by query pattern at POST /search
Cohere reranking - mock reranked document lists at POST /v2/rerank
OpenAI moderation - mock moderation decisions at POST /v1/moderations. Unmatched requests return unflagged by default.

import { LLMock } from "@copilotkit/aimock";

const mock = new LLMock();

// String pattern — case-insensitive substring match
mock.onSearch("weather", [
  { title: "Weather Report", url: "https://example.com/weather", content: "Sunny today" },
]);

// RegExp pattern
mock.onSearch(/stock\s+price/i, [
  { title: "ACME Stock", url: "https://example.com/stocks", content: "$42.00", score: 0.95 },
]);

// Catch-all — empty results for unmatched queries
mock.onSearch(/.*/, []);

mock.onRerank("machine learning", [
  { index: 0, relevance_score: 0.99 },
  { index: 2, relevance_score: 0.85 },
]);

mock.onModerate("violent", {
  flagged: true,
  categories: { violence: true, hate: false },
  category_scores: { violence: 0.95, hate: 0.01 },
});

If you just need catch-all responses to stop live requests, enable all three in config:

{
  "services": {
    "search": true,
    "rerank": true,
    "moderate": true
  }
}

String patterns use case-insensitive substring matching. RegExp patterns do full regex testing. First match wins. All service requests are recorded in the journal, so you can inspect exactly what got called. Full docs here.

Drift Detection: catch provider changes before your users do

This is one of the things no other mocking tool does. A mock that does not match reality is worse than no mock.

Here is the problem: mocks are a snapshot of how an API behaved when you wrote the fixture. OpenAI adds a field. Claude changes a default. Gemini tweaks its streaming format. Your mocks still pass. CI is green. Then your app breaks in production.

Drift Detection runs a three-way comparison in CI every day:

SDK types - what TypeScript type definitions say the shape should be
Real API responses - actual live requests to OpenAI, Anthropic, Gemini
AIMock output - what the mock returns for the same request

If any of those three disagree, you know within 24 hours. Not when a user files a bug.

Here is what the output looks like when drift is detected:

$ pnpm test:drift

[critical] AIMOCK DRIFT — field in SDK + real API but missing from mock
Path:    choices[].message.refusal
SDK:     null    Real: null    Mock: <absent>

[critical] TYPE MISMATCH — real API and mock disagree on type
Path:    content[].input
SDK:     object    Real: object    Mock: string

[warning] PROVIDER ADDED FIELD — in real API but not in SDK or mock
Path:    choices[].message.annotations
SDK:     <absent>    Real: array    Mock: <absent>

✓ 2 critical (test fails) · 1 warning (logged) · detected before any user reported it

Severity levels:

Critical : mock mismatches the real API, test fails immediately
Warning : provider added a new field that neither the SDK nor the mock knows about yet, logged as early warning
Ok : all three sources agree, nothing to do

Here is how the three-way comparison works under the hood:

import { extractShape, triangulate, formatDriftReport, shouldFail } from "./schema";

// 1. Get the SDK shape (what TypeScript says)
const sdkShape = openaiChatCompletionShape();

// 2. Call the real API and the mock in parallel
const [realRes, mockRes] = await Promise.all([
  openaiChatNonStreaming(config, [{ role: "user", content: "Say hello" }]),
  httpPost(`${instance.url}/v1/chat/completions`, { /* ... */ }),
]);

// 3. Extract response shapes
const realShape = extractShape(realRes.body);
const mockShape = extractShape(JSON.parse(mockRes.body));

// 4. Three-way comparison
const diffs = triangulate(sdkShape, realShape, mockShape);
const report = formatDriftReport("OpenAI Chat (non-streaming text)", diffs);

// 5. Critical diffs fail the test
if (shouldFail(diffs)) {
  expect.soft([], report).toEqual(
    diffs.filter(d => d.severity === "critical")
  );
}

Run it yourself against live endpoints:

# Run drift checks against live endpoints
pnpm vitest --config vitest.config.drift.ts

MSW, VidaiMock, Mokksy - none of them do this. Your mocks should never silently go stale. Full docs here.

Record and Replay: stop hand-writing fixtures

Writing fixtures by hand is fine for simple cases. It gets painful fast when you are dealing with multi-turn agent conversations with tool calls, streaming responses, and branching logic. MSW, VidaiMock, mock-llm, piyook - none of the LLM mocking tools solve this.

Record and Replay works like a VCR. Here is exactly what happens:

Client sends a request to AIMock
AIMock attempts fixture matching as usual
On miss: the request is forwarded to the configured upstream provider
The upstream response is relayed back to the client immediately
The response is collapsed if streaming and saved as a fixture to disk and memory
Subsequent identical requests match the newly recorded fixture

Quick start using CLI:

$ npx aimock --fixtures ./fixtures \
  --record \
  --provider-openai https://api.openai.com \
  --provider-anthropic https://api.anthropic.com

Recorded fixtures are saved automatically and look like this:

{
  "fixtures": [
    {
      "match": { "userMessage": "What is the weather?" },
      "response": { "content": "I don't have real-time weather data..." }
    }
  ]
}

AIMock handles stream collapsing automatically across six formats: OpenAI SSE, Anthropic SSE, Gemini SSE, Cohere SSE, Ollama NDJSON, and Bedrock EventStream. Auth headers are forwarded to the upstream provider but never saved in fixtures.

It also supports a programmatic API with enableRecording() and requestTransform for normalizing dynamic data like timestamps.

In CI, add --strict so that any unmatched request returns a 503 and fails immediately, rather than slipping through silently. Without it, missing fixtures return 404 responses, and your test suite may never tell you that a live API call was attempted.

Full docs here.

Chaos Testing: prove your app handles failure

Your app will eventually get a 500 from OpenAI. A malformed JSON response from a tool. A mid-stream disconnect from a vector database. The question is whether you find out in testing or in production.

Chaos Testing lets you inject failures at configurable probabilities at three levels: server-wide, per fixture, or per individual request.

Three failure modes:

drop - returns HTTP 500 with {"error":{"message":"Chaos: request dropped","code":"chaos_drop"}}
malformed - returns HTTP 200 with invalid JSON
disconnect - destroys the TCP connection immediately with no response

Server-level chaos applies to all requests:

import { LLMock } from "@copilotkit/aimock";

const mock = new LLMock();
mock.setChaos({
  dropRate: 0.1,        // 10% of requests return 500
  malformedRate: 0.05,  // 5% return broken JSON
  disconnectRate: 0.02, // 2% drop the connection
});

// remove all chaos later
mock.clearChaos();

Fixture-level chaos targets specific responses only:

{
  "fixtures": [
    {
      "match": { "userMessage": "unstable" },
      "response": { "content": "This might fail!" },
      "chaos": {
        "dropRate": 0.3,
        "malformedRate": 0.2,
        "disconnectRate": 0.1
      }
    }
  ]
}

Per-request headers override everything else - useful for forcing a specific failure in a single test:

// Force 100% disconnect on this specific request
await fetch(`${mock.url}/v1/chat/completions`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-aimock-chaos-disconnect": "1.0",
  },
  body: JSON.stringify({ model: "gpt-4", messages: [{ role: "user", content: "hello" }] }),
});

All chaos events are tracked in the journal with a chaosAction field and counted in Prometheus metrics. Full docs here.

What else is included

Beyond the core modules, AIMock offers a lot more worth knowing.

Embeddings - POST /v1/embeddings is fully supported. Return explicit vectors via fixtures or let AIMock generate them deterministically from a hash of the input text. Same input always produces the same vector, default dimensions are 1536.

WebSocket APIs - OpenAI Realtime, OpenAI Responses over WebSocket, and Gemini Live are all supported using raw RFC 6455 framing. If your app uses voice or real-time streaming agents, these are covered.

Sequential Responses - return different responses for the same prompt on each successive call. Useful for testing retry logic and multi-turn workflows where the same message should behave differently over time.

[
  { "match": { "userMessage": "retry", "sequenceIndex": 0 }, "response": { "content": "First attempt" } },
  { "match": { "userMessage": "retry", "sequenceIndex": 1 }, "response": { "content": "Second attempt" } },
  { "match": { "userMessage": "retry" }, "response": { "content": "Fallback" } }
]

Streaming Physics - configure ttft (time to first token), tps (tokens per second), and jitter to simulate realistic timing. Pre-built profiles cover fast models, reasoning models, and overloaded systems.

Prometheus Metrics - request counts, latency histograms, and current fixture count at /metrics. Enable with --metrics flag.

Docker and Helm - the official Docker image is available at ghcr.io/copilotkit/aimock (GitHub Container Registry) for CI/CD. It runs as a plain HTTP server, so any language can use it. No Node.js needed on the test runner side.

AG-UI uses AIMock in production

AG-UI is the open protocol that connects AI agents to frontend applications - adopted by LangGraph, CrewAI, Mastra, Google ADK, AWS Bedrock AgentCore, and more.

It uses AIMock for its end-to-end test suite, verifying AI agent behavior across LLM providers with fixture-driven responses. AG-UI is a protocol used in production. If you want to see what a real AIMock setup looks like at scale, that is a good place to start.

Migrating to AIMock

The docs include step-by-step migration guides for MSW, VidaiMock, mock-llm, Python mocks and Mokksy. You will find a side-by-side comparison and a breakdown of what you gain and keep.

If you are on MSW, you do not have to replace everything: you can keep MSW for general REST and GraphQL mocking and use AIMock only for AI endpoints.

If you are already using @copilotkit/llmock, the upgrade is a find-and-replace:

pnpm remove @copilotkit/llmock
pnpm add @copilotkit/aimock

The LLMock class, all fixture formats, and the programmatic API are unchanged. Your existing tests will work as-is.

Get started

npm install @copilotkit/aimock

Docs: aimock.copilotkit.dev
GitHub: github.com/CopilotKit/aimock
npm: @copilotkit/aimock
CLI: AIMock CLI

Your test suite should be as complete as your stack. That is what AIMock is for.

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Google Maps for Codebases: Paste a GitHub URL, Ask Anything

Anmol Baranwal — Mon, 06 Apr 2026 13:09:38 +0000

Navigating a large codebase for the first time is painful. You clone the repo, realize there are 300 files, and have no idea where anything lives.

You can ask an AI assistant, but it burns through context fast and you never know if it hallucinated a file path that does not even exist.

Codebase Navigator solves this. Paste a URL, ask anything in plain English, and watch a real dependency graph built from actual import statements appear in real time.

It is built using CopilotKit, Zenflow, GitHub API and React Flow. You can run it completely free using Ollama locally.

In this blog, we will go through the architecture, the key patterns and how everything works end-to-end.

What are we building?

Codebase Navigator lets you paste any public GitHub repo URL and ask questions about it in plain English. Instead of getting a wall of text back, you get four panels that all update at once.

Panel	What it does
Graph Canvas	Live dependency graph built from real import statements
Code Viewer	File content fetched live from GitHub, relevant lines highlighted
Repo Explorer	Full file tree, click any file to open it
Chat	Follow-up questions and plain English explanations

The graph morphs to show every relevant file connected by real imports, the code viewer opens the file, and the chat explains what each piece does. No context switching.

You can run it completely free using Ollama locally without any API key.

Here is the full request → response flow of what happens when you ask a question:

User types "how does auth work?"
     ↓
CopilotChat → POST /api/copilotkit
     ↓
LLM receives repo context (file paths, current selection, system rules)
     ↓
LLM calls analyzeRepository tool
     ↓
Tool fetches relevant files via /api/github/file
     ↓
Extracts import/require statements → resolves paths → builds dependency graph
     ↓
Zustand store updates
     ↓
All four panels re-render live: graph, file tree, code viewer, chat response

Tech Stack & Tools

At a high level, this project is built by combining:

Next.js 16 - frontend and API routes
CopilotKit - agent-UI state sync and chat interface. provides built-in hooks & components like useAgentContext, useFrontendTool, CopilotChat
Zenflow - workflow tool that planned, tested and orchestrated the build
React Flow & dagre - interactive dependency graphs with automatic layout
Octokit - GitHub API proxy for fetching repo trees and file contents
Zustand - shared state across all four panels
Tailwind CSS - styling
Ollama / OpenAI - local or cloud LLM backend switchable from the UI

The CopilotKit runtime is self-hosted inside a Next.js API route, which lets you plug into any OpenAI-compatible backend, including Ollama, for completely free local inference.

It connects your UI, agents, and tools into a single interaction loop.

What is Zenflow?

Zenflow is an AI development tool that treats building software as a structured engineering process rather than just autocompleting code.

This project was planned, built, tested, reviewed and deployed using Zenflow (Zencoder's workflow engine) in a single session.

It ran a six-phase process from scratch:

Architecture and spec
Scaffolding and foundation
Data layer (GitHub API integration)
AI layer (CopilotKit actions and context)
Visual layer (React Flow graphs)
Final assembly and wiring

Each phase was verified with lint, type checks and tests before moving to the next. After the build, it performed a code review, found 14 issues and fixed 11 systematically:

API keys exposed in headers → moved to httpOnly cookies
Fake star-shaped graphs → replaced with real import-based dependency resolution
Duplicate file-fetching logic → consolidated into a single cached utility

The .zenflow/ folder in the repo contains the task plan, spec, and report files it generated along the way.

That's how we built it with Zenflow. Here's the architecture that came out of it.

Architecture & Project Structure

The app is split into three layers: the browser handles all UI and AI tool logic, Next.js API routes act as a secure proxy to external services, and GitHub API, plus the LLM backend sits at the bottom. Nothing in the browser talks to GitHub or the LLM directly.

Below is a high-level overview of all the layers and how they are organized.

┌─────────────────────────────────────────────────────────┐
│                        BROWSER                          │
│                                                         │
│  ┌─────────┐  ┌──────────┐  ┌──────────┐  ┌────────┐    │
│  │  Chat   │  │  Graph   │  │  Code    │  │  File  │    │
│  │  Panel  │  │  Canvas  │  │  Viewer  │  │  Tree  │    │
│  └────┬────┘  └────┬─────┘  └────┬─────┘  └───┬────┘    │
│       └────────────┴─────────────┴────────────┘         │
│                          │                              │
│                   ┌──────▼──────┐                       │
│                   │   Zustand   │  ← shared state       │
│                   └──────┬──────┘                       │
│                          │                              │
│              ┌───────────▼────────────┐                 │
│              │      CopilotKit        │                 │
│              │   frontend tools       │                 │
│              │  analyzeRepository     │                 │
│              │  fetchFileContent      │                 │
│              │  highlightCode         │                 │
│              └───────────┬────────────┘                 │
└──────────────────────────┼──────────────────────────────┘
                           │
┌──────────────────────────▼──────────────────────────────┐
│                    NEXT.JS API ROUTES                   │
│                                                         │
│   /api/copilotkit     /api/github/*     /api/settings   │
│   (LLM runtime)       (proxy)          (httpOnly cookie)│
└──────────┬────────────────┬─────────────────────────────┘
           │                │
     ┌─────▼─────┐   ┌──────▼──────┐
     │  Ollama   │   │  GitHub API │
     │  / OpenAI │   │  (Octokit)  │
     └───────────┘   └─────────────┘

Here is a simplified view of how the project is structured.

src/
├── app/api/
│   ├── copilotkit/route.ts    → CopilotKit runtime endpoint
│   ├── github/
│   │   ├── tree/route.ts      → fetch repo tree
│   │   ├── file/route.ts      → fetch + base64-decode file
│   │   └── search/route.ts    → code search
│   └── settings/route.ts      → LLM config (httpOnly cookie)
├── components/
│   ├── panels/     → 5 UI panels (chat, graph, code, analysis, repo)
│   └── flow/       → custom React Flow node types
├── hooks/          → AI tool registration, agent context sync, repo loading
├── lib/            → import extraction, dagre layout, Octokit client
├── store/          → Zustand (AppState + SettingsState)
└── .zenflow/       → Zenflow task plan, spec and report

Let's get into how everything works behind the scenes. This will help you understand the key patterns.

How everything works under the hood

Now that you have the big picture, let's go through each piece in detail.

A lot is happening under the surface, so we will break it down into ten parts, starting from when you first load a repo all the way to how state flows across the panels.

1. Loading a repository

When you paste a GitHub URL and click Explore, useRepository.loadRepository() fires (from useRepository.ts hook).

It calls /api/github/tree on the server, which uses Octokit to:

Parse the repo URL into { owner, repo }
Fetch the default branch
Get the full recursive file tree from the GitHub Trees API
Convert the flat array into a nested TreeNode structure

Here is what the src/api/github/tree/route.ts route looks like:

export async function GET(request: NextRequest) {
  const { owner, repo } = parseRepoUrl(repoUrl);
  const resolvedBranch = branch || (await getDefaultBranch(owner, repo));
  const tree = await getRepoTree(owner, repo, resolvedBranch);

  return NextResponse.json({ owner, repo, branch: resolvedBranch, tree });
}

That nested tree powers the sidebar file explorer and becomes the starting point for everything else.

All GitHub calls are proxied through Next.js API routes. The browser never talks to GitHub directly to keep tokens secure.

2. From architecture view to dependency graph

This is the visual moment that makes the app feel alive. The graph you see on load (the architecture view) and the graph you see after asking a question (the dependency graph) are two completely different things, built two different ways.

Both live in src/lib/analyzer.ts - buildOverviewGraph for the architecture view and buildDependencyNodes for the dependency graph.

When the repo loads, useRepository.ts calls buildOverviewGraph right after the tree response comes back and pushes the result straight to the graph:

const overview = buildOverviewGraph(data.tree);
setVisualization(overview.nodes, overview.edges, "architecture");

buildOverviewGraph never fetches any files. It just walks the tree structure and builds a folder map: root node at the top, top-level directories as children, one level of sub-folders below that:

export function buildOverviewGraph(tree: TreeNode) {
  const rootId = `node-${nodeId++}`;
  nodes.push({ id: rootId, type: "module", label: tree.path || "root" });

  const topDirs = (tree.children || []).filter((c) => c.type === "directory");

  for (const dir of topDirs) {
    const fileCount = flattenTree(dir).length;
    nodes.push({ id, label: `${dir.name} (${fileCount})`, type: categorizeDirType(dir.path) });
    edges.push({ source: rootId, target: id, type: "flow" });
  }
}

The edges here mean nothing beyond "this folder is inside that folder."

Once you ask a question, useCopilotActions.ts fetches the actual file content and calls buildDependencyNodes instead:

const graph = buildDependencyNodes(fileData);
setVisualization(graph.nodes, graph.edges, "dependency");

buildDependencyNodes creates one node per file and one edge per real import statement:

export function buildDependencyNodes(files: { path: string; imports: string[] }[]) {
  const nodes = files.map((file) => ({
    id: file.path,
    type: "file",
    label: file.path.split("/").pop() || file.path,
    metadata: { fullPath: file.path },
  }));

  for (const file of files) {
    for (const imp of file.imports) {
      const resolved = resolveImportPath(imp, file.path, filePathSet);
      if (resolved) {
        edges.push({ source: file.path, target: resolved, type: "import" });
      }
    }
  }

  return { nodes, edges };
}

Same setVisualization call, different data. React Flow re-renders and the graph morphs from a folder map into a real dependency graph focused on exactly the files relevant to your question.

3. The CopilotKit runtime

The /api/copilotkit route is where LLM requests actually land. The API key and provider config are stored in an httpOnly cookie, meaning they live on the server and never get sent to the browser.

The route reads that config, creates an OpenAI-compatible client, and hands the request to the CopilotKit runtime:

export const POST = async (req: NextRequest) => {
  const { baseURL, apiKey, model } = await getLLMConfig();

  const openai = new OpenAI({ baseURL, apiKey });
  const serviceAdapter = new OpenAIAdapter({ openai, model });
  const runtime = new CopilotRuntime();

  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

The defaults point to Ollama on localhost:11434/v1 with qwen2.5.

Because the OpenAI SDK accepts a custom baseURL, swapping to OpenAI, Groq or any other provider is just a config change with no code changes needed.

4. Feeding the LLM context

Before the LLM can answer anything useful, it needs to know what repo is loaded and what files exist. That is the job of the useCopilotContext.ts hook.

It calls useAgentContext, a CopilotKit hook that attaches structured data to every message you send. You can call it multiple times to attach different pieces of context.

Here it is called twice: once for the file list and once for the system instructions that tell the LLM how to behave:

useAgentContext({
  description: "File paths in the repository (max 500), one per line",
  value: fileList,  // flattened from the TreeNode structure
});

useAgentContext({
  description: "System instructions",
  value: `You are a Codebase Navigator assistant. You MUST use tool calls to answer questions.
CRITICAL RULES:
1. For ANY question about the repository call the "analyzeRepository" tool.
2. To show a file, call "fetchFileContent" with the exact file path.
3. NEVER respond with only text. ALWAYS call a tool first.
4. Use ONLY file paths from the file list above.`
});

The file list is capped at 500 paths to stay within token limits.

Because this hook re-runs every time the Zustand store changes, the LLM always sees the current repo, selected file and analysis state rather than a stale snapshot from when the page loaded.

Note: If you are running a local model with a large context window, you can raise this limit in useCopilotContext.ts.

5. The four frontend tools

This is the core of how the app works. Instead of just replying with text, the LLM can call tools. Each tool has a name, a set of parameters it accepts and a handler, which is just a function that runs when the LLM calls it.

CopilotKit lets you register these tools directly in the browser using the useFrontendTool hook, which is defined in useCopilotActions.ts.

When a tool's handler runs, it updates Zustand state directly, which is why all four panels, the graph, code viewer, analysis panel, and chat, react at the same time without any extra wiring.

analyzeRepository is the main tool the LLM calls for almost every question. Here is what it does step by step:

useFrontendTool({
  name: "analyzeRepository",
  parameters: z.object({
    query: z.string(),
    explanation: z.string(),
  }),
  handler: async ({ query, explanation }) => {
    // 1. Find relevant files using keyword matching
    const matchedPaths = findFilesByQuery(repo.tree, query);

    // 2. Cap at 15 files to keep things fast
    const capped = matchedPaths.slice(0, 15);

    // 3. Fetch each file content (cached)
    const fileData = await Promise.all(
      capped.map(async (p) => ({
        path: p,
        imports: extractImports(await fetchFile(owner, repo, p, branch)),
      }))
    );

    // 4. Build real dependency graph from actual import statements
    const graph = buildDependencyNodes(fileData);

    // 5. Categorize each node by file type
    graph.nodes.forEach(node => {
      node.type = categorizeFileType(node.metadata.fullPath);
    });

    // 6. Push to Zustand - all four panels react
    setAnalysisResult({ explanation, relevantFiles, flowDiagram: graph });
    setVisualization(graph.nodes, graph.edges, "dependency");
  },
});

fetchFileContent opens a specific file in the code viewer. The LLM calls this when you ask it to show you a file, it just fetches the content and calls setCodeViewer:

useFrontendTool({
  name: "fetchFileContent",
  parameters: z.object({
    filePath: z.string(),
  }),
  handler: async ({ filePath }) => {
    const content = await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, filePath, repo.repoInfo.branch);
    setCodeViewer(filePath, content);
  },
});

generateFlowDiagram is similar to analyzeRepository but more focused. Instead of searching the whole repo, you give it a specific list of file paths and it builds a graph for just those files. Useful when you want to visualize a slice of the codebase like just the API layer:

useFrontendTool({
  name: "generateFlowDiagram",
  parameters: z.object({
    files: z.array(z.string()),
    diagramType: z.enum(["dependency", "flow", "architecture"]),
  }),
  handler: async ({ files, diagramType }) => {
    const fileData = await Promise.all(files.slice(0, 20).map(async (f) => ({
      path: f,
      imports: extractImports(await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, f, repo.repoInfo.branch)),
    })));
    const graph = buildDependencyNodes(fileData);
    setVisualization(graph.nodes, graph.edges, diagramType);
  },
});

highlightCode fetches a file and highlights specific line numbers with an explanation. The LLM uses this when it wants to point at exactly where something happens in the code:

useFrontendTool({
  name: "highlightCode",
  parameters: z.object({
    filePath: z.string(),
    lines: z.array(z.number()),
    explanation: z.string(),
  }),
  handler: async ({ filePath, lines, explanation }) => {
    const content = await fetchFile(repo.repoInfo.owner, repo.repoInfo.repo, filePath, repo.repoInfo.branch);
    setCodeViewer(filePath, content, lines, explanation);
  },
});

Notice that none of these tools returns text to the chat. They all just update the state. The LLM is not describing what it found, it is directly controlling what you see on screen.

For instance, I asked about the architecture of v2 packages.

6. Finding relevant files

When analyzeRepository gets a query like "how does authentication work?", it needs to figure out which files to actually look at. That is handled by findFilesByQuery in src/lib/analyzer.ts.

It checks if the query matches one of six predefined categories, each with a set of keywords and file path patterns:

const ANALYSIS_CATEGORIES = [
  {
    name: "authentication",
    keywords: ["auth", "login", "logout", "token", "jwt", "session", "oauth"],
    filePatterns: [/auth/i, /login/i, /session/i, /oauth/i, /jwt/i],
  },
  {
    name: "database",
    keywords: ["database", "db", "model", "schema", "prisma", "mongoose"],
    filePatterns: [/model/i, /schema/i, /migration/i, /prisma/i, /db/i],
  },
  // ... api, configuration, testing, styling
]

If the query matches a category, it filters all file paths using that category's regex patterns.

If nothing matches, it falls back to splitting the query into individual terms and checking if any file path contains them. So even a vague query like "where is the config" will still find files with "config" in the name.

This is intentionally simple. No embeddings, no vector search, just regex on file names. It works well because file names in well-structured projects are usually descriptive enough that matching on them gets you the right files.

7. File caching

Every time a tool fetches a file, it goes through src/lib/fetch-file.ts which maintains an in-memory cache with a 5-minute TTL:

const cache = new Map<string, { content: string; timestamp: number }>();
const CACHE_TTL = 5 * 60 * 1000; // 5 minutes

export async function fetchFile(owner, repo, path, ref) {
  const key = `${owner}/${repo}/${ref}/${path}`;
  const cached = cache.get(key);

  if (cached && Date.now() - cached.timestamp < CACHE_TTL) {
    return cached.content; // serve from cache
  }

  const res = await fetch(`/api/github/file?repo=...&path=...&ref=...`);
  const data = await res.json();
  cache.set(key, { content: data.content, timestamp: Date.now() });

  // LRU eviction: if over 200 items, drop the oldest 50
  if (cache.size > 200) {
    const oldest = [...cache.entries()].sort((a, b) => a[1].timestamp - b[1].timestamp);
    for (let i = 0; i < 50; i++) cache.delete(oldest[i][0]);
  }

  return data.content;
}

This matters because analyzeRepository fetches up to 15 files at once. A follow-up question about the same files hits the cache instead of making 15 more API requests.

Without a GitHub token, you are limited to 60 requests per hour, so caching is what makes repeated questions fast and free.

The cache also evicts the oldest entries once it crosses 200 items, so it never grows unbounded in long sessions.

8. Parsing imports and resolving paths

Once analyzeRepository has the relevant file paths, it fetches each file and parses it for import statements. Both functions that handle this live in src/lib/analyzer.ts.

extractImports runs on the raw file content and pulls out every import, handling both ES6 and CommonJS syntax:

export function extractImports(content: string): string[] {
  const imports: string[] = [];

  // ES6: import X from 'y' and import 'y'
  const esImports = content.matchAll(
    /(?:import\s+.*?\s+from\s+['"](.+?)['"]|import\s+['"](.+?)['"])/g
  );
  for (const match of esImports) imports.push(match[1] || match[2]);

  // CommonJS: require('y')
  const requires = content.matchAll(/require\s*\(\s*['"](.+?)['"]\s*\)/g);
  for (const match of requires) imports.push(match[1]);

  return imports;
}

But raw import strings like "./utils" or "@/lib/github" are not file paths yet.

buildDependencyNodes calls resolveImportPath on each one to turn them into actual paths that exist in the repo:

function resolveImportPath(importPath, fromFile, filePathSet) {
  // Skip npm packages (no ./ or @/)
  if (!importPath.startsWith(".") && !importPath.startsWith("@/")) return null;

  // Resolve @/ alias to src/
  if (importPath.startsWith("@/")) {
    resolved = "src/" + importPath.slice(2);
  }

  // Try with common extensions: .ts, .tsx, .js, /index.ts, etc.
  const extensions = ["", ".ts", ".tsx", ".js", ".jsx", "/index.ts", "/index.tsx"];
  for (const ext of extensions) {
    if (filePathSet.has(resolved + ext)) return resolved + ext;
  }
  return null;
}

npm packages like react or zod are skipped immediately since they do not start with ./ or @/. For local imports, it tries common extensions like .ts, .tsx, and /index.ts, so it handles both direct file imports and folders that export through an index file.

If the resolved path does not exist in the repo, it returns null and no edge is created. That is what keeps the graph honest.

9. Graph layout with Dagre

Once the nodes and edges are built, they need actual positions on screen before React Flow can render them. That is what applyDagreLayout does in src/lib/graph-layout.ts.

It uses Dagre, a JavaScript library that automatically calculates x and y coordinates for every node in a directed graph so nothing overlaps, all on the client side with no server needed:

export function applyDagreLayout(nodes, edges, options = {}) {
  const g = new dagre.graphlib.Graph();
  g.setGraph({ rankdir: "TB", ranksep: 80, nodesep: 40 });

  // Tell dagre the size of each node
  for (const node of nodes) {
    g.setNode(node.id, { width: 200, height: 60 });
  }
  for (const edge of edges) {
    g.setEdge(edge.source, edge.target);
  }

  dagre.layout(g); // dagre calculates x,y for every node

  return nodes.map((node) => {
    const pos = g.node(node.id);
    return { ...node, position: { x: pos.x - 100, y: pos.y - 30 } };
  });
}

The direction can be toggled between top-to-bottom (TB) and left-to-right (LR) from the UI.

Each node type maps to a React Flow component based on the file type categorizeFileType assigned earlier:

module / service → ModuleNode (indigo)
function → FunctionNode (teal)
file → FileNode (gray)

10. State management with Zustand

All four panels stay in sync because they all read from the same Zustand store in src/store/index.ts. The store has four slices, one for each major concern:

interface AppState {
  repo: {
    repoInfo: RepoInfo | null;   // owner, repo, branch
    tree: TreeNode | null;        // full file tree
    selectedFile: string | null;  // active file path
    loading: boolean;
    error: string | null;
  };
  analysis: {
    result: AnalysisResult | null; // explanation + relevant files + graph
    loading: boolean;
    error: string | null;
  };
  visualization: {
    nodes: FlowNode[];
    edges: FlowEdge[];
    graphType: "dependency" | "flow" | "architecture" | null;
  };
  codeViewer: {
    filePath: string | null;
    content: string | null;
    highlightedLines: number[];
    explanation: string | null;
  };
}

Each panel subscribes to only the slice it needs using a selector like useAppStore((s) => s.visualization).

So when a tool calls setVisualization, only the graph panel re-renders. When setCodeViewer fires, only the code viewer updates. Nothing else moves.

Settings like LLM provider, API key and model live in a separate SettingsState store. It syncs to two places:

localStorage so the browser remembers your config across reloads
an httpOnly cookie so the server-side /api/copilotkit route can read them without the API key ever appearing in request headers

This is how everything works behind the scenes. It's time to run it locally.

Running it locally

Clone the GitHub repository and install the dependencies.

git clone <repo-url>
cd codebase-navigator
npm install
npm run dev

Open http://localhost:3000 to view it live in the browser. Here is how to set up each provider.

Using OpenAI

You will need an OpenAI API Key. Then just click the Settings icon in the top right, switch the provider to OpenAI, paste your API key and pick a model like gpt-5.2. That's it.

Using Ollama (free, runs locally)

You can download it from ollama.com or install it via these commands based on the operating system you use.

brew install ollama #macOS
curl -fsSL https://ollama.com/install.sh | sh #linux

Then pull the model and start the server:

ollama serve
ollama pull qwen2.5 # in another terminal

The app points to Ollama by default so nothing else needs to change. Just open http://localhost:3000 and it works.

For GitHub, public repos work without a token (the only catch is that it's limited to 60 requests per hour). Add a GITHUB_TOKEN to a .env.local file for higher limits:

GITHUB_TOKEN=your_token_here

I tried it with the CopilotKit GitHub repo and it immediately maps out the entire repo structure.

I asked it, "tell me about architecture" and it gave me:

a graph of every relevant file connected by real imports
a plain english breakdown of what each file does
raw code fetched live from GitHub
a detailed chat response I could keep following up on

I tried difficult and vague questions and it handled all of them.

The bigger picture

This project started with wiring three ideas together and seeing what happened.

GitHub is the data source, not just a host. Every file the AI references gets fetched live, parsed, and reasoned about on the spot. No hallucination, no guessing from memory.

CopilotKit changes what the AI can actually do in the browser. Instead of replying with text, it calls tools that update the state directly. The graph changes. The code viewer opens. The AI is not describing what it found, it is showing you.

The dependency edges are real. A lot of tools draw graphs that look impressive but mean nothing. Every edge here exists because one file has an actual import statement pointing to another. That is it.

Wire those three together and you get something that actually understands the codebase instead of just searching through it.

There is a lot you could extend from here: PR diff exploration, multi-repo comparison, security audit mode. But as a starting point for understanding any codebase, it already does exactly what it needs to.

Let me know what you think in the comments!

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Deploying AG-UI Agents to Production with Amazon Bedrock AgentCore

Anmol Baranwal — Tue, 24 Mar 2026 17:00:06 +0000

The Agent-User Interaction Protocol (AG-UI) standardizes how AI agents communicate with user applications.

But deploying those agents means the same manual work: configuring SSE endpoints, writing auth middleware, managing session isolation, and setting up auto-scaling. None of that is your agent logic. All of it had to be built before you could ship.

Amazon recently announced native AG-UI support in Amazon Bedrock AgentCore Runtime across 14 AWS regions. You bring your agent code in a container, set one protocol flag, and AgentCore handles the rest.

So I deployed Strands Agents to learn how it actually works end-to-end. This post covers everything I picked up, including what AgentCore is and the full deployment flow from a local server to a live UI.

Amazon Bedrock AgentCore Runtime

Amazon Bedrock AgentCore is an agentic platform for building, deploying, and operating agents securely at scale. AgentCore Runtime is a fully managed hosting environment.

You bring your agent code in a container, and AgentCore checks who's allowed in (auth), keeps each user's session separate (Session isolation), adds capacity when traffic spikes (auto-scaling) and logs everything to CloudWatch (observability). You don't write any of this code.

It's also framework-agnostic: agents built with Strands, LangGraph, CrewAI, or any other framework can deploy to AgentCore. Think of it as the production layer between your agent and your users.

It supports four protocols:

HTTP: default agent invocation
MCP: agent ↔ tools
A2A: agent ↔ agent
AG-UI: agent ↔ user (new)

You can read more about the differences in message format, discovery and authentication in their docs.

You configure with --protocol AGUI and the protocol layer is taken care of. Your job is just the agent logic.

AG-UI: the interaction protocol

AG-UI (Agent-User Interaction Protocol) is an open, event-based protocol that standardises how AI agents communicate with user-facing applications.

Instead of a custom WebSocket or polling layer, AG-UI defines a shared event stream between your agent backend and your frontend UI. It covers:

Streaming text responses as they generate
Tool call lifecycle (start to result)
State updates and progress
User interactions sent back into the agent mid-execution

In practice, when your agent responds, the frontend receives a sequence of typed events:

data: {"type": "RUN_STARTED", "threadId": "test-123", "runId": "run-456"}
data: {"type": "TEXT_MESSAGE_START", "messageId": "msg-abc", "role": "assistant"}
data: {"type": "TEXT_MESSAGE_CONTENT", "messageId": "msg-abc", "delta": "Hello!"}
data: {"type": "TEXT_MESSAGE_END", "messageId": "msg-abc"}
data: {"type": "RUN_FINISHED", "threadId": "test-123", "runId": "run-456"}

Any AG-UI-compatible frontend, including CopilotKit, can consume that stream directly. In the next section, we will see how it all connects through an architecture diagram.

How It All Connects

CopilotKit is an AG-UI-compatible React client. It connects to an AG-UI endpoint, subscribes to the event stream and renders streaming responses, tool progress, and state updates inside your app.

Once your agent is deployed to AgentCore, HttpAgent points to the AgentCore invocation URL with a Cognito Bearer token in the headers, and user messages flow back over AG-UI into the running agent.

Here is what the full stack looks like when deployed:

Your agent framework  [Strands · LangGraph · CrewAI]
│
│  deployed to
▼
Amazon Bedrock AgentCore Runtime  [Bearer token via Cognito OAuth 2.0]
│  auth · session isolation · auto-scaling · observability
│
│  invokes container
▼
Agent container  [ARM64 · port 8080]
│  FastAPI · /invocations · /ping
│
│  calls model
▼
Amazon Bedrock model
│
│  streams back via
▼
AG-UI protocol  [SSE event stream]
│
│  consumed by
▼
CopilotKit  [HttpAgent · @ag-ui/client]
│
├─  renders streaming responses + tool progress
├─  syncs state updates in real time
└─  forwards user messages → agent via AG-UI
▼
Your application UI

If you want a complete full-stack starting point, AWS also released the Fullstack AgentCore Solution Template - a ready-made stack with AgentCore, LangGraph, CopilotKit and AG-UI all wired together, deployed via CDK. It's the fastest way to go from zero to a production-ready agentic app on AWS.

Deploying your AG-UI agent with Strands

This example uses AWS Strands as the agent framework but the same pattern works with LangGraph, CrewAI and any other AG-UI-compatible framework.

Before deploying, make sure you have:

Python 3.12+
AWS account with credentials configured (aws configure)
IAM permissions for AgentCore Runtime
Docker, Finch, or Podman for building container images
Amazon Cognito user pool configured (Setup guide here)

Start by installing the required packages:

pip install fastapi uvicorn ag_ui_strands strands-agents boto3

ag_ui_strands wraps your Strands agent with AG-UI protocol support, handling event encoding automatically.
strands-agents is the AWS agent framework used in this example.
boto3 is the AWS SDK, used internally by Strands to call Bedrock models.

Step 1: Build your AG-UI server

Create my_agui_server.py. This defines the model, wraps the agent with AG-UI support, and exposes the endpoints AgentCore needs.

import uvicorn
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse, JSONResponse
from ag_ui_strands import StrandsAgent
from ag_ui.core import RunAgentInput
from ag_ui.encoder import EventEncoder
from strands import Agent
from strands.models.bedrock import BedrockModel

model = BedrockModel(
    model_id="us.anthropic.claude-3-5-sonnet-20241022-v2:0",
    region_name="us-west-2",
)

strands_agent = Agent(
    model=model,
    system_prompt="You are a helpful assistant.",
)

agui_agent = StrandsAgent(
    agent=strands_agent,
    name="my_agent",
    description="A helpful assistant",
)

app = FastAPI()

@app.post("/invocations")
async def invocations(input_data: dict, request: Request):
    accept_header = request.headers.get("accept")
    encoder = EventEncoder(accept=accept_header)

    async def event_generator():
        run_input = RunAgentInput(**input_data)
        async for event in agui_agent.run(run_input):
            yield encoder.encode(event)

    return StreamingResponse(
        event_generator(),
        media_type=encoder.get_content_type()
    )

@app.get("/ping")
async def ping():
    return JSONResponse({"status": "Healthy"})

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8080)

Here, StrandsAgent wraps your Strands agent and handles all AG-UI event encoding so you don't write any protocol logic yourself.

AgentCore requires three things from your container:

POST /invocations - handles AG-UI requests, returns the SSE event stream.
GET /ping - health check, must return HTTP 200.
Port 8080 - required, do not change.

For LangGraph, CrewAI, and other frameworks, see the AG-UI framework integrations.

Step 2: Test locally

Start your server.

python my_agui_server.py

The server starts on http://localhost:8080. Send a test request to confirm the SSE stream is working before deploying.

curl -N -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "test-123",
    "run_id": "run-456",
    "state": {},
    "messages": [{"role": "user", "content": "Hello!", "id": "msg-1"}],
    "tools": [], "context": [], "forwardedProps": {}
  }'

You should see the SSE events stream back:

data: {"type": "RUN_STARTED", "threadId": "test-123", "runId": "run-456"}
data: {"type": "TEXT_MESSAGE_START", "messageId": "msg-abc", "role": "assistant"}
data: {"type": "TEXT_MESSAGE_CONTENT", "messageId": "msg-abc", "delta": "Hello!"}
data: {"type": "TEXT_MESSAGE_END", "messageId": "msg-abc"}
data: {"type": "RUN_FINISHED", "threadId": "test-123", "runId": "run-456"}

Step 3: Deploy to AgentCore

The AgentCore starter toolkit handles the entire deployment process for you.

It builds an ARM64 container image, pushes it to Amazon ECR, and creates the AgentCore runtime with protocol flag serverProtocol: "AGUI". You don't write a Dockerfile or configure any infrastructure manually.

Install it first:

pip install bedrock-agentcore-starter-toolkit

Create a requirements.txt so the toolkit knows what to install in the container:

fastapi 
uvicorn 
ag_ui_strands

Now configure and deploy. The configure command sets the protocol to AG-UI and links your Cognito auth setup. The deploy command does the rest:

# Configure with AG-UI protocol + OAuth authentication
agentcore configure -e my_agui_server.py --protocol AGUI

# Deploy to AWS
agentcore deploy

After a successful deploy, you receive an agent runtime ARN. You will need this to invoke the agent and connect your frontend.

arn:aws:bedrock-agentcore:us-west-2:ACCOUNT_ID:runtime/my_agui_server-xyz123

Step 4: Invoke your deployed agent

With the agent running on AgentCore, you can now invoke it and stream AG-UI events over the wire. Set your environment variables first:

export BEARER_TOKEN="<your-cognito-bearer-token>"
export AGENT_ARN="arn:aws:bedrock-agentcore:us-west-2:ACCOUNT_ID:runtime/my_agui_server-xyz123"

Install the HTTP client libraries for making SSE requests:

pip install httpx httpx-sse

Then invoke the agent:

import asyncio
import json
import os
from urllib.parse import quote
from uuid import uuid4

import httpx
from httpx_sse import aconnect_sse

async def invoke_agui_agent(message: str):
    agent_arn = os.environ.get('AGENT_ARN')
    bearer_token = os.environ.get('BEARER_TOKEN')
    escaped_arn = quote(agent_arn, safe='')

    url = f"https://bedrock-agentcore.us-west-2.amazonaws.com/runtimes/{escaped_arn}/invocations?qualifier=DEFAULT"
    headers = {
        "Authorization": f"Bearer {bearer_token}",
        "X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": str(uuid4()),
    }
    payload = {
        "threadId": str(uuid4()),
        "runId": str(uuid4()),
        "messages": [{"id": str(uuid4()), "role": "user", "content": message}],
        "state": {},
        "tools": [],
        "context": [],
        "forwardedProps": {},
    }

    async with httpx.AsyncClient(timeout=300) as client:
        async with aconnect_sse(client, "POST", url, headers=headers, json=payload) as sse:
            async for event in sse.aiter_sse():
                data = json.loads(event.data)
                event_type = data.get("type")
                if event_type == "TEXT_MESSAGE_CONTENT":
                    print(data.get("delta", ""), end="", flush=True)
                elif event_type == "RUN_ERROR":
                    print(f"Error: {data.get('code')} - {data.get('message')}")

asyncio.run(invoke_agui_agent("Hello from production!"))

You should see the agent's response stream token by token.

The same AG-UI events as local testing, now served from AgentCore with managed authentication, session isolation, and auto-scaling.

Step 5: Connect your frontend

With the agent deployed, point HttpAgent from @ag-ui/client at the AgentCore invocation URL instead of localhost. The rest of the client code stays the same.

This is the same AG-UI client used locally - you are just pointing it at the AgentCore URL instead of localhost.

import { HttpAgent } from "@ag-ui/client";

const agent = new HttpAgent({
  url: `https://bedrock-agentcore.us-west-2.amazonaws.com/runtimes/${encodedArn}/invocations?qualifier=DEFAULT`,
  headers: {
    "Authorization": `Bearer ${token}`,
    "X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": sessionId,
  }
});

For SigV4 authentication, sign requests using the AWS SDK. For OAuth 2.0, obtain a token from your configured Cognito user pool. Authentication configuration is the same across all AgentCore protocols. See the AgentCore auth docs for setup.

AG-UI support in AgentCore is available today across 14 AWS regions.

Resources

You can start building right away.

Explore the AgentCore AG-UI docs
Explore the AG-UI Protocol Docs
Check out the AWS Announcement

If you want a complete full-stack starting point, AWS also released the Fullstack AgentCore Solution Template - AgentCore, LangGraph, CopilotKit, and AG-UI all wired together and deployed via CDK.

What This Means

AG-UI works with LangGraph, CrewAI, AWS Strands, Google ADK, Mastra, Microsoft Agent Framework, LlamaIndex, Agno, and more. The protocol itself isn't tied to any of them - you pick the framework, AG-UI handles the interaction layer.

What was missing was a managed way to run these agents in production. That's what this support adds.

Beyond AG-UI hosting, AgentCore also covers persistent memory across sessions, tool connectivity via MCP Gateway, and production observability (though I didn't explore all of these in-depth).

thanks for reading!

You can connect me on GitHub, Twitter and LinkedIn.

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

How to add AI to your App in 5 minutes

Anmol Baranwal — Wed, 11 Mar 2026 15:10:31 +0000

Most AI integrations are overengineered.

Even basic tutorials on adding an AI assistant get mixed up with LangGraph workflows and FastAPI servers. You don't need any of that.

Today, we will build one without any external agent framework or orchestration layers. Just an AI that knows what's in your app and can actually do things.

You will also learn how to implement the Generative UI pattern, which lets you render actual components rather than just text responses.

Let's jump in.

Why AI Integrations Get Complicated

Adding AI to an app sounds simple until you actually try it.

The moment you go beyond a basic chat interface, you run into a fundamental problem: LLMs on their own can't do anything. They can only generate text.

To make an AI that actually takes actions: reads your app's state, updates data, calls APIs - you need to build the plumbing yourself.

That plumbing is what an AI agent is. At its core, every agent runs the same loop:

Observe - take in context (user input, tool results, memory)
Reason - decide what to do next
Act - call a tool, write to a file, hit an API, whatever's needed

Credit: Anthropic

Simple enough in theory. But in practice, you end up writing the orchestration layer that manages this loop, tool registries that expose your app's functions to the LLM, state management to track what the agent knows and error handling for when the model goes sideways.

context = [initial_event]
while True:
  next_step = await llm.determine_next_step(context)
  context.append(next_step)
  if next_step.intent == "done":
    return next_step.final_answer
  result = await execute_step(next_step)
  context.append(result)

In a full agent setup, the LLM sits at the center - orchestrating across data sources, tools, models, and external services.

Credit: Medium blog

That's why tutorials teach for LangGraph and FastAPI servers. And for complex automation pipelines, that complexity is justified.

But if your goal is just to add an AI assistant inside your product - one that understands your UI, reads your data, and can take actions on behalf of the user, you are building a lot of infrastructure to solve a much smaller problem.

CopilotKit handles the loop, the context, the streaming and the frontend integration so you don't have to wire any of it yourself.

For this tutorial, we will use CopilotKit without any external agent framework, just hooks and your LLM of choice.

What CopilotKit Actually Does

CopilotKit is an open source framework to add AI agents to your app that abstracts away everything you just read about.

Instead of building your own agent loop, tool registry and streaming layer, you get a set of hooks and pre-built components that plug directly into your existing frontend.

The mental model is simple, you provide the AI with two things:

Your app's state - so it understands what's on screen and what the user is working with
A set of tools - so it can actually do something useful, not just respond with text

In practice, this maps to two hooks:

useAgentContext - share app state with the AI. Whatever you pass in becomes part of the LLM's context: current user, selected items, loaded data - the AI knows what your app knows.
useFrontendTool - define actions the AI can trigger. You give it a name, a description, and a handler. The LLM decides when to call it based on what the user asks.

The result is that adding an AI assistant to your app becomes a frontend problem, not an infrastructure problem.

Your UI, agent, tools are in one single interaction loop.

How to wire everything (1 minute)

You can follow along with the official docs or just stick with this. I will walk through exactly what each piece does and why it's there.

I'm using Next.js with TypeScript, but this works with any React/Angular framework out there!

// creates a nextjs app  
npx create-next-app@latest .

Install the three CopilotKit packages you need:

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core/v2 : hooks and built-in chat components
@copilotkit/react-ui/v2 : styles
@copilotkit/runtime : backend runtime and LLM adapters

Create the API route at app/api/copilotkit/route.ts.

It's the single endpoint that receives messages from your UI, runs them through the agent loop and streams responses back.

BuiltInAgent is what runs the observe → reason → act loop
CopilotRuntime manages the session, streaming, and tool execution

That's your entire backend.

import { CopilotRuntime, copilotRuntimeNextJSAppRouterEndpoint } from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";

const builtInAgent = new BuiltInAgent({
  model: "openai:gpt-5",
  // apiKey: process.env.OPENAI_API_KEY,
});

const runtime = new CopilotRuntime({
  agents: { default: builtInAgent },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    endpoint: "/api/copilotkit",
  });
  return handleRequest(req);
};

Create .env.local in the root and add your OpenAI API key.

OPENAI_API_KEY=sk-proj-...

If you want to switch to any other LLM Provider, you just need to pass the modified string to BuiltInAgent in your API route. Everything else remains the same.

// Anthropic
const builtInAgent = new BuiltInAgent({ model: "anthropic:claude-sonnet-4-5" });

// Google
const builtInAgent = new BuiltInAgent({ model: "google:gemini-2.0-flash" });

Add the matching key to .env.local and you are done. For custom models like Azure OpenAI, AWS Bedrock or Ollama, check the model selection docs.

Now, connect the frontend to that backend by wrapping your app in app/layout.tsx.

import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/v2/styles.css";

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        <CopilotKit runtimeUrl="/api/copilotkit">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

CopilotKit is the context provider that every hook below it depends on. runtimeUrl points to the route you just created.

There are many in-built components like CopilotSidebar, CopilotChat and CopilotPopup. For this, I will be adding the chat sidebar.

import { CopilotSidebar } from "@copilotkit/react-core/v2";

export default function Page() {
  return (
    <main>
      <h1>Your App</h1>
      <CopilotSidebar />
    </main>
  );
}

Run npm run dev and you now have a working AI sidebar assistant! 🎉

It can respond to anything but it has no idea what's actually in your app. It doesn't know your data, your UI state or what the user is looking at. The next two steps fix that.

Give the AI Context

By default, the AI knows nothing about what's in your app. You can ask it, "how much did I spend on food?" and it has no idea - it only sees the conversation, not your app's state.

useAgentContext solves that. It pushes your React state into the agent's context window on every turn, so the AI always has a current snapshot of what's in your UI.

Let's build a simple expense tracker to see how this works.

"use client";
import { useState } from "react";
import { useAgentContext, CopilotSidebar } from "@copilotkit/react-core/v2";

type Expense = {
  id: number;
  description: string;
  amount: number;
  category: string;
};

const initialExpenses: Expense[] = [
  { id: 1, description: "Groceries", amount: 85, category: "Food" },
  { id: 2, description: "Netflix", amount: 15, category: "Entertainment" },
  { id: 3, description: "Uber", amount: 22, category: "Transport" },
];

export default function Page() {
  const [expenses, setExpenses] = useState<Expense[]>(initialExpenses);

  useAgentContext({
    description: "The user's current expense list. Each item has an id, description, amount in dollars, and category.",
    value: expenses,
  });

  return (
    <main className="p-8">
      <h1 className="text-2xl font-bold mb-4">My Expenses</h1>
      <ul className="space-y-2">
        {expenses.map((e) => (
          <li key={e.id} className="flex justify-between border-b py-2">
            <span>
              {e.description}{" "}
              <span className="text-gray-400 text-sm">({e.category})</span>
            </span>
            <span>${e.amount}</span>
          </li>
        ))}
      </ul>
      <CopilotSidebar />
    </main>
  );
}

expenses is your normal React state. useAgentContext takes it and injects it into the LLM's context on every turn.

Now, if the user asks "what's my biggest expense?", the AI is looking at the same list your UI renders.

Let the AI Do Things

Awareness is one thing. Being able to act is what makes an agent actually useful.

useFrontendTool lets you hand the AI a set of actions it can trigger. You define a tool with a name, a description, and a handler - the LLM decides when to call it based on what the user asks.

The LLM reads the description fields to understand what each tool does and what each parameter expects, so write them clearly and the model will fill them accurately even from casual conversational input.

Add useFrontendTool inside the same page.

parameters takes a Zod schema (a TypeScript-first validation library). If you haven't used it before, the idea is simple: you define the shape of your data with z.object({...}) and each field gets a .describe() call that tells the LLM what it means.

Install it with npm install zod.

"use client";
import { useState } from "react";
import { useAgentContext, useFrontendTool, CopilotSidebar } from "@copilotkit/react-core/v2";
import { z } from "zod";

// ... type and initialExpenses stay the same

export default function Page() {
  const [expenses, setExpenses] = useState<Expense[]>(initialExpenses);

  // ... useAgentContext

  useFrontendTool({
  name: "addExpense",
  description:
    "Add a new expense when the user mentions spending money on something.",
  parameters: z.object({
    description: z
      .string()
      .describe("What the expense was for, e.g. Lunch, Taxi, Coffee"),
    amount: z.number().describe("How much was spent in dollars"),
    category: z
      .string()
      .describe("Category: Food, Transport, Entertainment, Health, or Other"),
  }),
  handler: async ({ description, amount, category }) => {
    setExpenses((prev) => [
      ...prev,
      { id: Date.now(), description, amount, category },
    ]);
  },
});

  return (
    // ... remains the same
  );
}

You can send a sample query like "I spent $40 on dinner last night." The agent calls addExpense with { description: "Dinner", amount: 40, category: "Food" }, your handler updates expenses and the new item appears in the list instantly.

This is the observe → reason → act loop running end to end.

The agent observed your message and your current expenses via useAgentContext, reasoned that addExpense is the right call, and acted by invoking your handler with the right parameters. CopilotKit handled everything in between.

Bonus: Generative UI

The AI can read your data and update it. But so far, it only responds in text.

Generative UI is a new idea: instead of the agent describing results, it renders actual UI. For example, if someone asks about their spending, the app would display a breakdown card.

CopilotKit supports this through the render property on useFrontendTool. Instead of a text reply, the tool returns a React component - rendered inline in the chat, using your own design system.

Let's add a summary tool inside the same component.

import { ToolCallStatus, useFrontendTool } from "@copilotkit/react-core/v2";

useFrontendTool({
  name: "showSpendingSummary",
  description:
    "Call this when the user asks for a summary or overview of their expenses.",
  parameters: z.object({}),
  handler: async () => {
    const summary = expenses.reduce(
      (acc, e) => {
        acc[e.category] = (acc[e.category] ?? 0) + e.amount;
        return acc;
      },
      {} as Record<string, number>,
    );
    const total = expenses.reduce((sum, e) => sum + e.amount, 0);
    return JSON.stringify({ summary, total });
  },
  render: ({ result, status }) => {
    return (
      <div className="rounded-lg border p-4 mt-2 space-y-3">
        <p className="font-semibold text-sm">
          {status === ToolCallStatus.InProgress ? "Calculating..." : "Spending Breakdown"}
        </p>
        {status === ToolCallStatus.Complete && result && (
          <>
            {Object.entries(
              (JSON.parse(result) as { summary: Record<string, number>; total: number }).summary
            ).map(([category, amount]) => (
              <div key={category} className="flex justify-between text-sm">
                <span className="text-gray-600">{category}</span>
                <span className="font-medium">${amount}</span>
              </div>
            ))}
            <div className="flex justify-between text-sm font-semibold border-t pt-2">
              <span>Total</span>
              <span>${(JSON.parse(result) as { summary: Record<string, number>; total: number }).total}</span>
            </div>
          </>
        )}
      </div>
    );
  },
});

What's happening here:

parameters: z.object({}) is empty. The LLM doesn't need to pass anything, it just needs to decide when to call the tool
handler does the actual work, calculates category totals from the expenses state and returns them
ToolCallStatus gives you the exact lifecycle states of the tool call
result is the JSON string returned by your handler, parsed back into an object inside render

When you ask, "summarize my spending." LLM reads the query, looks at the context, recognizes that showSpendingSummary is the right tool to call and triggers it.

That's the pattern: the LLM decides when to act, your code does the work, your components display the result.

The working implementation of this minimal expense tracker is available in this GitHub repo in the example/basic branch.

The main branch takes the same pattern further with a full kanban board:

Move tasks between columns by asking the AI
Add, delete and reassign tasks from natural language
Get a visual board summary with generative UI

Here is the demo!

Most AI features that feel magical aren't. They are just an app that knows what's on screen and has a few actions wired up. That's it.

The hard part was always the infrastructure. Turns out it doesn't have to be.

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Give Your AG2 Agents a UI with AG‑UI and CopilotKit

Anmol Baranwal — Thu, 26 Feb 2026 15:01:48 +0000

Agent frameworks have become increasingly capable at reasoning, orchestration and tool execution. But most of them still operate as isolated systems.

AG2 (formerly AutoGen) changes that. It provides a shared runtime where specialized agents with defined roles, tools and conversation patterns can collaborate as one coordinated team across models and even frameworks.

But orchestration alone isn’t the same as shipping an interactive experience. Every application ends up building custom protocols and UI wiring to observe runs, show tool progress and capture user actions during execution.

AG-UI (Agent-User Interaction Protocol) defines that interaction contract. It standardizes how agent activity is streamed, so any frontend can stay synchronized in real time.

Today, AG2 integrates with AG‑UI and CopilotKit to bring agent orchestration into real applications.

You can refer to the docs and try the interactive AG-UI Dojo if you want to explore it yourself.

Let’s break down how these layers fit together, the end-to-end integration flow and what this unlocks for agent builders.

AG2: Runtime for Agent Teams

AG2 (formerly AutoGen) is an open-source programming framework built around multi-agent collaboration. You define specialized agents, wire tools and run structured conversations to solve tasks.

Instead of a single monolithic loop, AG2 treats agents as composable actors that collaborate through shared context.

At its core is ConversableAgent, the primitive that defines an agent’s role, reasoning model and tool access while handling message exchange.

from autogen import ConversableAgent, LLMConfig
from dotenv import load_dotenv
import os
load_dotenv()

# Create LLM configuration first
llm_config = LLMConfig({
  "api_type": "openai",
  "model": "gpt-5-mini",
  "api_key": os.getenv("OPENAI_API_KEY"),
})

# Create the agent
lesson_planner = ConversableAgent(
    name="planner_agent",
    system_message="You are a lesson planner for a fourth grade class.",
    description="Creates lesson plans.",
    llm_config=llm_config,
)

Agents can be organized into coordinated workflows using built-in patterns such as sequential flows, group conversations, nested calls and human-in-the-loop checkpoints.

In practice, you might model a planner, executor and reviewer as distinct agents: each with its own role and tools and let AG2 manage how they interact over multiple turns.

auto_selection = AutoPattern(
    agents=[teacher, lesson_planner, lesson_reviewer],
    initial_agent=lesson_planner,
    group_manager_args={"llm_config": llm_config},
)

response = run_group_chat(
    pattern=auto_selection,
    messages="Introduce students to the solar system.",
    max_rounds=20,
)

response.process()

This allows backend behavior to evolve by assembling agents differently, rather than rewriting control logic for every new workflow.

AG-UI integration currently wraps a single ConversableAgent. Multi-agent patterns (group chat, delegation, nested flows) still run inside AG2, but are exposed through a unified interaction surface.

Staged or multi-agent UX can be modeled using context variables. The pattern is covered in the integration section.

Read all the basic concepts involved in the docs and check the example repo, which includes many examples.

The Gap: Orchestration Without an Application Contract

Even if multi-agent logic is portable, it does not automatically become app-portable.

In a production application, the UI needs a stable runtime contract for:

Streaming messages and partial results.
Tool lifecycle (start → progress → finish/fail).
State snapshots and updates while the run is still executing.
Human-in-the-loop actions that can steer or interrupt a run

AG2 manages delegation, shared context and orchestration internally. But it does not define a standard event/state interface for how that execution is exposed to a frontend.

Without that contract, every AG2 app ends up inventing its own WebSocket/SSE payload shapes and UI mapping rules. Integration becomes tightly coupled glue code instead of a reusable interaction layer.

This is where AG-UI Protocol fits.

AG-UI: The Execution & Interaction Layer

AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes the live interaction stream between an agent runtime and an application.

During a run, the backend emits a stream of structured event types. The frontend consumes that stream and can send compatible interaction events back.

This matters because AG2 already manages multi-agent coordination internally. What AG-UI defines is how that coordination becomes observable and interactive at the application boundary.

Instead of every app inventing custom payload formats, AG-UI defines a small set of self-describing JSON event types.

Each event includes a clear type and structured payload. For example:

TEXT_MESSAGE_CONTENT : event streams LLM tokens
TOOL_CALL_START/END : convey function call progress
STATE_DELTA : carries JSON Patch deltas to sync state

In practice, AG‑UI standardizes:

Message streaming
Tool lifecycle events (start → progress → finish / fail)
State snapshots and incremental updates
User interaction events (clicks, form submits, confirmations)

Because the stream is standard and ordered, the frontend can reliably interpret what the backend is doing and render progress in real time.

The result is decoupling. The backend doesn’t need to know how the UI renders progress and the UI doesn’t need to understand internal orchestration logic.

Here, AG-UI uses Server-Sent Events (SSE) over HTTP as its primary transport. The AG2 integration serves a text/event-stream response per run.

In AG2, this protocol is implemented via AGUIStream, which wraps a ConversableAgent and exposes it as an ASGI endpoint that streams AG-UI events.

Explore the full event specification in the AG-UI documentation and implementation details in the AG2 user guide, including best practices for when to adopt this integration.

How CopilotKit wires AG2 to AG-UI

On the backend, AG2 exposes agent runs through an AG-UI-compatible endpoint using AGUIStream. That endpoint streams standardized AG-UI events for the lifecycle of a run.

CopilotKit acts as that client and UI runtime. It connects to the AG‑UI endpoint (via @ag-ui/client), subscribes to the event stream, renders agent activity inside your application and forwards user actions back into the same execution loop.

The wiring is straightforward:

Your backend runs an AG2 agent workflow
AGUIStream wraps the agent and translates execution into AG-UI events
Those events are exposed through an AG-UI endpoint over SSE
CopilotKit connects and subscribes to the live event stream
Agent messages, tool progress and state updates are rendered in the UI
User interactions are sent back over AG-UI into the same AG2 run

The same endpoint can be consumed by any other AG-UI-compatible client. At a high level, the full interaction loop looks like this:

AG2 Runtime (multi-agent orchestration)
│
│ executes ConversableAgents
│ manages delegation + shared state
│ invokes tools
▼
AG2 ↔ AG-UI Integration Layer (AGUIStream)
│
│ wraps ConversableAgent
│ maps AG2 activity → standardized AG-UI events
│ handles SSE transport
▼
AG-UI Endpoint (event + state stream)
│
│ agent ↔ application synchronization
│ TEXT_MESSAGE_CONTENT
│ TOOL_CALL_START / END
│ STATE_DELTA / STATE_SNAPSHOT
│ USER_INTERACTION
▼
CopilotKit (AG-UI client runtime)
│
├─ subscribes to live stream
├─ renders messages + tool lifecycle
├─ synchronizes state updates
└─ forwards user actions → AG2 via AG-UI
▼
Application UI

The backend remains purely AG2. CopilotKit simply consumes the AG-UI stream and renders it as product UI.

Integration Flow: AG2 → AG-UI → CopilotKit

We have seen the conceptual flow. Now let’s look at the actual implementation.

You can check the AG-UI integration examples in the build-with-ag2 repository, which consists of two patterns: weather/ directory shows a single-agent chat with tools & factory/ directory demonstrates a multi-stage pipeline using ContextVariables.

For complete CopilotKit + AG2 integration with full UI implementations, explore the ag2-samples repository.

The integration centers on AGUIStream, which wraps a ConversableAgent and exposes it as an AG‑UI endpoint over SSE.

Define your agent exactly as you normally would, then mount the stream on FastAPI. The only important detail is enabling streaming in the LLM configuration.

from autogen import ConversableAgent, LLMConfig
from autogen.ag_ui import AGUIStream
from fastapi import FastAPI

agent = ConversableAgent(
    name="weather_agent",
    system_message="You are a helpful weather assistant. Use the get_weather tool to look up current conditions for any city.",
    llm_config=LLMConfig({
        "model": "gpt-5-mini",
        "stream": True
    }),
    functions=[get_weather],
)

# wrap the agent and mount it as an ASGI endpoint
stream = AGUIStream(agent)  

app = FastAPI()  
app.mount("/chat", stream.build_asgi())

That’s the entire backend bridge.

AGUIStream automatically maps:

Token streaming → TEXT_MESSAGE_*
Tool execution → TOOL_CALL_START, TOOL_CALL_RESULT
Context updates → STATE_SNAPSHOT / STATE_DELTA
Run lifecycle → RUN_STARTED, RUN_FINISHED, RUN_ERROR

The endpoint serves Server-Sent Events (SSE). When a client sends a RunAgentInput payload with Accept: text/event-stream, the response becomes a live event stream.

Here's a simplified example of what flows back:

data: {"type":"RUN_STARTED", ...}
data: {"type":"TEXT_MESSAGE_CONTENT","delta":"The"}
data: {"type":"TEXT_MESSAGE_CONTENT","delta":" weather"}
data: {"type":"TOOL_CALL_START", ...}
data: {"type":"TOOL_CALL_RESULT", ...}
data: {"type":"RUN_FINISHED", ...}

The frontend does not need to understand AG2 internals. It reacts to event types. At the lowest level, consuming AG-UI is just opening an SSE stream:

const response = await fetch("/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Accept": "text/event-stream"
  },
  body: JSON.stringify(runAgentInput)
});

But in practice, you don’t parse SSE manually.

CopilotKit understands AG-UI natively. You keep the same backend (AGUIStream), and simply point CopilotKit to that endpoint. Proxy AG‑UI traffic through a Next.js API route.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { HttpAgent } from "@ag-ui/client";
import { NextRequest } from "next/server";

// You can use any service adapter here for multi-agent support
// Use ExperimentalEmptyAdapter since AG2 handles the LLM directly
const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    weather_agent: new HttpAgent({
      url: "http://localhost:8000/chat"
    })
  }
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Finally, wrap your app with CopilotKit and render the chat. CopilotKit subscribes to the AG‑UI stream and renders messages, tool progress and state updates in real time.

import { CopilotKit } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";

<CopilotKit runtimeUrl="/api/copilotkit" agent="weather_agent">
  <CopilotChat />
</CopilotKit>

That's the complete loop. The weather example in the build-with-ag2 repository shows this minimal streaming setup.

Multi-Agent Orchestration

The current AG-UI integration wraps a single ConversableAgent. Multi-agent workflows (group chats, delegation, nested conversations) still run inside AG2, but they are exposed through one interaction surface.

To model staged or multi-agent UX, use an orchestrator agent with ContextVariables. It is a shared state that agents use to coordinate, track progress, and make decisions across a conversation.

When a tool updates context_variables, AGUIStream automatically emits a STATE_SNAPSHOT event.

The frontend can consume that snapshot to:

Highlight which agent is active
Show pipeline progress
Render stage-specific UI

Here's the pattern:

from autogen import ConversableAgent
from autogen.ag_ui import AGUIStream

def submit_plan(plan: str, context): # context injected automatically
    """Submit the plan and move to draft stage."""
    context["active_agent"] = "drafter"
    context["stage"] = "drafting"
    context["plan"] = plan
    return {"status": "plan submitted"}

def submit_draft(draft: str, context):
    """Submit the draft and move to review stage."""
    context["active_agent"] = "reviewer"
    context["stage"] = "reviewing"
    context["draft"] = draft
    return {"status": "draft submitted"}

def submit_review(approved: bool, context):
    """Submit review and finalize."""
    context["stage"] = "complete" if approved else "revising"
    return {"status": "review complete"}

orchestrator = ConversableAgent(
    name="orchestrator",
    system_message="You coordinate a multi-stage workflow.",
    llm_config=llm_config,
    functions=[submit_plan, submit_draft, submit_review],
)

stream = AGUIStream(orchestrator)

Each tool call that updates context triggers a STATE_SNAPSHOT. The frontend reads active_agent and stage from the snapshot and updates the UI accordingly.

The factory example shows this pattern with a 3-panel UI: agent pipeline, document preview and conversation log.

Native multi-agent support is actively being developed. This orchestrator pattern is the current workaround.

What This Enables for Agent Builders

With this integration, you can keep AG2 where it’s strong and still ship real UIs without protocol glue. You can:

✅ Build AG2 agent teams and expose them as AG‑UI endpoints with one wrapper (AGUIStream).

✅ Connect those endpoints to CopilotKit (or any AG‑UI client) without custom WebSocket or polling infrastructure.

✅ Reuse AG2 agent logic across products while keeping a single frontend contract (AG-UI) instead of inventing new payloads for every application

✅ Stream agent runs, tool progress and state in real time using CopilotKit components, so users see what the system is doing instead of waiting on a single final response.

Getting Started

You can start building with this integration today.

👉 Explore the AG2 documentation

👉 Check out the CopilotKit / AG2 docs to get started

👉 Check out our example repo for a minimal integration setup

👉 Explore the build-with-ag2 AG‑UI examples: weather/ directory shows a single-agent chat with tools, factory/ directory demonstrates a multi-stage pipeline using ContextVariables

👉 Try the AG2 demo in the AG-UI Dojo playground

What's Next

This launch makes AG2 a first‑class backend for AG‑UI‑compatible frontends.

On the CopilotKit side, the focus is on richer AG‑UI rendering components, better devtools for inspecting AG‑UI streams and more starter projects that bundle AG2 + CopilotKit out of the box.

Native multi-agent support in AG‑UI is an active focus. The AG2 and CopilotKit teams are working together to bring first-class multi-agent patterns to the protocol so you won't need the orchestrator workaround.

Over the next few weeks, expect more reference implementations, clearer integration patterns and production-grade examples that show how these layers work together in real applications.

How to Build a Research Assistant using Deep Agents

Anmol Baranwal — Fri, 20 Feb 2026 14:22:05 +0000

LangChain's Deep Agents provide a new way to build structured, multi-agent systems that can plan, delegate and reason across multiple steps.

It comes with planning, a filesystem for context and subagent spawning built in. But connecting that agent to a live frontend and actually showing what’s happening behind the scenes in real time is still surprisingly hard.

Today, we will build a Deep Agents powered research assistant using Tavily and connect it to a live Next.js UI with CopilotKit, so every step the agent takes streams to the frontend in real time.

You will find architecture, the key patterns, how state flows between the UI ↔ agent and the step-by-step guide to building this from scratch.

Let's build it.

What is covered?

In summary, we are covering these topics in detail.

What are Deep Agents?
Core Components
What are we building?
Building Frontend
Building Backend (FastAPI + Deep Agents + AG-UI)
Running Application
Data flow (frontend ↔ Agent)

Here is the GitHub Repository, deployed link and official docs if you want to explore yourself.

1. What are Deep Agents?

Most agents today are just “LLM in a loop + tools”. That works but it tends to be shallow: no explicit plan, weak long-horizon execution and messy state as runs get longer.

Popular agents like Claude Code, Deep Research and Manus get around this by following a common pattern: they plan first, externalize working context (often via files or a shell) and delegate isolated pieces of work to sub-agents.

Deep Agents package those primitives into a reusable agent runtime.

Instead of designing your own agent loop from scratch, you call create_deep_agent(...) and get a pre-wired execution graph that already knows how to plan, delegate and manage state across many steps.

Credit: LangChain

At a practical level, a Deep Agent created via create_deep_agent is just a LangGraph graph. There’s no separate runtime or hidden orchestration layer.

The "context management" in deep agents is also very practical -- they offload large tool payloads to the filesystem and only fall back to summarization when token usage approaches the model’s context window. You can read more on Context Management for Deep Agents blog by LangChain.

The mental model (how it runs)

Conceptually, the execution flow looks like this:

User goal
  ↓
Deep Agent (LangGraph StateGraph)
  ├─ Plan: write_todos → updates "todos" in state
  ├─ Delegate: task(...) → runs a subagent with its own tool loop
  ├─ Context: ls/read_file/write_file/edit_file → persists working notes/artifacts
  ↓
Final answer

That gives you a usable structure for “plan → do work → store intermediate artifacts → continue” without inventing your own plan format, memory layer or delegation protocol.

You can check official docs.

Where CopilotKit Fits

Deep Agents push key parts into explicit state (e.g. todos + files + messages), which makes runs easier to inspect. That explicit state is also what makes Copilotkit integration possible.

CopilotKit is a frontend runtime that keeps UI state in sync with agent execution by streaming agent events and state updates in real time (using AG-UI under the hood).

This middleware (CopilotKitMiddleware) is what allows the frontend to stay in lock-step with the agent as it runs. You can read the docs at docs.copilotkit.ai/langgraph/deep-agents.

agent = create_deep_agent(
    model="openai:gpt-4o",
    tools=[get_weather],
    middleware=[CopilotKitMiddleware()], # for frontend tools and context
    system_prompt="You are a helpful research assistant."
)

2. Core Components

Here are the core components that we will be using later on:

1) Planning Tools (built-in via Deep Agents) - built-in planning/to‑do behavior so the agent can break the workflow into steps without you writing a separate planning tool.

# Conceptual example (not required in codebase)
@tool
def todo_write(tasks: List[str]) -> str:
    formatted = "\n".join([f"- {task}" for task in tasks])
    return f"Todo list created:\n{formatted}"

2) Subagents - let the main agent delegate focused tasks into isolated execution loops. Each sub-agent has its own prompt, tools and context.

subagents = [
    {
        "name": "job-search-agent",
        "description": "Finds relevant jobs and outputs structured job candidates.",
        "system_prompt": JOB_SEARCH_PROMPT,
        "tools": [internet_search],
    }
]

3) Tools - this is how the agent actually does things. Here, finalize() signals completion.

@tool
def finalize() -> dict:
    """Signal that the agent is done."""
    return {"status": "done"}

How Deep Agents are implemented (Middleware)

If you are wondering how create_deep_agent() actually injects planning, files and subagents into a normal LangGraph agent, the answer is middleware.

Each feature is implemented as a separate middleware. By default, three are attached:

To-do list middleware - adds the write_todos tool and instructions that push the agent to explicitly plan and update a live todo list during multi-step tasks.
Filesystem middleware - adds file tools (ls, read_file, write_file, edit_file) so the agent can externalize notes and artifacts instead of stuffing everything into chat history.
Subagent middleware - adds the task tool, allowing the main agent to delegate work to subagents with isolated context and their own prompts/tools.

This is what makes Deep Agents feel “pre-wired” without introducing a new runtime. If you want to go deeper, the middleware docs show the exact implementation details.

3. What are we building?

Let's create an agent that:

Accepts a research question from the user
Uses Deep Agents to plan multi-step and orchestrate sub-agents
Searches the web using Tavily
Writes intermediate research artifacts using the filesystem middleware
Streams tool results back to the UI via CopilotKit (AG-UI)

The interface is a two-panel app where the left side is a CopilotKit chat UI and the right side is a live workspace that shows the agent’s plan, generated files and sources as the agent works.

Here's a simplified call request → response flow of what will happen:

[User asks research question]
        ↓
Next.js Frontend (CopilotChat + Workspace)
        ↓
CopilotKit Runtime → LangGraphHttpAgent
        ↓
Python Backend (FastAPI + AG-UI)
        ↓
Deep Agent (research_assistant)
    ├── write_todos        (planning, built-in)
    ├── write_file         (filesystem, built-in)
    ├── read_file          (filesystem, built-in)
    └── research(query)
            └── internal Deep Agent [thread-isolated]
                    └── internet_search (Tavily)

We will see the concepts in action as we build the agent.

4. Frontend: wiring the agent to the UI

Let's first build the frontend part. This is how our directory will look.

The src directory hosts the Next.js frontend, including the UI, shared components and the CopilotKit API route (/api/copilotkit) used for agent communication.

.
├── src/                             ← Next.js frontend
│   ├── app/
│   │   ├── page.tsx          
│   │   ├── layout.tsx               ← CopilotKit provider
│   │   └── api/
│   │       └── copilotkit/route.ts  ← CopilotKit AG-UI runtime
│   ├── components/
│   │   ├── FileViewerModal.tsx      ← Markdown file viewer
│   │   ├── WorkSpace.tsx            ← Research progress display
│   │   └── ToolCard.tsx             ← Tool call visualizer
├── lib/
│   └── types.ts
├── package.json                     
├── next.config.ts                   
└── README.md

If you don’t have a frontend, you can create a new Next.js app with TypeScript.

// creates a nextjs app  
npx create-next-app@latest .

Step 1: CopilotKit Provider & Layout

Install the necessary CopilotKit packages.

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core provides the core React hooks and context that connect your UI to an AG-UI compatible agent backend.
@copilotkit/react-ui offers ready-made UI components like <CopilotChat /> to build AI chat or assistant interfaces quickly.
@copilotkit/runtime is the server-side runtime that exposes an API endpoint and bridges the frontend with an AG-UI compatible backend (e.g., a LangGraph HTTP agent).

The <CopilotKit> component must wrap the Copilot-aware parts of your application. In most cases, it's best to place it around the entire app, like in layout.tsx.

import type { Metadata } from "next";

import { CopilotKit } from "@copilotkit/react-core";
import "./globals.css";
import "@copilotkit/react-ui/styles.css";

export const metadata: Metadata = {
  title: "Deep Research Assistant | CopilotKit Deep Agents Demo",
  description: "A research assistant powered by Deep Agents and CopilotKit - demonstrating planning, memory, subagents, and generative UI",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body className="antialiased">
        <CopilotKit runtimeUrl="/api/copilotkit" agent="research_assistant">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

Here, runtimeUrl="/api/copilotkit" points to the Next.js API route CopilotKit uses to talk to the agent backend.

Each page is wrapped in this context so UI components know which agent to invoke and where to send requests.

Step 2: Next.js API Route: Proxy to FastAPI

This Next.js API route acts as a thin proxy between the browser and the Deep Agents. It:

Accepts CopilotKit requests from the UI
Forwards them to the agent over AG-UI
Streams agent state and events back to the frontend

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// Empty adapter since the LLM is handled by the remote agent
const serviceAdapter = new ExperimentalEmptyAdapter();

// Configure CopilotKit runtime with the Deep Agents backend
const runtime = new CopilotRuntime({
  agents: {
    research_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here's a simple explanation of the above code:

The code above registers the research_assistant agent.
LangGraphHttpAgent : defines a remote LangGraph agent endpoint. It points to the Deep Agents backend running on FastAPI.
ExperimentalEmptyAdapter : simple no-op adapter used when the agent backend handles its own LLM calls and orchestration
copilotRuntimeNextJSAppRouterEndpoint : small helper that adapts the Copilot runtime to a Next.js App Router API route and returns a handleRequest function

Step 3: Types (Research State)

Before building components, let's define shared state for todos, files, and sources in lib/types/research.ts. These are the contracts between the tool results from the agent and the local React state.

// Uses local state + useDefaultTool instead of CoAgent (avoids type issues with Python FilesystemMiddleware)

export interface Todo {
  id: string;
  content: string;
  status: "pending" | "in_progress" | "completed";
}

export interface ResearchFile {
  path: string;
  content: string;
  createdAt: string;
}

// Sources found via internet_search (includes content)
export interface Source {
  url: string;
  title: string;
  content?: string;
  status: "found" | "scraped" | "failed";
}

export interface ResearchState {
  todos: Todo[];
  files: ResearchFile[];
  sources: Source[];
}

export const INITIAL_STATE: ResearchState = {
  todos: [],
  files: [],
  sources: [],
};

Instead of dumping raw tool JSON into chat, each tool result routes into a dedicated state slot - write_todos updates todos, write_file appends to files and research appends to sources. This becomes the foundation of the Workspace panel.

Step 4: Building Key Components

I'm only covering the core logic behind each component since the overall code is huge. You can find all the components in the repository at src/components.

✅ ToolCard Component

This is the client component that renders every tool call inline inside chat. It has two modes:

SpecializedToolCard for known tools (write_todos, research, write_file, read_file) with icons, status indicators and result previews
DefaultToolCard for unknown tools that fall back to expandable JSON.

"use client";

import { useState } from "react";
import { Pencil, ClipboardList, Search, Save, BookOpen, Check, ChevronDown } from "lucide-react";

const TOOL_CONFIG = {
  write_todos: {
    icon: Pencil,
    getDisplayText: () => "Updating research plan...",
    getResultSummary: (result, args) => {
      const todos = (args as { todos?: unknown[] })?.todos;
      if (Array.isArray(todos)) {
        return `${todos.length} todo${todos.length !== 1 ? "s" : ""} updated`;
      }
      return null;
    },
  },
  research: {
    icon: Search,
    getDisplayText: (args) =>
      `Researching: ${((args.query as string) || "...").slice(0, 50)}${(args.query as string)?.length > 50 ? "..." : ""}`,
    getResultSummary: (result) => {
      if (result && typeof result === "object" && "sources" in result) {
        const { sources } = result as { summary: string; sources: unknown[] };
        return `Found ${sources.length} source${sources.length !== 1 ? "s" : ""}`;
      }
      return "Research complete";
    },
  },
  write_file: {
    icon: Save,
    getDisplayText: (args) => {
      const path = args.path as string | undefined;
      const filename =
        path?.split("/").pop() || (args.filename as string | undefined);
      return `Writing: ${filename || "file"}`;
    },
    getResultSummary: (_result, args) => {
      const content = args.content as string | undefined;
      if (content) {
        const firstLine = content.split("\n")[0].slice(0, 50);
        return firstLine + (content.length > 50 ? "..." : "");
      }
      return "File written";
    },
  },
  // read_todos, read_file follow the same pattern
};

export function ToolCard({ name, status, args, result }: ToolCardProps) {
  const config = TOOL_CONFIG[name];
  if (config) {
    return (
      <SpecializedToolCard
        name={name}
        status={status}
        args={args}
        result={result}
        config={config}
      />
    );
  }
  return (
    <DefaultToolCard name={name} status={status} args={args} result={result} />
  );
}

Here's a brief explanation:

research and write_todos are expandable, clicking reveals the full query + findings or the live todo checklist with pending / inprogress / completed states.
resultSummary appears as a small green line below the display text so you can glance at the output without expanding.
DefaultToolCard handles any tool not in TOOL_CONFIG, showing collapsible args and result JSON.

In this component, ExpandedDetails handles the per-tool expanded view. research and write_todos get structured layouts; everything else falls back to a JSON pre block.

function ExpandedDetails({ name, result, args }) {
  if (name === "research") {
    const summary = typeof result === "object" && result && "summary" in result
      ? (result as any).summary
      : "";
    return (
      <div>
        <p>Query: {args.query as string}</p>
        <p>{summary}</p>
      </div>
    );
  }

  if (name === "write_todos") {
    const todos = (args as any)?.todos;
    return (
      <div>
        {todos?.map((todo: any, i: number) => (
          <div key={todo.id || i}>
            <span>{todo.status === "completed" ? "✓" : todo.status === "inprogress" ? "●" : "○"}</span>
            <span>{todo.content}</span>
          </div>
        ))}
      </div>
    );
  }

  // Fallback
  return (
    <pre>
      {typeof result === "string" ? result : JSON.stringify(result, null, 2)}
    </pre>
  );
}

Check out the complete code at src/components/ToolCard.tsx.

✅ Workspace Component

This is the right-side panel that displays research progress as it happens. It has three collapsible sections: Research Plan, Files and Sources - each with a live badge count and an empty state when nothing has arrived yet.

export function Workspace({ state }: { state: ResearchState }) {
  const { todos, files, sources } = state;
  const fileCount = files.length;
  const todoCount = todos.length;
  const sourceCount = sources.length;
  
  // State for file viewer modal
  const [selectedFile, setSelectedFile] = useState<ResearchFile | null>(null);

  return (
    <div className="workspace-panel p-6">
      <div className="mb-6">
        <h2 className="text-xl font-bold">Workspace</h2>
        <p className="text-sm">Research progress and artifacts</p>
      </div>
      <Section title="Research Plan" icon={ListTodo} badge={todos.length}>
        <TodoList todos={todos} />
      </Section>
      <Section title="Files" icon={FileText} badge={files.length}>
        <FileList files={files} onFileClick={setSelectedFile} />
      </Section>
      <Section title="Sources" icon={Globe} badge={sources.length}>
        <SourceList sources={sources} />
      </Section>
      <FileViewerModal
        file={selectedFile}
        onClose={() => setSelectedFile(null)}
      />
    </div>
  );
}

function TodoList({ todos }: { todos: Todo[] }) {
  if (todos.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-1">
      {todos.map((todo) => (
        <div
          key={todo.id}
          className={`todo-item ${
            todo.status === "completed"
              ? "todo-item-completed"
              : todo.status === "in_progress"
                ? "todo-item-inprogress"
                : "todo-item-pending"
          }`}
        >
          <span>{/* Check / CircleDot / Circle icon based on status */}</span>
          <span className="text-sm">{todo.content}</span>
        </div>
      ))}
    </div>
  );
}

// FileList — same pattern, items are clickable (onFileClick)
// each row has a download button with e.stopPropagation()
function FileList({
  files,
  onFileClick,
}: {
  files: ResearchFile[];
  onFileClick: (file: ResearchFile) => void;
}) {
  if (files.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-2">
      {files.map((file, i) => (
        <div
          key={`${file.path}-${i}`}
          className="file-item"
          onClick={() => onFileClick(file)}
        >
          <div className="flex items-center gap-3">
            {/* FileText icon */}
            <div>
              <p className="text-sm font-medium">
                {file.path.split("/").pop()}
              </p>
              <p className="text-xs">{file.path}</p>
            </div>
          </div>
          <button
            onClick={(e) => {
              e.stopPropagation();
              downloadFile(file);
            }}
          >
            {/* Download icon */}
          </button>
        </div>
      ))}
    </div>
  );
}

// SourceList — same pattern, colour-codes by source.status 
// (scraped → green check, failed → red X, found → grey circle) 
// each source links to source.url
function SourceList({ sources }: { sources: Source[] }) {
  if (sources.length === 0) return <div>...</div>; // empty state

  return (
    <div className="space-y-2">
      {sources.map((source, i) => (
        <div
          key={`${source.url}-${i}`}
          className={`file-item ${source.status === "failed" ? "source-failed" : ""}`}
        >
          <span>{/* Check / X / Circle based on source.status */}</span>
          <div className="flex-1 min-w-0">
            <p className="text-sm font-medium truncate">
              {source.title || new URL(source.url).hostname}
            </p>
            <a
              href={source.url}
              target="_blank"
              rel="noopener noreferrer"
              className="text-xs truncate block"
            >
              {source.url}
            </a>
          </div>
        </div>
      ))}
    </div>
  );
}

A brief explanation:

TodoList renders each todo with a pending / inprogress / completed icon. Completed items get a strikethrough.
FileList items are clickable: they open FileViewerModal with full Markdown rendering and a download button.
SourceList colour-codes sources: green check for scraped, red X for failed, grey circle for found-but-not-yet-scraped.

Check out the complete code at src/components/WorkSpace.tsx.

✅ FileViewerModal Component

This component renders a modal that displays the contents of a research file written by the Deep Agent. It renders file content as formatted Markdown using the react-markdown package.

Install it using the following command.

npm install react-markdown

Here is the full core implementation:

export function FileViewerModal({ file, onClose }: FileViewerModalProps) {
  const handleKeyDown = useCallback(
    (e: KeyboardEvent) => { if (e.key === "Escape") onClose(); },
    [onClose]
  );

  useEffect(() => {
    if (file) {
      document.addEventListener("keydown", handleKeyDown);
      document.body.style.overflow = "hidden";
    }
    return () => {
      document.removeEventListener("keydown", handleKeyDown);
      document.body.style.overflow = "";
    };
  }, [file, handleKeyDown]);

  if (!file) return null;

  const filename = file.path.split("/").pop() || file.path;

  const handleDownload = () => {
    const blob = new Blob([file.content], { type: "text/markdown" });
    const url = URL.createObjectURL(blob);
    const a = document.createElement("a");
    a.href = url;
    a.download = filename;
    document.body.appendChild(a);
    a.click();
    document.body.removeChild(a);
    URL.revokeObjectURL(url);
  };

  return (
    <div className="fixed inset-0 z-50 flex items-center justify-center p-4">
      <div className="absolute inset-0 bg-black/30 backdrop-blur-sm" onClick={onClose} aria-hidden="true" />
      <div className="relative max-w-3xl w-full max-h-[85vh] flex flex-col" role="dialog" aria-modal="true">
        {/* Header -- filename + download + close */}
        <div className="flex items-center justify-between border-b ...">
          <div className="flex items-center gap-3">
            {/* FileText icon */}
            <h2 className="truncate max-w-md">{filename}</h2>
          </div>
          <div className="flex items-center gap-2">
            <button onClick={handleDownload}>...</button>
            <button onClick={onClose}>...</button>
          </div>
        </div>

        {/* Scrollable Markdown body */}
        <div className="flex-1 overflow-y-auto p-8">
          <div className="prose prose-sm prose-slate max-w-none">
            <ReactMarkdown>{file.content}</ReactMarkdown>
          </div>
        </div>

        {/* Footer -- full file path */}
        <div className="border-t ...">
          <code>{file.path}</code>
        </div>
      </div>
    </div>
  );
}

Here's a brief explanation:

useEffect registers the Escape listener and locks body scroll only when a file is open, and cleans both up on close.
handleDownload creates a temporary anchor element to trigger a browser download without any server round-trip.

Check out the complete code at src/components/FileViewerModal.tsx.

Step 5: Connecting the Chat UI to the Agent

At this point, all the pieces are in place. This page owns ResearchState, passes it to Workspace, uses CopilotChat component for the conversational UI, runs useDefaultTool to intercept every tool call and routes results into state.

research → appends to sources
write_todos → replaces todos
write_file → appends to files

Here's the code.

"use client";

import { useState, useRef } from "react";
import { CopilotChat } from "@copilotkit/react-ui";
import { useDefaultTool } from "@copilotkit/react-core";
import { Workspace } from "@/components/Workspace";
import { ResearchState, INITIAL_STATE, Todo } from "@/types/research";
import { ToolCard } from "@/components/ToolCard";

export default function Page() {
  const [state, setState] = useState<ResearchState>(INITIAL_STATE);
  const processedKeysRef = useRef<Set<string>>(new Set());

  useDefaultTool({
    render: (props) => {
      const { name, status, args, result } = props;

      // Deduplicate on re-renders
      if (status === "complete") {
        const resultStr = result ? JSON.stringify(result) : "";
        const resultHash = resultStr
          ? `${resultStr.length}-${resultStr.slice(0, 100)}`
          : "";
        const key = `${name}-${JSON.stringify(args)}-${resultHash}`;
        if (processedKeysRef.current.has(key)) return <ToolCard {...props} />;
        processedKeysRef.current.add(key);
      }

      if (name === "research" && status === "complete" && result) {
        const researchResult = result as {
          summary: string;
          sources: Array<{
            url: string;
            title: string;
            content?: string;
            status: "found" | "scraped" | "failed";
          }>;
        };
        if (researchResult.sources && researchResult.sources.length > 0) {
          queueMicrotask(() =>
            setState((prev) => ({
              ...prev,
              sources: [...prev.sources, ...researchResult.sources],
            })),
          );
        }
      }

      if (name === "write_todos" && status === "complete" && args?.todos) {
        const todosWithIds = (
          args.todos as Array<{ id?: string; content: string; status: string }>
        ).map((todo, index) => ({
          ...todo,
          id: todo.id || `todo-${Date.now()}-${index}`,
        }));
        queueMicrotask(() =>
          setState((prev) => ({ ...prev, todos: todosWithIds as Todo[] })),
        );
      }

      if (name === "write_file" && status === "complete" && args?.file_path) {
        queueMicrotask(() =>
          setState((prev) => ({
            ...prev,
            files: [
              ...prev.files,
              {
                path: args.file_path as string,
                content: args.content as string,
                createdAt: new Date().toISOString(),
              },
            ],
          })),
        );
      }

      return <ToolCard {...props} />;
    },
  });

  return (
    <div className="relative min-h-screen">
      <main className="relative z-10 h-screen flex overflow-hidden">
        {/* Chat panel -- left side */}
        <div className="w-[38%] h-full border-r flex flex-col">
          {/* Header */}
          <div style={{ flex: 1, minHeight: 0, overflow: "hidden" }}>
            <CopilotChat
              className="h-full"
              labels={{
                title: "Deep Research Assistant",
                initial: "What topic would you like me to research?",
                placeholder: "Ask me to research any topic...",
              }}
            />
          </div>
        </div>

        {/* Workspace panel -- right side */}
        <div className="w-[62%] h-full overflow-hidden">
          <Workspace state={state} />
        </div>
      </main>
    </div>
  );
}

A few things worth noting:

useDefaultTool intercepts every tool call and renders a ToolCard inline inside CopilotChat.
queueMicrotask defers setState so it never fires mid-render.
processedKeysRef deduplicates results since the render callback fires multiple times as status updates stream in.

4. Backend: Building the Agent Service (FastAPI + Deep Agents + Tavily)

We will now build the FastAPI backend that hosts our Deep Agent.

Under the /agent directory lives a FastAPI server that runs the research agent. Here's the project structure of the backend.

agent/
├── main.py           ← FastAPI server + AG-UI
├── agent.py          ← Deep Agents graph
├── tools.py          ← Tavily search tools
├── pyproject.toml    ← Python deps (uv)
├── uv.lock
└── .env

At a high level, the backend:

Builds a Deep Agent graph
Uses Tavily for real-time web search
Wraps research in an isolated sub-agent
Exposes a CopilotKit-compatible AG-UI endpoint
Streams tool calls to the frontend

The backend uses uv for dependency management. Install it if you don't have it:

pip install uv

Initialize a new uv project. This will generate a fresh pyproject.toml.

cd agent
uv init
uv python pin 3.12

Then install the dependencies:

uv add copilotkit deepagents fastapi langchain langchain-openai python-dotenv tavily-python "uvicorn[standard]" ag-ui-langgraph

copilotkit : connects agents to a frontend with streaming, tools and shared state.
deepagents : planning-first agent framework for multi-step execution
fastapi : web framework that exposes the agent API
langchain : agent and tool orchestration layer.
langchain-openai : OpenAI model integration for LangChain.
tavily-python : web search for real-time research
ag-ui-langgraph : AG-UI protocol adapter for LangGraph
uvicorn[standard] : ASGI server to run FastAPI

Now run the following command to generate a uv.lock file pinned with exact versions.

uv sync

Add necessary API Keys

Create a .env file under the agent directory and add your OpenAI API Key and Tavily API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
OPENAI_MODEL=gpt-5.2

OpenAI API Key

Tavily API Key

Step 1: Implement Research Tools

Let's define two tools (tools.py):

internet_search is the low-level Tavily wrapper. It runs a search and returns up to max_results formatted results.
research wraps an internal Deep Agent that runs in a separate thread to prevent its streaming events from leaking to the frontend via LangChain callback propagation.

Here is the code.

import os
from typing import Any
from concurrent.futures import ThreadPoolExecutor
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage
from tavily import TavilyClient


def _do_internet_search(query: str, max_results: int = 5) -> list[dict[str, Any]]:
    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("TAVILY_API_KEY not set")

    try:
        client = TavilyClient(api_key=tavily_key)
        results = client.search(
            query=query,
            max_results=max_results,
            include_raw_content=False,
            topic="general",
        )
        return [
            {
                "url": r.get("url", ""),
                "title": r.get("title", ""),
                "content": (r.get("content") or "")[:3000],
            }
            for r in results.get("results", [])
        ]
    except Exception as e:
        return [{"error": str(e)}]


@tool
def internet_search(query: str, max_results: int = 5) -> list[dict[str, Any]]:
    """Search the web and return results with content."""
    return _do_internet_search(query, max_results)


@tool
def research(query: str) -> dict:
    """Research a topic using web search. Returns structured data with sources."""

    from deepagents import create_deep_agent
    from langchain_openai import ChatOpenAI

    def _run_research_isolated():
        search_results = []

        def internet_search_tracked(query: str, max_results: int = 5):
            """Search the web and return results with content."""
            results = _do_internet_search(query, max_results)
            search_results.extend(results)
            return results

        llm = ChatOpenAI(
            model=os.environ.get("OPENAI_MODEL", "gpt-5.2"),
            temperature=0.7,
            api_key=os.environ.get("OPENAI_API_KEY"),
        )

        research_agent = create_deep_agent(
            model=llm,
            system_prompt="""You are a Research Specialist.
Use internet_search to find information. Return a prose summary of findings.
Rules:
- Call internet_search ONCE with a focused query
- Return a brief summary (2-3 sentences) of key findings
- No JSON, no code blocks, just prose""",
            tools=[internet_search_tracked],
        )

        result = research_agent.invoke({"messages": [HumanMessage(content=query)]})

        summary = result["messages"][-1].content
        sources = [
            {
                "url": r["url"],
                "title": r.get("title", ""),
                "content": r.get("content", "")[:3000],
                "status": "found",
            }
            for r in search_results
            if "url" in r and not r.get("error")
        ]

        return {"summary": summary, "sources": sources}

    with ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(_run_research_isolated)
        return future.result()

Here's what's happening:

_do_internet_search caps each result at 3000 chars to stay within token limits and returns an error dict instead of crashing.
internet_search_tracked extends search_results on every call, building a clean source list to return alongside the final summary.
research_agent is constrained to call internet_search once -- prevents looping and keeps latency predictable.
ThreadPoolExecutor isolates the subagent in a separate thread so its stream events don't leak into the frontend chat.

Step 2: Define the Agent Behavior

This is the brain of the system. The agent is a Deep Agents graph that:

Creates a research plan using write_todos
Calls research() for each research question
Writes a final report to /reports/final_report.md

Here is the main system prompt that enforces this workflow:

MAIN_SYSTEM_PROMPT = """You are a Deep Research Assistant, an expert at planning and
executing comprehensive research on any topic.

Hard rules (ALWAYS follow):
- NEVER output raw JSON, data structures, or code blocks in your messages
- Communicate with the user only in natural, readable prose
- When you receive data from research, synthesize it into insights

Your workflow:
1. PLAN: Create a research plan using write_todos with clear, actionable steps
2. RESEARCH: Use research(query) tool to investigate each topic
3. SYNTHESIZE: Write a final report to /reports/final_report.md using write_file

Important guidelines:
- Always start by creating a research plan with write_todos
- Call research() for each distinct research question
- The research tool returns prose summaries of findings
- You write all files - compile findings into a comprehensive report
- Update todos as you complete each step

Example workflow:
1. write_todos(["Research topic A", "Research topic B", "Synthesize findings"])
2. research("Find information about topic A") -> receives prose summary
3. research("Find information about topic B") -> receives prose summary
4. write_file("/reports/final_report.md", "# Research Report\n\n...")

Always maintain a professional, comprehensive research style."""

Here is the core agent graph (build_agent):

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from deepagents import create_deep_agent
from langgraph.checkpoint.memory import MemorySaver
from copilotkit import CopilotKitMiddleware
from tools import research

load_dotenv()

def build_agent():
    api_key = os.environ.get("OPENAI_API_KEY")
    if not api_key:
        raise RuntimeError("Missing OPENAI_API_KEY environment variable")

    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("Missing TAVILY_API_KEY environment variable")

    model_name = os.environ.get("OPENAI_MODEL", "gpt-5.2")
    llm = ChatOpenAI(
        model=model_name,
        temperature=0.7,
        api_key=api_key,
    )

    agent_graph = create_deep_agent(
        model=llm,
        system_prompt=MAIN_SYSTEM_PROMPT,
        tools=[research],
        middleware=[CopilotKitMiddleware()],
        checkpointer=MemorySaver(),
    )

    return agent_graph.with_config({"recursion_limit": 100})

Here's what's happening:

CopilotKitMiddleware() enables AG-UI streaming to the frontend
MemorySaver() enables stateful execution
Recursion limit increased to support multi-step research
research is the only explicit tool since write_todos, read_file, write_file are built into Deep Agents automatically

Step 3: Expose the Agent via FastAPI + AG-UI

The last step is to wire everything together and expose it as a FastAPI app. The main.py builds the agent graph, configures which tool calls to emit to the frontend and registers the AG-UI endpoint.

import os
import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from dotenv import load_dotenv
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent
from copilotkit.langgraph import copilotkit_customize_config

from agent import build_agent

load_dotenv()

app = FastAPI(title="Deep Research Assistant", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.get("/health")
def health():
    return {"status": "ok", "service": "deep-research-agent"}

agent_graph = build_agent()

# Only emit main agent tools — suppress internal subagent tool calls
agui_config = copilotkit_customize_config(
    emit_tool_calls=["research", "write_todos", "write_file", "read_file", "edit_file"]
)
agui_config["recursion_limit"] = 100

add_langgraph_fastapi_endpoint(
    app=app,
    agent=LangGraphAGUIAgent(
        name="research_assistant",
        description="A deep research assistant that plans, searches, and synthesizes research reports",
        graph=agent_graph,
        config=agui_config,
    ),
    path="/",
)

def main():
    uvicorn.run(
        "main:app",
        host=os.getenv("SERVER_HOST", "0.0.0.0"),
        port=int(os.getenv("SERVER_PORT", "8123")),
        reload=True,
    )

if __name__ == "__main__":
    main()

The emit_tool_calls config controls exactly which tool calls CopilotKit streams to the frontend. Without it, the subagent's internal internet_search calls leak through as raw JSON noise in the chat UI.

5. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the agent/.env.

From the project root, navigate to the agent directory and start the FastAPI server:

cd agent
uv run python main.py

The backend will start on http://localhost:8123.

In a new terminal window, start the frontend development server using:

npm run dev

Once both servers are running, open the frontend in your browser at http://localhost:3000/ to view it locally.

CopilotKit also provides the Agent Inspector, which is a live AG-UI runtime view that lets you inspect agent runs, state snapshots, messages and tool calls as they stream from the backend. It's accessible from a copilotkit button overlaid on your app.

Here is the complete demo!

6. Data flow

Now that we have built both the frontend and the agent service, this is how data actually flows between them. This should be easy to follow if you have been building along so far.

[User sends research question]
        ↓
Next.js Frontend (CopilotChat + Workspace)
        ↓
POST /api/copilotkit (AG-UI protocol)
        ↓
CopilotKit Runtime → LangGraphHttpAgent (localhost:8123)
        ↓
FastAPI Backend (AG-UI endpoint "/")
        ↓
Deep Agent (research_assistant)
        ↓
Deep Agents orchestration
    ├── write_todos  → todos state   → Workspace (Research Plan)
    ├── research()   → sources state → Workspace (Sources)
    │       └── internal Deep Agent [thread-isolated]
    │               └── internet_search (Tavily API)
    └── write_file   → files state   → Workspace (Files)
        ↓
AG-UI streaming (SSE)
        ↓
useDefaultTool intercepts tool calls
        ↓
ToolCard renders in chat + Workspace updates live

That's it! 🎉

You now have a fully working Deep Agents research assistant that plans, searches, and writes reports, with every step visible in the frontend as it happens.

The real win here is visibility. CopilotKit's AG-UI layer turns what would otherwise be a black-box agent into something users can actually follow and trust.

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

How to Build a LangGraph Research Agent that Embeds Dynamic Charts via MCP Apps (CopilotKit & Tako)

Anmol Baranwal — Tue, 03 Feb 2026 15:49:30 +0000

AI research assistants are getting smarter, but turning that reasoning into something usable is still hard.

Real research involves more than text: searching the web, exploring structured data, comparing sources, and gradually assembling a report. Doing that well requires state, tools, and UI -- not just a chat box.

Today, we will build a chat + canvas research assistant that searches the web, queries structured data via MCP, and embeds interactive charts directly into the report, while streaming progress to the frontend in real time.

We will use CopilotKit (Next.js) for the frontend, LangGraph to orchestrate the agent workflow, and MCP (Model Context Protocol) to connect external tools like Tako, which returns embeddable chart UI.

Check out CopilotKit's GitHub ⭐️

You will find architecture, key concepts, how state flows between the UI and the agent, how MCP Apps fit into the system, and a step-by-step guide to building this from scratch.

Let’s build it.

What is covered?

In summary, we will cover these topics in detail.

What are we building?
Tech Stack and Project Structure
Building the Frontend (CopilotKit)
Agent Orchestration (LangGraph)
Backend: MCP Integration & Tools
Running the Application
Complete Data Flow between UI ↔ agent

Here’s the GitHub Repo if you are interested in exploring it yourself.

1. What are we building?

We are building an AI research assistant with a live chat + canvas UI that searches, collects sources, and assembles a report while streaming progress to the frontend in real time.

It will:

Turn a user question into a research plan (research question + data sub-questions)
Run parallel retrieval: Tavily for web search + Tako via MCP apps for charts
Deduplicate results into a clean resources list
Generate a report with [CHART:title] placeholders and then embed the real charts inline
Stream intermediate state (logs/resources/report/charts) so the UI updates as the agent works.

We will see all of these concepts in action as we build the agent.

Core Components

A few days ago, the Anthropic team shipped MCP Apps as an official MCP extension, meaning tools can return interactive UI that the host can render, not just JSON or text.

That’s the pattern we will use here: our agent will call Tako (over MCP) to fetch charts as embeddable UI, so interactive visualizations can drop straight into the report/canvas instead of us building chart components.

At a high level, this project is built by combining:

CopilotKit for the agent chat + canvas UI
AG‑UI for streaming agent ↔ UI events (protocol)
LangGraph for stateful agent orchestration
MCP Apps for external tools
Tako for charts (search engine)
Tavily for web search

For context: AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes this “agent ↔ frontend” real-time communication.

Because it’s transport-agnostic (SSE, WebSockets, etc.), the same frontend can work across different agent backends without custom wiring. CopilotKit uses AG-UI as its synchronization layer.

Here's a simplified call request → response flow of what will happen:

User question
   ↓
CopilotKit UI (chat + canvas)
   ↓
LangGraph agent workflow
   ├─ chat: interpret intent + plan
   ├─ search: parallel retrieval (Tavily + Tako via MCP Apps)
   └─ report: narrative + [CHART:*] markers
   ↓
Streaming state updates (AG‑UI events)
   ↓
Canvas renders report + embedded charts

2. Tech Stack and Architecture

At the core, we are going to use this stack for building the agent:

Next.js 15 : frontend framework with TypeScript
CopilotKit 1.50 : agent UI components + streaming runtime (@copilotkit/react-core, @copilotkit/react-ui, @copilotkit/runtime)
LangGraph: stateful agent orchestration (StateGraph nodes, conditional routing)
FastAPI & Uvicorn : HTTP server for the agent
Tavily: web search API (tavily-python)
Model Context Protocol (MCP): connects the agent to Tako (mcp, httpx)
CopilotKit LangGraph SDK (Python): streams LangGraph state to CopilotKit (copilotkit)
OpenAI: LLM provider (openai)

There are also other libraries used, like react-markdown & remark-gfm for report rendering, langgraph-checkpoint-sqlite for state persistence, and Shadcn UI for components.

See package.json and agents/python/pyproject.toml in the repo for the complete dependency list.

Project structure

This is how our directory will look.

The agents/python/ directory hosts the Python LangGraph agent, exposed via FastAPI, which orchestrates the workflow (chat, search, report), streams state updates, and calls external tools (Tavily, Tako via MCP).

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the agent backend.

.
├── src/                           ← Next.js frontend (TypeScript)
│   ├── app/
│   │   ├── page.tsx               ← CopilotKit provider & model selector
│   │   ├── Main.tsx               ← Chat + ResearchCanvas split layout
│   │   └── api/
│   │       └── copilotkit/
│   │           └── route.ts      ← CopilotRuntime bridge to agent
│   ├── components/
│   │   ├── ResearchCanvas.tsx    ← Main canvas (orchestrates report + resources)
│   │   ├── Resources.tsx         ← Displays Tavily + Tako resources list
│   │   ├── MarkdownRenderer.tsx  ← Renders report + embeds charts
│   │   └── ui/                   ← Reusable Shadcn UI components
│   └── lib/
│       ├── types.ts                     ← AgentState type
│       ├── utils.ts                     ← Utility functions
│       └── model-selector-provider.tsx  ← Model selection context
│
├── agents/
│   ├── python/                        ← Python LangGraph agent (primary)
│   │   ├── src/
│   │   │   ├── agent.py               ← StateGraph definition & compile
│   │   │   └── lib/
│   │   │       ├── state.py           ← AgentState (Pydantic)
│   │   │       ├── model.py       ← LLM factory (OpenAI / Anthropic / Gemini)
│   │   │       ├── chat.py            ← Chat node & tool definitions
│   │   │       ├── search.py          ← Parallel Tavily + Tako search
│   │   │       ├── mcp_integration.py ← MCP client & iframe helpers
│   │   │       ├── download.py        ← Download node
│   │   │       └── delete.py          ← Delete node
│   │   ├── main.py                    ← FastAPI/Uvicorn entrypoint
│   │   ├── requirements.txt
│   │   └── pyproject.toml
│   │
│   └── src/                       ← TypeScript agent (optional, local dev)
│       └── server.ts              ← Express + CopilotRuntime
│
├── package.json                   ← Frontend deps & scripts
├── .env.example
├── .env.local
└── README.md

Here's the GitHub repository and deployed live at tako-copilotkit.vercel.app if you want to explore yourself. I will cover the implementation and all key concepts in the following sections.

The easiest way to follow along is to clone the repo.

git clone https://github.com/TakoData/tako-copilotkit.git
cd tako-copilotkit

Add necessary API Keys

You can copy the environment template (.env.example) by using this command to create a .env.local in the root directory.

cp .env.example .env.local

Add your OpenAI API Key, Tavily API Key and Tako API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
TAKO_API_TOKEN=your_api_token_here
TAKO_MCP_URL=https://mcp.tako.com  # URL of Tako's MCP server
TAKO_URL=https://tako.com  # URL of Tako's main API

The Tako/MCP integration is an optional data source for the research agent. If you want to query charts or structured datasets via a Tako MCP server, provide TAKO_API_TOKEN and the related URLs.

Otherwise, you can leave these unset & the agent will continue to work normally. For this tutorial, we will be using it.

OpenAI API Key

Tavily API Key

Tako API Key

3. Frontend: wiring the agent to the UI

Let's first build the frontend part.

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the Python agent backend.

At a high level, the frontend is responsible for:

Sending user queries to the agent backend via CopilotChat
Receiving and rendering real-time state updates from the agent
Embedding Tako chart iframes in the rendered report
Managing the CopilotKit runtime bridge between UI and Python agent

If you are building this from scratch, the easiest approach is to copy the existing package.json from the repo.

It already includes all the required dependencies for CopilotKit, LangGraph integration, UI components, and local dev tooling, so you don’t have to assemble them manually.

I'm only covering the core frontend dependencies you actually need to understand.

Step 1: CopilotKit Provider & Layout

Install the necessary packages:

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core: Core React hooks and context that connect your UI to the agent backend via AG-UI protocol
@copilotkit/react-ui: Ready-made UI components like <CopilotChat /> for building AI chat interfaces
@copilotkit/runtime: Server-side runtime that exposes an API endpoint and bridges the frontend with the LangGraph agent using HTTP and SSE

Everything else in the repo (radix, tailwind, react-split, etc.) is there to support layout, styling, and developer experience -- not the core agent wiring.

The <CopilotKit> component must wrap the agent-aware parts of your application and point to the runtime endpoint so it can communicate with the agent backend.

This is the main entry point (page.tsx):

// page.tsx
"use client";

import { CopilotKit } from "@copilotkit/react-core";
import Main from "./Main";
import {
  ModelSelectorProvider,
  useModelSelectorContext,
} from "@/lib/model-selector-provider";

export default function ModelSelectorWrapper() {
  return (
    <ModelSelectorProvider>
      <Home />
    </ModelSelectorProvider>
  );
}

function Home() {
  const { agent, lgcDeploymentUrl } = useModelSelectorContext();

  // This logic is implemented to demonstrate multi-agent frameworks in this demo project.
  // There are cleaner ways to handle this in a production environment.
  const runtimeUrl = lgcDeploymentUrl
    ? `/api/copilotkit?lgcDeploymentUrl=${lgcDeploymentUrl}`
    : `/api/copilotkit${
        agent.includes("crewai") ? "?coAgentsModel=crewai" : ""
      }`;

  return (
    <div style={{ height: "100vh", overflow: "hidden" }}>
      <CopilotKit runtimeUrl={runtimeUrl} showDevConsole={false} agent={agent}>
        <Main />
      </CopilotKit>
    </div>
  );
}

Here, ModelSelectorProvider is a convenience for switching agents/models in development.

Step 2: Chat + Canvas Layout

The Main.tsx component sets up the core UI. It sets up:

the chat interface (CopilotChat)
the research canvas (where results & charts appear)
state binding to the agent

// app/Main.tsx
import { useCoAgent } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import Split from "react-split";
import { ResearchCanvas } from "@/components/ResearchCanvas";
import { ChatInputWithModelSelector } from "@/components/ChatInputWithModelSelector";
import { AgentState } from "@/lib/types";
import { useModelSelectorContext } from "@/lib/model-selector-provider";

export default function Main() {
  const { model, agent } = useModelSelectorContext();

  const { state, setState } = useCoAgent<AgentState>({
    name: agent,
    initialState: {
      model,
      research_question: "",
      resources: [],
      report: "",
      logs: [],
    },
  });

  return (
    <div style={{ height: "100%", display: "flex", flexDirection: "column" }}>
      <h1>Research Helper</h1>

      <Split sizes={[30, 70]} style={{ flex: 1, display: "flex" }}>
        {/* Chat Panel */}
        <CopilotChat
          Input={ChatInputWithModelSelector}
          onSubmitMessage={async () => {
            setState({ ...state, logs: [] });
            await new Promise((r) => setTimeout(r, 30));
          }}
        />

        {/* Canvas Panel */}
        <ResearchCanvas />
      </Split>
    </div>
  );
}

Here's what's happening:

useCoAgent<AgentState>: bi-directional state synchronization between UI and agent. When the agent emits state updates via copilotkit_emit_state(), this hook automatically picks them up.
CopilotChat: drop-in chat UI that sends messages to the agent and renders tool calls inline
ResearchCanvas: custom component that renders the streaming report, resources list, and embedded charts
Split: provides a resizable split-pane layout

Step 3: Building Key Components

I'm only covering the core logic behind the main components since the overall code is huge. You can find all the components in the repository at src\components.

These components use CopilotKit hooks (like useCoAgentStateRender) to tie everything together.

✅ Research Canvas Component

ResearchCanvas.tsx is where the agent’s current state is rendered:

the accumulating report text
linked resources
embedded charts (iframes from MCP Apps like Tako)

This component listens to the agent state and displays elements as they arrive. It translates [CHART:title] markers into real embedded charts. This pattern is part of CopilotKit's support for generative UI.

// core logic
import { useCoAgent, useCoAgentStateRender, useCopilotAction } from "@copilotkit/react-core";
import { MarkdownRenderer } from "./MarkdownRenderer";
import { AgentState } from "@/lib/types";
import { useRef, useEffect } from "react";

export function ResearchCanvas() {
  const { state, setState } = useCoAgent<AgentState>({
    name: "research_agent",
  });

  // Use refs to prevent flicker during streaming updates
  const lastReportRef = useRef<string>("");
  const lastResourcesRef = useRef<Resource[]>([]);

  if (state.report) lastReportRef.current = state.report;
  if (state.resources?.length) lastResourcesRef.current = state.resources;

  const report = state.report || lastReportRef.current;
  const resources = state.resources || lastResourcesRef.current;

  // Render progress logs during execution
  useCoAgentStateRender({
    name: "research_agent",
    render: ({ state, status }) => {
      if (state.logs?.length) return <Progress logs={state.logs} />;
      return null;
    },
  });

  // Generative UI: Agent requests deletion confirmation from user
  useCopilotAction({
    name: "DeleteResources",
    description: "Prompt user for resource delete confirmation",
    available: "remote",
    parameters: [{ name: "urls", type: "string[]" }],
    renderAndWait: ({ args, handler }) => (
      <div>
        <h3>Delete these resources?</h3>
        <Resources resources={resources.filter(r => args.urls.includes(r.url))} />
        <button onClick={() => handler("YES")}>Delete</button>
        <button onClick={() => handler("NO")}>Cancel</button>
      </div>
    ),
  });

  return (
    <div>
      <h2>Research Question</h2>
      <div>{state.research_question || "Agent will identify your question..."}</div>

      {/* Resources Panel */}
      <Resources resources={resources} />

      {/* Report Panel with Embedded Charts */}
      <MarkdownRenderer content={report} />
    </div>
  );
}

The full component also includes collapsible sections for Tako charts vs web resources, and resource management dialogs. Check src/components/ResearchCanvas.tsx.

✅ Chart Embedding Pattern

The MCP Apps-style embedding happens in MarkdownRenderer.tsx. It embeds Tako charts without flickering during streaming updates. Instead of simple regex replacement, it uses a stable iframe pattern.

// core logic
import React, { useEffect, useRef, useMemo } from "react";
import ReactMarkdown from "react-markdown";

// Global registry: iframe src never changes for a given ID
const iframeRegistry = new Map<string, string>();

export function MarkdownRenderer({ content }: { content: string }) {
  // Split content into text and iframe segments
  const segments = useMemo(() => {
    const embedPattern = /<!doctype html>[\s\S]*?<\/html>/gi;
    const parts = [];
    let lastIndex = 0;

    content.replace(embedPattern, (match, offset) => {
      // Add text before iframe
      if (offset > lastIndex) {
        parts.push({ type: 'text', content: content.slice(lastIndex, offset) });
      }

      // Extract iframe src and create stable ID
      const src = match.match(/src=["']([^"']+)["']/)?.;
      if (src) {
        const id = `embed-${src.replace(/[^a-zA-Z0-9]/g, '-').slice(0, 50)}`;
        iframeRegistry.set(id, src);
        parts.push({ type: 'iframe', id, src });
      }

      lastIndex = offset + match.length;
      return match;
    });

    // Add remaining text
    if (lastIndex < content.length) {
      parts.push({ type: 'text', content: content.slice(lastIndex) });
    }

    return parts;
  }, [content]);

  return (
    <div>
      {segments.map((segment, i) => 
        segment.type === 'text' 
          ? <ReactMarkdown key={i}>{segment.content}</ReactMarkdown>
          : <StableIframe key={segment.id} src={segment.src} />
      )}
    </div>
  );
}

// Memoized iframe - never re-renders once mounted
const StableIframe = React.memo(({ src }: { src: string }) => {
  const iframeRef = useRef<HTMLIFrameElement>(null);

  // Handle Tako resize messages
  useEffect(() => {
    const handleResize = (event: MessageEvent) => {
      if (event.data?.type === "tako::resize" && 
          iframeRef.current?.contentWindow === event.source) {
        iframeRef.current.style.height = `${event.data.height}px`;
      }
    };
    window.addEventListener("message", handleResize);
    return () => window.removeEventListener("message", handleResize);
  }, []);

  return <iframe ref={iframeRef} src={src} style={{ height: "400px" }} />;
});

Tako returns fully-formed chart UI via MCP Apps so the frontend only needs to embed the iframe. Here's what's happening:

Iframes are rendered as stable siblings (outside ReactMarkdown), preventing remounts during streaming updates
Each embed uses a deterministic ID derived from its src, ensuring consistent DOM identity
Resize events from Tako (tako::resize) are handled via postMessage to keep charts responsive
rehypeRaw: Allows ReactMarkdown to render raw HTML (the iframe elements)

Check out the complete code at src/components/MarkdownRenderer.tsx.

✅ Model Selector Provider

The ModelSelectorProvider (src\lib\model-selector-provider.tsx) is a convenience wrapper for switching between different agent configurations during development. It reads URL query parameters to determine which agent/model to use.

"use client";

import React from "react";
import { createContext, useContext, useState, ReactNode } from "react";

type ModelSelectorContextType = {
  model: string;
  setModel: (model: string) => void;
  agent: string;
  lgcDeploymentUrl?: string | null;
  hidden: boolean;
  setHidden: (hidden: boolean) => void;
};

const ModelSelectorContext = createContext<ModelSelectorContextType | undefined>(undefined);

export const ModelSelectorProvider = ({ children }: { children: ReactNode }) => {
  // Read model from URL query params (e.g., ?coAgentsModel=google_genai)
  const model =
    globalThis.window === undefined
      ? "openai"
      : new URL(window.location.href).searchParams.get("coAgentsModel") ?? "openai";

  const [hidden, setHidden] = useState<boolean>(false);

  const setModel = (model: string) => {
    const url = new URL(window.location.href);
    url.searchParams.set("coAgentsModel", model);
    window.location.href = url.toString();
  };

  // Optional: LangGraph Cloud deployment URL for production
  const lgcDeploymentUrl =
    globalThis.window === undefined
      ? null
      : new URL(window.location.href).searchParams.get("lgcDeploymentUrl");

  // Map model to agent name
  let agent = "research_agent";
  if (model === "google_genai") {
    agent = "research_agent_google_genai";
  } else if (model === "crewai") {
    agent = "research_agent_crewai";
  }

  return (
    <ModelSelectorContext.Provider
      value={{ model, agent, lgcDeploymentUrl, hidden, setModel, setHidden }}
    >
      {children}
    </ModelSelectorContext.Provider>
  );
};

export const useModelSelectorContext = () => {
  const context = useContext(ModelSelectorContext);
  if (context === undefined) {
    throw new Error("useModelSelectorContext must be used within a ModelSelectorProvider");
  }
  return context;
};

The repo also includes UI components like Resources.tsx (resource cards), Progress.tsx (logs display), EditResourceDialog.tsx and AddResourceDialog.tsx for resource management, plus ChatInputWithModelSelector for model switching during development.

Step 4: Runtime Bridge (API Route)

This Next.js API route acts as a thin proxy between the browser and the Python LangGraph agent. It:

Accepts CopilotKit requests from the UI
Forwards them to the LangGraph agent via HTTP
Streams agent state and events back to the frontend via SSE

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  copilotRuntimeNextJSAppRouterEndpoint,
  EmptyAdapter,
} from "@copilotkit/runtime";
import {
  LangGraphHttpAgent,
  LangGraphAgent,
} from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
// const llmAdapter = new OpenAIAdapter({ openai } as any);
const llmAdapter = new EmptyAdapter();
const langsmithApiKey = process.env.LANGSMITH_API_KEY as string;

export const POST = async (req: NextRequest) => {
  const searchParams = req.nextUrl.searchParams;
  const deploymentUrl =
    searchParams.get("lgcDeploymentUrl") || process.env.LGC_DEPLOYMENT_URL;

  const baseUrl =
    process.env.REMOTE_ACTION_URL || "http://localhost:2024/copilotkit";

  let runtime = new CopilotRuntime({
    agents: {
      research_agent: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent`,
      }),
      research_agent_google_genai: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent_google_genai`,
      }),
    },
  });

  if (deploymentUrl) {
    runtime = new CopilotRuntime({
      agents: {
        research_agent: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent",
        }),
        research_agent_google_genai: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent_google_genai",
        }),
      },
    });
  }

  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter: llmAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

The frontend is fully reactive: as the agent works, the UI updates in real time.

4. Agent Orchestration (LangGraph)

Now that the frontend can communicate with an agent, we need to define how the agent thinks and operates. The backend agent lives in agents/python/src/.

It uses LangGraph to orchestrate a multi-step research workflow (chat → search → download → report...) through well-defined steps (nodes), each responsible for a specific task.

Each step:

receives the current AgentState
performs one responsibility
returns partial state updates
decides what node runs next

Step 1: Define the StateGraph (agent.py)

The workflow is defined using LangGraph's StateGraph in agent.py. It models the research assistant as a stateful workflow, where each step (node) operates on shared state and decides which step to run next.

At a high level:

AgentState is the shared memory across the entire agent (messages, report, resources, logs, ...)
Each node is an async function that:
- reads from the current state
- updates part of that state
- returns control-flow instructions
Edges control execution flow between nodes (download → chat → search → download loop)

One important detail is the interrupt_after hook, which allows the agent to pause execution for user confirmation before deleting a resource (delete_node).

# agents/python/src/agent.py
import os

from langgraph.graph import StateGraph

from src.lib.chat import chat_node
from src.lib.delete import delete_node, perform_delete_node
from src.lib.download import download_node
from src.lib.search import search_node
from src.lib.state import AgentState

workflow = StateGraph(AgentState)
workflow.add_node("download", download_node)
workflow.add_node("chat_node", chat_node)
workflow.add_node("search_node", search_node)
workflow.add_node("delete_node", delete_node)
workflow.add_node("perform_delete_node", perform_delete_node)


workflow.set_entry_point("download")
workflow.add_edge("download", "chat_node")
workflow.add_edge("delete_node", "perform_delete_node")
workflow.add_edge("perform_delete_node", "chat_node")
workflow.add_edge("search_node", "download")

# Compile with interrupt support for deletion confirmation
compile_kwargs = {"interrupt_after": ["delete_node"]}

# Check if we're running in LangGraph API mode
if os.environ.get("LANGGRAPH_FASTAPI", "false").lower() == "false":
    # When running in LangGraph API, don't use a custom checkpointer
    graph = workflow.compile(**compile_kwargs)
else:
    # For CopilotKit and other contexts, use MemorySaver
    from langgraph.checkpoint.memory import MemorySaver

    memory = MemorySaver()
    compile_kwargs["checkpointer"] = memory
    graph = workflow.compile(**compile_kwargs)

Why this structure works well:

The download → chat → search → download loop naturally supports iterative research
Optional checkpointing (MemorySaver) allows the same graph to run locally or on LangGraph Cloud (state persistence)

Step 2: Chat Node & Tools (chat.py)

The chat node (chat_node) is the brain of the agent. It orchestrates the LLM, defines tools, manages state streaming, and handles chart injection. It:

Define LLM-callable tools (Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions)
Configure intermediate state emission (stream research question and data questions as they are generated)
Build prompts with available resources and Tako chart context
Inject Tako chart iframes into the report after generation
Route to next node based on tool calls

Here are the core tools (agents/python/src/lib/chat.py).

@tool
def Search(queries: List[str]):
    """A list of one or more search queries to find good resources."""

@tool
def WriteReport(report: str):
    """Write the research report."""

@tool
def WriteResearchQuestion(research_question: str):
    """Write the research question."""

@tool
def GenerateDataQuestions(questions: List[DataQuestion]):
    """
    Generate 3-6 ATOMIC data-focused questions for Tako search.

    CRITICAL: Each query should ask for ONE metric/dimension only.
    BAD: "Compare US GDP and inflation" → compound query
    GOOD: "US GDP 2020-2024", "US inflation rate" → separate atomic queries
    """

Here is the chat node logic.

async def chat_node(state: AgentState, config: RunnableConfig):
    # Configure intermediate state emission
    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[
            {"state_key": "research_question", "tool": "WriteResearchQuestion", "tool_argument": "research_question"},
            {"state_key": "data_questions", "tool": "GenerateDataQuestions", "tool_argument": "questions"},
        ],
    )

    # Build Tako charts context for LLM
    tako_charts_map = {}
    available_tako_charts = []
    for resource in state["resources"]:
        if resource.get("resource_type") == "tako_chart":
            title = resource.get("title", "")
            card_id = resource.get("card_id")
            description = resource.get("description", "")
            tako_charts_map[title] = {"card_id": card_id, "embed_url": resource.get("embed_url")}
            available_tako_charts.append(f"  - **{title}**\n    Description: {description}")

    # Build prompt with workflow instructions
    response = await model.bind_tools([Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions]).ainvoke([
        SystemMessage(content=f"""
        You are a research assistant. 

        WORKFLOW:
        1. Use WriteResearchQuestion to extract the core research question
        2. Use GenerateDataQuestions to create 3-6 ATOMIC data questions (one metric per query)
        3. Use Search for web resources
        4. Write a comprehensive report using WriteReport

        AVAILABLE CHARTS ({len(tako_charts_map)}):
        {available_tako_charts_str}

        WRITING GUIDELINES:
        - Write a detailed analysis with substantial narrative text
        - For EACH chart, write 1-2 paragraphs discussing insights and trends
        - DO NOT include chart markers or image syntax - charts will be inserted automatically
        - Reference specific data points from chart descriptions
        """),
        *state["messages"],
    ], config)

    # Handle tool calls
    if response.tool_calls:
        if response.tool_calls[0]["name"] == "WriteReport":
            report = response.tool_calls[0]["args"]["report"]

            # Clean up any markdown images LLM incorrectly added
            report = re.sub(r'!\[[^\]]*\]\([^)]+\)', '', report)

            # Inject charts using second LLM pass
            if tako_charts_map:
                chart_list = "\n".join([f"- {title}" for title in tako_charts_map.keys()])

                inject_response = await model.ainvoke([
                    SystemMessage(content=f"""Insert [CHART:exact_title] markers at appropriate positions.

                    AVAILABLE CHARTS:
                    {chart_list}

                    RULES:
                    - Place markers AFTER relevant paragraphs
                    - NEVER place more than two charts consecutively
                    - Each chart is used exactly once
                    """),
                    HumanMessage(content=f"Insert chart markers:\n\n{report}")
                ])

                report_with_markers = inject_response.content

                # Replace markers with actual iframe HTML
                async def replace_marker(match):
                    chart_title = match.group(1).strip()
                    chart_info = tako_charts_map.get(chart_title)
                    if chart_info:
                        iframe_html = await get_visualization_iframe(
                            item_id=chart_info.get("card_id"),
                            embed_url=chart_info.get("embed_url")
                        )
                        return "\n" + iframe_html + "\n" if iframe_html else ""
                    return ""

                # Apply replacements
                markers = list(re.finditer(r'\[CHART:([^\]]+)\]', report_with_markers))
                processed_report = report_with_markers
                for match in reversed(markers):
                    replacement = await replace_marker(match)
                    processed_report = processed_report[:match.start()] + replacement + processed_report[match.end():]

            return Command(goto="chat_node", update={"report": processed_report})

        elif response.tool_calls[0]["name"] == "GenerateDataQuestions":
            data_questions = response.tool_calls[0]["args"]["questions"]
            return Command(goto="search_node", update={"data_questions": data_questions})

    # Route based on tool call
    goto = "__end__"
    if response.tool_calls:
        tool_name = response.tool_calls[0]["name"]
        if tool_name == "Search":
            goto = "search_node"
        elif tool_name == "DeleteResources":
            goto = "delete_node"

    return Command(goto=goto, update={"messages": response})

How this node works in practice:

The LLM never writes free-form text directly. Every meaningful action goes through an explicit tool (WriteResearchQuestion, GenerateDataQuestions, Search, WriteReport). This keeps state changes deterministic and easy to reason about.
Intermediate state (research question and data questions) is streamed immediately using copilotkit_customize_config. The report itself is intentionally not streamed to avoid UI flicker while charts are being injected later.
Chart handling is deliberately two-phase: the model first writes a clean narrative report, then a second pass inserts [CHART:title] markers, which are finally replaced with real Tako iframes. To keep the generation stable and layout-safe.
Control flow is explicit. Each tool call results in a Command that routes execution to the next node (search_node, delete_node, or back to chat_node), making the agent’s behavior predictable and easier to debug.

Step 3: Search Node (search.py)

The search node (search_node) runs web search (Tavily) and structured chart data search (Tako via MCP) in parallel, streams progress to the UI, deduplicates results, and extracts the most relevant resources.

Tavily and Tako searches run concurrently using asyncio.gather, so web pages and structured datasets are fetched at the same time
TavilyClient is synchronous (uses requests library), so we wrap it with loop.run_in_executor() to avoid blocking the async event loop
As each search completes, copilotkit_emit_state is called to update state.logs, allowing the frontend to show real-time progress
Tako chart results are deduplicated using the title to avoid embedding the same visualization multiple times
Web results and chart results are passed through an LLM tool (ExtractResources) to select only the most relevant resources
Each resource is tagged as web or tako_chart, which the frontend uses to decide whether to render a link or an embedded chart

Here is the code for Tavily async wrapper (bridging sync client with async workflow):

# agents/python/src/lib/search.py

from tavily import TavilyClient
import asyncio

_tavily_client = None

def get_tavily_client():
    """Lazy initialization to avoid repeated client creation"""
    global _tavily_client
    if _tavily_client is None:
        _tavily_client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))
    return _tavily_client

async def async_tavily_search(query: str) -> Dict[str, Any]:
    """
    Async wrapper for synchronous Tavily client.
    TavilyClient.search() is blocking, so we run it in a thread pool.
    """
    loop = asyncio.get_event_loop()
    return await loop.run_in_executor(
        None,
        lambda: get_tavily_client().search(
            query=query,
            search_depth="advanced",
            include_answer=True,
            max_results=5,
        ),
    )

Search node implementation (parallel execution and resource extraction):

# agents/python/src/lib/search.py (core search pattern)
async def search_node(state: AgentState, config: RunnableConfig):
    """
    Perform Tavily web search and Tako MCP chart search in parallel.
    Stream progress, dedupe results, extract top resources.
    """
    # Extract queries from tool_calls or fall back to research_question
    # (Actual code handles dual routing: Search tool OR GenerateDataQuestions)
    queries = state.get("search_queries", [])
    data_questions = state.get("data_questions", [])

    # PHASE 1: Build parallel tasks
    tavily_tasks = [async_tavily_search(query) for query in queries]
    tako_tasks = [
        search_knowledge_base(q["question"], search_effort="fast")
        for q in data_questions
    ]
    all_tasks = tavily_tasks + tako_tasks

    # Initialize logs for progress tracking
    for query in queries:
        state["logs"].append({"message": f"Web search: {query}", "done": False})
    for q in data_questions:
        state["logs"].append({"message": f"Tako search: {q['question']}", "done": False})

    if all_tasks:
        await copilotkit_emit_state(config, state)

    # PHASE 2: Run all searches concurrently
    all_results = await asyncio.gather(*all_tasks, return_exceptions=True)

    # Split results back into Tavily and Tako
    num_tavily = len(tavily_tasks)
    tavily_results = all_results[:num_tavily]
    tako_results_raw = all_results[num_tavily:]

    # PHASE 3: Stream completion and process results
    search_results = []
    tako_results = []

    # Process Tavily results
    for i, result in enumerate(tavily_results):
        if isinstance(result, Exception):
            search_results.append({"error": str(result)})
        else:
            search_results.append(result)
        state["logs"][i]["done"] = True
        await copilotkit_emit_state(config, state)

    # Process Tako results
    tako_log_offset = num_tavily
    for i, result in enumerate(tako_results_raw):
        if isinstance(result, Exception):
            tako_results.append({"error": str(result)})
        elif result:
            tako_results.extend(result)
        state["logs"][tako_log_offset + i]["done"] = True
        await copilotkit_emit_state(config, state)

    # PHASE 4: Deduplicate Tako charts by title
    # (Same chart may appear in multiple searches)
    seen_titles = set()
    unique_tako = []
    for chart in tako_results:
        title = chart.get("title", "")
        if title and title not in seen_titles:
            seen_titles.add(title)
            unique_tako.append(chart)
        elif not title:
            unique_tako.append(chart)

    # PHASE 5: Use LLM to extract the most relevant resources
    model = get_model(state)

    # Prepare search message with both web and Tako results
    search_message = f"Web search results: {search_results}"
    if unique_tako:
        search_message += f"\n\nTako chart results: {unique_tako}"

    # Build extraction prompt
    extract_messages = [
        SystemMessage(content="""
        Extract the 3-5 most relevant resources from the search results.
        Prioritize Tako charts when they match the research topic.
        """),
        *state["messages"],
        SystemMessage(content=f"Search results:\n{search_message}")
    ]

    # Call LLM with ExtractResources tool
    response = await model.bind_tools(
        [ExtractResources], 
        tool_choice="ExtractResources"
    ).ainvoke(extract_messages, config)

    # PHASE 6: Tag resources with type and attach content
    resources = response.tool_calls[0]["args"]["resources"]

    for resource in resources:
        # Check if this is a Tako chart by matching URL or title
        is_tako = False
        for tako_result in unique_tako:
            if (resource.get("url") == tako_result.get("url") or 
                resource.get("title") == tako_result.get("title")):
                is_tako = True
                resource["resource_type"] = "tako_chart"
                resource["source"] = "Tako"
                resource["card_id"] = tako_result.get("id")
                resource["embed_url"] = tako_result.get("embed_url")
                resource["content"] = tako_result.get("description", "")
                break

        if not is_tako:
            resource["resource_type"] = "web"
            resource["source"] = "Tavily Web Search"
            # Find matching Tavily result for content summary
            for search_result in search_results:
                if isinstance(search_result, dict) and "results" in search_result:
                    for tavily_item in search_result["results"]:
                        if tavily_item.get("url") == resource.get("url"):
                            resource["content"] = tavily_item.get("content", "")
                            break

    # Deduplicate and add to state resources
    # (Production code enforces MAX_TOTAL_RESOURCES limit and generates iframe HTML)
    state["resources"].extend(resources)

    return state

This simplified version shows the core search pattern.

The production code adds fallback searches when Tako returns no results, generates iframe HTML for chart embedding via get_visualization_iframe(), enforces resource limits, and handles dual routing for different tool paths.

Check out the complete implementation at agents/python/src/lib/search.py.

5. Backend: MCP Integration & Tools

At this point, the UI is wired, the agent graph is defined, and the execution flow is clear. What’s left is the backend work that actually fetches data, talks to external systems, and turns results into something the agent can reason about.

The most interesting part here is the MCP integration for Tako charts. The rest of the backend nodes are intentionally simple.

MCP Apps Pattern for Tako Charts

Tako is a data visualization platform with searchable charts on economics, markets, demographics, and more.

What Tako provides:

Searchable chart catalog via knowledge_search tool
Chart search with configurable effort: "fast" (quick lookups), "medium", or "deep" (thorough searches)
Ready-to-embed iframe HTML via open_chart_ui tool
Auto-resizing charts via postMessage protocol

This is the core idea behind MCP Apps: tools return UI components, not just data.

This module mcp_integration.py is where the agent connects to Tako via the Model Context Protocol (MCP). I have broken the core parts of this integration, so it's easier to follow along.

The MCP Client: Session Management

The SimpleMCPClient class handles the Model Context Protocol lifecycle:

SSE connection establishes and maintains the session ID
HTTP messages send JSON-RPC tool calls to the server
Automatic reconnection on session expiry (404/410 errors)

# core client logic -- agents/python/src/lib/mcp_integration.py

class SimpleMCPClient:
    """
    Minimal MCP client following the Model Context Protocol spec.
    Handles connection lifecycle, session management, and message passing.
    """

    async def connect(self):
        """Connect to MCP server via SSE to get session ID."""
        self._sse_task = asyncio.create_task(self._sse_reader())

        # Wait for session_id establishment
        for _ in range(50):
            if self.session_id:
                return True
            await asyncio.sleep(0.1)
        return False

    async def reconnect(self):
        """Reconnect with new session on expiry."""
        if self._sse_task:
            self._sse_task.cancel()

        # Clear state and reconnect
        self.session_id = None
        self._responses.clear()
        await self.connect()
        await self.initialize()

    async def _send(self, method: str, params: dict = None, _retry: bool = True):
        """
        Send JSON-RPC message to server via HTTP.
        Automatically reconnects and retries once on session expiration (410/404).
        """
        msg = {"jsonrpc": "2.0", "id": self.message_id, "method": method}
        if params:
            msg["params"] = params

        future = asyncio.get_event_loop().create_future()
        self._responses[self.message_id] = future

        try:
            resp = await self._client.post(
                f"{self.base_url}/messages/?session_id={self.session_id}",
                json=msg,
            )

            # Handle session expiration
            if resp.status_code in (404, 410):
                if _retry:
                    await self.reconnect()
                    return await self._send(method, params, _retry=False)
                raise SessionExpiredException("Session expired")

        except httpx.HTTPStatusError as e:
            if e.response.status_code in (404, 410) and _retry:
                await self.reconnect()
                return await self._send(method, params, _retry=False)
            raise

        return await asyncio.wait_for(future, timeout=120.0)

# Global client instance (reused across all calls)
_mcp_client: Optional[SimpleMCPClient] = None

A single global client instance maintains the session across all tool calls. If a session expires mid-conversation, the client automatically reconnects and retries once.

Searching Tako's Knowledge Base

The search_knowledge_base function wraps the MCP knowledge_search tool to find relevant charts.

Each result becomes a Resource with resource_type = "tako_chart". The search_node runs these queries in parallel with Tavily web searches using asyncio.gather() for speed (as we discussed in the previous section).

async def search_knowledge_base(
    query: str,
    count: int = 5,
    search_effort: str = "fast"
) -> List[Dict[str, Any]]:
    """
    Search Tako knowledge base via MCP 'knowledge_search' tool.

    Returns a list of charts with metadata (id, title, description, embed_url).
    """
    result = await _call_mcp_tool("knowledge_search", {
        "query": query,
        "api_token": TAKO_API_TOKEN,
        "count": count,
        "search_effort": search_effort
    })

    if result and "results" in result:
        formatted_results = []
        for item in result["results"]:
            item_id = item.get("card_id") or item.get("id")
            title = item.get("title", "")
            description = item.get("description", "")
            embed_url = f"{DATA_SOURCE_URL}/embed/{item_id}/?theme=dark" if item_id else None

            formatted_results.append({
                "type": "data_visualization",
                "id": item_id,
                "title": title,
                "description": description,
                "embed_url": embed_url,
                "url": f"{DATA_SOURCE_URL}/card/{item_id}"
            })

        return formatted_results

    return []

Fetching Embeddable Chart UI

The get_visualization_iframe function is where MCP Apps shines. Instead of returning chart data, it returns complete iframe HTML.

There are two paths involved here:

MCP UI (open_chart_ui): Returns complete iframe HTML with interactive controls and resize logic
Fallback embed URL: Generates basic iframe HTML manually if the MCP call fails

The returned HTML is injected directly into the markdown report. The frontend's MarkdownRenderer extracts these iframe blocks and renders them as siblings to the markdown content (preventing flickering during streaming updates).

async def get_visualization_iframe(item_id: str = None, embed_url: str = None) -> Optional[str]:
    """
    Get embeddable iframe HTML for a Tako chart.
    Uses the MCP 'open_chart_ui' tool with automatic session reconnection.

    Returns ready-to-embed iframe HTML with resize script.
    """
    if item_id:
        result = await _call_mcp_tool("open_chart_ui", {
            "pub_id": item_id,
            "dark_mode": True,
            "width": 900,
            "height": 600
        })

        # Extract HTML from MCP response
        if result and result.get("type") == "resource":
            resource = result.get("resource", {})
            html_content = resource.get("htmlString") or resource.get("text")
            if html_content:
                return html_content

    # Fallback: Generate iframe HTML with embed_url
    if embed_url:
        return f'''<iframe
  width="100%"
  height="600"
  src="{embed_url}"
  scrolling="no"
  frameborder="0"
></iframe>

<script type="text/javascript">
window.addEventListener("message", function(e) {{
  if (e.data.type !== "tako::resize") return;

  for (let iframe of document.querySelectorAll("iframe")) {{
    if (iframe.contentWindow === e.source) {{
      iframe.style.height = e.data.height + "px";
    }}
  }}
}});
</script>'''

    return None

This is how MCP Apps actually works. The agent writes [CHART:title] markers in the report, and the backend replaces them with full iframe HTML from Tako.

The frontend never needs to know how to render Tako charts; it just embeds the iframe. Check out the complete code at agents/python/src/lib/mcp_integration.py.

Other backend nodes (briefly)

The repository includes a few more helper nodes, but they follow standard patterns and don’t introduce new concepts, so I have left them out to avoid unnecessary length.

download_node: Downloads web resource content and caches it (truncated to 3000 chars to reduce context bloat)
delete_node: handles resource deletion with user confirmation (interrupt pattern for generative UI)
state.py: defines TypedDict schemas for AgentState, Resource, DataQuestion, and Log
model.py: function to get LLM instance (OpenAI/Anthropic/Google) based on env config

6. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the .env in the root directory.

Install all dependencies (frontend + Python agent) by running this command from the root. This installs the Next.js frontend packages and also runs uv sync in agents/python to install Python dependencies.

npm install

This repository runs the frontend and backend together in development mode. Just run the following command from the root:

npm run dev

Behind the scenes, this is wired up using concurrently in package.json:

"dev": "concurrently -k -n ui,agent -c blue,red \"PORT=3000 npm run dev:ui\" \"PORT=2024 npm run dev:agent\"",
"dev:ui": "next dev",
"dev:agent": "cd agents/python && uv run main.py"

dev:ui → runs Next.js frontend on port 3000
dev:agent → runs Python FastAPI agent backend on port 2024 (using uv run)
concurrently runs both in parallel with colored output (blue for UI, red for agent)

You can navigate to http://localhost:3000 in your browser to view it locally.

Here is the demo!

6. Data flow

Now that we've built the frontend (Next.js + CopilotKit), the agent orchestration (LangGraph) and the backend integrations (MCP + Tavily), this is how data actually flows between them.

This should be easy to follow if you have been building along so far.

[User enters research question]
      ↓
Next.js UI (CopilotKit Provider + CopilotChat + ResearchCanvas)
        ↓
POST /api/copilotkit (CopilotKit runtime endpoint)
        ↓
CopilotRuntime → Python Agent (LangGraph)
        ↓
LangGraph StateGraph orchestration
   ├─ chat_node: interpret intent, decide tools 
   │   └─ (generates report with [CHART:title] markers, then replaces with iframe HTML)
   ├─ search_node: parallel gather + dedupe/normalize resources
   │      ├─ Tavily search (web results)
   │      └─ Tako MCP search (returns chart metadata)
   └─ download_node: cache web resource content
        ↓
copilotkit_emit_state(): stream intermediate AgentState with report containing iframe HTML (SSE)
        ↓
Frontend receives state updates (resources/report with embedded iframes)
        ↓
MarkdownRenderer extracts <!doctype html>...</html> iframe blocks and renders as siblings to markdown
        ↓
User sees: streaming report + resources list + interactive embedded charts

That’s it! 🎉

You now have a research assistant that streams agent state into a chat + canvas UI, blends web search with MCP-powered charts, and produces an interactive report.

What makes this work is the structure: clear agent state, explicit tool boundaries, and a UI that renders progress instead of hiding it. Once those pieces are in place, adding new MCP tools or constraints becomes an implementation detail.

I hope you learned something valuable. Have a great day!

The Developer's Guide to Generative UI in 2026

Anmol Baranwal — Thu, 29 Jan 2026 14:41:12 +0000

AI agents have become much better at reasoning and planning. The UI layer has mostly stayed the same, and it is holding back the experience.

Most agent experiences still rely on chat, even when the task clearly needs forms, previews, controls, or step-by-step feedback.

Generative UI is the idea that allows agents to influence the interface at runtime, so the UI can change as context changes. This is usually done through UI specs like A2UI, Open-JSON-UI, or MCP Apps.

Check out CopilotKit's GitHub ⭐️

Today, we will break down Generative UI, the three practical patterns, and how CopilotKit supports them (using AG-UI protocol under the hood).

Let's jump in.

What is covered?

In summary, we will cover these topics in detail.

What is Generative UI, and why does it matter?
Three Patterns of Generative UI
- Static Generative UI (AG-UI)
- Declarative Generative UI (A2UI and Open-JSON-UI)
- Open-ended Generative UI (MCP Apps)
How AG-UI enables Generative UI at runtime

Here’s the GitHub Repo for the live playground that implements all three approaches.

1. What is Generative UI?

Generative UI (sometimes called GenUI) is a pattern in which parts of the user interface are generated, selected, or controlled by an AI agent at runtime rather than being fully predefined by developers.

Instead of hardcoding every button, for,m or panel upfront, the UI is created dynamically based on the user's needs and context. The agent can decide what UI to show, what inputs it needs from the user, and how the state should update as the task progresses.

Instead of only generating text, the agent can:

request structured inputs
render task-specific UI
update the interface as tools run and state changes

This turns the UI into an active part of the agent’s execution rather than a static wrapper around a chat box. The frontend no longer has to infer intent from text.

Generative UI = LLM output → live, interactive UI

Examples:

A travel query → itinerary cards + map + expandable sections
A comparison query → sortable tables

In the CopilotKit ecosystem, Generative UI is not a single abstraction. It is a set of patterns built on top of agentic UI specifications (such as A2UI and MCP Apps) that define how agents communicate UI updates to applications. You can read more on the website and refer to the docs.

Google describes Generative UI as a way to give models a rich, custom, and interactive user experience that adapts to any prompt. If you are interested in reading more, you can check out their research blog.

A high-level system overview of the generative UI implementation (from Google Research).

Source: Google Research

Why does it matter?

Most agent experiences today still look like chat apps. When agents move beyond chatting and start doing things, text alone becomes a bottleneck.

Common problems with text-only agents:

tool execution and progress are hidden behind chat messages
user inputs are vague, easy to misinterpret, and hard to validate
multi-step flows feel opaque, so users do not trust the outcome

With Generative UI:

render task-specific UI exactly when it is needed
collect structured, validated input
show progress and intermediate results as real UI
adapt the interface as plans evolve

This makes agent systems easier to understand and guide. Instead of interacting with a black box, users interact with a system that exposes its state through UI.

2. Three Patterns of Generative UI

Most Generative UI implementations fit into three patterns. What separates them is how much control stays in the frontend versus how much freedom is given to the agent.

Here are the three types (along with example specs):

1) Static Generative UI (high control, low freedom) - streamed via AG-UI

The frontend owns the UI. The agent only selects which predefined component to show and fills it with data.

2) Declarative Generative UI (shared control) - using A2UI, Open-JSON-UI

The agent returns a structured UI spec (cards, lists, forms). The frontend renders it with its own constraints and styling.

3) Open-ended Generative UI (low control, high freedom) - enabled by MCP Apps

The agent returns a full UI surface (often embedded or free-form). The frontend mainly hosts it, so you trade consistency and safety for flexibility.

If you want to see these patterns side by side, there’s a live playground showing all three, along with a code implementation:

Live playground: go.copilotkit.ai/gen-ui-demo
Code: github.com/CopilotKit/generative-ui-playground

Here is a short demo!

You can check the docs and pick the integration you are using to see detailed implementation guides and examples.

Now, let's cover each pattern in depth and see how to implement them.

✅ Static Generative UI (AG-UI)

Static Generative UI is the most constrained pattern.

Developers pre-build UI components and the agent’s role is limited to deciding when a component appears and what data it receives. The layout and interactions are fully owned by the application.

In CopilotKit, this pattern is implemented using the useFrontendTool hook, which binds predefined UI components to the lifecycle of an action.

// Weather tool - a callable tool that displays weather data in a styled card
useFrontendTool({
  name: "get_weather",
  description: "Get current weather information for a location",
  parameters: z.object({ location: z.string().describe("The city or location to get weather for") }),
  handler: async ({ location }) => {
    await new Promise((r) => setTimeout(r, 500));
    return getMockWeather(location);
  },
  render: ({ status, args, result }) => {
    if (status === "inProgress" || status === "executing") {
      return <WeatherLoadingState location={args?.location} />;
    }
    if (status === "complete" && result) {
      const data = JSON.parse(result) as WeatherData;
      return (
        <WeatherCard
          location={data.location}
          temperature={data.temperature}
          conditions={data.conditions}
          humidity={data.humidity}
          windSpeed={data.windSpeed}
        />
      );
    }
    return <></>;
  },
});

Here, the useFrontendTool hook lets the application register the get_weather tool and attach a predefined React render to each phase of its lifecycle.

Output:

The agent can invoke the tool and stream arguments, but it never controls layout or invents UI.

You can check the complete implementation at github.com/CopilotKit/generative-ui-playground.

✅ Declarative Generative UI (A2UI and Open-JSON-UI)

Declarative Generative UI sits between static and fully open-ended approaches.

Instead of selecting predefined components, the agent returns a structured UI description. This description defines things like cards, lists, forms, or widgets, and the frontend renders them.

Two common declarative specifications used here are:

1) A2UI → declarative Generative UI spec from Google, described as JSONL-based, and agents can use it to return UI widgets as part of their responses

2) Open‑JSON‑UI → open standardization of OpenAI’s internal declarative Generative UI schema

Both define structured, platform-agnostic ways for agents to describe UI in JSON.

Let's first understand the basic flow for implementing A2UI.

Instead of writing A2UI JSON by hand, you can use the A2UI Composer to generate the spec for you. Copy the output and paste it into your agent’s prompt as a reference template.

In your prompt_builder.py, pick one representative A2UI template (e.g., a simple form or list). This is what the agent learns to emit:

UI_EXAMPLES = """
---BEGIN FORM_EXAMPLE---
[
  { "beginRendering": { "surfaceId": "form-surface", "root": "form-column",
                        "styles": { "primaryColor": "#9B8AFF", "font": "Plus Jakarta Sans" } } },
  { "surfaceUpdate": {
      "surfaceId": "form-surface",
      "components": [
        { "id": "form-column", "component": { "Column": {
            "children": { "explicitList": ["form-title", "name-field", "submit-button"] }
        } } },
        { "id": "form-title", "component": { "Text": {
            "usageHint": "h2", "text": { "literalString": "Contact Us" }
        } } },
        { "id": "name-field", "component": { "TextField": {
            "label": { "literalString": "Your Name" },
            "text": { "path": "name" },
            "textFieldType": "shortText"
        } } },
        { "id": "submit-button", "component": { "Button": {
            "child": "submit-text", "primary": true,
            "action": { "name": "submit_form", "context": [
              { "key": "name", "value": { "path": "name" } }
            ] }
        } } },
        { "id": "submit-text", "component": { "Text": {
            "text": { "literalString": "Send Message" }
        } } }
      ]
    }
  },
  { "dataModelUpdate": {
      "surfaceId": "form-surface", "path": "/", "contents": [
        { "key": "name", "valueString": "" }
      ]
    }
  }
]
---END FORM_EXAMPLE---
"""

This single JSONL‑formatted block defines a “Contact Us” form in A2UI. You feed it to the agent so it learns the three message envelopes A2UI expects: surfaceUpdate (components), dataModelUpdate (state), then beginRendering (render signal).

Then add UI_EXAMPLES into the system prompt that the agent uses:

if use_ui:
    instruction = AGENT_INSTRUCTION + get_ui_prompt(self.base_url, UI_EXAMPLES)
else:
    instruction = get_text_prompt()

return LlmAgent(
    model=LiteLlm(model=LITELLM_MODEL),
    name="ui_generator_agent",
    description="Generates dynamic UI via A2UI declarative JSON.",
    instruction=instruction,
    tools=[],
)

By appending UI_EXAMPLES (and the A2UI schema) to your agent’s instruction, you ensure it streams back valid JSON based on your templates whenever the user asks for a UI.

The final step is to use createA2UIMessageRenderer, which takes the agent’s A2UI JSON and your theme, and then renderActivityMessages tells CopilotKit to use it when rendering chat messages.

import { CopilotKitProvider, CopilotSidebar } from "@copilotkitnext/react";
import { createA2UIMessageRenderer } from "@copilotkit/a2ui-renderer";
import { a2uiTheme } from "../theme";

// Instantiate the A2UI renderer once with your theme
const A2UIRenderer = createA2UIMessageRenderer({ theme: a2uiTheme });

export function A2UIPage({ children }: { children: React.ReactNode }) {
  return (
    <CopilotKitProvider
      runtimeUrl="/api/copilotkit-a2ui"
      renderActivityMessages={[A2UIRenderer]}   // ← hook in the A2UI renderer
      showDevConsole={false}
    >
      {children}
      <CopilotSidebar defaultOpen labels={{ modalHeaderTitle: "A2UI Assistant" }} />
    </CopilotKitProvider>
  );
}

Output:

The playground repo is wired for A2UI, but the pattern is the same for Open‑JSON‑UI.

An agent can respond with a JSON payload describing a UI “card,” and the frontend renders it.

// Example: Agent returns Open-JSON-UI specification
{
  type: "open-json-ui",
  spec: {
    components: [
      {
        type: "card",
        properties: {
          title: "Data Visualization",
          content: {...}
        }
      }
    ]
  }
}

Output:

You can check the specs overview at docs.copilotkit.ai/learn/generative-ui/specs/open-json-ui to understand how CopilotKit's AG-UI protocol natively supports Open-JSON-UI.

This pattern is a good fit if you need more flexibility than static UI, while still keeping clear boundaries and predictable rendering.

✅ Open-ended Generative UI (MCP Apps)

Open-ended Generative UI is the most flexible and powerful.

In this pattern, the agent returns an entire UI surface. This can be HTML, an iframe, or other free-form content. The frontend mainly acts as a container that displays whatever the agent provides.

This approach is commonly used with MCP Apps, where external applications expose their own UI that can be embedded into an agent experience.

Traditional MCP tools return text, images, resources or structured data that the host displays as part of the conversation. MCP Apps extend this pattern by allowing tools to declare a reference to an interactive UI in their tool description that the host renders in place.

The trade-offs are real. Rendering arbitrary UI introduces security and performance concerns, inconsistent styling, and harder portability outside the web.

But for complex tools and rich applications, this pattern unlocks capabilities that are impossible with static or declarative UI alone.

In CopilotKit, MCP Apps support is enabled by attaching MCPAppsMiddleware to your agent, which allows the runtime to connect to one or more MCP Apps servers.

// app/api/copilotkit/route.ts
import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

// 1. Create your agent and add the MCP Apps middleware
const agent = new BuiltInAgent({
  model: "openai/gpt-4o",
  prompt: "You are a helpful assistant.",
}).use(
   new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: "http://localhost:3108/mcp",
        serverId: "my-server" // Recommended: stable identifier
      },
    ],
  }),
)

// 2. Create a service adapter, empty if not relevant
const serviceAdapter = new ExperimentalEmptyAdapter();

// 3. Create the runtime and add the agent
const runtime = new CopilotRuntime({
  agents: {
    default: agent,
  },
});

// 4. Create the API route
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here is the GitHub Repo for the complete code implementation, and you can check the complete integration flow in the blog.

3. How AG-UI enables Generative UI at runtime

Across all three patterns, AG-UI (Agent-User Interaction Protocol) serves as the underlying runtime interaction layer in CopilotKit. It provides a bidirectional connection between the agent and the application that makes Generative UI work.

It is not a UI specification. It is an event/state protocol that defines how UI updates, agent state, and user interactions flow while the agent is working.

At a high level, AG-UI handles:

Tool lifecycles (started → streaming → finished / failed)
User interactions (clicks, form submissions, selections)
Agent state updates (progress, partial results, next steps)
Real-time coordination between agent ↔ UI ↔ application

For instance, here is the role of AG-UI protocol across Generative UI patterns:

Static → signals tool lifecycle → frontend renders predefined components
Declarative → carries UI specs + state → frontend renders structured UI
Open-ended → coordinates tools + UI references → app embeds external UI

Because AG-UI sits below specs like A2UI, Open-JSON-UI, and MCP Apps, CopilotKit can support multiple Generative UI approaches without locking you into a single approach.

In many cases, the real bottleneck is the interface that sits between the agent and the user.

Generative UI provides a practical way forward, and once you build one of these patterns yourself, it becomes clear which approach fits which problem.

I hope you learned something valuable. Have a great day!

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Don't forget to follow CopilotKit on Twitter and join the Discord community of agent builders.

Happy building!

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

Bring MCP Apps into your OWN app with CopilotKit & AG-UI

Anmol Baranwal — Thu, 22 Jan 2026 15:06:34 +0000

Agent frameworks have become very good at calling tools and returning structured data.

The gap appears when an agent has to present information (charts, dashboards) or collect user input (forms, approvals). If the tool only returns JSON, every host ends up rebuilding the same UI logic for the same tools.

MCP Apps was proposed to solve this exact problem: a standardized way for MCP servers to ship interactive HTML UIs alongside tool outputs.

Today, we are excited to announce CopilotKit’s support for MCP Apps. Now, MCP servers can finally ship an interactive UI that works out of the box in real agent applications.

But showing UI is only step one. The hard part is keeping everything coordinated while the user is interacting:

Tool lifecycle (started → streaming → finished / failed)
User interactions (clicks, form submits, selections)
Agent state updates (progress, partial results, next steps)
Real-time back-and-forth between agent ↔ UI ↔ app

That’s why this integration is powered by AG‑UI: an event/state protocol that ships structured events back and forth (tool progress, UI actions, state updates) so the agent, the UI and your app stay synchronized while MCP Apps run.

The easiest way to get started is using the CopilotKit CLI:

npx copilotkit create -f mcp-apps

You can refer to the official docs or try the live demo if you want to explore it yourself. Now, let's understand the technical details of what is happening under the hood.

1. MCP-UI: where the idea first worked

This proposed standard is inspired by MCP-UI and OpenAI's Apps SDK.

MCP-UI was the community’s first proof that UI can be a first-class MCP resource (not just text/JSON). Tools could ship HTML rendered in sandboxed iframes and hosts could display rich, interactive content instead of rebuilding the UI for every tool.

MCP Apps take the core idea behind MCP-UI and formalize it into a shared, interoperable spec.

SEP-1865 was authored by MCP core maintainers at OpenAI and Anthropic, together with MCP‑UI creators and lead maintainers of the MCP-UI Community Working Group.

You can review the full SEP at modelcontextprotocol/ext-apps.

2. MCP Apps: standardizing UI within ecosystem

MCP Apps doesn’t just say “tools can return UI.” It standardizes how a server exposes UI in a way that hosts can reliably discover, render and interact with.

Credit: MCP Apps post

At a high level, it introduces a few core pieces:

MCP server declares UI resources (using the ui://... URI scheme).
Tools reference UI resources through the _meta field so the host knows what to render.
Hosts can prefetch and review templates before running tools
UI runs in sandboxed iframes for security
All UI ↔ host communication happens using the standard MCP JSON-RPC over postMessage.

A simplified example from a tool response might look like this:

{
  "result": {
    "data": {
      "revenue": [280, 400, 560]
    }
  },
  "_meta": {
    "ui/resourceUri": "ui://charts/revenue"
  }
}

On the server side, that UI is declared explicitly:

{
  "ui://charts/revenue": {
    "type": "html",
    "entrypoint": "/ui/revenue-chart.html"
  }
}

The point is: a server can ship a UI once and multiple hosts can render it consistently because the “resource + link + message channel” pattern is standardized.

What's still missing?

MCP Apps defines the UI surface and a standard way for the UI to talk to the host, but it doesn’t tell you how to keep agent state, tool lifecycle and UI events synchronized in real-time.

That orchestration layer is where AG‑UI protocol fits in.

3. AG‑UI: the sync layer that makes it usable

You still need a consistent way to keep agent ↔ UI ↔ app aligned while the user interacts.

AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes this “agent ↔ frontend” real-time communication.

Because it’s transport-agnostic (SSE, WebSockets, etc.), the same frontend can work across different agent backends without custom wiring.

CopilotKit uses AG-UI as its synchronization layer, which makes it a natural fit for running interactive MCP Apps.

What it looks like (events on the wire)

When an MCP App UI is on screen, AG‑UI is the stream CopilotKit (or any compatible agent framework) uses to keep tool progress and shared state synced while the user interacts.

When the agent starts a tool call, you will see lifecycle events like:

{ "type": "TOOL_CALL_START", "toolCallId": "tool_1", "toolCallName": "get_revenue_data" }

As the agent/frontend state changes, you will see state snapshots like:

{ "type": "STATE_SNAPSHOT", "snapshot": { "status": "loading" } }

Followed by incremental updates (such as JSON Patch deltas) as the workflow progresses:

{
  "type": "STATE_DELTA",
  "delta": [{ "op": "replace", "path": "/status", "value": "ready" }]
}

STATE_DELTA uses JSON Patch so the UI can update without re-sending the entire state each time.

This is how it works in practice.

4. How CopilotKit wires MCP Apps ↔ AG‑UI

CopilotKit acts as the runtime that binds MCP Apps and AG-UI together inside a real application. Here is the simple flow:

The agent calls an MCP tool and receives structured output
The tool response includes a ui://... reference, indicating which UI to render
CopilotKit renders that UI inside your app using a sandboxed iframe
UI sends interactions back and CopilotKit forwards them into the agent loop
AG-UI streams state/progress updates back to the UI while the agent runs.

Agent  
│  
│ calls tool  
▼  
MCP Tool ──────► returns data + ui:// reference  
│  
▼  
CopilotKit (runtime)  
│  
├─ renders MCP App UI (iframe)  
│  
├─ forwards UI messages → agent  
│  
└─ participates in AG-UI event stream  
│ (state / progress updates)  
▼  
MCP App UI

MCP Apps provide the UI surface, AG-UI provides the sync contract and CopilotKit runs the loop.

5. Integration flow: MCP Apps with CopilotKit

The fastest way to get started is to use the CopilotKit CLI:

npx copilotkit create -f mcp-apps

It creates a starter repo, which shows an end-to-end MCP Apps integration with CopilotKit and uses the official Three.js MCP Server.

If you are integrating into an existing Next.js app, here's how you can set things up manually.

To run MCP Apps inside a real agent, you need a runtime layer that can fetch UI resources, sandbox them and keep UI state in sync with the agent.

That’s what the MCPAppsMiddleware provides. Install it along with the required CopilotKit packages:

npm install @copilotkit/react-ui @copilotkit/react-core @copilotkit/runtime @ag-ui/mcp-apps-middleware

Next, configure the CopilotKit API route. This is where the agent is created, the MCP Apps middleware is attached and the CopilotKit runtime is exposed.

BuiltInAgent(...) : defines the agent’s LLM (model) and behavior (prompt)
mcpServers : the list of MCP servers to connect to (you can add multiple)
serverId : stable name for that server so CopilotKit can reload the same MCP App UI later, even if the server URL changes (local, staging, prod)

// app/api/copilotkit/route.ts
import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

// 1. Create your agent and add the MCP Apps middleware
const agent = new BuiltInAgent({
  model: "openai/gpt-4o",
  prompt: "You are a helpful assistant.",
}).use(
   new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: "http://localhost:3108/mcp",
        serverId: "my-server" // Recommended: stable identifier
      },
    ],
  }),
)

// 2. Create a service adapter, empty if not relevant
const serviceAdapter = new ExperimentalEmptyAdapter();

// 3. Create the runtime and add the agent
const runtime = new CopilotRuntime({
  agents: {
    default: agent,
  },
});

// 4. Create the API route
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

This will allow the agent to understand ui:// references returned by MCP tools and resolve them into rendered UI.

Create a .env.local file in your frontend directory and add your API key. The example uses OpenAI’s GPT-4o by default but you can also configure other language models.

OPENAI_API_KEY=your_openai_api_key

Wrap your application with the CopilotKit provider so the frontend can connect to the runtime (layout.tsx).

import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/styles.css";

// ...

export default function RootLayout({ children }: {children: React.ReactNode}) {
    return (
        <html lang="en">
            <body>
                <CopilotKit runtimeUrl="/api/copilotkit">
                    {children}
                </CopilotKit>
            </body>
        </html>
    );
}

Now add the CopilotSidebar chat component to your page.

// app/page.tsx
"use client";

import { CopilotSidebar } from "@copilotkit/react-ui";

export default function Page() {
    return (
        <main>
            <h1>Your App</h1>
            <CopilotSidebar />
        </main>
    );
}

That's it! MCP Apps will now render automatically whenever the agent uses tools that reference UI resources.

The middleware supports both HTTP and SSE transports. You can find the full config options in the official docs.

You can try some of the open source MCP Apps servers from the official repo at github.com/modelcontextprotocol/ext-apps.

What this enables for Agent Builders

This brings interactive UIs into MCP in a way that works across hosts. You can:

Ship interactive tool UIs without rebuilding them per host
Keep UI + agent state in sync without custom wiring
Go from prototype to production with fewer rewrites

You can even collect structured input and include human-in-the-loop steps inside your app.

5. Getting started

To start building:

👉 Start with CopilotKit & MCP Apps: npx copilotkit create -f mcp-apps

👉 Check out the CopilotKit / MCP Apps docs to get started

👉 Check out our example repo for a minimal integration setup

👉 Try the live CopilotKit demo and interact with an MCP App in real time

What's Next

AG-UI is shaping up to be the bridge. Over the next few weeks, expect more examples, tighter primitives and clearer patterns for this integration.

If you want to follow along (or help shape what ships next), join the CopilotKit or AG-UI communities.

There’s a lot more coming soon.

How to build a Frontend for LangChain Deep Agents with CopilotKit!

Anmol Baranwal — Tue, 20 Jan 2026 14:22:27 +0000

LangChain recently introduced Deep Agents: a new way to build structured, multi-agent systems that can plan, delegate, and reason across multiple steps.

It comes with built-in planning, a filesystem for context, and subagent spawning. But connecting that agent to a real frontend is still surprisingly hard.

Today, we will build a Deep Agents powered job search assistant and connect it to a live Next.js UI with CopilotKit, so the frontend stays in sync with the agent in real time.

You will find architecture, the key patterns, how state flows between the UI ↔ agent and the step-by-step guide to building this from scratch.

Let's build it.

Check out CopilotKit's GitHub ⭐️

1. What are Deep Agents?

Most agents today are just “LLM in a loop + tools”. That works, but it tends to be shallow: no explicit plan, weak long-horizon execution, and messy state as runs get longer.

Popular agents like Claude Code, Deep Research, and Manus get around this by following a common pattern: they plan first, externalize working context (often via files or a shell), and delegate isolated pieces of work to sub-agents.

Deep Agents package those primitives into a reusable agent runtime.

Credit: LangChain

At a practical level, a Deep Agent created via create_deep_agent is just a LangGraph graph. There’s no separate runtime or hidden orchestration layer.

That means standard LangGraph features work as-is:

streaming
checkpoints and interrupts
human-in-the-loop controls

The mental model (how it runs)

Conceptually, the execution flow looks like this:

User goal
  ↓
Deep Agent (LangGraph StateGraph)
  ├─ Plan: write_todos → updates "todos" in state
  ├─ Delegate: task(...) → runs a subagent with its own tool loop
  ├─ Context: ls/read_file/write_file/edit_file → persists working notes/artifacts
  ↓
Final answer

That gives you a usable structure for “plan → do work → store intermediate artifacts → continue” without inventing your own plan format, memory layer or delegation protocol.

You can read more at blog.langchain.com/deep-agents and check official docs.

Where CopilotKit Fits

Deep Agents push key parts into explicit state (e.g. todos + files + messages), which makes runs easier to inspect. That explicit state is also what makes Copilotkit integration possible.

CopilotKit is a frontend runtime that keeps UI state in sync with agent execution by streaming agent events and state updates in real time (using AG-UI under the hood).

This middleware (CopilotKitMiddleware) is what allows the frontend to stay in lock-step with the agent as it runs. You can read the docs at docs.copilotkit.ai/langgraph/deep-agents.

agent = create_deep_agent(
    model="openai:gpt-4o",
    tools=[get_weather],
    middleware=[CopilotKitMiddleware()], # for frontend tools and context
    system_prompt="You are a helpful research assistant."
)

The diagram below shows how a user action in the UI is sent via AG-UI to any agent backend and responses flow back as standardized events.

2. Core Components

Here are the core components that we will be using later on:

1) Planning Tools (built-in via Deep Agents) - built-in planning/to‑do behavior so the agent can break the workflow into steps without you writing a separate planning tool.

# Conceptual example (not required in codebase)
@tool
def todo_write(tasks: List[str]) -> str:
    formatted = "\n".join([f"- {task}" for task in tasks])
    return f"Todo list created:\n{formatted}"

2) Subagents - let the main agent delegate focused tasks into isolated execution loops. Each sub-agent has its own prompt, tools and context.

subagents = [
    {
        "name": "job-search-agent",
        "description": "Finds relevant jobs and outputs structured job candidates.",
        "system_prompt": JOB_SEARCH_PROMPT,
        "tools": [internet_search],
    }
]

3) Tools - this is how the agent actually does things. Here, finalize() signals completion.

@tool
def finalize() -> dict:
    """Signal that the agent is done."""
    return {"status": "done"}

How Deep Agents are implemented (Middleware)

If you are wondering how create_deep_agent() actually injects planning, files and subagents into a normal LangGraph agent, the answer is middleware.

Each feature is implemented as a separate middleware. By default, three are attached:

To-do list middleware - adds the write_todos tool and instructions that push the agent to explicitly plan and update a live todo list during multi-step tasks.
Filesystem middleware - adds file tools (ls, read_file, write_file, edit_file) so the agent can externalize notes and artifacts instead of stuffing everything into chat history.
Subagent middleware - adds the task tool, allowing the main agent to delegate work to subagents with isolated context and their own prompts/tools.

This is what makes Deep Agents feel “pre-wired” without introducing a new runtime. If you want to go deeper, the linked middleware docs show the exact implementation details.

What are we building?

Let's create an agent that:

Accepts a resume (PDF) and extracts skills + context
Uses Deep Agents to plan and orchestrate sub-agents
Searches the web for relevant jobs using tools (Tavily)
Streams tool results back to the UI via CopilotKit (AG-UI)

We will see some of these concepts in action as we build the agent.

3. Frontend: wiring the agent to the UI

Let's first build the frontend part. This is how our directory will look.

The src directory hosts the Next.js frontend, including the UI, shared components and the CopilotKit API route (/api/copilotkit) used for agent communication.

.
├── src/                               ← Next.js frontend
│   ├── app/
│   │   ├── page.tsx                      
│   │   ├── layout.tsx                 ← CopilotKit provider
│   │   └── api/
│   │       ├── upload-resume/route.ts ← upload endpoint
│   │       └── copilotkit/route.ts    ← CopilotKit AG-UI runtime
│   ├── components/
│   │   ├── ChatPanel.tsx              ← Chat + tool capture
│   │   ├── ResumeUpload.tsx           ← PDF upload UI
│   │   ├── JobsResults.tsx            ← Jobs table renderer
│   │   └── LivePreviewPanel.tsx          
│   └── lib/
│       └── types.ts   
├── package.json                     
├── next.config.ts                   
└── README.md

Step 1: CopilotKit Provider & Layout

Install the necessary CopilotKit packages.

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

@copilotkit/react-core provides the core React hooks and context that connect your UI to an AG-UI compatible agent backend.
@copilotkit/react-ui offers ready-made UI components like <CopilotChat /> to build AI chat or assistant interfaces quickly.
@copilotkit/runtime is the server-side runtime that exposes an API endpoint and bridges the frontend with an external AG-UI compatible agent backend using HTTP and SSE.

The <CopilotKit> component must wrap the Copilot-aware parts of your application. In most cases, it's best to place it around the entire app, like in layout.tsx.

import type { Metadata } from "next";

import { CopilotKit } from "@copilotkit/react-core";
import "./globals.css";
import "@copilotkit/react-ui/styles.css";

export const metadata: Metadata = {
  title: "Job Finder | Deep Agents with CopilotKit",
  description: "A job search assistant powered by Deep Agents and CopilotKit",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body className={"antialiased"}>
        <CopilotKit runtimeUrl="/api/copilotkit" agent="job_application_assistant">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}

Here, runtimeUrl="/api/copilotkit" points to the Next.js API route CopilotKit uses to talk to the agent backend.

Each page is wrapped in this context so UI components know which agent to invoke and where to send requests.

Step 2: Next.js API Route: Proxy to FastAPI

This Next.js API route acts as a thin proxy between the browser and the Deep Agents. It:

Accepts CopilotKit requests from the UI
Forwards them to the agent over AG-UI
Streams agent state and events back to the frontend

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    job_application_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Here's a simple explanation of the above code:

The code above registers the job_application_assistant agent.
LangGraphHttpAgent : defines a remote LangGraph agent endpoint. It points to the Deep Agents backend running on FastAPI.
ExperimentalEmptyAdapter : simple no-op adapter used when the agent backend handles its own LLM calls and orchestration
copilotRuntimeNextJSAppRouterEndpoint : small helper that adapts the Copilot runtime to a Next.js App Router API route and returns a handleRequest function

Step 3: Resume upload API endpoint

This API route (src\app\api\upload-resume\route.ts) handles resume uploads from the frontend and forwards them to the FastAPI backend. It:

Accepts multipart file uploads from the browser
Proxies the file to the backend resume parser
Returns extracted text and skills to the UI

Keeping resume parsing in the backend lets the agent reuse the same logic and keeps the frontend lightweight.

import { NextRequest, NextResponse } from "next/server";

export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const file = formData.get("file") as File;

    if (!file) {
      return NextResponse.json({ error: "No file provided" }, { status: 400 });
    }

    const backendFormData = new FormData();
    backendFormData.append("file", file);

    const backendUrl = process.env.BACKEND_URL || "http://localhost:8123";
    const response = await fetch(`${backendUrl}/api/upload-resume`, {
      method: "POST",
      body: backendFormData,
    });

    if (!response.ok) {
      throw new Error("Backend upload failed");
    }

    const data = await response.json();
    return NextResponse.json(data);
  } catch (error) {
    return NextResponse.json(
      { error: error instanceof Error ? error.message : "Upload failed" },
      { status: 500 }
    );
  }
}

Step 4: Building Key Components

I'm only covering the core logic behind each component since the overall code is huge. You can find all the components in the repository at src\components.

These components use CopilotKit hooks (like useCopilotReadable) to tie everything together.

✅ Resume Upload Component

This client component handles resume selection and forwards the file to the backend for parsing.

It accepts a PDF/TXT file, POSTs it to /api/upload-resume and lifts the extracted text and skills back up to the parent component.

"use client";
import { useRef, useState } from "react";

type ResumeUploadResponse = { success: boolean; text: string; skills: string[]; filename: string };

export function ResumeUpload({ onUploadSuccess }: { onUploadSuccess(d: ResumeUploadResponse): void }) {
  const [selectedFile, setSelectedFile] = useState<File | null>(null);
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const inputRef = useRef<HTMLInputElement>(null);

  const onSelect = (e: React.ChangeEvent<HTMLInputElement>) => {
    setError(null);
    const f = e.target.files?.[0] ?? null;
    if (f && !["application/pdf", "text/plain"].includes(f.type)) {
      setSelectedFile(null);
      setError("Please upload a PDF or TXT file");
      e.target.value = ""; // allow re-selecting same file
      return;
    }
    setSelectedFile(f);
  };

  const onSubmit = async (e: React.FormEvent) => {
    e.preventDefault();
    if (!selectedFile) return;

    setIsLoading(true);
    setError(null);

    try {
      const fd = new FormData();
      fd.append("file", selectedFile);

      const res = await fetch("/api/upload-resume", { method: "POST", body: fd });
      if (!res.ok) throw new Error("Upload failed");

      onUploadSuccess((await res.json()) as ResumeUploadResponse);

      setSelectedFile(null);
      if (inputRef.current) inputRef.current.value = "";
    } catch (err) {
      setError(err instanceof Error ? err.message : "Failed to upload resume");
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <form onSubmit={onSubmit}>
      <input ref={inputRef} type="file" accept=".pdf,.txt" onChange={onSelect} />
      <button disabled={!selectedFile || isLoading}>{isLoading ? "Uploading..." : "Upload Resume"}</button>
      {error && <p>{error}</p>}
      {/* ... UI/styling omitted ... */}
    </form>
  );
}

Here's a brief explanation:

Accepts a PDF/TXT file from the user
Sends the file to /api/upload-resume using FormData
Receives extracted text + skills from the backend
Lifts that data via onUploadSuccess so it can be injected into the agent later

Check out the complete code at src/components/ResumeUpload.tsx.

✅ Chat Panel Component

This is the core UI that connects the user, the agent, and the tool outputs. The Chat Panel:

Embeds CopilotChat to handle conversational input and streaming agent responses
Uses useCopilotReadable to continuously sync resume text, detected skills and user preferences into the agent’s context
Intercepts tool calls (like update_jobs_list) to update local UI state without dumping job JSON into chat

We also build the conversational UI using <CopilotChat /> in this component.

"use client";

import { useState, useRef } from "react";
import { useDefaultTool, useCopilotReadable } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import { ResumeUpload } from "./ResumeUpload";
import { JobsResults } from "./JobsResults";

export function ChatPanel() {
  // form + resume state (title, location, skills, resume text…)
  const [jobs, setJobs] = useState<JobPosting[]>([]);
  const processedKeyRef = useRef<string | null>(null); // dedupe repeated tool calls

  // Capture tool output
  useDefaultTool({
    render: ({ name, status, args, result }) => {
      if (name === "update_jobs_list" && status === "complete" && result?.jobs_list) {
        const key = JSON.stringify({
          len: result.jobs_list.length,
          first: result.jobs_list[0]?.url,
        });

        if (processedKeyRef.current !== key) {
          processedKeyRef.current = key;

          // Avoid setState during render
          queueMicrotask(() => {
            setJobs(result.jobs_list);
          });
        }
      }

      // Render tool calls inline
      return (
        <details>
          ...
        </details>
      );
    },
  });

  // Send UI state + resume data into agent context
  useCopilotReadable({
    description: "Job search preferences",
    value: {
      targetTitle,
      targetLocation,
      skillsHint,
      resumeText,
      detectedSkills,
    },
  });

  return (
    <div>
      {/* Resume upload + extracted skills UI */}
      {!resumeUploaded && <ResumeUpload onUploadSuccess={handleUploadSuccess} />}

      {/* Job search inputs (title / location / skills) */}

      {/* CopilotKit chat UI */}
      <CopilotChat />

      {/* Structured output rendered outside chat */}
      <JobsResults jobs={jobs} />
    </div>
  );
}

Check out the complete code at src/components/ChatPanel.tsx.

✅ Jobs Results Component

This is a pure presentational component. It receives the jobs array (populated when update_jobs_list completes) and renders it as a table, keeping the chat output clean.

"use client";
import { JobPosting } from "@/lib/types";

export function JobsResults({ jobs }: { jobs: JobPosting[] }) {
  if (!jobs.length) return null;

  return (
    <div className="mt-4 bg-white border border-slate-200 rounded-lg shadow-sm overflow-hidden">
      <div className="px-4 py-3 border-b border-slate-200">
        <h3 className="font-semibold text-slate-900">Jobs</h3>
      </div>

      <div className="overflow-x-auto">
        <table className="w-full text-sm">
          <thead>{/* Company | Title | Location | Link | Good match */}</thead>
          <tbody>
            {jobs.map((j, idx) => (
              <tr key={idx} className="border-t border-slate-100 text-black">
                <td className="px-4 py-2">{j.company}</td>
                <td className="px-4 py-2">{j.title}</td>
                <td className="px-4 py-2">{j.location}</td>
                <td className="px-4 py-2">
                  <a className="text-blue-600 hover:underline" href={j.url} target="_blank" rel="noreferrer">
                    Open
                  </a>
                </td>
                <td className="px-4 py-2">{j.goodMatch || "Yes"}</td>
              </tr>
            ))}
          </tbody>
        </table>
      </div>
    </div>
  );
}

Check out the complete code at src/components/JobsResults.tsx.

Step 5: Connecting the Chat UI to the Agent

At this point, all the pieces are already in place. This page simply renders the ChatPanel, which is fully wired to the Deep Agents backend via CopilotKit.

A secondary LivePreviewPanel is mounted alongside it. Since tool calls are already rendered inline inside CopilotChat, this panel is optional for now and acts as a work-in-progress space for richer debugging and visualization.

"use client";

import { ChatPanel } from "@/components/ChatPanel";
import { LivePreviewPanel } from "@/components/LivePreviewPanel";

export default function Page() {
  return (
    <main className="min-h-screen flex flex-col">
      {/* App header (branding + description) */}
      <header>
        <h1>Job Application Assistant</h1>
        <p>Find personalized jobs with AI.</p>
        {/* ... badges / styling omitted ... */}
      </header>

      <div className="grid lg:grid-cols-3 gap-6">
        <section className="lg:col-span-2">
          <ChatPanel />
        </section>

        <aside className="lg:col-span-1">
          <LivePreviewPanel />
          {/* Tool calls are already rendered inside CopilotChat */}
          {/* This panel is optional and currently used for experimentation */}
        </aside>
      </div>

      {/* Footer */}
      {/* ... footer content omitted ... */}
    </main>
  );
}

4. Backend: Building the Agent Service (FastAPI + Deep Agents + AG-UI)

We will now build the FastAPI backend that hosts our Deep Agent.

Under the /agent directory lives a FastAPI server that runs the Job Application agent. Here's the project structure of the backend.

.
├── agent/                             ← Deep Agents backend
│   ├── main.py                        ← FastAPI + AG-UI endpoint
│   ├── agent.py                       ← Deep Agents graph & tools
│   ├── pyproject.toml                 ← Python deps (uv)
│   └── uv.lock
...

At a high level, the backend is responsible for:

Exposes a CopilotKit-compatible agent endpoint (for streaming agent state and tool calls)
Provides a /api/upload-resume endpoint for parsing resumes
Constructs a Deep Agents graph that plans, delegates to sub-agents, and searches the web for matching jobs

The backend uses uv for dependency management. Install it if you don't have it in your system.

pip install uv

Initialize a new uv project using the following command. This will generate a fresh pyproject.toml.

cd agent
uv init

Most of the AI tooling used in this backend (especially AG-UI Strands) currently expects Python 3.12+, so make sure to tell uv to use a compatible Python version by using this command:

uv python pin 3.12

Then install the dependencies. This will also create the project’s virtual environment.

uv add copilotkit deepagents fastapi langchain langchain-openai pypdf python-dotenv python-multipart tavily-python "uvicorn[standard]"

copilotkit : connects agents to a frontend with streaming, tools, and shared state.
deepagents : planning-first agent framework for multi-step execution.
fastapi : web framework that exposes the agent API.
langchain : agent and tool orchestration layer.
langchain-openai : OpenAI model integration for LangChain.
pypdf : extracts text from PDF files.
python-dotenv : loads environment variables from .env
python-multipart : enables file uploads in FastAPI.
tavily-python : web search for real-time agent research.
uvicorn[standard] : ASGI server to run FastAPI.

Now run the following command to generate a uv.lock file pinned with exact versions.

uv sync

Add necessary API Keys

Create a .env file under both the agent directory and add your OpenAI API Key and Tavily API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
OPENAI_MODEL=gpt-4-turbo

OpenAI API Key

Tavily API Key

Step 1: Define the agent’s behavior

We start by defining the agent’s behavior using a single, strict system prompt in agent.py.

In Deep Agents, the system prompt acts as the control layer for the workflow, combining planning and delegation to decompose complex tasks into ordered steps.

The MAIN_SYSTEM_PROMPT coordinates tools and sub-agents by enforcing a fixed execution sequence. This prompt ensures:

external actions always happen via tools
UI state is updated in a controlled way
execution ends deterministically with finalize()

MAIN_SYSTEM_PROMPT = """
You are a tool-using agent.

Hard rules:
- Never include job details, URLs, or JSON in assistant messages.
- Only output jobs via update_jobs_list(jobs_json).
- A valid job must be a single job detail page on an ATS or company careers page.
- Do NOT use job boards or listing/search pages.
- company MUST be the hiring company (never Lever/Greenhouse/Ashby/Workday/Talent.com/etc).

Schema (exact keys):
- company, title, location, url, goodMatch

Steps:
1) Call internet_search(query) exactly once.
2) From the returned results, select up to 5 valid individual job postings.
3) Call update_jobs_list(jobs_json) once.
4) Call finalize().
5) Output: Found N jobs.

If you cannot find 5 valid jobs, return as many valid ones as possible.
"""

JOB_SEARCH_PROMPT defines the behavior of a specialized sub-agent. Its responsibility is limited to finding relevant jobs and returning structured results in a controlled format.

JOB_SEARCH_PROMPT = (
    "Search and select 5 real postings that match the user's title, locations, and skills. "
    "Output ONLY this block format (no extra text before/after the wrapper):\n"
    "<JOBS>\n"
    '[{"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"},'
    ' {"company":"...","title":"...","location":"...","link":"https://...","Good Match":"one sentence"}]'
    "\n</JOBS>"
    "Each job MUST:"
    "- Be a single opening (not a job board, filter page or company jobs index)"
    "- Belong to a specific company with a dedicated job description page"
    "You must:"
    "- Use internet_search to find relevant jobs."
    "- Do NOT output job listings, JSON, or URLs in messages."
    "- Return everything ONLY by calling the parent tool `update_jobs_list` with a JSON string."
)

Step 2: Add resume parsing and skill extraction utilities

We extract raw text from uploaded PDFs using pypdf. This function is used by the FastAPI upload endpoint to turn resumes into plain text.

def parse_pdf_resume(file_path: str) -> str:
    with open(file_path, "rb") as file:
        reader = PdfReader(file)
        return "".join(page.extract_text() for page in reader.pages)

Next, we extract lightweight structured signals (languages, frameworks, tools) from the resume. This influences job search queries and match quality.

def extract_skills_from_resume(resume_text: str) -> List[str]:
    skills_db = {
        "languages": ["Python", "JavaScript", "Go"],
        "frameworks": ["React", "FastAPI", "Django"],
        "cloud": ["AWS", "Docker", "Kubernetes"],
    }

    found = set()
    text = resume_text.lower()

    for skills in skills_db.values():
        for skill in skills:
            if skill.lower() in text:
                found.add(skill)

    return list(found)

Step 3: Define tools for search, UI updates, and termination

Tools are the integration surface between the agent and the external world / UI.

The internet_search tool is responsible for discovering real job postings. It intentionally fetches extra search results, filters out any URL containing a “bad” substring (job boards/search pages) via BAD_URL_SUBSTRINGS and returns only the first max_results clean hits.

BAD_URL_SUBSTRINGS = [
    "linkedin.com/jobs/search",
    "linkedin.com/jobs/",
    "builtin.com/jobs",
    "naukri.com",
    "glassdoor.",
    "/jobs/search",
    "/search?",
]


def _is_bad(url: str) -> bool:
    u = (url or "").lower()
    return any(p in u for p in BAD_URL_SUBSTRINGS)


@tool
def internet_search(query: str, max_results: int = 10) -> List[Dict[str, Any]]:
    """
    Search for jobs using Tavily API. Always returns up to 5 results.
    """
    tavily_key = os.environ.get("TAVILY_API_KEY")
    if not tavily_key:
        raise RuntimeError("TAVILY_API_KEY not set")

    client = TavilyClient(api_key=tavily_key)
    res = client.search(
        query=query,
        max_results=max_results * 3,  # get more, then filter
        include_raw_content=False,
        topic="general",
    )

    trimmed = []
    for r in res.get("results", []):
        url = r.get("url") or ""
        if _is_bad(url):
            continue
        trimmed.append(
            {
                "title": r.get("title"),
                "url": url,
                "content": (r.get("content") or "")[:400],
            }
        )
        if len(trimmed) == max_results:
            break

    print(f"[SEARCH] Returning {len(trimmed)} filtered results")
    print(trimmed)
    return trimmed

The update_jobs_list tool is the only way structured job data reaches the frontend, keeping UI updates explicit and JSON out of chat messages.

@tool
def update_jobs_list(jobs_json: str) -> Dict[str, Any]:
    """Send jobs list to UI state."""
    jobs = json.loads(jobs_json)
    print(f"[TOOL] update_jobs_list: {len(jobs)} jobs")
    return {"jobs_list": jobs}

finalize tool signals that the agent has completed its workflow.

@tool
def finalize() -> dict:
    """Signal completion."""
    print("[TOOL] finalize: Job search complete")
    return {"status": "done"}

Step 4: Assemble the Deep Agents graph with sub-agents

Now we connect everything and build the Deep Agents graph with build_agent().

def build_agent():
    """Build Deep Agents graph with proper recursion limit"""
    api_key = os.environ.get("OPENAI_API_KEY")
    if not api_key:
        raise RuntimeError("Missing OPENAI_API_KEY")

    llm = ChatOpenAI(
        model=os.environ.get("OPENAI_MODEL", "gpt-4-turbo"),
        temperature=0.7,
        api_key=api_key,
    )

    tools = [
        internet_search,
        update_jobs_list,
        finalize,
    ]

    subagents = [
        {
            "name": "job-search-agent",
            "description": "Finds relevant jobs and outputs <JOBS> JSON.",
            "system_prompt": JOB_SEARCH_PROMPT,
            "tools": [internet_search],
        },
    ]

    agent_graph = create_deep_agent(
        model=llm,
        system_prompt=MAIN_SYSTEM_PROMPT,
        tools=tools,
        subagents=subagents,
        middleware=[CopilotKitMiddleware()],
        checkpointer=MemorySaver(),
    )

    print("[AGENT] Deep Agents graph created")
    print(agent_graph)

    return agent_graph

Step 5: FastAPI setup

The last step is to initialize the backend and expose it as a FastAPI app. It also handles resume uploads and PDF parsing, turning raw files into clean text and skills before they are ever sent to the agent.

import os
from fastapi import FastAPI, HTTPException, File, UploadFile
import uvicorn
from dotenv import load_dotenv
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent
from agent import build_agent, parse_pdf_resume, extract_skills_from_resume
import tempfile

load_dotenv()

app = FastAPI(
    title="Job Application Assistant",
    description="Find personalized job openings based on skills and preferences",
    version="1.0.0",
)

try:
    agent_graph = build_agent()
    print(agent_graph)
    add_langgraph_fastapi_endpoint(
        app=app,
        agent=LangGraphAGUIAgent(
            name="job_application_assistant",
            description="Job finder",
            graph=agent_graph,
        ),
        path="/",
    )
    print("[MAIN] Agent registered")
except Exception as e:
    print(f"[ERROR] Failed to build agent: {str(e)}")
    raise


@app.get("/healthz")
async def health_check():
    """Health check"""
    return {
        "status": "healthy",
        "service": "job-application-assistant",
        "version": "1.0.0",
    }


@app.post("/api/upload-resume")
async def upload_resume(file: UploadFile = File(...)):
    """
    Upload and parse resume (PDF, DOCX, TXT).
    Returns extracted text and skills.
    """
    if not file:
        raise HTTPException(status_code=400, detail="No file provided")

    try:
        with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
            content = await file.read()
            tmp.write(content)
            tmp_path = tmp.name

        if file.filename.endswith(".pdf"):
            resume_text = parse_pdf_resume(tmp_path)
        else:
            # for other formats, just read as text
            resume_text = content.decode("utf-8", errors="ignore")

        skills = extract_skills_from_resume(resume_text)

        os.unlink(tmp_path)

        return {
            "success": True,
            "text": resume_text[:1000],
            "skills": skills,
            "filename": file.filename,
        }

    except Exception as e:
        print(f"[ERROR] Resume upload failed: {str(e)}")
        raise HTTPException(status_code=500, detail=str(e))


def main():
    """Run server"""
    host = os.getenv("SERVER_HOST", "0.0.0.0")
    port = int(os.getenv("SERVER_PORT", 8123))

    uvicorn.run(
        "main:app",
        host=host,
        port=port,
        reload=True,
        log_level="info",
    )


if __name__ == "__main__":
    main()

5. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the agent/.env.

From the project root, navigate to the agent directory and start the FastAPI server:

cd agent
uv run python main.py

The backend will start on http://localhost:8123.

In a new terminal window, start the frontend development server using:

npm run dev

Once both servers are running, open the frontend in your browser at http://localhost:3000/ to view it locally.

You then upload your resume and search for a job query.

Based on the job query, it can fetch a different number of results. Here is another output!

6. Data flow

Now that we have built both the frontend and the agent service, this is how data actually flows between them. This should be easy to follow if you have been building along so far.

[User uploads resume & submits job query]
        ↓
Next.js UI (ResumeUpload + CopilotChat)
        ↓
useCopilotReadable syncs resume + preferences
        ↓
POST /api/copilotkit (AG-UI protocol)
        ↓
FastAPI + Deep Agents (/copilotkit endpoint)
        ↓
Resume context + skills injected into the agent
        ↓
Deep Agents orchestration
   ├─ internet_search (Tavily)
   ├─ job filtering & normalization
   └─ update_jobs_list (tool call)
        ↓
AG-UI streaming (SSE)
        ↓
CopilotKit runtime receives the tool result
        ↓
Frontend captures the tool output
        ↓
Jobs rendered in table + chat stay clean

That’s it! 🎉

You now have a Deep Agents powered job application assistant with CopilotKit as the frontend layer.

I hope you learned something valuable. Have a great day!

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow CopilotKit on Twitter and say hi, and if you'd like to build something cool, join the Discord community.

CopilotKit Follow

React UI + elegant infrastructure for AI Copilots, in-app AI agents, AI chatbots, and AI-powered Textareas 🪁

How Rube MCP Solves Context Overload When Using Hundreds of MCP Servers

Anmol Baranwal — Mon, 12 Jan 2026 07:18:45 +0000

Over the past year, I have used ChatGPT for everything from drafting emails to debugging code and it’s wildly useful.

But I kept finding myself doing the tedious glue work. That repeated work never felt like a good use of time so I started looking for something that closes the loop.

I recently found Rube MCP, a Model Context Protocol (MCP) server that connects ChatGPT and other AI assistants to 500+ apps (such as Gmail, Slack, Notion, GitHub, Supabase, Stripe, Reddit).

For the last few weeks, I have been testing it out & trying to understand how it think and works. This post covers everything I picked up.

What is covered?

What is MCP? (Brief Intro)
What is Rube MCP & how does it help?
How Rube Thinks: Meta Tools, Planning & Execution
How does it work under the hood?
Connecting Rube to ChatGPT
Three practical examples with demos.
What’s Still Missing.

We will be covering a lot so let's get started.

What is MCP?

MCP (Model Context Protocol) is Anthropic's attempt at standardizing how applications provide context and tools to LLMs. Think of it like HTTP for AI models - a standardized protocol for AI models to “plug in” to data sources and tools.

Instead of writing custom integrations (GitHub, Slack, databases, file systems), MCP lets a host dynamically discover available tools (tools/list), invoke them (tools/call) and get back structured results.

This mimics function-calling APIs but works across platforms and services. If you are interested in reading more, here are a couple of good reads:

The guide to MCP I never had
What is the Model Context Protocol (MCP)? by the Builder.io team

credit: ByteByteGo

Now that you understand MCP, here’s where Rube MCP comes in.

What is Rube MCP & why it matters?

MCP provides a standardized “tool directory” so AI can discover and call services using JSON-RPC, without each model having to memorize all the API details.

Rube is a universal MCP server built by Composio. It acts as a bridge between AI assistants and a large ecosystem of tools.

It implements the MCP standard for you, serving as middleware: the AI assistants talk to Rube via MCP and Rube talks to all your apps via pre-built connectors.

You get a single MCP endpoint (like https://rube.app/mcp) to plug into your AI client, while Composio manages everything (adapters, authentication, versioning and connector updates) behind the scenes.

You can learn more at rube.app.

Once you sign up on the website, you can check recent chats, connect/remove apps at any time and find/enable any supported app in the marketplace.

You can integrate Rube with:

ChatGPT (Developer Mode or Custom GPT)
Agent Builder (OpenAI)
Claude Desktop
Claude Code
Cursor/VS Code
Any generic MCP-compatible client via HTTP / SSE transport (you point it to https://rube.app/mcp)

You can also do it for systems like N8N, custom apps or automation platforms using Auth Headers (signed tokens or API keys) with HTTP requests.

We will be covering how to set it up later but you can find the detailed instructions on the website.

Where it helps

Sure, AI Assistants can write beautiful code, explain complex concepts and even help you debug, but when it comes to actually doing anything in your workflow, they are not really useful.

Rube solves this major problem. Instead of the model guessing which API to use or dealing with OAuth, Rube translates your plain-English request into the correct API calls.

For example, Rube can parse:

“Send a welcome email to the latest sign-up” into Gmail API calls
“Create a Linear ticket titled ‘Bug in checkout flow” into the right Linear endpoint

Because Rube standardizes these integrations, you don’t have to update prompts every time an API changes.

In the dashboard, you will also find a library of recipes. They are basically a list of shortcuts that turn your complex tasks into instant automations.

You can view the workflow in detail, run directly, even fork it to modify based on your needs.

If you are wondering how automation would actually work, you can try asking Rube itself. In my case, it would be as following.

Security & Privacy

Rube is designed with privacy-first principles. Credentials flow directly between you and the app (via OAuth 2.1).

Rube (Composio) never sees your raw passwords. All tokens are end-to-end encrypted.

You choose which apps to connect and which permissions each has. You can remove an app at any time from the Rube dashboard.

For teams, Rube supports shared connections: which means one person can connect Gmail for the team and everyone’s ChatGPT can use it (without re-authenticating).

How Rube Thinks: Meta Tools, Planning & Execution

Rube thinks through your requests using built-in meta tools. The smart layer (planning, workbench) handle the hard parts automatically.

These meta tools run on top of Rube's MCP protocol.

✅ RUBE_SEARCH_TOOLS is a meta tool that inspects your task description and returns the best tools, toolkits and connection status to use for that task.

For example, to check Medium emails in Gmail and write summaries into a Notion page, you can call:

RUBE_SEARCH_TOOLS({
  session: { id: "vast" },                 
  use_case: "Fetch unread Gmail messages from last 30 days that mention Medium Daily Digest or are from [noreply@medium.com](mailto:noreply@medium.com), then add rows to a Notion database page titled Medium Blogs with columns for Title, Summary and Priority",
  known_fields: "sender:noreply@medium.com, query_keyword:Medium Daily Digest, days:30, notion_page_title:Medium Blogs"
})

A successful response for this task looks like (trimmed to the most important fields):

{
  "data": {
    "main_tool_slugs": [
      "NOTION_SEARCH_NOTION_PAGE",
      "NOTION_CREATE_NOTION_PAGE",
      "NOTION_ADD_MULTIPLE_PAGE_CONTENT"
    ],
    "related_tool_slugs": [
      "NOTION_FETCH_DATA",
      "NOTION_GET_ABOUT_ME",
      "NOTION_UPDATE_PAGE",
      "NOTION_APPEND_BLOCK_CHILDREN",
      "NOTION_QUERY_DATABASE"
    ],
    "toolkits": [
      { "toolkit": "NOTION", "description": "NOTION toolkit" }
    ],
    "connection_statuses": [
      {
        "toolkit": "notion",
        "active_connection": true,
        "message": "Connection is active and ready to use"
      }
    ],
    "memory": {
      "all": [
        "Medium Blogs page has ID 287a763a-b038-802e-b22f-f8d853f591a0",
        "User is fetching unread emails from last 30 days from noreply@medium.com or with subject containing Medium Daily Digest"
      ]
    },
    "query_type": "search",
    "reasoning": "NOTION is required to create the new page titled \"Medium Blogs\" and to add formatted text blocks. GMAIL is needed to retrieve the email titles, summaries, and priority information that will populate those text blocks.",
    "session": { "id": "vast" },
    "time_info": {
      "current_time": "2025-10-10T16:15:36.452Z"
    }
  },
  "successful": true
}

✅ RUBE_CREATE_PLAN is a planning meta tool that takes your task description and returns a structured, multi‑step workflow: tools to call, ordering, parallelization, edge‑case handling and user‑confirmation rules.

For the same Gmail→Notion workflow, you can call:

RUBE_CREATE_PLAN({
  session: { id: "vast" },
  use_case: "Fetch unread Gmail messages from last 30 days that mention Medium Daily Digest or are from noreply@medium.com, summarize them with an LLM, and insert one row per email into the Medium Blogs Notion database with Title, Summary and Priority columns.",
  difficulty: "medium",
  known_fields: "sender:noreply@medium.com, query_keyword:Medium Daily Digest, days:30, notion_database:Medium Blogs"
})

A successful response for this task looks like (trimmed to the most important fields):

{
  "data": {
    "workflow_instructions": {
      "plan": {
        "workflow_steps": [
          {
            "step_id": "S1",
            "tool": "GMAIL_FETCH_EMAILS",
            "intent": "Fetch IDs of unread Gmail messages from the last 30 days that are either from noreply@medium.com or mention 'Medium Daily Digest' in subject/body.",
            "parallelizable": true,
            "output": "List of message IDs with minimal metadata.",
            "notes": "Paginate with nextPageToken until all results are fetched."
          },
          {
            "step_id": "S2",
            "tool": "GMAIL_FETCH_MESSAGE_BY_MESSAGE_ID",
            "intent": "...",
            "parallelizable": true,
            "output": "...",
            "notes": "..."
          },
          {
            "step_id": "S3",
            "tool": "COMPOSIO_REMOTE_WORKBENCH",
            "intent": "...",
            "parallelizable": true,
            "output": "..."
          },
          {
            "step_id": "S4",
            "tool": "NOTION_SEARCH_NOTION_PAGE",
            "intent": "...",
            "parallelizable": false,
            "output": "..."
          },
          {
            "step_id": "S5",
            "tool": "NOTION_FETCH_DATABASE",
            "intent": "...",
            "parallelizable": false,
            "output": "..."
          },
          {
            "step_id": "S6",
            "tool": "USER_CONFIRMATION",
            "intent": "...",
            "parallelizable": false,
            "output": "..."
          },
          {
            "step_id": "S7",
            "tool": "COMPOSIO_MULTI_EXECUTE_TOOL",
            "intent": "...",
            "parallelizable": true,
            "output": "..."
          }
        ],
        "complexity_assessment": {
          "overall_classification": "Complex multi-step workflow",
          "data_volume": "...",
          "time_sensitivity": "..."
        },
        "decision_matrix": {
          "tool_order_priority": [
            "GMAIL_FETCH_EMAILS (S1)",
            "GMAIL_FETCH_MESSAGE_BY_MESSAGE_ID (S2)",
            "COMPOSIO_REMOTE_WORKBENCH (S3)",
            "NOTION_SEARCH_NOTION_PAGE (S4)",
            "NOTION_FETCH_DATABASE (S5)",
            "USER_CONFIRMATION (S6)",
            "COMPOSIO_MULTI_EXECUTE_TOOL (S7)"
          ],
          "edge_case_strategies": [
            "If S1 yields no IDs, skip S2–S7 and return an empty result.",
            "If the Notion database cannot be found, prompt the user for an alternate target.",
            "If the user denies confirmation in S6, stop before any Notion inserts."
          ]
        },
        "failure_handling": {
          "gmail_fetch_failure": ["..."],
          "notion_discovery_failure": ["..."],
          "insertion_failure": ["..."]
        },
        "user_confirmation": {
          "requirement": true,
          "prompt_template": "...",
          "post_confirmation_action": "..."
        },
        "output_format": {
          "final_delivery": "...",
          "links_and_references": "..."
        }
      },
      "critical_instructions": "Use pagination correctly, respect time-awareness, and always get explicit user approval before irreversible actions.",
      "time_info": {
        "current_date": "...",
        "current_time_epoch_in_seconds": 1760112500
      }
    },
    "session": {
      "id": "vast",
      "instructions": "..."
    }
  },
  "successful": true
}

If you are using Rube through any other assistant like ChatGPT, you will still be able to inspect all the calls.

✅ RUBE_MULTI_EXECUTE_TOOL is a high‑level orchestrator that runs multiple tools in parallel, using the plan from RUBE_CREATE_PLAN to batch calls like Gmail fetches or Notion inserts into a single step.

RUBE_MULTI_EXECUTE_TOOL({
  session_id: "vast",
  tools: [ /* up to 20 prepared tool calls, e.g. NOTION_INSERT_ROW_DATABASE for each email */ ],
  sync_response_to_workbench: true
})

Here tools is the list of concrete tool invocations generated from the plan and sync_response_to_workbench: true streams the results into the Remote Workbench so the agent can post‑process outputs or handle failures across the whole batch.

If you have been following the examples, the final “execution engine” in S7 of the last plan used this step.

After the emails are summarized and the user confirms, COMPOSIO_MULTI_EXECUTE_TOOL fires many NOTION_INSERT_ROW_DATABASE (and similar) calls concurrently so each Medium email becomes a Notion row without the agent manually looping over tools.

{
  "step_id": "S7",
  "tool": "COMPOSIO_MULTI_EXECUTE_TOOL",
  "intent": "After user confirmation, insert rows into Notion for all summarized emails by leveraging the discovered database schema, in parallel batches.",
  "parallelizable": true,
  "output": "Created Notion rows and returned row identifiers/links.",
  "notes": "Use NOTION_INSERT_ROW_DATABASE per email. If there are many emails, batch in parallel calls (up to tool limits). Reference: Notion insert API usage [Notion API — Create a page in a database](https://developers.notion.com/docs/working-with-dilters#create-a-page-in-a-database)."
}

✅ RUBE_REMOTE_WORKBENCH is a cloud sandbox for running arbitrary Python between tool calls, perfect for parsing raw API responses, batching LLM calls and preparing data for Notion or other apps (such as parsing email HTML → format for Notion).

For this workflow, it was first called like this:

RUBE_REMOTE_WORKBENCH({
  session_id: "vast",
  current_step: "PROCESSING_EMAILS",
  next_step: "GENERATING_SUMMARIES",
  file_path: "/home/user/.composio/output/multi_execute/multi_execute_response_1760112513002_akfm55.json",
  code_to_execute: "import json\n# Load the Gmail response file\ndata = json.load(open('/home/user/.composio/output/multi_execute/multi_execute_response_1760112513002_akfm55.json'))\n# Extract Medium emails, print counts and a few subjects for debugging..."
})

Inside the sandbox, the script loads the multi‑execute Gmail response JSON from disk, filters out the Medium emails and prints diagnostics to stdout such as “Found 30 Medium emails”, “Extracted 30 emails”, plus the first three subjects.

Basically to confirm that upstream Gmail calls worked and giving the agent something human‑readable to reason about.

The corresponding (trimmed) result looks like:

{
  "data": {
    "stdout": "Found 30 Medium emails\nExtracted 30 emails\n\nFirst 3 email subjects:\n1. ...\n2. ...\n3. ...\n",
    "stderr": "",
    "results_file_path": null,
    "session": { "id": "vast" }
  },
  "successful": true
}

The agent then reuses the same workbench in later steps (with different current_step/next_step and code_to_execute) to batch LLM calls and write cleaned summaries to a file that the Notion tools consume. But the calling pattern are always the same.

The next section goes deep on how it works under the hood, which is the MCP transport layer (JSON-RPC, connectors).

How Rube MCP Works (Under the Hood)

At a high level, Rube is an MCP server that implements the MCP spec on the server side. Your AI client communicates with Rube over HTTP/JSON-RPC.

Whenever you ask the AI to “send a message” or “create a ticket,” here’s what happens behind the scenes:

✅ 1. AI Client → Rube MCP Server

It first requests a tool catalog, a machine-readable list of everything Rube can do. When you trigger a command, the client sends a tool call (a JSON-RPC request) to Rube with parameters like:

{ "method": "send_email", "params": { "to": "user@example.com" } }

Rube returns structured JSON responses. The ChatGPT and other AI interfaces have built-in support for this MCP format, so they handle the conversation around it.

✅ 2. Rube MCP Server → Composio Platform

Rube runs on Composio’s infrastructure, which manages a massive library of pre-built tools (connectors).

When Rube receives a tool call from your AI client (say send_slack_message), it looks up the right connector and hands off the request to Composio. It then routes it to the appropriate adapter.

Think of Rube as a universal translator: it takes structured MCP requests from your AI and turns them into actionable API calls using Composio’s connector ecosystem.

✅ 3. Connector / Adapter Layer

Each app has its own connector or adapter that knows how to talk to that app’s API.

The adapter knows how to handle:

API routes and parameters
Pagination, rate limits, and retries
Response formatting and error handling
return structured JSON back to Rube

This design makes Rube easily extensible: when Composio adds a new connector, it becomes instantly available in Rube without any code changes.

✅ 4. OAuth Flow

When you first connect an app, Rube triggers an OAuth flow. For example, saying “send an email” will prompt you to sign in to Gmail and grant access.

Composio then encrypts and stores your tokens securely, which you can always view, revoke or reauthorize in the dashboard.

✅ 5. Response & Chaining

Once the connector completes the API call, it returns a structured JSON response back to Rube, which then sends it to your AI client.

The AI can use that data in the next step like summarizing results, creating follow-up actions or chaining multiple tools together in one flow.

A simple example can be: Hey Rube -- check our shared Sales inbox for unread messages from the last 48h. Summarize each in 2 bullets, draft polite replies for high-priority pricing requests and post a summary to #sales-updates on Slack.

So that’s the magic under the hood. Let’s set it up with ChatGPT and see it in action.

Connecting Rube to ChatGPT

There are a few ways to start using Rube: the simplest is to chat directly from the dashboard.

But since we will be using ChatGPT for the demo later, let’s start there.

You can use Rube inside ChatGPT in two ways:

1) Custom Rube Custom GPT : Just open the Custom GPT, connect your apps and start running Rube-powered actions directly in chat. No setup needed.

2) Developer Mode : for advanced users who want to manually add Rube as an MCP server. You will need a ChatGPT Pro plan for this but it only takes a few minutes to set up. Let’s walk through it step-by-step:

Start by opening ChatGPT Settings and navigating to Apps & Connectors, then click on Advanced Settings.

Next, enable Developer Mode to access advanced connector features.

Once enabled, copy the MCP URL (https://rube.app/mcp). You will now see a Create option on the top-right under Connectors.

Click Create, enter the details, including the MCP URL you copied and save.

You will have to sign up for Rube to complete the OAuth flow.

After this, Rube will be fully connected and ready to use inside ChatGPT.

I will also show how you can plug Rube into other assistants like Claude and Cursor in case you use those too.

Rube with Cursor

You can also connect Rube with Cursor using this link. It will take you straight to the settings, with the MCP URL already filled in so you can get started right away.

Rube with Claude Code (CLI)

Start by running the following command in your terminal:

claude mcp add --transport http rube -s user "https://rube.app/mcp"

This registers Rube as an MCP server in Claude Code. Next, open Claude Code and run the /mcp command to list available MCP servers. You should see Rube in the list.

Claude Code will then prompt you to authenticate. A browser window will open where you can complete the OAuth login. Once authentication is finished, you can use Rube inside Claude Code.

Rube with Claude Desktop

If you are on the Pro plan, you can simply use the Add Custom Connector option in Claude Desktop and paste the MCP URL.

For free plan users, you can set it up via the terminal by running:

npx @composio/mcp@latest setup "https://rube.app/mcp" "rube" --client claude

This will register Rube as an MCP server and you will be ready to start using it with Claude Desktop.

If you want to connect to any other platforms that need an Authorization header, you can generate a signed token from the official website.

Practical examples

Seeing Rube in action is always better than just reading about it. Here, we will go through five real-world workflows. The first one will show the full flow, so you can see how Rube saves time and links multiple apps together.

Before you start, make sure your apps are enabled in the marketplace from the dashboard.

You can easily disconnect and modify scopes depending on your use case.

✅ 1. Gmail to Notion Workflow

Let’s start with a simple but incredibly useful example: turning important emails into Notion tasks automatically.

Every day, Medium sends a “Daily Digest” email (usually from noreply@medium.com) full of trending stories and recommendations.

Rube can automatically fetch the unread Daily Digest emails, pick the most relevant stories and organize them neatly in Notion so you get a clean reading list ready to go.

Here is the prompt:

Hey Rube -- check my Gmail inbox for unread messages from the last 30 days that mention “Medium Daily Digest” or are from noreply@medium.com.  

Extract all story titles and links. Summarize each story in 2 lines, assign a priority (High, Medium, Low) based on potential impact,  

and add them as new rows in my Notion page titled “Medium Blogs”  
with columns: Title, Summary, Priority and URL.

As you can see, this isn’t a simple task. I intentionally made it quite challenging since it:

has to find unread emails
extract links from messy, HTML-heavy content
generate meaningful summaries
and decide what’s actually “relevant” or “high impact”

The overall flow looks like this:

Finding unread emails → Filtering only “Medium Daily Digest” ones → Parsing stories and summaries from HTML → Prioritizing by relevance or impact → Creating a structured format in Notion

Here are the snapshots of the flow.

Rube automatically calls the right tools and you can see everything that happens before approving the actions.

You will also see the full breakdown of tool calls and step-by-step progress:

And the best part: it solved everything automatically and completed the task cleanly.

Here is the output notion page after all the entries were added. The links are accurate and clickable.

I tried this through multiple ways (ChatGPT with Developer mode, Rube Custom ChatGPT, Rube dashboard Chat) and they all worked equally well.

Here are a couple of snapshots from the dashboard chat:

✅ 2. GitHub → Slack (Developer Notifications)

Let’s say your dev team wants to stay synced after every code change, without having to manually update each other.

Here is the prompt:

Hey Rube -- monitor my GitHub repo “acme/auth-service” for new pull requests or merges.  

When a PR is merged, post a message in Slack channel #dev-updates  
with the PR title, author, link.

✅ 3. Customer Support → GitHub

Support teams often have to bridge the gap between customer tickets and dev work.

Rube can quietly handle most of that coordination for you.

Checks for new or updated Zendesk tickets
Looks up each customer’s record in Salesforce
Finds tickets that mention technical bugs
Creates matching GitHub issues for the dev team

Here is the prompt:

Hey Rube -- check for new Zendesk tickets, pull the customer's order history from Salesforce and create github issues for any technical problems mentioned

A simple message but it connects three different systems end-to-end.

✅ 4. Research → Notion (Idea Board)

Let’s say you want to brainstorm ideas and need to see what people are talking about online. Rube can search Hacker News and Reddit, pick out the most relevant posts, summarize them quickly and add everything neatly to your Notion page.

This way, your research board stays up-to-date without you having to comb through dozens of links yourself.

Here is the prompt:

Hey Rube -- search Hacker News and Reddit for "MCP server" mentions in the last 14 days  

Extract the top 10 posts, summarize each in 3 bullet points and add them to my Notion page "MCP Research Board"  

with: Source, Title, Summary, Link.

✅ 5. Gmail → Draft Replies

Let's say you have got a bunch of emails to respond to, but you don’t want to spend hours drafting them. Rube can check your inbox for recent “Sales” emails, summarize each one and draft polite replies for the high-priority threads.

You can instruct to save them as drafts to review before sending.

Here is the prompt:

Hey Rube -- check my Gmail inbox for unread messages from the last 48 hours labeled "Sales"

Summarize each email in 2 bullet points and for high-priority messages asking for pricing or proposals 

draft a polite reply and save it as a Gmail draft under the label "Rube-Drafts"

Think about what this means for productivity. Tasks that once required switching between 5+ apps can now happen in a single conversation with Rube.

What’s Still Missing

While I was trying Rube, I realized there are some practical limitations and gaps to keep in mind:

1) While Rube can chain actions across apps, very complex sequences (like if this, do that; else do something else) can still confuse the AI. You may need to break tasks into smaller steps, manually supervise or guide the assistant more explicitly.

2) Rube supports 500+ apps but not every niche tool is integrated yet. And not everyone will switch apps because they always have a preference.

3) Rube can’t execute tasks locally since it relies on cloud connectivity and Composio’s servers.

4) There is no 100% guarantee that AI will never misinterpret ambiguous instructions or generate unintended actions. So always review critical commands, especially when automating emails, payments or database updates.

I personally don't feel there is any learning curve, since you can start using it with a single click.

This is the closest I have seen to a real “AI that gets things done”.

If you have ever wished ChatGPT could just take care of the boring stuff, Rube is as close as it gets right now.

Have a great day! Until next time :)

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

I built a self-hosted Google Forms alternative and made it open source! 🎉

Anmol Baranwal — Sat, 22 Nov 2025 10:58:37 +0000

Over the past few weeks, I have been building a Google Forms alternative but with a huge twist.

Rather than creating forms manually, you can chat to develop forms and those forms go live instantly for submissions.

Under the hood, it’s powered by a streaming LLM connected to Next.js & C1 by Thesys. The form spec and submissions are stored in MongoDB.

It would be hard to cover the complete codebase but here are all the important details and everything I learned building this.

What is covered?

In summary, we are going to cover these topics in detail.

The vision behind the project.
Tech Stack Used.
Architecture Overview.
Data Flow: From Prompt → Form → Save
How It Works (Under the Hood).

You can check the GitHub Repository.

1. The vision behind the project.

I was using Google Forms a few months back and realized it still requires you to build forms manually, which works fine but feels outdated.

So I wondered: what if creating a form were as easy as describing it?

Something like: “I want a registration form with name, email, password and a feedback textbox”. The AI would take that, generate the UI spec automatically and render it instantly in the browser.

Each form gets its own unique URL for collecting submissions and with a simple cookie-based authentication so only you can create & view the list of forms with their submissions.

Around this time, I came across Thesys and its C1 API, which made it easy to turn natural language descriptions into structured UI components. That sparked the idea to build this project on top of it.

Unlike hosted form tools, this one is completely self-hosted, your data and submissions stay in your own database.

Here is the complete demo showing the flow!

This wasn’t about solving a big problem. It was more of an experiment in understanding how chat-based apps and generative UI systems work under the hood.

To use the application:

Fork the repository.
Set your admin password and other credentials in .env (check the format below).
Deploy it on any hosting provider (Vercel, Netlify, Render) or your own server.
Visit /login and enter your admin password.
After successful login, you will be redirected to the chat interface at /.
You can now create forms as needed (see the demo above).

You can copy the .env.example file in the repo and update environment variables.

THESYS_API_KEY=<your-thesys-api-key>
MONGODB_URI=<your-mongodb-uri>
THESYS_MODEL=c1/anthropic/claude-sonnet-4/v-20250930
ADMIN_PASSWORD=<your-admin-password>

If you want to use any other model, you can find the list of stable models recommended for production and how their pricing is calculated in the documentation.

Setting up Thesys

The easiest way to get started is using CLI that sets up an API key and bootstraps a NextJS template to use C1.

npx create-c1-app

But let's briefly understand how Thesys works (which is the core foundation):

✅ First, update the OpenAI client configuration to point to Thesys by setting the baseURL to api.thesys.dev and supplying your THESYS_API_KEY. You get an OpenAI‐style interface backed by Thesys under the hood.

// Prepare Thesys API call (OpenAI‑compatible)
    const client = new OpenAI({
      baseURL: 'https://api.thesys.dev/v1/embed',
      apiKey: process.env.THESYS_API_KEY,
    })

✅ Calling a streaming chat completion. Setting stream: true lets us progressively emit tokens back to the browser for real‑time UI rendering.

const llmStream = await client.chat.completions.create({
      model: process.env.THESYS_MODEL || 'c1/anthropic/claude-sonnet-4/v-20250930',
      messages: messagesToSend,
      stream: true,
    })

✅ By wrapping the raw LLM stream in our transformStream helper, we extract just the tokenized content deltas and pipe them as an HTTP-streamable response.

const responseStream = transformStream(
      llmStream,
      (chunk) => chunk?.choices?.[0]?.delta?.content ?? '',
    )

    return new NextResponse(responseStream, {
      headers: {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache, no-transform',
        Connection: 'keep-alive',
      },
    })

You can try it live on the playground.

2. Tech Stack Used

The tech stack is simple enough with:

Next.js (App Router) : routing, server rendering and API endpoints
Thesys GenUI SDK (@thesysai/genui-sdk) : powers the chat interface (C1Chat) and renders generated forms (C1Component)
C1 by Thesys (Anthropic model) : augments LLMs to respond with interactive UI schemas based on prompts
@crayonai/stream : streams LLM output to the front end in real time
MongoDB & Mongoose : stores form definitions and user submissions in DB
Node.js (Next.js API routes + middleware) : handles backend logic for chat, CRUD, and authentication

If you want to read more about how Thesys Works using C1 API and GenUI React SDK, check out this blog.

Project Structure

Most of the work lives under the src directory, with the Next.js App Router and API routes:

.
├── .env.example              
├── .gitignore
├── LICENSE                   
├── next.config.ts            
├── package.json              
├── postcss.config.mjs        
├── tsconfig.json       
├── middleware.ts      
├── public/                   
└── src/
    ├── app/                  # Next.js App Router
    │   ├── api/              # Serverless API routes
    │   │   ├── chat/route.ts # Chat endpoint
    │   │   └── forms/        # Form CRUD + submissions
    │   │       ├── [id]/     # Form-specific endpoints
    │   │       │   ├── submissions/
    │   │       │   │   ├── [submissionId]/
    │   │       │   │   │   └── route.ts   # Delete submission of a form
    │   │       │   │   └── route.ts       # GET form submissions
    │   │       ├── create/route.ts        # Create new form
    │   │       ├── delete/route.ts        # Delete form by ID
    │   │       ├── get/route.ts           # Get form by ID
    │   │       ├── list/route.ts          # List all forms
    │   │       └── submit/route.ts        # Handle form submission
    │   │
    │   ├── assets/            # Local fonts
    │   ├── forms/             
    │   │   ├── [id]/          # Dynamic form route
    │   │   │   ├── submissions/
    │   │   │   │   └── page.tsx  # Show all submissions for a form
    │   │   │   └── page.tsx      # Show a single form (renders via C1Component)
    │   │   └── page.tsx          # All forms listing page
    │   │
    │   ├── home/              # Landing page (when not logged in)
    │   │   └── page.tsx
    │   ├── favicon.ico
    │   ├── globals.css
    │   ├── layout.tsx
    │   └── page.tsx              
    │
    ├── components/  
    │   ├──C1ChatWrapper.tsx
    |   ├──ClientApp.tsx
    |   ├──FormsListPage.ts
    │   └──SubmissionsPage.tsx
    │
    └── lib/                      
        ├── dbConnect.ts          # MongoDB connection helper
        ├── fonts.ts              # Next.js font setup
        ├── models/               # Mongoose models
        │   ├── Form.ts
        │   └── Submission.ts
        └── utils.ts

Page Routes

Here are all the Page Routes:

/home – Landing page (shown when not logged in)
/login – Admin login page
/ – Chat interface (requires authentication)
/forms – List all forms
/forms/[id] – Render a specific form
/forms/[id]/submissions – List submissions for a specific form

Here are all the API Routes:

POST /api/login – Authenticate and set session cookie
POST /api/chat – AI chat endpoint
GET /api/forms/list – Get all forms
POST /api/forms/create – Create a new form
GET /api/forms/get – Get form schema by ID
DELETE /api/forms/delete – Delete a form by ID
POST /api/forms/submit – Submit a form response
GET /api/forms/[id] – List submissions for a form
DELETE /api/forms/[id]/submissions – Delete a submission by ID

3. Architecture Overview.

Here's the high-level architecture of the project.

4. Data Flow: From Prompt → Form → Save

Here is the complete sequence diagram from Form Creation → Save.

5. How It Works (Under the Hood)

Below is an end‑to‑end walkthrough of the main user‑facing flows, tying together chat, form generation, rendering and everything in between.

System Prompt

Here is the system prompt:

const systemPrompt = `
You are a form-builder assistant.
Rules:
- If the user asks to create a form, respond with a UI JSON spec wrapped in <content>...</content>.
- Use components like "Form", "Field", "Input", "Select" etc.
- If the user says "save this form" or equivalent:
  - DO NOT generate any new form or UI elements.
  - Instead, acknowledge the save implicitly.
  - When asking the user for form title and description, generate a form with name="save-form" and two fields:
    - Input with name="formTitle"
    - TextArea with name="formDescription"
    - Do not change these property names.
  - Wait until the user provides both title and description.
  - Only after receiving title and description, confirm saving and drive the saving logic on the backend.
- Avoid plain text outside <content> for form outputs.
- For non-form queries reply normally.
<ui_rules>
- Wrap UI JSON in <content> tags so GenUI can render it.
</ui_rules>
`

As you can see, it behaves like a Form Builder Assistant.

You can read the official docs for a step-by-step guide on how to add a system prompt to your application.

Chat‑Driven Form Design

User types a prompt in the chat widget (C1Chat).
The frontend sends the user message(s) via SSE (fetch('/api/chat')) to the chat API.
/api/chat constructs an LLM request:
- Prepends a system prompt that tells the model to emit JSON UI specs inside <content>…</content>.
- Streams responses back to the client.
As chunks arrive, @crayonai/stream pipes them into the live chat component and accumulates the output.
On the stream end, the API:
- Extracts the <content>…</content> payload.
- Parses it as JSON.
- Caches the latest schema (in a global var) for potential “save” actions.
- If the user issues a save intent, it POSTs the cached schema plus title/description to /api/forms/create.

export async function POST(req: NextRequest) {
  try {
    const incoming = await req.json()
    // Normalize client structure...
    const messagesToSend = [
      { role: 'system', content: systemPrompt },
      ...incomingMessages,
    ]

    const client = new OpenAI({
      baseURL: 'https://api.thesys.dev/v1/embed',
      apiKey: process.env.THESYS_API_KEY,
    })


    const llmStream = await client.chat.completions.create({
      model:
        process.env.THESYS_MODEL ||
        'c1/anthropic/claude-sonnet-4/v-20250930',
      messages: messagesToSend,
      stream: true,
    })

    const responseStream = transformStream(
      llmStream,
      (chunk) => chunk?.choices?.[0]?.delta?.content ?? '',
      {
        onEnd: async ({ accumulated }) => {
          const rawSpec = Array.isArray(accumulated)
            ? accumulated.join('')
            : accumulated
          const match = rawSpec.match(/<content>([\s\S]+)<\/content>/)
          if (match) {
            const schema = JSON.parse(decodeHtmlEntities(match[1].trim()))
            globalForFormCache.lastFormSpec = schema
          }
          if (isSaveIntent(incomingMessages)) {
            const { title, description } = extractTitleDesc(incomingMessages)
            await fetch(`${req.nextUrl.origin}/api/forms/create`, {
              method: 'POST',
              headers: { 'Content-Type': 'application/json' },
              body: JSON.stringify({
                title,
                description,
                schema: globalForFormCache.lastFormSpec,
              }),
            })
          }
        },
      }
    ) as ReadableStream<string>

    return new NextResponse(responseStream, {
      headers: {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache, no-transform',
        Connection: 'keep-alive',
      },
    })
  } catch (err: any) {
    /* …error handling… */
  }
}

Here is the chat page.

Storing a New Form

Here's how it stores form:

POST /api/forms/create (serverless route) receives { title, description, schema }.
It calls dbConnect() to get a Mongo connection (with connection‑caching logic).
It writes a new Form document (Mongoose model) with your UI‑schema JSON.

export async function POST(req: NextRequest) {
  await dbConnect()
  const { title, description, schema } = await req.json()
  const form = await Form.create({ title, description, schema })
  return NextResponse.json({ id: form._id, success: true })
}

Let's also see how all the forms are listed using this API route src\app\api\forms\list\route.ts.

export const runtime = 'nodejs'
import { NextResponse } from 'next/server'
import { dbConnect } from '@/lib/dbConnect'
import Form from '@/lib/models/Form'
export async function GET() {
  await dbConnect()
  const forms = await Form.find({}, '_id title description createdAt')
    .sort({ createdAt: -1 })
    .lean()

  const formattedForms = forms.map((f) => ({
    id: String(f._id),
    title: f.title,
    description: f.description,
    createdAt: f.createdAt,
  }))
  return NextResponse.json({ forms: formattedForms })
}

Here is the listing page.

Rendering the Generated Form

Visitors navigate to /forms/[id].
The page’s useEffect() fetches the stored schema from GET /api/forms/get?id=[id].
It wraps the raw JSON in <content>…</content> and passes it to C1Component, which renders the fields, inputs, selects and more.

Each form is rendered at src/app/forms/[id]/page.tsx.

useEffect(() => {
  async function fetchForm() {
    const res = await fetch(`/api/forms/get?id=${id}`)
    const data = await res.json()
    const wrappedSpec = `<content>${JSON.stringify(data.schema)}</content>`
    setC1Response(wrappedSpec)
  }
  if (id) fetchForm()
}, [id])

if (!c1Response) return <div>Loading...</div>

return (
  <C1Component
    key={resetKey}
    c1Response={c1Response}
    isStreaming={false}
    onAction={/* …see next section… */}
  />
)

Here is an example of a generated form.

Handling Form Submission

When the user fills and submits the form, C1Component fires an onAction callback.
The callback POSTs { formId, response } to /api/forms/submit.
The server writes a new Submission document linking back to the Form.

Here is the Submission Mongoose Model.

const SubmissionSchema = new mongoose.Schema({
   formId: { type: mongoose.Schema.Types.ObjectId, ref: 'Form' },
   response: Object,    // The filled (submitted) data JSON
   createdAt: { type: Date, default: Date.now },
 })
 export default mongoose.models.Submission ||
   mongoose.model('Submission', SubmissionSchema)

Let's also see how all submissions are shown using API route src\app\api\forms\[id]\submissions\route.ts.

import { NextRequest, NextResponse } from 'next/server'
import { dbConnect } from '@/lib/dbConnect'
import Submission from '@/lib/models/Submission'

export async function GET(
  req: NextRequest,
  context: { params: Promise<{ id: string }> }
) {
  await dbConnect()

  const { id } = await context.params
  try {
    const submissions = await Submission.find({ formId: id }).sort({
      createdAt: -1,
    })
    return NextResponse.json({ success: true, submissions })
  } catch (err) {
    console.error('Error fetching submissions:', err)
    return NextResponse.json(
      { success: false, error: 'Failed to fetch submissions' },
      { status: 500 }
    )
  }
}

Here’s the submissions view. You can also delete entries or export all responses in Markdown.

Admin Listing & Deletion

Under the authenticated area, you can list all forms (GET /api/forms/list), view/delete individual forms and inspect submissions: each via dedicated API routes.

Here is a small snippet to list all forms (src/app/api/forms/list/route.ts).

export async function GET() {
  await dbConnect()
  const forms = await Form.find({}, '_id title description createdAt')
    .sort({ createdAt: -1 })
    .lean()

  const formattedForms = forms.map(f => ({
    id: String(f._id),
    title: f.title,
    description: f.description,
    createdAt: f.createdAt,
  }))
  return NextResponse.json({ forms: formattedForms })
}

Here is a small snippet to delete the form (src/app/api/forms/delete/route.ts).

export async function DELETE(req: NextRequest) {
  await dbConnect()

  try {
    const { id } = await req.json()
    if (!id) {
      return NextResponse.json(
        { success: false, error: 'Form ID is required' },
        { status: 400 }
      )
    }
    await Form.findByIdAndDelete(id)
    return NextResponse.json({ success: true })
  } catch (err) {
    console.error('Error deleting form:', err)
    return NextResponse.json(
      { success: false, error: 'Failed to delete form' },
      { status: 500 }
    )
  }
}

Authentication Flow

It's a simple admin auth for creating and deleting forms. This is how it's implemented:

✅ Login Endpoint - POST /api/login checks the provided { password } against process.env.ADMIN_PASSWORD. On success, it sets a secure, HTTP-only cookie named auth.

✅ Middleware Protection - A middleware file (middleware.ts) inspects the auth cookie. If the user isn’t authenticated, they’re redirected from / to the public /home page.

✅ Environment Variable - Add ADMIN_PASSWORD in your .env (also included in .env.example) so the login route can verify credentials.

So the listing forms page, chat page and the submissions page are protected using this method.

There’s a lot to improve: better schema validation, versioning, maybe even multi-user sessions.

But it already does what I hoped for: you talk, it builds and suddenly you have something that works. Let me know what you think of the app in the comments.

Have a great day! Until next time :)

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Forem: Anmol Baranwal

AIMock: One Mock Server For Your Entire AI Stack

Why we built AIMock

Table of Contents

AI apps in 2026 call a lot more than just an LLM

Introducing AIMock: mock your entire agentic stack

LLMock: full streaming, tool calls, reasoning across every major provider

MCPMock: a mock MCP server for testing tool integrations

A2AMock: test multi-agent workflows without live agents

VectorMock: deterministic retrieval for RAG pipelines

Services: search, rerank, and moderation

Drift Detection: catch provider changes before your users do

Record and Replay: stop hand-writing fixtures

Chaos Testing: prove your app handles failure

What else is included

AG-UI uses AIMock in production

Migrating to AIMock

Get started

CopilotKit Follow

Google Maps for Codebases: Paste a GitHub URL, Ask Anything

What are we building?

Tech Stack & Tools

What is Zenflow?

Architecture & Project Structure

How everything works under the hood

1. Loading a repository

2. From architecture view to dependency graph

3. The CopilotKit runtime

4. Feeding the LLM context

5. The four frontend tools

6. Finding relevant files

7. File caching

8. Parsing imports and resolving paths

9. Graph layout with Dagre

10. State management with Zustand

Running it locally

Using OpenAI

The bigger picture

CopilotKit Follow

Deploying AG-UI Agents to Production with Amazon Bedrock AgentCore

Amazon Bedrock AgentCore Runtime

AG-UI: the interaction protocol

How It All Connects

Deploying your AG-UI agent with Strands

Step 1: Build your AG-UI server

Step 2: Test locally

Step 3: Deploy to AgentCore

Step 4: Invoke your deployed agent

Step 5: Connect your frontend

Resources

What This Means

CopilotKit Follow

How to add AI to your App in 5 minutes

Why AI Integrations Get Complicated

What CopilotKit Actually Does

How to wire everything (1 minute)

Give the AI Context

Let the AI Do Things

Bonus: Generative UI

CopilotKit Follow

Give Your AG2 Agents a UI with AG‑UI and CopilotKit

AG2: Runtime for Agent Teams

The Gap: Orchestration Without an Application Contract

AG-UI: The Execution & Interaction Layer

How CopilotKit wires AG2 to AG-UI

Integration Flow: AG2 → AG-UI → CopilotKit

Multi-Agent Orchestration

What This Enables for Agent Builders

Getting Started

What's Next

How to Build a Research Assistant using Deep Agents

What is covered?

1. What are Deep Agents?

The mental model (how it runs)

Where CopilotKit Fits

2. Core Components

How Deep Agents are implemented (Middleware)

3. What are we building?

4. Frontend: wiring the agent to the UI

Step 1: CopilotKit Provider & Layout