Forem: Farhan Ahmad

Build stateful AI agents using Mem0

Farhan Ahmad — Sat, 30 Aug 2025 11:52:35 +0000

You may have interacted with (or built) many AI bots in the last few years. Most of them serve a basic purpose (like helping with customer support) but many lack the ability to remember the user's preferences and conversations from previous sessions (which means the agents are stateless), so every new session with the AI agent feels like you're starting with a blank slate.

This is where the concept of stateful agents comes in.
An an Agent that actually remembers your user's preferences and past behaviors (a stateful agent), and then uses that info to guide the user is extremely helpful. This helps the user reach their goals faster and the conversation flows more smoothly which ultimately keeps your users happy with your product/service.

Let's see how Mem0 makes it possible to build stateful agents in a few easy steps.

What's Mem0?

To put it simply, Mem0 is a framework that helps your agent remember the user's past behaviors and preferences by storing the conversation and also helping you retrieve it as needed, using natural language. It does the heavy lifting for us, all we have to do is add a couple of lines of code in our app and we're good to go.

With Mem0 you can:

Store conversations
Get older conversations
Search Conversations using natural language
Update stored conversations as needed

In this tutorial we'll build a very simple Next.js app that showcases the power of this framework. We'll keep it super easy to follow and beginner friendly :)

Overview of what we're building today

We're going to be building a very basic but powerful stateful agent. We'll be using the below stack for this:

Typescript framework - Next.js
LLM provider - OpenAI
Memory layer - Mem0

Our goal is to demonstrate the concept of stateful agents using Mem0, which is why we'll keep this tutorial simple and beginner-friendly.

Getting Started (Next.js project initialization)

Alright, we'll start with initializing a Next.js app using create-next-app (I know, I'm a lazy guy).

If you don't already have npx installed, use this command to install it:
npm install -g npx

Fire up your favorite text editor, open the command line and type the below command to initialize a new project:
npx create-next-app@latest

It's going to ask you a bunch of questions, starting with the project name, linter choice, etc. If you're new, you can just follow my lead according to the screenshot below:

As you can see, the project name I specified was "smart-agent". After this step, a new project gets initialized with the name "smart-agent" (or whatever name that you chose to specify) on the left hand side in VS Code (see screenshot below).

A good way to get familiar with Next.js is to expand/collapse each folder in this project to get a feel for how things are structured and how different components inside the app interact with each other.

Try running the app to see all's good

Use the command below to switch to the project directory inside the CLI (remember, in my case the name of the agent is "smart-agent"):
cd smart-agent

Now run the below command to run the server:
npm run dev

(Screenshot for reference)

Now when you open your browser and search for "localhost:3000" you should see something like this:

Cool! That means our app is officially set up. Let's add some features now.

Creating a simple chat feature

For the next step we'll create a simple app that will help us interact with the LLM using the OpenAI API. We're not going to worry about adding the "remembering feature" just now to keep things simple.

The very first thing we'll need is the OpenAI API key. Let's go grab it from OpenAI's website. Go to the URL https://platform.openai.com/api-keys where you can either use an existing API key or create a new one.

Once you have copied the API key, come back to VS code and add it to our project's environment variable. To do this we'll create a new file called ".env.local" signifying that this file is for the local environment. So when we run npm run dev and start the server, the server will pick up all values specified here. You can refer to the screenshot below.

Cool, now that we have added the API key, let's go ahead and install the official openai package to interact with the LLM API.

Run the below command to install the "openai" package:
npm i openai

You can see below that the openai package was added to the package.json file:

Alright let's now build the UI and logic for our chat app

Since our app was initialized using create-next-app, the page.tsx file is the easiest way to update the UI of our app. To keep this tutorial super easy, we'll only update page.tsx file (and one server file) to build out our app.

Go ahead and copy the below code and paste it in your page.tsx file (replacing the older code):

"use client"

import { useEffect, useRef, useState } from "react"

type Role = "system" | "user" | "assistant"

export type ChatMessage = {
  role: Role
  content: string
}

export default function Page() {
  return <Chat />
}

function Chat() {
  const [messages, setMessages] = useState<ChatMessage[]>([
    {
      role: "system",
      content: "You are a concise, friendly assistant. Keep answers under ~120 words unless asked.",
    },
  ])

  const [input, setInput] = useState("")
  const [pending, setPending] = useState(false)
  const [error, setError] = useState<string | null>(null)
  const bottomRef = useRef<HTMLDivElement>(null)

  useEffect(() => {
    bottomRef.current?.scrollIntoView({ behavior: "smooth" })
  }, [messages, pending])

  async function send() {
    const text = input.trim()
    if (!text || pending) return

    // Add the user's message locally first
    const next = [...messages, { role: "user" as const, content: text }]
    setMessages(next)
    setInput("")
    setPending(true)
    setError(null)

    try {
      const res = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ messages: next }),
      })

      if (!res.ok) throw new Error(`HTTP ${res.status}`)

      const data: { reply?: string } = await res.json()
      const reply = data.reply ?? "(No response)"

      setMessages((cur) => [...cur, { role: "assistant", content: reply }])

      /* eslint-disable @typescript-eslint/no-explicit-any */
    } catch (e: any) {
      console.error(e)
      setError(e?.message || "Something went wrong.")
      setMessages((cur) => [...cur, { role: "assistant", content: "Sorry—something went wrong. Try again." }])
    } finally {
      setPending(false)
    }
  }

  function onSubmit(e: React.FormEvent) {
    e.preventDefault()
    void send()
  }

  return (
    <div className="flex flex-col h-[100svh] max-w-2xl mx-auto p-4 gap-3">
      <header className="flex items-center justify-between">
        <h1 className="text-xl font-semibold">Simple Chat</h1>
      </header>

      <main className="flex-1 overflow-y-auto rounded-2xl border bg-white p-3 space-y-3 text-black">
        {/* Hide system message from the transcript */}
        {messages
          .filter((m) => m.role !== "system")
          .map((m, i) => (
            <Bubble key={i} role={m.role} text={m.content} />
          ))}
        {pending && <Bubble role="assistant" text="…thinking" muted />}
        <div ref={bottomRef} />
      </main>

      <form onSubmit={onSubmit} className="flex gap-2">
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Type a message and press Enter…"
          className="flex-1 px-4 py-2 rounded-xl border outline-none"
          aria-label="Message"
        />
        <button type="submit" disabled={!input.trim() || pending} className="px-4 py-2 rounded-xl border shadow disabled:opacity-50">
          Send
        </button>
      </form>

      {error && (
        <p className="text-xs text-red-600" role="alert">
          {error}
        </p>
      )}
    </div>
  )
}

function Bubble({ role, text, muted }: { role: Role; text: string; muted?: boolean }) {
  const isUser = role === "user"
  return (
    <div
      className={
        "max-w-[85%] whitespace-pre-wrap px-4 py-2 rounded-2xl shadow " +
        (isUser ? "bg-gray-200 ml-auto" : "bg-white border") +
        (muted ? " text-gray-500" : "")
      }
    >
      {text}
    </div>
  )
}

Once you're done updating your page.tsx file, open your browser and visit localhost:3000 URL and you'll see something like this:

Our app is taking shape for sure! But if you try sending a message it'll throw an error, because we have only implemented the UI, and we still need to add the logic to communicate with OpenAI's LLM.

Adding a new route to handle requests to OpenAI API

Whenever the user sends a message, we want that message to be sent to OpenAI's LLM and receive back a response. Finally, we want to add this LLM's response to the chat UI. Age old game of state updates and API requests. Let's get it going.

Create a new folder inside the app folder named api, and within api folder create another folder named chat. Inside the chat folder create a new file called route.ts as shown in the screenshot:

As you can see that our route.ts file is empty. Let's paste the below code into the file:

import { NextRequest, NextResponse } from "next/server"
import OpenAI from "openai"

export const runtime = "nodejs"
export const dynamic = "force-dynamic" // avoid caching

export async function POST(req: NextRequest) {
  if (!process.env.OPENAI_API_KEY) {
    return NextResponse.json({ error: "Missing OPENAI_API_KEY on the server." }, { status: 500 })
  }

  let body: unknown
  try {
    body = await req.json()
  } catch {
    return NextResponse.json({ error: "Invalid JSON body." }, { status: 400 })
  }
  /* eslint-disable @typescript-eslint/no-explicit-any */
  const { messages } = (body as any) || {}

  if (!Array.isArray(messages) || messages.length === 0) {
    return NextResponse.json({ error: "'messages' must be a non-empty array." }, { status: 400 })
  }

  try {
    const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

    const completion = await openai.chat.completions.create({
      model: "gpt-4.1-nano", // an affordable model here
      messages,
    })

    const reply = completion.choices?.[0]?.message?.content?.trim() ?? ""

    return NextResponse.json({ reply })
  } catch (err: any) {
    console.error("/api/chat error:", err?.response?.data || err?.message || err)
    return NextResponse.json({ error: "Something went wrong generating a reply." }, { status: 500 })
  }
}

Great, now run the server if it isn't already using npm run dev and you should be able to talk to the AI as shown below:

But.. why do we want our Agent to remember?

Before we get into implementing the memory feature, lets quickly demonstrate why we need this memory feature.

As shown in the GIF below, try telling the agent something about your likes/preferences (for example, "My favorite travel destination is Japan"). Then go ahead and refresh the page. Once the page is refreshed ask the agent about your preference that you had mentioned earlier ("What's my favorite travel destination?").

You'll quickly notice that the agent doesn't remember anything about you:

And this is why we want to add memory to the agent. This will help the agent remember user's preferences across different user sessions, as you'll see soon.

Adding the ability to remember using Mem0

Okay now that our simple chat feature is ready let's give our app the ability to remember user's preferences from past conversations. We'll use Mem0 for implementing this feature. If you are someone who prefers reading docs first, feel free to read Mem0's quick start guide if you want to dive deeper using this link: Mem0's Quick-start guide.

Two ways to use Mem0

Broadly speaking, there are two ways to use Mem0. You can either:

Use Mem0's cloud platform to manage your conversations using a MEM0_API_KEY (easiest)
or, you can choose to host Mem0's open-source version on your own infra (hard + time consuming)

In this tutorial, for the sake of keeping things simple, we'll use Mem0's platform for building out our agent in an instant.

Setting up Mem0 in our project

Let's start by installing the npm package mem0ai using the below command:
npm install mem0ai

After running the command our package.json should be updated as shown below:

Now that we have the package installed, there is one more step that's required before we can start using Mem0. We need to grab an API key for Mem0's website to actually make it work.

Head over to this URL https://app.mem0.ai/login and create a new account if you haven't already.

Once you're done, head over to https://app.mem0.ai/dashboard/get-started and copy your API key. At the time when I'm writing this, that page should look something like below:

Once you've copied the API key come back to your code editor and open the .env.local file and add the API key you just copied as MEM0_API_KEY. See the screenshot below for reference.

Nice! let's start implementing the memory feature using the mem0 library.

Implementing Mem0 in our route.ts file

We know that Mem0 allows us to store conversations and search from older one's using natural language. Let's now see it in action by implementing it in our app.

Go into the route.ts file that we created earlier inside app/api/chat folder. Next, replace the code inside route.ts file with the code below:

import { NextRequest, NextResponse } from "next/server"
import OpenAI from "openai"
import MemoryClient from "mem0ai"
import { ChatCompletionMessageParam } from "openai/resources"

export const runtime = "nodejs"
export const dynamic = "force-dynamic" // avoid caching

export async function POST(req: NextRequest) {
  const mem0 = new MemoryClient({ apiKey: process.env.MEM0_API_KEY! })

  if (!process.env.OPENAI_API_KEY) {
    return NextResponse.json({ error: "Missing OPENAI_API_KEY on the server." }, { status: 500 })
  }

  let body: unknown
  try {
    body = await req.json()
  } catch {
    return NextResponse.json({ error: "Invalid JSON body." }, { status: 400 })
  }
  /* eslint-disable @typescript-eslint/no-explicit-any */
  const { messages } = (body as any) || {}

  if (!Array.isArray(messages) || messages.length === 0) {
    return NextResponse.json({ error: "'messages' must be a non-empty array." }, { status: 400 })
  }

  const userId: string = (body as any)?.userId || "demo-user"
  const lastUserContent = [...messages].reverse().find((m: any) => m.role === "user")?.content || ""

  let memoryContext = ""
  try {
    console.log("searching from memory...")
    const hits = await mem0.search(lastUserContent || "recent preferences", { user_id: userId })
    // Each hit can be an object; we try common fields (text/memory/value)
    const lines = (Array.isArray(hits) ? hits : [])
      .map((m: any) => m?.text ?? m?.memory ?? m?.value ?? "")
      .filter(Boolean)
      .slice(0, 6)
    if (lines.length) {
      memoryContext = "Known about user (from memory): \n" + lines.join(" - ")
    }
  } catch (e) {
    console.log("mem0.search error", e)
  }

  // Prepend memory context so the model can personalize
  const messagesWithMemory: ChatCompletionMessageParam[] = memoryContext
    ? [
        { role: "system", content: "Use the following user memory if helpful. Don’t repeat it verbatim." },
        { role: "system", content: memoryContext },
        ...messages,
      ]
    : messages

  try {
    const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

    const completion = await openai.chat.completions.create({
      model: "gpt-4.1-nano", // an affordable model here
      messages: messagesWithMemory,
    })
    const reply = completion.choices?.[0]?.message?.content?.trim() ?? ""

    try {
      // Grab the latest user turn as a natural-language search query for Mem0
      console.log("adding to memory...")
      const lastUserMessage = [...messages].reverse().find((m: any) => m?.role === "user")
      const messagesToStore = [
        lastUserMessage ? { role: "user", content: lastUserMessage.content } : messages[messages.length - 1],
        { role: "assistant", content: reply },
      ]
      await mem0.add(messagesToStore as any, { user_id: userId, metadata: { category: "chat" } })
    } catch (e) {
      console.log("mem0.add failed:", e)
    }

    return NextResponse.json({ reply })
  } catch (err: any) {
    console.error("/api/chat error:", err?.response?.data || err?.message || err)
    return NextResponse.json({ error: "Something went wrong generating a reply." }, { status: 500 })
  }
}

Can you spot the changes that we made? Let's go through these new changes one by one below.

First, we initialized a new mem0 client:
const mem0 = new MemoryClient({ apiKey: process.env.MEM0_API_KEY! })

Notice that we're passing the MEM0_API_KEY environment variable we had added earlier to .env.local file.

Next, we added this block of code to search the conversation in Mem0 using user's recent message:

const userId: string = (body as any)?.userId || "demo-user"
  const lastUserContent = [...messages].reverse().find((m: any) => m.role === "user")?.content || ""

  let memoryContext = ""
  try {
    console.log("searching from memory...")
    const hits = await mem0.search(lastUserContent || "recent preferences", { user_id: userId })
    // Each hit can be an object; we try common fields (text/memory/value)
    const lines = (Array.isArray(hits) ? hits : [])
      .map((m: any) => m?.text ?? m?.memory ?? m?.value ?? "")
      .filter(Boolean)
      .slice(0, 6)
    if (lines.length) {
      memoryContext = "Known about user (from memory): \n" + lines.join(" - ")
    }
  } catch (e) {
    console.log("mem0.search error", e)
  }

As you can see, every conversation thats stored needs some kind of identifier. In our app we haven't implemented authentication which is why we'll simply mock the user_id value to be "demo-user".
Then, we use mem0.search() to search for any memory that could help us learn about the user's preferences that belongs to the unique user_id.

And now finally, we've also added logic to store the conversations in Mem0 as shown below:

try {
      // Grab the latest user turn as a natural-language search query for Mem0
      console.log("adding to memory...")
      const lastUserMessage = [...messages].reverse().find((m: any) => m?.role === "user")
      const messagesToStore = [
        lastUserMessage ? { role: "user", content: lastUserMessage.content } : messages[messages.length - 1],
        { role: "assistant", content: reply },
      ]
      await mem0.add(messagesToStore as any, { user_id: userId, metadata: { category: "chat" } })
    } catch (e) {
      console.log("mem0.add failed:", e)
    }

The above code adds the last assistant message + the last user message to conversation. We do this so that only the new messages get stored in Mem0 and not the entire conversation history. This helps us save cost and prevents duplicate memory from being saved in Mem0.

And that's pretty much all we need! Our agent now has the ability to learn user's preferences and remember them across sessions. Let's try it out.

Trying out our Mem0 Agent

Okay let's take for a run now. We'll tell the agent about our favorite color, refresh the page and ask it about our favorite color again to see if it remembers anything.

Below is a GIF of me doing it:

Voila! Your agent will now remember anything you talk about, even across sessions.

How can we see the stored memories in Mem0?

Yes you can also see the memories that are stored with Mem0 for every user_id. Go to your Mem0 dashboard here: https://app.mem0.ai/dashboard/memories and you'll see something like this:

Notice that every memory is associated with a user_id. The memory "favorite color is blue" is associated with "demo-user".

Conclusion

There's no doubt stateful agents will become the standard in the near future. We'll stop seeing generic customer chat bots and instead, chat bots that remember everything about you from your past interactions will become the norm.
This is still the beginning and the reason I like solutions like Mem0 so much is because how easy it is to use for beginners to build complex stateful agents in no time.

As always, if you found this article useful, feel free to follow me here or on Linkedin (https://www.linkedin.com/in/farhans-profile/).

Cheers!

Step-by-Step Guide to Building Domain-Specific AI Agents with Phidata SDK

Farhan Ahmad — Wed, 22 Jan 2025 09:21:03 +0000

AI Agents are gaining a lot of popularity these days and businesses are rushing to build their own agents. Some prefer building an AI Agent from scratch, but what often bothers devs the most is the time is takes to build and test different AI Tools (for function calling). This can lead to a high development time.

However as time passes we're seeing new solutions popping up every now and then to help develoeprs streamline the process of building AI Agents. One such solution is Phidata.

What is Phidata?

Phidata SDK makes it super easy to build AI Agents in just a few lines of Python code. The best part is that it provides useful AI Tools right out of the box, so you don't need to write them from scratch yourself.

Some of the AI Tools Phidata offers out of the box:

Google Search, Exa (for Web Search)
Resend (to send Emails)
Crawl4AI and Firecrawl (to crawl the web)
DuckDB (for Data Analysis)
Python Agent (to write and run Python code)
Files (for reading files to build a RAG Agent)
GitHub (to interact with GitHub)
CalCom Agent (to schedule meetings using Cal.com)

and many more!
You can see an exhaustive list of Tools provided by Phidata here.

Using one or a combination of the above tools, we can build really complex and exciting AI Agents, like:

Data Analyst Agent
Research Agent
Sales Agent
Shopping Agent

just to name a few.

Creating Advanced Agents using "Team Agent"

Something cool about Phidata SDK is that you have the option to combine multiple Tools into a team, known as a "Team Agent". For example, you can create a team with the "DuckDuckGo" and "Yahoo Finance" Tools which can get the data from both sources, the web as well as Yahoo Finance.
This Team Agent will work the following way:

The user gives the following command to the Agent: "Summarize analyst recommendations and share the latest news for NVDA"
Our agent searches the web for the latest news on the company Nvidia
Agent also searches for the ticker "NVDA" on Yahoo Finance to get the company's financial data
Finally, our Agent presents the data from both sources (the Web and Yahoo Finance) in a nice table.

Pretty cool huh?

So now that we have some idea of what's possible using the Phidata SDK, let's go ahead and see how we can use it to create a simple Financial Analyst Agent.

Getting Started with Phidata SDK

Let's start building our Financial Analyst Agent. It's going to be super beginner-friendly so don't worry about falling behind.

We're going to create our Agent inside a Jupyter Notebook on Google Colab, which is very interactive and easy to share.

Step 1 - Create a new Notebook on Google Colab

Go to Google Colab using by clicking here and you should see this screen:

Now click on the "New Notebook" button:

It may load for a while, but after that you should arrive at the newly created notebook which will look like this:

Great, let's move to the next step.

Step 2 - Installing all required libraries in the Notebook

Before we can start creating our AI agent, we need to make sure we have the required dependencies available inside the Notebook. Note that Google Colab comes with some common libraries pre-installed in the Notebook, but to make sure we have all the libraries we need, we'll install all of them anyway.

We'll install the following libraries:

yfinance - To get company's financial data
openai - To allow Phidata to use OpenAI's LLM to make the AI agent come to life
duckduckgo-search - To search the web using DuckDuckGo
Phidata - To load up the pre-written AI Tools for function calling and to create a Team Agent.

To install these libraries, copy the below command and paste it inside the first cell block:

pip install openai yfinance duckduckgo-search phidata

It should look like this:

Next, click on the play icon on the left as shown below:

Now give it some time to install all dependencies. Once done installing, you should see a tiny green check mark on the left of the Run button like this:

Let's hide the cell's output since it's taking a lot of space in the Notebook. Click on the button right below the "Run" button and then click on "Show/hide output".

Step 3 - Adding the OPENAI_API_KEY environment variable

Now we need to add the OpenAI API key to our environment. Go ahead and add a new cell to the Notebook by clicking on the button as shown below:

Now inside this new cell paste the following code and Run it. Replace the your_api_key value with your actual OpenAI API key that you can get from https://platform.openai.com/api-keys.

import os

os.environ['OPENAI_API_KEY'] = "your_api_key"

It should look something like this:

Step 4 - Write code for the Agent

In this last step we'll write the actual code for the Agent. Since this is an "Agent Team" (which means that it is an AI agent made of multiple agents), we'll start by creating 2 agents using the Phidata SDK, namely web_agent and finance_agent. The web agent will search the web for news about the company and the finance agent will search Yahoo Finance for the company's financial data. Finally, we'll create a third Agent by passing these 2 agents to the "teams" array of this Agent, which will result in the creation of an "Agent Team".
This third Agent will be the final Agent that we'll use for getting a company's data from web and Yahoo Finance.

from phi.agent import Agent
from phi.model.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo
from phi.tools.yfinance import YFinanceTools

web_agent = Agent(
    name="Web Agent",
    role="Search the web for information",
    model=OpenAIChat(id="gpt-4o"),
    tools=[DuckDuckGo()],
    instructions=["Always include sources"],
    show_tool_calls=True,
    markdown=True,
)

finance_agent = Agent(
    name="Finance Agent",
    role="Get financial data",
    model=OpenAIChat(id="gpt-4o"),
    tools=[YFinanceTools(stock_price=True, analyst_recommendations=True, company_info=True)],
    instructions=["Use tables to display data"],
    show_tool_calls=True,
    markdown=True,
)

agent_team = Agent(
    team=[web_agent, finance_agent],
    instructions=["Always include sources", "Use tables to display data"],
    show_tool_calls=True,
    markdown=True,
)

agent_team.print_response("Summarize analyst recommendations and share the latest news for NVDA", stream=True)

Add a new cell to your Notebook. Then copy the code above and paste it into the new cell.

That's pretty much it! Go ahead and run the cell. It will take some time to finish running and once it's done you'll be able to see the output as follows (you'll need to scroll to the bottom):

The Output text looks really small because I had zoomed out in order to fit the entire output in a single screenshot.

So we were able to build this Financial Analyst Agent in a very short amount of time. It's obvious the report is sort of basic and could be more detailed, but we can always improve our Agent to add more data from different sources by adding new Agents to the Team (or by building our own function Tools from scratch).

You can Follow me on LinkedIn to learn more about AI Agents!

Managing AI Tools for Function Calling with Toolhouse SDK

Farhan Ahmad — Tue, 14 Jan 2025 20:13:46 +0000

When building AI agents, one of the most powerful aspects is their ability to manage and execute tools (function calls). Tools can help an agent perform tasks like scraping data, summarizing content, or even solving complex workflows. But as your AI agent grows in size and capabilities, it becomes increasingly difficult to manage/maintain multiple tools.
In this tutorial, we’ll focus on using the Toolhouse SDK to demonstrate how to use pre-built tools and how we can track every single tool call using the platform.

For this example, we’ll build a very simple interface where a user can input a URL and a prompt, and an AI agent will use tools to scrape the webpage and process the data.

Why Tool Management Matters in AI Agents

AI agents are nothing without Tools. They're like arms and legs of the AI agent. Each Tool is a specialized skill or function that the AI relies on to complete a specific task.
User-facing AI agents need to be flawless in their execution of different tasks. And writing AI tools from scratch to implement API integration or web scraping logic is like re-inventing the wheel that also requires maintenance over the long run by the dev team.

These problems are taken care of by Toolhouse. It helps you to:

Choose from a variety of specialized AI Tools for tasks such as web scraping, sending emails, taking screenshots, integrating APIs like the LinkedIn API for searching for profiles on LinkedIn, etc.
Track you AI agent's tool calls on the Toolhouse app.
Use the Toolhouse SDK to implement powerful AI features in no time.

These capabilities simplify your tool management and lets you focus on building smarter AI agents instead of worrying about building/maintaining Tools.

Getting Started with Toolhouse

Alright so let's build an AI-powered web-scraper. Sounds fancy but it's just a single page app that lets you input a URL to scrape and an optional prompt that you want to execute along with the scraped data.
Here's what you'll need:

Node.js (v16 or later).
An OpenAI API key
A Toolhouse API key

Step 1: Set Up Your React Project

We’ll use React to create a simple frontend for managing tool calls. Make sure you have create-react-app installed which we'll use to initialize a new React application. If you don't have it installed, you can do so by running:

npm install -g create-react-app

Open your favorite code editor and inside the terminal type the following:

npx create-react-app ai-scraper

Once it's done creating a new app, change into the project directory:

cd ai-scraper

If you expand the ai-scraper folder, it should look like this:

Great! Let's now start the server:

npm start

It should automatically start a new app at localhost:3000:

Neat! Let's install all the essential libraries now.

Step 2: Install Toolhouse and OpenAI SDKs

These SDKs will let our app interact with the Toolhouse platform and OpenAI models.

npm install @toolhouseai/sdk openai

Step 3: Add the API keys

Create a new .env inside the project folder ai-scraper and add the following API keys:

REACT_APP_TOOLHOUSE_API_KEY=your_toolhouse_api_key
REACT_APP_OPENAI_API_KEY=your_openai_api_key

You can find your OpenAI API key at platform.openai.com/api-keys. In the .env file replace "your_openai_api_key" with the actual OpenAI key.

Let's now see how we can set-up our Toolhouse account for our AI web scraping app. In order to get your Toolhouse API key you'll first need to create an account at Toolhouse.ai.

Once you've signed up, go to API Keys page. This page should look something like the following:

Clicking on the eye icon should reveal your API key. Copy this and paste it in your .env file in the place of "your_toolhouse_api_key".

Step 4: Creating and setting up your Bundle in Toolhouse

This is how your Dashboard looks like:

On the left menu click on "Bundles". This will take us to a new page where we can create a new Bundle. The purpose of Bundles is to organize our AI Tools into groups or packs.

Once created, you will then be taken to this page, where you can find different pre-made tools and add them to your Bundle:

If you scroll further down, you'll find a Tool named Tavily web search. Enable this Tool and it will be added to your Bundle:

Step 5: Building the App.js Component

Coming back to our app, we’ll now create a simple React component to showcase how tools are managed and executed. Go to your App.js file (or App.ts if you're using TypeScript) inside the src folder and replace the entire code inside with the following code:

import React, { useState } from "react";
import { Toolhouse } from "@toolhouseai/sdk";
import OpenAI from "openai";
import "./App.css";

const MODEL = "gpt-4o-mini";

function App() {
  const [url, setUrl] = useState("");
  const [prompt, setPrompt] = useState("");
  const [result, setResult] = useState("");
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState("");

  const handleSubmit = async (e) => {
    e.preventDefault();
    setIsLoading(true);
    setError("");
    setResult("");

    try {
      const toolhouse = new Toolhouse({
        apiKey: process.env.REACT_APP_TOOLHOUSE_API_KEY,
        metadata: {
          id: "user_id",
          timezone: "0",
        },
      });

      const client = new OpenAI({
        apiKey: process.env.REACT_APP_OPENAI_API_KEY,
        dangerouslyAllowBrowser: true,
      });

      const messages = [
        {
          role: "user",
          content: `Get the contents of ${url} and ${prompt}`,
        },
      ];

      const tools = await toolhouse.getTools();

      const chatCompletion = await client.chat.completions.create({
        messages,
        model: MODEL,
        tools,
      });

      const openAiMessage = await toolhouse.runTools(chatCompletion);

      const newMessages = [...messages, ...openAiMessage];

      const chatCompleted = await client.chat.completions.create({
        messages: newMessages,
        model: MODEL,
        tools,
      });

      setResult(chatCompleted?.choices[0]?.message?.content);
    } catch (err) {
      console.error("Error occurred:", err);
      setError(
        `An error occurred while processing your request. ${
          err.message || JSON.stringify(err)
        }`
      );
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <div className="container">
      <h1>AI Scraper with Toolhouse</h1>
      <form onSubmit={handleSubmit}>
        <div className="form-group">
          <label htmlFor="url">URL</label>
          <input
            type="url"
            id="url"
            value={url}
            onChange={(e) => setUrl(e.target.value)}
            placeholder="https://example.com"
            required
          />
        </div>
        <div className="form-group">
          <label htmlFor="prompt">Prompt</label>
          <textarea
            id="prompt"
            value={prompt}
            onChange={(e) => setPrompt(e.target.value)}
            placeholder="Summarize the content..."
            required
          />
        </div>
        <button type="submit" disabled={isLoading}>
          {isLoading ? "Processing..." : "Scrape and Process"}
        </button>
      </form>

      {error && <div className="error">{error}</div>}

      {result && (
        <div className="result">
          <h2>Result:</h2>
          <pre>{result}</pre>
        </div>
      )}
    </div>
  );
}

export default App;

Step 6: Style our App

To make the app look better, add some styles in App.css file inside the src folder:

.container {
  max-width: 600px;
  margin: 0 auto;
  padding: 20px;
  font-family: Arial, sans-serif;
}

h1 {
  text-align: center;
}

.form-group {
  margin-bottom: 15px;
}

label {
  display: block;
  margin-bottom: 5px;
  font-weight: bold;
}

input, textarea {
  width: 100%;
  padding: 8px;
  margin-bottom: 10px;
  border: 1px solid #ccc;
  border-radius: 4px;
}

button {
  padding: 10px 15px;
  background-color: #007bff;
  color: white;
  border: none;
  border-radius: 4px;
  cursor: pointer;
}

button:disabled {
  background-color: #ccc;
  cursor: not-allowed;
}

.error {
  color: red;
  margin-top: 10px;
}

.result {
  margin-top: 20px;
  padding: 10px;
  background-color: #f9f9f9;
  border: 1px solid #ddd;
  border-radius: 4px;
}

Step 7: Re-start the App

Stop the React server if it's already running by typing Ctrl+C inside the terminal. Run the following command to start the server again in order to load up the environment variables:

npm start

Final App

This is how your app should look like:

You can enter any URL and then a prompt, then our AI agent will scrape the URL and summarize the webpage. Note that some websites like microsoft.com don't allow scraping and hence our scraper will fail in those cases, so make sure the URLs you use allow scraping.

Here's me playing around with the scraper:

Monitoring Tool calls using the Execution Logger in Toolhouse

You can also monitor every single Tool call made to the Tools hosted on Toolhouse. This can help you estimate the number of Tool calls and optimize your Tool calls to save time and money.
Here's how the Execution Logs look like:

As you can see you'll find the exact time of each Tool call as well as the ouput of each Tool call in the Execution Logs.

That's about it for this tutorial. If you want to learn more about building AI agents, feel free to follow me here or on LinkedIn.

Building AI-Powered Applications: Getting Started with Botpress Framework in React

Farhan Ahmad — Sat, 11 Jan 2025 21:16:39 +0000

Writing code for implementing LLM logic can be really tiring and this is what got me experimenting with different frameworks and tools to make my life easier. One such framework that I came across is Botpress. It's an easy to use framework for building AI based apps quickly and it comes with a lot of flexibility as well. More importantly, it offers React components out of the box for you. And you can still customize the look and feel of the components as you wish.

Why Botpress though?

Botpress is not just a framework, it's an all-in-one platform for building AI-powered apps. It greatly simplifies creating, deploying, and managing AI agents. So if you're looking for a complete package of scalable AI solutions, this is it.

Some of the advanced things that you can do with Botpress:

Visual Workflow Design: Drag-and-drop interface for no-code bot creation.
Code Flexibility: Customizable tools and integrations for advanced logic.
Multi-Channel Support: Deploy bots on websites, WhatsApp, Slack, and more.
AI Capabilities: Includes NLU, knowledge integration, and personality customization.

Getting started with Botpress framework - The Webchat SDK

To show you how you can use Botpress framework in your app, we're going to build a very basic AI chat application using Botpress' Webchat SDK which provides ready to use components. It offers a variety of different components that we'll need:

The WebchatProvider - Needs to wrap all chat components and has to be initialized with a Botpress client. We can also customize the theme of the chat interfaces using this provider.
FAB (Floating Action Button) - A fancy name for the small circle that upon clicking opens up the chat.
WebChat - The main chat interface where you can see all messages.

Enough theory, let's get our hands dirty.

We'll use create-react-app to initialize a new React project. Open a folder in VS Code and in the integrated terminal write the following:

npx create-react-app botpress-chat or yarn create react-app botpress-chat

This will create a new project for you automatically. This is how your project structure will look like:

Cool! Let's now run the server to make sure our project was initialized perfectly. Go inside the project directory:

cd botpress-chat

Now run the server:

npm start

Voila, here's your app running on the default port, 3000:

Note: Sometimes, you might get the following error upon running npm start:

In that case, you can overcome this by installing the latest ajv package:
npm install ajv@latest

Setting up Botpress framework in React app

We're now going to install the Botpress libraries, namely the Webchat SDK. These libraries will help us build the chat interface and interact with our chat bot.

Type the following command in your terminal:
npm install @botpress/webchat @botpress/webchat-generator

Note: At the time of this writing, Botpress' Webchat libraries are using react version 18, however latest version of create-react-app is using react version 19. Which means that you downgrade to version 18 for react and react-dom libraries. You can do this by running:
npm install react@18.2.0 react-dom@18.2.0

Once these libraries have been installed, we can start integrating Botpress into our app.
Go to your App.js file (App.ts if you're using Typescript) inside the src folder and replace the code inside the file with the following code:

import { Webchat, WebchatProvider, Fab, getClient } from "@botpress/webchat";
import { buildTheme } from "@botpress/webchat-generator";
import { useState } from "react";
const { theme, style } = buildTheme({
  themeName: "prism",
  themeColor: "#634433",
});
//Add your Client ID here ⬇️
const clientId = "YOUR_CLIENT_ID";
export default function App() {
  const client = getClient({ clientId });
  const [isWebchatOpen, setIsWebchatOpen] = useState(false);
  const toggleWebchat = () => {
    setIsWebchatOpen((prevState) => !prevState);
  };
  return (
    <div style={{ width: "100vw", height: "100vh" }}>
      <style>{style}</style>
      <WebchatProvider
        theme={theme}
        client={client}
      >
        <Fab onClick={toggleWebchat} />
        <div
          style={{
            display: isWebchatOpen ? "block" : "none",
          }}
        >
          <Webchat />
        </div>
      </WebchatProvider>
    </div>
  );
}

Notice the line no. 9 in this file with the following code:

const clientId = "YOUR_CLIENT_ID";

The constant "clientId" has been assigned a dummy placeholder value. This is because we'll need to get the client id of the bot that we want to use in our app from the Botpress website.
First, head over to Botpress.com and sign-up for free. Once you've created an account and filled some basic info you should land on the Dashboard (app.botpress.cloud) which should look like this:

Cool, let's go ahead and create a new bot! Click on the button "Create Bot":

You should see this next:

And once you're done you'll see this:

However, our bot hasn't been published yet. We will need to publish it from the Botpress studio. Go ahead and click the "Open in Studio" button.

This will take you to a different page called the Botpress studio (notice that the URL has changed from app.botpress.cloud to studio.botpress.cloud). Over here we'll need to set-up one more thing before our bot is ready to use.
We have to add the OpenAI integration so that we can use OpenAI's LLM through our bot. To do this follow the below steps.
First, click on "installed integrations" on the left panel.

This will open the Botpress hub explorer. Inside the search box type "openai" and click on the first result that appears.

After clicking you'll be taken to the OpenAI integration page. Click on the button "install integration".

Great, the integration will get installed. You can close the popups. Now on the top right, click on the "Publish" button as the final step.

That's it! Our new chatbot has been created and published.
You can close the Botpress studio window and go back to your workspace (app.botpress.cloud). Now in the workspace page on the left hand side, you will notice there is an option that says "Webchat" as shown below. Click on it.

This will open a bunch of customization options for our bot in the "general" tab. You can give it a name, description, picture, color, etc. Once you're done customizing make sure to scroll down and click on "save" button to save your changes.

Now let's get the client id of the bot we've just created. We will need to go to the "advanced settings" tab and copy the client id of the bot from there:

Great! Now go back to your codebase and find the below line (should be on line number 9) and replace the placeholder value "YOUR_CLIENT_ID" with your client id that you've just copied:

const clientId = "YOUR_CLIENT_ID";

Cool, now in your terminal type the following command to start the server:
npm start
Go to localhost:3000 and you should see something like this:

Click on the FAB (Floating Action Button) component on top left of the page and it will open up a chat interface. You can then try talking to the bot.

Pretty cool! You can explore the official documentation on Botpress' website to further customize these chat UI components over here.

That's it for this tutorial, let me know if you'd like to learn more about Botpress!

How to Set Up CopilotKit in Your React App: A Step-by-Step Guide

Farhan Ahmad — Tue, 31 Dec 2024 13:06:24 +0000

If you've ever built an AI based application, you may have noticed that developing an AI heavy feature for your app that uses the power of LLMs through APIs (like that of OpenAI, Anthropic etc) can be super time consuming and frustrating. It's not just about coding the API integration, it's the overall time spent writing the business logic thats on top of it that takes so much effort.

This is where CopilotKit comes in. It allows you to build scalable LLM-powered applications with little to no time. It also offers endless ways of building cool, unique features for your app.

In this article we'll cover how to get started the easy way with an example repo provided by CopilotKit and then later we'll also cover how to integrate CopilotKit into a new/existing React app from scratch.

What's CopilotKit and why use it?

CopilotKit is a framework that helps you integrate AI assistants and agents in your apps easily. If you're using React it especially helps to have hooks and functions for building complex components that interact with LLM APIs.
By using CopilotKit you no longer have to write LLM API integration logic and it simplifies implementing business logic on top of it.

Here are some features you can build using CopilotKit:

AI chatbots
AI Agents
AI Assistants
Dynamically generated Components

...and much more!

If you're not familiar with AI agents, think of them as autonomous digital workers that can perform tasks proactively. They don't require user inputs to execute a task as they make decisions independently based on their own understanding. AI assistants on the other hand are comparatively less powerful as they only perform specific actions whenever a user instructs them, like setting reminders, booking meetings, etc.

Getting Started with an Example Repo

Lets go ahead and set up a simple React application with CopilotKit. The fastest way to see CopilotKit in action is by using one of the official example repositories.
Head over to their GitHub homepage by clicking here. You'll notice they have multiple example repos for you to explore their powerful framework.

At the time of writing, they have the following example repos:

Each repo demonstrates the unique abilities of CopilotKit framework.

Let's see how we can make these repos work locally with an example. We'll be cloning the repo named example-todos-app.

In your VS Code, open the terminal and type the following command to clone the example-todos-app repository:

git clone https://github.com/CopilotKit/example-todos-app.git

Great, once you have cloned the repo, go to the project directory by doing:

cd example-todos-app

Now that we have the codebase locally we'll need to add an important environment variable to get the app fully set-up. That environment variable is NEXT_PUBLIC_COPILOT_CLOUD_PUBLIC_API_KEY which requires us to create a new account at cloud.copilotkit.ai and then get the API key.

Copilotkit offers a free trial which is more than enough to play around with it. Once you have signed up you should see this screen:

After filling the form you'll be taken to the project dashboard page:

Here you can find important stuff like your Copilot Cloud API key, number of copilot cloud requests made, etc. Note that it says you can make up to 50 requests without providing an OpenAI API key, which is perfect for those who don't have an active OpenAI subscription!

Ok moving on. Copy your Copilot Cloud API key and then go back to your VS Code and create a new .env file in your project folder (inside example-todos-app) and add the following line inside the .env file:
NEXT_PUBLIC_COPILOT_CLOUD_PUBLIC_API_KEY="paste_your_copilot_cloud_api_key"

Perfect, let's install all the required dependencies now:

npm install
or
yarn install

Cool! Once the dependencies are done installing, lets go ahead and start the server to get our app running.

npm run dev
or
yarn dev

That's about it! You can see the app working in your browser at the URL localhost:3000.
It'll look something like the following:

You'll notice on the bottom right there's a chat icon. If you click on that it'll open the chat interface. Here, you can command the AI to modify your To Do list as you want.

Try sending these messages one by one:

Delete all to do list items.
Come up with 3 random business ideas and add them to the list.
Mark all ideas as completed.

Cool stuff huh? You can find more example repos on their GitHub homepage here.

Integrating CopilotKit into your React app

For those who want to integrate CopilotKit into their existing or new React app, here’s how to set it up from scratch. Let's build a simple chat app that lets you talk to AI like chat gpt. Nothing big, but enough to show you the power of CopilotKit.

I'll be using create-react-app for initializing a new project.
So let's go ahead and make sure we actually have create-react-app installed globally:

npm install -g create-react-app

Great, now go ahead and open a new window in VS Code, open a folder and go to the VS Code terminal and type this:

npx create-react-app chat-app
or
yarn create react-app chat-app

This should automatically create a new react project along with all the started files.
Before we move forward make sure you are in the project directory.

cd chat-app

Voila, we can now run the server to ensure our project is set up.

npm start

Note that in some rare cases, you may get the following error when running npm start:

To fix this, go ahead and install the missing dependency called "ajv" as follows:

npm install ajv@latest

Now try running npm start again and it should work normally. It will start the server at the default port 3000. Go to localhost:3000 and it should look like this:

Cool, so let's go ahead and start modifying our react app. We will try implementing a "talk to AI" feature. CopilotKit provides a component to do this out of the box, so we just have to import and use it. Nothing too fancy, but it goes a long way in showing the power of CopilotKit.
But first we need the CopilotKit framework installed and set up in order to use it.

Adding CopilotKit to your React app

The first step is to install the CopilotKit libraries in our app. There are essentially 2 libraries that we're going to need. The first is copilotkit/react-ui which provides ready to use AI powered components right out of the box.
The second library is copilotkit/react-core that contains the main logic for interacting with LLM APIs.

Go to the terminal and type:

npm install @copilotkit/react-ui @copilotkit/react-core

[To view CopilotKit's official Quickstart guide, go here]

Setting up CopilotKit

Great, let's go ahead and modify the code so that we can use the capabilities of CopilotKit in our app.
In order to use all features of the CopilotKit framework we need to add the CopilotKit Provider to the app. This provider should wrap around all the components of your app, which is why we'll need to add it to the Parent component or at the top of the component tree. This parent component may differ depending on your app structure. For instance, if your app was initialised using create-react-app (like in our case) then the parent component is the index.js component. On the other hand if you're using _Next.js _then you need to add the CopilotKit provider inside the layout.tsx component.

As we're using create-react-app for this example, we'll add the CopilotKit provider inside the src/index.js file.
This is how your index.js file should look like:

import React from "react";
import ReactDOM from "react-dom/client";
import "./index.css";
import App from "./App";
import reportWebVitals from "./reportWebVitals";
import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/styles.css";

const root = ReactDOM.createRoot(document.getElementById("root"));
root.render(
  <React.StrictMode>
    <CopilotKit publicApiKey={process.env.REACT_APP_COPILOTKIT_API_KEY}>
      <App />
    </CopilotKit>
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

We're done setting up the Copilotkit provider. All we need now is to host the CopilotKit runtime, and we can start using CopilotKit.
There are 2 ways to use CopilotKit: Either by self-hosting the runtime or by using CopilotKit cloud.
To keep this guide simple and straight-forward we'll use CopilotKit cloud to build the "talk to AI" page.

We've already covered above how you can register for CopilotKit cloud and get the API key for your app. Its fairly easy and you can do it at cloud.copilotkit.ai.
Once you're done signing up, you'll find your API keys on the Dashboard which should look something like this:

Copy the Copilot Cloud Public API Key as we'll need to add it to our environment variable. Create a new file inside the chat-app folder called .env and add the following value:
REACT_APP_COPILOTKIT_API_KEY="paste_your_copilot_cloud_api_key"
And.. we're done setting up CopilotKit! All we need to do now is to import the CopilotKit functions/hooks inside any component and we can start building out LLM powered features supported by CopilotKit. Let's do just that.

Since we're building a talk to AI feature we're going to use a React component called . This is provided by CopilotKit and we just have to import it to start using it.

Go to your src/App.js file inside the src folder and make sure it looks like this:

import "./App.css";
import { CopilotChat } from "@copilotkit/react-ui";
function App() {
  return (
    <div className="App">
      <CopilotChat />
    </div>
  );
}

export default App;

That's it, we're done! Now let's run the command to start our server like this:

npm start

Your server should start automatically at localhost:3000. Here is how it should look:

You can now start talking to AI like you normally talk to chat GPT!

As you saw, it was fairly simple to set everything up, because of the ready-made UI components provided by CopilotKit library and because the API calls to the OpenAI are being taken care of by CopilotKit as well.
You can build LLM based applications quickly without re-writing the same API call logic and creating the UI components that update dynamically based on LLM responses.