Forem: Tereza Tizkova

Llama 3 with Function Calling and Code Interpreter

Tereza Tizkova — Fri, 26 Apr 2024 02:55:58 +0000

At this moment, Llama 3 is one of the most capable open-source models. In this guide, we give Llama 3 code interpreter capabilities and test it on data analysis and data visualization task.

Get full code

We have a full code on GitHub.

Code Interpreter SDK

We will show how to build a code interpreter with Llama 3 on Groq, and powered by open-source Code Interpreter SDK by E2B. The E2B Code Interpreter SDK quickly creates a secure cloud sandbox powered by Firecracker. Inside this sandbox is a running Jupyter server that the LLM can use.

Key links

Overview

Setup
Configuration and API keys
Creating code interpreter
Calling Llama 3
Connecting Llama 3 and code interpreter

1. Setup

We will be working in Jupyter notebook. First, we install the E2B code interpreter SDK and Groq's Python SDK.



%pip install groq e2b_code_interpreter



Collecting groq
  Downloading groq-0.5.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.0/75.0 kB 937.3 kB/s eta 0:00:00
Collecting e2b_code_interpreter
  Downloading e2b_code_interpreter-0.0.3-py3-none-any.whl (10.0 kB)
Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from groq) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from groq) (1.7.0)
Collecting httpx<1,>=0.23.0 (from groq)
  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 4.2 MB/s eta 0:00:00
Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from groq) (2.7.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from groq) (1.3.1)
Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from groq) (4.11.0)
Collecting e2b>=0.14.11 (from e2b_code_interpreter)
  Downloading e2b-0.14.14-py3-none-any.whl (100 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.3/100.3 kB 4.8 MB/s eta 0:00:00
Requirement already satisfied: websocket-client<2.0.0,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from e2b_code_interpreter) (1.7.0)
Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->groq) (3.7)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->groq) (1.2.1)
Collecting aenum>=3.1.11 (from e2b>=0.14.11->e2b_code_interpreter)
  Downloading aenum-3.1.15-py3-none-any.whl (137 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.6/137.6 kB 2.5 MB/s eta 0:00:00
Requirement already satisfied: aiohttp>=3.8.4 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (3.9.5)
Collecting jsonrpcclient>=4.0.3 (from e2b>=0.14.11->e2b_code_interpreter)
  Downloading jsonrpcclient-4.0.3-py3-none-any.whl (7.0 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.8.2)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.31.0)
Requirement already satisfied: urllib3>=1.25.3 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.0.7)
Collecting websockets>=11.0.3 (from e2b>=0.14.11->e2b_code_interpreter)
  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 4.2 MB/s eta 0:00:00
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->groq) (2024.2.2)
Collecting httpcore==1.* (from httpx<1,>=0.23.0->groq)
  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 4.7 MB/s eta 0:00:00
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->groq)
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 3.9 MB/s eta 0:00:00
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->groq) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->groq) (2.18.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (4.0.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->e2b>=0.14.11->e2b_code_interpreter) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->e2b>=0.14.11->e2b_code_interpreter) (3.3.2)
Installing collected packages: aenum, websockets, jsonrpcclient, h11, httpcore, httpx, e2b, groq, e2b_code_interpreter
Successfully installed aenum-3.1.15 e2b-0.14.14 e2b_code_interpreter-0.0.3 groq-0.5.0 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonrpcclient-4.0.3 websockets-12.0

2. Configuration and API keys

Then we store the Groq and E2B API keys and set the model name for the Llama 3 instance you will use. In the system prompt we define sets the rules for the interaction with Llama.



# TODO: Get your Groq AI API key from https://console.groq.com/
GROQ_API_KEY = ""

# TODO: Get your E2B API key from https://e2b.dev/docs
E2B_API_KEY = ""

# Or use 8b version
# MODEL_NAME = "llama3-8b-8192"
MODEL_NAME = "llama3-70b-8192"

SYSTEM_PROMPT = """you are a python data scientist. you are given tasks to complete and you run python code to solve them.
- the python code runs in jupyter notebook.
- every time you call `execute_python` tool, the python code is executed in a separate cell. it's okay to multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to but the usual packages for data analysis are already preinstalled.
- you can run any python code you want, everything is running in a secure sandbox environment"""


tools = [
  {
    "type": "function",
      "function": {
        "name": "execute_python",
        "description": "Execute python code in a Jupyter notebook cell and returns any result, stdout, stderr, display_data, and error.",
        "parameters": {
          "type": "object",
          "properties": {
            "code": {
              "type": "string",
              "description": "The python code to execute in a single cell.",
            }
          },
          "required": ["code"],
        },
      },
  }
]

3. Creating code interpreter

We define the main function that uses the E2B code interpreter to execute code in a Jupyter Notebook cell. We'll be calling this function a little bit further when we're parsing the Llama's response with tool calls.



def code_interpret(e2b_code_interpreter, code):
  print("Running code interpreter...")
  exec = e2b_code_interpreter.notebook.exec_cell(
    code,
    on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
    on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
    # You can also stream code execution results
    # on_result=...
  )

  if exec.error:
    print("[Code Interpreter ERROR]", exec.error)
  else:
    return exec.results

4. Calling Llama 3

Now we're going to define and implement chat_with_llama method. In this method, we'll call the LLM with our tools dictionary, parse the output, and call our code_interpret method we defined above.

See the Groq documentation to get started.



import os
import json
import re
from groq import Groq

client = Groq(api_key=GROQ_API_KEY)

def chat_with_llama(e2b_code_interpreter, user_message):
  print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")

  messages = [
      {"role": "system", "content": SYSTEM_PROMPT},
      {"role": "user", "content": user_message}
  ]

  response = client.chat.completions.create(
      model=MODEL_NAME,
      messages=messages,
      tools=tools,
      tool_choice="auto",
      max_tokens=4096,
  )

  response_message = response.choices[0].message
  tool_calls = response_message.tool_calls

  if tool_calls:
    for tool_call in tool_calls:
      function_name = tool_call.function.name
      function_args = json.loads(tool_call.function.arguments)
      if function_name == "execute_python":
        code = function_args["code"]
        code_interpreter_results = code_interpret(e2b_code_interpreter, code)
        return code_interpreter_results
      else:
        raise Exception(f"Unknown tool {function_name}")
  else:
    print(f"(No tool call in model's response) {response_message}")
    return []

5. Connecting Llama 3 and code interpreter

Finally, we can instantiate the code interpreter and pass the E2B API key. Then we call the chat_with_llama method with our user message and the code_interpreter instance.



from e2b_code_interpreter import CodeInterpreter

with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
  code_results = chat_with_llama(
    code_interpreter,
    "Visualize a distribution of height of men based on the latest data you know"
  )
  if code_results:
    first_result = code_results[0]
  else:
    print("No code results")
    exit(0)


# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result



==================================================
User message: Visualize a distribution of height of men based on the latest data you know
==================================================
Running code interpreter...

Get full code

We have a full code on GitHub.

Key links

Limitations of Running AI Agents Locally

Tereza Tizkova — Fri, 09 Feb 2024 15:47:23 +0000

Like many developers, I have been recently building my own coding AI agent.
I was inspired by a lot of popular code interpreters and AI coding frameworks I have seen lately, like Open Interpreter, Autogen, or ChatGPT Code Interpreter plugin.

The coding agents' value for daily usage has increased after they have evolved from just chatting, to doing the actual work. Developers are equipping their agents with the ability to execute LLM-generated code output.

Imagine you are using an AI agent that can analyze and visualize given data, but instead of just providing you a code that you can copy-paste to do the task, the agent conducts the next step and also runs the code to produce a chart and saves it to your local filesystem.

For example, when I used AutoGen to analyze stock data. The AI agents generated the required code, executed it, and saved the resulting data analysis chart in a PDF format on my computer:

Image source

The code execution is often done locally via containers, e.g., Docker, like in the case of Autogen. Many developers that I talked to started building their own in-house solutions for running AI code.

Running the AI output on the user's computer is an achievable, but risky way. Here are a few obvious and less obvious issues that I had, and noticed other AI developers mentioning:

1. 🔒 Security + isolation

Allowing AI tools to autonomously run untrusted LLM-generated operations on users' computers can be problematic.

Take for example Docker containers, that (according to their docs) possess a risk of providing incomplete isolation, either independently, or when used in combination with kernel vulnerabilities.

When using containers, ensuring isolation is very difficult even when adding barriers.

2. ⭐ User-centric approach

As is typical for any emerging technology, the approach to building AI agents has begun as developer-centric. Many AI products are being built for running in a terminal, using Docker or another type of local solution.

Non-technical people most often ask me how to find AI tools with nice no-code UI. Apps don't necessarily have to be used solely on desktop and some provide better functionalities on a mobile version, but the AI assistants built as browser apps often (not always) offer a nicer user experience.

Installing and running an app locally, let alone controlling it via a terminal is a struggle for non-technical users.
In such a setting, it's also challenging to allow users to collaborate, e.g., share their files with their app in a simple way or share outputs within a team.
Within a browser UI, users can for example share resources and templates so they don't have to build the same thing from scratch.

3. 📈 Scalability

Another challenge that is particularly important for agent-building frameworks is deploying AI agent instances so they can scale. An AI platform or service that accommodates thousands of users, each developing its own AI applications, will require thousands of containers, as each agent instance needs to run in its own isolated environment.

4. ⌛ Session length

A lot of developers want their end users to be able to return to their work after some time or even after closing and re-opening the app.
Developers need to ensure long-running sessions for their AI products.

In conclusion, end users apply their UX and security standards from using "traditional" software to AI apps.
Issues like security and data privacy have been discussed even more in relation to AI.
Within the new AI tech stack that is being formed, developers need to think of a solution that would be tailored for building AI products.

E2B - Remote execution of AI-generated output

E2B (5.7K+ ⭐️) provides the cloud runtime for AI agents and apps.

The E2B Sandbox is a secure way to run your AI app. It is a long-running cloud environment where you can let any LLM (GPTs, Claude, local LLMs, etc) use tools as you would do locally.

Check out the quick start in the docs to start with E2B for free: https://e2b.dev/docs.

150+ AI agents sorted by use cases

Tereza Tizkova — Tue, 23 Jan 2024 16:28:11 +0000

Few months ago I started building a GitHub repo called the Awesome List of AI Agents. It got very popular, reached 3.6k stars and I got people making pull requests and submissions with new product weekly.

Since then, the hype around AI agents hasn't ended, but the opposite. It was time to step up the game and give the list of agents a nice UI too.

This website with AI agents and agent frameworks is my side project and I will be very happy for feedback.

The first version has these features:

🟠 150+ AI agents and frameworks
🟠 Filter by use cases
🟠 Filter by open/closed source
🟠 Filter new AI products
🟠 Request update on particular AI agent

People have been asking how to tell which agents actually work. I haven't tested every single one of the products, but here are some quick examples of what is worth trying.

Coding

My top pick is Open Interpreter which is controllable via terminal and recently added a web UI.

Data analysis

I think not many people know Dot, but I tried it and it worked surprisingly well.

My long-time favorite tool is Superagent, which allows you to simply create custom agent instances for different purposes.

Productivity

I noticed that Taskade has been very popular recently, and its focus is different from other productivity tools. It generates flowcharts, mind mapping, task management

Saga AI is my go-to tool for writing my thoughts and drafts of articles or posts. It offers Grammar correction or paraphrasing parts of the text.

Research

Cognosys AI offers interesting features, e.g. sending you a daily news digest.

I and other team members from E2B use Phind daily. It is able to even generate a functioning code, plus it always provides sources.

Build-your-own

Fine provides an app where you can build agents that work on coding tasks with you.

Crew AI is getting big popularity recently. It's a framework for orchestrating role-playing agents, similar to AutoGen or ChatDev.

AI Agents in 2024

Tereza Tizkova — Wed, 03 Jan 2024 13:31:24 +0000

We are E2B. Try our cloud runtime for AI agents. Check out our GitHub, and support us with a star if you like what we’re building. ✴️

What will the new year bring to the AI field? We asked developers and AI startup founders what they think will happen with AI products in 2024.

Big thanks for the insights go to:

Ismail Pelaseyed, founder of Superagent
Ted Spare, co-founder of Rubric Labs
Jonathan Harel, co-founder of Fine
Kaveen Kumarasinghe, student at University of Waterloo, contractor at OpenAI
John Xie, co-founder of Taskade
Rahul Behal, co-founder of AgentHub
Max Musing, founder of Basedash
Brendon Geils, founder of Athena Intelligence.

1. What do you predict for AI agents/AI products in 2024?

_It's positive to see that developers perceive 2024 as the year of AI agents. See what are their predictions for the new year.
_

💬 Ismail, Superagent:

We will see a lot of LLM extensions/adapters. A concept similar to database extensions. These extensions would augment the abilities of the LLM without using prompt techniques. Examples of this would be extensions for structured outputs (function calling) similar to outlines-dev or JSONformer packages on GitHub. Future examples would be adapters for memorization of new content (an alternative to RAG).

💬 Ted, Rubric Labs:

AI agents will show results in 2024. Barriers to entry will continue to drop through low-code tools and education.

💬 Jonathan, Fine:

Product will become the main moat for AI startups.

💬 Kaveen, Waterloo & OpenAI:

There will be a significant focus on personalization and context-awareness, allowing AI to provide more tailored and relevant interactions within environments that we use daily. I think we’ll also see more drivable and coherent collaborative AI, where agents work in tandem with humans to accomplish tasks.

💬 John, Taskade:

AI agents will become even more integrated into our daily workflows, especially in project management and collaborative spaces. They’ll not only automate tasks but also provide deeper insights and predictions, helping us make smarter decisions faster.

💬 Rahul, AgentHub:

Browsing agents that interface with web browsers at a high level will be able to robustly and reliably autonomously perform at the same level as humans or better for 90% of the things we spend our time with browsers doing. Tasks like shopping on Amazon, browsing/posting to social media, reading/sending emails, etc.

💬 Max, Basedash:

Agents will be functional with minimal prompt engineering, and remarkable with sophisticated prompt engineering.

💬 Brendon, Athena:

My prediction for AI agents (or AI products in general) is that the way we guide Agents will not come primarily from chat inputs.

2. What was your favorite (or most used) AI product in 2023?

The last year was a boom for AI-powered tools and a lot of new startups building AI agents. See some developers' top picks of 2023.

💬 Ismail, Superagent:

Perplexity AI.

💬 Ted, Rubric Labs:

Perplexity. Real-time search results make LLM chat much more useful since much of what I’m looking for is <2-year-old docs.

💬 Jonathan, Fine:

Ollie.ai was pretty neat, found out a few cool things I can gift friends and myself!

💬 Kaveen, Waterloo & OpenAI:

My most used AI product in 2023 was the OpenAI API, specifically GPT-4 and, more recently, the GPT-4 vision preview!

💬 John, Taskade:

It’s hard to pick just one given 100% biased towards Taskade AI, but I’m particularly impressed with the advancements in AI Workflow Generators and Custom AI Agents by Taskade. These tools have significantly streamlined task management and creative processes for me.

💬 Rahul, AgentHub:

Cursor. It just completely changed and enhanced my development workflow and experience. I had several issues with both Co-Pilot and only using ChatGPT, Cursor solves most of them. It’s not perfect, but the product is getting better every day.

💬 Max, Basedash:

Raycast AI proved to me that speed is often more important than capabilities.

💬 Brendon, Athena:

I’ll give this to our favorite framework, Langchain. Their team’s ability to ship days if not hours after AI news hit made it our first stop for AI integration.

To get inspired, check out the landscape of AI products and agents, sorted by their use cases.

3. What is your AI-related resolution for 2024?

Developers shared a broad spectrum of goals for the upcoming year, from building a product that complements human work to replacing one real job with AI.

💬 Ismail, Superagent:

Keep shipping.

💬 Ted, Rubric Labs:

To spend more time listening to worries around AI from family and friends. Disruption can be scary. If we're about to see AI generate the greatest wealth in history, I think it can only be sustained if there's widespread buy-in from regular people.

💬 Jonathan, Fine:

I hope to reach the end of 2024 with an AI that helps me manage my own life.

💬 Kaveen, Waterloo & OpenAI:

To stay ahead of the latest developments in generative AI. This also entails continuing to build and maintain a strong network of professionals to draw information and inspiration from.

💬 John, Taskade:

We want to ensure that we keep building a tool that enhances human creativity and efficiency, rather than replacing it.

💬 Rahul, AgentHub:

Strive to be more platform-agnostic, experimenting with different types of models and services including OSS. And building in a modular way so models can be swapped in and out easily.

💬 Max, Basedash:

Render one job title obsolete with agents.

💬 Brendon, Athena:

Talk more politely to any AI…. just in case.

What is your view on the future of AI agents? Join the discussion on our Discord server.

Build AI developer that makes pull requests for you

Tereza Tizkova — Thu, 21 Dec 2023 15:23:43 +0000

Python guide with complete code

🏁 Final code: E2B Cookbook - AI GitHub Developer

What we will do

In this guide, we build a custom AI developer that clones a GitHub repository of your choice to its remote cloud environment, works on it, and then makes a pull request with the changes.

We use E2B Sandboxes for the remote execution of AI developer's actions, and the OpenAI's Assistants API for the AI assistant.

Find the complete final code here. 🏁

Here is an example of a Next.js project that the AI developer built.

Prerequisites

We are using two key concepts:

OpenAI API - Find your API key here, read the intro to the Assistants API, and Function Calling.
E2B Sandbox - Find your free API key here, read how E2B sandboxes work here.

1. Create files

Let's start with creating files:

main.py for the main program
assistant.py for defining AI developer's behavior
actions.py for defining actions (Python functions) for the developer.

Prepare also .env file where you store your API keys.

2. Define actions for the assistant

In the actions.py file, we define Python functions as runnable actions for the AI assistant and the LLM.

First, let's import the E2B Sandbox and everything else we need.

Here we are using the "default" E2B sandbox. For different use cases, we could use different custom sandboxes with extra packages. For example, a code interpreter sandbox useful for advanced data analysis, or a cloud browser sandbox.

2.1 Import packages

We use Python Rich library for formatting the terminal output of the program.

import os
import random
import string
from typing import Any, Dict
from e2b import Sandbox
from rich.console import Console
from rich.theme import Theme

2.2 Print sandbox actions

We determine the directory where the AI developer will clone the user's repo in the sandbox. We add a way to print what is going on in the sandbox (and pick a theme for the prints, using Rich).

REPO_DIRECTORY = "/home/user/repo"

custom_theme = Theme(
    {
        "sandbox_action": "bold #E57B00",  # Adjust color as needed
    }
)

console = Console(theme=custom_theme)


def print_sandbox_action(action_type: str, action_message: str):
    console.print(
        f"[sandbox_action] [Sandbox Action][/sandbox_action] {action_type}: {action_message}"
    )

2.3 Specify actions for AI developer

Then we define actions that the AI developer can use. You can later modify the program by adding more actions for your specific use case in the same principle. We are adding actions that allow the AI developer the following:

Create a directory in the remote sandbox
Save content (e.g., code) to a file
List files in a directory
Read content of files
Commit changes
Make a pull request

"Actions" are Python functions automatically called in the program by E2B SDK. Inside the actions
Each action corresponds to exactly one OpenAI Function (see the next steps of the guide).

For each action, we need to specify arguments and add printing of relevant information. For example, for list_files it makes sense to return a list of files within the folder.
Inside actions, various operations are called within the sandbox.

# List of actions for the assistant
def create_directory(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    directory = args["path"]
    print_sandbox_action("Creating directory", directory)

    try:
        sandbox.filesystem.make_dir(directory)
        return "success"
    except Exception as e:
        return f"Error: {e}"


def save_content_to_file(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    path = args["path"]
    content = args["content"]
    print_sandbox_action("Saving content to", path)

    try:
        dir = os.path.dirname(path)
        sandbox.filesystem.make_dir(dir)
        sandbox.filesystem.write(path, content)
        return "success"
    except Exception as e:
        return f"Error: {e}"


def list_files(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    path = args["path"]
    print_sandbox_action("Listing files on path", path)

    try:
        files = sandbox.filesystem.list(path)
        response = "\n".join(
            [f"dir: {file.name}" if file.is_dir else file.name for file in files]
        )
        return response
    except Exception as e:
        return f"Error: {e}"


def read_file(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    path = args["path"]
    print_sandbox_action("Reading file on path", path)

    try:
        return sandbox.filesystem.read(path)
    except Exception as e:
        return f"Error: {e}"


def commit(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    repo_directory = "/home/user/repo"
    commit_message = args["message"]
    print_sandbox_action("Committing with the message", commit_message)

    git_add_proc = sandbox.process.start_and_wait(f"git -C {repo_directory} add .")
    if git_add_proc.exit_code != 0:
        error = f"Error adding files to staging: {git_add_proc.stdout}\n\t{git_add_proc.stderr}"
        console.print("\t[bold red]Error:[/bold red]", error)
        return error

    git_commit_proc = sandbox.process.start_and_wait(
        f"git -C {repo_directory} commit -m '{commit_message}'"
    )
    if git_commit_proc.exit_code != 0:
        error = f"Error committing changes: {git_commit_proc.stdout}\n\t{git_commit_proc.stderr}"
        console.print("\t[bold red]Error:[/bold red]", error)
        return error

    return "success"


def make_pull_request(sandbox: Sandbox, args: Dict[str, Any]) -> str:
    base_branch = "main"
    random_letters = "".join(random.choice(string.ascii_letters) for _ in range(5))
    new_branch_name = f"ai-developer-{random_letters}"

    title = args["title"]
    body = ""

    print_sandbox_action(
        "Making a pull request", f"from '{new_branch_name}' to '{base_branch}'"
    )

    git_checkout_proc = sandbox.process.start_and_wait(
        f"git -C {REPO_DIRECTORY} checkout -b {new_branch_name}"
    )
    if git_checkout_proc.exit_code != 0:
        error = f"Error creating a new git branch {new_branch_name}: {git_checkout_proc.stdout}\n\t{git_checkout_proc.stderr}"
        console.print("\t[bold red]Error:[/bold red]", error)
        return error

    git_push_proc = sandbox.process.start_and_wait(
        f"git -C {REPO_DIRECTORY} push -u origin {new_branch_name}"
    )
    if git_push_proc.exit_code != 0:
        error = (
            f"Error pushing changes: {git_push_proc.stdout}\n\t{git_push_proc.stderr}"
        )
        console.print("\t[bold red]Error:[/bold red]", error)
        return error

    gh_pull_request_proc = sandbox.process.start_and_wait(
        cmd=f'gh pr create --base "{base_branch}" --head "{new_branch_name}" --title "{title}" --body "{body}"'.replace(
            "`", "\\`"
        ),
        cwd=REPO_DIRECTORY,
    )
    if gh_pull_request_proc.exit_code != 0:
        error = f"Error creating pull request: {gh_pull_request_proc.stdout}\n\t{gh_pull_request_proc.stderr}"
        console.print("\t[bold red]Error:[/bold red]", error)
        return error

    return "success"

3. Build the assistant

Now we create the AI developer itself inside the assistant.py. The specific feature of the OpenAI's Assistants API that we'll take advantage of is Function calling.

Function calling feature gives our AI assistant the ability to decide to call the sandbox actions we defined in the actions.py.

3.1 Import packages

from typing import List

from dotenv import load_dotenv
import openai
from openai.types.beta.assistant_create_params import Tool

load_dotenv()

3.2 Define the assistant

Now we create the assistant and equip it with six functions (remember, these are the OpenAI Functions, not Python functions) where each corresponds to one action defined previously in the actions.py.

def create_assistant():
    client = openai.Client()

    functions: List[Tool] = [
        {
            "type": "function",
            "function": {
                "name": "create_directory",
                "description": "Create a directory",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "The path to the directory to be created",
                        },
                    },
                    "required": ["path"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "save_content_to_file",
                "description": "Save content (code or text) to file",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "content": {
                            "type": "string",
                            "description": "The content to save",
                        },
                        "path": {
                            "type": "string",
                            "description": "The path to the file, including extension",
                        },
                    },
                    "required": ["content", "path"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "list_files",
                "description": "List files in a directory",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "The path to the directory",
                        },
                    },
                    "required": ["path"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "read_file",
                "description": "Read a file",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "The path to the file",
                        },
                    },
                    "required": ["path"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "commit",
                "description": "Commit changes to the repo",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "message": {
                            "type": "string",
                            "description": "The commit message",
                        },
                    },
                    "required": ["message"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "make_pull_request",
                "description": "Creates a new branch and makes a pull request",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "title": {
                            "type": "string",
                            "description": "The title of the pull request",
                        }
                    },
                    "required": ["title"],
                },
            },
        },
    ]

3.3 Write system prompt

Still inside the create_assistant() function, we give instructions to the assistant and choose its parameters. Once we run this file, it prints the assistant's ID which we can save as an environment variable.
Don't forget to re-run the file with assistant and create new ID every time you update it.

💡 Tip: Adjust the instructions as needed. For example, you can decide how much the AI developer engages in discussion with user vs limiting itself to performing given task.
The OpenAI's prompt engineering guide may come handy.

    ai_developer = client.beta.assistants.create(
        instructions="""You are an AI developer. You help user work on their tasks related to coding in their codebase. The provided codebase is in the /home/user/repo.
    When given a coding task, work on it until completion, commit it, and make pull request.

    If you encounter a problem, communicate it promptly, please.

    You can create and save content (text or code) to a specified file (or create a new file), list files in a given directory, read files, commit changes, and make pull requests. Always make sure to write the content in the codebase.

    By default, always either commit your changes or make a pull request after performing any action on the repo. This helps in reviewing and merging your changes.
    Name the PR based on the changes you made.

    Be professional, avoid arguments, and focus on completing the task.

    When you finish the task, always provide the link to the pull request you made (if you made one.)
    Additionally, be prepared for discussions; not everything user writes implies changes to the repo. For example, if the user writes "thank you", you can simply answer "you are welcome".
    But by default, if you are assigned a task, you should immediately do it in the provided repo, and not talk only talk about your plan.
    """,
        name="AI Developer",
        tools=functions,
        model="gpt-4-1106-preview",
    )

    print("AI Developer Assistant created, copy its id to .env file:")
    print(ai_developer.id)


if __name__ == "__main__":
    create_assistant()

4. Create the main program

Now we code the core program. The assistant calls OpenAI Functions through JSON, which the E2B SDK parses and automatically invokes defined actions.

4.1 Import packages

First, we import the necessary packages - openai, e2b Sandbox, and the actions we created in the other file.

import os
from dotenv import load_dotenv
from e2b import Sandbox
import openai
import time
from ai_github_developer.actions import (
    create_directory,
    read_file,
    save_content_to_file,
    list_files,
    commit,
    make_pull_request,
    REPO_DIRECTORY,
)

from rich import print
from rich.console import Console
from rich.theme import Theme
from rich.prompt import Prompt


class MyPrompt(Prompt):
    prompt_suffix = ""


custom_theme = Theme(
    {
        "theme": "bold #666666",
    }
)
console = Console(theme=custom_theme)

4.2 Retrieve assistant

We call the assistant using its ID and use the OpenAI API to retrieve the assistant. Don't forget to save assistant's ID, OpenAI API key and E2B API key as environment variables.

load_dotenv()
client = openai.Client()

AI_ASSISTANT_ID = os.getenv("AI_ASSISTANT_ID")
USER_GITHUB_TOKEN = os.getenv("GITHUB_TOKEN")

assistant = client.beta.assistants.retrieve(AI_ASSISTANT_ID)

4.2 Prompt user for GitHub repo, authentication, and task

We define three functions that ask the user for

GitHub repository URL
Specifying the task for the AI agent (e.g., "Please create a calculator in JavaScript and save it to a new file")
GitHub authentication token.

The user is asked for their GitHub personal access token (classic) as a standard way to interact with the GitHub API securely.

💡 Tip: Save your GitHub token as GITHUB_TOKEN in your .env file to avoid having to paste it every time you run the program.

def prompt_user_for_github_repo():
    user_repo = MyPrompt.ask(
        "\nWhat GitHub repo do you want to work in? Specify it like this: [bold #E0E0E0]your_username/your_repo_name[/bold #E0E0E0].\n> "
    )
    print("\n🔄[#666666] Cloning the repo...[/#666666]", end="\n")
    print("", end="\n")

    repo_url = f"https://github.com/{user_repo.strip()}.git"

    return repo_url


def prompt_user_for_task(repo_url):
    user_task_specification = MyPrompt.ask(
        "\n\n🤖[#E57B00][bold] The AI developer is working in the cloned repo[/bold][/#E57B00]\n\nWhat do you want to do?\n> "
    )
    user_task = (
        f"Please work with the codebase repo called {repo_url} "
        f"that is cloned in the /home/user/repo directory. React on the following user's comment: {user_task_specification}"
    )
    print("", end="\n")
    return user_task


def prompt_user_for_auth():
    user_auth = MyPrompt.ask(
        "\nProvide [bold]GitHub token[/bold] with following permissions:\n\n\u2022 read:org\n\u2022 read:project\n\u2022 repo\n\nFind or create your token at [bold #0096FF]https://github.com/settings/tokens[/bold #0096FF]\n\nToken:",
        password=True,
    )
    print("", end="\n")
    return user_auth

4.3 Setup git

We set up the Git configuration and authentication for the user's account within the specified sandbox environment. It involves configuring the user's email and name, logging in with a GitHub personal access token, and setting up Git credentials for GitHub. To monitor the process, we add printing exit codes in each step.

def setup_git(sandbox):
    print("Logging into GitHub...")
    # Identify AI developer in git
    sandbox.process.start_and_wait(
        "git config --global user.email 'ai-developer@email.com'"
    )
    sandbox.process.start_and_wait("git config --global user.name 'AI Developer'")

    # Login user to GitHub
    proc = sandbox.process.start_and_wait(
        f"echo {USER_GITHUB_TOKEN} | gh auth login --with-token"
    )
    if proc.exit_code != 0:
        print("Error: Unable to log into GitHub", end="\n")
        print(proc.stderr)
        print(proc.stdout)
        exit(1)

    # Setup user's Git credentials
    proc = sandbox.process.start_and_wait("gh auth setup-git")
    if proc.exit_code != 0:
        print("Error: Unable to set up Git auth with GitHub")
        print(proc.stderr)
        print(proc.stdout)
        exit(1)
    else:
        print("\n✅ [#666666]Logged in[/#666666]")

4.4 Clone the repo

Use the sandbox environment to execute a Git clone command and check if the process was successful. We define a way to print the standard output or standard error output from the sandbox (with a specific visual theme).

def clone_repo_in_sandbox(sandbox, repo_url):
    # Clone the repo
    git_clone_proc = sandbox.process.start_and_wait(
        f"git clone {repo_url} {REPO_DIRECTORY}"
    )
    if git_clone_proc.exit_code != 0:
        print("Error: Unable to clone the repo")
        exit(1)


def handle_sandbox_stdout(message):
    console.print(f"[theme][Sandbox][/theme] {message.line}")


def handle_sandbox_stderr(message):
    console.print(f"[theme][Sandbox][/theme] {message.line}")

4.5 Spawn the sandbox

Now we can define the main function to spawn the E2B sandbox.

Inside the function, we choose the preferred E2B sandbox, which is called simply "Sandbox", since we chose the default one.

💡 Tip: E2B offers premade sandboxes or an option to build your own custom one with preferred packages. To keep this guide simple, we picked the Default Sandbox and equipped it with just the actions we have defined in the actions.py.

We use a sandbox.add_action() method to register the actions with the sandbox.
We start the sandbox and configure the AI assistant in git. We log the user to GitHub via the authentication token.
We assign the user's task to the assistant. Then we create a thread, send messages to the thread, and finally run the thread.

We register actions with the sandbox using a sandbox.add_action() method. We start the sandbox, configure the AI developer in Git, and ask the user for a GitHub token, if they haven't added it as the environment variable already.

Then, we assign the user's task to the assistant and create and run a thread to send messages to complete the task.

Here, we are using OpenAI's concept of threads, messages, and runs.

def main():
    global USER_GITHUB_TOKEN

    # Create the E2B sandbox
    sandbox = Sandbox(
        on_stderr=handle_sandbox_stderr,
        on_stdout=handle_sandbox_stdout,
    )
    sandbox.add_action(create_directory).add_action(read_file).add_action(
        save_content_to_file
    ).add_action(list_files).add_action(commit).add_action(make_pull_request)

    print("\n🤖[#E57B00][bold] AI developer[/#E57B00][/bold]")
    if USER_GITHUB_TOKEN is None:
        USER_GITHUB_TOKEN = prompt_user_for_auth()
    else:
        print("\n✅ [#666666]GitHub token loaded[/#666666]\n")

    # Setup git right away so user knows immediately if they passed wrong token
    setup_git(sandbox)

    # Clone repo
    repo_url = prompt_user_for_github_repo()
    clone_repo_in_sandbox(sandbox, repo_url)

    while True:
        user_task = prompt_user_for_task(repo_url)

        thread = client.beta.threads.create(
            messages=[
                {
                    "role": "user",
                    "content": f"Carefully plan this task and start working on it: {user_task} in the {repo_url} repo",
                },
            ],
        )

        run = client.beta.threads.runs.create(
            thread_id=thread.id, assistant_id=assistant.id
        )

The threads, messages, and runs are a concept from the OpenAI's Assistants API:

4.6 Print assistant's runs

Still inside the main function, we print the assistant's process to the terminal. Each time the developer chooses to use one of the actions, the user can see logs about the chosen action and its success/failure in the terminal.

Note: The assistant's runs duration is by huge part determined by OpenAI.

At the end, we use time.sleep() to specify how often we want to poll the assistant's status.

 spinner = ""
        with console.status(spinner):
            previous_status = None
            while True:
                if run.status != previous_status:
                    console.print(
                        f"[bold #FF8800]>[/bold #FF8800] Assistant is currently in status: {run.status} [#666666](waiting for OpenAI)[/#666666]"
                    )
                    previous_status = run.status
                if run.status == "requires_action":
                    outputs = sandbox.openai.actions.run(run)
                    if len(outputs) > 0:
                        client.beta.threads.runs.submit_tool_outputs(
                            thread_id=thread.id, run_id=run.id, tool_outputs=outputs
                        )
                elif run.status == "completed":
                    console.print("\n✅[#666666] Run completed[/#666666]")
                    messages = (
                        client.beta.threads.messages.list(thread_id=thread.id)
                        .data[0]
                        .content
                    )
                    text_messages = [
                        message for message in messages if message.type == "text"
                    ]
                    console.print("Thread finished:", text_messages[0].text.value)
                    break

                elif run.status in ["queued", "in_progress"]:
                    pass

                elif run.status in ["cancelled", "cancelling", "expired", "failed"]:
                    break

                else:
                    print(f"Unknown status: {run.status}")
                    break

                run = client.beta.threads.runs.retrieve(
                    thread_id=thread.id, run_id=run.id
                )
                time.sleep(0.5)

Finally, under the main function, we write the code that is run if the script is the main program being executed.

if __name__ == "__main__":
    main()

Output

Find the complete final code here. 🏁

Here is an example of a Next.js project bootstrapped with create-next-app that the AI developer built.

Thank you for reading this guide - and I will appreciate your feedback!

Reach out to me on my X (Twitter).

Where to find E2B:

💻 Website
🎮 Discord server
📝 Docs
🧑🏼‍💻 GitHub
💬 X (Twitter)