Forem: LangDB

Designing Smart Multi-Agent Workflows with Agno & LangDB

Mrunmay — Thu, 24 Jul 2025 14:48:53 +0000

Build a multi-agent financial analysis team with LangDB and Agno that can reason, research, and report on complex financial data.

In the world of finance, staying ahead requires more than just data; it requires deep analysis, contextual awareness, and collaborative reasoning. What if you could build a team of AI agents to do this for you? In this post, we'll show you how to build a sophisticated, multi-agent financial analysis team using LangDB and Agno.

TL;DR:

This guide walks you through building a multi-agent workflow using Agno for orchestration and LangDB as the AI Gateway. We'll use a financial analysis team as a practical example to show how you can build sophisticated agent systems that are easy to manage and debug, thanks to LangDB's end-to-end tracing, dynamic tooling, and access to over 350 LLMs.

This team of agents collaborates to deliver in-depth insights on publicly traded companies by combining web research for market sentiment with hard financial data analysis. You can see a full trace of the final agent's execution.

The Code

You can find the complete source code for this project on GitHub:

LangDB Samples: https://github.com/langdb/langdb-samples/tree/main/examples/agno/reasoning-finance-team

The Architecture: A Trio of Financial Experts

Our system is composed of two specialist agents orchestrated by a coordinating team:

Web Search Agent: Gathers the latest news and market sentiment from the internet.
Finance Agent: Equipped with YFinanceTools to fetch and analyze quantitative stock data, including pricing, fundamentals, and analyst recommendations.
Reasoning Finance Team: A coordinator that directs the two agents, synthesizes their findings, and produces a final, comprehensive report.

LangDB provides the backbone for this system. As an AI Gateway, it enables seamless access to over 350 LLMs, simplifies tool integration, and provides full end-to-end tracing and observability into each agent's actions and the team's collaborative process.

Checkout: https://docs.agno.com/models/langdb and https://docs.agno.com/observability/langdb

Enhanced Tracing with `pylangdb.agno.init()`

While you can use LangDB as a provider in Agno directly, calling pylangdb.agno.init() unlocks deeper, end-to-end tracing. This function provides additional metadata and observability by automatically instrumenting the entire Agno framework, giving you complete visibility into your agent's workflows.

# main.py
from pylangdb.agno import init

# Initialize LangDB for enhanced tracing *before* importing any Agno modules.
init()

from agno.agent import Agent
from agno.team import Team
# ... other imports

As an official provider integrated with Agno, LangDB requires you to set up your credentials. You'll need to export your LangDB API key and Project ID as environment variables. You can find these in your LangDB project settings.

export LANGDB_API_KEY="<your_langdb_api_key>"
export LANGDB_PROJECT_ID="<your_langdb_project_id>"

Code Walkthrough: Building the Team

Let's look at how the agents and the team are defined.

The Web Search Agent: Decoupled and Dynamic

Instead of hard-coding a search tool, we assign the web_agent a LangDB Virtual Model. This decouples the agent's logic from the specific tools it uses.

web_agent = Agent(
    name="Web Search Agent",
    role="Search the web for the information",
    model=LangDB(id="langdb/search_agent_xmf4v5jk"),
    instructions="Always include sources"
)

This virtual model is configured in the LangDB UI to provide search capabilities, which we'll cover in the next section.

The Finance Agent: The Quantitative Analyst

This agent is equipped with YFinanceTools to access a wide range of financial data. It's powered by a powerful model like Grok-4 and has specific instructions to format its output professionally.

finance_agent = Agent(
    name="Finance AI Agent",
    role="Analyse the given stock",
    model=LangDB(id="xai/grok-4"),
    tools=[YFinanceTools(
        stock_price=True,
        stock_fundamentals=True,
        analyst_recommendations=True,
        company_info=True,
        company_news=True
    )],
    instructions=[
        "Use tables to display stock prices, fundamentals (P/E, Market Cap), and recommendations.",
        "Clearly state the company name and ticker symbol.",
        "Focus on delivering actionable financial insights."
    ]
)

The Coordinating Team: The Orchestrator

The ReasoningFinanceTeam orchestrates the two specialist agents. It operates in coordinate mode, allowing it to delegate tasks, synthesize information, and ensure the final output is a comprehensive report.

reasoning_finance_team = Team(
    name="Reasoning Finance Team",
    mode="coordinate",
    model=LangDB(id="xai/grok-4"),
    members=[web_agent, finance_agent],
    tools=[ReasoningTools(add_instructions=True)],
    instructions=[
        "Collaborate to provide comprehensive financial and investment insights",
        "Consider both fundamental analysis and market sentiment",
        "Present findings in a structured, easy-to-follow format",
    ],
    success_criteria="The team has provided a complete financial analysis with data, visualizations, risk assessment, and actionable investment recommendations."
)

Dynamic Tooling with Virtual Models and Virtual MCPs

To empower the web_agent with live web search capabilities without hard-coding tools, we configure a Virtual Model in LangDB. This model is backed by a Virtual MCP Server that provides the actual search functionality.

Create a Virtual MCP Server: In the LangDB UI, create a new Virtual MCP Server named web-search-mcp that uses the Tavily Search MCP.
Create and Configure the Virtual Model: Create a new virtual model (e.g., search-agent) and attach the web-search-mcp to it.
Use the Virtual Model ID: Copy the ID of your new virtual model and use it in the web_agent definition.

This setup allows you to change the tools and models your agents use on the fly from the LangDB UI, without changing a single line of code.

Running the Team and Observing the Results

To run the team, simply call the print_response method with a detailed prompt:

reasoning_finance_team.print_response(
    """Compare the tech sector giants (AAPL, GOOGL, MSFT) performance:\n
    1. Get financial data for all three companies\n
    2. Analyze recent news affecting the tech sector\n
    3. Calculate comparative metrics and correlations\n
    4. Recommend portfolio allocation weights"""
)

Every execution is captured in LangDB, providing a complete trace of the team's operations. This includes the initial prompt, each agent's contributions, the tools they used, and the final synthesized output. You can explore the full, shareable trace.

Here is a snippet of the final report generated by the agent team:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                   Comprehensive Comparative Analysis of Tech Giants: AAPL, GOOGL, and MSFT                    ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

As the Reasoning Finance Team, we've conducted a thorough analysis of Apple Inc. (AAPL), Alphabet Inc. (GOOGL), and Microsoft Corporation (MSFT) based on the user's request. This includes financial data retrieval, recent news analysis, comparative metrics and correlations, and portfolio allocation recommendations. Our evaluation incorporates fundamental analysis (e.g., valuations, growth), market sentiment (e.g., news and analyst views), quantitative metrics (e.g., betas, correlations), and risk assessments. Data is current as of July 2025.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────

                                             1. Financial Data Overview                                             

Key financial data for each company, sourced from reliable APIs.                                                  

                                                   Stock Prices                                                   

   Metric               AAPL      GOOGL     MSFT                                                                   
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━                                                               
   Current Price        $210.16   $182.97   $505.62                                                                
   52-Week High         $260.10   $207.05   $508.30                                                                
   52-Week Low          $169.21   $140.53   $344.79                                                                
   50-Day Moving Avg    $203.87   $170.88   $472.41                                                                
   200-Day Moving Avg   $222.55   $173.43   $427.18

And here's a view of the full trace in the LangDB UI, showing how the agents collaborated to produce the report.

Full Observability with LangDB Tracing

The "full observability" promised in our subtitle is delivered through LangDB's detailed tracing capabilities. When you run your Agno team, every action is captured, providing a transparent, hierarchical view of the entire workflow. Here’s what you can see in the trace:

Hierarchical Span View: The trace isn't a flat list of events but a tree of "spans." The top-level span represents the entire team's execution, with child spans for each agent's turn, tool call, and model invocation. This shows the exact flow of control and delegation.
Input/Output for Each Step: For every span, you can inspect the exact inputs and outputs. This means you can see the precise query sent to the Web Search Agent, the articles it returned, the data requested by the Finance Agent, and the final synthesized response from the team. This level of detail is crucial for debugging.
Latency and Performance: Each span is timestamped and includes latency information, allowing you to instantly identify bottlenecks. You can see exactly how long each tool call, model response, or agent deliberation took.
Cost and Token Usage: For every LLM call, the trace displays the number of input and output tokens and the associated cost, giving you full transparency into your operational expenses.
Automatic Metadata: Thanks to pylangdb.agno.init(), traces are automatically enriched with metadata, including agent names, the team name (Reasoning Finance Team), and the models used, making it easy to filter and search for specific traces in the LangDB UI.

This granular, end-to-end visibility is what makes building, debugging, and managing complex agentic workflows with LangDB and Agno so powerful.

You can check out the full conversation with tracing here: https://app.langdb.ai/sharing/threads/630b2ded-15ae-43d9-8a7a-d6dd9d649655

Conclusion

By combining Agno with the LangDB AI Gateway, we've built a financial analysis team that is:

Modular: Each agent has a specific, firewalled responsibility.
Dynamic: We can change models and grant new tools on the fly from the LangDB UI without redeploying our agent.
Observable: We get detailed traces of every interaction, making debugging and performance analysis straightforward.

This architecture allows for rapid development and iteration, enabling you to build truly powerful and intelligent agentic systems for any domain.

Ready to build your own? Start building for free on LangDB or Explore Agno to orchestrate your agent workflows.

Discover End-to-End Tracing on Google ADK with LangDB

Mrunmay — Mon, 07 Jul 2025 04:25:59 +0000

Before diving into the code, watch this 2-minute video to see a complete demonstration of what we'll be building. You'll learn how to integrate LangDB tracing into the Google ADK Travel Concierge sample with no code chages.

In this quick demo you’ll see:

How to install and initialize the pylangdb[adk] package.
The single line of code that enables full observability for every ADK agent and tool.
Running a sample prompt like “Find me flights from JFK to London”.
Inspecting your workflow in the LangDB AI Gateway dashboard, including:
- Threads view for step-by-step conversation logs.
- Traces view for Gantt charts, cost & token breakdowns, and dependency graphs.
Drilling into any agent or tool (like the planning_agent on Claude 3 Sonnet) for full observability.

In this tutorial, we'll walk through the architecture of a sophisticated Travel Concierge agent built with Google's Agent Development Kit (ADK). We'll explore how to leverage the LangDB AI Gateway to use any LLM—from OpenAI, Google, Anthropic, and more—and harness powerful features like Virtual Models and Virtual MCPs (Model Context Protocol) to create a dynamic, observable, and easily maintainable agent system.

Our travel_concierge agent is not just a single agent; it's a hierarchy of specialized sub-agents that handle everything from vacation inspiration to booking and in-trip assistance. Here's a look at the overall architecture:

This project is based on the official Google ADK Travel Concierge sample and has been modified to showcase the integration with the LangDB AI Gateway.

You can find the complete source code for this agent on GitHub: LangDB Samples

The Magic Behind the Curtain: `pylangdb.adk.init()`

First, let's talk about the most important line of code in this integration:

# travel_concierge/agent.py
from pylangdb.adk import init
# Initialize LangDB *before* importing any ADK modules.
init()

This single function call is the key to unlocking the LangDB AI Gateway's observability features. By placing it at the very top of our script, before any google.adk modules are imported, we enable automatic instrumentation for the entire agent framework.

Here’s what init() does automatically:

Discovers Agents: It recursively finds all agent and sub-agent definitions within your project.
Patches Runtimes: It automatically patches the necessary ADK components to emit traces.
Links Sessions: It intelligently links all the interactions—from the root agent's initial processing to the deepest sub-agent and tool calls—into a single, cohesive trace in the LangDB Tracing.

This "zero-instrumentation" approach means you get complete, end-to-end visibility into your agent's complex workflows just by adding that one line of code.

The Architecture: Root Agent and Sub-Agents

Our travel_concierge is a hierarchical agent. At the top is the root_agent, which acts as a smart router or orchestrator. Its job is not to answer queries directly, but to delegate them to a specialized sub-agent.

Here's its actual definition:

# travel_concierge/agent.py
root_agent = Agent(
    model="openai/gpt-4.1",
    name="root_agent",
    description="A Travel Conceirge using the services of multiple sub-agents",
    instruction=prompt.ROOT_AGENT_INSTR,
    sub_agents=[
        inspiration_agent,
        planning_agent,
        # ... and other sub-agents
    ],
    # ...
)

As you can see, it uses a standard model ("openai/gpt-4.1") and has a list of sub_agents. It doesn't have any tools of its own. The real power comes from the sub-agents.

Dynamic Tooling with Virtual Models and Virtual MCPs

A LangDB Virtual Model is a powerful abstraction that decouples your agent's code from its runtime configuration. It acts as a pointer to a configuration that you can manage entirely from the LangDB UI.

This is where the Model Context Protocol (MCP) comes in. MCP is a standard that allows language models to interact with external tools and services in a uniform way. However, managing connections to multiple MCP-enabled tools can be complex.

The LangDB AI Gateway simplifies this with Virtual MCP Servers. A Virtual MCP is a single, managed endpoint that you configure in the UI. It can bundle multiple tools (like Google Maps, Tavily Search, or your own custom APIs), handle their authentication securely, and lock them to specific versions.

You then connect this Virtual MCP to your agent's Virtual Model. This is how you can dynamically grant new capabilities to your agents without changing a single line of code.

Here are all the virtual models for our project, as seen in the LangDB AI Gateway dashboard. You can see the inspiration_agent, google_search_agent, and planning_agent all configured here, ready to be assigned to our agents.

Example: The `inspiration_agent` and Google Maps

Let's look at our inspiration_agent. It needs access to location data to give travel ideas. Instead of hardcoding a Google Maps MCP, we use a Virtual Model.

Here's the agent's definition:

# travel_concierge/sub_agents/inspiration/agent.py
inspiration_agent = Agent(
    model= "langdb/inspiration_agent_z73m3wmd",
    name="inspiration_agent",
    description="A travel inspiration agent...",
    # ...
)

Notice its model is langdb/inspiration_agent_z73m3wmd. In the LangDB AI Gateway UI, we've configured this virtual model to use a Virtual MCP server that has the Google Maps API attached as a tool. Now, when the inspiration_agent is active, it can seamlessly query Google Maps, even though the tool isn't explicitly listed in its code.

Example: Grounding with Google Search

We also have a specialized agent tool for web searches, google_search_grounding.

# travel_concierge/tools/search.py
_search_agent = Agent(
    model= "langdb/google_search_agent_hsz7lf9q",
    name="google_search_grounding",
    description="An agent providing Google-search grounding capability",
    # ... instruction ...
)

google_search_grounding = AgentTool(agent=_search_agent)

Just like our inspiration_agent, the _search_agent uses a virtual model, langdb/google_search_agent_hsz7lf9q. We've attached a Virtual MCP server that provides the Tavily Search tool to this model in LangDB.

Example: The `planning_agent` for Flights and Hotels

Finally, let's look at the planning_agent, which handles the core booking tasks.

# travel_concierge/sub_agents/planning/agent.py
planning_agent = Agent(
    model="langdb/planning_agent_w1l8sygt",
    name="planning_agent",
    description="Helps users with travel planning...",
    # ...
)

This agent's virtual model, langdb/planning_agent_w1l8sygt, is connected to a Virtual MCP that provides an Airbnb search tool. This allows the agent to handle complex booking-related queries by leveraging this external service, all without having the tool logic hardcoded in the agent's definition.

The Flow: From Query to Answer

A user asks the travel_concierge: "What are some good museums to visit in Paris?"
The root_agent receives the query and, based on its instructions, delegates the task to the inspiration_agent.
The inspiration_agent is activated. Its virtual model configuration is loaded from the LangDB AI Gateway.
The agent now knows it has access to the Google Maps tool (via its Virtual MCP).
It uses the tool to find museums in Paris and provides a list to the user.
All of these steps—the delegation, the model calls, the tool usage—are automatically captured as traces in the LangDB AI Gateway, giving us complete observability into our agent's behavior.

You can explore a complete, shareable trace of a conversation with this agent here: https://app.langdb.ai/sharing/threads/8425e068-77de-4f41-8aa9-d1111fc7d2b7

When you open the trace, you'll see a detailed breakdown of the entire workflow. This includes:

A Gantt chart visualizing the sequence and duration of each agent and tool invocation.
Cost and token counts for every LLM call, helping you monitor usage and optimize performance.
Detailed input/output payloads for each step, allowing you to inspect the exact data being passed between components.
A dependency graph showing how agents and tools are interconnected, making it easy to debug complex interactions.

Conclusion

By combining Google ADK with the LangDB AI Gateway's virtual models and MCPs, we've built a travel_concierge agent that is:

Modular: Each sub-agent has a specific responsibility.
Dynamic: We can change models and grant new tools on the fly from the LangDB UI without redeploying our agent.
Observable: We get detailed traces of every interaction, making debugging and performance analysis easy.

This architecture allows for rapid development and iteration, enabling us to build truly powerful and intelligent agentic systems.

Ready to build your own? Check out the LangDB AI Gateway documentation to get started

Choose the right AI model: Comparision of gpt-4o, claude and gemini using LangDB

Mrunmay — Thu, 27 Feb 2025 11:49:55 +0000

Wondering which model to pick for your AI integration? It often comes down to response quality, time, and cost. LangDB’s Chat lets you compare GPT, Claude, and Gemini and many other models side by side:

Send identical prompts to all three models in one interface.
Analyze response accuracy, coherence, and tone.
Measure execution time, latency, and cost.

Chat Interface in LangDB

Query: "Summarize the potential risks and benefits of AI-driven automated trading in financial markets, focusing on efficiency, transparency, and ethical concerns."

LangDB provides detailed insights into the API calls made to various providers. Here's an example of a traces in LangDB:

Model	Execution Time	Cost
GPT-4o	12.44 sec	$0.637
Claude 3.5 Sonnet	11.40 sec	$0.013
Gemini 1.5 Pro	7.47 sec	$0.384

These metrics allow developers to track performance, latency, and associated costs for each model efficiently.

Supported Models

Why Perform Model Comparisons?

Directly comparing LLMs helps developers:

Select the Best Model: Choose the model that performs best for your specific use case.
Optimize Costs and Performance: Compare API costs, execution times, and token efficiency.

LangDB’s Chats feature eliminates the operational friction of testing models. It provides a clean, user-friendly platform for experimenting with the latest AI models without any extra configuration.

Get Started with LangDB

Stop guessing which model works best—test them side by side with LangDB Chats. Compare the latest and best AI models effortlessly, optimize performance, and unlock new possibilities without writing or managing infrastructure.

Start building smarter AI systems today—and let LangDB handle the heavy lifting.

How to Integrate LangChain with LangDB

Mrunmay — Thu, 27 Feb 2025 11:06:50 +0000

LangDB integrates seamlessly with libraries like LangChain to provide advanced tracing and logging support for workflows, allowing developers to streamline the development process while maintaining detailed logs. If you're familiar with LangChain, adding LangDB to your workflow can offer enhanced functionality without adding complexity.

In this blog post, we'll walk through how to use LangDB with LangChain, including a practical example. By the end, you'll understand how to capture detailed logs and take advantage of LangDB’s features in your own LangChain projects.

Pre-requisites

Tavily API token
OpenAI API token
Python v3.11
Pip packages: langchain (at least v0.1.0), openai, wikipedia, langchain-community, tavily-python, langchainhub, langchain-openai, python-dotenv

pip install langchain wikipedia langchain-community tavily-python langchainhub langchain-openai openai python-dotenv

Example: Using LangDB with LangChain

Below is an example of how you can integrate LangDB into your LangChain workflow. The integration is designed to be as simple as possible, letting you focus on writing logic without worrying about setup complexities.

from langchain import hub
from langchain.agents import (
    AgentExecutor,
    create_tool_calling_agent,
)
from langchain_openai import ChatOpenAI
import os
from langchain_community.agent_toolkits.load_tools import load_tools
from langchain_community.tools.tavily_search.tool import TavilySearchResults
from langchain_community.utilities.tavily_search import TavilySearchAPIWrapper
import uuid


api_base = "https://api.us-east-1.langdb.ai"  ### LangDB API base URL
pre_defined_run_id = uuid.uuid4()
default_headers = {
    "x-project-id": "xxxxx",  ### LangDB Project ID
    "x-thread-id": str(pre_defined_run_id),
}
os.environ["OPENAI_API_KEY"] = "xxxx"  ### LangDB API key
os.environ["TAVILY_API_KEY"] = "tvly-xxxx"


def get_function_tools():
    search = TavilySearchAPIWrapper()
    tavily_tool = TavilySearchResults(api_wrapper=search)

    tools = [tavily_tool]

    tools.extend(load_tools(["wikipedia"]))

    return tools


def init_action():
    llm = ChatOpenAI(
        model_name="gpt-4o-mini",
        temperature=0.3,
        openai_api_base=api_base,
        default_headers=default_headers,
        disable_streaming=True,
    )
    prompt = hub.pull("hwchase17/openai-functions-agent")
    tools = get_function_tools()
    agent = create_tool_calling_agent(llm, tools, prompt)
    agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
    agent_executor.invoke(
        {"input": "Who is the owner of Tesla company? Let me know details about owner."}
    )


init_action()

In this example:

We define an API base URL for LangDB (api_base), which points to the LangDB.
We add a project-id to the headers to specify the LangDB project being used, and a thread-id to create a unique thread for tracking and logging the execution within LangDB.
Once executed, the agent is able to process an input query ("Who is the owner of Tesla company? Let me know details about owner.") and use the tools integrated into the agent, like Tavily search and Wikipedia.

Dynamic Model Switching

One of LangDB’s most powerful features is the ability to seamlessly switch between different models without major changes in the existing codebase. In the above example, if you want to use Claude 3.5 Sonnet, all you need to do is update the model name in your configuration:

# Switch to Anthropic's Claude model
llm = ChatOpenAI(
    model_name='claude-3-sonnet-20240229',  # Change model here
    temperature=0.3, 
    openai_api_base=api_base, 
    default_headers=default_headers
)

With this small change, LangDB takes care of the rest, ensuring that your application can dynamically adapt to new models without the need for rewriting large parts of your codebase.

Tracing in LangDB

LangDB’s tracing feature provides real-time visualizations of your AI workflows, breaking down the time spent in each stage. The trace shows key stages like:

API Invoke: Total time for the request.
Model Call: Time spent interacting with the model.
Tool Usage: Duration of specific tool calls.

The trace visualization below highlights these stages, helping you identify bottlenecks and optimize your workflow.

This detailed view makes it easier to diagnose performance issues and fine-tune your LangChain integrations.

Conclusion

Using LangDB with LangChain is a powerful yet straightforward way to manage and trace your AI workflows. By leveraging LangDB’s capabilities, you can focus on developing complex workflows without worrying about the operational overhead. The ability to seamlessly switch between models also ensures that you can stay agile as new AI technologies emerge.

Start integrating LangDB with LangChain today, and enjoy the flexibility and scalability it offers; Check out LangDB!

Why We Built an AI Gateway in Rust: A Performance-Centric Decision

Mrunmay — Thu, 13 Feb 2025 18:44:14 +0000

When building our AI gateway, we knew performance would be a critical factor. Unlike most AI software written in Python, an AI gateway acts as the proxy layer between users and inference engines. This gateway must handle high concurrency, low latency, and large data volumes efficiently. Python, while dominant in the AI ecosystem, struggles under these demands due to its runtime overhead and limitations with concurrency.

To demonstrate why we chose Rust, we benchmarked three popular programming environments—Rust, Python, and JavaScript (Node.js)—to evaluate their performance under high-load conditions. Rust emerged as the clear winner, offering predictable and stable performance even at scale.

Benchmark Setup: Simulating Real-World AI Traffic

We built an HTTP/2 streaming server and a corresponding client to mimic real-world AI workloads. Here’s how the setup worked:

Server:

Streams tokens at a fixed inter-token latency of 25ms, similar to the tokenized output of an AI inference engine.
Uses HTTP/2 to deliver tokenized data efficiently to multiple clients.
Implements asynchronous programming to support thousands of connections concurrently.

Client:

Gradually establishes up to 15,000 simultaneous connections to the server.
Measures the intra-token latency—the time between consecutive tokens received from the server. This metric reflects the server’s ability to scale under increasing load.
Ensures that connections remain stable and records latency for each connection.

Test Workflow:

The server was implemented in Rust, Python, and JavaScript (Node.js) to ensure a fair comparison.
The client progressively increased the number of active connections, starting with a small number and scaling up to 15,000.
Intra-token latency measurements were collected for each implementation to evaluate performance under load.

Results: Rust vs. Python vs. JavaScript (Node.js)

The chart below illustrates the intra-token latency (in milliseconds) as the number of concurrent connections increases:

Key Observations:

Rust:
- Rust exhibited the most stable performance, maintaining a near-linear increase in latency as connections scaled.
- At 15,000 connections, Rust's intra-token latency reached approximately 75ms, only 3x the baseline inter-token latency of 25ms.
- Rust’s efficiency highlights its ability to handle high concurrency without significant degradation.
Python:
- Python's intra-token latency grew exponentially, exceeding 200ms at 15,000 connections.
- This exponential growth demonstrates Python's inherent limitations in managing large-scale concurrency and resource contention.
JavaScript (Node.js):
- Node.js initially performed better than Python, maintaining lower latency up to 7,500 connections.
However, its performance began to degrade significantly beyond this point, reaching over 150ms at 15,000 connections.
- This result underscores Node.js’s event-driven model, which works well for moderate concurrency but struggles under extreme loads.

Why Rust is the Best Choice for an AI Gateway

Predictable, Scalable Performance:

Rust’s ability to maintain 75ms latency at 15,000 connections demonstrates its scalability. Its near-linear latency growth makes it ideal for high-concurrency systems.
Concurrency Without Compromise:

Rust’s async programming model (e.g., Tokio) efficiently manages thousands of simultaneous connections. Unlike Python, Rust avoids the bottlenecks of the Global Interpreter Lock (GIL) and utilizes system resources optimally.
Resource Efficiency:

Rust compiles directly to machine code, ensuring minimal runtime overhead. Its memory safety and zero-cost abstractions allow for predictable and efficient resource management.
Low-Level Control:

Rust provides fine-grained control over threading and memory, making it the best choice for performance-critical applications like AI gateways.

Why Python and JavaScript Fall Short

Python:

Concurrency Limitations: The GIL prevents true multi-threading, causing severe bottlenecks under high load.

Runtime Overhead: Python's interpreted nature adds significant latency, making it unsuitable for latency-sensitive applications.

Exponential Growth: As connections increase, Python's performance deteriorates rapidly, with latency exceeding acceptable thresholds.
JavaScript (Node.js):

Event-Driven Model: Node.js performs well under moderate concurrency but struggles as the number of simultaneous connections grows beyond 7,500.

Resource Contention: While Node.js handles asynchronous I/O well, it lacks the low-level control offered by Rust, leading to degraded performance at scale.

Why AI Gateways Must Be Built with Performance in Mind

An AI gateway is more than a simple intermediary. It plays a critical role in ensuring:

Real-Time Responses: Users expect tokenized outputs to arrive with minimal delay, making low latency essential.
Scalability: AI gateways must handle thousands or tens of thousands of simultaneous connections to accommodate large-scale applications.
Reliability: Inconsistent performance or connection drops can severely impact user experience and application reliability.

Rust excels in all these areas, delivering predictable, stable performance at scale, making it the ideal language for building high-performance AI gateways.

The Takeaway: Rust is the Future of AI Gateways

Our benchmark results clearly show that while Python and JavaScript (Node.js) have their strengths, they are ill-suited for building performance-critical AI gateways:

Python struggles with concurrency and runtime overhead, leading to exponential latency growth.
Node.js performs better but falters under extreme loads, making it unreliable for high-concurrency scenarios.

Rust, on the other hand, delivers consistent, scalable performance with low latency, even at 15,000 connections. By choosing Rust for our AI gateway, we’ve built an infrastructure that can handle the demands of modern AI applications with ease.

If you’re building an AI gateway or any performance-critical infrastructure, Rust isn’t just an option—it’s the solution. When every millisecond matters, Rust is the language that ensures you meet the challenge head-on.

Introduction to AI Gateway

Mrunmay — Tue, 10 Dec 2024 17:04:23 +0000

Rise Of LLMs

In the ever-evolving landscape of technology, few innovations have created waves as transformative as artificial intelligence (AI).

The rise of AI—fueled by large language models (LLMs)—is reshaping how we think about software, automation, and the user experience. Much like the pivotal shifts brought by mobile computing, cloud infrastructure, and microservices architecture, AI represents a foundational shift in how we design and deliver technology.

As we embrace this new era of AI, large language models are not just reshaping what’s possible—they’re redefining how we approach technology itself. However, this transformation is only as strong as the underlying frameworks that support it. APIs, as the silent workhorses of modern software, serve as the crucial bridge connecting AI’s potential with real-world applications, ensuring its seamless integration into the fabric of our digital ecosystem.

What is an AI Gateway

An AI Gateway is a middleware platform that simplifies the integration, management, and scaling of artificial intelligence (AI) models and services within an organization’s IT infrastructure. It serves as a critical bridge between AI systems—such as large language models (LLMs)—and the applications or services that consume them.

LangDB’s AI Gateway provides a simplified way to connect with multiple Large Language Models (LLMs) using a single line of code. It’s designed to help developers integrate and manage LLMs efficiently while keeping operations cost-effective.

Key Features of LangDB's AI Gateway

LangDB’s AI Gateway provides developers with a practical, streamlined solution for integrating and managing LLMs. Here’s how it stands out:

Cost Management: Gain control over LLM usage by tracking and optimizing spending, ensuring cost-effective operations.
Dynamic Routing: Automatically route requests to the most suitable LLM based on performance, cost, or availability, optimizing results.
Scalability: Seamlessly scale your AI integrations across projects and environments without added complexity.
Seamless Integration: Connect with multiple AI providers using a single line of code, reducing development overhead and increasing productivity.
Future-Ready: Stay adaptable as new LLMs emerge, ensuring your workflows remain at the cutting edge of AI technology.

Built with developers in mind, LangDB’s AI Gateway empowers you to integrate multiple AI models using just a single line of code. Whether you’re scaling enterprise applications or experimenting with new AI tools, LangDB ensures your workflows are efficient, cost-effective, and future-ready.

Conclusion

As AI adoption grows, LangDB’s AI Gateway simplifies the process, helping developers focus on building smarter, faster applications without operational overhead. By offering seamless integration, cost optimization, and scalable solutions, it reduces complexity and empowers developers to focus on innovation and delivering impactful results.

Exploring Different Chunking Strategies and Working with Unstructured Data

Mrunmay — Fri, 23 Aug 2024 04:13:32 +0000

LangDB provides a powerful arsenal of functions for developers to deal with unstructured data. These functions are designed to streamline common tasks in data extraction, and text chunking. Let's dive into some of the key functions and see how they can make your life easier.

load

The load function converts any webpage/file into bytes. These bytes can be used to extract text or layout from the file/webpage.

SELECT * FROM load('s3://sample-onlineboutique-codefiles/onlineboutique-codefiles/just-deserts-spring-obooko-small.pdf');

content
[37,80,68,70,45,49,46,54,10,37,-30,-29,-49,-45,10,53,32,48,32,111,98,106,10,60,60,10,47,66,77,32,47,78,111,114,109,97,108,10,47,99,97,32,49,10,62,62,10,101,110,100,111,98,106,10,56,32,48,32,111,98,106,10,60,60,10,47,70,105,108,116,101,114,32,47,70,108,97,116,101,68,101,99,111,100,101,10,47,76,101,110,103,116,104,32,50,57,54,10,47,78,32,51,10,62,62,10,115,116,114,101,97,109,10,120,-100,125,-112,-67,74,-61,96,20,-122,31,107,65,20,-59,65,-121,14,14,25,28,92,-44,-2,104,127,-64,-91,-83,88,92,91,-123,86,-89,52,77,-117,-40,-97,-112,-90,-24,5,-24,-26,-32,-22,38,46,-34,-128,-24,101,40,8,14,-30,-32,37,-120,-96,-77,111,26,36,5,-87,-25,-16,-26,123,120,-13,-110,47,-25,64,36,-122,42,26,-121,78,-41,115,-53,-91,-126,81,-83,29,24,83,-17,76,-88,-121,101,90,125,-121,-15,-91,-44,-9,75,-112,125,94,-3,39,55,-82,-90,27,118,-33,-46,-7,33,121,-82,46,-41,39,27,-30,-59,86,-64,-89,62,-41,3,-66,-16,-7,-60,115,60,-15,-75,-49,-18,94,-71,40,-66,19,-81,-76,70,-72,62,10]

content

[37,80,68,70,45,49,46,54,10,37,-30,-29,-49,-45,10,53,32,48,32,111,98,106,10,60,60,10,47,66,77,32,47,78,111,114,109,97,108,10,47,99,97,32,49,10,62,62,10,101,110,100,111,98,106,10,56,32,48,32,111,98,106,10,60,60,10,47,70,105,108,116,101,114,32,47,70,108,97,116,101,68,101,99,111,100,101,10,47,76,101,110,103,116,104,32,50,57,54,10,47,78,32,51,10,62,62,10,115,116,114,101,97,109,10,120,-100,125,-112,-67,74,-61,96,20,-122,31,107,65,20,-59,65,-121,14,14,25,28,92,-44,-2,104,127,-64,-91,-83,88,92,91,-123,86,-89,52,77,-117,-40,-97,-112,-90,-24,5,-24,-26,-32,-22,38,46,-34,-128,-24,101,40,8,14,-30,-32,37,-120,-96,-77,111,26,36,5,-87,-25,-16,-26,123,120,-13,-110,47,-25,64,36,-122,42,26,-121,78,-41,115,-53,-91,-126,81,-83,29,24,83,-17,76,-88,-121,101,90,125,-121,-15,-91,-44,-9,75,-112,125,94,-3,39,55,-82,-90,27,118,-33,-46,-7,33,121,-82,46,-41,39,27,-30,-59,86,-64,-89,62,-41,3,-66,-16,-7,-60,115,60,-15,-75,-49,-18,94,-71,40,-66,19,-81,-76,70,-72,62,10]

extract_text

The extract_text() function extracts text from various file types, with specific options available for PDF files.

Parameters

Parameter	Type	Optional	Description	Possible Values	Sample Value
`path`	String	No	The file path to extract text from	Any valid URL	`'https://example.com'`
`type`	String	Yes	Type of file	PDF, Markdown, Text, HTML	`'pdf'`
`page_rage`	Array(Int)	Yes	Extra parameter for PDF file type for the range of page numbers	Array of Start and Ending page numbers	[1, 10]
`per_page`	Bool	Yes	Extra parameter for PDF file type to chunk per Page	true, false	true

Usage with `load` function

SELECT * FROM extract_text((SELECT * from load('s3://sample-onlineboutique-codefiles/onlineboutique-codefiles/just-deserts-spring-obooko-small.pdf')),
    type => 'pdf' ,
    per_page => false
);

content	metadata	page_no
JUST DESERTS Aniela Spring © Copyright Aniela Spring 2024 This is an authorised free edition from www.obooko.com Although you do not have to pay for this book, the author’s intellectual property rights remain fully protected by international Copyright laws. You are licensed to use this digital copy strictly for your personal enjoyment only: it must not be redistributed commercially or offered for sale in any form. If you paid for this free edition, or to gain access to it, we suggest you demand an immediate refund and report the transaction to the author and Obooko. All characters are fictitious and any resemblance to real persons, living or dead, is utterly coincidental. 1	{"total_pages":2,"page_range":"(0, 2)"}	0

These all functions are best suited for raw text. However, if you want to get the layout information from a document, LangDB has support for it too.

extract_layout

The extract_layout function enables structured data extraction with layout information from a document.

Parameters

Parameter	Type	Optional	Description	Possible Values	Sample Value
`path`	String	No	The file path to extract text from	Any valid file URL	'https://example.pdf'
`type`	String	Yes	Type of file	Raw, PDF, Image	'pdf'
`page_range`	Array(Int)	Yes	Extra parameter for PDF file type for the range of page numbers	Array of Start and Ending page numbers	[1, 10]
`parallelism`	Int	Yes	Extra parameter for PDF file type to process pages parallelly	2, 4, 5	2

Extracting Layout information from a PDF

SELECT * FROM extract_layout(
    path => 's3://sample-onlineboutique-codefiles/onlineboutique-codefiles/just-deserts-spring-obooko-small.pdf',
    type=> 'pdf'
);

block_idx	block_id	block_type	text	confidence	entity_types	relationships
0	c7261e9c-be58-4776-a1de-70adf6e4e6e6	PAGE		0	[]	[["CHILD",["23112b0d-4062-424d-bbb3-4f4aa82f4d80","3e3c5562-b018-4f75-85d9-6e7771489ba0","f08a9210-eedb-4150-99e2-a5d22b26e029","f3087bee-7680-4024-aeff-60ab0bdc1dac"]]]
1	23112b0d-4062-424d-bbb3-4f4aa82f4d80	LINE	Don't forget about your past, because it never forgets about you.	99.88849	[]	[["CHILD",["102e10d5-fd45-46ee-9890-b70279c6e532","af6bad3a-34fc-462e-9033-c1af2bd5aa1a","aab2849a-4a4b-499c-a16f-43c55fb5dffd","78ef1f76-d8a5-413f-be87-cff88194b7e1","9f41e657-f307-487f-872e-569272305ad4","01ae2b2a-755f-4ef9-9ddc-24eaefbaabd4","7484031d-5259-48ad-a3c7-bbcb862d34f0","783ddab6-47a3-48aa-b56b-adc564daa8cd","d7a69ab3-c601-4d9d-9632-7d4f176b2462","d8f537c7-4c64-4a01-9792-088660b1631d","fb7ad2cb-e72d-4013-8399-fa32d46cb21d"]]]
2	3e3c5562-b018-4f75-85d9-6e7771489ba0	LINE	JUST DESERTS	98.635315	[]	[["CHILD",["5e4fc404-7326-4195-a3d7-343a4dea7a8f","f3efd0b2-0c54-49bc-9867-6830eab05403"]]]
3	f08a9210-eedb-4150-99e2-a5d22b26e029	LINE	ANIELA SPRING	99.87999	[]	[["CHILD",["f4e95636-470f-45eb-a599-4d3e00f754d6","5e8676d2-f90c-4455-b179-083da72c647e"]]]
4	102e10d5-fd45-46ee-9890-b70279c6e532	WORD	Don't	99.96765	[]	[]
5	af6bad3a-34fc-462e-9033-c1af2bd5aa1a	WORD	forget	99.908676	[]	[]
6	aab2849a-4a4b-499c-a16f-43c55fb5dffd	WORD	about	99.9353	[]	[]
7	78ef1f76-d8a5-413f-be87-cff88194b7e1	WORD	your	99.92315	[]	[]
8	9f41e657-f307-487f-872e-569272305ad4	WORD	past,	99.73978	[]	[]
9	01ae2b2a-755f-4ef9-9ddc-24eaefbaabd4	WORD	because	99.9515	[]	[]

Extracting Layout information from an Image

Similarly, you can extract layout information from an image through the following code:

SELECT * FROM extract_layout(
    path => 'https://langdb-sample-data.s3.ap-southeast-1.amazonaws.com/Screenshot+from+2024-08-09+09-49-18.png',
    type => 'image'
);

chunk

The chunk function breaks down large texts into smaller, manageable pieces. This is particularly useful for processing long documents, especially when working with models that have input size limitations.

Parameters

Parameter	Type	Optional	Description	Possible Values	Sample Value
`raw_text`	String	No	The raw text which needs to be chuncked	Any String	'In a quaint village...'
`type`	String	No	Unit of chunking	Char, Word, Sentence, Paragraph	'Char'
`chunk_size`	Int	Yes	Number of units to be present in a Chunk	Any non-negative integer	100
`overlap`	Int	Yes	Number of units to overlap between consecutive chunks	Any non-negative integer	20
`trim`	Bool	Yes	Whether to trim whitespace from the start and end of each chunk	true, false	true

Chunking Raw text into Char with Chunk Size

SELECT * FROM chunk('In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.

One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.',
    type => 'Char',
    trim => true,
    chunk_size => 200);

text	index
In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure.	0
Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.	1
One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before.	2
As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen.	3
The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty.	4
She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.	5

Chunking Raw text into Word with Chunk Size and Overlap

SELECT * FROM chunk('In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.

One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.',
    type => 'Word',
    chunk_size => 30,
    overlap => 10);

text	index
In a quaint village nestled in the heart of the countryside there lived a young girl named Lily She was known throughout the village for her vibrant imagination and her	0
known throughout the village for her vibrant imagination and her love for adventure Every day Lily would set out to explore the lush forests and rolling hills that surrounded her	1
explore the lush forests and rolling hills that surrounded her home always eager to discover something new and exciting One particularly sunny morning Lily decided to venture deeper into the	2
particularly sunny morning Lily decided to venture deeper into the woods than she ever had before As she walked she stumbled upon a hidden grove filled with the most beautiful	3
stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen The colors were so vivid and the petals so delicate that Lily couldnt help but	4
and the petals so delicate that Lily couldnt help but marvel at their beauty She spent hours in the grove carefully examining each flower and breathing in their sweet fragrance	5

Chunking Raw Text into Sentences

SELECT * FROM chunk('In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.

One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.',
    type => 'Sentence');

text	index
In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily	0
She was known throughout the village for her vibrant imagination and her love for adventure	1
Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting	2
One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before	3
As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen	4
The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty	5
She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance	6

Chunking Raw Text into Paragraphs

SELECT * FROM chunk('In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.

One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.',
    type => 'Paragraph');

text	index
In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.	0
One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.	1

text

index

In a quaint village nestled in the heart of the countryside, there lived a young girl named Lily. She was known throughout the village for her vibrant imagination and her love for adventure. Every day, Lily would set out to explore the lush forests and rolling hills that surrounded her home, always eager to discover something new and exciting.

One particularly sunny morning, Lily decided to venture deeper into the woods than she ever had before. As she walked, she stumbled upon a hidden grove filled with the most beautiful flowers she had ever seen. The colors were so vivid and the petals so delicate that Lily couldnt help but marvel at their beauty. She spent hours in the grove, carefully examining each flower and breathing in their sweet fragrance.

Combining functions

We have seen how these functions behave individually, but the real power of these functions and LangDB lies within combining. Let's take an example of a job description pdf.

Firstly, we will use load to convert the file into bytes and then extract_text to get all the raw text from it.
After that, we will Chunk by Char with a chunk_size of 2000.

select * from chunk(
    (
        select content from extract_text((
            select * from load('https://www.stjohneyehospital.org/wp-content/uploads/2024/05/Job-Description-Accountant.pdf',
            type=> 'pdf')
        ))
    ),
    chunk_size => 2000,
    type => 'Char',
    trim => false
)

text	index
ST. JOHN EYE HOSPITAL – JERUSALEM JOB DESCRIPTION Title Accountant Department Finance Section Reports to Director of Finance Hours 40 hrs per week (inc of lunch breaks) Date February 24 formulated/updated General Statement of Duties: To play a major role in controlling the costing system of purchases and payroll by supporting the existing accountants and providing reports as instructed by the Director of Finance. Main Responsibilities: To act as a substitute for the senior/payroll accountant during her absence. Act as the Projects’ accountant and point of contact by providing reports and supporting documents for projects and any other assistance as needed. Act as the Cafeteria’s accountant which includes recording of expenses and income, produce reports for management, as well as reporting to the tax authority. Responsible for examining, recording, and summarizing the organization’s West Bank costs, mainly payroll and purchases. The Accountant records and classifies expenditures to create financial statements for senior management. Ensure that all costs are identified and recorded accurately. Maintaining accurate costing records in relation to labour and supplies. Process accounting transactions using the existing accounting software. Assist in the preparation of the monthly local management accounts and comparing it to budget, and report on any variance to DOF and other heads of departments. Process Palestinian payroll transactions using accounting and payroll systems and assist with the Israeli payroll system when needed (and ensure that the payroll taxes and national insurance are paid to the regulatory bodies on timely basis). Revision of purchases recorded at the pharmacy system. Monitor and coordinate payments for West Bank Suppliers Any other duties as assigned by the Director of Finance. General Responsibilities:	0
1. All staff are expected to report for work on time and fulfil their hours of duty, from time to time some flexibility may be required in order to meet the needs of the job and this may be outside regular hours of work. All staff are expected to promote and contribute to a cooperative and productive work environment. Staff are also expected to show respect and consideration to their colleagues and all patients and visitors to the hospital. All staff are expected to follow the dress code for their area of work. All uniforms as required by different work areas should be worn at all times. Staff who do not have a uniform are expected to wear appropriate, respectful, modest business dress. Jeans are not considered appropriate attire. The hospital is a no smoking hospital and smoking is only permitted in the designated smoking areas and only during official break periods. All staff will abide by confidentiality rules and will not disclose any information about patients, the staff or the workings of the hospital, except in certain circumstances where express permission is given as per the Confidentiality Policy. All staff are expected to comply at all times with the requirements of Health and Safety regulations and to take responsibility for the health and safety and welfare of others in the working environment ensuring that agreed safety procedures are carried out to maintain a safe environment. The Hospital has a Control of Visits in the Hospital and Security of Workers policy in order to help protect patients, visitors and staff and to safeguard their property. All employees have a responsibility to ensure that those persons using the Hospital and its service are as secure as possible. The Hospital is committed to equality and all staff are expected to treat colleagues, patients and visitors to the Hospital with dignity and respect, regardless of their ethnic background, religion, race, gender, age or	1
sexual orientation. All staff are expected to familiarise themselves with the requirements of the Hospitals policies and procedures for staff and also their specific area of work. All appointments within the Hospital are subject to pre-employment health screening. All staff are responsible for ensuring that all risks of cross infection to patients are minimised and that all policies, procedures and guidance relating to infection control practice are adhered to. All staff are responsible, where relevant, for ensuring that all equipment used by patients is clean/decontaminated as instructed by manufacturers and in line with the infection control/guidelines protocol and policy. The job description gives a general outline of the duties of the position and is not intended to be an inflexible or finite list of tasks. It may be varied, from time to time, after consultation with the member of staff. Any other duties as designated by your manager and which are commensurate with the grade. Essential requirements for the post: Bachelor’s degree in accounting. At least one year experience in the accounting field mainly in the payable’s sections. At least one year experience in processing payroll. Knowledge and experience of the Israeli & Palestinian Payroll systems is required. Previous experience in projects is a plus. Very Good in English and Hebrew languages. Computer literate especially excel spread sheets. Good eye for details. Methodical and organised. Ability to work under pressure. Ability to meet deadlines. Ability to lead & contribute to team work as necessary. Name ______________________________________________ Date ________________________ Signed ______________________________________	2

By leveraging functions like load, extract_text, extract_layout and chunk, LangDB equips developers with a powerful toolkit for overcoming unstructured data challenges. Whether you're dealing with disorganized text, intricate document layouts, or vast amounts of data, these functions provide the versatility and efficiency needed to convert raw information into structured, actionable insights. LangDB not only simplifies the complexity of data extraction and processing but also enhances the overall productivity of your development workflow.

Unlocking the Power of Text-to-SQL with LangDB

Sayantan Bera — Thu, 22 Aug 2024 17:28:55 +0000

The Text-to-SQL model in LangDB transforms natural language queries into precise Clickhouse SQL commands, streamlining data extraction from your databases. By understanding user intent and targeting specific tables, it delivers highly accurate results, bridging the gap between natural language and complex SQL operations.

Tools required for Text-to-SQL Model

The model relies on several built-in tools provided by LangDB:

langdb_raw_query: Executes SQL queries directly on the database.
langdb_validate_query: Ensures the generated SQL query is correct before execution.
list_tables: Retrieves available table names within the database.
describe_tables: Provides detailed information about specific tables.

These tools work together to simplify natural language to SQL translation and ensure query accuracy.

Step 1: Prompt Creation for Query Agent

CREATE PROMPT query_prompt(
    system "You are an advanced query agent specializing in Clickhouse SQL. Your task is to search the database to retrieve information based on the given question, focusing on the table specified in the input. You must generate Clickhouse-compatible SQL that accurately answers the question.

    Follow these steps for EVERY query:

    Step 1: Get Database Schema
    Action: list_tables tool
    Action Input: all
    Observation: [Output from list_tables tool list all tables in the specified schema]


    Step 2: Analyze Tables and Focus on pick tables to query
    Thought: Based on the question, and the available tables, I will focus on the required tables and identify any potential relationships with other tables if needed.
    Related Tables (if any): [Names of potentially related tables from the schema]
    Reason: [Explanation for why the specified table is appropriate and how any related tables might be useful]

    Step 3: Describe Table
    Action: describe_tables tool
    Action Input: [Related Table names from Step 2]
    Observation: [Output from list_tables tool list all tables in the specified schema]


    Step 4: Generate and Execute Clickhouse SQL Query
    Thought: I will now generate a Clickhouse SQL query that accurately answers the question, focusing on the specified table. Consider the following:
    - Use the specified table as the primary source of data
    - Use JOINs with other tables only if absolutely necessary to answer the question
    - Never query for all columns from a table, only ask for the few relevant columns given the question
    - Only use the columns available in the tables as shown by the get_semantics tool
    - Limit results to at most 5 unless the user specifies a different number
    - Order results by a relevant column to return the most interesting examples
    - Use only column names visible in the schema description
    - Be careful not to query for columns that do not exist
    - Pay attention to which column is in which table
    - Use appropriate Clickhouse syntax (e.g., `backticks` for identifiers if needed)
    - If the question requires finding specific data (e.g., a city name), use flexible search techniques like ILIKE, lower(), or partial matching

    Step 4: Validate the SQL Query
    Action: langdb_validate_query tool
    Action Input: [Your generated ClickHouse SQL query]
    Observation: [true or false]

    Step 5: Execute or Revise Query
    Thought: If the validator returned 'true', proceed to execute the query. If 'false', revise the query and go back to Step 3.
    Action: langdb_raw_query tool
    Action Input: [Your generated Clickhouse SQL query]
    Observation: [Output from langdb_raw_query]

    Always provide your final response in this EXACT format:

    Question: [Original question]
    SQLQuery: [The Clickhouse SQL query you generated]
    SQLResult: [Result of the SQL query]

    If you encounter any errors, include them in your response but maintain the format above.

    Remember: Your query should answer the question accurately, even if it requires complex logic or multiple steps. Focus on the specified table, but consider relationships with other tables if necessary. Use flexible search techniques when looking for specific data. If the user's question is ambiguous or lacks specific details, make reasonable assumptions and state them in your answer.",

    human "{{input}}"
)

Step 2: Model Creation

Now, we can create the models that can leverage the tools that were created earlier.

CREATE MODEL text_to_sql(
    input
) USING open_ai_provider(model_name = 'gpt-4o')
PROMPT query_prompt
TOOLS (
    list_tables COMMENT 'Tool to get the tables from the database',
    langdb_raw_query COMMENT 'Tool to execute SQL query over the database',
    describe_tables COMMENT 'Tool to describe a table in the database',
    langdb_validate_query COMMENT 'Tool to validate generated SQL query'
)
SETTINGS retries = 3;

And just like that, we’re done! Building this Text-to-SQL model with LangDB was this straightforward.

Give it a shot yourself Text-to-SQL Notebook — LangDB makes it so easy for you.

Join our Slack community for more discussions and support:

langdbcommunity.slack.com

Running Text-To-SQL on MongoDB: Working with Postgres, MySQL, MongoDB and Others

Mrunmay — Wed, 07 Aug 2024 13:42:52 +0000

When working with LangDB, one question that developers ask how langdb integrates with their existing data infrastructure. Fortunately, LangDB offers seamless connectivity to your pre-existing databases, allowing you to leverage your data without significant modifications.

Database Integrations

Table Engines

LangDB, built on top of Clickhouse, provides native support for integrating with a wide range of databases. This means you can use the Integration Engines of Clickhouse to continue using your existing databases while benefiting from LangDB's SQL arsenal.

Here are a few examples of how you can connect to some popular databases. You can find more about them here: Table Engines

NOTE: Original article can be found at https://app.langdb.ai/share/apps/4250cb28-5e2b-421f-9711-0f334452aef5

PostgreSQL

CREATE TABLE [IF NOT EXISTS] table_name
(
    col_name1 type1 ,
    col_name2 type2 ,
    ...
) ENGINE = PostgreSQL(host:port, database, table, user, password)

MySQL

CREATE TABLE [IF NOT EXISTS] table_name 
(
    col_name1 type1 ,
    col_name2 type2 ,
    ...
) ENGINE = MySQL(host:port, database, table, user, password)

MongoDB

CREATE TABLE [IF NOT EXISTS] table_name 
(
    col_name1 type1 ,
    col_name2 type2 ,
    ...
) ENGINE = MongoDB(host:port, database, collection, user, password [, 'ssl=true&tls=true&tlsAllowInvalidCertificates=true&connectTimeoutMS=30000&socketTimeoutMS=60000&authSource=admin']);

Table Functions

Apart from the table engines, you can also query your existing databases directly using table functions There are Postgresql, MongoDB, MySQL, and many other table functions.
You can read more about them here: Table Functions

Let's take the example of MongoDB. On MongoDB Atlas, I will use Sample Analytics Dataset.

The sample_analytics database contains three collections for a typical financial services application. It has customers, accounts, and transactions.

We will focus on accounts, but feel free to play around. The accounts collection contains the details of the users.

We can use the mongodb table function to query the collection quickly.

SELECT * from mongodb(
    'host:port',
    'sample_analytics',
    'customers',
    'user',
    'password',
   '_id String, 
     username String, 
     name String, 
     address String, 
     birthdate DateTime, 
     email String, 
     accounts Array(Int32)',
    'connectTimeoutMS=10000&ssl=true&authSource=admin'
) limit 5;

_id	username	name	address	birthdate	email	accounts
5ca4bbcea2dd94ee58162a69	valenciajennifer	Lindsay Cowan	Unit 1047 Box 4089 DPO AA 57348	1994-02-19 23:46:27	cooperalexis@hotmail.com	[116508]
5ca4bbcea2dd94ee58162b29	crodriguez	Charles Jones	183 Young Mountain Rossmouth, DC 11579	1974-01-24 12:12:53	vancejohnny@hotmail.com	[467666]
5ca4bbcea2dd94ee58162a76	portermichael	Lauren Clark	1579 Young Trail Jessechester, OH 88328	1980-10-28 16:25:59	briannafrost@yahoo.com	[883283,980867,164836,200611,528224,931483]
5ca4bbcea2dd94ee58162a72	wesley20	James Sanchez	8681 Karen Roads Apt. 096 Lowehaven, IA 19798	1973-01-13 16:17:26	josephmacias@hotmail.com	[987709]
5ca4bbcea2dd94ee58162bf4	skinnercraig	Ashley Lindsey	Unit 6691 Box 1189 DPO AP 53029	1994-06-16 07:38:48	robertwalker@gmail.com	[375655,892096,401997,253554,890055,959435]

However, to use LangDB's collection of tools like Text-to-SQL and Embeddings, it is better to use the table engines. You can do something like this for the above query to get read-only access to the remote MongoDB collection.

CREATE TABLE IF NOT EXISTS customers_mongodb
(
     _id "String", 
     username "String", 
     name "String", 
     address "String", 
     birthdate DateTime, 
     email "String", 
     accounts Array("Int32")
) ENGINE = MongoDB(
    'host:port',
    'sample_analytics',
    'customers',
    'user',
    'password',
    'ssl=true&tls=true&tlsAllowInvalidCertificates=true&connectTimeoutMS=30000&socketTimeoutMS=60000&authSource=admin'
);

SELECT * FROM customers_mongodb limit 5

_id	username	name	address	birthdate	email	accounts
5ca4bbcea2dd94ee58162a69	valenciajennifer	Lindsay Cowan	Unit 1047 Box 4089 DPO AA 57348	1994-02-19 23:46:27	cooperalexis@hotmail.com	[116508]
5ca4bbcea2dd94ee58162b29	crodriguez	Charles Jones	183 Young Mountain Rossmouth, DC 11579	1974-01-24 12:12:53	vancejohnny@hotmail.com	[467666]
5ca4bbcea2dd94ee58162a76	portermichael	Lauren Clark	1579 Young Trail Jessechester, OH 88328	1980-10-28 16:25:59	briannafrost@yahoo.com	[883283,980867,164836,200611,528224,931483]
5ca4bbcea2dd94ee58162a72	wesley20	James Sanchez	8681 Karen Roads Apt. 096 Lowehaven, IA 19798	1973-01-13 16:17:26	josephmacias@hotmail.com	[987709]
5ca4bbcea2dd94ee58162bf4	skinnercraig	Ashley Lindsey	Unit 6691 Box 1189 DPO AP 53029	1994-06-16 07:38:48	robertwalker@gmail.com	[375655,892096,401997,253554,890055,959435]

Now that we have created a table, we can use a Text-To-SQL model directly onto the database.

SELECT text_to_sql('How Many accounts are from people who were born in 1994')

Question: How many accounts are from people who were born in 1994?
SQLQuery: `SELECT COUNT(*) as account_count FROM customers_mongodb WHERE toYear(birthdate) = 1994`
SQLResult: [{"account_count":"19"}]

Let's verify and run that query for ourselves to confirm it.

SELECT COUNT(*) as account_count FROM customers_mongodb WHERE toYear(birthdate) = 1994

account_count
19

LangDB's features like Text-to-SQL and embeddings can streamline your data querying and analysis process, making extracting valuable insights from your databases more accessible. The example we walked through illustrates the simplicity of getting started with LangDB and how it can complement your existing setup.

As you explore LangDB further, remember that its flexibility and compatibility can significantly enhance your development workflow.

So, if you're ready to take your database experience to the next level with Gen AI, try LangDB. Its integration capabilities and robust features might become a go-to tool in your development arsenal.

Building a PDF Chatbot with RAG using SQL

Mrunmay — Mon, 15 Jul 2024 06:41:04 +0000

In the realm of AI chatbot development, there are many ways to create a PDF chatbot using Retrieval Augmented Generation. There are tools like LangChain and LlamaIndex, which are the popular choice. However, in this post, we’ll be exploring how you can achieve this using SQL.

Let’s go over the general steps of building a RAG chatbot:

Loading and Extracting PDF
Generating and Storing Vector Embeddings
Retrieving relevant information using Vector Search
Generating and Returning Answer

Why SQL?

SQL queries are straightforward and efficient, making data manipulation and retrieval easier on large datasets.
Combining structured and unstructured data is accessible on the go.
Streamlines the development process by enabling the creation of workflows with minimal code.

Here are a few examples of how easily you can create embeddings, create a prompt for the LLM and finally create a model.

-- Embeddings Model 
CREATE EMBEDDING MODEL IF NOT EXISTS generate_embed(
input COMMENT 'This is the input of the content whose embeddings are created'
) USING open_ai_provider(embedding_model='text-embedding-ada-002', encoding_format='float', dimensions=100)

-- Completions Model
CREATE MODEL IF NOT EXISTS search(
  input
) USING open_ai_provider()
PROMPT (system "You are a helpful assistant. You have been provided with similar tool to search over the SEC fillings. 
                 Your task is also to help users with any queries they might have about the SEC filings. 
                 Go through all the content and then respond.",
  human "{{input}}")
TOOLS (similar)
SETTINGS retries = 1;

Using SQL in this manner provides a robust framework for building a PDF chatbot, leveraging the power of SQL for workflow management.
So, the final execution of the model will look like this:

You can follow the full tutorial here: https://app.langdb.ai/samples/preview?file_name=QA%20on%20PDF%20%26%20RAG%20using%20LangDB

Join our Slack community for more discussions and support: https://join.slack.com/t/langdbcommunity/shared_invite/zt-2haf5kj6a-d7NX6TFJUPX45w~Ag4dzlg

Forem: LangDB

Designing Smart Multi-Agent Workflows with Agno & LangDB

TL;DR:

The Code

The Architecture: A Trio of Financial Experts

Enhanced Tracing with pylangdb.agno.init()

Code Walkthrough: Building the Team

The Web Search Agent: Decoupled and Dynamic

The Finance Agent: The Quantitative Analyst

The Coordinating Team: The Orchestrator

Dynamic Tooling with Virtual Models and Virtual MCPs

Running the Team and Observing the Results

Full Observability with LangDB Tracing

Conclusion

Discover End-to-End Tracing on Google ADK with LangDB

The Magic Behind the Curtain: pylangdb.adk.init()

The Architecture: Root Agent and Sub-Agents

Dynamic Tooling with Virtual Models and Virtual MCPs

Example: The inspiration_agent and Google Maps

Example: Grounding with Google Search

Example: The planning_agent for Flights and Hotels

The Flow: From Query to Answer

Conclusion

Choose the right AI model: Comparision of gpt-4o, claude and gemini using LangDB

Chat Interface in LangDB

Supported Models

Why Perform Model Comparisons?

Get Started with LangDB

How to Integrate LangChain with LangDB

Pre-requisites

Example: Using LangDB with LangChain

Dynamic Model Switching

Tracing in LangDB

Conclusion

Why We Built an AI Gateway in Rust: A Performance-Centric Decision

Benchmark Setup: Simulating Real-World AI Traffic

Results: Rust vs. Python vs. JavaScript (Node.js)

Why Rust is the Best Choice for an AI Gateway

Why Python and JavaScript Fall Short

Why AI Gateways Must Be Built with Performance in Mind

The Takeaway: Rust is the Future of AI Gateways

Introduction to AI Gateway

Rise Of LLMs

What is an AI Gateway

Key Features of LangDB's AI Gateway

Conclusion

Exploring Different Chunking Strategies and Working with Unstructured Data

load

extract_text

Parameters

Usage with load function

extract_layout

Parameters

Extracting Layout information from a PDF

Extracting Layout information from an Image

chunk

Parameters

Chunking Raw text into Char with Chunk Size

Chunking Raw text into Word with Chunk Size and Overlap

Chunking Raw Text into Sentences

Chunking Raw Text into Paragraphs

Combining functions

Unlocking the Power of Text-to-SQL with LangDB

Tools required for Text-to-SQL Model

Step 1: Prompt Creation for Query Agent

Step 2: Model Creation

Running Text-To-SQL on MongoDB: Working with Postgres, MySQL, MongoDB and Others

Database Integrations

Table Engines

PostgreSQL

MySQL

MongoDB

Table Functions

Building a PDF Chatbot with RAG using SQL

Why SQL?

Enhanced Tracing with `pylangdb.agno.init()`

The Magic Behind the Curtain: `pylangdb.adk.init()`

Example: The `inspiration_agent` and Google Maps

Example: The `planning_agent` for Flights and Hotels

Usage with `load` function