Forem: Timon Vonk

Announcing Swiftide 0.31

Timon Vonk — Tue, 16 Sep 2025 15:39:36 +0000

Just released Swiftide 0.31 🚀 A Rust library for building LLM applications. From performing a simple prompt completion, to building fast, streaming indexing and querying pipelines, to building agents that can use tools and call other agents.

The release is absolutely packed:

Graph like workflows with tasks
Langfuse integration via tracing
Ground-work for multi-modal pipelines
Structured prompts with SchemaRs

... and a lot more, shout-out to all our contributors and users for making it possible <3

Even went wild with my drawing skills.

Full write up on all the things in this release at our blog and on github.

Swiftide 0.26 - Streaming agents

Timon Vonk — Thu, 08 May 2025 09:54:20 +0000

Funny how time flies and you forget to write a blog post every time there is a major release. We are now at 0.26, and a lot has happened since our last update (January, 0.16!). We have been working hard on building out the agent framework, fixing bugs, and adding features. Shout out to all the contributors who have helped us along the way, and to all the users who have provided feedback and suggestions.

Swiftide is a Rust library for building LLM applications. Index, query, run agents, and bring your experiments right to production.

To get started with Swiftide, head over to swiftide.rs, check us out on github, or hit us up on discord.

Finally they stream!

Better late than never, agents can now stream their output. Under the hood, the ChatCompletion trait received an additional method called complete_stream, which returns a stream of responses, both the accumulated response and the delta. All OpenAI like providers and Anthropic are supported. We decided on including the accumulated response for convenience. Let us know if that is too many bytes for you, and we're happy to take a look.

The kicker is that it can also be used with agents, like this:

    agents::Agent::builder()
        .llm(&anthropic)
        .on_stream(|_agent, response| {
            // We print the message chunk if it exists. Streamed responses also include
            // the full response (without tool calls) in `message` and an `id` to map them to
            // previous chunks for convenience.
            //
            // The agent uses the full assembled response at the end of the stream.
            if let Some(delta) = &response.delta {
                print!(
                    "{}",
                    delta
                        .message_chunk
                        .as_deref()
                        .map(str::to_string)
                        .unwrap_or_default()
                );
            };

            Box::pin(async move { Ok(()) })
        })
        // Every message added by the agent will be printed to stdout
        .on_new_message(move |_, msg| {
            let msg = msg.to_string();
            Box::pin(async move {
                println!("\n---\nFinal message:\n {msg}");
                Ok(())
            })
        })
        .limit(5)
        .build()?
        .query("Why is the rust programming language so good?")
        .await?;

Streaming the response back to users makes for a much snappier user experience. We've also implemented it in kwaak.

The completion response (streaming and non-streaming) now also includes the token usage.

MCP Support

Agents can now use tools provided by MCP (Model Context Protocol). The ecosystem is growing rapidly and there are quite a few cool tools available. Under the hood we're using the 'official' Rust implementation. We don't support creating MCP servers, as I think it's a bit out of scope.

Here is an example of adding MCP tools to an agent:


    // First set up our client info to identify ourselves to the server
    let client_info = ClientInfo {
        client_info: Implementation {
            name: "swiftide-example".into(),
            version: env!("CARGO_PKG_VERSION").into(),
        },
        ..Default::default()
    };

    // Use `rmcp` to start the server
    let running_service = client_info
        .serve(TokioChildProcess::new(
            tokio::process::Command::new("npx")
                .args(["-y", "@modelcontextprotocol/server-everything"]),
        )?)
        .await?;

    // Create a toolbox from the running server, and only use the `add` tool
    //
    // A toolbox reveals it's tools to the swiftide agent the first time it starts (if the state of
    // the agent was pending). You can add as many toolboxes as you want. MCP services are an
    // implementation of a toolbox. A list of tools is another.
    let everything_toolbox = McpToolbox::from_running_service(running_service)
        .with_whitelist(["add"])
        .to_owned();

    agents::Agent::builder()
        .llm(&openai)
        // Add the toolbox to the agent
        .add_toolbox(everything_toolbox)
        // Every message added by the agent will be printed to stdout
        .on_new_message(move |_, msg| {
            let msg = msg.to_string();
            let tx = tx.clone();
            Box::pin(async move {
                tx.send(msg).unwrap();
                Ok(())
            })
        })
        .build()?
        .query("Use the add tool to add 1 and 2")
        .await?;

Resuming agents from history

You can now resume agents from a pre-existing history. Technically this was already possible, we've made it a bit easier. By creating the agent context from an existing history, the agent will resume where it left off:

  let mut first_agent = agents::Agent::builder().llm(&openai).build()?;

  first_agent.query("Say hello!").await?;

  // Let's store the messages in a database, retrieve them back, and start a new agent
  let stored_history = serde_json::to_string(&first_agent.history().await)?;
  let retrieved_history: Vec<_> = serde_json::from_str(&stored_history)?;

  let restored_context = DefaultContext::default()
      .with_message_history(retrieved_history)
      .to_owned();

  let mut second_agent = agents::Agent::builder()
      .llm(&openai)
      .context(restored_context)
      // We'll use the one from the first agent, alternatively we could also pop it from the
      // previous history and add a new one here
      .no_system_prompt()
      .build()?;

  second_agent.query("What did you say?").await?;

  Ok(())

So much more

Since January there have been so many improvements, new integrations, and a myriad of fixes, it's hard to keep track. Luckily we have a changelog here
Some more highlights since then:

Broader adoption of Backoff for more controlled LLM error handling
Tool macros now support generics and nearly all types common for tool calling
Prompt templating setup is drastically simplified, just templates that can use Tera and that's it.
Duckdb support
Token estimation with tiktoken.rs

Feedback, suggestions, ideas, and contributions are super welcome. Swiftide aims to make LLM app development in Rust easier, while providing opinionated building blocks to get you started. It's your feedback that makes it worthwhile <3.

To get started with Swiftide, head over to swiftide.rs or check us out on github.

Kwaak, a different take on AI coding tools

Timon Vonk — Tue, 25 Feb 2025 22:12:35 +0000

Just did a major release of Kwaak (open source), adding a whole host of new features, model support (sonnet 3.7 and azure!) and other niceties.

Kwaak is different than other AI coding tools in that it tries to get out of your way. You can throw your backlog at it and have it burn through it, so we as engineers can work on the cool stuff. Under the hood it uses rust based rag shenanigans to index large codebases fast and can solve multiple tasks in parallel.

Oh and its open-source.

You can find the project at https://github.com/bosun-ai/kwaak

Kwaak 0.8 - More LLMs and usability improvements

Timon Vonk — Fri, 07 Feb 2025 15:59:20 +0000

Hey everyone,

A month ago we released Kwaak, a terminal app that allows you to spawn many coding agents in parallel, on your own machine (open and free). The idea: Have AI burn through your tech debt and backlog, so we as engineers can work on the fun creative stuff.

Since we released it, we received a lot of positive and constructive feedback <3 I'd just like to share some of the highlights we released since.

Ollama + OpenRouter support
A lot is now configurable; no more opinionated workflows
Pull and show the diff of the agent
Interactive configurator
Many, many usability fixes and improvements

Swiftide 0.16 brings AI agents to Rust

Timon Vonk — Tue, 07 Jan 2025 17:05:41 +0000

A pretty huge release for Swiftide, 0.16 allows you to build AI agents in Rust. You know, that seemingly hot AI thing everybody talks about, with promises of doing it all for you.

I'm a bit more optimistic, I think we just are not there yet. And when doing agents, performance and reliability suddenly really start to matter. At the same time, the space is moving extremely, ridiculously fast, so we need building blocks to keep innovating at the same pace, and not get stuck on infrastructure (or lifetime) issues.

I think with this release we put a step forward in doing exactly that.

Check out the full https://bosun.ai/posts/swiftide-0-16 or jump right into it on https://github.com/bosun-ai/swiftide

Swiftide 0.12 - Hybrid Search, search filters, parquet loader, and a giant speed bump

Timon Vonk — Fri, 13 Sep 2024 15:59:55 +0000

Excited to announce Swiftide 0.12 🚀 A Rust library for building AI applications using retrieval augmented generation.

Retrieving the most relevant information for a given query is the key challenge in when building AI applications. Research and our own experience shows that similarity search on vectors is not enough. The idea behind hybrid search is fairly simple:

Retrieve n documents with similarity search
Retrieve n documents with another kind of search (ie full text, sparse vectors)
Rerank the documents for relevancy
Take the top k documents

Swiftide can now do hybrid search with #Qdrant, supports filters in search, load parquet files for indexing (i.e. from #HuggingFace) and is a lot faster!

Trumpets and a big thanks to @ephraimkunz for his first contribution! Read the full release post here, check us out on github

Swiftide 0.9, a Rust native library for building LLM applications with RAG, brings Fluvio, Lancedb and Ragas support

Timon Vonk — Mon, 02 Sep 2024 14:30:36 +0000

Introducing Swiftide 0.9 with Fluvio as a starting point for indexing streams, lancedb for querying and indexing and RAGAS support

To get started with Swiftide, head over to swiftide.rs, check us out on github, or hit us up on discord.

Fluvio support

Fluvio is a lightweight high-performance distributed data streaming system written in Rust and Web Assembly. In a production environment, data could come and go to many places at the same time. With the Fluvio loader, you can hook into a Fluvio topic and index right away.

The integration is fully configurable with Fluvio's own configuration. Here is an example:

static TOPIC_NAME: &str = "hello-rust";
static PARTITION_NUM: u32 = 0;

let loader = Fluvio::builder()
    .consumer_config_ext(
        ConsumerConfigExt::builder()
            .topic(TOPIC_NAME)
            .partition(PARTITION_NUM)
            .offset_start(fluvio::Offset::from_end(1))
            .build()?,
    )
    .build()?;

indexing::Pipeline::from_loader(loader)
    .then_in_batch(10, Embed::new(FastEmbed::try_default().unwrap()))
    .then_store_with(
        Qdrant::builder()
            .batch_size(50)
            .vector_size(384)
            .collection_name("swiftide-examples")
            .build()?,
    )
    .run()
    .await?;

Lancedb

Lancedb is a popular vector and text database that separates storage from compute. It enables a whole new class of applications where the database is embedded into the application itself. Under the hood it uses Apache Arrow and Tantivy.

Both indexing with different embedded fields and querying is supported. Note that Lancedb does not really need Sparse vectors, as it provides full text search as well. Additionally, Swiftide does not index the data.

An example:

let lancedb = LanceDB::builder()
  .uri(tempdir.child("lancedb").to_str()?)
  .vector_size(1536)
  .with_vector(EmbeddedField::Combined)
  .with_metadata(METADATA_QA_TEXT_NAME)
  .table_name("swiftide_test")
  .build()?;

// and in your indexing pipeline
indexing_pipeline.then_store_with(lancedb.clone())

// and in your query pipeline
query_pipeline.then_retrieve(lancedb.clone())

RAGAS support

Evaluating RAG pipelines is a whole topic by itself. A query pipeline can now take an evaluator, and we have build one for RAGAS to kick it off. The evaluator
can export to json, which can be imported in a python hugging face dataset and used with RAGAS.

This is great, as it will allow you to evaluate the quality of your data, indexing and querying.

A full guide is coming soon!

An example:

let ragas = evaluators::ragas::Ragas::from_prepared_questions(questions);

let pipeline = query::Pipeline::default()
    .evaluate_with(ragas.clone())
    .then_transform_query(GenerateSubquestions::from_client(context.openai.clone()))
    .then_transform_query(query_transformers::Embed::from_client(
        context.openai.clone(),
    ))
    .then_retrieve(context.qdrant.clone())
    .then_answer(Simple::from_client(context.openai.clone()));

pipeline.query_all(ragas.questions().await).await?;

let json = ragas.to_json().await;
std::fs::write("output.json", json)?;

And then in a Python notebook:

from datasets import load_dataset
dataset = load_dataset("json", data_file='output.json')

from ragas.metrics import (
    answer_relevancy,
    faithfulness,
    context_recall,
    context_precision,
)

import pandas as pd

from ragas import evaluate

result = evaluate(dataset, metrics=[answer_relevancy,faithfulness,context_recall,context_precision]).to_pandas()

# ... And create some amazing plots!

From our own experimentation, Rust feature flags are great to try out many different options fast to get data in.

What's next?

We are working hard on hybrid search in the query pipeline for both Qdrant and Lancedb, improving documentation for the query pipeline and many more improvements. If you have feedback, suggestions or need help, feel free to reach out to us on Discord or via a Github issue.

Call for contributors

There is a large list of desired features, and many more unlisted over at our issues page; ranging from great starter issues, to fun, complex challenges.

You can find the full changelog here.

To get started with Swiftide, head over to swiftide.rs or check us out on github.

Announcing Swiftide, blazing fast data pipelines for RAG

Timon Vonk — Wed, 26 Jun 2024 20:15:29 +0000

While working with other Python-based tooling, frustrations arose around performance, stability, and ease of use.

Excited to announce Swiftide, blazing fast data pipelines for Retrieval Augmented Generation written in Rust. Python bindings soon!

Check it out at https://swiftide.rs

IngestionPipeline::from_loader(FileLoader::new(".").with_extensions(&["md"]))
        .then_chunk(ChunkMarkdown::with_chunk_range(10..512))
        .then(MetadataQACode::new(openai_client.clone()))
        .then_in_batch(10, Embed::new(openai_client.clone()))
        .then_store_with(
            Qdrant::try_from_url(qdrant_url)?
                .batch_size(50)
                .vector_size(1536)
                .collection_name("swiftide-examples".to_string())
                .build()?,
        )
        .run()
        .await?;

Questions, feedback, complaints and great ideas are more than welcome in the comments <3

bosun-ai / swiftide

Fast, streaming indexing and query library for AI (RAG) applications, written in Rust

Table of Contents

Swiftide

Fast, streaming indexing and query library for AI applications, written in Rust

Read more on swiftide.rs »

<a href="https://docs.rs/swiftide/latest/swiftide/" rel="nofollow">API Docs</a>
·
<a href="https://github.com/bosun-ai/swiftide/issues/new?labels=bug&amp;template=bug_report.md">Report Bug</a>
·
<a href="https://github.com/bosun-ai/swiftide/issues/new?labels=enhancement&amp;template=feature_request.md">Request Feature</a>
·
<a href="https://discord.gg/3jjXYen9UY" rel="nofollow">Discord</a>

About The Project

Swiftide is a Rust native library for building LLM applications. Large language models are amazing, but need context to solve real problems. Swiftide allows you to ingest, transform and index large amounts of data fast, and then query that data so it it can be injected into prompts. This process is called Retrieval Augmented Generation.

With Swiftide, you can build your AI application from idea to production in a few lines of code.

While working with other Python-based tooling, frustrations arose around performance, stability, and ease of use. Thus, Swiftide was born…

View on GitHub