Forem: Kirk Crenshaw

What the Heck Are Hybrid Knowledge Bases? (And Why They Matter for LLM Apps)

Kirk Crenshaw — Fri, 25 Apr 2025 17:51:56 +0000

If you're building with LLMs and trying to give your agents or copilots real context, you've probably used a vector database.

They're great for unstructured data — PDFs, HTML, markdown, text blobs. But what happens when your data isn't just text?

Enter Hybrid Knowledge Bases.

Now available in Griptape Cloud, they let you store and retrieve structured and unstructured data — together — and query them intelligently in your apps.

🧠 So... What Is a Hybrid Knowledge Base?

A Hybrid Knowledge Base compliments a vector store by combining:

🔢 Structured data: things like location, job titles, timestamps, metadata fields
📝 Unstructured data: resumé text, emails, notes, paragraphs, docs

You can use natural language queries or programmatic ones — and get results that combine exact-match filters with vector similarity searches.

🛠️ Example Use Case: Candidate Search

You're building a recruiter assistant. You have:

Structured data: candidate name, location, years of experience
Unstructured data: resumes, LinkedIn profiles, cover letters

With a hybrid knowledge base, your app can answer:

"Which candidates are in New York and have experience in data analysis with Python?"

It will:

Filter by location == New York (structured)
Perform semantic search across profiles and resumes for "data analysis with Python" (unstructured)

📊✅ Combined results. No hacky joins. No second queries. Just clean, LLM-ready responses.

💡 Why It Matters

Most LLM apps fail when the data isn’t flat text.

Real-world knowledge is messy. It’s structured and unstructured.

And most stacks treat those as separate systems.

With Griptape Hybrid Knowledge Bases, you get:

A unified query layer
Tight integration with agents, workflows, and pipelines
Real-time, semantic + structured retrieval

Hybrid Knowledge Bases are now available in Griptape Cloud.

🙋‍♂️ What Would You Build?

Got a use case that blends structured and unstructured data?

Want to give your agents actual intelligence without cobbling multiple tools together?

Improving the dev experience for building apps that integrate up-to-date and private data with large language models

Kirk Crenshaw — Mon, 31 Mar 2025 20:25:51 +0000

We’re delighted to announce a new feature in Griptape Cloud that will improve the experience for developers building applications that integrate up-to-date and private data with large language models. Griptape Cloud has supported retrieval augmented generation applications through similarity search for some time. Check out this post for more details on existing Retrieval Augmented Generation features in Griptape Cloud > https://www.griptape.ai/blog/retrieval-augmented-generation-with-griptape-cloud

Retrievers
Today, we are adding to that capability with Griptape Cloud Retrievers. Retrievers are a fully-managed implementation of the RAG Engine within Griptape Framework, and add query modification, reranking capabilities, and the ability to apply rules to query responses. These features enable you to generate more accurate and tailored results in your RAG applications built with Griptape Cloud.

Query Modification
Query modification in RAG allows you to improve matching by transforming queries before embeddings are generated for the query and used for similarity search against the data stored in a vector store. They use techniques such as query expansion, where an LLM is used to add additional context or terms to improve each query; or Hypothetical Document Embedding (HyDE), which is the generation of hypothetical answers to a query, which are then embedded and used with the original query to enhance the query that is made against the vector store.

What is Reranking?
Reranking works by comparing the relatedness of search results returned from a vector search query to the original query and reordering the results in descending order of their relatedness. This gives a ‘reranked’ list of results that you can use in your application. Let’s explore an example of reranking with Griptape Framework to illustrate why reranking is valuable, and then we’ll move on to explain how to get started with Retrievers in Griptape Cloud. Let’s assume we asked the question “What is the capital of France?” and the results that came back from a vector search operation across our data sources were as follows:

results = [ “Hotdog", "San Francisco", "Lille", "Paris", "Rome", "Baguette", "Eiffel Tower", "French Hotdog" ]

To rerank these results, we compare the embedding for each result to the embedding for the original question “What is the capital of France?” and then order the results in descending order of relatedness. In this use-case, answering a question, we would likely use the top result, but there are other use-cases, a research agent might be one example, where we might want to take the top n results from the reranking operation to perform a secondary operation on those results.

Implementing reranking is simple with Griptape Framework. Griptape Framework supports reranking with a local rerank driver using a simple relatedness calculation, and also has the capabilities to use Cohere’s reranking model through the CohereRerankDriver. The sample code below uses the local rerank driver to rerank these results from the example query.

from griptape.artifacts import TextArtifact
from griptape.drivers.rerank.local import LocalRerankDriver

list = ["Hotdog","San Francisco","Lille","Paris","Rome","Baguette","Eiffel Tower","French Hotdog"]

artifact_list = []

for i in list:
   artifact_list.append(TextArtifact(i))

rerank_driver = LocalRerankDriver()

artifacts = rerank_driver.run("What is the capital of France?", artifact_list)

print("Reranked list:")

for artifact in artifacts:
    print("\t", artifact.value)
Reranked list:
         Paris
         Eiffel Tower
         Lille
         San Francisco
         Rome
         Baguette
         French Hotdog
         Hotdog

You can see that the reranking operation correctly identifies Paris as the best answer to the question that we posed.

The example above shows the LocalRerankDriver being used as a standalone module, but it is more commonly used within a Griptape Framework RagEngine. The code sample below shows how we might create a tool for an Agent to use where we define a RetrievalRagStage that includes a VectorStoreRetrievalRagModule and a TextChunksRerankRagModule, where we use the LocalRerankDriveras the rerank_driver.

rag_tool = RagTool(
   description="Contains information about the judgements and applications relating to legal cases",
   off_prompt=False,
   rag_engine=RagEngine(
       retrieval_stage=RetrievalRagStage(
           retrieval_modules=[
               VectorStoreRetrievalRagModule(
                   vector_store_driver=vector_store_driver,
                   query_params={"namespace": "legal documents", "top_n": 20},
               )
           ],
           rerank_module=TextChunksRerankRagModule(rerank_driver=LocalRerankDriver()),
       ),
       response_stage=ResponseRagStage(
           response_modules=[
               PromptResponseRagModule(
                   prompt_driver=OpenAiChatPromptDriver(model="gpt-4o")
               )
           ]
       ),
   ),
)

As we mentioned earlier, Retrievers are a fully-managed implementation of the RAG Engine within the Griptape Framework. So you don’t need to worry about this complexity if you’re using Griptape Cloud.

Query Response Types
Retrievers support two different response types, text chunk responses and prompts with rulesets. The Text Chunk response type is intended for use in conjunction with an LLM for Retrieval Augmented Generation use-case.

The Prompt with Rulesets response type is used to generate natural language responses to your queries directly from the Retriever without the need to pass text chunks to an LLM for response generation. As you might expect from the name of this response type, you can control the behaviour of the Retrieverer in generating natural language responses by attaching a Ruleset.

Using Retrievers in Griptape Cloud
Retrievers bring the benefits of query modification, reranking, and control over query responses to Griptape Cloud. Retrievers can rerank the results returned from multiple Knowledge Bases. This means that they are particularly valuable when combining the results from multiple different data sources where they can help ensure that your applications get the results that are most relevant to the search queries they are making.

Let’s walk through the process of setting up a Retriever on Griptape Cloud. In this example, we’re going to create a Retriever that uses the Knowledge Base that we configured when we set up the Assistant in the 'Brush Up on NVIDIA's Q4 Earnings’ sample on the Griptape Cloud console home page. If you want to learn more about that sample application, it’s covered in this blog post.

Retrievers can be found under the Libraries Navigation header in the left navigation menu in the Griptape Cloud Console. To create a Retriever, select the yellow highlighted option in the left navigation menu and then select Create Retriever on the Retrievers page.

You will then be prompted to provide the details for your new Retriever. We are going to create a Retriever for the RAG use-case that I will connect to an Assistant, so I completed the Retriever details as shown below. Once the details have been entered, click the Create button to create the Retriever.

I can then connect the new Retriever to my Assistant and use it to retrieve answers to my questions from the NVIDIA Q4 Earnings Knowledge Base.

In this example, I am using a Ruleset to guide the Assistants behavior. If you want to experiment with this, you can use my rules as inspiration for your own, or just copy them for yourself. The rules I used are as follows:

Only provide answers that you can verify using the Knowledge Base or Retriever. Check all answers against either the Knowledge Base or Retriever. If you cannot verify an answer from the Knowledge Base or Retriever, say so and decline to answer. Do not make things up that cannot be verified.
Only answer questions related to NVIDIA's Q4 2025 earnings. Decline to answer all other questions.

We hope you find the Griptape Cloud Retrievers a valuable addition to your RAG toolkit. As usual, we’re excited to hear how you put this new capability to work. Please join us in the Griptape Discord if you have any questions, or use-cases that you would like to share.

Announcing Griptape AI Framework 1.5

Kirk Crenshaw — Thu, 20 Mar 2025 19:37:37 +0000

We’re pleased to announce that Griptape Framework 1.5 is now available. The 1.5 release brings enhancements to embeddings to add support for generating image embeddings and the addition of image search via the framework’s vector store drivers. We also have updates to the default models in several drivers, support for Perplexity with a new prompt driver and web search driver, and more. Let’s head down to the skatepark and explore some of the new features added in this release.

Updated Getting Started Guide & Improved Samples
First up, we have improved the Griptape Framework getting started guide on the Framework Overview documentation page. The new guide uses the uv python dependency manager, though of course we also provide instructions for pip users, and provides a tour of the key features in the framework. We recommend that you check this out, even if you're already familiar with Griptape Framework, as it covers new features added over the last few releases. We have also updated all the code samples in the documentation to include a tab showing the output logs from running the sample, so you can see the results from each sample without having to run them yourself.

Support for Image Embeddings
Griptape Framework 1.5 includes support for generating embeddings from ImageArtifact objects, together with new embedding drivers for Amazon Bedrock with Amazon’s Titan Multimodal Embeddings G1 model, and for Voyage AI’s voyage-multimodal-3 model.

In the example below, I generate embeddings for 4 images that I generated using FLUX.2 with the Black Forest Labs extension for Griptape. After generating and saving two snowboarding images and two skateboarding images, in the code sample I use Amazon’s amazon.titan-embed-image-v1 model to calculate embeddings for these images and a simple relatedness calculation to compare the vectors that the embedding model generated.

from griptape.drivers.embedding.amazon_bedrock import AmazonBedrockTitanEmbeddingDriver
from griptape.loaders import ImageLoader
import numpy as np

def calc_relatedness(
   x, y
):  # using the same relatedness function as the localrerankdriver
   return np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y))

# create a driver for multi-modal embedding with Amazon Bedrock and Amazon Titan
multi_modal_embedding_driver = AmazonBedrockTitanEmbeddingDriver(
   model="amazon.titan-embed-image-v1"
)

# calculate embeddings for our four sample images
blue_snowboarder_embeddings = multi_modal_embedding_driver.embed(
   ImageLoader().load("images/blue_snowboarder.jpeg")
)
orange_snowboarder_embeddings = multi_modal_embedding_driver.embed(
   ImageLoader().load("images/orange_snowboarder.jpeg")
)
beach_skater_embeddings = multi_modal_embedding_driver.embed(
   ImageLoader().load("images/beach_skater.jpeg")
)
paris_skater_embeddings = multi_modal_embedding_driver.embed(
   ImageLoader().load("images/paris_skater.jpeg")
)

print(  # compare the two snowboarding images
   "blue_snowboarder vs orange_snowboarder: ",
   calc_relatedness(blue_snowboarder_embeddings, orange_snowboarder_embeddings),
)

print(  # compare a snowboarding image with a skateboarding image
   "blue_snowboarder vs beach_skater: ",
   calc_relatedness(blue_snowboarder_embeddings, beach_skater_embeddings),
)

print(  # compare the two skateboarding images
   "beach_skater vs paris_skater: ",
   calc_relatedness(beach_skater_embeddings, paris_skater_embeddings),
)

The results are that the two snowboarding images are very related with a score of greater than 0.9 (scores closer to 1 indicate higher levels of relatedness), while the comparison of the snowboarding and skateboarding images gives a score just over 0.55. Comparing the two skateboarding images generates a relatedness score of greater than 0.75, despite one being on Santa Monica beach and the other in the skatepark at the Paris Olympics.

Here are the images, together with the results for you to take a look at. I really like the results that I got with Black Forest Labs, and the results from the relatedness calculations.

In addition to the changes to embedding drivers, vector store drivers have been updated to support upserting and querying with ImageArtifact objects, and the framework’s local vector store has been updated to support persisting multi-modal entries. These changes mean that you can store the embeddings generated for the images into a vector store and then you can query this to find the top x images nearest to the embeddings for a query image. These changes enable Griptape Framework to support image-based use cases such as image similarity search.

Support for Perplexity
In this release we’ve added support for the popular AI-powered search and research engine, Perplexity, with the addition of PerplexityPromptDriver and PerplexityWebSearchDriver.

import os

from griptape.drivers.prompt.perplexity import PerplexityPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent
from griptape.tasks import PromptTask

agent = Agent(
   tasks=[
       PromptTask(
           prompt_driver=PerplexityPromptDriver(
               model="sonar-pro", api_key=os.environ["PERPLEXITY_API_KEY"]
           ),
           rules=[
               Rule("Be precise and concise"),
           ],
       )
   ],
)

agent.run("tell me about the griptape framework 1.4 release")

print(agent.output.value)

If you experiment with the PerplexityPromptDriver using the code sample above, you will notice that the responses are generated using search as well as an LLM, meaning that you get up-to-date answers from the web. In this case, the model does a great job finding some of the highlights from the last release of the framework, using the blog post about that release as a source. This makes the PerplexityPromptDriver a little different to the other prompt drivers in Griptape Framework. We are excited to see what you build with this exciting new capability.

Updates to Default Models
The default model in the AnthropicPromptDriver has been updated to claude-3-7-sonnet-latest.

If you're using Google models, the default model for the GooglePromptDriver has been updated to gemini-2.0-flash and the default model for the GoogleEmbeddingDriver has been updated to embedding-004.

For developers using Amazon Bedrock to access Anthropic’s model, the default model in the AmazonBedrockPromptDriver has been updated to anthropic.claude-3-7-sonnet-20250219-v1:0. In addition, the Amazon Titan model used for text embeddings has been updated to amazon.titan-embed-text-v2:0 and the default Amazon Titan model used in the AmazonBedrockImageGenerationDriver has been updated to amazon.titan-image-generator-v2:0.

As usual, you can continue to use previous generation models by setting the model keyword argument to the model that you wish to use when creating an instance of each driver.

Improved Control over Tool Use
If you’ve having trouble getting less powerful LLMs to function correctly when calling tools, the 1.5 release allows you to set reflect_on_tool_use to False and have the LLM return tool outputs directly. If you're using a less powerful LLM, you should consider using this setting so that you can make tool calls yourself rather than having the LLM coordinate them. In the very simple code sample below I run three tasks where I provide a DateTimeTool. The code is commented to provide details on the behavior that you should expect, and you can see the results from running this sample below the sample.

from griptape.tasks import PromptTask
from griptape.tools import DateTimeTool
from griptape.artifacts.list_artifact import ListArtifact

# When disabling `reflect_on_tool_use`, Task results will be returned as a ListArtifact.
# Each item in the ListArtifact will be the result of a single tool execution.

date_task = PromptTask(
    tools=[DateTimeTool()],
    reflect_on_tool_use=False,
)
results = date_task.run("How many days until it's 2026?")
# This will fail as the model will not reflect and figure out that an additional tool run is needed to calculate the date delta

if isinstance(results, ListArtifact):
    for result in results:
        print("Simple prompt output without reflection:", result)

date_task = PromptTask(
    tools=[DateTimeTool()],
    reflect_on_tool_use=False,
)
results = date_task.run("How many days from 12:44 on March 19th 2025 to Jan 1st 2026?")
# This will succeed as the prompt is more specific and the model can calculate the date delta using a single tool invocation
# Note that the output is the raw output from the tool

if isinstance(results, ListArtifact):
    for result in results:
        print("Detailed prompt output without reflection:", result)

date_task = PromptTask(
    tools=[DateTimeTool()],
    reflect_on_tool_use=True,
)
results = date_task.run("How many days until it's 2026?")
# This will succeed as the model will invoke the tool multiple times to get the current date & calculate the date delta
# Note that the output is more descriptive as the model has reflected on the final tool inovcation to generate the response

print("Simple prompt output with reflection:", results)
Simple prompt output without reflection: 2025-03-19 12:47:10.882934

Detailed prompt output without reflection: 287 days, 11:16:00

Simple prompt output with reflection: There are 287 days until it's 2026.

‍How to get started
Griptape Framework 1.5 is available now and you can download it with uv, poetry, pip or another Python package manager of your choice. As usual, we would love to hear your feedback on these changes, together with ideas and suggestions for future imrpovements to the framework. If you want to ask any questions about the other features in this release or discuss your image embedding use-cases, please head over to the Griptape Discord.