Forem: Mael Kerichard

Creating a Knowledge-Based Chatbot with OpenAI Embedding API, Pinecone, and Langchain.js

Mael Kerichard — Sun, 09 Apr 2023 11:10:54 +0000

In this tutorial, we'll walk you through the process of creating a knowledge-based chatbot using the OpenAI Embedding API, Pinecone as a vector database, and langchain.js as a large language model (LLM) framework. This chatbot will be able to accept URLs, which it will use to gain knowledge from and provide answers based on that knowledge.

Introduction

To create our knowledge-based chatbot, we will use the following technologies:

Pinecone: A vector database that helps us store and query embeddings.
OpenAI Embedding API: An API that provides embeddings for text inputs.
langchain.js: A JavaScript library for LLM frameworks that makes it easier to work with Pinecone and OpenAI.

What is a Vector Database?

Source: pinecone.io

Vector databases are used to store and query vectors efficiently. They allow you to search for similar vectors based on their similarity in a high-dimensional space. In this tutorial, we will use Pinecone as our vector database.

Embeddings are dense vector representations of data, such as text, images, or audio. In our case, we'll be using text embeddings generated by the OpenAI Embedding API. These embeddings help us find semantically similar content in the vector space.

Prerequisites

Before we dive into the details, make sure you have the following:

Familiarity with JavaScript, TypeScript, and Svelte
Basic understanding of web scraping and natural language processing (NLP)
Node.js and NPM installed on your system
An OpenAI API key
A Pinecone API key

Overview

Our chatbot will consist of the following components:

Web scraper: to extract content from given URLs
Text splitter: to split the acquired text into chunks for processing
Embedding generator: to create embeddings from the text chunks using OpenAI's Embedding API
Pinecone vector store: to store and retrieve embeddings efficiently
Langchain LLM: to provide question-answering capabilities based on the embeddings

Let's dive into each component.

1. Web Scraper

To build the web scraper, we will use Puppeteer, a Node.js library that provides a high-level API to control headless Chrome or Chromium browsers.

async function scrape_researchr_page(url: string, browser: Browser): Promise<string> {
    const page = await browser.newPage();
    await page.setJavaScriptEnabled(false);
    await page.goto(url);

    const element = await page.waitForSelector('#content > div.row > div', {
        timeout: 100
    });

    if (!element) {
        throw new Error('Could not find element');
    }

    // keep only content elements (like p, h1, h2, h3, h4, h5, h6, li, blockquote, pre, code, table, dl, div)
    await element.evaluate((element) => {
        const elements = element.querySelectorAll(
            '*:not(p, h1, h2, h3, h4, h5, h6, li, blockquote, pre, code, table, dl, div)'
        );
        for (let i = 0; i < elements.length; i++) {
            elements[i].parentNode?.removeChild(elements[i]);
        }
    });

    const html_of_element = await element.evaluate((element) => element.innerHTML);

    return turndownService.turndown(html_of_element);
}

Of course, you will need to adapt the scrapper for your pages.

At the end of the function, we use the turndown library. It is a library that allows to convert HTML into Markdown. We have added that step because it will be later easier to split and GPT have a better understanding of Markdown than HTML. It is also a lighter syntax than HTML, meaning less tokens, thus cheaper API calls.

2. Text Splitter

The next step is to split the acquired text into smaller chunks for processing. We will use the MarkdownTextSplitter class from the langchain/text_splitter package. This class takes a chunkSize and chunkOverlap parameter to control the size and overlap of the generated text chunks.

const textSplitter = new MarkdownTextSplitter({
    chunkSize: 1000,
    chunkOverlap: 20
});
const chunks = await textSplitter.splitText(markdowns.join('\n\n'));

Ideally, we want one information per chunk. If you have very structured markdown files, one chunk could be equal to one subsection. In our case, the markdown comes from HTML and is badly structured, we then really on fixed chunk size, making our knowledge base less reliable (one information could be split into two chunks).

3. Embedding Generator

We will use OpenAI's Embedding API to generate embeddings for our text chunks. We will create an instance of the OpenAIEmbeddings class from the langchain/embeddings package and pass it our OpenAI API key via the ENV OPENAI_API_KEY=<your_openai_api_key>.

const embeddingModel = new OpenAIEmbeddings({ maxConcurrency: 5 });

4. Pinecone Vector Store

Pinecone is a vector database designed for efficient storage and retrieval of high-dimensional vectors. We will use the PineconeStore class from the langchain/vectorstores package to store our generated embeddings.

We first initialize the client and connect to the index created on Pinecone dashboard (the vectors have 1536 dimensions).

import {PineconeClient} from "@pinecone-database/pinecone";

const client = new PineconeClient();
await client.init({
    apiKey: PINECONE_API_KEY,
    environment: PINECONE_ENVIRONMENT,
});

export const pineconeIndex = client.Index(PINECONE_INDEX);

This code will get embeddings from the OpenAI API and store them in Pinecone.

5. Langchain

To provide question-answering capabilities based on our embeddings, we will use the VectorDBQAChain class from the langchain/chains package. This class combines a Large Language Model (LLM) with a vector database to answer questions based on the content in the vector database.

In our example, we use the ChatOpenAI class from the langchain/chat_models package as our LLM, and the PineconeStore instance as our vector database.

const model = new ChatOpenAI({ temperature: 0.9, openAIApiKey: OPENAI_API_KEY, modelName: 'gpt-3.5-turbo' });

const chain = VectorDBQAChain.fromLLM(model, vectorStore, {
k: 5,
returnSourceDocuments: true
});

With these components in place, we can now create a SvelteKit endpoint to handle user requests. The POST request handler will receive the user's query, call the VectorDBQAChain, and return the chatbot's response along with the source document (if available).

export const POST = (async ({ request }) => {
    const { text } = await request.json();
    if (!text) {
        throw error(400, 'Missing text');
    }
    if (text.length > 200) {
        throw error(400, 'Text too long');
    }

    try {
        const response = await chain.call({ query: text });
        const { text: responseText, sourceDocuments } = response;

        return json({
            text: responseText,
            sources: sourceDocuments
        });
    } catch (e) {
        console.log(e);
        throw error(500, 'Internal Server Error');
    }
}) satisfies RequestHandler;

Conclusion

We've demonstrated how to build a knowledge-based chatbot using the OpenAI Embedding API, Pinecone as a vector database, and Langchain. This chatbot can acquire knowledge from given URLs and answer user queries based on that knowledge.

By combining web scraping, text processing, embeddings, vector databases, and LLMs, you can create powerful chatbots that can learn and provide useful answers based on the content they consume.

Find the reference code on GitHub.

Next.js 13 App directory + Prisma

Mael Kerichard — Sun, 05 Feb 2023 20:06:13 +0000

Introduction

Next.js 13 introduced the app directory with new features and conventions. This feature is still in beta, but we already have a nice overview of what the future will be like.

See https://beta.nextjs.org/docs/app-directory-roadmap for the roadmap.

Installation

To automatically create a new Next.js project using the app directory:

npx create-next-app@latest --experimental-app

After executing this command, you will see the new project architecture :

app/
├─ globals.css
├─ head.tsx
├─ layout.tsx
├─ page.module.css
├─ page.tsx

Where is the index.tsx file ?

To create a route for /profile, you may be used to create a profile.tsx file or /profile/index.tsx. With the app directory, you now must create a new folder with a page.tsx file.

For example, the route /profile would be created as following :

app/
├─ globals.css
├─ head.tsx
├─ layout.tsx
├─ page.module.css
├─ page.tsx
├─ profile/
│  ├─ page.tsx

Introducing Prisma

Prisma is a next-generation Node.js and TypeScript ORM for PostgreSQL, MySQL, SQL Server, SQLite, MongoDB, and CockroachDB. It provides type-safety, automated migrations, and an intuitive data model.

Before, we may use tRPC or API routes to call Prisma. But now, with the app directory and server components, Prisma can be called directly inside the component.

Adding Prisma to the project

npm install prisma --save-dev
npx prisma init --datasource-provider sqlite

This creates a new prisma directory with your Prisma schema file and configures SQLite as your database. You're now ready to model your data and create your database with some tables.

The Prisma schema provides an intuitive way to model data. Add the following models to your schema.prisma file:

model Todo {
  id        Int      @id @default(autoincrement())
  title     String
  completed Boolean  @default(false)
  createdAt DateTime @default(now())
}

We now need to migrate the database. Run the following :

npx prisma migrate dev --name init

To generate the typescript types, run :

npx prisma generate

Seed the database

Create a prisma/seed.ts file and add the following :

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

async function main() {
  await prisma.todo.create({
    data: {
      title: "Learn Next.js",
    },
  });
  await prisma.todo.create({
    data: {
      title: "Learn Prisma",
    },
  });
  await prisma.todo.create({
    data: {
      title: "Learn GraphQL",
    },
  });
}

main()
  .then(async () => {
    await prisma.$disconnect();
  })
  .catch(async (e) => {
    console.error(e);
    await prisma.$disconnect();
    process.exit(1);
  });

And in you package.json file :

{
  "name": "nextjs13-app-prisma",
  "version": "0.1.0",
  "private": true,
...
  "prisma": {
    "seed": "ts-node --compiler-options {\"module\":\"CommonJS\"} prisma/seed.ts"
  },
...
}

Now, when you run the npx prisma seed command, it will seed your database.

Link Prisma and Next

Create a utils/prisma.ts file :

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient()

export default prisma;

This will ensure that only one Prisma instance is running.

By default, all components are Server Components. Therefore, you can use async code in every one of them.

For example, to fetch all the to-dos in the home page, use :

import prisma from "@/utils/prisma";

export default async function Home() {
  const todos = await prisma.todo.findMany();
  return (
    <main>
      <h1 className="font-bold">Todos</h1>
      <ul>
        {todos.map((todo) => (
          <li key={todo.id}>{todo.title}</li>
        ))}
      </ul>
    </main>
  );
}

(I added Tailwind for styling, it is of course not required)

Go to localhost:3000 and you will see all your to-dos :

Data mutation

To mutate data, it is a bit more complicated. Because we need Client Component for user interactivity and because we can not have server code inside Client Component, we need to set up an API route.

Create the pages/api/todo.ts file :

import { NextApiRequest, NextApiResponse } from "next";
import prisma from "@/utils/prisma";

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  if (req.method === "PUT") {
    await prisma.todo.update({
      where: {
        id: parseInt(req.body.id),
      },
      data: {
        completed: req.body.completed,
      },
    });
    res.status(200).json({ message: "Updated" });
  } else {
    // 404
    res.status(404).json({ message: "Not found" });
  }
}

In this API endpoint, we get the to-do id and state in the body and update the corresponding to-do.

We will now create a Todo Client Component.

In a app/todo.tsx file (remember that it will not create a /todo route) :

"use client";

import { Todo } from "@prisma/client";
import { useRouter } from "next/navigation";

export default function TodoComponent({ todo }: { todo: Todo }) {
  const router = useRouter();
  const update = async (todo: Todo) => {
    await fetch(`/api/todo`, {
      method: "PUT",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        completed: !todo.completed,
        id: todo.id,
      }),
    });
    router.refresh();
  };

  return (
    <li key={todo.id} className="space-x-4">
      <input
        onChange={() => update(todo)}
        type="checkbox"
        checked={todo.completed}
      />
      {todo.title}
    </li>
  );
}

The UI is updated using the router.refresh() method.
Note the "use client" at the top, it is required to let Next know that it is a Client Component.

We can then update the Home page :

import prisma from "@/utils/prisma";
import Todo from "@/app/todo";

export default async function Home() {
  const todos = await prisma.todo.findMany();
  return (
    <main>
      <h1 className="font-bold">Todos</h1>
      <ul>
        {todos.map((todo) => (
          <Todo todo={todo}></Todo>
        ))}
      </ul>
    </main>
  );
}

Enjoy your simple but working to-do in Next.js 13 using Prisma.

Find the code for the article on GitHub.