Forem: Ayush

Getting started with LLM and LangChain

Ayush — Wed, 13 Mar 2024 14:02:46 +0000

Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) and Artificial Intelligence (AI). However, building and training LLMs can be a complex and resource-intensive process, requiring significant computational power and expertise. That's where langchain comes in—LangChain provides a range of tools and features that address various complexities with building applications using LLMs, such as prompt crafting, response structuring, and model integration.

In this blog post, we will go over some 101-level information on LLMs, and then walk through a demo on how to build a very simple LLM-based application to query for information. Let's dive into the world of Large Language Models and explore the ways in which LangChain is making them accessible and affordable for everyone.

First, what are Large Language Models?

Large language models (LLMs) are a class of artificial intelligence (AI) algorithms that have been trained on massive amounts of text data, for example, text from books, articles, websites, or other written content. LLMs use this data to learn how to predict the next word or words that will occur in a sequence. Given some initial words in a sentence or paragraph, an LLM can generate a natural-sounding continuation of the text.

There are several different types of LLMs, each with its own unique architecture and training methodologies. Some of the most popular types of LLMs include:

Recurrent Neural Networks (RNNs): These neural networks are designed to handle sequential data, meaning language models with context, such as long sentences or paragraphs.
Transformers: Transformers were developed in 2017 by OpenAI models and have since become among the most popular LLM designs. Transformers are trained using a self-supervised learning learning method that enables them to analyze and encode vast amounts of text data while still maintaining the context of the information.
Generative Pre-trained Transformer 3 (GPT-3): This is one of the most advanced LLMs currently available, with 175 billion parameters as of 2021. GPT-3 is a neural network-based language model that is incredibly powerful at generating and understanding human language.

Large language models have many applications, including natural language search engines, text-to-speech systems, chatbots and automated customer service, and language translation software. As LLMs continue to improve, they are becoming increasingly useful tools for businesses in a wide range of industries.

LLMs are already starting to change the way we interact with written content, and as their capabilities continue to grow, businesses and other organizations are starting to realize the benefits of using these systems to streamline their operations. For those interested in AI’s future, large language models are an exciting area to follow, and LangChain is a great platform to learn more and build new AI skills.

What is LangChain?

LangChain offers a comprehensive framework for creating applications that leverage the power of large language models. With LangChain, users are able to develop chatbots, personal assistants, and other AI-powered tools that can perform a range of tasks such as summarization, analysis, Q&A generation, code comprehension, API interaction, and more.

The LangChain framework comes equipped with pre-designed chains for efficiently executing complex tasks. Users can customize existing chains or build entirely new ones by utilizing the vast library of components that LangChain offers.

The LangChain framework has numerous components that can be used to develop custom applications. These components include:

Prompts: Prompt templates play a crucial role in streamlining user input to ensure effective communication with a natural language model. Its primary function is to format user input in a fashion that matches the expectations of the language model. These templates are crucial in providing context and specifying inputs to help the model perform the intended task efficiently. As an illustration, a chatbot prompt template may incorporate the user's name alongside their question to enhance contextual relevance and improve the performance of the language model.

Chains: Chains are pre-designed language model components that accomplish specific tasks. These are pre-designed sets of components that can process input and generate output. Chains can be used as-is or customized to meet the specific requirements of a project. Custom chains can be designed by combining multiple components, creating powerful and flexible applications.

Agents: Agents are responsible for language model training and communication. Agents train models by determining the optimal input for the model and optimizing its output. They are also responsible for receiving user input, processing it, and providing appropriate output from the model. The LangChain Agent handles different input/output formats, language models, and APIs.

LLMs: Language models (LLMs) are at the core of LangChain's functionality. They analyze, summarize, generate Q&A, and understand code. LangChain enables users to create their own models or use pre-trained models from the LangChain library. The flexibility of LangChain allows users to train models on any type of data or use pre-trained models for any type of use case.

Callbacks: Callbacks enable developers to enhance the functionality of the LangChain components by adding customized algorithms. The callbacks can be used to modify the output generated by the language model or modify the input before input pre-processing.

Memory: Memory allows users to store data across multiple sessions, enabling LangChain to provide a more personalized experience to the users. Memory can be used to store user preferences, chat history, and interaction patterns, allowing LangChain to learn from previous interactions and adapt to user's needs.

Output Parsers: Output parsers take responsibility for organizing the response output that Language Model (LLM) generates. The parser performs some functions, such as the elimination of unwanted or irrelevant content, addition of supplementary information, or alteration of response format. As a significant component of the LangChain framework, output parsers play an essential role in maintaining outcome relevance, accuracy, and structure. It ensures that the LLM's responses are more decipherable, thus improving ease of interpretation and increases in practicality of such responses.

One of the most significant benefits that LangChain offers is the ability to access LLMs models without expending high development costs or needing extensive technical expertise. LangChain’s suite of tools deliver valuable assistance for prompt crafting, response structuring, model integration and other programming-related requirements.

Now that we’re familiar with the concepts, let’s walk through a simple example to better understand how various components work together. We’ll build a service that checks whether Wikipedia contains cat recordings (who doesn’t love a cat video?):

Installation and Setup

Let’s first configure the environment. Assuming Python is already installed in your working environment, the next step is to install the LangChain library using pip:

pip install langchain

As we are going to use the OpenAI’s language models, we will install OpenAI SDK as well:

pip install openai

Next, as a critical step in connecting to OpenAI's API, it is necessary to set the OPENAI_API_KEY as an environment variable. First, sign in to your OpenAI account and navigate to account settings. Then, head over to the View API Keys option and generate a secret key, which you will need to copy. Within your Python script, use the os module to access the dictionary of available environment variables and set the "OPENAI_API_KEY" to the secret API key you copied earlier.

import os
os.environ["OPENAI_API_KEY"] = "your-api-key-here"

Get predictions
Import the LLM wrapper:

from langchain.llms import OpenAI

We can then initialize the wrapper with any arguments.

gen_llm = OpenAI(openai_api_key=”…”, verbose=False)

Initialize the APIChain to query from the API endpoint:

from langchain.chains.api import open_meteo_docs

chain_new = APIChain.from_llm_and_api_docs(gen_llm, “https://cat-fact.herokuapp.com/facts/", verbose=True)

Now we’re ready to ask a question to the LLM:

chain_new.run(‘Does wikipedia have a cat recording?’)

You should get an output like this:

> Entering new APIChain chain…
https://cat-fact.herokuapp.com/facts/


[{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008800aac31001185ed07",”user”:”58e007480aac31001185ecef”,”text”:”Wikipedia has a recording of a cat meowing, because why not?”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–06T21:20:03.505Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008630aac31001185ed01",”user”:”58e007480aac31001185ecef”,”text”:”When cats grimace, they are usually \”taste-scenting.\” They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–07T21:20:02.903Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e00a090aac31001185ed16",”user”:”58e007480aac31001185ecef”,”text”:”Cats make more than 100 different sounds whereas dogs make around 10.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–11T21:20:03.745Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e009390aac31001185ed10",”user”:”58e007480aac31001185ecef”,”text”:”Most cats are lactose intolerant, and milk can cause painful stomach cramps and diarrhea. It’s best to forego the milk and just give your cat the standard: clean, cool drinking water.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–04T21:20:02.979Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008780aac31001185ed05",”user”:”58e007480aac31001185ecef”,”text”:”Owning a cat can reduce the risk of stroke and heart attack by a third.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–29T20:20:03.844Z”,”deleted”:false,”used”:false}]

> Finished chain.

Out[4]:
‘ Yes, Wikipedia has a recording of a cat meowing.’

Congratulations! Your basic setup of an LLM-based application using LangChain is now working successfully.

Building AI-powered search using LangChain and Milvus

Ayush — Mon, 11 Dec 2023 06:11:32 +0000

This blog is co-authored by Ayush Pandey, Senior Software Engineer and Amit Kaushal, Software Manager at Intuit.

Artificial Intelligence (AI) has revolutionized the way we interact with technology, and one of the most significant applications of AI is in search. With the help of AI, search tools can surface more accurate and relevant results to users. In this blog, we will discuss how to build an AI-powered search engine using LangChain and Milvus.

Before we dive into the demo, let’s talk through some of the concepts and tools involved.

What is Langchain?

LangChain is a framework for developing applications powered by language models. Use cases include applications for document question answering, building conversational interfaces for database interactions, and much more. We believe that the most powerful and differentiated applications will not only leverage a language model, but will also be:
Data-aware: connect a language model to other sources of data
Agentic: allow a language model to interact with its environment

LangChain provides the modular components used to build such applications, which can be used standalone or combined for more complexity.

What is Milvus?

Milvus was created in 2019 with a singular goal: to store, index, and manage massive embedding vectors generated by deep neural networks and other machine learning (ML) models.

As a database specifically designed to handle queries over input vectors, it is capable of indexing vectors on a trillion scale. Unlike existing relational databases, which mainly deal with structured data following a pre-defined pattern, Milvus is designed from the bottom-up to handle embedding vectors converted from unstructured data.

Vector embeddings: why the hype?

Vector embeddings are a powerful tool for developers working with natural language processing (NLP) and ML applications. Vector embeddings are a way of representing words or phrases as vectors in a high-dimensional space, where each dimension represents a different feature of the word or phrase. This allows developers to perform complex operations on text data, such as sentiment analysis, text classification, and machine translation.

Let’s go over a simple explainer of semantic features: scientists sort different types of animals in the world into categories based on certain characteristics. For example, Birds are a type of warm-blooded vertebrate that are adapted to fly. Based on these features, we created word coordinates to represent animals based on their Type and Domestication score. These scores are called “semantic features,” which capture parts of the meanings of each word. Now that the words have corresponding numerical values, we can then plot these words as points on a graph, where the x-axis represents Type, and the y-axis represents Domestication score.

We can add new words to the plot based on their meanings. For example, where should the words "Lions" and "Parrots" go? How about "Whales"? Or "Snakes"?

There are also several libraries and tools available for developers who want to work with vector embeddings. Some popular libraries include Gensim, TensorFlow, and PyTorch. These libraries provide pre-trained models for word2vec and GloVe, as well as tools for training custom models on specific datasets.

Demo: Using a similarity search for asking questions from a wikipedia

First, let’s go through some prerequisites.

Install LangChain and Milvus on your local system:

! python -m pip install --upgrade pymilvus langchain openai tiktoken

Then, import required modules:

from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Milvus
from langchain.document_loaders import TextLoader
from langchain.document_loaders import WebBaseLoader

Next, import your OpenAI API key:

import os
import getpass

os.environ['OPENAI_API_KEY'] = "your-openai-api-key"

Then, load in a Wikipedia document (here we’re grabbing the article for Intuit QuickBooks) using WebBaseLoader client, and split it into chunks:

loader = WebBaseLoader([
   "https://en.wikipedia.org/wiki/QuickBooks",
])

docs = loader.load()
# Split the documents into smaller chunks
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(docs)

Afterwards, use OpenAIEmbeddings and store everything in a Milvus vector database.

from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vector_db = Milvus.from_documents(
   docs,
   embeddings,
   connection_args={"host": "HostName", "port": "19530"},
)

It’s time to try semantic searching! Let’s ask a question using LangChain and Milvus:

query = "What is quickbooks?"
docs = vector_db.similarity_search(query)
docs[0].page_content

Output:

'Retrieved from "https://en.wikipedia.org/w/index.php?title=QuickBooks&oldid=1155606425"\nCategories: Accounting softwareIntuit softwareHidden categories: CS1 maint: url-statusArticles with short descriptionShort description is different from WikidataUse mdy dates from March 2019Articles containing potentially dated statements from May 2014All articles containing potentially dated statements\n\n\n\n\n\n\n This page was last edited on 18 May 2023, at 23:04\xa0(UTC).\nText is available under the Creative Commons Attribution-ShareAlike License 3.0;\nadditional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.\n\n\nPrivacy policy\nAbout Wikipedia\nDisclaimers\nContact Wikipedia\nMobile view\nDevelopers\nStatistics\nCookie statement\n\n\n\n\n\n\n\n\n\n\n\nToggle limited content width'
The results above are decent, but need quite a lot of formatting help. Let’s try using load_qa_with_sources_chain to ask the questions instead for a cleaner output:

from langchain.chains.qa_with_sources import load_qa_with_sources_chain
from langchain.llms import OpenAI

chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)
query = "What is quickbooks?"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)

Output:
{'intermediate_steps': [' No relevant text.', ' QuickBooks is an accounting software package developed and marketed by Intuit. First introduced in 1983, QuickBooks products are geared mainly toward small and medium-sized businesses and offer on-premises accounting applications as well as cloud-based versions that accept business payments, manage and pay bills, and payroll functions.', " Intuit also offers a cloud service called QuickBooks Online (QBO). The user pays a monthly subscription fee rather than an upfront fee and accesses the software exclusively through a secure logon via a Web browser. QuickBooks Online is supported on Chrome, Firefox, Internet Explorer 10, Safari 6.1, and also accessible via Chrome on Android and Safari on iOS 7. Quickbooks Online offers integration with other third-party software and financial services, such as banks, payroll companies, and expense management software. QuickBooks desktop also supports a migration feature where customers can migrate their desktop data from a pro or prem SKU's to Quickbooks Online.", ' QuickBooks - Wikipedia \nInitial release, Subsequent releases, QuickBooks Online, QuickBooks Point of Sale, Add-on programs.'], 'output_text': ' QuickBooks is an accounting software package developed and marketed by Intuit. It offers on-premises accounting applications as well as cloud-based versions that accept business payments, manage and pay bills, and payroll functions. QuickBooks Online is a cloud service that offers integration with other third-party software and financial services.\nSOURCES: https://en.wikipedia.org/wiki/QuickBooks'}

Other search-related use cases using LangChain and Milvus:

E-commerce search engine: the language model can be trained on product descriptions and reviews, and the data can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant products based on user queries.
Image search engine: the language model can be trained on image captions and tags, and the images can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant images based on user queries.
Video search engine: the language model can be trained on video titles and descriptions, and the videos can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant videos based on user queries.

By following the simple steps we’ve outlined here, developers can use LangChain and Milvus to build search engines for various use cases ranging from a simple document search to applications in e-commerce, image, and video search. We hope this was a helpful starter guide, please leave a comment if you have any further questions!

*References and further reading: *