Forem: qudrat ullah

Build Your First AI Search in 30 Minutes: A Complete RAG Tutorial

qudrat ullah — Thu, 16 Apr 2026 20:15:29 +0000

What is RAG and Why Should You Care?

RAG stands for Retrieval-Augmented Generation. Think of it like having a super-smart assistant who can quickly search through your documents and then give you intelligent answers based on what they found.

Imagine you have thousands of company documents, and instead of spending hours searching through them manually, you could just ask questions like "What's our vacation policy?" and get instant, accurate answers. That's exactly what RAG does.

The "Retrieval" part finds relevant information, and the "Generation" part creates human-like responses using that information. It's like combining Google search with ChatGPT, but for your own data.

How RAG Works: The Simple Explanation

RAG works in three simple steps:

Store: Break your documents into small chunks and convert them into numbers (called embeddings) that computers understand
Search: When you ask a question, find the most relevant chunks from your stored data
Generate: Feed those relevant chunks to an AI model (like GPT) to create a natural answer

Think of it like organizing a massive library. Instead of browsing every book, you have a librarian who knows exactly where to find information and can summarize it for you.

Building Your First RAG System

Let's build a simple RAG system that can answer questions about your documents. We'll use Python and some helpful libraries.

Step 1: Install Required Libraries

pip install langchain openai chromadb sentence-transformers

Step 2: Prepare Your Documents

# documents.py
documents = [
    "Our company offers 20 days of paid vacation per year. Employees can carry over up to 5 unused days to the next year.",
    "The office hours are 9 AM to 6 PM, Monday through Friday. Remote work is allowed up to 3 days per week.",
    "Health insurance covers 100% of employee premiums and 80% of family member premiums.",
    "The annual performance review happens in December. Salary increases are effective from January."
]

Step 3: Create the RAG System

# rag_system.py
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Chroma
from langchain.llms import OpenAI
from langchain.chains import RetrievalQA
import os

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "your-api-key-here"

class SimpleRAG:
    def __init__(self, documents):
        # Split documents into smaller chunks
        text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)
        texts = text_splitter.create_documents(documents)

        # Create embeddings (convert text to numbers)
        embeddings = HuggingFaceEmbeddings()

        # Store in vector database
        self.vectorstore = Chroma.from_documents(texts, embeddings)

        # Set up the question-answering chain
        llm = OpenAI(temperature=0)
        self.qa_chain = RetrievalQA.from_chain_type(
            llm=llm,
            chain_type="stuff",
            retriever=self.vectorstore.as_retriever()
        )

    def ask_question(self, question):
        return self.qa_chain.run(question)

# Usage
from documents import documents

rag = SimpleRAG(documents)
answer = rag.ask_question("How many vacation days do I get?")
print(answer)

Step 4: Test Your RAG System

# test_rag.py
questions = [
    "How many vacation days do I get?",
    "Can I work from home?",
    "When do performance reviews happen?",
    "What does health insurance cover?"
]

for question in questions:
    print(f"Q: {question}")
    print(f"A: {rag.ask_question(question)}")
    print("-" * 50)

Making It Better: Pro Tips

1. Chunk Your Data Smartly

Don't just split text randomly. Split by paragraphs, sentences, or logical sections:

# Better text splitting
from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    separators=["\n\n", "\n", " ", ""]
)

2. Use Better Embeddings

For better search results, use more powerful embedding models:

from langchain.embeddings import OpenAIEmbeddings

# More accurate but costs money
embeddings = OpenAIEmbeddings()

# Free alternative that works well
from sentence_transformers import SentenceTransformer
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)

3. Add Memory for Conversations

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory(memory_key="chat_history")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=vectorstore.as_retriever(),
    memory=memory
)

Common Mistakes to Avoid

Mistake 1: Using chunks that are too big or too small

Too big: AI gets confused with too much information
Too small: Important context gets lost
Sweet spot: 500-1500 characters per chunk

Mistake 2: Not handling edge cases

def safe_ask_question(self, question):
    if not question.strip():
        return "Please ask a valid question."

    try:
        return self.qa_chain.run(question)
    except Exception as e:
        return f"Sorry, I couldn't process your question: {str(e)}"

Mistake 3: Ignoring document quality

Clean your documents first
Remove unnecessary formatting
Fix typos and inconsistencies

Scaling Up: Next Steps

Once you have a basic RAG system working, you can:

Add more document types: PDFs, Word docs, web pages
Improve the UI: Build a web interface with Streamlit or Flask
Use better databases: PostgreSQL with pgvector for production
Add authentication: Secure your system for multiple users
Monitor performance: Track which questions work well and which don't

Key Takeaways

RAG combines search and AI generation to answer questions about your documents
You need three main components: document storage, similarity search, and text generation
Start simple with basic libraries, then improve gradually
Good document chunking is crucial for accurate results
Test with real questions your users would ask
Always handle errors gracefully in production systems

RAG isn't magic, but it's incredibly powerful when done right. Start with this simple example, experiment with your own documents, and gradually add more features. Before you know it, you'll have built an AI assistant that actually knows about your specific domain.

The best part? This is just the beginning. RAG technology is evolving rapidly, and mastering the basics now will set you up for even more exciting developments ahead.

About the Author

Hi, I'm Qudrat Ullah, an Engineering Lead with 10+ years building scalable systems across fintech, media, and enterprise. I write about Node.js, cloud infrastructure, AI, and engineering leadership.

Find me online: LinkedIn · qudratullah.net

If you found this useful, share it with a fellow engineer or drop your thoughts in the comments.

Originally published at qudratullah.net.

Generative AI vs Agentic AI

qudrat ullah — Sun, 25 Jan 2026 15:32:52 +0000

Generative AI vs Agentic AI

Artificial Intelligence is moving fast, and two terms are showing up everywhere:

Generative AI
Agentic AI

They sound similar, but they serve very different purposes.

This article explains the difference in a simple, practical way
no buzzwords, no heavy theory.

🤖 What Is Generative AI?

Generative AI creates content.

You give it a prompt, and it generates an output such as text, images, or code.

Think of Generative AI as a very smart content creator.

It responds to your request, and then it stops.

Common examples

ChatGPT writing text
DALL·E generating images
GitHub Copilot suggesting code
AI summarising documents
AI translating languages

Simple example

You ask:

Write a job description for a frontend developer.

Generative AI:

Generates the text
Returns the response
Ends the interaction

✅ Task completed

❌ No follow-up

❌ No action taken

⚙️ What Is Agentic AI?

Agentic AI doesn’t just generate responses, it takes action.

Instead of only answering, it can:

Understand a goal
Break it into steps
Use tools
Make decisions
Execute tasks
Repeat until the goal is achieved

Think of Agentic AI as a digital worker.

Simple example

You say:

Find family-friendly events in London this weekend and email me the best five.

An Agentic AI might:

Search multiple event websites
Open and analyse pages
Extract dates, prices, and locations
Filter family-friendly events
Rank the best options
Send you an email
Store results for future use

This is not just AI responding — this is AI working toward a goal.

🔑 The Key Difference

Generative AI answers questions.
Agentic AI completes goals.

🧩 Side-by-Side Comparison

Feature	Generative AI	Agentic AI
Generates text or images	✅ Yes	✅ Yes
Makes decisions	❌ No	✅ Yes
Uses tools or APIs	❌ No	✅ Yes
Executes actions	❌ No	✅ Yes
Works autonomously	❌ No	✅ Yes
Has goals	❌ No	✅ Yes

🧠 Everyday Analogy

Generative AI is like a skilled writer.

You ask a question, they write an answer, and the task ends.

Agentic AI is like a personal assistant.

You give a goal, they figure out how to achieve it, and they get it done.

👨‍💻 Why This Matters for Developers

If you’re building:

Chatbots or writing tools → Generative AI
Content generation platforms → Generative AI
Automation systems → Agentic AI
AI assistants or workers → Agentic AI
End-to-end workflows → Agentic AI

Most real-world business value is shifting toward agentic systems.

🚀 Final Takeaway

Generative AI creates content
Agentic AI achieves goals
One responds
The other acts

If Generative AI is the brain,
Agentic AI is the brain plus hands.

🧠 One Last Question for You

We’ve moved very quickly from:

AI that generates answers
to
AI that takes action
The real question now is:

Where should AI stop acting on our behalf?

Should AI agents be fully autonomous?
Or should humans always stay in the loop?
Where do you draw the line between assistant and decision-maker?

👇

I’d love to hear your thoughts in the comments.

How to Fix Chromium on Vercel: A Complete Guide to Solving the “libnspr4.so” Error

qudrat ullah — Fri, 23 Jan 2026 14:26:47 +0000

If you're deploying a Next.js app with Playwright and Chromium to Vercel, you've probably hit this error:

/tmp/chromium: error while loading shared libraries: libnspr4.so: cannot open shared object file: No such file or directory

It works perfectly on your local machine but fails on Vercel. This is one of the most frustrating deployment issues developers face when working with headless browsers in serverless environments.

After analyzing 50+ GitHub issues and testing multiple solutions, I discovered the root cause and the complete fix. Here's everything you need to know.

Why It Works Locally But Fails on Vercel

The fundamental difference between your local environment and Vercel's serverless environment causes this issue.

On your local machine:

System libraries like libnspr4.so and libnss3.so are pre-installed
You have full file system access
Complete Node.js environment with all dependencies

On Vercel serverless:

Minimal Linux container with no system libraries
Read-only file system except for the /tmp directory
Environment variables must exist before modules load

The critical issue: The sparticuz/chromium package checks for AWS_LAMBDA_JS_RUNTIME when the module imports, but if you set it in your code, it happens after the module is already loaded. This timing mismatch breaks everything.

The Five Problems and Their Solutions

Problem 1: Wrong Package Choice

Many developers try to use the full Playwright package, which doesn't work on Vercel.

Don't use:

"playwright": "^1.39.0"

This package exceeds Vercel's 50MB limit and tries to download the browser at runtime, which fails in serverless environments.

Use instead:

{
  "dependencies": {
    "playwright-core": "1.39.0",
    "@sparticuz/chromium": "^131.0.0"
  }
}

Why this works: playwright-core is only about 5MB, and the sparticuz chromium package is about 40MB. Together they stay under Vercel's 50MB limit, and the chromium package provides a pre-bundled Chromium optimized for serverless environments.

Problem 2: Environment Variable Timing

This is where most developers get stuck. The timing of when environment variables are set matters.

The problem:

When you import the package, it immediately checks for AWS_LAMBDA_JS_RUNTIME. But if you set it in your code like this:

import chromiumPack from "@sparticuz/chromium";
process.env.AWS_LAMBDA_JS_RUNTIME = "nodejs22.x";

The module has already loaded and checked for the variable. It's too late.

The solution:

Set the environment variable in the Vercel Dashboard, not just in your code.

Here's how:

Go to Vercel Dashboard
Select your project
Go to Settings → Environment Variables
Add a new variable: AWS_LAMBDA_JS_RUNTIME with value nodejs22.x
Apply to Production, Preview, and Development environments
Redeploy your application

Why the Dashboard matters: Environment variables set in the Vercel Dashboard are available before any modules load, which is exactly when the chromium package needs them.

Problem 3: Libraries Extracted But Not Found (The Critical Fix)

This was the breakthrough that solved the issue. Even when libraries are extracted correctly, Chromium can't find them.

The problem:

Libraries are extracted to /tmp/libnspr4.so and /tmp/libnss3.so
Chromium executable is in /tmp/chromium
But Chromium can't find the libraries because the system doesn't know where to look

The solution:

Set LD_LIBRARY_PATH to the executable directory before launching the browser.

Here's the code:

const executablePath = await chromiumPack.executablePath();
const execDir = path.dirname(executablePath);

// CRITICAL: Set LD_LIBRARY_PATH so Chromium can find libraries
process.env.LD_LIBRARY_PATH = execDir;

Why this works: LD_LIBRARY_PATH tells the Linux system loader where to find shared libraries. By setting it to the same directory where the Chromium executable and libraries are located, Chromium can successfully load the required libraries.

This was the missing piece that most solutions don't mention. Without this, even with the correct environment variable, Chromium will fail to find the libraries.

Problem 4: Old Package Version

Using an outdated version of the chromium package can cause compatibility issues with Node.js 22.x.

Don't use:

"@sparticuz/chromium": "119.0.2"

Use:

"@sparticuz/chromium": "^131.0.0"

Version 131.0.0 has better support for Node.js 22.x and includes fixes for common serverless deployment issues.

Problem 5: Browser Freezing

Some developers report that the browser freezes after creating a new page, even when everything else is configured correctly.

The problem: Browser freezes after "Creating new page" message appears.

The solution:

Disable graphics mode before launching the browser:

if (typeof chromiumPack.setGraphicsMode === 'function') {
  chromiumPack.setGraphicsMode(false);
}

Why: Serverless environments don't have GPU support. Disabling graphics mode prevents the browser from trying to use GPU features that aren't available, which causes freezing.

Complete Working Solution

Here's the complete implementation that works on Vercel:

1. Package.json Configuration

{
  "dependencies": {
    "@sparticuz/chromium": "^131.0.0",
    "playwright-core": "1.39.0"
  },
  "engines": {
    "node": "22.x"
  }
}

2. Next.js Configuration

In your next.config.ts file:

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  serverExternalPackages: [
    'playwright-core',
    '@sparticuz/chromium',
  ],
};

export default nextConfig;

This prevents Next.js from bundling these packages, which is essential for them to work correctly in the serverless environment.

3. Browser Launch Code

Create a file lib/browser.ts with this code:

import { chromium, Browser } from "playwright-core";
import chromiumPack from "@sparticuz/chromium";
import * as path from "path";

let browserInstance: Browser | null = null;

export async function getBrowser(): Promise<Browser> {
  if (browserInstance) return browserInstance;

  const isVercel = !!(process.env.VERCEL || process.env.VERCEL_ENV);

  if (isVercel) {
    // Set runtime (fallback if not in Dashboard)
    if (!process.env.AWS_LAMBDA_JS_RUNTIME) {
      process.env.AWS_LAMBDA_JS_RUNTIME = "nodejs22.x";
    }

    // Set graphics mode to prevent freezing
    if (typeof chromiumPack.setGraphicsMode === 'function') {
      chromiumPack.setGraphicsMode(false);
    }

    // Get executable and set library path
    const executablePath = await chromiumPack.executablePath();
    const execDir = path.dirname(executablePath);

    // CRITICAL: Set LD_LIBRARY_PATH
    process.env.LD_LIBRARY_PATH = execDir;

    browserInstance = await chromium.launch({
      args: chromiumPack.args,
      executablePath,
      headless: true,
    });
  } else {
    // Local development
    browserInstance = await chromium.launch({
      args: ["--no-sandbox"],
      headless: true,
    });
  }

  return browserInstance;
}

4. API Route Configuration

In your API route file (e.g., app/api/screenshot/route.ts):

import { NextRequest, NextResponse } from "next/server";
import { getBrowser } from "@/lib/browser";

export const runtime = "nodejs";
export const maxDuration = 300;
export const dynamic = "force-dynamic";

export async function POST(request: NextRequest) {
  const browser = await getBrowser();
  // Your screenshot code here
}

The runtime = "nodejs" ensures the function runs in the Node.js runtime, and maxDuration = 300 gives you 5 minutes for screenshot operations (requires Vercel Pro plan).

5. Vercel Dashboard Settings

Before deploying, configure these settings in your Vercel Dashboard:

Required settings:

Environment Variable: Add AWS_LAMBDA_JS_RUNTIME with value nodejs22.x
Disable Fluid Compute: Go to Settings → Functions → Fluid Compute → Turn OFF
Pro Plan: Required for 300-second timeouts (Hobby plan only allows 10 seconds)

The Three Critical Fixes

You need all three of these fixes for Chromium to work on Vercel:

Correct packages: Use playwright-core and sparticuz chromium version 131.0.0 or higher
Environment variable: Set AWS_LAMBDA_JS_RUNTIME=nodejs22.x in Vercel Dashboard
Library path: Set LD_LIBRARY_PATH to the executable directory in your code

Missing any one of these will cause the deployment to fail. They work together:

The environment variable tells the package which runtime to use
The library path tells Chromium where to find the extracted libraries
The correct packages ensure everything fits within Vercel's limits

Quick Troubleshooting Checklist

If you're still experiencing issues, check each of these:

[ ] AWS_LAMBDA_JS_RUNTIME=nodejs22.x is set in Vercel Dashboard (not just in code)
[ ] LD_LIBRARY_PATH is set in code before browser launch
[ ] Using playwright-core (not the full playwright package)
[ ] Using sparticuz chromium version 131.0.0 or higher
[ ] setGraphicsMode(false) is called before launching the browser
[ ] Fluid Compute is disabled in Vercel settings
[ ] You're on Vercel Pro plan for extended timeouts
[ ] serverExternalPackages is configured in next.config.ts

Why This Solution Works

This solution addresses all the fundamental differences between local and serverless environments:

Package choice: playwright-core and the chromium package are designed for serverless and stay within size limits
Environment variable: Setting it in the Dashboard ensures it's available when modules load
Library path: LD_LIBRARY_PATH tells the system where to find shared libraries in the minimal serverless environment

The critical insight: Both the environment variable (for runtime detection) and the library path (for finding libraries) are required. Most solutions only mention one or the other, but you need both.

Conclusion

Deploying Chromium on Vercel requires three essential components:

The right packages (playwright-core and the chromium package)
Environment variable configured in Vercel Dashboard
Library path set in your code

Once you have all three configured correctly, your screenshot application will work reliably on Vercel's serverless infrastructure. The key is understanding that serverless environments are fundamentally different from local development, and each difference requires a specific solution.

This solution was discovered after analyzing 50+ GitHub issues and testing multiple approaches. The breakthrough was realizing that LD_LIBRARY_PATH must be set to the executable directory, which most documentation doesn't mention.

Resources

If you found this helpful, feel free to share your experience or ask questions in the comments below!