Forem: shailendra khade

Build a Simple Local Pune Travel AI with FAISS + Ollama LLM - POC

shailendra khade — Fri, 06 Feb 2026 07:40:45 +0000

Ever wondered how to create your own local AI assistant for city tours or travel recommendations? In this POC, we build a Pune Grand Tour AI using FAISS vector database for embeddings and Ollama LLM for generating answers. No Docker, no cloud costs — just local Python and embeddings.

Let’s go step by step.

🔹Purpose of this POC

Local AI Assistant: A mini ChatGPT specialized for Pune tourism.

Quick Retrieval: Use embeddings for fast similarity search over a curated dataset.

Cost-efficient: No cloud vector DB required — FAISS runs entirely locally.

Hands-on AI Exploration: Learn practical AI pipeline: embeddings → vector DB → LLM.

🔹 Why FAISS?

FAISS (Facebook AI Similarity Search) is a high-performance library for:

Storing vector embeddings.

Performing fast similarity search (like nearest neighbor search).

Working locally, without needing cloud infrastructure.

Key point: FAISS is ideal for projects like ours because all Pune data can fit in memory, and retrieval is very fast.

🔹 Dataset

We used a simple text dataset (pune_places_chunks.txt) containing Pune's:

Historical forts
Monuments
Museums
Tourist spots

Each line or chunk represents one document. Example:

[PLACE] Shaniwar Wada
Shaniwar Wada is a historic fort located in Pune, built in 1732 by Peshwa Bajirao I.
It served as the administrative center of the Maratha Empire.


[PLACE] Aga Khan Palace
The Aga Khan Palace is known for its association with Mahatma Gandhi and history

🔹 Step 1: Create Embeddings & FAISS Vector Store

Python script: ingest.py

from langchain_community.document_loaders import TextLoader
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS


# 1 Load processed Pune data
loader = TextLoader("../data/processed/pune_places_chunks.txt")
documents = loader.load()


# 2 Create embeddings
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)


# 3 Create FAISS vector store
vectorstore = FAISS.from_documents(documents, embeddings)


# 4 Save vector DB locally
vectorstore.save_local("../embeddings/pune_faiss")


print("Pune embeddings created successfully")

Run:

python ingest.py

Expected output:

Pune embeddings created successfully

Explanation:

TextLoader loads our text chunks.

HuggingFaceEmbeddings converts each document into vector representation.

FAISS.from_documents builds a searchable vector store.

save_local persists the FAISS DB for later retrieval.

🔹Step 2: Query FAISS with Ollama LLM

Python script: chat.py

from langchain_community.vectorstores import FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_ollama import OllamaLLM


# 1 Load embeddings
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)
print("Embeddings loaded")


# 2 Load FAISS DB (allow pickle deserialization)
vectordb = FAISS.load_local(
    "../embeddings/pune_faiss",
    embeddings,
    allow_dangerous_deserialization=True
)
print("FAISS Vector DB loaded")


# 3 Ask a question
question = "Tell me famous places to visit in Pune"
docs = vectordb.similarity_search(question, k=3)


if len(docs) == 0:
    print("No documents retrieved. Check embeddings folder.")
    exit(1)


context = "\n".join([d.page_content for d in docs])
print(f"Retrieved docs count: {len(docs)}")
print("Context preview (first 300 chars):")
print(context[:300])


# 4 Initialize Ollama LLM
llm = OllamaLLM(model="llama3")
print("Ollama LLM loaded")


# 5 Build prompt
prompt = f"""
You are a Pune travel guide AI.
Answer using only the context below.


Context:
{context}


Question:
{question}
"""


# 6 Generate AI response
response = llm.invoke(prompt)


print("\nPune AI says:\n")
print(response)

Run:

python chat.py

Sample output:

Embeddings loaded
FAISS Vector DB loaded
Retrieved docs count: 3
Context preview (first 300 chars):
[PLACE] Shaniwar Wada
Shaniwar Wada is a historic fort located in Pune, built in 1732...
Ollama LLM loaded


Pune AI says:


Pune is famous for Shaniwar Wada, Sinhagad Fort, Aga Khan Palace, and Dagdusheth Ganpati Temple.

Explanation:

similarity_search retrieves the top 3 most relevant documents.

Context is combined and sent as a prompt to Ollama LLM.

LLM generates human-like answers based on retrieved Pune knowledge.

🔹 Step 3: Make it Interactive with Streamlit

We can upgrade this to a fully interactive web app (app.py):

import streamlit as st
from langchain_community.vectorstores import FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_ollama import OllamaLLM


st.title("Pune Grand Tour AI")
st.write("Ask about Pune's forts, monuments, and travel tips!")


@st.cache_resource
def load_vectorstore():
    embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
    return FAISS.load_local("../embeddings/pune_faiss", embeddings, allow_dangerous_deserialization=True)


@st.cache_resource
def load_llm():
    return OllamaLLM(model="llama3")


vectordb = load_vectorstore()
llm = load_llm()


question = st.text_input("Ask a question about Pune:")


if question:
    docs = vectordb.similarity_search(question, k=3)
    if not docs:
        st.warning("No documents found!")
    else:
        context = "\n".join([d.page_content for d in docs])
        prompt = f"You are a Pune travel guide AI.\n\nContext:\n{context}\n\nQuestion:\n{question}"
        response = llm.invoke(prompt)
        st.subheader("Retrieved Context")
        st.text(context[:500] + ("..." if len(context) > 500 else ""))
        st.subheader("AI Answer")
        st.write(response)

Run:

pip install streamlit
streamlit run app.py

Result:
A browser UI opens where you can ask any Pune-related question, and the AI responds interactively.

🔹 Key Benefits of this POC

Fully Local — No cloud or Docker dependency.
Fast Retrieval — FAISS provides instant similarity search.
Context-aware AI — Ollama LLM answers based on curated Pune knowledge.
Expandable — Add more documents, images, or travel tips.
Interactive UI — Streamlit allows anyone to use the AI easily.

🔹 Common Issues & Fixes

🔹 Use Cases

Local city guide AI for tourism apps
Educational assistant for geography/history lessons
Personal knowledge assistant for any curated dataset
Prototype for RAG (Retrieval-Augmented Generation) projects

🔹 Conclusion
With FAISS + Ollama LLM + Streamlit, you can build fast, local, context-aware AI assistants without relying on cloud services or Docker.

This Pune AI POC demonstrates how a specialized knowledge base can power a chatbot capable of giving accurate, context-specific answers.

Building a Containerized GenAI Chatbot with Docker, Ollama, FastAPI & ChromaDB

shailendra khade — Tue, 03 Feb 2026 15:17:03 +0000

Introduction

Modern AI systems are not just Python scripts — they are distributed systems involving:

LLM engines
APIs
UI applications
Vector databases
Container orchestration

In this blog, I share how I built a GenAI Chatbot using Docker-based microservices, similar to real-world AI platforms, and the real DevOps issues I faced while building it.

System Architecture (AI Architect View)
User (Browser)
|
v
[ Streamlit UI ]
|
v
[ FastAPI Backend ]
|
+----> [ Ollama LLM Engine ]
|
+----> [ ChromaDB Vector Database ]

Why Microservices?

Separating AI components into services improves scalability, maintainability, and fault isolation.

Project Structure
genai-docker-project/
│
├── backend/ # FastAPI + AI logic
├── ui/ # Streamlit UI
├── docker-compose.yml
├── README.md

Docker Compose (Core of System)

version: "3.8"

services:

  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"

  backend:
    build: ./backend
    ports:
      - "8000:8000"
    depends_on:
      - ollama
      - chroma

  chroma:
    image: chromadb/chroma
    ports:
      - "8001:8000"

  ui:
    build: ./ui
    ports:
      - "8501:8501"
    depends_on:
      - backend

Backend (FastAPI + Ollama Integration)

from fastapi import FastAPI
import requests

app = FastAPI()

OLLAMA_URL = "http://ollama:11434/api/generate"

@app.post("/ask")
def ask_ai(question: str):
    payload = {"model": "mistral", "prompt": question}
    response = requests.post(OLLAMA_URL, json=payload)
    return response.json()

UI (Streamlit)

import streamlit as st
import requests

st.title("GenAI Chatbot")

question = st.text_input("Ask a question:")

if st.button("Ask AI"):
    res = requests.post("http://backend:8000/ask", params={"question": question})
    st.write(res.json())

Real Errors & DevOps Solutions
Error 1: Docker Permission Denied
permission denied while trying to connect to the Docker daemon socket
Root Cause
User was not part of the docker group.
Solution
sudo usermod -aG docker $USER newgrp docker

Error 2: Port Already in Use (11434)
failed to bind host port 0.0.0.0:11434: address already in use
Root Cause
Ollama was already running on host via Snap:
ps -ef | grep ollama
Output:
/snap/ollama/.../ollama serve
Solution:
sudo snap stop ollama sudo snap disable ollama
(or change Docker port)
`ports:

"21434:11434"`

Error 3: Model Not Found in Ollama
"model 'mistral' not found"
Root Cause
Ollama runtime was running, but model was not downloaded inside the container.

Solution
docker exec -it genai-docker-project-ollama-1 bash
ollama pull mistral

Error 4: Container Networking Issues
Problem
Backend could not connect to Ollama.

Root Cause
Using localhost instead of container DNS name.
Fix
OLLAMA_URL = "http://ollama:11434/api/generate"

Key Learnings (AI + DevOps)

AI systems are distributed systems.
Docker is essential for reproducible ML environments.
LLM platforms require careful networking and resource management.
MLOps is the bridge between DevOps and AI.

Autonomous DevOps AI Agent (FastAPI + Ollama)

shailendra khade — Thu, 22 Jan 2026 16:05:39 +0000

Building an Autonomous DevOps AI Agent using FastAPI and Ollama
Introduction

DevOps is no longer just about CI/CD pipelines and monitoring dashboards.
With the rise of Large Language Models (LLMs), DevOps is evolving into AI-powered automation (AIOps).

In this project, I built an Autonomous DevOps AI Agent using:

Ollama (Local LLM – Mistral)
FastAPI (API Layer)
Linux System Tools
Git & Deployment Scripts

This AI agent can analyze system metrics, interact with Git repositories, trigger deployments, and reason about DevOps tasks using an LLM.

This post explains the architecture, implementation, challenges, and learnings.

What is an Autonomous DevOps AI Agent?

An Autonomous DevOps AI Agent is a system that:
Collects infrastructure data (CPU, RAM, logs, etc.)
Executes DevOps operations (Git, scripts, deployments)
Uses AI (LLM) to analyze and respond intelligently
Exposes everything via REST APIs

In simple words:

AI + DevOps + Automation = Autonomous Agent

Architecture

Components:

FastAPI – API Gateway and Controller
Ollama – Local LLM runtime (Mistral model)
Linux Tools – System metrics and logs
Git & Shell Scripts – DevOps automation

Step 1: Install Ollama and Model

Install Ollama on Linux:

curl -fsSL https://ollama.com/install.sh | sh

Pull an LLM model:

ollama pull mistral

Verify:

ollama list

Step 2: Setup Python Environment

Create virtual environment:

python3 -m venv ai-env source ai-env/bin/activate

Install dependencies:

pip install fastapi uvicorn requests psutil

Step 3: Project Structure
devops-ai-agent/
│
├── agent.py # AI reasoning (Ollama)
├── system.py # System metrics
├── git_ops.py # Git automation
├── deploy.py # Deployment scripts
├── main.py # FastAPI entry point

Step 4: Core Implementation
system.py – System Metrics

import psutil

def get_system_info():
    return {
        "cpu": psutil.cpu_percent(),
        "memory": psutil.virtual_memory().percent,
        "disk": psutil.disk_usage("/").percent
    }

agent.py – AI Brain (Ollama Integration)

import requests

OLLAMA_URL = "http://localhost:11434/api/generate"

def ai_think(prompt):
    payload = {
        "model": "mistral",
        "prompt": prompt,
        "stream": False
    }
    response = requests.post(OLLAMA_URL, json=payload)
    return response.json()["response"]

git_ops.py – Git Automation

import os

def clone_repo(repo_url):
    os.system(f"git clone {repo_url}")
    return {"status": "repo cloned", "repo": repo_url}

deploy.py – Deployment Trigger

import os

def deploy_app():
    os.system("echo Deploying application...")
    return {"status": "deployment triggered"}

main.py – FastAPI Autonomous Agent

from fastapi import FastAPI
from system import get_system_info
from git_ops import clone_repo
from deploy import deploy_app
from agent import ai_think

app = FastAPI()

@app.get("/")
def home():
    return {"message": "Autonomous DevOps AI Agent Running"}

@app.get("/system")
def system():
    return get_system_info()

@app.post("/chat")
def chat(prompt: str):
    return {"ai_response": ai_think(prompt)}

@app.post("/git/clone")
def git_clone(repo_url: str):
    return clone_repo(repo_url)

@app.post("/deploy")
def deploy():
    return deploy_app()

@app.post("/agent")
def autonomous_agent(task: str):
    reasoning = ai_think(f"You are DevOps AI. Task: {task}")
    return {"task": task, "ai_decision": reasoning}

Step 5: Run the Agent
uvicorn main:app --host 0.0.0.0 --port 7000

Example Use Cases
1 Check System Health
GET /system

2 Ask AI about DevOps
POST /chat?prompt=Explain Kubernetes architecture

3 Clone Git Repository
POST /git/clone?repo_url=https://github.com/user/repo.git

4 Autonomous DevOps Task
POST /agent?task=Check server health and suggest actions

Key Learnings

Ollama can act as a local LLM inference server.
FastAPI is ideal for building AI-powered microservices.
DevOps automation can be enhanced using LLM reasoning.
This architecture is similar to real-world AIOps systems.

Ollama + FastAPI API, Building My Own AI API Using Ollama and FastAPI on a Linux VM

shailendra khade — Thu, 22 Jan 2026 10:54:23 +0000

Introduction

Large Language Models (LLMs) like ChatGPT are usually accessed via cloud APIs.
But what if we could run our own AI model locally and expose it as an API?

In this project, I built a custom AI API using Ollama + FastAPI on a Linux virtual machine.
This API exposes LLM capabilities via REST endpoints, similar to how real-world AI microservices work.

This post covers the architecture, implementation, challenges, and learnings.

What is Ollama?

Ollama is a tool that allows us to run LLM models like Mistral, Llama, and Gemma locally.

It provides a local API endpoint:

http://localhost:11434

We can wrap this with FastAPI to build our own AI service.

Architecture

Step 1: Install Ollama on Linux VM
curl -fsSL https://ollama.com/install.sh | sh

Verify installation:
ollama --version

Step 2: Pull an LLM Model
ollama pull mistral

Check available models:
ollama list

Step 3: Setup Python Environment
python3 -m venv ai-env source ai-env/bin/activate pip install fastapi uvicorn requests

Step 4: Build Ollama API using FastAPI
Create file ollama_api.py:

from fastapi import FastAPI
import requests

app = FastAPI()

OLLAMA_URL = "http://localhost:11434/api/generate"

@app.get("/")
def home():
    return {"message": "Ollama AI API is running"}

@app.get("/health")
def health():
    return {"status": "UP", "model": "mistral"}

@app.post("/chat")
def chat(prompt: str):
    payload = {
        "model": "mistral",
        "prompt": prompt,
        "stream": False
    }
    response = requests.post(OLLAMA_URL, json=payload)
    return response.json()

Step 5: Run the API Server
uvicorn ollama_api:app --host 0.0.0.0 --port 9000

Step 6: Test the AI API
Test with curl
curl -X POST "http://localhost:9000/chat?prompt=Explain%20DevOps"

Test from Host Machine
curl -X POST "http://<VM-IP>:9000/chat?prompt=explain AI"

Challenges Faced
1 Networking Issues in VM

0.0.0.0 cannot be used as a browser address.
Required using the VM IP address to access the API.

2 HTTPS vs HTTP

Browser attempted HTTPS while API was running on HTTP.
Solved by explicitly using HTTP.

3 Python PEP 668 Error

System Python was protected.
Solved using Python virtual environment (venv).

Key Learnings

Ollama can be used to run LLMs locally.
FastAPI is a great framework to expose AI models as microservices.
Virtual environments are essential in modern Linux systems.
Building APIs on VMs helps understand real DevOps workflows.
This architecture is similar to production AI services.

Building My First REST API on Linux VM using FastAPI (with Real DevOps Troubleshooting)

shailendra khade — Thu, 22 Jan 2026 08:10:13 +0000

Introduction

As a DevOps & AI enthusiast, I wanted to understand how APIs actually work at the system level — not just theory, but hands-on on a Linux VM.

So I decided to build a simple REST API using FastAPI on a Linux virtual machine and expose it to my host system.
During this journey, I faced real-world networking and Python environment issues — and solved them like a DevOps engineer.

This post documents my complete journey, steps, and learnings.

What is an API?
An API (Application Programming Interface) allows applications or users to communicate with a server using HTTP requests.

Example:

Client → sends request
Server → processes logic
API → returns JSON response

Architecture

Simple Architecture

Step 1: Setting up FastAPI on Linux VM
Install Python and pip
sudo apt update sudo apt install python3 python3-pip -y
Create Virtual Environment (Best Practice)
python3 -m venv api-env source api-env/bin/activate

Install FastAPI and Uvicorn
pip install fastapi uvicorn

Step 2: Create a Simple API
Create main.py

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def home():
    return {"message": "Hello from Linux VM API"}

@app.get("/health")
def health():
    return {"status": "UP"}

Step 3: Run the API Server
uvicorn main:app --host 0.0.0.0 --port 8000

Step 4: Test API
From VM terminal
curl http://localhost:8000

Output:
{"message":"Hello from Linux VM API"}

From Host Machine (Desktop Browser)
http://192.168.204.128:8000/

Real Issues I Faced (and Solved)
1 SSL / HTTPS Error

_Problem:
Browser tried HTTPS but API was HTTP.

Solution:
Use HTTP explicitly._

2 0.0.0.0Access Issue

Problem:
Tried accessing API using 0.0.0.0.

Reality:
0.0.0.0 is a bind address, not a client address.

Solution:
Use VM IP or localhost.

3 Python PEP 668 Error
externally-managed-environment
Reason:
System Python is protected in modern Linux.

Solution:
Use virtual environment (venv).

Key Learnings

0.0.0.0 is for server binding, not browser access.
Always use virtual environments in Linux.
Networking issues are common in VM-based setups.
FastAPI is powerful and developer-friendly.
Real DevOps work is about debugging, not just coding.

Linux for Beginners: A Clear, Simple, and Practical Introduction

shailendra khade — Tue, 20 Jan 2026 16:12:19 +0000

Introduction

Linux is one of the most powerful and widely used operating systems in the world.
From servers, cloud platforms, DevOps pipelines, and cybersecurity to embedded systems — Linux is everywhere.

In this article, we will understand Linux in the simplest way possible:
What it is, how it works, and why it matters for beginners.

What is Linux?

Linux is an open-source operating system based on the Unix architecture.
Being open-source means:

Its code is free to view, modify, and distribute

Anyone can contribute

Multiple distributions (Ubuntu, RedHat, CentOS, Fedora, etc.) exist

Linux controls the system resources like:

CPU
Memory
Storage
Processes
Network

Why is Linux Important?

Linux powers over 90% of cloud servers and is the backbone of:

AWS, Azure, GCP
Kubernetes & Docker
DevOps CI/CD
Cybersecurity & Ethical hacking
High-performance computing

If you want a career in Cloud, DevOps, SRE, Platform Engineering, or Cybersecurity — Linux is mandatory.

Key Components of Linux
1️⃣ Kernel

The core of Linux.
It manages hardware, processes, memory, and system resources.

2️⃣ Shell

A command-line interface that interacts with the OS.
Popular shells:

Bash
Zsh
Cshell

3️⃣File System

Linux follows a hierarchical structure:

/
├── bin
├── etc
├── home
├── var
└── usr

4️⃣ Package Manager

Helps install, update, and remove software.

apt → Ubuntu/Debian
yum/dnf → RHEL/CentOS/Fedora

Basic Linux Commands for Beginners

Here are the most essential commands:

Purpose	Command
List files	`ls`
Change directory	`cd`
Create directory	`mkdir`
View file content	`cat`
Check current path	`pwd`
Copy file	`cp source dest`
Move file	`mv source dest`
Remove file	`rm`
System info	`uname -a`
View running processes	`ps -aux`

Why Developers Prefer Linux

Stable
Secure
Highly customizable
Better performance in servers
Free and open-source
Strong community support

Where to Practice Linux?

You can use:

Ubuntu (VMware/VirtualBox)
WSL (Windows Subsystem for Linux)
Cloud providers (AWS EC2, GCP VM)
Online terminals like Katacoda / Webminal

Conclusion

Linux is not just an operating system — it is a foundation for modern technology.
Learning Linux opens doors to Cloud, DevOps, SRE, Automation, and more.

Start small, practice commands daily, and explore the power of open-source.