I Replaced a $200/mo AI Stack with OpenClaw + Free Models. Here's the Exact Setup (and Why Security Almost Killed It)

Dhaivat Jambudia — Fri, 03 Apr 2026 14:10:22 +0000

I spent 2 weeks building a setup with OpenClaw, free/cheap models for the boring stuff, and only routing the hard problems to expensive models. The result? Our AI bill went from ~$200/mo to around $10/mo. And honestly, the quality for 90% of tasks is the same.

But this is the part few people talks about on twitter the security side of OpenClaw nearly burnt us. I'll get into that too because if you're a founder or CTO thinking about deploying this, you need to hear the ugly parts.

Lets get into it.

What is OpenClaw, and Why Should You Care

If you haven't heard of OpenClaw yet... where have you been? Its the fastest growing open source project in GitHub history. Over 163k stars. The project started as a personal AI assistant by Peter Steinberger (yes, the iOS dev legend) and exploded because it does something no other tool does well — it gives you a persistent AI agent that lives in your messaging apps and actually does things on your behalf.

WhatsApp, Telegram, Slack, Discord, email — OpenClaw connects to all of them through a single gateway. Its not a chatbot. Its an agent that can run shell commands, browse the web, manage your calendar, read and write files, and more. Think of it like having a junior employee that never sleeps, never complains, and costs pennies per hour.

Quick Context
OpenClaw is model-agnostic. You can plug in Claude, GPT, Gemini, or run completely free local models through Ollama. This is the key that makes the cost optimization possible.

For founders and CTOs, heres why it matters: you can automate 80% of your operational busywork without building custom software. No Python scripts, no Zapier chains, no hiring a developer to glue APIs together. You write a SOUL.md config file, connect your channels, and you're live.

The Core Idea: Not Every Task Needs a $75/M-token Model

This is the thing that took me embarrassingly long to realize. When someone emails us saying "hey can you send me the latest invoice?", your AI doesn't need Claude Opus to understand that. A 7B parameter model can handle it. When someone fills a form and you need to parse it into structured data — again, small model territory.

// openclaw.json — the config that saved us $$$/mo
{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen3:32b",
        "thinking": "anthropic/claude-sonnet-4-20250514",
        "fallbacks": [
          "openai/gpt-4.1-mini",
          "google/gemini-2.5-flash"
        ]
      }
    }
  }
}

What We Actually Automate (With Examples)

1. Email Reply Drafts — Free Model

Our support inbox gets maybe 60-80 emails a day. Most of them are variants of the same 15 questions. We wrote a skill that reads incoming emails, matches them against our FAQ knowledge base, and drafts a reply. The human just reviews and hits send.

Model used: Qwen3 32B (local, $0). For templated email replies, this model is more then adequate. It follows instructions well, keeps the tone professional, and doesn't hallucinate company policies because we feed it the exact docs.

# skills/email-drafter/SKILL.md
name: email-reply-drafter
trigger: new email in support inbox
model_override: ollama/qwen3:32b

# Steps:
1. Read incoming email
2. Search FAQ knowledge base for matching topics
3. Draft reply using company voice guidelines
4. Send draft to #email-review Slack channel
5. Wait for human approval before sending

2. Form Processing — Free Model

We have clients who fill out onboarding forms. The data comes in messy — sometimes PDF, sometimes Google Form, sometimes literally a photo of a handwritten form (yes, in 2026). The OpenClaw skill extracts the data, structures it, and pushes it to our CRM.

3. Code Reviews & Bug Fixes — Expensive Model (Sub-Agent)

This is where it gets interesting. When our agent encounters a coding task — someone reports a bug, or we need to generate a script — it spawns a sub-agent that uses Claude Sonnet 4 or routes to Claude Code.

Why not use the free model here? Because I tried, and the results were... lets just say, not production-ready. Qwen 32B can write a for loop fine. But ask it to debug a race condition in an async Node.js service and it falls apart. The bigger models just get it in ways that smaller models don't.

// Sub-agent config for coding tasks
{
  "skills": {
    "code-reviewer": {
      "model_override": "anthropic/claude-sonnet-4-20250514",
      "tools": ["shell", "browser", "file-edit"],
      "sandbox": true
    }
  }
}

You can also pipe coding tasks directly to Claude Code or OpenAI Codex as external tools. OpenClaw's tool system lets you call any CLI tool, so you literally just wrap claude-code or codex as a skill and the agent delegates to them when needed.

Now the important part: Security

OpenClaw's security track record is... not great. And I say this as someone who genuinely loves the project. The speed of adoption outpaced the security hardening by a huge margin. which is user's mistake not openclaw's.

What We Actually Did to Lock It Down

After reading the Cisco blog and the Microsoft post, I spent a full weekend hardening our setup. Heres the non-negotiable stuff:

1. Docker isolation. Our OpenClaw instance runs in a container with no access to the host filesystem except for a single mounted volume for the workspace. The agent physically cannot touch anything outside its sandbox.

2.Dedicated credentials. The agent has its own email account, its own API keys, its own everything. If it gets compromised, the blast radius is limited. Its not sharing my personal Google account or our company's AWS root credentials.

3.No third-party skills. Zero. We write all our skills in-house. I don't care how cool a ClawHub skill looks — its not going on our machine until the ecosystem has proper code review and signing. Maybe in 6 months.

4.Network segmentation. The container runs on an isolated network. It can reach the model APIs and our internal services, but nothing else. No random outbound connections to god-knows-where.

5.Human-in-the-loop for destructive actions. Any action that sends an email, modifies a file, or runs a command requires approval in Slack. The agent proposes, the human approves. No autonomous destructive operations.

The Bottom Line: Is It Worth It?

Absolutely Yes, You need technical chops. This is not a "click three buttons and you're done" setup. You need to understand Docker, networking, API keys, model capabilities, and security basics. If your team doesn't have someone who can set this up and maintain it, pay for a managed solution instead.

Want help setting this up for your team?
I've helped people deploy this exact architecture over the past month. If your burning money on AI API bills and want to cut costs without losing quality, reach out. I do a free 30-minute audit call where we look at your current setup and identify where free models can replace expensive ones.

DM me on Twitter/X or on LinkedIn

I Built a RAG System to Chat With Newton's Entire Wikipedia

Dhaivat Jambudia — Thu, 02 Apr 2026 16:29:02 +0000

Most RAG tutorials just say "chunk your PDF and call OpenAI". I wanted to build something more real — a proper pipeline that actually ingests, cleans, embeds, and serves knowledge from Isaac Newton's Wikipedia page end to end.

The result is Newton LLM. You can now ask things like "What are Newton's contributions in Calculus?" and get proper answers with sources instead of made-up stuff.
Here's how I actually built it and what I learned.
The Problem With Most RAG Demos
Every YouTube RAG tutorial follows the same boring steps: load PDF, split into chunks, put in vector store, done.
But nobody talks about the real issues:

How do you keep the data fresh when the source changes?
How do you clean messy web data before embedding?
How do you separate the ingestion part from the serving part?
How do you make the whole thing actually deployable?

Newton LLM tries to solve these. Its not just a notebook — its a small system.
Architecture Overview
The system has two main layers:

Data Ingestion Layer (the offline part)

Source → Airflow → MongoDB
I pull data from Wikipedia about Newton — his life, physics, math, optics etc.
Apache Airflow runs the whole ETL pipeline through a DAG. It fetches, cleans, and transforms the raw content. No random scripts or cron jobs. Airflow handles retries, scheduling and monitoring.
MongoDB stores the cleaned documents. This is my "source of truth" before anything gets embedded.
Why not embed straight from Wikipedia? Because raw scraped pages are full of garbage — menus, references, bad HTML. You need to clean it first. MongoDB gives me a clean staging area.

RAG Serving Layer (the online part)

Qdrant ← Batch Embeddings ← MongoDB
Since Newton's Wikipedia doesn't change every day, I use batch embedding instead of doing it live. Documents go from MongoDB → embedding model → Qdrant in scheduled batches. Its cheaper and faster.
When user asks a question:
User Question
→ FastAPI gets it
→ Query gets embedded
→ Qdrant finds similar chunks
→ Retrieved docs + question → LLM
→ Answer with sources
The LLM always gets context. It helps a lot with hallucinations.
Tech Stack

Orchestration: Apache Airflow (for DAGs, retries, monitoring)
Document Store: MongoDB (flexible for messy Wikipedia data)
Vector Store: Qdrant (fast and open source)
Backend: FastAPI (quick and clean)
Frontend: Next.js / Streamlit (Next for real use, Streamlit for quick tests)

Key Decisions
Batch Embedding > Real-time Embedding
Most tutorials embed on the fly. For static data like this, its stupid to keep re-embedding the same things. I run batch embedding once or on schedule and save a lot of time and money.
Airflow instead of simple Python script
I could have just written one scrape_and_embed.py file. But Airflow gives retries, proper logging, scheduling and makes each step separate. If Wikipedia is down, it retries automatically. For anything bigger than a toy project, orchestration actually matters.
Separating Ingestion from Serving
The scraping/cleaning part and the answering part are completely separate. Ingestion can break or update without touching the live RAG system. The serving layer just reads from Qdrant.
What I'd Do Differently Next Time

Add a reranker — simple vector search isn't enough. A reranker would make results much better.
Build evaluation from the start — without proper eval, you don't know if your changes actually help.
Add more sources — right now only Wikipedia. Academic papers would make it way stronger.
Try hybrid search — combine vector search with keyword search (BM25).

Final Thoughts
Building a simple RAG demo is easy. Building something that actually works properly is much harder. Most of the work is in the boring parts: cleaning data, setting up orchestration, separating concerns, and deciding when to use batch vs real-time.
Newton LLM showed me that good retrieval matters more than which LLM you use. If your pipeline is solid, even a smaller model gives good answers.
If you're building RAG, focus on the data pipeline first, not fancy prompts.

Forem: Dhaivat Jambudia