Forem: miwaty

I Built a Fully Local OSINT Agent with Ollama, LangChain, Telegram and Qwen3.5 14B — Running 24/7 on My Homelab, Zero Cloud, Zero Compromises

miwaty — Sun, 12 Apr 2026 21:45:28 +0000

I Built a Fully Local OSINT Agent with Ollama, LangChain, Telegram and Qwen3.5 14B — Running 24/7 on My Homelab, Zero Cloud, Zero Compromises

Disclaimer — This project was built for educational purposes only, as a hands-on way to learn LangChain, local LLMs and OSINT tooling. Every technique described here should only be used on targets you have explicit written authorisation to analyse. Unauthorised use of OSINT tools may be illegal in your jurisdiction. Always act ethically and responsibly.

I've been wanting to seriously learn LangChain for a while. Not just tutorials — actually build something real that I'd use.

So I built OSINT Marin AI: a fully local OSINT agent running on my homelab, accessible from anywhere via Telegram, powered by Qwen3.5 14B running locally via Ollama, with Tor for IP rotation, SQLite for caching, and Matplotlib for statistical charts sent directly to my phone.

No cloud. No API keys. No data leaving my network. Just Python, open-source tools, and a lot of trial and error.

Why local?

If you work anywhere near cybersecurity, you already know the answer. Sending queries about IP addresses, domains, Instagram profiles or email addresses to a cloud API means that data is leaving your network and hitting someone else's servers. In a professional context that's often a hard no.

Local models solve this completely. Everything stays on your hardware, under your control, with no third-party visibility into what you're analysing.

For this project I used Qwen3.5 14B in Q6_K quantization, served by Ollama. Thinking mode disabled — it adds latency with no real benefit for this use case.

What the agent can actually do

I'll be concrete. These are the Telegram commands that work right now:

Instagram:

/analizza_profilo @username — full profile metadata + AI summary
/scarica_foto @username — downloads photos, shows a paginated interactive list with buttons to view each photo or its metadata (likes, comments, date, hashtags, caption) directly in Telegram
/scarica_reel @username — downloads reels
/stories @username — downloads stories (requires auth)
/foto_profilo @username — downloads and sends the profile picture
/statistiche @username — generates and sends 7 statistical charts as PNG images

Web & OSINT:

/analizza_sito https://... — full site analysis with AI summary
/whois dominio.com — WHOIS + DNS records + AI interpretation
/sherlock username — username search across platforms + AI digital profile
/ip 1.2.3.4 — geolocation, ISP, ASN + AI comment
/email info@x.com — MX records, provider, domain status + AI analysis
/telefono +39... — carrier, country, number type + AI comment
/chiedi <domanda> — free natural language question to the LLM

Every command that hits an external service first checks the local SQLite database. If the data is already there, it returns it instantly without making a new request.

The stack

python-telegram-bot 21.6     ← user interface
LangChain 0.3.7              ← agent framework
LangChain-Ollama 0.2.1       ← LLM connector
Ollama                       ← local model server
Qwen3.5 14B (Q6_K)          ← the model (no thinking mode)
Instaloader 4.13.1           ← Instagram scraping
SQLAlchemy 2.0.35 + SQLite   ← local database
Matplotlib 3.9.2             ← statistical charts
Sherlock                     ← username OSINT
python-whois + dnspython     ← WHOIS and DNS
phonenumbers                 ← phone number analysis
stem 1.8.2                   ← Tor integration
Loguru 0.7.2                 ← structured logging

Architecture

You (Telegram)
      ↓
  Telegram Bot (python-telegram-bot)
      ↓
  handlers.py — command routing + auth check
      ↓
  tools/ — instagram, web, osint, stats
      ↓
  SQLite DB — always checked before external requests
      ↓
  External sources (Instagram, ip-api, Sherlock...)
      ↓ (optional)
  Tor proxy — IP rotation for Instagram requests
      ↓
  agent/llm.py — ChatOllama, generates AI summary
      ↓
  Back to Telegram — structured data + AI block

The LLM is the last step, not the first. The tools collect the data, the model synthesises it. If Ollama isn't running, the bot works anyway — the AI block is just skipped silently.

The caching layer — the decision I'm most proud of

Every external request is expensive: rate limits, latency, risk of getting blocked. So before every call to Instagram, ip-api, Sherlock or WHOIS, the agent checks SQLite first.

def analizza_profilo(username: str) -> dict:
    # Check local DB first — never hit Instagram twice for the same target
    profilo_db = get_profile(username)
    if profilo_db:
        logger.info(f"Profile {username} found in local database")
        return {...}  # instant response

    # Only here if not cached
    loader = _get_loader(autenticato=False)
    profile = instaloader.Profile.from_username(loader.context, username)
    dati = { ... }
    save_profile(dati)
    return dati

Five tables: profiles, posts, websites, osint_results, logs. Every result stored with full metadata. Over time the local database becomes genuinely useful — a growing knowledge base of everything you've ever analysed.

Tor integration for IP rotation

Instagram rate-limits aggressively. After a few requests, you start getting 401/403 from their GraphQL API. My solution: route Instagram traffic through Tor and rotate the circuit when blocked.

brew install tor
# Add to torrc:
# ControlPort 9051
brew services start tor

In Python, the utils/tor.py module handles circuit rotation via stem. When Instaloader hits a block, the agent automatically requests a new Tor circuit and retries. Not perfect, but it works well enough for homelab use.

The LLM integration — keeping it simple

I didn't build a complex ReAct agent for this. The pattern is simpler and more reliable:

Tool runs, collects structured data
Data is sent to Telegram as formatted text
Same data is passed to _llm_analizza() which selects the right prompt for that data type
Model generates a natural language summary
Summary sent to Telegram as a separate Analisi AI block

llm = ChatOllama(
    model="qwen3.5:14b",
    base_url=os.getenv("OLLAMA_BASE_URL", "http://localhost:11434"),
    temperature=0.1,      # deterministic — no hallucination games
    num_predict=2048,
)

Temperature 0.1 because I want the model to interpret data, not invent it. The system prompt instructs it to never speculate beyond what the tools returned.

The 7 statistical charts

For Instagram profiles, /statistiche @username generates and sends 7 PNG charts:

Monthly posting frequency — bar chart
Most active hours — heatmap (day of week × hour)
Engagement rate over time — line chart (likes + comments / followers × 100)
Like evolution — line chart
Top hashtags — horizontal bar chart (top 20)
Content type breakdown — pie chart (photo / video / reel)
Average caption length by month — bar chart

All generated with Matplotlib in Agg mode (no display needed), saved to /tmp/, sent to Telegram, then deleted. The underlying data comes from SQLite — no additional Instagram requests.

Project structure — designed to be extended

osint-marin-ai/
├── main.py               ← entrypoint
├── agent/
│   └── llm.py            ← ChatOllama + verifica_ollama + analizza_con_llm
├── bot/
│   ├── handlers.py       ← all Telegram command handlers
│   └── keyboards.py      ← inline keyboards
├── tools/
│   ├── instagram.py      ← Instaloader
│   ├── web.py            ← site analysis
│   ├── osint.py          ← Sherlock, IP, email, phone, WHOIS
│   └── stats.py          ← Matplotlib charts
├── db/
│   ├── models.py         ← SQLAlchemy models
│   └── database.py       ← CRUD + always-check-db-first pattern
└── utils/
    ├── logger.py         ← Loguru
    └── tor.py            ← Tor proxy + circuit rotation

Adding a new tool takes four steps:

Create a file in /tools/
Save results via db/database.py
Register the command in bot/handlers.py
Add a prompt entry in _llm_analizza() if you want AI summaries

That's the entire extension contract. No hidden coupling, everything is documented in Italian with docstrings.

What I actually learned building this

LangChain is powerful but you don't always need the full agent loop. For most of my commands, a simple tool → LLM → response pattern is faster, more reliable and easier to debug than a full ReAct agent. I'll use LangGraph for the more complex multi-step workflows I'm planning next.

Local LLMs are genuinely usable now. A year ago this would have been frustratingly slow. Today, with Q6_K quantization and thinking mode disabled, the model responds fast enough for interactive Telegram use. The ecosystem has caught up.

Caching is not optional. Instagram will block you. Sherlock takes 60-120 seconds. Without the SQLite caching layer this would be unusable. Cache everything, always check the database first.

Tor helps, but isn't a silver bullet. Instagram's bot detection is sophisticated. Authenticated requests with a dedicated account are still more reliable than Tor rotation for heavy scraping.

What's next

LangGraph for multi-step OSINT workflows (gather → correlate → report)
SearXNG for fully local web search integrated into the agent
PDF report generation — export a complete OSINT report from Telegram
Scheduled monitoring — alert when a tracked profile changes
Face recognition on downloaded photos

Final thoughts

I started this to learn LangChain. I ended up with a tool I actually use. That's the best possible outcome from a homelab project built for educational purposes.

If you're in security and haven't explored local LLMs yet — the barrier is lower than you think. Ollama makes model management trivial, LangChain handles the agent plumbing, and python-telegram-bot gives you a mobile interface in 50 lines of code.

The stack is mature. The models are capable. The only thing missing is someone building with them.

Written by miwaty — cybersecurity enthusiast, homelab builder, eternal work in progress.

How I Used the Model Context Protocol (MCP) to Coordinate My DIY Smart Home

miwaty — Sun, 06 Jul 2025 20:50:23 +0000

How I Used the Model Context Protocol (MCP) to Coordinate My DIY Smart Home

In recent months, I embarked on a personal project to upgrade my home with a smart automation system. The challenge was to integrate multiple components—motion sensors, temperature sensors, lights, and an IP camera—into a cohesive and context-aware environment.

Initially, each component was working in isolation. A motion sensor could turn on a light, a camera could start recording when triggered, and the temperature sensor could send its data periodically. But none of these devices had any understanding of the overall context. This limitation became apparent as the system grew more complex.

This article explores how I introduced a custom implementation of a Model Context Protocol (MCP) to unify the behavior of all these components and create a more intelligent and coordinated smart home system.

System Overview

The hardware and software stack included:

A mini PC running Ubuntu acting as the central server
ESP32 boards equipped with motion and temperature sensors, communicating via MQTT
A Raspberry Pi connected to an IP camera
Smart lights controlled via RESTful API
A Python-based automation controller
A WebSocket server for context synchronization

Initially, each component operated using simple event-action scripts. For example, when the motion sensor detected movement, a script would trigger the lights to turn on. However, the logic quickly became fragile and difficult to manage as additional devices were added.

The Core Issue: Lack of Shared Context

The primary issue was the absence of a shared understanding of the environment. Consider the following scenarios:

The lights turned on even when I was not home.
The IP camera kept recording even while I was in the living room.
Motion detection events were triggering the same reactions repeatedly, regardless of other conditions.
Environmental conditions such as brightness or time of day were not considered.

Each of these problems stemmed from the lack of a global state that all components could refer to. Each device operated based on its own inputs, without knowledge of what others were sensing or deciding.

Introducing the Model Context Protocol (MCP)

To solve this, I implemented a lightweight version of the Model Context Protocol. The goal was to establish a centralized context server that all components could publish to and subscribe from.

Key Principles

Each agent (sensor, light, camera, etc.) publishes its local state to a central context server.
The server maintains a global context model in memory, updated incrementally as new events arrive.
When significant changes occur in the context, interested agents are notified.
Agents react locally, using the latest context to decide whether or not to act.

This approach decouples the behavior logic from hardcoded trigger rules and instead allows decisions to be made based on consistent, shared context.

Technical Implementation

The Context Server was implemented in Python using the asyncio and websockets libraries. MQTT was used for communication with ESP32 devices, and TinyDB was used to persist the context state for inspection and recovery.

Each component sends updates in JSON format. For example:

{
  "source": "motion_sensor_living_room",
  "motion": true,
  "timestamp": "2025-07-06T14:21:45Z"
}

The context server processes this update and modifies the global context accordingly. A simplified context might look like this:

{
  "presence": true,
  "motion": true,
  "is_dark": false,
  "temperature": 27.8,
  "last_motion": "2025-07-06T14:21:45Z"
}

The context server then notifies all subscribers (e.g., the light controller, the camera module) about the updated context.

Real-World Use Case: Automated Presence Management

A particularly useful application of MCP was for presence detection. Here's how the system behaves with MCP:

When motion is detected while the context indicates that the house is "empty", the context updates presence to true.
The camera module, upon detecting presence: true, automatically stops recording to respect privacy.
The lighting controller turns on lights only if presence: true and is_dark: true.
Context updates are logged and stored for historical analysis or future machine learning applications.

Benefits of Using MCP

Implementing MCP brought several advantages:

Improved consistency: All agents operate based on a unified context model.
Reduced false triggers: Actions are taken only when multiple conditions are satisfied.
Easier debugging: Centralized logging of context changes makes it easier to trace issues.
Extensibility: New agents or logic can be added without modifying existing ones, as long as they follow the context protocol.

Security and Reliability Considerations

While this system is not exposed to the public internet, I added basic measures:

Token-based authentication for WebSocket clients
Input validation using jsonschema
Context versioning and timestamp tracking
Manual override controls via command line interface

Future improvements may include:

TLS encryption for WebSocket communication
Integration with a dashboard for visual monitoring
Rule-based automation engine using YAML or JSON configuration

Conclusion

Building a smart home system that behaves intelligently requires more than just connecting devices. The key lies in providing a shared understanding of context that all devices can refer to.

By implementing a simple Model Context Protocol, I was able to transform a fragile collection of scripts into a robust and extensible system where each component plays its part based on the global state.

This approach is scalable, reusable, and applicable well beyond home automation—any distributed system can benefit from having a centralized or decentralized context synchronization strategy.

If you're interested in the source code for the context server or the agents, feel free to reach out. I’d be happy to publish it in a follow-up article.

Getting Started with Podman: My First Splunk Test Lab

miwaty — Fri, 13 Jun 2025 10:50:39 +0000

A few weeks ago, I stumbled upon a LinkedIn post that mentioned Podman as a drop-in replacement for Docker—daemonless, rootless, and open-source. I had heard about it before but never gave it much thought. This time, the post got my attention.

Disclaimer: I'm not a professional writer or seasoned blogger.
I mostly use Dev.to as a notebook or public library for my tech experiments.
That said—I genuinely hope you’ll find something useful here that helps you replicate, improve, or build your own version of this small Splunk lab with Podman.
If you have suggestions, I’m always open to learning more.

I'm working a lot with Splunk, and I often spin up quick labs to test different components like Indexers, Heavy Forwarders, and Search Heads. I figured—why not try doing this with Podman?

Here’s how I set up a basic Splunk architecture using Podman and podman-compose.

Create the podman-compose.yml

**
I used a file nearly identical to what I’d write for Docker Compose, since podman-compose is compatible with the Compose Specification.
**

version: '3.8'

services:
  idx:
    image: docker.io/splunk/splunk:latest
    container_name: idx
    environment:
      - SPLUNK_START_ARGS=--accept-license
      - SPLUNK_PASSWORD=Splunk@00
      - SPLUNK_ROLE=splunk_indexer
      - SPLUNK_ENABLE_LISTEN=9997
    ports:
      - "8000:8000"
      - "9997:9997"
      - "8089:8089"
    networks:
      - splunk-net

  hf:
    image: docker.io/splunk/splunk:latest
    container_name: hf
    environment:
      - SPLUNK_START_ARGS=--accept-license
      - SPLUNK_PASSWORD=Splunk@00
      - SPLUNK_ROLE=splunk_heavy_forwarder
    ports:
      - "8001:8000"
    networks:
      - splunk-net
    depends_on:
      - idx

  sh:
    image: docker.io/splunk/splunk:latest
    container_name: sh
    environment:
      - SPLUNK_START_ARGS=--accept-license
      - SPLUNK_PASSWORD=Splunk@00
      - SPLUNK_ROLE=splunk_search_head
    ports:
      - "8003:8000"
    networks:
      - splunk-net
    depends_on:
      - idx

networks:
  splunk-net:
    driver: bridge

Run It with podman-compose

podman-compose -f podman-compose.yml up -d

This brought up the three containers: idx, hf, and sh, running on the same network.

Post-Startup Configuration

Search Head

podman exec -u splunk -it sh bash
/opt/splunk/bin/splunk add search-server idx:8089 -remoteUsername admin -remotePassword Splunk@00 -auth admin:Splunk@00
exit

Heavy Forwarder

podman exec -u splunk -it hf bash
/opt/splunk/bin/splunk add forward-server idx:9997 -auth admin:Splunk@00
exit

Verify the Configuration
Go to Splunk Web Access:
On the Search Head, go to
Settings > Distributed Search > Search Peers
and verify that the indexer appears and is connected.
On the Heavy Forwarder, check
Settings > Forwarding and receiving > Forwarded Data
to confirm that data is being forwarded to the indexer.

Honestly, I didn’t expect Podman to work this smoothly. The only real change I had to make was adding the full image path (docker.io/splunk/splunk) to avoid name resolution issues. Otherwise, the experience felt familiar and lightweight.

have fun with Zeek

miwaty — Sun, 16 Mar 2025 21:34:18 +0000

What a wonderful tool zeek is!

This is the thought I had after realising the versatility and potential of the instrument.

A network security monitoring tool, I needed this to perform a task. Its extensibility through the installation of protocol recognition packages is what impressed me the most, but let's take two steps back and start from the beginning.

I am using Zeek on my Raspberry(ubuntu), connected to a switch port(configured in monitor mode), Zeek operating to analyse and catalogue traffic, and a Universal Forwarder to make my life easier for post-capture data analysis.

Once the traffic capture begins, the captured packets are sent to an Event engine which is able to transform the flow of packets into a series of events representing network activity in a natural form. For example, an event of type HTTP is converted into its equivalent to know the ip addresses involved, http version used and the uri involved. Likewise an SSH type event contains the authentication, the IPs involved and the status. All that is extracted and converted is thanks to scripts already included in Zeek, which already allow a lot of information to be obtained with just the basis of the tool. All this is carefully placed in .log files for reading.

Are all types of protocols recognised?

The answer is yes.
but
But if we install the appropriate plugins, which I will explain in detail in another post. I had to perform an assessment discovery to compile a list of everything on that network segmentation.

Then knowing the domain context (small company) i.e. a small 3D printing company, I could have made a list by hand and then checked Zeek's results against my list.

But that was no fun!!!

So what I did was ask them to mirror a port and thus start Zeek.
Zeek fills a file called conn.log with all captured IP addresses primarily, located in the logs/current/ directory.

Main Log: conn.log
This file records all observed network connections, including:

Source IP (id.orig_h)
Destination IP (id.resp_h)
Ports, protocol, connection duration, and data transferred

ts uid id.orig_h id.resp_h proto service
1693401923.01 C5mAqR1T7kl 192.168.1.10 8.8.8 udp dns

This provides a full list of observed IPs in Zeek's network monitoring.

Then from there knowing the context and also recognising some services I was able to do some research and be able to install the exact modules (zeek-packages). One of these was ICSNPP-Modbus. Making a second round I completed the list and checking the parameters I got a good 90% coverage.

What did I learn from this experience?

Definitely a new tool to study in depth. A little more about the world of PLCs.
The architecture of Zeek itself.

How can you experiment?

Surely if you have a knowledge that allows you to monitor a network legally, you can try anything and for as long as you want since scanning is passive you do not risk doing damage or crashing networks.

Otherwise if you do not have the possibility of having a small infrastructure at home, you can always analyze pcap packets!!

Small Example:

Take a pcap of your interest, I recommend this list:
ICS-pcap

run the command:

**zeek -r <pcap>**

and at the end of the procedure you will have your files with the data.

How to analyze files more comfortably?

I use Splunk, it allows me to gather all the files under an index and launch SPL queries to facilitate the analysis.

Some tips

- Perform a diagnosis with zeekctl check

When installing modules never skip the tests
Do not install zkg as an external module with apt, but use the one under the --/zeek/bin folder
Contact me if you have errors.
Arm yourself with patience and remember that you must have in mind the infrastructure on which you are operating with Zeek