Forem: Sam Hartley

3 Months Running Everything Locally — What Broke, What Worked, What I'd Do Differently

Sam Hartley — Thu, 09 Apr 2026 08:03:48 +0000

It's been about three months since I made the switch. No more ChatGPT Plus. No more Claude subscription. No more Copilot. Everything I use day-to-day now runs on hardware sitting in my apartment — a Mac mini M4 and a PC with a mix of consumer GPUs.

I wrote the enthusiastic "I ditched OpenAI" post back in early March. This is the honest follow-up, because a lot of people asked me to come back after the honeymoon phase.

Some of it worked better than I expected. Some of it was genuinely annoying. Here's the unfiltered version.

The setup (short version)

Mac mini M4, 16 GB RAM — runs the orchestrator, small models, all the glue code
PC with an RTX 3060 12GB (plus spare 3070s and 3080s in a drawer) — runs the heavier models via Ollama
Everything talks over my LAN. Nothing leaves the house unless I tell it to.

I use this stuff for coding help, writing drafts, summarizing articles, transcribing voice notes, and a handful of personal automations.

What actually worked

1. Coding help for "normal" tasks

qwen3-coder:30b on the 3060 handles maybe 80% of what I used to ask GPT-4 for. Refactoring a function, explaining a gnarly regex, writing a quick shell script, sketching a React component. It's fast enough that I don't miss the cloud.

The latency is actually better than cloud APIs because there's no round trip to the US. I'll type a prompt and have tokens streaming back in under a second.

2. Voice notes and transcription

I didn't expect this to be the killer app, but it is. I talk to my Mac mini while I'm cooking, and Whisper on-device dumps the text into a daily markdown file. Zero cost, zero privacy worries, zero "oh I forgot to renew my API key."

This alone probably saved me more time than the coding stuff.

3. Batch jobs that used to rack up API bills

Summarizing 200 PDFs. Tagging a folder of screenshots. Generating alt text for a blog's image archive. These were the things that made me nervous to click "run" on OpenAI. Now I just let them chew overnight on the PC. Electricity is cheaper than tokens.

What broke or annoyed me

1. The "it's almost right" gap on hard stuff

For anything genuinely hard — like debugging a weird async bug in a codebase the model hasn't seen, or reasoning about a tricky algorithm — the gap between local 30B models and frontier cloud models is still real. It's not huge, but it's there.

I caught myself a few times thinking "I bet GPT-5 would get this in one shot" while I was on my fifth prompt with a local model. That's the honest truth.

My workaround: I keep a very small pay-as-you-go budget for the 2-3 times a month I actually need a frontier model. Probably $5/month total. Way cheaper than any subscription.

2. Context windows

Local models with huge contexts exist, but they get slow. Pasting a 50-file codebase into an 8B model and waiting is painful. I ended up writing a little script that does smart file selection instead of just dumping everything — basically a poor man's RAG. Worked better than I expected, but it was a weekend of yak-shaving I wasn't planning on.

3. Model sprawl

I have like 14 models pulled right now. qwen3-coder for code. deepseek-r1 for reasoning. A vision model for screenshots. Whisper for audio. A small embedding model. A translator. Each one made sense at the time. Now my Ollama directory is 180 GB and I can't remember what half of them are for.

I need to do a spring cleaning. I keep putting it off.

4. The "is it plugged in" problem

My PC is in another room. Sometimes my wife moves stuff and unplugs the switch. Sometimes Windows decides to reboot for updates at 3am. Sometimes the LAN cable gets bumped.

Cloud APIs just... work. Local stuff requires you to be your own SRE. I have health checks now. I have a Telegram bot that pings me when Ollama stops responding. This is not the kind of "home lab tinkering" I signed up for, but here we are.

What I'd do differently if I started today

Skip the "run everything locally or nothing" purity thing. I wasted a few weeks trying to make local models do things they're just not good at yet. The sweet spot is a hybrid setup: local for the 90% of boring stuff, a small cloud budget for the 10% that genuinely needs a bigger brain.

Buy one strong GPU instead of three mediocre ones. I have a drawer full of 3070s and 3080s I thought I'd use for "multi-GPU inference." In practice, a single 12GB card running a good 30B model handles almost everything I need, and juggling multiple cards adds complexity I don't want.

Put the models behind one API, not five. Ollama, llama.cpp, a Whisper server, a vision endpoint, embeddings... I should have stood up a single gateway that routes to the right backend. Instead I have five different base URLs in five different config files. It's fine, but it's ugly.

Write down what each model is for. Past-me assumed future-me would remember. Future-me does not remember.

Would I go back?

No. But I'd stop telling people "local AI is ready, ditch the subscriptions" like it's some binary choice. It's not.

If you're a developer who likes tinkering, has a decent GPU already, and wants to cut a $20-40/month subscription — yeah, it's great. You'll learn a lot, you'll own your stack, and you won't feel weird about pasting private code into someone else's API.

If you just want the best possible model for your work and don't want to babysit anything — stay on the cloud. There's no shame in it.

I'm in the first camp, but I was wrong to pretend the second camp was being lazy. They were being reasonable.

Anyone else running a mixed setup like this? I'm curious what other people's "local for X, cloud for Y" split looks like. Drop it in the comments — I'm especially interested in how you're handling the context-window problem without building your own RAG from scratch.

I Let AI Coding Agents Build My Side Projects for a Month — Here's My Honest Take

Sam Hartley — Sun, 05 Apr 2026 08:04:00 +0000

Last month I ran an experiment: instead of writing code myself, I delegated as much as possible to AI coding agents. Not just autocomplete — full autonomous agents that read files, run commands, and ship features.

I've been running a home lab (Mac Mini M4 + a Windows PC with GPUs + an Ubuntu box) for a while now, and I already had my dev workflow automated with AI agents. But this time I pushed further: what if the agents didn't just help me code, but actually wrote the code?

Here's what happened.

The Setup

I used a mix of tools:

Claude Code (CLI) — my go-to for complex, multi-file tasks
Codex (OpenAI) — good for one-shot generation with clear specs
Local models via Ollama — for quick iterations without burning API credits

The workflow: I'd describe what I wanted in plain English, point the agent at the right directory, and let it work. Sometimes I'd review the output. Sometimes I'd just run the tests and ship it.

What Actually Worked

1. Boilerplate and scaffolding — 10/10

This is where agents shine. "Set up a FastAPI project with SQLite, async endpoints, and Pydantic models for a subscriber management system." Done in 90 seconds. Would've taken me 20 minutes of copy-pasting from old projects.

2. Refactoring — 8/10

"Refactor this 400-line script into separate modules with proper error handling." The agents were surprisingly good at this. They understood the intent, split things logically, and even added type hints I'd been too lazy to write.

The two points I'm docking: they sometimes over-abstract. I'd ask for "cleaner code" and get an enterprise-grade factory pattern for a 50-line script. You still need taste.

3. Writing tests — 9/10

This was the biggest win I didn't expect. I hate writing tests. The agents love writing tests. "Write pytest tests for this module, cover edge cases" → comprehensive test suite in under a minute.

The catch: you need to actually read the tests. I caught a few that were testing the wrong thing — they'd pass, but they weren't testing what mattered. Still, it's a massive time saver.

4. Debugging — 7/10

Hit or miss. For straightforward bugs ("this function returns None when it should return a list"), agents crush it. For subtle timing issues or race conditions? They'd suggest fixes that looked right but didn't address the root cause.

My rule now: agents for the first pass, then I debug the debugger's output.

5. Documentation — 9/10

README files, docstrings, API docs. Agents are better at this than I am, honestly. They're more thorough, more consistent, and they don't get lazy halfway through.

What Didn't Work

Complex architecture decisions

"Should I use Redis or just in-memory caching for this?" The agent will give you a perfectly reasonable answer either way. That's the problem — it doesn't know your constraints the way you do. How many users? What's your memory budget? Are you running on a Raspberry Pi or a 128GB server?

I stopped asking agents for architecture advice. I make the decision, then let them implement it.

Multi-service orchestration

When I needed to coordinate between my Telegram bot, a background worker, and a database — and they all needed to agree on a shared state model — the agent would nail each piece individually but miss the integration points. I'd end up with three perfectly written services that didn't quite talk to each other.

Anything involving my specific hardware

"Configure this for my RTX 3060 with 12GB VRAM running on Windows with WSL2." The agents would give me generic CUDA setup instructions instead of what actually works on my rig. Local knowledge is still human knowledge.

The Numbers

Over the month, across 4 side projects:

Metric	Before (manual)	After (agent-assisted)
Time to MVP	~2 weeks	~4 days
Lines written by me	~80%	~30%
Bugs in first deploy	~8-12	~5-8
Time spent reviewing agent code	0	~2h/day
Overall velocity	1x	~2.5x

The 2.5x multiplier is real but misleading. I'm faster at producing code, but I spend more time reviewing code. The net gain is still significant — maybe 1.8x when you account for review time.

What I Changed About My Workflow

1. I write specs, not code.
My job shifted from "programmer" to "technical product manager." I write clear descriptions of what I want, define the interfaces, and let the agent fill in the implementation.

2. I review harder.
When I wrote the code myself, I'd eyeball it and move on. When an agent writes it, I actually read every line. Paradoxically, this has made me a better reviewer.

3. I prototype faster, throw away more.
Since generating a prototype takes minutes instead of hours, I build 2-3 approaches and pick the best one. This was a luxury I couldn't afford before.

4. I still write the hard parts.
State machines, complex business logic, performance-critical paths — I write these myself. Not because the agents can't, but because I need to understand them deeply to maintain them later.

The Uncomfortable Truth

Here's what nobody in the "AI will replace developers" discourse talks about: you need to be a good developer to use AI coding agents well.

Every time the agent produced something subtly wrong, I caught it because I knew what right looked like. Every time it over-engineered a solution, I could simplify it because I understood the problem. Every time it picked the wrong library or pattern, I could redirect it because I had opinions forged by years of mistakes.

Junior devs using these tools will ship faster. They'll also ship more bugs, more over-engineering, and more "it works but nobody can maintain it" code. The tools amplify whatever skill level you bring.

My Recommendation

If you're not using AI coding agents yet, start with:

Tests first. Let agents write your test suites. Low risk, high reward.
Boilerplate second. Project setup, CRUD endpoints, config files.
Refactoring third. Point it at your worst file and say "clean this up."
Complex features last. Only after you trust the tool and know its limits.

And always, always review the output. The agent is your fastest junior developer. It's also your most confident one — and confidence without experience is how bugs get shipped.

I write about building things with AI, self-hosting, and turning side projects into income. If you're into that, I post a new article every few days.

Running my own AI setup locally? Here's how I do it for $0/month.

I Expanded My GPU Rental Fleet to 6 Cards — Here's What Happened to My Earnings

Sam Hartley — Sat, 04 Apr 2026 15:44:11 +0000

I Expanded My GPU Rental Fleet to 6 Cards — Here's What Happened to My Earnings

A few weeks ago I wrote about renting out my single RTX 3060 on Vast.ai for passive income. The experiment worked better than I expected, so I did what any reasonable person would do: I went and dug out the five other GPUs sitting in my storage room.

This is the honest follow-up. What actually happened when I went from 1 GPU to 6.

The Backstory

I had a bunch of GPUs from an older setup — two RTX 3070s, one RTX 3080, and two more RTX 3060s. They were collecting dust. The PC they came from got upgraded, the cards went into cardboard boxes, the boxes went under a shelf.

Total VRAM across all six: around 62GB. Combined retail value when new: probably $3,000+. Current value sitting in boxes: $0/month.

The math wasn't complicated.

What the Expansion Actually Took

Here's what I underestimated: it's not just "plug cards in, profit."

The hardware side

You can't just stack 6 GPUs into a regular PC case. I had to think about:

PCIe slots and bandwidth. A standard ATX board has maybe 2-3 real x16 slots. For 6 cards, you're looking at risers, which means a mining-style open frame or a server chassis.
Power. Each card pulls 150-250W under load. Six cards = potentially 1,200-1,500W just in GPU power. Plus CPU, drives, RAM. My existing 850W PSU was not going to cut it.
Cooling. Cards in a tight case thermal-throttle each other. Open frame was the answer.

I ended up using an open-air mining frame I found used for cheap, two PSUs daisy-chained (a sketchy-but-common approach in the mining world), and PCIe risers.

Setup time: about a full weekend.

The software side

Getting all six cards recognized wasn't plug-and-play either. I run Windows on the main PC (easier driver support for NVIDIA), and Vast.ai has a Windows daemon that mostly works — except when it doesn't.

A few issues I hit:

Two risers were flaky and caused cards to drop off
One 3070 had a driver conflict until I did a clean DDU reinstall
Vast.ai's host dashboard showed 5 GPUs after setup; took me an hour to figure out the sixth wasn't being detected

Total debugging time before everything was stable: another weekend.

The Earnings Comparison

Setup	Cards	VRAM	Weekly Earnings
Before (1 card)	RTX 3060	12GB	~$12-18
After (6 cards)	3060 × 3, 3070 × 2, 3080 × 1	62GB	~$65-95

Not exactly linear scaling. Here's why:

Demand is unpredictable. Sometimes 4 of my 6 cards are rented simultaneously. Sometimes 1. The RTX 3080 gets picked up more often than the 3060s — higher VRAM matters for LLM inference jobs that need room to load bigger models.

Not all hours are equal. Utilization spikes during US business hours and drops overnight (Turkey time). I'm in a timezone where "overnight for me" overlaps with "peak US working hours," which actually helps.

Pricing matters more than I thought. I dropped my per-card price slightly and saw utilization go up noticeably. A few cents per hour makes a real difference when renters are comparing a dozen similar options.

Current Monthly Run Rate

Across all six cards, I'm averaging around $280-340/month before electricity.

Power costs are real. Six GPUs under load is serious wattage. My electricity bill went up — I haven't calculated the exact delta yet because my bill is shared (I'm not the only one using power in my building), but I'd estimate $40-60/month in additional costs.

Net: roughly $220-280/month in real passive income.

Is that life-changing? No. Is it meaningful for money that was doing nothing? Absolutely.

What I'd Do Differently

1. Start with a proper open-frame rig, not a cobbled-together case.

The mining frame was cheap but took time to source. If I were doing this again I'd budget for it from day one.

2. Get a proper high-wattage PSU setup.

Running two PSUs linked together works but it's inelegant. A server PSU with the right adapter is cleaner and safer.

3. Test each card individually before combining them.

I wasted time troubleshooting "which card is the problem" when I could've confirmed each one worked before building the full rig.

4. Set minimum job duration.

Short jobs (under an hour) rack up overhead — container spin-up time, handshaking — without much earnings. I set a minimum of 2 hours and earnings-per-hour improved.

The Unexpected Part

I expected this to be a boring passive income setup. It mostly is. But I've learned a surprising amount about how the AI inference market actually works by watching what gets rented and when.

Most renters are running:

Fine-tuning jobs (need sustained GPU hours)
LLM inference (need VRAM more than raw compute)
Image generation (FLUX, Stable Diffusion variants)
Dev environments (people testing stuff without committing to a cloud contract)

Watching the demand patterns is actually interesting data about what the AI dev community is building right now. The 3080 almost always goes first — 10GB VRAM hits a sweet spot for smaller Llama and Mistral models.

Is It Worth It?

Depends on your situation.

Yes, if: You already have the GPUs and they're sitting idle. The marginal cost of setting this up is mostly your time, and the monthly return is real.

Maybe, if: You'd have to buy the GPUs. At current used-market prices, payback period is 6-12 months depending on utilization. That's not terrible but it's not obvious.

No, if: You're renting out your daily-driver GPU. The rental platform can grab your card at inconvenient times. Keep at least one card reserved for your own use.

What's Next

I'm looking at adding the Ubuntu server I have running as a CPU-only Vast.ai host for smaller workloads. Less money per unit but zero additional hardware cost.

Also thinking about whether it makes sense to eventually get into the dedicated hosting side rather than the rental marketplace — more stable income, more setup required. Still researching.

For now, 6 cards, ~$250/month net, and a weekend's worth of setup. I'll take it.

Questions about the rig setup or Vast.ai specifics? Drop them in the comments.

→ Check out my automation work on Fiverr

→ Follow along on Telegram

I Use Telegram as My DevOps Dashboard — No Web UI, No VPN, Just Works

Sam Hartley — Mon, 23 Mar 2026 08:02:32 +0000

I have a bunch of things running 24/7 on a Mac Mini. GPU rental jobs, a Garmin watch face updater, a Fiverr inbox monitor, a funding rate tracker, a few cron jobs.

For a while I ran a Grafana dashboard to keep an eye on them. It looked impressive. I never opened it.

What I actually do is check my phone. So I built the monitoring layer there.

Here's the setup: a lightweight Telegram bot that serves as my entire DevOps interface. Status checks, alerts, and even simple commands — all from the Telegram app I already have open.

Why Not a Proper Dashboard?

Honest answer: dashboards are for teams. If you're a solo dev with a few projects, a fancy web UI creates more overhead than it solves.

Problems I had with Grafana:

VPN required to reach it from outside my home network
Needs to stay running (another thing to maintain)
I never actually opened the browser tab
It didn't push me information — I had to pull it

Telegram flips this: it pushes alerts to me. I glance at my phone, see what's happening, and move on.

The Architecture

Services (cron jobs, Python scripts, shell scripts)
  ↓
Central alert script: notify.sh
  ↓
Telegram Bot API → my phone
  ↓ (optional)
Command bot → runs queries on server

Two pieces:

Outbound alerts — services send me messages when things happen
Inbound commands — I can ask the bot questions from my phone

Part 1: Dead Simple Alert Script

Every service on my server can call this:

#!/bin/bash
# notify.sh — send a Telegram message from any script
# Usage: ./notify.sh "Your GPU job finished"

BOT_TOKEN="your_bot_token_here"
CHAT_ID="your_chat_id_here"
MESSAGE="$1"

curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
  -d chat_id="${CHAT_ID}" \
  -d text="${MESSAGE}" \
  -d parse_mode="HTML" > /dev/null

That's it. Any script can now send me a message in one line:

./notify.sh "✅ GPU rental job completed — earned $2.40"
./notify.sh "⚠️ Funding rate dropped below threshold on LYN_USDT"
./notify.sh "📬 New Fiverr inquiry from user987"
./notify.sh "❌ Garmin watch face API returned 503"

I spent maybe 20 minutes on this. It replaced a monitoring stack I spent days configuring.

Real Examples from My Setup

GPU rental monitor:

# Runs every 30 min
earnings=$(./check_gpu_earnings.sh)
if [ "$earnings" -gt "0" ]; then
  ./notify.sh "💰 GPU earned: $earnings today"
fi

Funding rate watcher:

# Python script, runs every 15 min via cron
rate = get_funding_rate("LYN_USDT")
if rate < -0.5:  # negative = people paying longs
    notify(f"🔥 LYN funding rate: {rate}% — worth checking")

Daily summary (9 AM cron):

#!/bin/bash
msg="📊 Daily Summary — $(date +%Y-%m-%d)

GPU Jobs: $(get_gpu_count) completed
Funding Earned: $(get_funding_total)
Fiverr Inquiries: $(get_fiverr_count)
Watch Face Updates: $(get_garmin_count)

Server uptime: $(uptime -p)"

./notify.sh "$msg"

I wake up, check my phone, and immediately know if anything needs attention. No browser, no VPN, no dashboard.

Part 2: The Command Interface

Outbound alerts are great. But sometimes I want to query the server from my phone.

I wrote a simple Python bot that listens for commands:

import telebot
import subprocess

BOT_TOKEN = "your_token"
ALLOWED_USER = 123456789  # your Telegram user ID

bot = telebot.TeleBot(BOT_TOKEN)

COMMANDS = {
    "/status": "uptime && free -h && df -h /",
    "/gpu": "./check_gpu_status.sh",
    "/funding": "python3 check_funding_rates.py --summary",
    "/services": "ps aux | grep -E '(python|node|ollama)' | grep -v grep",
}

@bot.message_handler(commands=list(COMMANDS.keys()))
def handle_command(message):
    if message.from_user.id != ALLOWED_USER:
        bot.reply_to(message, "Not authorized.")
        return

    cmd_text = message.text.split()[0]  # handle /status@botname format
    if cmd_text in COMMANDS:
        result = subprocess.run(
            COMMANDS[cmd_text], 
            shell=True, 
            capture_output=True, 
            text=True,
            timeout=15
        )
        output = result.stdout[:3000] or result.stderr[:3000] or "No output"
        bot.reply_to(message, f"```
{% endraw %}
\n{output}\n
{% raw %}
```", parse_mode="Markdown")

bot.polling(none_stop=True)

Now from Telegram I can type /status and get:

 11:23:15 up 14 days, 3:41,  1 user
Mem:   16Gi   8.2Gi   7.8Gi
/dev/sda1        245G   82G  163G  34%

Or /funding and get the current rate snapshot.

The key detail: ALLOWED_USER check. Only my Telegram ID can run commands. Everyone else gets "Not authorized." Bot tokens are public in the sense that anyone can message your bot — you need to validate the sender.

Keeping It Running

The command bot needs to stay alive. I use a simple systemd service:

[Unit]
Description=Telegram DevOps Bot
After=network.target

[Service]
ExecStart=/usr/bin/python3 /home/user/telegram-bot/bot.py
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

systemctl enable telegram-bot && systemctl start telegram-bot — and it survives reboots.

On macOS (my setup) I use a launchd plist, same concept.

What I Actually Get Alerts For

Not everything. Alert fatigue is real.

Alert on:

✅ Job completions (GPU task done, funding cycle closed)
❌ Errors that need action
📬 New customer inquiries (Fiverr inbox)
⚠️ Thresholds crossed (rate drops, disk usage, memory spikes)
📊 Daily summaries (once a day, morning)

Silence:

Routine successful runs (no news is good news)
Health checks that pass
Regular cron completions with no anomalies

The goal is: every message I receive from the bot is something I actually care about. If I'm ignoring 80% of notifications, I'm alerting on the wrong things.

The Full Cost

Telegram Bot API: free
curl command: comes with your OS
Python + telebot library: free
Running this bot: negligible CPU, ~20MB RAM

My entire monitoring setup costs $0/month and runs on the same Mac Mini as everything else. No SaaS, no cloud logging, no dashboard subscription.

Three Months Later

I send about 15-20 alerts per day. Daily summary at 9 AM, event-driven messages the rest of the day. I check my phone, see green checkmarks and earnings summaries, and know the server is doing its job.

The one time the GPU host went offline, I got a message within 5 minutes. Fixed it from my phone during lunch.

That's the whole point: not more tooling, just the right interface for how I actually work.

Got a monitoring setup you like? Drop it in the comments — always curious what others are running.

→ I build these kinds of automation setups on Fiverr
→ Follow CelebiBots on Telegram

I Rented Out My GPU for Passive Income — Here’s What Happened After My First Week

Sam Hartley — Sat, 21 Mar 2026 08:02:08 +0000

I had an RTX 3060 sitting on a shelf.

Not broken. Not old. Just... not doing anything. My Windows PC runs models when I need them, but most of the time it's idle. The fans spin, the power draw ticks along, and that 12GB of VRAM just sits there.

A week ago I connected it to Vast.ai — a GPU marketplace where people rent compute time. No code required. You install a daemon, set a price, and wait for someone to rent your machine.

Here's what actually happened.

Why I Didn't Just Mine Crypto

First thing people ask: "Why not just mine?"

Short answer: it's 2026, the margins are brutal, and I didn't want to deal with it. GPU compute rental is different — you're renting raw processing power, and the demand right now is AI inference and training. People building LLMs, running diffusion models, doing batch jobs.

The upside: no mining pool setup, no daily coin price anxiety, no special software. Your machine runs Docker containers, gets paid per second of use, you get a payout.

The Setup (Genuinely About 90 Minutes)

Created a Vast.ai account
Installed the host daemon on Windows (it's a one-click installer)
Set my RTX 3060 12GB at $0.15/hour
Went to bed

That's it. No configuration rabbit holes, no drivers to hunt down. The daemon manages everything — spinning up containers, cleaning up after renters, reporting uptime.

I set the minimum rental duration to 1 hour so I wouldn't get hit with a dozen 5-minute jobs.

The First Week Numbers

Day	Hours Rented	Earnings
Day 1	3.2h	$0.48
Day 2	11.5h	$1.73
Day 3	0h	$0.00
Day 4	16.8h	$2.52
Day 5	9.1h	$1.37
Day 6	22.0h	$3.30
Day 7	14.4h	$2.16

Week 1 total: ~$11.56

Annualized naively? About $600/year. Which would be great except day 3 was $0 and utilization is inconsistent.

A more realistic steady-state: $50–130/month depending on demand.

What People Actually Rent It For

Vast.ai shows you the jobs (anonymized). Mine has been used for:

Running vllm inference servers (Mistral, Qwen, LLaMA variants)
Stable Diffusion batch jobs
Some kind of PyTorch training run that lasted 8 hours

The 3060 with 12GB VRAM is actually sweet for inference — fits most 7B–13B models at 4-bit quantization without breaking a sweat. It's not the fastest card, but it's affordable to rent, which means demand is there.

The Honest Downsides

You can't use your GPU while it's rented. Sounds obvious, but the practical implication: if you need your machine for local inference and someone's rented it, tough luck. I started routing heavy tasks to my Mac Mini during rental periods.

Electricity. My RTX 3060 at load pulls about 150W. At Turkish electricity rates, that's roughly $8–15/month in power at typical utilization. So the net is lower than the gross numbers above.

It's genuinely passive but not predictable. Day 3 was $0. Day 6 was near-full utilization. There's no way to forecast demand.

Payouts have a minimum. Vast.ai pays out once you hit a threshold. Nothing to worry about, just something to know going in.

What I'm Going to Try Next

The obvious play is adding more GPUs. I have a few more in storage — an RTX 3080 and some older 3060s. If I rack those up, the math gets interesting:

GPU	Rate	Monthly (50% util)
RTX 3060 (current)	$0.15/h	~$54
RTX 3080 10GB	$0.20/h	~$72
2x RTX 3070 8GB	$0.16/h	~$115

That's ~$240/month without doing anything after setup. At 70% utilization: ~$340.

The real work is physical — pulling GPUs from storage, getting them into a rig, managing thermals. But the software side is almost zero maintenance.

Should You Try This?

If you have a spare GPU collecting dust: yes, probably. The setup is low friction, the risk is near-zero (worst case, you uninstall the daemon and move on), and even modest earnings beat $0.

If you're thinking about buying a GPU specifically for this: do the math carefully. At current rates, an RTX 3060 costs ~$300–350 used. Payback period at $50/month is 6–7 months, which is fine — but don't expect to fund your retirement from a single card.

The real value for me isn't the income (yet). It's that I now have a system running, I understand the demand patterns, and I know the path to scale looks viable.

The Setup Summary

Platform: Vast.ai (there's also RunPod if you want alternatives)
Time to set up: ~90 minutes
Technical skill required: Know how to install software on Windows
Ongoing maintenance: Almost none
Realistic earnings: $50–130/month per mid-tier GPU

Happy to answer questions if you try this and run into something weird. The daemon is pretty solid but there's always an edge case or two.

I write about running AI locally, automation side projects, and occasionally making money from hardware that would otherwise just collect dust. If any of this is useful, feel free to follow.

Local LLMs vs Cloud APIs — A Real Cost Comparison (2026)

Sam Hartley — Thu, 19 Mar 2026 08:03:20 +0000

"Just use ChatGPT" — sure, until your API bill hits $500/month.

I've been running both local and cloud AI for over a year. Here are the real numbers.

The Test Setup

Cloud: OpenAI GPT-4o, Anthropic Claude Sonnet, Google Gemini Pro
Local: Ollama with Qwen 3.5 9B (Mac Mini M4) + Qwen 3 Coder 30B (RTX 3060)

Workload: ~500 queries/day — code review, content generation, customer support, data analysis.

Monthly Cloud API Costs

For 500 queries/day:

OpenAI GPT-4o (200 queries): ~$90/month
Anthropic Claude Sonnet (200 queries): ~$72/month
Google Gemini Pro (100 queries): ~$25/month
Total: ~$187/month

Monthly Local Setup Costs

Mac Mini M4 (already owned): $0
RTX 3060 12GB (used, eBay): $150 one-time
Electricity 24/7: ~$12/month
Total: ~$12/month ongoing

Break-even: less than 1 month.

Quality Comparison (What Surprised Me)

For 80% of daily tasks, local models are good enough:

General chat: Qwen 3.5 9B is roughly GPT-4o quality (~90%)
Code generation: Qwen 3 Coder 30B is close to Claude Sonnet (~85-90%)
Simple Q&A and extraction: any 7B model matches cloud (~95%+)
Complex multi-step reasoning: cloud still wins here

The Hybrid Approach I Use

User query
  -> Simple? (Q&A, formatting, extraction)
       -> Local Qwen 3.5 9B  (free, instant)
  -> Code-heavy?
       -> Local Qwen 3 Coder 30B  (free, ~12s)
  -> Complex reasoning?
       -> Cloud Claude Sonnet  ($0.003-0.015 per query)

Result: cloud costs dropped from ~$187/month to ~$25/month.

Hidden Costs of Cloud

Things people forget:

Rate limits — hit the ceiling during a deadline? Too bad.
Latency — 500-2000ms per request vs 100-500ms local
Privacy — your code and data live on someone else's server
Vendor lock-in — OpenAI changes pricing, you're stuck
Downtime — their outage = your workflow stops

Hidden Costs of Local

Being fair:

Initial hardware — $150-500 for a GPU (pays off in under a month)
Setup time — 30 minutes with Ollama these days
Storage — models are 4-40GB each
Power — $10-15/month for 24/7 operation
No frontier models — you won't run GPT-4 locally yet

Getting Started in 10 Minutes

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull qwen3.5:9b

# Start chatting
ollama run qwen3.5:9b

Total time: 10 minutes. Total cost: $0.

Need help setting up a local AI server? I do this professionally.

Follow along: Telegram @celebibot_en

Sam Hartley — building AI things that actually work.

How I Automated My Entire Dev Workflow with AI Agents (Running 24/7 on a Mac Mini)

Sam Hartley — Sun, 15 Mar 2026 08:01:19 +0000

I used to spend 3 hours a day on repetitive tasks. Now an AI agent handles them while I sleep.

This isn't a concept post — this setup is running right now on a Mac Mini in my home office. Here's the full picture.

What Gets Automated

Task	Before (Manual)	After (AI Agent)
Email triage	30 min/day	Automatic
Code review	45 min/day	12 seconds per PR
Customer inbox check	20 min/day	Hourly cron job
Calendar reminders	Forget constantly	Proactive alerts
Weather check	Open app manually	Agent tells me before I leave
Market data	Check 5 websites	Live on my watch face

Total time saved: ~2.5 hours/day. That's 75 hours a month I got back.

The Architecture

Mac Mini M4 (always on)
├── AI Agent (orchestrator)
│   ├── Email checker (hourly)
│   ├── Calendar scanner (every 4 hours)
│   ├── Inbox monitor (Fiverr/orders, every hour)
│   └── Market data fetcher (every 15 min)
├── Ollama (local LLM — free inference)
│   ├── Qwen 3.5 9B (general tasks)
│   └── Qwen 3 Coder 30B (via network GPU)
├── Notification layer
│   └── Telegram bot → my phone
└── Background services
    ├── Garmin watch face data feed
    └── System health monitoring

Total hardware cost: $0 additional (Mac Mini was already there).
Monthly API cost: ~$25 (only complex reasoning hits the cloud).

Key Design Principles

1. Don't Automate Everything

The 80/20 rule applies hard:

✅ Automate: Checking, monitoring, formatting, reminders
❌ Don't automate: Decisions that need judgment, creative work, human relationships

I tried automating replies to clients once. Deleted it after day one. Some things need a human touch.

2. Alert, Don't Act

My agent tells me when something needs attention — it doesn't reply to customers, send emails, or make purchases.

One wrong automated email can destroy a client relationship. AI is excellent at detection. It's not great at nuance.

3. Local First, Cloud Fallback

if query_is_simple(prompt):
    # Free, instant, private
    response = ollama.generate(model="qwen3.5:9b", prompt=prompt)
else:
    # Rare — complex reasoning only
    response = claude.complete(prompt)

95% of automation tasks are simple: "Is there a new message? What does it say? Is it urgent?"

A 9B model handles that perfectly at zero cost.

4. Fail Silently, Alert Loudly

If the weather API is down → no notification (who cares).
If a paying customer messages → instant alert to my phone.

Not all failures are equal. Treat them that way.

Real Example: The Inbox Monitor

Every hour, the agent:

Opens the browser → inbox
Scans for new messages
Checks against known spam/scam patterns (regex + AI)
If legitimate: sends me a Telegram notification with the preview
If junk: logs it and moves on quietly

Scam detection rate: 100% so far (pattern matching + LLM analysis).
False positives: 0 (I still review every alert manually before responding).
Time saved: 20 min/day × 30 days = 10 hours/month.

The Full Stack (All Free or Open Source)

Component	Tool	Cost
LLM inference	Ollama + Qwen 3.5	$0
Orchestration	Cron jobs + Python scripts	$0
Notifications	Telegram Bot API	$0
Email access	Apple Mail + CLI bridge	$0
Calendar	EventKit (macOS)	$0
Browser automation	Safari + accessibility APIs	$0
Cloud reasoning (10%)	Anthropic API	~$25/mo

Monthly total: ~$25. Before this setup I was paying $150+ just in API costs.

How I Started (Without Going Crazy)

I didn't build this in a weekend. Here's the honest timeline:

Week 1: Just the email checker. One script, one cron job.
Week 2: Added Telegram notifications. Suddenly I could see it working.
Week 3: Calendar alerts. This was a game changer.
Month 2: Inbox monitoring, market data feed.
Month 3: Everything orchestrated together.

The lesson: start with one automation, get it reliable, then add the next. Trying to build everything at once is how you end up with a half-working mess you don't trust.

Lessons From Running This in Production

Logging is everything. When something fails at 3 AM, logs are all you have.
Cron intervals matter. Checking email every minute wastes resources. Every hour is fine for most things.
AI doesn't replace thinking — it replaces the mechanical parts of thinking.
Test on your actual workflow before scaling. My setup works for me; yours will look different.
The value isn't speed — it's offloading mental overhead. I stopped worrying about forgetting to check things.

What's Next

I'm building a Telegram-based version of this that others can use — subscribe to signals, set their own alert rules, powered by the same local AI stack. If that sounds interesting, follow along.

Have you automated parts of your workflow? What was the first thing you tackled? Drop it in the comments — I'm genuinely curious what others are automating.

If you want to set something like this up for your own workflow or business, I do this as a service too: Custom AI Automation Workflows

Daily automation tips: t.me/celebibot_en

I Built a Personal AI That Actually Knows My Projects (RAG + Ollama, Zero Cloud)

Sam Hartley — Fri, 13 Mar 2026 08:01:50 +0000

I got tired of explaining my own codebase to an AI every single session.

"Here's the architecture. Here's the README. Here's what I tried last time." Every. Single. Time.

So I built a local RAG (Retrieval-Augmented Generation) system that knows my projects, my notes, and my docs — permanently. No cloud. No API costs. No context window resets.

Here's exactly how it works.

The Problem with Context Windows

LLMs don't remember. You paste the same 200 lines of context every session, hit the token limit, and start over. It's fine for one-off questions. It's exhausting for ongoing projects.

The standard solution is RAG: instead of stuffing everything into the prompt, you store docs in a vector database and retrieve only the relevant chunks when you ask a question. The model sees 3-5 paragraphs of targeted context instead of your entire repo.

Result: faster, cheaper, and the AI actually answers the right question.

The Architecture

Your Documents (markdown, code, PDFs, notes)
  → Chunked + embedded (Ollama nomic-embed-text)
  → Stored in Chroma (local vector DB)

Query
  → Embedded (same model)
  → Top-5 relevant chunks retrieved
  → Stuffed into Ollama prompt (Qwen 3.5 9B)
  → Answer

Zero cloud. Zero API keys. Runs on a Mac Mini or any machine with 8GB RAM.

What I Index

Everything that would normally eat my context window:

Project READMEs and architecture docs
My personal notes (Obsidian vault)
Code snippets and past solutions
API documentation I use regularly
Stack Overflow answers I bookmarked (because I always forget them again)
Config files and deployment notes

Total indexed: ~4,800 chunks. Query time: under 2 seconds.

Step 1: Install the Stack (15 minutes)

# Ollama (already installed? skip)
curl -fsSL https://ollama.com/install.sh | sh

# Pull models
ollama pull qwen3.5:9b          # LLM for answers
ollama pull nomic-embed-text    # Embedding model

# Python dependencies
pip install chromadb langchain ollama pypdf markdown

That's the entire stack. No Docker required (though Chroma has a Docker option if you want a persistent server).

Step 2: Index Your Documents

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import DirectoryLoader
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.vectorstores import Chroma

# Load your docs folder
loader = DirectoryLoader("~/projects/", glob="**/*.md", recursive=True)
docs = loader.load()

# Split into chunks (400 tokens, 50 overlap)
splitter = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=50)
chunks = splitter.split_documents(docs)

print(f"Indexed {len(chunks)} chunks from {len(docs)} documents")

# Embed and store locally
embeddings = OllamaEmbeddings(model="nomic-embed-text")
db = Chroma.from_documents(
    chunks, 
    embeddings, 
    persist_directory="./my-knowledge-base"
)
db.persist()

Run once. Done. Your docs are now searchable by meaning, not just keywords.

Step 3: Query It

import ollama
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.vectorstores import Chroma

# Load existing DB
embeddings = OllamaEmbeddings(model="nomic-embed-text")
db = Chroma(persist_directory="./my-knowledge-base", embedding_function=embeddings)

def ask(question: str) -> str:
    # Retrieve top 5 relevant chunks
    results = db.similarity_search(question, k=5)
    context = "\n\n".join([r.page_content for r in results])

    # Query local LLM with context
    response = ollama.chat(
        model="qwen3.5:9b",
        messages=[{
            "role": "user",
            "content": f"Based on this context:\n\n{context}\n\nAnswer: {question}"
        }]
    )
    return response['message']['content']

# Example
print(ask("How does my Garmin watch face fetch stock data?"))
print(ask("What's the API rate limit for the crypto bot?"))
print(ask("How do I deploy the Telegram bot to the VPS?"))

Real answers from your own documentation. No hallucinations about your specific setup.

The Killer Feature: Incremental Updates

Don't re-index everything when one file changes. Just update what's new:

import hashlib
import json
from pathlib import Path

def get_file_hash(path):
    return hashlib.md5(Path(path).read_bytes()).hexdigest()

def update_index(docs_dir, db, index_cache="./index-cache.json"):
    cache = json.loads(Path(index_cache).read_text()) if Path(index_cache).exists() else {}

    changed_files = []
    for f in Path(docs_dir).rglob("*.md"):
        h = get_file_hash(f)
        if cache.get(str(f)) != h:
            changed_files.append(str(f))
            cache[str(f)] = h

    if changed_files:
        print(f"Re-indexing {len(changed_files)} changed files...")
        # [load, chunk, embed, upsert only changed files]

    json.dumps(cache) and Path(index_cache).write_text(json.dumps(cache))

Run this as a cron job every hour. Your knowledge base stays current automatically.

What Actually Changed for Me

Before RAG:

"Explain the background service memory limit in my Garmin project" → paste 200 lines → wait → answer
Every new chat session: context reset, start explaining again

After RAG:

ask("Garmin background service memory limit") → "64KB sandbox, pass data via Background.exit(dictionary)" — in 1.8 seconds

My LLM now answers questions about projects I haven't touched in 6 months. No context management. No pasting. Just ask.

Hardware Requirements

Setup	RAM	Embedding Speed	Query Speed
Mac Mini M4 8GB	8GB	~500 docs/min	~2s
RTX 3060 12GB	12GB VRAM	~3000 docs/min	~0.5s
Old laptop 8GB	8GB	~100 docs/min	~5-8s

The embedding step (indexing) is the slow part — run it once, then it's instant.

Tips from Running This for 3 Months

Chunk size matters — 400 tokens works well for prose and docs. For code, try 200 with more overlap.
Metadata is your friend — store filename and section in chunk metadata. When the AI says "see the deployment notes," you know exactly where to look.
Re-rank when accuracy matters — if top-5 chunks aren't enough, add a re-ranker step (Cohere has a free API, or use a local cross-encoder).
Watch your embed model — nomic-embed-text beats most larger models for RAG. Don't use your chat LLM for embeddings.
Hybrid search — combine vector search with BM25 keyword search for better results on technical queries with specific names/functions.

The Bigger Picture

This is step one of something bigger: a personal AI that grows with your projects instead of resetting every session.

Next phase I'm building: automatic indexing from Git commits (index diffs in real-time as you code) + a simple web UI for non-terminal queries.

Total current cost of this setup: $0/month. It runs on the same Mac Mini I already had.

Want This for Your Business?

RAG systems are one of the highest-value AI implementations you can build. A local knowledge base trained on your company's docs, SOPs, and code can:

Answer customer support questions without hallucinating
Help your team find answers instantly across years of internal docs
Run 24/7 with zero ongoing API costs

→ Custom RAG setup on Fiverr
→ Follow us on Telegram

Built with 🤖 by CelebiBots — AI that runs on your terms.

5 Telegram Bot Ideas That Actually Make Money (Built a Few of These Myself)

Sam Hartley — Wed, 11 Mar 2026 08:01:24 +0000

Everyone and their grandmother builds Telegram bots these days. But most of them are glorified toys that never earn a cent.

I've been building and selling custom Telegram bots as a side income for the past year. Here are 5 ideas that have real revenue models behind them — not wishful thinking, not "build an audience first" advice.

1. Crypto Price Alert Bot ($5–20/month per user)

What it does: Monitors crypto prices in real-time and sends instant Telegram messages when assets hit thresholds the user defines.

Why it actually works:
Traders check prices constantly. An alert bot eliminates the anxiety loop of "let me just check one more time." The free tier (3 alerts) hooks them. The paid tier (unlimited alerts + portfolio tracking) converts the ones who actually trade.

API costs for this? Basically zero — free tiers of CoinGecko or Twelve Data cover thousands of requests per day.

What I'd charge: $5/month basic, $15/month with portfolio tracking. With 50 paid users that's $750/month.

Tech: Python + python-telegram-bot + any crypto API. Runs on a cheap VPS or a home server.

2. AI Customer Support Bot ($50–500/month per client)

What it does: Handles customer questions using an LLM trained on the client's FAQ and documentation.

Why it actually works:
Small businesses pay $15–25/hour for human support. Their support reps answer the same 20 questions every day. A bot handles 80% of that. The business keeps humans only for the edge cases.

The pitch practically writes itself: "Your customers get instant answers at 3 AM. You save $2,000/month in support costs."

What I'd charge: $300–500 setup + $100–200/month retainer. One client pays for a month of work.

Key detail: Running this on Ollama locally eliminates API costs entirely. Qwen 3.5 9B handles most customer service scenarios well.

3. Content Scheduler Bot ($10–30/month per user)

What it does: Channel owners send content to the bot, set a schedule, and it auto-posts at the right time.

Why it actually works:
Here's a gap I noticed: Telegram scheduling tools are surprisingly weak. Buffer and Hootsuite barely support Telegram. Channel operators with 10K+ subscribers are either posting manually at odd hours or using jank workarounds.

Built-in analytics (which post got the most views) would be a huge differentiator.

What I'd charge: 1 channel free, 5 channels $10/month, unlimited $30/month. The free tier does the marketing for you — channel owners recommend it to other channel owners.

4. Group Moderation Bot ($100–300 one-time)

What it does: Anti-spam, welcome messages, auto-moderation, member verification, analytics for large Telegram groups.

Why it actually works:
Bots like Combot and Rose exist, but they're generic. Large communities — gaming clans, trading groups, local neighborhood chats — have specific rules and specific problems. A custom bot fits their exact workflow.

You're not competing with Combot. You're selling "exactly what your community needs."

What I'd charge: $100–300 one-time, plus $10–20/month optional hosting. The hosting is passive income that compounds.

5. Notification Bridge Bot ($200–500 one-time)

What it does: Bridges alerts from external systems (servers, IoT sensors, security cameras, any API) into Telegram.

Why it actually works:
DevOps engineers want server alerts in Telegram, not email. Smart home tinkerers want camera motion alerts on their phone. Small businesses want order notifications in their team chat.

This is pure infrastructure work — not glamorous, but people pay well for it because it saves them hours of setup.

Server monitoring (Prometheus)
  → Webhook fires on alert
  → Bot sends formatted message to Telegram group
  → Team sees "🔴 Server CPU at 95%" instantly

What I'd charge: $200–500 per project depending on complexity. These are usually one-day builds.

What All of These Have in Common

Looking back at what actually converts:

They solve annoyances people already have — not problems you invented
Low infrastructure cost — most run on a $5 VPS or a home server with Ollama
Clear ROI for the buyer — "you save X dollars" or "saves you Y hours" is an easy sell
Telegram's built-in payments make subscriptions surprisingly easy to implement

The biggest mistake I see: people build a bot first, then look for users. Do it backwards. Find someone who already has the problem, ask what they'd pay, then build it.

Where to Start

Pick the one that matches a community you're already in. If you're in crypto Twitter, start with the alert bot — you'll have early users from day one. If you do DevOps, the notification bridge is a weekend project that pays for itself immediately.

The MVP doesn't need to be fancy. Mine were usually 200 lines of Python and a weekend of work.

What bots have you built that actually made money? Curious to hear what's worked for others.

I Built a Custom Garmin Watch Face With Live Stock Sparklines — Here's What I Learned

Sam Hartley — Mon, 09 Mar 2026 08:01:24 +0000

Last spring I got annoyed enough at constantly pulling out my phone to check stock prices that I decided to just... put them on my watch.

Not as notifications. Not as a widget. A proper watch face with live sparkline charts for 5 configurable assets — crypto, stocks, commodities — updating every 15 minutes automatically.

Here's what building StockFaceTC for the Garmin Venu 2 Plus actually looked like.

The Language Nobody Talks About: Monkey C

First thing you'll notice: there's basically no community around Garmin development compared to iOS or Android. Stack Overflow has maybe 200 questions. The official forums are a ghost town. The documentation has dead links.

The language itself — Monkey C — is a mix of Java and JavaScript with Garmin-specific APIs bolted on. It's fine. The runtime constraints are the real challenge:

124KB total memory for the watch face
64KB separate sandbox for background services
No npm, no frameworks, no familiar tooling

The simulator is decent but has one critical limitation: it can't make real web requests. You test your API calls on real hardware connected to a phone via Bluetooth, or you don't test them at all.

The Font Problem That Cost Me a Week

I wanted custom text for the sparkline labels. Garmin has a VectorFont API for this.

The Venu 2 Plus doesn't support it.

Neither do most Garmin devices, it turns out.

So I built PrimitiveFont — a complete vector font rendered entirely with drawLine() calls. Every character is defined on a 5×7 grid:

// "A" on a 5×7 grid
const CHAR_A = [0,6, 0,2,  0,2, 2,0,  2,0, 4,2,  4,2, 4,6,  0,4, 4,4];

Multiply by scale factor (sizePx / 7.0) and you have arbitrary text at any size.

I went further and added proportional widths (so "i" is narrow, "M" is wide), bold, italic, and — the fun one — arc text. Characters curved along a circular path for the ring labels around the chart.

The whole library is ~21KB. Fits comfortably.

The 64KB Background Sandwich

Here's the architecture problem: Garmin watch faces have a background service that runs separately from the main app, in its own 64KB memory sandbox. It can't share memory with the main face directly.

The only data transfer mechanism: Background.exit(dictionary) — pass a dictionary out, receive it in onBackgroundData() in the main app.

This means you can't just parse a full API response and hand it over. You have to:

Fetch the data
Extract only what you need (closing prices for 24 candles × 5 symbols)
Pack it into the tightest dictionary you can
Exit

My fetch loop handles 5 symbols sequentially, parses the JSON inline, and builds a compact model before exiting. The background service uses maybe 40KB peak. Close enough.

Getting Labels Right (Not by Hardcoding Them)

Early version I hardcoded display names: "XAU/USD" → "GOLD". Obviously terrible.

The Twelve Data API returns metadata with every response:

{
  "meta": {
    "symbol": "BTC/USD",
    "currency_base": "Bitcoin",
    "type": "Digital Currency"
  }
}

So now:

Stocks → meta.symbol → "AAPL"
Crypto → meta.currency_base → "Bitcoin" → "BITCOIN" (truncated to fit)
Commodities → meta.currency_base → "Gold Spot" → "GOLD"

Zero hardcoded aliases. Add any valid ticker, get the right label. Works for everything in their catalog.

Budget Arithmetic

Twelve Data's free tier gives you 800 API calls/day.

My setup: 5 symbols × 4 fetches/hour × 24 hours = 480 calls/day.

Fits with 320 to spare. No paid tier needed.

Gotchas I Wish Someone Had Documented

Properties are cached forever. Changing defaults in properties.xml after the first install does nothing. You have to "Reset All App Data" in the simulator menu. Spent an embarrassing amount of time on this.

makeWebRequest() is stubbed in the simulator. It runs without errors, triggers the callback with null data, and you wonder why your code is broken. It's not — you just need a real device.

getSettingsView() must be implemented. Even if you return null. Otherwise the settings option in Connect app stays greyed out and you'll waste 30 minutes thinking it's a permissions issue.

The simulator's memory limits aren't accurate. It'll run code that crashes on device. Always test on hardware before you call it done.

The End Result

The watch face has been on my wrist every day for about 4 months now. The sparklines are genuinely useful — you can see at a glance whether BTC had a rough night or AAPL is doing something interesting.

The hardest part wasn't the code. It was the documentation gaps and the "you'll just have to test on device" reality of Connect IQ development.

But it runs. Updates every 15 minutes. Uses a free API tier. And cost exactly $0 beyond the hardware I already owned.

If you're thinking about building a Garmin app: the platform is rough around the edges, but it's real code running on real hardware on your wrist. That's genuinely satisfying in a way that web apps aren't.

The source is on GitLab if you want to poke around. Happy to answer questions in the comments — there aren't many of us in the Garmin dev community, might as well help each other.

garmin #connectiq #monkeyc #wearables #devlog

I Ditched OpenAI and Run AI Locally for Free — Here's How

Sam Hartley — Sat, 07 Mar 2026 12:33:19 +0000

I Ditched OpenAI and Run AI Locally for Free — Here's How

I was spending ~$80/month on API calls. ChatGPT Plus, some Anthropic credits, the occasional Gemini Pro request. It adds up fast when you're prototyping things.

Then I discovered you can run surprisingly good models on hardware you probably already own. I've been running a fully local AI setup for about a month now, and my API bill went to zero.

Here's the exact setup I'm using.

The Hardware (Nothing Fancy)

My main inference machine is a desktop PC with an RTX 3060 (12GB VRAM). You can find these used for ~$150. That's it. No A100, no cloud GPU rental.

For context:

8B parameter models (like Qwen 3.5) run at ~40 tokens/sec on this card
30B parameter models (like Qwen 3 Coder) run at a comfortable ~12 tokens/sec
Even on a MacBook M1 with 16GB RAM, 8B models are perfectly usable

If you have any modern GPU with 8GB+ VRAM, or an Apple Silicon Mac, you're good.

Step 1: Install Ollama

This is the part that surprised me. No Docker, no conda environments, no dependency nightmares:

curl -fsSL https://ollama.com/install.sh | sh

On macOS, you can also brew install ollama. Windows has an installer. That's literally it.

Step 2: Pull a Model

# Good general-purpose model
ollama pull qwen3.5:9b

# Great for code
ollama pull qwen3-coder:30b

# Solid reasoning
ollama pull deepseek-r1:8b

Models download once (~5-18GB depending on size) and run forever. No recurring costs.

Step 3: Use It

Interactive:

ollama run qwen3.5:9b
>>> Explain the difference between async and parallel execution

Or via API — and here's the killer feature: it's OpenAI-compatible:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5:9b",
    "messages": [{"role": "user", "content": "Explain Docker in 3 sentences"}]
  }'

Any tool built for the OpenAI API works by just changing the base URL. I've connected it to VS Code extensions, custom scripts, even a Telegram bot.

What I Actually Use This For

This isn't theoretical. Here's what I run daily:

Code reviews — I pipe diffs through the 30B coder model. It catches things I miss, especially in languages I'm less familiar with
A Telegram bot — Runs 24/7 on a Mac Mini, answers questions using Qwen 3.5. Nobody can tell it's local
Document Q&A — RAG pipeline with local embeddings. Load PDFs, ask questions
Quick lookups — Instead of context-switching to ChatGPT, I just ollama run in the terminal

The Real Cost Comparison

	Monthly Cost	Privacy	Latency
OpenAI GPT-4o	$20-200+	Data leaves your machine	~1-3s
Local Ollama	~$5 electricity	Everything stays local	~0.5-2s

The electricity cost is real but negligible. My PC draws about 250W under GPU load, and I'm not running inference 24/7.

Things I Wish I Knew Earlier

GPU matters way more than CPU. A $150 used RTX 3060 is 10-15x faster than even a high-end CPU for inference
Start with smaller models. 7-9B models are shockingly capable. Don't jump to 70B thinking bigger = better — the speed tradeoff isn't worth it for most tasks
Different models for different jobs. I use the coder model for code, the reasoning model for analysis, and the general model for chat. Specialization matters
Make it a network service. Set OLLAMA_HOST=0.0.0.0 and every device in your house can use it
It works offline. Plane, cabin, whatever. No internet needed after the initial model download

Is It As Good As GPT-4?

Honestly? For 80% of what I was using GPT-4 for, yes. The 9B models handle everyday coding questions, text generation, and analysis just fine.

For the really hard stuff — complex multi-step reasoning, very long context — the cloud models still have an edge. But I find myself needing that maybe once a week. Not worth $80/month.

Try It

If you have a gaming PC or a recent Mac, you can be up and running in literally 5 minutes. curl | sh, ollama pull, ollama run. That's the whole setup.

The worst that happens is you wasted 10 minutes. The best? You save thousands of dollars a year and keep your data private.

What models are you running locally? I'm always looking for recommendations — drop them in the comments.