Forem: Vineeth N Krishnan

I upgraded our 2.5-year-old self-hosted Sentry without losing a single byte

Vineeth N Krishnan — Wed, 22 Apr 2026 05:34:38 +0000

I upgraded our 2.5-year-old self-hosted Sentry without losing a single byte

The server nobody touched

Every team has one. A server that works. A server that has been working for so long that everyone has collectively agreed to never look at it directly, the way you never look directly at the sun, or at the production database during a demo.

Ours was the Sentry box.

Self-hosted Sentry, version 23.9.1, installed in October 2023 on a Hetzner machine running Ubuntu 20.04. It collected errors from six applications — three TypeScript backends, a React frontend, a PHP monolith, and one project cryptically named "internal" that I am not going to explain.

It worked. It had been working. Nobody was going to touch it.

Then I touched it.

The crime scene

Before you upgrade anything on a production server, you check free -h. This is a tradition. It is also, in this case, a jump scare.

              total        used        free      shared  buff/cache   available
Mem:           30Gi        26Gi       454Mi       349Mi       3.5Gi       3.1Gi
Swap:           9Gi         9Gi         0Ki

Zero. Zero bytes of swap free. Not "low." Not "concerning." Zero. The server had been running like this for — checking uptime — 373 days. Over a year with no swap headroom at all, and nobody noticed because nothing had crashed yet. The server was basically doing a continuous trust fall into the OOM killer's arms, and the OOM killer just kept catching it.

The culprit was MySQL. Not Sentry's MySQL — Sentry uses Postgres. This was the old PHP monolith's MySQL, running on the same machine, calmly eating 41% of RAM (12.7 GB) like it was paying rent.

On top of that: 59 Docker containers, a monitoring agent (Netdata) consuming 3.6 GB of RAM for the privilege of watching the server struggle, and Docker Compose v2.21.0 — which was about to become a problem.

The plan: four hops and a runbook

Sentry self-hosted has this concept of hard stops — mandatory version checkpoints you cannot skip. If you are on 23.9.1 and the latest is 26.3.1, you do not just git checkout 26.3.1 and run the installer. You will get a polite error message, or more likely, a database migration that assumes tables exist from a version you never installed.

The upgrade path looked like this:

23.9.1 → 23.11.0 → 24.8.0 → 25.5.1 → 26.3.1
         (hop 1)    (hop 2)   (hop 3)   (hop 4)

Four hops. Each one means: stop Sentry, checkout the version tag, run the installer, wait for database migrations, start Sentry, verify it works, then move on to the next. Skip one, and you are in uncharted territory with your production error tracking.

I wrote a runbook. The runbook was 1,020 lines. For context, some of my entire side projects have fewer lines than that runbook. It had phase gates, verification blocks, trace templates, and a log format you were supposed to paste into a scratch file on the server.

Was the runbook overkill? We will find out.

Pre-flight: making the server survivable

The first problem was RAM. Running database migrations on a server with zero swap is how you get a partially-migrated Postgres database and a very bad afternoon. The mitigation plan was three steps:

Step one: Stop Netdata. Sorry, monitoring agent. You are consuming 3.6 GB to watch a server I am about to intentionally stress. You can come back later.

Step two: Create a temporary 4 GB swap file.

sudo fallocate -l 4G /swapfile-temp
sudo chmod 600 /swapfile-temp
sudo mkswap /swapfile-temp
sudo swapon /swapfile-temp

Step three: Shrink MySQL's buffer pool from 13 GB to 8 GB. This is the server equivalent of asking your roommate to please move their stuff so you can fit a couch through the door.

mysql -u root -e "SET GLOBAL innodb_buffer_pool_size = 8589934592;"

After all three: 9.8 GB available RAM, 5.1 GB swap free. The server could breathe.

Backup everything that matters

Before touching Sentry, I backed up:

Config files (.env, sentry.conf.py, config.yml with the Slack credentials)
A full Sentry export (orgs, projects, teams — 278 KB of JSON)
Seven Docker volumes as tarballs — Postgres alone was 7.6 GB compressed
All six DSN URLs (the connection strings every client app uses)
A snapshot of every running container

Total backup: around 8 GB. Most of the time was spent waiting for the Postgres tar to finish.

The DSN recording is the one you absolutely cannot skip. If those change, every client app needs a redeployment. They did not change. But you check anyway.

The hops

Hop 1 (23.9.1 → 23.11.0): Under ten minutes, but not without drama. The install script asked me if I wanted to send telemetry data to Sentry (the company). I did not have an environment variable set to skip this, so it just sat there, waiting for a y/n I was not expecting. A few minutes wasted staring at a frozen terminal before I figured out what it wanted. Lesson learned: REPORT_SELF_HOSTED_ISSUES=0.

Hop 2 (23.11.0 → 24.8.0): Maybe ten, twelve minutes. This one had a breaking config change — Django's cache backend switched from MemcachedCache to PyMemcacheCache. The options API is completely different between the two. If you do not edit sentry.conf.py before running the installer, things break. I knew about this from the planning phase, so I had the fix ready. The runbook earned its keep.

Between hop 2 and 3: Docker Compose needed upgrading. Sentry 25.5.1 requires Compose v2.32.2 minimum. We had v2.21.0. One curl and a sudo later, we had v2.32.4. Quick detour, nothing dramatic.

Hop 3 (24.8.0 → 25.5.1): Smooth. Under ten minutes. This is the version that introduced the taskbroker architecture (replacing Celery) and PgBouncer for connection pooling, but the transition does not complete until 26.x.

Hop 4 (25.5.1 → 26.3.1): Another ten-ish minutes, after a false start. The installer introduced a new interactive prompt asking about S3 nodestore migration. In non-interactive mode, read -p fails and kills the script. Fix: APPLY_AUTOMATIC_CONFIG_UPDATES=1. This is the kind of thing that makes you wonder why "non-interactive install" is not just a single flag.

Then the 503.

After docker compose up -d, the Sentry URL returned "Service Unavailable." I stared at the screen for what felt like a long time but was probably a couple of minutes, with the specific kind of calm that only appears when you have 8 GB of backups and know exactly where they are. The nginx container was restarting. The web container was healthy. It resolved itself. The runbook said "no action needed — just wait." The runbook was right.

How long did it actually take?

The upgrade itself — backups, four hops, blockers, the 503 scare, verification — took maybe two, three hours. The install scripts were about ten minutes each. The migrations I had been dreading ran without a sound. The real time sink was backing up the Postgres volume, which took almost half an hour on its own.

But that is not the real answer.

The real answer is that the planning took longer than the execution. I spent over four hours before I touched a single thing on that server. SSH in, document the exact state of everything — RAM, swap, disk, Docker versions, every running container, every volume, every config file. Then figure out the hop path, read every release note between 23.9.1 and 26.3.1, find out which hops have breaking config changes and which need newer Docker Compose. Then write the runbook. Then dry-run it in my head.

This server had years of error data — the kind you go back to when checking regressions. Team member accounts. Project configs. Slack integrations. Six applications relying on it. If a migration failed halfway through and corrupted the Postgres data, that history is gone. There is no "oops, let me try again." You either have backups that work or you have a very uncomfortable conversation with your team.

So yes, the upgrade took a morning. But the work that made it take only a morning? That took days.

No data lost. All six DSNs unchanged. Fifty-nine containers went in, seventy-eight came out.

The victory lap that wasn't

Here is where the story should end. Sentry upgraded. All six DSNs intact. Seventy-eight containers healthy. The new version came with the things we actually upgraded for — AI Agent Monitoring with MCP tracing, the new taskbroker architecture, PgBouncer, SeaweedFS nodestore. All of that was working out of the box.

But then I noticed Sentry Logs in the docs. Structured logging that goes directly into Sentry alongside your errors and traces. No separate ELK stack, no Grafana Loki, just logs flowing into the same place your exceptions already live. This was not part of the upgrade plan — Logs is a separate feature you have to enable manually. But now that I was on 26.3.1, I could.

The upgrade was done by late morning. By lunch I was already in a new terminal session trying to enable it.

First thing I did was grep for ourlogs in the Sentry config. Empty result. The feature flags did not exist.

Right. The install script from 26.3.1 added the base services but not the feature flags. You need to re-run ./install.sh with the right options for it to inject the ourlogs-* flags into your config.

Before re-running the installer, I actually paused and asked myself: "Does ./install.sh clear all my existing data, users, and config?"

I had run this script four times that morning. But somehow, the fifth time, on a server I had just spent hours carefully upgrading, the thought of running it again made me hesitate. The answer is no, it does not clear your data. I knew this. I asked anyway.

Re-ran the installer. Grepped again. Ten ourlogs-* feature flags now present in sentry.conf.py. The EAP items consumer container was running. The Logs page was available in the UI.

I clicked on it.

Empty.

"I did everything and the logs are not there."

I double-checked the feature flags. All enabled. Checked the containers. All healthy. Checked the Sentry UI. The Logs section was right there, accessible, with a nice empty state saying "no logs found."

Then the obvious thing hit me. None of my projects actually use the Sentry logger.

I had enabled the server-side feature. I had not shipped any code that actually sends logs to it. The kitchen was built, the stove was on, and nobody was cooking.

Enabling Sentry Logs on the server is step one. Step two is updating your application's Sentry SDK to use the new Sentry.logger API. You need enableLogs: true in your Sentry.init(), and a logger that calls Sentry.logger.info() / Sentry.logger.error() instead of (or alongside) console.log. Without that, the Logs page will sit there empty, politely waiting for data that never arrives.

A colleague had actually written the NestJS integration weeks ago — a SentryLogger class that extends NestJS's ConsoleLogger and forwards log, error, and fatal calls to Sentry. It was sitting in a branch. Waiting for the server-side feature to be turned on. Waiting for today.

We deployed it. The logs appeared. I took a screenshot.

Minutes later I was taking screenshots of logs flowing in.

From "I did everything and the logs are not there" to showing it off — that is the developer emotional arc in its purest form. Panic, confusion, realization, deployment, screenshot. In that order. Every time.

What actually changed

For anyone running self-hosted Sentry and considering the same upgrade, here is what went from 23.9.1 to 26.3.1:

Celery → taskbroker: The worker architecture is completely new. Celery workers are replaced by taskworker and taskscheduler containers.
PgBouncer: PostgreSQL connection pooling is now built in. One less thing to configure yourself.
SeaweedFS nodestore: Event data that used to bloat Postgres is now stored in an S3-compatible object store (SeaweedFS). This is the biggest architectural change. Existing data is transparently migrated on read.
Docker registry: Images moved from getsentry/ (Docker Hub) to ghcr.io/getsentry/ (GitHub Container Registry).
AI Agent Monitoring + MCP tracing: Track agent runs, tool calls, and MCP server interactions directly in Sentry. This was the main reason we upgraded.
Container count: 59 → 78. Yes, nineteen new containers. Your docker ps output now needs a wider terminal.
Sentry Logs (manual enable): Structured logging, directly in Sentry. Not part of the upgrade — you have to enable feature flags and re-run the installer separately. Worth it, once you actually ship the client code to use it.

The non-interactive install cheat sheet

If you take one thing from this post, take this. For every Sentry self-hosted version from 23.11.0 through 26.3.1, this is the incantation:

REPORT_SELF_HOSTED_ISSUES=0 \
APPLY_AUTOMATIC_CONFIG_UPDATES=1 \
./install.sh --skip-user-creation

Saves you from every interactive prompt I hit. Tape it to your monitor.

Was the 1,020-line runbook worth it?

Yes.

Not because the upgrade was complex — it turned out to be straightforward. But because when the Logs page was empty and I did not know why, and when the URL returned 503 and I did not know how long it would last, having a document that said "this is expected, here is what to check, here is when to worry" was the difference between a methodical next step and a panicked docker compose down.

The runbook was not overkill. The runbook was the reason the upgrade took a morning instead of a weekend.

And the Astro documentation site I built afterward to record everything? That one might have been overkill. But it does look nice.

The day I realised I had never tested a production backup

Vineeth N Krishnan — Wed, 22 Apr 2026 05:33:57 +0000

The day I realised I had never tested a production backup

TL;DR — I had run dozens of test-backup restore drills and felt safe. What I had never done was pull a real production snapshot, decrypt it, and bring it up in a sandbox. The day I finally did, it worked — even a big database came back clean. This post is the story, plus the small Docker Compose file I keep on my laptop for exactly this kind of check. No repo, no project, just a compose file. Copy it if you want it.

The cold little question

So there I was, one afternoon, coffee going cold, staring at my own Hetzner Storage Box.

Neat folders. Neat snapshots. A long green column of audit rows. Every cron had fired, every upload had a size, every week had a tick.

And a question at the back of my head that I had been avoiding for longer than I want to admit: have I ever actually restored a real one of these?

The honest answer was no. Not properly.

I want to be fair to myself here, because I had not been careless. Every new engine I added to my backup tool came with its own test drill — spin up a throwaway database, seed it with some rows, take a backup, bring it back up somewhere else, confirm the rows are there. I had run those drills many, many times. Every time I touched the audit layer I reran them. Every time I changed an adapter. I genuinely felt covered.

But test drills and a real production restore are not the same thing, and part of me knew it. The seed database is five megabytes. The real one is tens of gigabytes, schema-migrated across versions, touched by years of application code, full of columns I had not thought about in a long time. Some classes of problem only show up at that scale. And at some point you have to stop quietly trusting your drills and go pull the real snapshot.

So that afternoon, I did.

What actually happened

I opened the throwaway Docker Compose folder I keep on my laptop exactly for this — I call it backup-verify because that is what I am doing, nothing fancier than that. Picked the Postgres profile. Pulled the latest production snapshot out of the storage box, decrypted it, dropped the .dump file into ~/Downloads, and ran pg_restore from inside the container.

And it just worked.

The whole database came back clean. Every schema, every foreign key, every bytea column, every ancient jsonb field I had half-expected to be the thing that blew up. I opened pgAdmin, clicked around, pulled up a user I had onboarded earlier that week, checked their orders, checked their uploads, checked the payment rows — all of it, all readable, all in the shape the application expects.

I sat back for a minute. That was a good minute.

There is a particular quiet pride in watching a pipeline you built yourself — that you had only ever proven on tiny test data — land an actual production-sized restore on the first try. The drills had been telling me the tool was correct. This run told me the backups themselves are correct. Two different claims. I had been conflating them for months. Today they both turned out to be true, but that could very easily have gone the other way, and I would have found out at the worst possible time.

If you have been putting this off the way I was, tell me I am not the only one.

Right. Now, the lab I ran it through.

The compose file

This is not a service. There is no CLI, no scheduler, no diff engine. The whole thing is a single docker-compose.yml, about 175 lines of it, and a data/ folder it creates as it boots. It lives in one folder on my laptop and has never been a repo. That is it.

Seven database engines, each behind a Docker Compose profile, each paired with a companion admin UI:

services:
  postgres:
    image: pgvector/pgvector:pg17
    profiles: ["postgres"]
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: verify
    ports: ["55432:5432"]
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
      - ~/Downloads:/dumps:ro

  pgadmin:
    image: dpage/pgadmin4:latest
    profiles: ["postgres"]
    ports: ["5050:80"]
    depends_on: [postgres]

That block, six more times, for:

postgres with pgAdmin
mysql with phpMyAdmin
mariadb with its own phpMyAdmin (yes, separate — MariaDB has its own quirks)
mongo with Mongo Express
redis with Redis Commander
mssql with Adminer
elasticsearch with Kibana

Each profile is independent. docker compose --profile postgres up -d boots only Postgres and pgAdmin. --profile mongo boots only Mongo and Mongo Express. You never spin up the whole zoo at once, because you almost never need to.

Two small choices that make me actually use it

Two things in this file sound boring on paper but are the reason I open the folder instead of putting it off.

~/Downloads:/dumps:ro on every service.

The workflow collapses to this: pull the snapshot out of the storage box, let the .sql or .dump or .bson land in ~/Downloads like any other file, and inside the container it is already sitting at /dumps/myapp.dump, ready to feed into pg_restore or mysql or mongorestore. No docker cp. No volume gymnastics. No temp folders. The read-only flag is there so I cannot accidentally scribble junk back into my Downloads folder from inside a container.

Weird port numbers, on purpose.

Postgres → 55432
MySQL → 33306
MariaDB → 33307
Mongo → 37017
Redis → 56379
MSSQL → 11433
Elasticsearch → 19200

Every one of these sits a digit or two away from the default. The reason is petty but load-bearing. I run a real Postgres on 5432 for other projects. I do not want the verify lab colliding with it, ever. Ten minutes of picking weird ports on the day I built this have saved me from "wait, which database am I actually connected to right now" more times than I would like to admit.

Why I still refuse to automate the verify step

Every time I show someone this compose file, the suggestion is the same: "Why don't you wrap it in a CLI that does the restore, counts rows, and reports?"

I have thought about it. I am not going to, and the reason matters.

The compose file works because it forces me to open an admin UI and scroll. The moment I replace scrolling with row-count assertions, I will start getting green ticks on the day the backup is silently wrong. A checksum says the bytes match. A row count says the count matches. Neither of those says the orders.status enum still decodes correctly, or that a bytea column came back as a string, or that a timestamp came back in UTC when the application wants it in IST. Those are the problems you catch by putting human eyes on the data.

There is a clean split in this system I want to protect. The making of backups is automated, scheduled, audited, alerted, and I trust it. The checking of backups is manual, slow, and not my favourite way to spend a Sunday morning. That is correct. Verification is a ritual, not a job. If I automate it away, I will lose the one thing that actually makes me look.

The UIs are hilarious in aggregate

A small appreciation, because these tools are a time capsule of twenty years of database tooling:

pgAdmin is the serious one. Loads slowly, does everything, is the reason I keep using Postgres.
phpMyAdmin is twenty years of PHP in a trench coat, and somehow still the fastest way to eyeball a MySQL dump. I do not know how. I have stopped asking.
Mongo Express is sparse but it does its one job.
Redis Commander is what I use to remember whether a cached key is JSON or a raw string.
Adminer is the single-file PHP hero of my generation, now driving my MSSQL tab because Microsoft's own tooling refuses to make this easier.
Kibana is absurd overkill for "did the Elasticsearch restore work", but when the only dump you have is an ES snapshot, you use what the ecosystem hands you.

All of them are pinned to latest, because this is not production. If any of them break after a version bump, I delete ./data and move on. That is a luxury the rest of my infrastructure does not get.

What I would still add

Two small items on the list, nothing structural.

A helper script for Redis, because restoring an .rdb snapshot is the one place where "drop the file into /dumps and import it" is not quite enough — you have to stop the container, replace dump.rdb at the right path, fix permissions, and restart. Every other engine is a one-liner. Redis is the awkward cousin at the dinner table, and I keep re-googling the procedure.

And bacpac support for MSSQL. The current setup handles .bak fine, but bacpac has bitten people I know, and I do not want to be fighting sqlpackage on the day a client actually needs their data back.

Go do the boring thing

If you have a backup setup you trust, the honest question is whether "trust" is built on a test drill or on a real restore. If it is the drill, go pull a real snapshot tonight. Not next quarter, not next incident review — tonight. Boot a throwaway database. Put eyes on a table you know by heart. If the rows are where you expect, that is a feeling worth having. If they are not, you have caught it on your terms instead of the worst possible day.

There is no repo for this one. backup-verify is just a folder on my laptop with the compose file above in it and a .gitignored data/ directory it writes to. It is not going to become a published project — the moment it grows a README, someone will file an issue, and then it stops being the thing I can delete without guilt when it breaks. The snippet in this post is the whole thing. Copy it, change the ports that collide with yours, drop the engines you will never restore, and you are done.

Not going to pretend this was a perfect writeup. But if even one part of it nudges someone to finally open a backup they have been politely ignoring, then it was worth putting down. See you in the next one.

The second half of shipping a CLI: Homebrew tap, Scoop bucket, and the SHA dance

Vineeth N Krishnan — Wed, 22 Apr 2026 05:33:46 +0000

The second half of shipping a CLI: Homebrew tap, Scoop bucket, and the SHA dance

The gap nobody mentioned

So you publish a CLI to npm. You put the install line in the README — npm i -g ipwhoami. You tell a few friends. You move on with your life.

Then someone on a Mac opens a terminal, reads the README, and thinks, why do I need Node on my machine for a tool that just looks up an IP? Someone on Windows reads the same line and thinks, what is npm, exactly? Both of them close the tab. Not because the tool is bad. Because the install line is not the one they are used to.

This is the part of shipping a CLI that nobody warns you about. Getting the tool working is half the job. Getting it installed the way people on their specific OS actually install things is the other half.

For Mac, that means brew install. For Windows, that means scoop install. Both of them are one line. Both of them look boring from the outside. And setting them up turned out to be a lot more work than I thought it would be — two extra repos, a Ruby file I had never written before, a JSON manifest with PowerShell inside it, and a SHA256 dance that I broke a few times before getting it right.

This is the packaging side-quest for ipwhoami, written down honestly.

The tap, and the weirdly specific naming rule

The first thing I learned about Homebrew is that it has opinions. Strong ones.

If you want to publish a package under your own GitHub namespace, the repo holding your formula must be named homebrew-<something>. Not close to it. Not similar. Exactly that prefix. Because when a user types brew tap vineethkrishnan/ipwhoami, brew quietly goes looking for github.com/vineethkrishnan/homebrew-ipwhoami. If the repo is named anything else, nothing happens. No error worth reading. Just a polite failure.

So I made the repo. One file inside it that matters — Formula/ipwhoami.rb. Yes, Ruby. A language I have probably written three lines of in my entire career, for a package manager I use every other day without ever looking inside.

The good news is the formula itself is tiny.

class Ipwhoami < Formula
  desc "IP geolocation lookup from your terminal"
  homepage "https://github.com/vineethkrishnan/ipwhoami"
  url "https://github.com/vineethkrishnan/ipwhoami/archive/refs/tags/ipwhoami-v1.2.1.tar.gz"
  sha256 "8a85739e0a8ac46a6195241d458a4108a9a2c5f042cde90846719161021852ab"
  license "MIT"

  depends_on "node" => ">=18"

  def install
    libexec.install "bin", "src", "package.json"
    bin.install_symlink libexec/"bin/ipwhoami.js" => "ipwhoami"
  end

  test do
    assert_match "ipwhoami", shell_output("#{bin}/ipwhoami --version")
  end
end

The bad news is every one of those lines cost me a read of the docs before I fully understood what it was up to.

The url is not some release asset or a zip I uploaded somewhere. It is the tarball GitHub generates for you, automatically, for every tag you push. Free CDN, basically. The sha256 is how brew makes sure the file it downloaded is the one I signed off on. One byte off and the install fails loud, which is exactly what you want.

The install block was the part I spent most time on. You do not want to scatter loose bin/ and src/ folders into /opt/homebrew like some tragic zip file exploded there. The pattern is — dump everything into libexec (a sandboxed folder brew gives every package), then create a single symlink into the real bin so the user has ipwhoami on their PATH. That is what bin.install_symlink is doing in one line.

The test do block is optional, but I liked the idea of it. You run brew test ipwhoami and brew actually runs the installed binary with --version to confirm it is not silently broken. Shipping a tiny smoke test with the package felt like a nice thing to do.

And then you push all this, install it on a clean Mac, watch it pull down Node and then your CLI, and for a second you feel like you have figured this whole thing out. That feeling lasts until you remember you still have to do the same thing for Windows.

Scoop: same job, thankfully not Ruby

Scoop is Windows' answer to Homebrew. If you have not used it — it is a PowerShell-based package manager, and most of the developers I know on Windows prefer it to Chocolatey because the install model is simpler and does not need admin rights.

The relief with Scoop is that the manifest is JSON, not Ruby. I know it sounds petty, but after enough rounds of squinting at def install and wondering if I was supposed to close the block with end, opening a JSON file felt like coming home.

The repo is scoop-ipwhoami. The manifest is ipwhoami.json. Most of it is the same information as the Homebrew formula wearing a different outfit — same tarball URL, same SHA256 (here it is called hash), same pointer to Node as a dependency. The only two parts actually worth looking at are the installer block and the auto-update hooks at the bottom.

"installer": {
    "script": [
        "New-Item -ItemType Directory -Force -Path \"$dir\\libexec\" | Out-Null",
        "Copy-Item -Recurse \"$dir\\bin\" \"$dir\\libexec\\bin\"",
        "Copy-Item -Recurse \"$dir\\src\" \"$dir\\libexec\\src\"",
        "Copy-Item \"$dir\\package.json\" \"$dir\\libexec\\package.json\""
    ]
},
"checkver": { "github": "https://github.com/vineethkrishnan/ipwhoami" },
"autoupdate": {
    "url": "https://github.com/vineethkrishnan/ipwhoami/archive/refs/tags/v$version.tar.gz",
    "extract_dir": "ipwhoami-$version"
}

The installer is a small PowerShell script, because Scoop does not hand you a "symlink my file into bin" helper the way Homebrew does. You write the copying yourself — make a libexec folder, copy bin, src, and package.json into it. A bin array elsewhere in the manifest then registers libexec\bin\ipwhoami.js as a shim called ipwhoami, which is what puts it on the user's PATH. The mental model is identical to Homebrew, you just do more of it by hand.

checkver and autoupdate are nice touches Homebrew does not really have a clean equivalent of. You tell Scoop, hey, my latest version is whatever the latest GitHub tag says it is, and Scoop figures out new versions for users automatically. That word — automatically — matters a lot for what comes next.

The SHA chicken-and-egg

This is the part that tripped me up the most.

Both the Homebrew formula and the Scoop manifest need the SHA256 of the release tarball. But the tarball is generated by GitHub only after you push a tag. So the order of operations goes something like this:

Bump the version in package.json
Push a tag
Wait — GitHub generates the tarball
Download the tarball yourself
Compute its SHA256
Paste the SHA into both the formula and the manifest
Push those to the tap and bucket repos
Test the install on a real machine and pray

Skip step 3 and you have nothing to SHA. Guess the SHA and every install fails with a loud mismatch. Copy the SHA wrong — swap a zero with an O, miss a character at the end — every install fails the same way, except now you also do not realise it is you who broke it until somebody opens an issue.

I did this whole thing by hand the first time. And yes, I got it wrong. Pasted the SHA from the previous tag, pushed it to the tap, installed on a fresh machine, watched brew scream at me about mismatch. Great. Fine. Fixed it, pushed again, then forgot to run brew update before re-testing — so brew was still happily using the broken cached formula. I sat there wondering why my fix had not taken effect, until the penny dropped.

Has this happened to you too? You fix a thing, you are sure you fixed it, and then you spend ages confused because the cache on your own machine is lying to you. The debugging is never the broken thing. It is always something nearby that you had not thought about.

Anyway, this is the kind of dumb mistake you make exactly once, after which you decide a machine should be doing this job, not you.

Automating the whole thing so I never touch it again

The main ipwhoami repo uses release-please to handle versions. You write conventional commits, release-please keeps an open PR with the next version and changelog, and merging it cuts a tag. That part I had been using from the start.

What I added was a set of release-triggered jobs that run after the tag gets cut. The full dance now looks like this:

release-please opens a release PR. I merge it, a new tag gets pushed.
The workflow fires. It publishes to npm, builds a multi-arch Docker image, and then — the point of all this — updates the Homebrew tap and the Scoop bucket.
For each of those, the job sleeps a moment so GitHub can finish generating the tarball, then it curls the tarball, computes the SHA, checks out the tap or bucket repo with a cross-repo PAT, writes a fresh formula or manifest, commits, and pushes.

The one step worth looking at is the SHA-compute — the heart of the whole thing the CI is doing for me so I do not have to:

- name: Wait for tarball availability
  run: sleep 10

- name: Compute SHA256
  id: sha
  run: |
    TAG="${{ needs.release-please.outputs.tag_name }}"
    URL="https://github.com/vineethkrishnan/ipwhoami/archive/refs/tags/${TAG}.tar.gz"
    SHA=$(curl -sL "$URL" | sha256sum | cut -d' ' -f1)
    echo "sha256=$SHA" >> $GITHUB_OUTPUT

After that, the job checks out the tap repo with a cross-repo token, writes a fresh formula with the new URL and SHA baked in, and pushes. The Scoop updater does the same, just writes JSON instead of Ruby.

One thing that quietly matters here — RELEASE_PAT. The default GITHUB_TOKEN a workflow gets cannot push to a different repo. For cross-repo commits (the tap and the bucket both live outside the main repo) you need a fine-grained personal access token with contents: write on those two repos. I missed this on my first run. The job failed at the push step with a 403 I spent longer reading than I will admit.

Both jobs sleep for ten seconds before curling because GitHub's tarball generation is async — move too fast and you get a 404 or a zero-byte file. The sleep 10 is not elegant. It is honest.

After all this, my release loop turned into something much nicer. Merge the release PR. Everything downstream updates on its own. npm gets the new version. Docker Hub and GHCR get the new image. Homebrew users see it when they run brew update. Scoop users see it when they run scoop update. I do not touch anything.

What I would tell past me

If I was starting this whole side-quest over, three things would go differently.

One — write the release workflow first. Before the formula. Before the manifest. Get the automation scaffolded in a branch, even if the first version just prints the computed SHA to the log and exits. The whole hand-paste-a-hash phase I went through was avoidable. I just did not realise how cheap a GitHub Actions job is until I actually wrote one.

Two — test the first install on a clean VM or a fresh machine. I lost a good bit of time chasing "why is brew not seeing my changes" before realising my own Mac had a cached formula from my first broken upload. A clean box has no opinions. It shows you exactly what a stranger on the internet would see, which is all that actually matters.

Three — keep the tap and the bucket each in their own repo, even if it feels like two more repos to manage. Homebrew forces you to. Scoop does not, so I was tempted for a while to dump the Scoop manifest into the main CLI repo and save a repo. Glad I did not. Both of those repos are now boring, stable, and have not needed a single manual commit from me since the automation went in. That is exactly how I want it.

The install section of the README now reads:

# Mac
brew tap vineethkrishnan/ipwhoami && brew install ipwhoami

# Windows
scoop bucket add ipwhoami https://github.com/vineethkrishnan/scoop-ipwhoami
scoop install ipwhoami

# Anywhere
npm i -g ipwhoami

Three lines instead of one. Same CLI under the hood. But now Mac and Windows users do not need to know what npm is to use a tool someone built for them. That small thing ended up mattering more than I thought it would.

That is pretty much the packaging half of ipwhoami. The tap lives at github.com/vineethkrishnan/homebrew-ipwhoami, the bucket at github.com/vineethkrishnan/scoop-ipwhoami, and the release workflow driving both is in the main ipwhoami repo.

Okay, that is enough from me for today. If any of this saved you some time, that is the whole point of writing it down. Until the next one — take it easy.

Setting Up a MinIO CDN with Nginx Reverse Proxy on Docker

Vineeth N Krishnan — Wed, 22 Apr 2026 05:33:05 +0000

Setting Up a MinIO CDN with Nginx Reverse Proxy on Docker

TL;DR - You can run your own S3-compatible CDN on a single small server with MinIO, Nginx, and a free Let's Encrypt cert. The setup is straightforward. The only places that usually bite people are the Host header, body size limits, and buffering. Get those three right and the whole thing just works.

Why bother self-hosting a CDN?

AWS S3 plus CloudFront is great when someone else is paying the bill. For side projects, staging environments, or small production apps where you already have a VPS sitting around, a MinIO instance behind Nginx gives you the same S3 API with your own SSL, your own data, and a bill that does not surprise you at the end of the month. You also get full control - no region lock-in, no egress fees that scale faster than your traffic.

What we are building

Three boxes, one arrow each way:

Client makes a request to cdn.example.com over HTTPS
Nginx (port 443, SSL) terminates TLS and forwards the request
MinIO container (port 9000, localhost only) does the actual storage work

MinIO is never exposed to the public internet directly. Nginx is the only thing the outside world sees. We will point a subdomain like cdn.example.com at this whole setup.

Prerequisites

Before starting, make sure you have:

An Ubuntu or Debian server (anything recent, 22.04 LTS is fine)
Docker and Docker Compose installed
A domain with a DNS A record pointing to your server's IP
Ports 80 and 443 open on your firewall
Some idea of what a reverse proxy does (since you are here, I will assume yes)

Right. Let us get into it.

Step 1: Run MinIO with Docker Compose

Create a folder somewhere sensible - I usually go with /opt/minio - and drop this in as docker-compose.yml:

services:
  minio:
    image: minio/minio:latest
    container_name: minio
    restart: unless-stopped
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
      MINIO_SERVER_URL: https://cdn.example.com
      MINIO_BROWSER_REDIRECT_URL: https://cdn.example.com/console
    ports:
      - "127.0.0.1:9000:9000"
      - "127.0.0.1:9001:9001"
    volumes:
      - ./data:/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 10s
      retries: 3

A few things worth calling out here.

The 127.0.0.1: prefix on the port mapping is the important bit. Without it, Docker binds to 0.0.0.0 and MinIO ends up open to the whole internet. With it, only processes on the host machine can talk to those ports - which is exactly what we want, since Nginx is going to be the front door.

The MINIO_SERVER_URL tells MinIO what its public URL is. This matters for presigned URLs - MinIO needs to know what hostname to sign against, otherwise you get signature mismatches when clients try to use the URL.

The healthcheck is basic but enough. MinIO has a built-in liveness endpoint, and Docker will mark the container unhealthy if it stops responding.

Drop your credentials in a .env file next to the compose file:

MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=please-change-this-to-something-long-and-random

Then bring it up:

docker compose up -d
docker compose logs -f minio

You should see MinIO reporting healthy. If you curl http://127.0.0.1:9000/minio/health/live from the host, you should get a 200. So far so good.

Step 2: Configure Nginx as a reverse proxy

This is the part where most tutorials hand-wave. Do not skip the details here - the defaults are wrong for MinIO in a few specific ways.

Drop this into /etc/nginx/sites-available/cdn.example.com:

# Redirect plain HTTP to HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name cdn.example.com;
    return 301 https://$host$request_uri;
}

# The main SSL block
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name cdn.example.com;

    # Certbot will fill these in for you in Step 3
    # ssl_certificate     /etc/letsencrypt/live/cdn.example.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/cdn.example.com/privkey.pem;

    # Allow large uploads
    client_max_body_size 5G;

    # Do not buffer - MinIO streams, and buffering breaks large transfers
    proxy_buffering off;
    proxy_request_buffering off;

    # Stop Nginx from chunking responses that are already fine
    chunked_transfer_encoding off;

    # Timeouts that make sense for big files
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    proxy_read_timeout 300;
    send_timeout 300;

    location / {
        proxy_pass http://127.0.0.1:9000;

        # These headers are the difference between working and "SignatureDoesNotMatch"
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Now the why, because this is where I lost time the first time I did this.

client_max_body_size 5G - Nginx defaults to 1MB. One megabyte. If anyone tries to upload a 20MB image, Nginx rejects it before MinIO even sees the request, and the error message is unhelpful. Set this to whatever your biggest expected file is, plus some headroom.

proxy_buffering off and proxy_request_buffering off - by default Nginx buffers the whole request before passing it upstream. For a multi-gigabyte upload, that means Nginx writes the file to its own temp folder first, then sends it to MinIO. You pay twice in disk IO and you might run out of /tmp space. Turning buffering off makes Nginx stream the request straight through, which is how S3-compatible clients expect things to work anyway.

proxy_set_header Host $host - this is the one everyone forgets. S3 signatures are computed over a canonical request that includes the Host header. If the client signed the request against cdn.example.com but Nginx forwards it with Host: 127.0.0.1:9000, the signature MinIO computes will not match the one the client sent, and you will get SignatureDoesNotMatch errors that make no sense.

chunked_transfer_encoding off - MinIO already handles its own transfer encoding. Letting Nginx add another layer on top causes intermittent breakage with larger files, especially when clients use multipart uploads.

Timeouts - default Nginx timeouts are measured for serving HTML, not for pushing gigabyte files around. Bump them up.

Enable the site and reload Nginx:

sudo ln -s /etc/nginx/sites-available/cdn.example.com /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

If you have been down the "why is my presigned URL failing" rabbit hole before, you know the pain.

Step 3: Get an SSL certificate with Certbot

Let's Encrypt makes this painless. Install Certbot:

sudo apt update
sudo apt install -y certbot python3-certbot-nginx

Then issue the cert. The --nginx plugin will edit your Nginx config automatically and uncomment the SSL lines we left commented out above:

sudo certbot --nginx -d cdn.example.com

Follow the prompts, say yes to the HTTPS redirect (we already have one, but Certbot is smart enough to not duplicate). Certbot sets up a systemd timer for auto-renewal. Check it works with:

sudo certbot renew --dry-run

If that comes back clean, you are set - renewal will happen on its own every 60 days.

Step 4: Create a bucket and test

The MinIO console runs on port 9001, but we bound it to localhost. So you have two ways to reach it.

Option A - SSH tunnel:

ssh -L 9001:127.0.0.1:9001 you@your-server

Then open http://localhost:9001 in your browser. Log in with the root credentials from your .env file.

Option B - separate subdomain like console.cdn.example.com with its own Nginx block pointing to 127.0.0.1:9001. Do this if you access the console often. For a one-off bucket setup the tunnel is fine.

Inside the console, create a bucket - call it public-assets for this example. Then go to the bucket's Anonymous Access tab and add a rule allowing GetObject on public-assets/* for anonymous users. That makes it a public read bucket.

You can also do this from the mc CLI, which is usually faster:

# Install mc (MinIO client)
brew install minio/stable/mc   # or the appropriate install for your OS

# Configure it to point at your CDN
mc alias set cdn https://cdn.example.com admin your-password

# Make a bucket and set it public
mc mb cdn/public-assets
mc anonymous set download cdn/public-assets

# Upload a test file
mc cp ~/Downloads/test.jpg cdn/public-assets/

And now the moment of truth:

curl -I https://cdn.example.com/public-assets/test.jpg

You should get HTTP/2 200 back with proper content-type headers. If you do, congratulations - you have a working CDN.

If you do not, jump to the gotchas section below.

Step 5: Using it as a CDN from your app

Here is what actually using this looks like from a Node.js app. Install the AWS SDK:

npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

Then a minimal setup:

import { S3Client, PutObjectCommand, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

// point the SDK at your MinIO
const s3 = new S3Client({
  endpoint: "https://cdn.example.com",
  region: "us-east-1", // MinIO ignores this but the SDK insists
  credentials: {
    accessKeyId: process.env.MINIO_ACCESS_KEY,
    secretAccessKey: process.env.MINIO_SECRET_KEY,
  },
  forcePathStyle: true, // important - MinIO uses path-style, not virtual-host style
});

// upload a file
async function uploadFile(key, body, contentType) {
  await s3.send(new PutObjectCommand({
    Bucket: "public-assets",
    Key: key,
    Body: body,
    ContentType: contentType,
  }));
  return `https://cdn.example.com/public-assets/${key}`;
}

// generate a short-lived URL the browser can upload to directly
async function getUploadUrl(key, contentType) {
  const command = new PutObjectCommand({
    Bucket: "public-assets",
    Key: key,
    ContentType: contentType,
  });
  return await getSignedUrl(s3, command, { expiresIn: 900 }); // valid for 15 minutes
}

The forcePathStyle: true line is non-negotiable. MinIO serves buckets at /bucket-name/ paths, not as bucket-name.cdn.example.com subdomains. Leave that off and everything breaks in weird ways.

Presigned URLs are the pattern you want for browser uploads. The server signs the URL, hands it to the client, and the browser uploads directly to MinIO without the file ever passing through your app server. Saves you bandwidth and a lot of memory.

Common gotchas

I have hit all of these at least once. Saving you the time.

SignatureDoesNotMatch on presigned URLs. Ninety percent of the time this is the Host header. Make sure Nginx is forwarding Host: cdn.example.com and that MINIO_SERVER_URL in the compose file matches exactly. Mismatch between what the client signs and what MinIO computes equals failed signatures, every single time.

Large uploads failing at around 1MB or 10MB. That is Nginx's client_max_body_size. Bump it up. If uploads fail at bigger sizes (say 100MB+), check proxy_request_buffering is off - without that, Nginx buffers the whole thing and may run out of disk or hit proxy_max_temp_file_size.

Browser uploads failing with CORS errors. MinIO does not send CORS headers by default. Set them on the bucket:

mc anonymous set download cdn/public-assets
mc cors set cdn/public-assets cors.json

Where cors.json looks something like:

{
  "CORSRules": [{
    "AllowedOrigins": ["https://your-app.com"],
    "AllowedMethods": ["GET", "PUT", "POST"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }]
}

Do not expose port 9000 to the public. I mean it. MinIO on its own, without TLS, without a firewall, is not something you want on the open internet. Someone finds the admin password, your bucket is their bucket now. The whole reason we bound it to 127.0.0.1 in the compose file is to force all traffic through Nginx, which is the layer that actually has TLS, rate limiting, and logging. Keep it that way.

Wrap-up

This setup is solid for staging and small-to-medium production workloads - I have run it under real traffic without drama. If you are pushing heavy traffic, put Cloudflare in front of Nginx for the edge caching, or look at MinIO's distributed mode across multiple nodes. For most of us though, a single box with Nginx and MinIO is more than enough and costs roughly what a decent lunch does per month.

So that is where I will stop. If you have a different way of doing this, or hit a gotcha I missed, I genuinely want to hear it - drop me a note. Otherwise, see you when the next interesting problem shows up.

My .zshrc is 350 lines and I mass-replaced every core Unix command

Vineeth N Krishnan — Wed, 22 Apr 2026 05:32:53 +0000

My .zshrc is 350 lines and I mass-replaced every core Unix command

Every developer has a dirty secret. Mine lives in two files.

.zshrc — 350 lines. Organized, sectioned, commented like production code. The refined version. The tuxedo.

.bash_aliases — 68 lines. No sections. No comments. A hardcoded password on line 48. A meeting reminder with a typo I never fixed. The digital equivalent of a shoebox full of receipts from 2016.

Together, they are over 400 lines of shell configuration that I have carried — like a suitcase I refuse to unpack — across operating systems, machines, jobs, and an entire decade of my career.

This is the archaeology of my dotfiles.

The secret gist

Before we dig in, let me explain the preservation method.

Years ago, I was about to move to a new machine. The thought of leaving my config behind felt like abandoning a pet at a rest stop. Physical devices are a burden to carry around. Hard drives fail. Laptops get replaced. But a GitHub gist? A gist is forever.

So I created a secret gist. Pasted in my .bashrc, my .bash_aliases, and a handful of other snippets I'd collected — the kind of things a developer hoards "just in case." That gist is still there. Still secret. Still the first thing I pull on any new machine, before I even open a code editor.

New machine ritual:

Install the OS
Pull the gist
Then start working

Everything else is replaceable. The config is not.

The bash_aliases: a time capsule you can source

Here's the thing about my .bash_aliases file — it still gets loaded. Every single time I open a terminal in 2026, this line runs:

[ -f ~/.bash_aliases ] && source ~/.bash_aliases

That line has survived the migration from bash to zsh, from Linux to macOS, from PHP to TypeScript. It's the most resilient piece of code I've ever written, and it's not even code — it's a conditional file include.

And what does it load? Sixty-eight lines of a Linux PHP developer's entire worldview, frozen in amber.

The aliases that tell my whole career story

alias html="cd /var/www/html/"

If you know, you know. This is the alias of someone who typed cd /var/www/html/ forty times a day and finally snapped. Classic LAMP stack energy. I haven't had a /var/www/html/ directory in years, but the alias stays. It's heritage.

alias art="php artisan"

The Laravel era. Two characters instead of eleven. At my peak, I was running art migrate, art serve, art make:controller so often that php artisan felt like typing a formal letter when a text would do.

alias pst="sh ~/PhpStorm-172.3198.4/bin/phpstorm.sh"

This one sends me. I was launching PhpStorm from a shell script with a hardcoded version path. Not from a desktop shortcut. Not from a /usr/local/bin symlink. From ~/PhpStorm-172.3198.4/bin/phpstorm.sh. Like a gentleman.

alias screenup="xrandr --output eDP1 --rotate inverted"
alias screendown="xrandr --output eDP1 --rotate normal"

I had a Linux laptop where I needed to flip the screen. I don't remember why. I don't want to remember why. But I aliased it, because apparently I was doing it often enough to justify two entries.

alias open="xdg-open"

This is a Linux developer making their terminal feel like macOS. On Linux, open doesn't exist — you need xdg-open. So I aliased it. Years later, I moved to macOS where open actually works natively. The alias became unnecessary. It stayed anyway.

The ones I probably shouldn't show you

alias sql_mode="echo '********' | sudo -S ~/./set-sql-mode.sh"
alias stop_server="echo '********' | sudo -S ~/./stop_server.sh"
alias start_server="echo '********' | sudo -S ~/./start_server.sh"

Yes. That's a plaintext password. In a shell alias. Piped into sudo. I was young. I was reckless. I was tired of typing my password three times to restart Apache. Don't do this. I did this. We've all done this.

The typo that will outlive me

alias weekly='notify-send -t 5000 --icon=clock "Knock Kncok" "Agency weekly is about to start"'

"Knock Kncok." Not "Knock Knock." Kncok. This alias sent me a desktop notification before my weekly agency meeting, and for however long I used it, it greeted me with a typo. I never fixed it. It's still there. It will remain there until the heat death of the universe or until GitHub deletes my gist, whichever comes first.

The Shopware chapter

A solid chunk of the file is Shopware plugin development aliases:

alias bcl="bin/console cache:clear"
alias pcr="bin/console plugin:create"
alias pin="bin/console plugin:install --reinstall -c --activate"
alias pact="bin/console plugin:activate -c"
alias pli="bin/console plugin:list"

Ten aliases for one framework's CLI. That's not configuration, that's muscle memory committed to disk.

The .zshrc: the evolved form

If .bash_aliases is the archaeological dig, .zshrc is the modern city built on top of it. Three hundred and fifty lines. Organized into sections with dividers. Comments that explain things. The version of my config that I wouldn't be embarrassed to show in a blog post. So here we are.

I replaced every core command

At some point, I decided that the standard Unix tools weren't good enough. So I replaced them. All of them.

alias cat='bat --paging=never'
alias grep='rg'
alias find='fd'
alias cd='z'
alias ls='eza --icons --group-directories-first'

cat is now bat. grep is now ripgrep. find is now fd. cd is now zoxide. ls is now eza with icons and git status.

I have effectively replaced the core Unix experience. If you SSH'd into my machine and tried to use it, nothing would behave the way you expect. cat has syntax highlighting. ls has icons. cd remembers where you've been. It's not a terminal anymore — it's my terminal.

The fzf setup that took longer than some of my projects

export FZF_DEFAULT_OPTS='
  --height 40%
  --layout=reverse
  --border rounded
  --preview "bat --style=numbers --color=always --line-range=:200 {}"
  --color=bg+:#1a1b26,bg:#12131b,spinner:#8a96fd,hl:#f7768e
  --color=fg:#d5dbe8,header:#66ccf0,info:#e0c07b,pointer:#8a96fd
  --color=marker:#66da8e,fg+:#d5dbe8,prompt:#8a96fd,hl+:#f7768e
'

That's a custom color scheme. For a fuzzy finder. I color-matched my file search tool to my terminal theme. This is what happens when a developer has opinions about aesthetics and access to hex codes.

But the real power is in the functions built on top of it:

# Search file contents, preview with syntax highlighting, open in editor
function fzg() {
  local file line
  read -r file line <<< "$(rg --line-number --no-heading "$@" | \
    fzf -d ':' --preview 'bat --style=numbers --color=always \
    --highlight-line {2} {1}' | awk -F: '{print $1, $2}')"
  [[ -n "$file" ]] && ${EDITOR:-vim} "+$line" "$file"
}

Type fzg "TODO", and it ripgreps your entire project, shows results in a fuzzy finder with a syntax-highlighted preview pane, and opens the file at the exact line in your editor. Three tools chained together into one command. This is the kind of thing that makes you feel like a wizard the first time it works and completely unemployable on anyone else's machine.

The extract function everyone has

function extract() {
  if [ -f "$1" ]; then
    case "$1" in
      *.tar.bz2)   tar xjf "$1"     ;;
      *.tar.gz)    tar xzf "$1"     ;;
      *.zip)       unzip "$1"       ;;
      *.7z)        7z x "$1"        ;;
      # ... six more formats
    esac
  fi
}

I am convinced that every developer who has maintained a dotfile for more than two years has a version of this function. Nobody memorizes tar flags. Nobody ever will. We all just write this function once and carry it forever.

The little things that add up

alias weather='curl -s "wttr.in?format=3"'
alias myip='echo "Local: $(ipconfig getifaddr en0)" && echo "Public: $(curl -s ifconfig.me)"'
alias killport='function _killport(){ lsof -ti:$1 | xargs kill -9; }; _killport'
alias duh='du -sh * | sort -rh | head -20'

Weather in the terminal. IP address in one command. Kill whatever's hogging a port. See what's eating disk space. None of these are impressive on their own. Together, they're the reason I can't use anyone else's terminal without getting frustrated.

The line that bridges two eras

If I had to pick one line that captures this entire story, it's this:

[ -f ~/.bash_aliases ] && source ~/.bash_aliases

A zsh config file, on a MacBook, in 2026, loading a bash aliases file that contains cd /var/www/html/ and a hardcoded password from a Linux machine that no longer exists.

That's not configuration. That's identity.

The point

I've been a developer since 2012. I've changed languages, frameworks, operating systems, editors, jobs, and countries. Through all of it, the one constant has been a shell config that grew with me — from a handful of PHP shortcuts on a Linux laptop to a 350-line productivity setup on a MacBook.

Is it excessive? Probably. Is every line necessary? Definitely not. Could I start fresh with a clean .zshrc? Technically.

But I won't. Because every alias is a bookmark in my career. art is the Laravel years. html is the LAMP stack era. pst is the kid who launched his IDE from a shell script. And somewhere in a secret gist, there's a meeting reminder that still says "Knock Kncok."

I wouldn't change a character.

Dotfiles: Some things are too personal to share. This was one of them. Until now.

My family thinks WhatsApp can send anything. So I wrote a Python CLI.

Vineeth N Krishnan — Wed, 22 Apr 2026 05:32:12 +0000

My family thinks WhatsApp can send anything. So I wrote a Python CLI.

The weekend plot twist

It was a normal weekend. My older cousin was visiting. Lunch was done, tea was going, the mood was relaxed. I had one of those rare afternoons where nobody was asking me to "just fix one small thing" on their laptop.

And then someone in the room brought up the wedding video.

Not mine. My cousin's wedding. From 2005. Ripped from a DVD years ago. Sitting on some old hard drive I kept because apparently I am the family digital archivist now — a role I never applied for.

Another cousin was suddenly excited.

Cousin: Hey, can you send that wedding video to the family group?
Me: You mean the full video?
Cousin: Yeah yeah, everyone wants to see it. Nostalgia and all.

Innocent enough. Family group, nostalgia, share the video. Five-minute task. Except.

The .VOB situation

The file was in .VOB format. Three parts. Each one around 2 to 3 GB. Total: roughly 8 GB of early 2000s DVD goodness.

If you are under thirty, .VOB might not mean much to you. It is the container format DVD players used, back when DVDs were a thing people rented from shops. Old, not streaming friendly, and most modern phones look at it the same way your cat looks at a cucumber.

Modern phone stock video player meets a 2005 .VOB file:

Stock Video Player: Sorry, this file format is not supported.
Me: Of course it is not.

So step one was already impossible. Even if WhatsApp somehow let me upload 8 GB, nobody in the family would be able to play the thing on their phone anyway. WhatsApp also has a file size limit that is not 8 GB. It is not even close to 8 GB. You can google the exact number, but the point is — my 2005 wedding archive was not going through it.

Family tech support mode

My family group is the classic Indian family group. Some very young, some very old, most somewhere in the middle. The group has a very specific worldview which I will call WhatsApp University syndrome: whatever it is, someone will try to send it through WhatsApp.

A 40-page PDF? Send it in the group.
A 20 MB meme image? Group.
A video of the dog? Group.
An 8 GB three-part DVD rip from 2005? Also group.

Sending files to my family through WhatsApp is never a technical question. It is a faith question. They believe WhatsApp will handle it, the way previous generations believed the postman would definitely find the house just from a vague description like "opposite the old banyan tree, near the temple."

So when I said "the video is too big and in the wrong format, we can't send it through WhatsApp," what they actually heard was "I have personally failed the family."

Cousin offered the backup plan.

Cousin: Then just upload it to Google Drive no?
Me: Uhh.

Why I did not just use Google Drive

Look. Technically, Google Drive would have solved this. Upload the file, share the link, anyone in the family opens the link, Drive plays the video in the browser. Done. Move on.

But there is a small, very personal thing I have with giving Google one more file about my family. Google already knows an uncomfortable amount about me. My location history. My search history. My email. My calendar. My photos. Probably my blood type.

At this point I am pretty sure Google knows me better than my own wife. My wife does not remember every restaurant I have been to in the last six years. Google does. And now I was supposed to hand it a private 8 GB wedding video of a 2005 family event?

No thanks. This is a small thing, but it is my small thing.

There is also a purely technical annoyance on top of the privacy one. If I had used some online media conversion service, I would have had to:

Upload 8 GB to someone's server
Wait for their queue
Wait for the conversion
Download the converted file
Then upload it to Drive or wherever

That is a lot of babysitting for one video. And every step in that list is another server I do not control touching a private family file.

Then I looked at my laptop. And my laptop had something those online services did not know about me: it already had ffmpeg.

Wait. I can just do this myself.

Here is the moment this post becomes a project.

I have a unix system. I have ffmpeg. I can install any package I need. I can convert any media format into any other format right here in my terminal, without uploading my cousin's wedding video to a single outside server. The compute is free. The storage is mine. The privacy is obvious.

The only problem was: writing

ffmpeg -i input.VOB -c:v libx264 -crf 23 -preset medium -c:a aac -b:a 128k output.mp4

once is fine. Writing it three times for three parts is fine. But I do this kind of thing often enough — family members handing me random files in random formats — that I knew I wanted something slightly nicer than "remember the exact ffmpeg flags forever."

So over that same weekend, I started writing medix.

What medix actually is

Medix is a small Python CLI that wraps ffmpeg in a friendlier interface. The name came from "Media to X" — you give it a media file, you pick X, it gives you X. Short, cli-friendly, does not sound generic like mediaconverter or mcc. MediaX → medix.

The stack is honestly boring, which is the point:

Python 3.9+ — ffmpeg as a subprocess is trivial in Python, Python already ships on macOS and most Linux distros, and I already know the language. No runtime to install for most users.
click for the CLI plumbing — small, reliable, has been around forever.
rich for the terminal UI — tables, colours, actual progress bars that do not lie.
questionary for the interactive prompts — after you give it a path, it asks you which format, which codec, which preset, which resolution, all with keyboard arrows.
ffmpeg for the actual conversion. Medix is a polite interface; ffmpeg does all the real work.

A few things I am quietly proud of:

Auto-install prerequisites. If you do not have ffmpeg, medix detects your operating system, finds the right package manager (Homebrew, APT, DNF, Pacman, winget, Chocolatey, Scoop, and a few more) and offers to install it for you. You do not need to know what apt-get install ffmpeg is.
Batch mode. Point it at a folder, optionally add -r for recursive, and it converts every media file in it.
File discovery. Before doing anything, it scans your path and shows you a clean table of what it found — resolution, duration, size — so you know exactly what is about to happen. You can bail with Ctrl-C if you scanned the wrong folder.
Real progress bars. Not fake ones that jump from 0 to 99 and freeze. It reads ffmpeg's progress output and updates per file and overall.
--dry-run flag. Because sometimes you just want to see what it would do without actually running ffmpeg on 8 GB of your cousin's wedding.

You install it with pip install medix. It is on PyPI at pypi.org/project/medix, indexed on libraries.io at libraries.io/pypi/medix, and the source lives at github.com/vineethkrishnan/medix.

The quiet victory

The 2005 wedding video converted in about 15 minutes. Three parts, all merged into one clean MP4. The file size dropped enough to share comfortably. I uploaded the result, sent it in the family group, and then I just watched.

Within minutes the group was lighting up. Elderly aunts were sending voice notes saying "oh look at so-and-so, they were so young then." Uncles were zooming in on their own faces from twenty years ago. My cousin — the actual groom — sent three laughing-crying emojis in a row, which is the closest he gets to a standing ovation.

Nobody in that group knew what medix was. Nobody knew I had written a Python CLI over the weekend just to make this share possible. Nobody asked what format the file was in, or which package manager installed ffmpeg, or whether I ran it in batch mode. As far as they were concerned, I had sent a video to the group. That was the full extent of the technical appreciation.

And honestly? That was the best part.

When a tool you built disappears completely into the thing it was supposed to do, it means the tool worked.

Try it yourself

pip install medix
medix ./some-old-video.vob

Or point it at a whole folder:

medix ./wedding-dvd-rip -r

That is the whole setup. The repo is at github.com/vineethkrishnan/medix if you want to read the code, open issues, or suggest features. I do accept contributions on this one — unlike a certain Laravel 4 starter kit of mine from another era, which is probably the next blog post.

And if anyone in your family ever hands you an 8 GB .VOB file and asks you to "just send it in the group" — now at least you have a weekend project.

jquery.verticalScroll.js: a love letter to jQuery, written ten years later

Vineeth N Krishnan — Wed, 22 Apr 2026 05:32:01 +0000

jquery.verticalScroll.js: a love letter to jQuery, written ten years later

There's a moment every developer has had. You're browsing a website — usually Apple's — and something scrolls so smoothly that your brain short-circuits from "wow, that's beautiful" to "I bet I could build that" in under three seconds.

For me, it was Apple's iPhone launch page. Full-screen sections. Buttery vertical scroll. Pagination dots on the side like a quiet tour guide. I stared at it the way a dog stares at a squirrel.

And then I did what any reasonable developer would do in 2016: I opened a file, typed $.fn.verticalScroll = function(), and started building a jQuery plugin.

Ten years later, I'm still maintaining it. It has 13 pagination themes, 16 animations, TypeScript support, visual regression tests, and a documentation site. It also has zero GitHub stars. Not one. Not even from me.

This is the story of a plugin that nobody asked for, nobody uses, and I absolutely refuse to let die.

The project I can't remember building it for

Here's the thing about personal projects from a decade ago: I genuinely cannot tell you what I originally built this for. I don't even remember what I had for breakfast. What I do remember is the feeling — I wanted full-page vertical scrolling in my next project, fullpage.js existed but I wanted my own, and jQuery was the hammer that fit every nail in 2016.

So I built it. A lightweight plugin — pass it a container with <section> elements, call .verticalScroll(), and suddenly your page moves like an Apple keynote. Mouse wheel, keyboard arrows, touch swipe. Pagination dots that actually knew which section you were on.

It worked. I used it. I loved it. And then I had the thought that turns a weekend hack into a ten-year commitment:

"This should be open source."

Zero stars, zero promotion, zero shame

Let's talk about the elephant in the repo: zero GitHub stars.

Ten years. Not a single star. And I've never once been tempted to log into an alt account and give myself one. My philosophy is simple — if someone actually likes it, they'll speak out. Spread the word, don't manufacture it.

Is it a little absurd to maintain a jQuery plugin for a decade with zero external validation? Absolutely. But there's something honest about it. This project doesn't exist to pad a resume or farm engagement. It exists because I built something, I liked it, and I wasn't ready to let it rot.

Some developers have side projects with thousands of stars that they abandoned two months after launch. I have one with zero stars that I gave a TypeScript rewrite last week. I know which one I respect more.

Dressing a ten-year-old in a tuxedo

The original code was... let's say "young." If the first version was a newborn — tiny, adorable, running around naked and somehow still getting the job done — then what I've done over the years is slowly teach it table manners and put it in a suit.

The latest version? It's wearing a tuxedo:

13 pagination themes — Neon, Git Graph, Chain, Diamond, and nine others. Each one with a custom animation that matches. Because if you're going to have pagination dots, they should at least have personality.
SCSS 7-1 architecture — because managing 13 themes in a single CSS file is a war crime.
Playwright visual regression tests — every theme, every animation, pixel-checked. Because the one thing worse than a jQuery plugin nobody uses is a broken jQuery plugin nobody uses.
Docsify documentation site — full API docs, examples, theme previews.
Release-please automation — semantic versioning, changelogs, the works.
v3 alpha: TypeScript + ESM — yes, a jQuery plugin with TypeScript types and ES module support. In 2026. I know how that sounds.

Is it overengineered for what it is? Probably. But here's the thing — I didn't add all of this at once. Each upgrade came from a genuine desire to do the thing properly. If I'm going to maintain something for a decade, it might as well be something I'm proud to open the source of.

Will this ever be anything other than a jQuery plugin?

Honestly? The odds that jquery.verticalScroll.js becomes react-vertical-scroll or @vue/vertical-scroll are roughly the same as the odds that it gets its first GitHub star: nonzero, but I'm not holding my breath.

The v3 TypeScript rewrite was a modernization experiment. A "what if" exercise. Could the core logic live outside jQuery? Could it become framework-agnostic? The answer is yes, technically. But "technically possible" and "something I'll actually do" are two very different things in the side-project economy.

More likely, this plugin will get another quiet upgrade in 2036. jQuery 5 will exist by then (probably). I'll add Web Component support or something. The star count will still be zero. And I'll still be fine with that.

The point

Every developer has a project like this. The one that doesn't make your portfolio shine. The one that nobody clones. The one that exists purely because you wanted it to exist, and you kept choosing to make it a little better instead of letting it decay.

jquery.verticalScroll.js is mine. Born because Apple made a pretty website and I had jQuery loaded. Raised through a decade of quiet commits. Dressed up in TypeScript because even old code deserves new clothes.

If you somehow find it, try it, and it saves you twenty minutes — that would genuinely make my day. And if you star it? Well. That would be a first.

Code: github.com/vineethkrishnan/jquery.verticalScroll.js

Hello World — Welcome to My Blog

Vineeth N Krishnan — Wed, 22 Apr 2026 05:31:20 +0000

Hello World

Welcome to my corner of the internet. I'm Vineeth, a Full Stack Developer with a love for building tools that solve real problems.

Why a Blog?

After years of writing code professionally, I've accumulated a lot of knowledge — patterns that work, mistakes that teach, and tools that save hours. This blog is my way of sharing all of that.

What to Expect

Technical deep dives into TypeScript, Go, Rust, and Python
CLI tool development tips and patterns
DevOps and infrastructure insights
Lessons learned from production systems
Open source project walkthroughs

Let's Connect

If you find something useful here, or want to discuss any topic, feel free to reach out through my GitHub or drop me an email.

Thanks for stopping by. More posts coming soon.

diskdoc and dockit: same problem, two languages, different answers

Vineeth N Krishnan — Wed, 22 Apr 2026 05:31:09 +0000

diskdoc and dockit: same problem, two languages, different answers

I built two tools that clean up disk space. Both are CLIs. Both deal with Docker. Both ship as a single binary. And if you put them side by side, they look like the same project in two languages.

They're not.

diskdoc is a Rust TUI that walks your entire filesystem in parallel, classifies what it finds — logs, caches, Docker artifacts, build output — and lets you browse and delete interactively. It answers the question "where did my disk space go?"

dockit is a Go CLI that talks directly to the Docker daemon, scores every resource by deletion risk, and gives you a cleanup plan. It answers the question "which Docker resources can I safely delete?"

One is wide and visual. The other is narrow and opinionated. Building both taught me things about Rust, Go, and tool design that building either alone wouldn't have.

This is the third post in a series on my open-source projects. The first covered backupctl, the second covered agent-sessions. This one is a two-for-one.

Why two tools

The honest version: I didn't plan to build two.

diskdoc came first. I wanted to learn Rust, and "build a disk usage analyzer" is the kind of project that forces you to deal with the filesystem, concurrency, error handling, and a real UI — all the things that make Rust interesting and hard. I chose it as a learning vehicle, and the tool itself was the byproduct.

It worked well for general disk analysis. But when I used it to investigate Docker disk usage specifically, the limitations were obvious. diskdoc sees Docker the way du sees Docker: a pile of directories under /var/lib/docker. It can tell you that the overlay2 directory is 40GB, but it can't tell you which images are eating that space, whether those images back running containers, or which ones are dangling. For that, you need to talk to the Docker daemon — not the filesystem.

dockit started there. Not "let me rebuild diskdoc in Go" but "let me build the Docker-specific tool that diskdoc can't be." Go was the natural fit: Docker's own SDK is in Go, the tooling ecosystem is mature, and a single static binary is the default output, not a build target you have to configure.

Two tools, two languages, two scopes. But the interesting part isn't the difference in scope — it's how the language shaped the design in ways I didn't expect.

Rust shaped diskdoc into a TUI

Rust's type system and ownership model made me think about the program as a state machine before I wrote any UI code.

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AppMode {
    Scanning,
    Browsing,
    DeleteConfirmation,
    Dashboard,
    DashboardCleanupConfirmation,
    About,
}

Six states. Every keypress is a transition between them. The App struct holds the current mode, the file list, the selection index, and the injected dependencies — and nothing else. There's no "accidentally in two states at once" because AppMode is an enum, not a pair of booleans.

The TUI followed naturally. If you already have an explicit state machine, rendering becomes a pure function: take the current state, produce a frame. Ratatui (the Rust TUI library) is designed exactly for this — you call terminal.draw(|f| draw(f, &app)) in a loop, and the draw function pattern-matches on the mode to decide what to show.

loop {
    terminal.draw(|f| tui::draw(f, app))?;

    if crossterm::event::poll(Duration::from_millis(100))? {
        if let Event::Key(key) = crossterm::event::read()? {
            match app.mode {
                AppMode::DeleteConfirmation => match key.code {
                    KeyCode::Char('y') | KeyCode::Enter => app.confirm_delete(),
                    KeyCode::Char('n') | KeyCode::Esc => app.cancel_delete(),
                    _ => {}
                },
                // ... other modes
            }
        }
    }

    app.on_tick();
}

100ms polling. Drain up to 100 scan events per frame. Render. Repeat. The TUI stays responsive while the scanner runs on a separate thread, sending results through an mpsc channel.

I don't think I would have built a TUI in Go. Not because Go can't do it — it can — but because Rust's enums and match exhaustiveness made the state machine feel safe to extend. Every time I added a new mode, the compiler told me every place I needed to handle it. In Go, I'd have been maintaining that discipline manually with a switch statement and hoping I didn't miss a case.

Go shaped dockit into an opinion

Go didn't push me toward a TUI. It pushed me toward decisions.

The Docker SDK is a Go library. The standard CLI toolkit (Cobra) gives you subcommands with flags. The ecosystem defaults are: structured output, composable commands, --json for automation. So dockit became a tool with opinions instead of a tool with a canvas.

The biggest opinion is the risk scoring system:

const (
    ScoreSafe      Score = "SAFE"
    ScoreReview    Score = "REVIEW"
    ScoreProtected Score = "PROTECTED"
)

Every Docker resource — container, image, volume — gets classified. Running containers are PROTECTED. Dangling images older than 7 days are SAFE. Stopped containers created last week are REVIEW. The scorer is 60 lines of Go, and it encodes a real position: recently-created resources deserve a second look, even if they're unused.

func (s *Scorer) ScoreContainer(c *models.Container) {
    if c.State == "running" || c.State == "restarting" || c.State == "paused" {
        c.Score = models.ScoreProtected
        c.Reason = "Container is currently active"
        return
    }

    age := time.Since(c.Created)
    threshold := time.Duration(s.Config.ReviewDays) * 24 * time.Hour

    if age < threshold {
        c.Score = models.ScoreReview
        c.Reason = "Container stopped, but created recently"
        return
    }

    c.Score = models.ScoreSafe
    c.Reason = "Container is stopped and old"
}

This is what docker system prune doesn't do. prune is a binary: delete or don't. dockit's scoring system introduces a middle ground — "you should probably look at this before I delete it" — which is the actual mental model most developers have when cleaning up Docker resources.

The cleanup command defaults to dry-run. You have to pass --apply to actually delete anything, and even then it asks for confirmation:

if !applyCleanup {
    plan.PrintDryRun()
    return nil
}

diskdoc has confirmation dialogs too, but they're modal — you select an item, press d, see the path, confirm. dockit shows you the entire plan before you touch anything. Different UX, different safety model. The modal works for interactive browsing. The plan works for auditing.

The parallel scanning story (Rust)

The piece of diskdoc that made me appreciate Rust the most is the filesystem scanner.

pub fn start_scan(root: &Path, tx: Sender<ScanEvent>) {
    let root_path = root.to_path_buf();

    thread::spawn(move || {
        let walk = WalkDir::new(&root_path)
            .skip_hidden(false)
            .sort(true)
            .parallelism(Parallelism::RayonNewPool(4));

        for entry in walk {
            match entry {
                Ok(dir_entry) => {
                    let path = dir_entry.path();
                    let meta = dir_entry.metadata();
                    // ... classify and send
                    if tx.send(ScanEvent::NewEntry(stats)).is_err() {
                        break; // Receiver dropped
                    }
                }
                Err(e) => {
                    let _ = tx.send(ScanEvent::Error(e.to_string()));
                }
            }
        }
        let _ = tx.send(ScanEvent::Complete);
    });
}

jwalk is the crate doing the heavy lifting — it wraps Rayon to walk directories across 4 threads, and the results feed into an unbounded mpsc channel. The TUI's on_tick drains up to 100 events per frame, so the UI stays responsive even when the scanner is churning through millions of files.

What I like about this code is what it doesn't need. No mutex around the file list. No lock on the scan state. The channel is the synchronization primitive, and Rust's ownership model guarantees that the sender and receiver can't race on shared data because there is no shared data. The scanner owns its thread. The UI owns the App struct. They communicate through messages.

In Go I would have reached for a goroutine and a channel, which is structurally similar — but the safety guarantee would be convention, not compilation.

The Docker SDK story (Go)

dockit's advantage is that it doesn't guess about Docker. It asks.

func (c *Client) GetContainers(ctx context.Context) ([]*models.Container, error) {
    rawContainers, err := c.api.ContainerList(ctx, types.ContainerListOptions{
        All: true, Size: true,
    })
    // ...
}

Size: true is the detail that matters. Without it, the Docker daemon returns containers without calculating their writable layer size — which makes the output useless for disk analysis. With it, the daemon does the work (it takes a bit longer), and you get real numbers.

But the SDK has quirks. Image.Containers — the field that tells you how many containers use an image — is unreliable. ImageList returns -1 for this field unless you've recently called /system/df. So dockit does its own cross-reference:

func Analyze(containers []*models.Container, images []*models.Image, volumes []*models.Volume) *CorrelatedData {
    imgUsage := make(map[string]int64)
    for _, c := range containers {
        imgUsage[c.ImageID]++
    }

    for _, img := range images {
        localCount := imgUsage[img.ID]
        if localCount > img.Containers {
            img.Containers = localCount
        }
    }

    return &CorrelatedData{Containers: containers, Images: images, Volumes: volumes}
}

Build a map of image ID to container count. If the local count is higher than what Docker reported, use the local count. This is the kind of detail you only discover by testing against real Docker daemons — the SDK documentation doesn't warn you.

diskdoc can't do any of this. It sees /var/lib/docker/overlay2 as a directory and reports its size. Useful, but it can't tell you that half of those layers belong to a dangling image you pulled six months ago and forgot about.

The heuristics chain (Rust) vs the scorer (Go)

Both tools need to classify what they find. They solve this differently, and the difference reveals something about what each tool values.

diskdoc uses a chain-of-responsibility pattern with pluggable heuristics:

pub struct HeuristicsEngine {
    heuristics: Vec<Box<dyn Heuristic + Send + Sync>>,
}

impl HeuristicsEngine {
    pub fn new() -> Self {
        Self {
            heuristics: vec![
                Box::new(LogHeuristic),
                Box::new(NpmHeuristic),
                Box::new(ComposerHeuristic),
                Box::new(AptHeuristic),
                Box::new(CacheHeuristic),
                Box::new(DockerHeuristic),
            ],
        }
    }

    pub fn analyze(&self, path: &Path, is_dir: bool) -> FileType {
        for h in &self.heuristics {
            if let Some(t) = h.detect(path, is_dir) {
                return t;
            }
        }
        FileType::Normal
    }
}

First match wins. Order matters — NpmHeuristic runs before CacheHeuristic so that .npm/_cacache gets flagged as npm cache, not generic cache. Each heuristic is a trait implementation, so adding a new one means writing a struct with a detect method and inserting it at the right position in the chain.

This works for filesystem analysis because the input is a path. Paths are simple. You can match on extensions, directory names, well-known locations. The heuristic doesn't need context beyond the path itself.

dockit's scorer is different. It doesn't look at paths — it looks at state:

func (s *Scorer) ScoreImage(img *models.Image) {
    if img.Containers > 0 {
        img.Score = models.ScoreProtected
        img.Reason = "Image is currently backing a container"
        return
    }

    if !img.Dangling {
        img.Score = models.ScoreReview
        img.Reason = "Image specifies a repository/tag but is unused"
        return
    }

    age := time.Since(img.Created)
    if age < threshold {
        img.Score = models.ScoreReview
        img.Reason = "Image is dangling but was created recently"
        return
    }

    img.Score = models.ScoreSafe
    img.Reason = "Image is dangling and old"
}

The scorer knows about relationships (does this image back a container?), temporal context (how old is it?), and naming (is it dangling?). It's not matching strings — it's reasoning about the resource graph. This is only possible because dockit has the Docker SDK, which gives it the full picture.

The heuristics chain is extensible. The scorer is opinionated. Both are right for their context.

The runaway log problem

One of dockit's most practical features is dockit logs — it finds containers whose log files are silently eating disk.

func (c *Client) GetLogMetrics(ctx context.Context) ([]*models.LogMetrics, error) {
    for _, cnt := range containers {
        info, err := c.api.ContainerInspect(ctx, cnt.ID)
        if err != nil {
            continue
        }

        m := &models.LogMetrics{
            ContainerID:   cnt.ID,
            ContainerName: name,
            LogPath:       info.LogPath,
        }

        if info.LogPath != "" {
            stat, err := os.Stat(info.LogPath)
            if err == nil {
                m.LogSize = stat.Size()
            }
        }

        metrics = append(metrics, m)
    }
    return metrics, nil
}

Docker stores container logs at /var/lib/docker/containers/{id}/{id}-json.log. If a container writes without rotation — no --log-opt max-size — that file grows until the disk fills. I've seen 30GB log files from a single container that printed debug output nobody was reading.

The trick is the ContainerInspect call. The container list doesn't include the log path — you have to inspect each container individually to get info.LogPath, then os.Stat the file. It's an N+1 query pattern, but for Docker containers (usually dozens, not thousands) it's fine.

diskdoc would see the same files, but it would flag them as generic log files in /var/lib/docker. It wouldn't know which container produced them, whether that container is still running, or whether the log driver is misconfigured. Context matters.

What the comparison taught me

Language shapes design more than you expect. I didn't choose to build a TUI in Rust and a CLI in Go because I sat down and evaluated the tradeoffs. Rust's type system made a state machine feel natural, and ratatui was right there. Go's Docker SDK and Cobra ecosystem made structured CLI output feel natural, and --json was right there. The languages didn't force the designs, but they made certain designs frictionless — and friction is what kills side projects.

Scope is a design decision, not a limitation. diskdoc's broad scope (all files, all types) means it's useful everywhere but shallow on any specific domain. dockit's narrow scope (Docker only) means it's useless outside Docker but deep where it matters — risk scoring, log detection, SDK-level introspection. Neither is better. They serve different moments.

Safety has more than one shape. diskdoc uses confirmation modals: select, press delete, confirm. dockit uses a three-layer system: classify, plan, confirm. The modal works when you're browsing and spot something to delete. The plan works when you want to audit before you act. I use both, for different situations.

Two small tools beat one clever tool. I could have tried to merge diskdoc and dockit — a Rust TUI that also talks to the Docker daemon. It would have been harder to build, harder to test, harder to release. Instead I have two binaries that do one thing each. The Unix way isn't always the right way, but for developer tools that solve a specific itch, it usually is.

Where to go from here

Both tools are open source:

diskdoc (Rust): github.com/vineethkrishnan/diskdoc
dockit (Go): github.com/vineethkrishnan/dockit

This is the third post in the series. If you're curious about the projects or want to tell me why I should have written both in Zig, find me on GitHub.

dfree: stop digging with your hands, you've got an axe now

Vineeth N Krishnan — Wed, 22 Apr 2026 05:30:28 +0000

dfree: stop digging with your hands, you've got an axe now

There's a special kind of panic that hits when someone pings you at work and says:

"Vineeth, we can't upload anymore."

Not "something is slow." Not "there's a bug." Just — the server is full. Done. No more room at the inn.

The setup was innocent enough. A small internal hosting for a knowledge base and academy videos. Upload feature included — so the customer support team could add training materials themselves. Worked great. Worked too great. Turns out, when you give people an upload button and no quota, they will fill your disk with the enthusiasm of a golden retriever discovering a mud puddle.

So there I was, SSH'd into the server, running du -sh /* like a caveman, trying to figure out which directory ate 90% of the disk. Docker images from six months ago? Still there. npm cache from a build pipeline nobody remembers? Thriving. Log files older than some of my git repositories? Absolutely.

I needed a tool. Not a monitoring dashboard. Not an alert system. Just something that could look at a server and say: "Here's what's eating your disk. Want me to clean it? Yes or no."

That's dfree.

One command. That's it.

dfree

That's the entire interface. Run it, and it scans your system in four passes:

=== System Analysis ===

[INFO] Scanning disk usage...
500G 387G 113G 78%

[INFO] Scanning Docker usage...
Images: 12.4GB (8.2GB reclaimable)
Containers: 1.1GB (900MB reclaimable)
Build Cache: 5.6GB

[INFO] Scanning Developer Caches...
  - /home/vineeth/.npm/_cacache: 1240MB
  - /home/vineeth/.cache/pip: 680MB
  - /home/vineeth/.cargo/registry/cache: 340MB
  - /home/vineeth/.gradle/caches: 2100MB

That Gradle cache sitting at 2.1GB and contributing absolutely nothing to society? Yeah. dfree found it.

It asks before it touches anything

This is the part I care about most. dfree will never delete something without asking first. Every single action gets a confirmation:

=== Cleanup Process ===

Prune Docker system (images, containers, networks)? [y/N] y
[INFO] Pruning Docker...
Total reclaimed space: 9.1GB

Clean system cache at /var/cache/apt/archives? [y/N] y
Clean system cache at /var/log/journal? [y/N] n
Clean developer cache at /home/vineeth/.npm/_cacache? [y/N] y
Clean developer cache at /home/vineeth/.gradle/caches? [y/N] y

Empty Trash? [y/N] y

[SUCCESS] Cleanup complete.

No flags to memorize. No config files to write. Just a conversation between you and your disk, one directory at a time. Say yes to what you want gone, no to what you don't.

If you're the kind of person who doesn't trust a tool that says "I'll be careful" — good, me neither. That's what --simulate is for:

dfree --simulate

[WARN] RUNNING IN SIMULATION MODE (No files will be deleted)
[INFO] SIMULATE: docker system prune -f
[INFO] SIMULATE: rm -rf /home/vineeth/.npm/_cacache
[INFO] SIMULATE: rm -rf /home/vineeth/.gradle/caches

All the analysis, none of the consequences. Like window shopping, but for disk space.

It knows what NOT to delete

Behind every rm -rf there's a safety check. dfree maintains a blocklist of paths that should never be touched:

# You literally cannot delete these, even if you try
/ /bin /usr /etc /boot /System /Applications

It also rejects relative paths (no ../../oops) and checks your exclusion list from .dfreerc. Because the only thing worse than a full disk is an empty one that used to have your operating system on it.

Custom rules for your mess

Every server has its own flavor of clutter. dfree lets you teach it yours with a ~/.dfreerc file:

# ~/.dfreerc

# "Please also look at these"
DFREE_CUSTOM_PATHS=("/data/academy/temp-uploads" "/var/tmp/build-artifacts")

# "Never touch these, I mean it"
DFREE_EXCLUDED_PATHS=("/data/academy/production-videos")

The custom paths show up in the analysis and get offered for cleanup. The excluded paths are invisible to the cleaner — dfree won't even ask about them. Because that /production-videos directory? That one is sacred. That one stays.

What it hunts

Here's the full hit list, depending on your OS:

Docker — dangling images, stopped containers, orphan networks, build cache. The stuff Docker hoards like it's preparing for winter.

System caches — /var/cache/apt/archives on Linux, ~/Library/Caches on macOS. The files your OS downloaded once and will never look at again but refuses to throw away, like a digital hoarder.

Developer caches — npm, yarn, pip, cargo, go-build, gradle. Six package managers, six caches, one combined guilt trip. That node_modules might be gone, but the cache remembers everything.

Logs — journal logs on Linux, ~/Library/Logs on macOS. Your system's diary, except nobody reads it and it never stops writing.

Trash — yes, the actual trash. Because "I'll empty it later" is a lie we all tell ourselves.

Works everywhere you SSH into

dfree auto-detects whether it's on Linux or macOS and adapts. Different paths for caches, different commands for disk stats, same interactive flow. You don't configure this — it just figures it out.

# Linux
get_system_cache_paths → /var/cache/apt/archives, /var/log/journal, ~/.cache

# macOS
get_system_cache_paths → ~/Library/Caches, ~/Library/Logs

One tool, two operating systems, zero flags to remember.

Install in 10 seconds

git clone https://github.com/vineethkrishnan/dfree.git
cd dfree
./install.sh

That's it. It's a shell script. It doesn't need a runtime, a package manager, or a prayer. If your server has bash, it has dfree.

The point

dfree isn't clever. It doesn't predict what you should delete. It doesn't auto-schedule. It doesn't send you a weekly report with pie charts.

It does one thing: it walks through your disk, shows you what's taking space, and asks — one item at a time — if you want it gone. That's it. That's the whole tool.

Because when your server is full and someone is waiting to upload the next training video, you don't need a dashboard. You need an axe.

Code: github.com/vineethkrishnan/dfree

Building mcp-pool: one week, eleven MCP servers, one shared OAuth library

Vineeth N Krishnan — Wed, 22 Apr 2026 05:30:17 +0000

Building mcp-pool: one week, eleven MCP servers, one shared OAuth library

This is another post in the series where I walk through my open-source projects. Earlier ones covered backupctl, agent-sessions, and a few smaller tools. This one is about mcp-pool — a monorepo of MCP servers for the SaaS tools I actually use at work.

It started as a Stripe MCP server. One package, one weekend. Somewhere along the way it grew into eleven — Stripe, Sentry, Notion, Linear, Datadog, Vercel, PagerDuty, HubSpot, Intercom, Shopify, Google Workspace — plus a shared OAuth library holding them together. Most of that happened faster than I'd planned, partly because the monorepo setup I did on day one kept paying off, and partly because the Claude Code sessions I used to scaffold each server got better every time I ran one.

The interesting part is not the package count. Pasting SDK code into eleven folders is not engineering. The interesting part is what happened a few days in, when I realised six of those servers were going to need OAuth and I was about to write more or less the same auth flow six times.

Why Stripe first

Before any of the architecture decisions, there was a small thing that kept bugging me.

I was using Claude Code heavily for day job stuff, and at some point I noticed I had three browser tabs open all the time — Stripe dashboard, Sentry, and a Notion page with our on-call runbook. Every "why did this customer's payment fail" or "what did this error look like yesterday" started the same way. Switch to browser. Find the tab. Log in again because the session expired. Copy something. Come back to the editor. Paste. Ask the agent to keep going.

MCP was already in the picture by then. I had wired up a couple of third-party servers and they mostly worked, but the ones I wanted most were either missing, or heavyweight, or maintained by someone whose priorities were clearly different from mine. So one evening I sat down and told myself: just build one. Start with Stripe because it's the one I open most, and also the one where the API is so well-documented that getting a clean read-only subset working is basically a weekend project.

That was the whole plan. One server. Ship it to npm. Move on.

I did not move on.

Day one: the monorepo decision

This is where I made the first choice that paid off for the rest of the week.

A tiny voice said: just ship Stripe as a single repo. Call it stripe-mcp, npm it, be done. I ignored the voice. If I was going to write one MCP server, there was at least a fifty percent chance I'd write another one next month. And if I wrote two, I was absolutely going to hate myself for duplicating the build config, the test setup, the release pipeline, the lint rules.

So day one, before writing any Stripe code, I set up a monorepo. npm workspaces. TypeScript strict mode. Jest with a 100% line coverage target. ESLint, Prettier, husky, commitlint. release-please for independent versioning per package. A Docusaurus site scaffolded next to the packages folder.

This felt like overengineering for one server. In practice it paid for itself as soon as the second package showed up — no rebuilding CI, no rewriting release config, no fighting the test runner. Every package I added after Stripe was basically drop-in.

The initial commit went in as 406ad19: initial monorepo setup with stripe-mcp and docusaurus.

Sentry next, and the self-hosted thing

Stripe was in a working state within a few hours. I published it, opened Claude Desktop, pointed it at the package, and asked "how many active subscriptions do we have". It answered. That was a good moment.

Next one was obvious. Most of my error-investigation workflow goes through Sentry, but we run a self-hosted Sentry instance, not the SaaS one. And every Sentry MCP server I checked on npm had the base URL hardcoded to sentry.io. Fine for most users. Useless for me and for everyone else I know who runs Sentry on their own infrastructure for compliance or cost reasons.

So I built the Sentry MCP with a SENTRY_BASE_URL env var from the very first commit. Default to sentry.io if not set. Point it at https://sentry.yourdomain.tld if you self-host. No forks, no patches, same npm package for both worlds:

{
  "mcpServers": {
    "sentry": {
      "command": "npx",
      "args": ["-y", "@vineethnkrishnan/sentry-mcp"],
      "env": {
        "SENTRY_AUTH_TOKEN": "sntrys_...",
        "SENTRY_BASE_URL": "https://sentry.mydomain.tld"
      }
    }
  }
}

This is the one I use most, and it has quietly changed how I deal with errors during the day. Before, a Sentry issue in a Slack alert meant: click the link, switch tabs, log back in because the session expired, read the stack trace, copy a snippet, go back to the editor, paste, think. Now it is:

"Pull up the latest unresolved issue in project api, show me the stack trace, and suggest a fix."

The agent calls the Sentry MCP, gets the issue, pulls the event with the stack frames, and either proposes a code change or drills into the file directly. When I'm happy with the fix, I tell it to resolve the issue in Sentry and it does. The write-ops version (more on that later) means the whole loop — investigate, fix, resolve — stays in one place, against my own self-hosted instance, without me touching the dashboard. That part is genuinely the reason I kept building the rest of the pool.

Once Stripe and Sentry were both shipping, I had two packages in the monorepo — and the duplication started showing up. Not in a scary way, but enough that my lint tool flagged it.

The wall I was about to hit

I opened a new Claude Code session and just said:

Yes I need to add oAuth to the supporting mcp packages, so what could be the architecture?

That sentence was the entire setup for the next two days of work.

Here's what I was staring at. Stripe uses a static API key. Sentry uses a static API token. Easy. But the ones I wanted next — Notion, Linear, HubSpot, Intercom, Shopify, Google Workspace — all need OAuth2. And "OAuth2" is one of those phrases that sounds like a single protocol but is actually six slightly different protocols that agree on the overall shape and disagree on every small thing that matters.

Each one was going to need:

A local token file at some path like ~/.mcp-tokens/<server>.json
A browser-based login flow with a callback server
Token exchange, refresh, and caching
A CLI command to log in and log out
The actual SDK integration on top of all that

If I wrote this six times, copy-paste style, three things would happen. One, I'd get bored around server three and start cutting corners. Two, the first time I found a bug in the refresh logic I'd have to fix it in six places. Three, jscpd (a code-duplication detector in my CI) would start blocking PRs because the overlap would be enormous.

The correct answer was obvious in hindsight. The only reason it was not obvious earlier is that I had two packages. With two, you don't see the pattern. With six pending, you can't not see it.

oauth-core

I extracted a package called @vineethnkrishnan/oauth-core. Its job is small and boring, which is exactly what you want from an infrastructure package.

It exposes:

A TokenProvider interface that hides the difference between OAuth and static-key auth, so the MCP tool code doesn't have to care which one it's using.
A LocalFileTokenStore that handles reading, writing, and migrating those ~/.mcp-tokens/<server>.json files.
A set of OAuth strategies — authorization code, refresh, PKCE — that each server can pick from by passing a small config object.
A CLI helper that wires up <server> auth login and <server> auth logout commands so every server has the same UX.

The per-server OAuth config ends up looking like this:

export const notionAuthConfig: OAuthProviderConfig = {
  name: "notion",
  authorizationUrl: "https://api.notion.com/v1/oauth/authorize",
  tokenUrl: "https://api.notion.com/v1/oauth/token",
  scopes: [],
  clientIdEnv: "NOTION_CLIENT_ID",
  clientSecretEnv: "NOTION_CLIENT_SECRET",
};

That's basically the whole auth setup for Notion. Same for Linear, HubSpot, Intercom, Shopify, Google Workspace — each one is a dozen lines of config instead of a few hundred lines of duplicated flow code. The commit that landed this was 417d8a7: feat(oauth): add shared oauth-core package and integrate across 6 mcp servers.

One thing I want to be honest about: the first version of oauth-core was too clever. I tried to abstract every possible OAuth variant behind a single interface and ended up with a config object that had more fields than some of the SDKs I was wrapping. I threw it away and started over with the rule that if a provider doesn't need a field, the field doesn't exist. The second version was boring, readable, and has not needed a breaking change since.

Adding the rest

Once oauth-core was in place, writing a new server stopped feeling like a project and started feeling like filling in a form.

Each new one was a small amount of actually-new code:

A thin services/ layer wrapping the official SDK (if one existed) or fetch (if not).
A tools/ folder defining the MCP tool schemas — what the AI agent sees as callable functions.
Tests. Real tests, not aspirational tests. Every package had to hit the coverage target before it was allowed to merge.

I added Linear, Notion, Vercel, Datadog, HubSpot, Intercom, Shopify, PagerDuty, and Google Workspace this way — one after another, each building on what the previous one had already figured out. The commit that landed them is f137e0c: add 9 new mcp servers for popular saas platforms, but the commit itself isn't the interesting thing. The interesting thing is that the auth config was small, the SDK wrappers were small, and the tool definitions were the only piece that actually needed thought per-provider.

The tests, for what it's worth, were not negotiable. I wasn't moving fast because I was skipping them. I was moving fast because every previous package had tests, which meant when I copied its structure I inherited a working test pattern instead of a blank file.

Shipping is not enough

At some point the eleven packages were on npm, the docs site was live, and oauth-core was carrying the shared weight. I thought the hard part was done.

Then I went to Glama — the community registry that a lot of people use to discover MCP servers — and noticed something. My entries were there, but they showed up with "security and quality not tested" badges. Which is a polite way of saying "this might be junk". Even the great-looking packages in my monorepo were sitting behind those badges, invisible to anyone who filters by verified servers.

So there was a whole second phase of work that was just about being a good citizen of the ecosystem. I added a Dockerfile so the Glama sandbox could spin up each server and verify it responded to introspection. I added read-only tool annotations so clients could show users which tools were safe and which ones wrote data. I added glama.json metadata and an MCP Registry manifest. I added a CI workflow that built the Docker image and ran smoke tests. None of this was code users would ever see, and all of it was required to actually be findable.

This was the moment the project stopped being "I built some MCP servers" and started being "I run an MCP server pool that people can actually adopt without vetting from scratch". Different kind of work, same project.

Write operations, eventually

The first ten days of the project were deliberately read-only. Read Stripe balances, read Sentry issues, read Linear tickets. No creating, updating, or deleting anything. I wanted the safety story to be obvious — if the AI agent goes off the rails, at worst it reads something it shouldn't have.

Write ops landed in commit 24d762c: add write operations to all 11 mcp servers, and I structured them with one hard rule: write tools are opt-in per server, not global. You set an env var to enable them. Otherwise the server starts in read-only mode and the write tools are not even exposed to the agent. This keeps the default safe and lets users flip the switch only for the servers where they actually want the agent to take action.

If you're wondering why this is a big deal: an MCP tool is just a function signature in the agent's context. If the tool exists, the agent may call it. The only way to really prevent it is to not advertise the tool in the first place. Env-gated tools make this a boring one-line config instead of a runtime check that might leak.

Retrospective

Looking back at the repo, a few things worked and a few I'd do differently.

What worked:

Monorepo before the second package. Setting up the monorepo before writing any Stripe code was the most useful decision of the whole project. Every package after Stripe was drop-in.
Extracting oauth-core at the right moment. Too early, and I would have built the wrong abstraction from Stripe's static-key auth. Too late, and I would have been refactoring six packages at once. Roughly the right time was when the third OAuth server was about to be written — hold off until then, then stop and extract.
100% line coverage as a merge gate. Sounds harsh. In practice it mostly just forced me to structure code in ways that were naturally testable.
Self-hosted support from day one in Sentry. Small environmental detail, but it's the thing that actually made the MCP useful for my own day-to-day.

What I'd change:

The first version of oauth-core. As mentioned, it was too abstract. I caught it in the same week, but I'd catch it earlier next time by writing the second caller of a new abstraction before finalising the API.
Docker and Glama from day one. I treated discoverability as a phase-two concern. Should have been phase-one, alongside npm publishing. A package nobody finds doesn't help anyone.
The docs site. Docusaurus is fine, but the default templates eat a lot of configuration time. Next time I'd pick something more minimal or start with the README-only approach and let docs grow organically.

Where it goes next

The public roadmap is in roadmap/ inside the repo. Short version: v0.2.0 is landing write ops across the board (already done, mostly), SSE transport, and streaming responses. v0.3.0 is about webhooks, multi-account support — "show me Stripe balances for both tenants" — and a few more servers people have asked for (GitHub, Slack, Airtable are the three loudest).

If you're someone who also spends their day in three SaaS dashboards and an AI agent, mcp-pool is on npm and the source is on GitHub. The docs are at mcp-pool.vineethnk.in.

That's it for this one. Thanks for reading — see you in the next post.

Bis bald.

Building docling-server: a one-command document API for our AI pipeline

Vineeth N Krishnan — Wed, 22 Apr 2026 05:21:01 +0000

Building docling-server: a one-command document API for our AI pipeline

This is another one in the series where I walk through my open-source projects. Earlier ones covered backupctl, mcp-pool, and a few smaller tools. Today it is docling-server — the thing I built when our AI project needed a proper way to turn messy documents into clean markdown, and calling docling directly from the app started feeling like a bad idea.

If you have not seen docling yet, it is IBM's document processing library. PDF, DOCX, PPTX, scanned images, tables, the whole lot — out comes structured output. Very good at its job. The problem is not docling. The problem is everything around it.

The itch

So the AI project needed document ingestion. And by "document" I do not just mean PDFs. Users would drop in whatever they had lying around — Word files, rich text, PowerPoint decks, scanned contracts, invoices, internal reports in weird old formats, the kind of unstructured stuff nobody formats nicely — and the pipeline had to pull clean text out of every one of them. Clean enough that the downstream steps could actually reason about it.

You can call docling as a library. In a notebook it is lovely. In a production service it gets awkward fast. First, the conversions are slow. A scanned document running through OCR can easily take longer than any sane HTTP request should. Second, docling pulls in a small forest of dependencies — OCR engines, model weights, CUDA bits if you want the GPU path. You do not really want that zoo living inside your main app container.

And there was one more thing. We had this Hetzner GEX box sitting there with a proper GPU on it, exactly for heavy lifting like this. It made no sense to install docling into the app. The AI app should ask a question. The GPU box should do the work.

What was missing was the thin layer in between. An HTTP endpoint that says here is a document, give me markdown back. That layer is what became docling-server.

What "one command" actually means

The pitch is small. Clone the repo, fill in a .env, run make init. You get a protected HTTP endpoint over SSL — API key in the header, Let's Encrypt doing the certificate bit, nginx enforcing TLS on the edge — that takes a file or a URL and returns markdown, JSON, or plain text. That is it.

Under the hood it is less small. The compose file spins up:

nginx for the reverse proxy, with Let's Encrypt doing the TLS bit
FastAPI for the actual API endpoints
Celery workers doing the real document conversion
Redis as the broker and result backend
Flower for a dashboard to peek at what the workers are up to

The split matters. FastAPI accepts the request, hands off to Celery, and hands you back a task ID. The worker chews through the document in the background, writes the result to Redis, and you come back to /tasks/{id} when you want it. No waiting thirty seconds on an HTTP connection while a scanned invoice is being OCR'd. No timeouts. No retry headaches.

A minimal request looks like this:

# kick off a conversion
curl -X POST https://docling.yourdomain.com/convert \
  -H "X-API-Key: your-token" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://arxiv.org/pdf/2408.09869"}'

# come back later for the result
curl https://docling.yourdomain.com/tasks/TASK_ID \
  -H "X-API-Key: your-token"

That is the whole surface area you need on the calling side. The AI project does not care that there is a GPU, an OCR engine, six model files, and a CUDA runtime behind that URL. It just asks, and markdown comes back.

Has your team also had this moment — where you realised the clean thing was to hide all the mess behind a single HTTP call and move on? I feel like half the services I build end up being exactly that.

The CUDA-in-Docker saga

I will not pretend the setup was smooth. The part that ate the most time was getting Docker to actually see the GPU.

CUDA on bare metal is one thing. CUDA inside a Docker container using a GPU that belongs to the host is a different level of patience. I honestly do not remember every exact error message I ran into — this was some time ago and I have done a decent job of repressing it — but I remember the shape of the problem. The container would happily start. The Python process inside would happily import torch. And then torch would calmly tell me there were zero CUDA devices available, thank you very much.

If you have ever been through this, you know the drill. Is the nvidia driver installed on the host? Yes. Is the container toolkit installed? Yes. Is the runtime configured in /etc/docker/daemon.json? Pretty sure yes. Is the compose file using runtime: nvidia or the new deploy.resources.reservations.devices block? Which one is the right one this year? Who knows anymore.

I got it working. I got it working the way most of us get these things working — by reading five GitHub issues, trying four combinations, and eventually landing on the one that did not throw. The annoying bit is that once it works, it just keeps working, and you forget exactly which of the four things was the actual fix. If you are setting this up on your own GPU box, budget a bit of time for this specifically. It is not docling's fault. It is the price of doing business with NVIDIA and Docker in the same sentence.

The moment the first OCR conversion came back and I saw the tables from a scanned document rendered as clean markdown — that was a good moment. Small win. But the kind of small win that makes a whole evening of driver wrangling feel worth it.

What the AI project actually gets

From the app's point of view, the contract is dumb simple. Upload a file, or point at a URL. Poll a task. Get markdown. Feed it into whatever downstream chain was waiting for clean text.

A few things that turned out to matter more than I expected:

Batch endpoint. Users rarely upload one file. They upload a folder. /convert/batch lets the worker pool chew through them in parallel instead of us queueing sequentially from the app side.
Embeddings on the same box. Since we already had the GPU warm, generating vector embeddings right there saved an entire network hop. The app gets text and vectors from one call.
API key + rate limits at nginx. It is an internal service, but internal services have a way of getting called from places you did not plan. A token and a per-IP rate limit at the edge cost almost nothing to set up and save a lot of explaining later.
Flower on localhost only. If a Celery worker is stuck, you want to know. If a Flower dashboard is accidentally open on the public internet, you really do not want that. Binding it to localhost and tunnelling over SSH when I need it is the lazy, correct answer.

None of this is clever. Most of it is the boring scaffolding you end up writing for any long-running task service. The good part is that once it is all in one repo with a make init, the next person who needs a document processing endpoint can stand it up on their own GPU box without redoing any of the thinking.

Who is this for

Honestly, future me. The next time I need docling behind an API, I am not redoing the Celery setup and the nginx config from scratch.

But also anyone else going down this path. If you are trying to get docling into a production shape — on your own hardware, with GPU, with OCR, with a queue, with TLS — and you do not want to glue it together from six different blog posts, this repo is pretty much that glue already. Not polished. Not fancy. But it works, and it is documented enough that you are not guessing what make init is about to do.

The repo is at github.com/vineethkrishnan/docling-server. The deployment doc has the actual commands. If you hit the CUDA-in-Docker thing too, you have my sympathy — we have all been there.

So that is where I will stop. If you have a different way of doing this, or a cleaner trick for the GPU-in-Docker bit, I genuinely want to hear it — drop me a note. Otherwise, see you when the next interesting problem shows up.