I built an open-source Python tool to detect drift in embedding spaces

Praful Reddy — Fri, 17 Apr 2026 00:19:56 +0000

I built an open-source Python tool to detect drift in embedding spaces

Most monitoring pipelines wait for a downstream metric to break: accuracy drops, retrieval quality slips, or user-facing behavior gets worse.

By then, the shift has already happened.

I wanted a simpler way to catch changes earlier by looking directly at the embeddings themselves.

So I built drift-lens-monitor — an open-source Python package for detecting drift in embedding spaces.

GitHub: https://github.com/PRAFULREDDYM/Drift_lense
PyPI: https://pypi.org/project/drift-lens-monitor/

What problem this solves

A lot of modern ML systems depend on embeddings:

semantic search
RAG pipelines
recommenders
clustering
classification pipelines with embedding-based features

Even when the raw system looks “healthy,” the embedding space can start changing underneath you.

That change may come from:

new user behavior
model updates
data source changes
upstream preprocessing differences
gradual distribution shift over time

If you only monitor business metrics or model accuracy, you often detect the issue late.

The idea behind this project is simple:

take snapshots of embeddings over time and compare them directly.

What drift-lens-monitor includes

The package currently supports three drift detection approaches.

1) Fréchet Embedding Distance (FED)

This is inspired by FID-style comparison, but applied to arbitrary embeddings.

At a high level, it compares the mean and covariance of two embedding distributions.

Useful when you want a compact statistical distance between a reference snapshot and a current snapshot.

2) Maximum Mean Discrepancy (MMD)

This is a kernel-based, non-parametric method for comparing two samples.

I included permutation-based p-values so it can be used not just as a raw distance, but also as a statistical test.

Useful when you want a more flexible distribution comparison without assuming Gaussian structure.

3) Persistent homology

This is the unusual one.

Instead of only asking whether two embedding clouds differ statistically, this looks at whether their shape changes.

It builds topological summaries over the point cloud and compares them using Wasserstein distance.

Why that matters:

A system can preserve rough averages while still changing structurally. For example:

clusters merge
clusters split
holes or loops appear/disappear
local geometry shifts in ways mean/covariance may not capture

This makes persistent homology an interesting complement to more standard drift metrics.

Design goals

I wanted the tool to stay practical:

local-first
no cloud dependency
no API keys
simple snapshot storage
easy to inspect and extend

Snapshots are stored as parquet files, so the workflow stays lightweight and reproducible.

The package can be used:

as a Python library
through a Streamlit dashboard for visual exploration

Install


bash
pip install drift-lens-monitor

## Example workflow

The intended workflow is straightforward:

1. Save a reference embedding snapshot
2. Save a new embedding snapshot later
3. Compare them using one or more drift methods
4. Inspect drift scores and visualize the changes

This makes it usable for both experimentation and production-adjacent monitoring workflows.

## Why I built it

I was interested in a gap I keep seeing in ML tooling:

we monitor model outputs, latency, costs, and downstream metrics heavily, but we often do much less direct monitoring of the representation space itself.

Embeddings are doing a huge amount of work in modern AI systems. They deserve first-class monitoring too.

I also wanted to explore whether more unusual techniques like **topological drift detection** could add signal beyond standard statistical distances.

## What I’d love feedback on

I’d especially love feedback on three things:

1. Does persistent homology feel genuinely useful here, or too heavyweight?
2. What baselines or benchmark datasets would make this more convincing?
3. How should the package/API be improved to make it easier to use in real workflows?

## Links

- GitHub: https://github.com/PRAFULREDDYM/Drift_lense
- PyPI: https://pypi.org/project/drift-lens-monitor/

A zero-token progress bar for Claude Code

Praful Reddy — Fri, 10 Apr 2026 11:48:59 +0000

Every Claude Code extension I've seen shows the same thing: token counts, API costs, model info. Useful, but it doesn't answer the question I actually care about mid-session — how much of the work is done.
So I built task-progress-bar. It reads Claude Code's native task list from disk and renders a live ASCII progress bar with time estimates. It runs as a PostToolUse hook, which means it consumes zero tokens — it's a subprocess that Claude never sees.

Tasks [████████░░] 8/10 (~3m left) | ✓8 ⟳1 ○1

How it works

Claude Code persists tasks as JSON files in ~/.claude/tasks/. The plugin watches for TodoWrite, TodoRead, TaskCreate, and TaskUpdate tool calls via the PostToolUse hook, then:

Parses every JSON file in the tasks directory
Counts completed, in-progress, and pending tasks
Computes a time estimate using an exponential moving average (EMA)
Outputs a single status line to Claude Code's statusLine renderer

The time estimation is straightforward. Each time a task moves to completed, the timestamp is logged. The interval between consecutive completions feeds into an EMA with α=0.3:

EMA_new = 0.3 × latest_interval + 0.7 × EMA_old
estimated_remaining = EMA × tasks_left

Intervals over 1 hour are clamped to avoid skew from session breaks. The first 3 completions show "calculating..." until there's enough data.
The entire plugin is a single Python file — stdlib only, no pip dependencies. It uses json, pathlib, time, and sys. Nothing else.

Install

curl -fsSL https://raw.githubusercontent.com/PRAFULREDDYM/task-progress-bar/main/install.sh | bash

The installer checks for Python 3.8+, copies the script to ~/.claude/, and patches settings.json with the hook configuration. There's a matching uninstall.sh for clean removal.

What it looks like

The progress bar color-codes by completion percentage — red below 33%, yellow from 33–66%, green above 66%. When all tasks finish, it shows ✅ All done! for 30 seconds and then hides.
If you run it standalone (python3 task_progress_bar.py), you get a full multi-line colored view with a task breakdown and per-task average.

Forem: Praful Reddy

I built an open-source Python tool to detect drift in embedding spaces

I built an open-source Python tool to detect drift in embedding spaces

What problem this solves

What drift-lens-monitor includes

1) Fréchet Embedding Distance (FED)

2) Maximum Mean Discrepancy (MMD)

3) Persistent homology

Design goals

Install

A zero-token progress bar for Claude Code

How it works

Install

What it looks like

Links