Forem: Stephen Collins

AI Agents Can Pass Tests. They Still Can't Maintain Systems.

Stephen Collins — Thu, 12 Mar 2026 13:43:00 +0000

AI coding tools have made writing software dramatically easier.

Cursor can scaffold entire modules from a prompt. Claude can produce working CLIs in minutes. Copilot fills in entire implementations as you type.

Code production is no longer the primary bottleneck in software engineering.

But a new research benchmark from Alibaba highlights something important:

Writing code is not the same as maintaining a system.

The SWE-CI Benchmark

Most AI coding benchmarks test one thing: can a model produce working code once?

Examples include HumanEval, MBPP, and SWE-bench. These evaluate a single moment in time. A model writes code, tests pass, and the task is complete.

Real software does not work that way.

Codebases evolve for years. Features are added. Requirements change. Dependencies shift. Bugs appear in unexpected places.

The SWE-CI benchmark was designed to simulate this reality.

Instead of a single coding task, SWE-CI runs models through a simulated continuous integration loop across real repository history. Each task includes:

a real open source repository
a starting commit
dozens of historical changes
evolving test suites

On average, each task spans 233 days of evolution across 71 commits.

Agents must repeatedly:

read the repository
analyze failing tests
modify the code
run CI
avoid breaking existing behavior

This process repeats over many iterations. The goal is not just to make code work once. The goal is to maintain a system over time.

What Happens to Most AI Agents

The results are revealing.

Across 18 models from 8 providers, most agents struggled to maintain stability under continuous evolution. The most common failure pattern looks like this:

Fix failing test
↓
Break another module
↓
Patch that module
↓
Break something else
↓
System becomes unstable

Researchers observed several recurring failure modes.

Local Patch Myopia

Agents apply minimal fixes to the failing test without understanding broader dependencies. The result is a cascading sequence of regressions.

Interface Drift

Function signatures change but callers are not updated across the codebase.

Architectural Erosion

Over many iterations, the codebase accumulates duplicated logic, temporary helpers, and inconsistent abstractions. In other words: AI agents generate technical debt.

Test Misinterpretation

Models often treat tests as constraints to bypass rather than signals about system invariants. Passing the test becomes more important than preserving the intended behavior.

Even the Best Models Struggle

One model family stood out: Claude Opus produced significantly fewer regressions than other models.

But even the best-performing agents still broke existing behavior in roughly half of the maintenance iterations.

That is a critical observation.

AI models can often produce working implementations. But maintaining a complex system requires something deeper:

understanding dependencies
preserving architectural structure
predicting change impact
managing technical debt

These are system comprehension problems, not code generation problems.

The Bottleneck Has Moved

For decades, writing code was the limiting factor in software development. AI is rapidly removing that constraint.

But SWE-CI highlights the next bottleneck: understanding large systems and how they evolve.

Two implementations may both pass tests today. Only one will remain stable as the system changes. Those differences only emerge over time — and that's exactly what the benchmark is designed to surface.

The Rise of Meta Tooling

As AI increases the rate of code production, the importance of tools that help engineers understand systems will grow.

A new category of tooling is emerging around:

repository evolution
architectural risk
change impact
technical debt concentration

These are tools that help engineers answer questions like:

Which files actually matter in this codebase?
Where is complexity accumulating?
Which parts of the system are most fragile?

┌─────────────────────────────────────────────────────┐
│  Layer 3 - Codebase Intelligence                    │
│  Hotspot analysis · architecture evolution ·        │
│  system risk signals                                │
├─────────────────────────────────────────────────────┤
│  Layer 2 - Code Validation                          │
│  Linters · type systems · static analysis ·         │
│  security scanners                                  │
├─────────────────────────────────────────────────────┤
│  Layer 1 - Code Creation                            │
│  IDEs · Copilot · Cursor · AI coding agents         │
└─────────────────────────────────────────────────────┘

This is the motivation behind Hotspots.

Hotspots analyzes repository history and code structure to identify the small set of files that account for most maintenance risk. In large systems, these hotspots often represent a tiny fraction of the codebase that drives most engineering effort.

Understanding those areas is essential for both humans and AI systems trying to maintain software over time. The SWE-CI failure modes tend to concentrate in the same places hotspot analysis flags: files with high churn and high complexity are exactly where cascading regressions and architectural erosion accumulate.

The Future of AI-Assisted Engineering

AI will keep getting better at generating code.

But software engineering has always been more than generation. It is about managing complexity, evolving architecture, and maintaining invariants across thousands of changes.

The SWE-CI benchmark shows that these challenges remain significant for current AI agents. Which means the next generation of developer tooling will likely focus less on writing code and more on understanding systems.

The developers who will be most effective in an AI-accelerated world won't just be the ones who can generate code fastest. They'll be the ones who can reason clearly about systems — who understand which parts matter, where risk lives, and where to focus attention.

If you're curious where risk concentrates in your own codebase, hotspots.dev is a good place to start.

AI Made Code Cheap. The Bottleneck Is Now Understanding Systems.

Stephen Collins — Mon, 09 Mar 2026 13:51:00 +0000

This post was originally published on hotspots.dev.

Software used to be expensive to produce. Developers were constrained by how fast they could write code, which meant teams had to be deliberate about what they built. Every feature was an investment.

That constraint is eroding quickly. AI tools can now generate large amounts of working software from relatively small prompts. Cursor scaffolds full modules from a description. Claude Code produces working CLIs in minutes. Copilot fills in entire implementations as you type. One prompt can produce hundreds of lines of code — complete API handlers, even full prototypes.

Code production is no longer the bottleneck.

The constraint has moved somewhere else.

The Bottleneck Has Moved

When the cost of producing something drops dramatically, volume increases. Software is no exception.

As code becomes cheaper to produce, the hard problem shifts from writing to understanding. Developers increasingly spend their time asking different questions:

Which parts of this codebase actually matter?
Where are bugs most likely to appear?
Which modules are fragile?
Where should we direct engineering effort?

These are questions about the system itself, not individual files or functions. And they get harder to answer as systems grow, something AI accelerates.

The engineering bottleneck used to be code creation. Now it's system comprehension.

Three Structural Effects of Cheap Code

When something becomes dramatically cheaper to produce, predictable effects follow.

Codebases grow faster. AI removes friction from writing new components. Developers spin up modules they previously would have avoided, scaffold services that once took days, and generate helper libraries on the fly. The number of files, abstractions, and moving parts increases. Systems get larger.

Systems change faster. AI also accelerates commit velocity. Refactors become cheaper. Experiments are easier to try. Iteration cycles compress. This is broadly positive — faster feedback loops are good — but it also means the codebase changes at a rate that can outpace a team's ability to reason about it.

Architectural entropy increases. AI tools are optimized for local correctness — does this function do what I asked? — not for global coherence. Over time, the first two effects compound. Systems still work, but duplicated patterns, inconsistent abstractions, sprawling modules, and accidental complexity pile up.

What Existing Tooling Misses

Most developer tooling falls into two categories.

There are tools that help you write code: IDEs, autocomplete, and AI assistants like Copilot or Cursor. These help developers produce code faster. They've gotten remarkably good at this.

There are tools that help you check code: linters, static analyzers, type systems, security scanners. These validate correctness — does the code follow rules, does it compile, does it have obvious vulnerabilities.

What's mostly missing is a third category: tools that answer the question what parts of the system matter most?

Today, developers answer this question through intuition and tribal knowledge. Someone on the team "just knows" that the auth module is fragile, or that the payments service has accumulated a lot of technical debt. That knowledge lives in people's heads, doesn't transfer well, and becomes harder to maintain as systems grow and teams change.

As AI accelerates codebase growth, relying on intuition becomes increasingly untenable.

The Meta Tool Layer

┌─────────────────────────────────────────────────────┐
│  Layer 3 - Codebase Intelligence                    │
│  Hotspot analysis · architecture evolution ·        │
│  system risk signals                                │
├─────────────────────────────────────────────────────┤
│  Layer 2 - Code Validation                          │
│  Linters · type systems · static analysis ·         │
│  security scanners                                  │
├─────────────────────────────────────────────────────┤
│  Layer 1 - Code Creation                            │
│  IDEs · Copilot · Cursor · AI coding agents         │
└─────────────────────────────────────────────────────┘

One way to think about this missing category is meta tools.

Where conventional tools analyze individual files — this function is too complex, this import is unused — meta tools analyze the codebase itself. They examine system structure, change patterns, and architectural behavior over time. They answer questions like:

Where does change concentrate?
Where does complexity accumulate?
Which modules are stable, which are volatile?

Meta tools sit above the code.\
They don't tell you how to fix a specific bug.\
They tell you where to look.

Hotspot Analysis as a Concrete Example

One useful approach combines two signals already present in almost every repository: commit history and complexity.

Some files change constantly. Some files are extremely complex. When those two traits overlap, you have a problem — a function that is both hard to understand and being actively modified is a high-probability site for bugs, regressions, and compounding technical debt.

The core heuristic is simple:

risk ≈ change frequency × complexity

In practice, this surface area is small. In most systems, perhaps 5–10% of the codebase accounts for the majority of the structural risk. Identifying that slice lets engineering teams make better decisions about where to invest time — what to refactor first, what to review most carefully, where to add test coverage.

Hotspot analysis is one concrete example of a meta tool.

Hotspots applies this idea to any git repository, ranking functions by activity-weighted risk and surfacing the small portion of the codebase that dominates engineering effort. The goal isn't to tell you your code is bad. It's to give you a map.

Why This Matters More in the AI Era

AI tools make hotspot signals stronger, not weaker.

When developers iterate faster, commit velocity increases — and high-velocity files accumulate churn signals more quickly. When AI generates code that gets repeatedly revised, rewritten, or extended, those integration points show up clearly in the analysis. When architectural entropy increases, complexity metrics climb.

The problem hotspot analysis solves — identifying where risk concentrates — becomes more valuable as the forces that create risk accelerate.

Codebase Observability

The broader idea behind this goes beyond hotspot analysis.

Modern production systems have strong observability. We measure logs, metrics, traces, error rates, latency. We have dashboards for system health. When something goes wrong in production, we have tools to understand why.

We don't have equivalent tooling for the codebase itself.

We don't routinely measure how architecture evolves, where complexity accumulates over time, which modules are trending toward instability, or how AI-generated code changes the system's structural properties. That information exists in git history and static analysis — it just hasn't been assembled into something useful.

The next step is codebase observability: treating the evolution of software systems as something worth measuring, monitoring, and understanding over time. Not just "is the code correct?" but "how is the system changing, and where is that change creating risk?"

What Comes Next

AI is making code abundant.

System understanding is becoming the scarce skill.

Tooling for this problem is still early.\
But it's the right problem to be solving.

If you're curious where risk concentrates in your own codebase, hotspots.dev is a good place to start.

I Built a CLI to Find the Riskiest Code in Any Repo — Introducing Hotspots

Stephen Collins — Tue, 03 Mar 2026 13:38:00 +0000

Every team has those files. The ones everyone knows are dangerous. The ones where a "simple" change takes three days of careful testing. The ones that keep showing up in postmortems.

I spent a long time noticing that pattern—the same 10% of a codebase causing 80% of the pain—and wondering why our tooling didn't just show us where that 10% was. ESLint can tell you a function has high cyclomatic complexity. But that doesn't tell you whether it's actively being changed, who's touching it, or whether it's likely to bite you in the next sprint.

So I built Hotspots — a Rust CLI that finds risky code by combining structural complexity with real git activity.

The Core Idea: Complexity × Change

Raw complexity metrics are useful, but they're incomplete. A gnarly function that hasn't been touched in two years isn't your emergency today. The real danger is complexity plus active change — functions that are structurally hard to reason about and are being modified regularly.

Hotspots computes a risk score for every function in your codebase by combining:

Structural signals: cyclomatic complexity (CC), nesting depth (ND), fan-out (FO), non-structured exits (NS)
Activity signals: git churn in the last 30 days, touch frequency (commit count), recency, call-graph influence

The result is an activity-weighted risk score — a prioritized list of functions that are both dangerous and active. Not functions you should eventually clean up. Functions you should care about this sprint.

Getting Started

Install with a single curl:

curl -fsSL https://raw.githubusercontent.com/Stephen-Collins-tech/hotspots/main/install.sh | sh

Point it at any repo and run a snapshot:

hotspots analyze . --mode snapshot --format text

You'll get a ranked output grouped into risk bands:

Critical (risk ≥ 9.0):
  processPlanUpgrade    src/api/billing.ts:142    risk 12.4  CC 15  ND 4  FO 8

High (6.0 ≤ risk < 9.0):
  validateSession       src/auth/session.ts:67    risk 9.8   CC 11  ND 3  FO 7
  applySchema           src/db/migrations.ts:203  risk 8.1   CC 10  ND 2  FO 5

For a shareable HTML report:

hotspots analyze . --mode snapshot --format html --output report.html

Explain Mode: Why Is This Function Risky?

Once you have the list, you want to know why something ranked high — and what to do about it. Pass --explain-patterns and Hotspots annotates each function with antipattern labels:

hotspots analyze . --mode snapshot --format json --explain-patterns > snapshot.json

In v1.2.0, Hotspots detects two tiers of patterns:

Tier 1 — Structural:
complex_branching, deeply_nested, exit_heavy, long_function, god_function

Tier 2 — Relational & Temporal:
hub_function, cyclic_hub, middle_man, neighbor_risk, stale_complex, churn_magnet, shotgun_target, volatile_god

A function tagged god_function + cyclic_hub is both monolithic and at the center of a dependency cycle — a very different refactoring situation than one tagged exit_heavy + long_function. The labels make the action obvious.

CI Policy Checks: Stop Regressions Before They Merge

Identifying hotspots is useful. Preventing new ones from landing is better. Hotspots has a delta mode that compares the current branch against the baseline and applies configurable policy checks:

hotspots analyze . --mode delta --policy

Policies you can configure:

Critical Introduction — fail if a new function lands in the critical band
Excessive Regression — fail if a function's risk score jumps by a large delta
Rapid Growth — flag unusually fast complexity growth
Watch/Attention — warn when functions approach thresholds

Add it to CI and complexity regressions fail the PR before review. Start it in warn-only mode, get the team used to the signal, then flip to blocking when you're ready.

Here's a minimal GitHub Actions step:

- name: Hotspots policy check
  run: hotspots analyze . --mode delta --policy

Exit code 1 if any blocking policy fails. Zero if clean.

hotspots.dev — Automated OSS Analysis

Alongside the CLI, I've been building hotspots.dev — a blog that runs automated Hotspots analyses on trending open-source repos every night.

The pipeline:

A GitHub Actions crawl job selects a fresh trending repo
Hotspots analyzes it and extracts the top functions and antipatterns
An AI draft gets generated and opened as a PR for review
Once merged, it deploys automatically

It's a real-world showcase of what the tool surfaces in popular codebases — eslint, Flowise, and more. The HTML reports are hosted at reports.hotspots.dev so you can drill into the full ranked analysis for any repo.

I find it genuinely interesting to run Hotspots on codebases I use every day and see where the structural risk actually lives. It's rarely where you'd guess.

Try It

# Install
curl -fsSL https://raw.githubusercontent.com/Stephen-Collins-tech/hotspots/main/install.sh | sh

# Snapshot your current repo
hotspots analyze . --mode snapshot --format text

# Full JSON with pattern labels
hotspots analyze . --mode snapshot --format json --explain-patterns > snapshot.json

# CI policy check
hotspots analyze . --mode delta --policy

📖 Docs: docs.hotspots.dev
🔬 Live analyses: hotspots.dev
🦀 Source: github.com/Stephen-Collins-tech/hotspots

It's MIT licensed, written in Rust, and works on any language — since it operates on git history and AST-level metrics rather than language-specific rules.

If you've ever looked at a PR and thought "this feels risky but I can't explain why" — run Hotspots. It'll tell you why.

TL;DR:

Complexity metrics alone miss the point. Hotspots combines structural analysis with real git activity to surface the functions that are both hard to change and actively being changed. It's a Rust CLI, it works on any language, and it can gate PRs in CI. Try it.

Merkle Trees in SQLite with Python: A Practical Tutorial

Stephen Collins — Mon, 22 Sep 2025 20:45:00 +0000

Merkle trees are one of those concepts that quietly power some of the most important technologies we use today—Git, blockchains, and peer-to-peer systems like IPFS. They provide a compact way to prove that a piece of data belongs to a larger dataset, without needing to reveal or send everything.

But Merkle trees are often introduced only in the context of massive distributed systems. What if you just want to play with them locally, on your laptop, using tools you already know? That's where SQLite comes in.

In this tutorial, we'll build a Merkle tree in Python and persist it to SQLite. By the end, you'll have a working project that can:

Build a Merkle tree from arbitrary strings.
Store all nodes—leaves and internal hashes—in a relational database.
Query inclusion proofs for any leaf.
Verify proofs against the stored root.
Inspect and visualize the tree with SQL or helper scripts.

This combination gives you a verifiable, tamper-evident, and queryable data structure inside a single .db file.

All the code is available on GitHub.

Why SQLite?

Before we dive into code, let's answer the obvious question: why store a Merkle tree in SQLite at all?

SQLite is a lightweight, embeddable database. You don't need a server, a cluster, or even an internet connection—just a file. That makes it ideal for:

Tamper-evident logs
Store logs or events as leaves, and use the Merkle root to prove no row was modified. If someone changes even one character in the database, the root hash won't match.
Lightweight blockchain-like systems
You can experiment with blockchain concepts without spinning up heavy infrastructure. Exchange Merkle roots and proofs between peers instead of full tables.
Efficient synchronization
Two devices with the same dataset can exchange just their root hashes. If they differ, you can reconcile branch by branch instead of copying everything.
Verifiable queries
SQL gives you speed and structure; Merkle trees give you proofs. Together, you can run a query and provide a cryptographic guarantee that the results belong to a specific database state.
Education and demos
SQLite makes the concept tangible. You can see every node in a table, inspect it with SQL, and visualize the tree.

So while it may seem unusual, storing Merkle trees in SQLite is a neat way to bridge cryptographic structures with the databases we already use.

How Merkle Trees Work

A Merkle tree is a binary tree where:

Leaves contain the hash of the raw data.
Internal nodes contain the hash of the concatenation of their children.
The root hash commits to the entire dataset.

If you want to prove that “gamma” is part of the dataset, you don't need to show every other value. Instead, you provide its sibling hashes along the path to the root. The verifier recomputes and checks if it matches the stored root hash.

The Project Structure

Our Python project is minimal and dependency-free. The main files are:

├── main.py            # Example usage
├── merkle_tree.py     # Core MerkleTree class
├── print_proof.py     # CLI to print and verify proofs
├── visualize_tree.py  # CLI to render tree as ASCII or Graphviz
├── pyproject.toml     # Metadata and scripts
└── README.md

Everything uses only the Python standard library (hashlib, sqlite3, pathlib).

The SQLite Schema

The heart of persistence is a single table:

CREATE TABLE IF NOT EXISTS merkle_nodes (
  id INTEGER PRIMARY KEY,
  parent_id INTEGER,
  left_child_id INTEGER,
  right_child_id INTEGER,
  hash TEXT NOT NULL,
  level INTEGER NOT NULL,
  is_leaf BOOLEAN NOT NULL,
  data TEXT,
  FOREIGN KEY(parent_id) REFERENCES merkle_nodes(id),
  FOREIGN KEY(left_child_id) REFERENCES merkle_nodes(id),
  FOREIGN KEY(right_child_id) REFERENCES merkle_nodes(id)
);

Each row is one node. Leaves carry original data; internal nodes have only hash. Parent and child links let you traverse the tree in either direction.

The `MerkleTree` Class

The core logic lives in merkle_tree.py. It's wrapped in a class that manages both hashing and database storage.

Initialization and Context Manager

with MerkleTree("merkle_tree.db") as tree:
    root_id = tree.build_tree(["alpha", "beta", "gamma"])

The class opens and closes its SQLite connection automatically. Using a with block ensures resources are cleaned up.

Building the Tree

build_tree(data_blocks) does three things:

Hashes the leaves with SHA-256.
Iteratively pairs them, hashing concatenated child hashes until a root is formed. If there's an odd number, the last hash is duplicated (Bitcoin-style).
Inserts each node into the database and links parent-child relationships.

The method returns the root node's ID for convenience.

Generating Proofs

get_proof(leaf_id) walks from a leaf up to the root, collecting sibling hashes along the way. Each step also records whether the current node was a left or right child. That orientation matters because the concatenation order of hashes changes the result.

If the current node was the left child, the sibling is on the right → (sibling_hash, True).
If the current node was the right child, the sibling is on the left → (sibling_hash, False).

This way, the verifier knows whether to compute sha256(current + sibling) or sha256(sibling + current).

Example proof path:

[
    ("hash_of_right_sibling", True),   # current node was left
    ("hash_of_left_sibling", False),   # current node was right
    ("hash_of_right_sibling", True)    # current node was left
]

During verification, you start from the leaf's hash, then fold in each sibling in the correct order until you recompute the root.

Verification

Finally, verify_proof(data_item, proof_path, expected_root) recomputes the path from a leaf to the root.

It works like this:

Start with the SHA-256 hash of the original data.
For each (sibling_hash, is_left_child) in the proof path:

If is_left_child == True, the current node was the left child, so compute:
```
 current = sha256((current + sibling_hash).encode())
```
If is_left_child == False, the current node was the right child, so compute:
```
 current = sha256((sibling_hash + current).encode())
```

When you reach the top, compare the result to the stored root.

Example:

current = hashlib.sha256(data_item.encode()).hexdigest()
for sibling_hash, is_left_child in proof_path:
    if is_left_child:
        current = hashlib.sha256((current + sibling_hash).encode()).hexdigest()
    else:
        current = hashlib.sha256((sibling_hash + current).encode()).hexdigest()
print(current == expected_root)  # True if inclusion is valid

This guarantees that only the correct leaf, combined with the right sequence of sibling hashes, can reconstruct the root.

Running the Demo

main.py ties everything together:

uv run main.py

Output looks like:

Root node ID: 31
Leaf: 3 5f9a...
Proof for 'gamma': [('4f4a...', True), ('0cb0...', False), ('673e...', True)]
Verification: True
Merkle tree stored in /absolute/path/merkle_tree.db

This shows:

The root node was built.
A proof for "gamma" was generated.
Verification succeeded.
The full tree is persisted in merkle_tree.db.

Inspecting with SQL

Because everything is in SQLite, you can poke around yourself:

sqlite3 merkle_tree.db

View schema and sample nodes:

.schema merkle_nodes
SELECT id, level, is_leaf, substr(hash,1,16) AS h, data
FROM merkle_nodes ORDER BY level, id LIMIT 10;

Find the root:

SELECT id, hash FROM merkle_nodes WHERE parent_id IS NULL;

You can even extract inclusion proofs with a recursive CTE.

Printing and Verifying Proofs

The helper script print_proof.py lets you fetch proofs from an existing database:

uv run python print_proof.py gamma

Output:

Data: gamma
DB: merkle_tree.db
Leaf: id=3 hash=be9d...
Root: afc48e...
Proof (sibling_hash, is_left_child):
   ('4f4a94...', True)
   ('0cb030...', False)
   ('673e72...', True)
Verification: True

Visualizing the Tree

The optional visualize_tree.py script shows the tree in ASCII or Graphviz DOT:

uv run python visualize_tree.py --format ascii

Example output:

Root: id=31 hash=abc123
Level 0: [N id=31 h=abc123]
Level 1: [N id=25 h=1122aa]  [N id=30 h=99ee77]
Level 2: [L id=3 h=5f9a... data='gamma']  [L id=4 h=...]

How This Compares to Git and Bitcoin

It's worth noting how our simple SQLite-backed Merkle tree relates to real-world systems:

Git uses Merkle trees to track file versions. Every commit points to a tree object (directory), which points to blobs (files). Hashes are stored in a content-addressable database (.git/objects). Our implementation is simpler but follows the same principle: content → hash → parent hash → root.
Bitcoin uses Merkle trees to commit to all transactions in a block. Each leaf is a transaction hash; the Merkle root is included in the block header. Our example uses strings like "alpha" instead of transactions, but the process—pairing, hashing, and duplicating odd leaves—is identical.

The difference is scale and purpose. Git and Bitcoin optimize for billions of objects or high-security consensus. Our SQLite example is educational: small, transparent, and easy to query.

Lessons Learned

This little project highlights some powerful ideas:

Cryptography meets databases: By storing Merkle trees in SQLite, you blend verifiability with queryability.
Proofs are compact: Verifying inclusion doesn't require downloading everything—just log(N) sibling hashes.
SQLite is versatile: It's more than a toy database; you can model even cryptographic structures.
Educational clarity: Inspecting the tree in SQL makes the concept far less abstract.

Where to Go Next

This is a minimal, educational implementation. Real-world extensions could include:

Append-only logs for tamper-evident auditing.
Multiple independent Merkle trees in one database.
Performance tuning for large datasets.
More advanced hash strategies (domain separation, byte-level concatenation).
Integration with APIs that require verifiable responses.

Conclusion

Merkle trees are often taught as abstract cryptographic structures behind big systems like Bitcoin. But as this tutorial shows, you can implement them locally with just Python and SQLite.

The result is a verifiable, tamper-evident database in a single .db file—no servers, no dependencies. Whether you're prototyping, teaching, or just exploring, it's a powerful way to make Merkle trees tangible.

The Technical Challenges I've Faced in Interviews (and Given as an Interviewer)

Stephen Collins — Sat, 13 Sep 2025 15:50:35 +0000

I've been on both sides of the technical interview table. Sometimes I'm the one sweating through a coding challenge, other times I'm the one deciding how to evaluate a candidate. Along the way, I've seen a few clear patterns in what works (and what doesn't).

Here's what that looks like from both perspectives.

As an Interviewer

When I've run interviews, I've leaned heavily on technical conversations rather than puzzles or trick questions.

That means sitting down with someone and asking them to talk through what they know, what they don't know, and what they actually like doing. I'll often dig into past projects:

What trade-offs did you make?
What would you do differently if you had twice the time?
What parts of the system frustrated you?

I'm not looking for “perfect” answers. I'm looking for signals:

Can you clearly explain your reasoning?
Do you know your own limits?
Do you light up when talking about certain problems, and shrink away from others?

The job isn't about memorizing solutions-it's about how you think, communicate, and make trade-offs. These conversations often reveal more than any contrived exercise ever could.

And honestly, they're more enjoyable for both sides. Instead of watching someone struggle through a binary search tree implementation, we get to talk shop. That's a better predictor of whether I'd want to work with them every day.

As an Interviewee

I've had my fair share of experiences across the spectrum. Some good, some less so.

Esoteric algorithm challenges

These are the “can you do dynamic programming on a whiteboard in 20 minutes?” style questions. I'll be honest: I still bomb these. They're mostly about drilling patterns until you can regurgitate them under pressure.

I once got asked to implement Conway's Game of Life that, with time, I probably could've solved-but in the moment, I froze. Unless it's two-sum (which I've got memorized), I'm not your whiteboard hero.

I understand why these persist-they're standardized, and you can compare candidates on a consistent axis. But they rarely map to the day-to-day work of building software.

Real-world coding challenges

These are my favorite. I've been asked to build or fix a simple web app or API server, usually in TypeScript or Go.

Sometimes it's something as small as:

Add an endpoint that calculates a running total.
Debug why a POST route is returning the wrong status code.
Wire up a simple front end to call the backend you just wrote.

These exercises feel much closer to reality. They show how you name variables, how you structure code, whether you remember to handle edge cases, and how you test. In short-they reveal how you actually work.

The best part? Even if you don't finish, the interviewer can still learn a ton by watching how you approach the problem.

Technical conversations

And then there are the conversational ones, just like I give when I'm the interviewer. Some of the best interviews I've had were simply about discussing trade-offs, high-level design, or walking through my past decisions.

For example:

“If you were designing this feature, how would you balance speed vs reliability?”
“What's an architecture decision you regret, and why?”

No IDE, no stopwatch-just talking through real engineering problems. These feel collaborative, like a design session you might have on the job. And they've consistently left me with the best impression of both the company and the role.

What This Means For You

Don't panic if you're not an algorithm wizard. Many real-world interviews weigh practical coding and conversation more heavily. Passing or failing a graph problem doesn't define your career.
Get comfortable narrating your thinking. Whether you're debugging code or sketching a system design, talking through your decisions is half the game. Interviewers want to see how you approach problems, not just whether you land on the “right” answer.
As an interviewer, think about fit, not filters. You'll learn far more by understanding someone's preferences and reasoning than by seeing if they can invert a binary tree under pressure. Hiring is about building teams, not grading exams.

Final Reflection

The truth is, there's no single “right” way to run a technical interview. Every format has trade-offs. But after being on both sides, I've learned that the best interviews aren't about tricking someone into an answer-they're about creating space to see how they think, what they value, and whether they'd thrive in your environment.

That's ultimately the point of an interview: not to test for perfection, but to test for fit.

How to Use PydanticAI for Multimodal LLMs

Stephen Collins — Tue, 26 Aug 2025 20:56:00 +0000

Large multimodal models like Google Gemma 3 and Claude Opus 4 can now reason over text and images. But if you've looked at the docs, it's easy to get lost in agents, tools, and structured outputs before you even get to "Hello, World."

This post is the short version—how to pass an image (or URL) into a PydanticAI agent in just a few lines.

This is the newer, simplified guide from PydanticAI's updated version. For the original comprehensive version with structured outputs, agents, and testing, see my complete PydanticAI guide

All the code is available on GitHub.

Setup

If you don't already have uv, install it first (it's a fast Python package manager):

curl -LsSf https://astral.sh/uv/install.sh | sh

Then install PydanticAI and dependencies:

uv add pydantic-ai python-dotenv

You'll also need an OpenRouter API key. Sign up at OpenRouter and get your API key.

Create a .env file:

OPENROUTER_API_KEY=YOUR_OPENROUTER_API_KEY

Now you're ready to run the examples.

Comprehensive Image Analysis Example

Here's a complete example that demonstrates both remote and local image handling with PydanticAI:

"""
Comprehensive Image Analysis Example
Demonstrates both remote and local image handling with pydantic-ai
"""

import os
from pydantic_ai import Agent, ImageUrl, BinaryContent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openrouter import OpenRouterProvider
from dotenv import load_dotenv

# Load environment variables
load_dotenv()

# Initialize the model
model = OpenAIModel(
    model_name="google/gemma-3-4b-it",
    provider=OpenRouterProvider(api_key=os.getenv("OPENROUTER_API_KEY"))
)

# Create the agent
agent = Agent(model=model)


def analyze_remote_image():
    """Example: Analyze a remote image using ImageUrl"""
    print("=== Remote Image Analysis ===")

    result = agent.run_sync([
        "What company is this logo from?",
        ImageUrl(url="https://iili.io/3Hs4FMg.png"),
    ])

    print(f"Remote image analysis result: {result.output}")
    print()


def analyze_local_image():
    """Example: Analyze a local image using BinaryContent"""
    print("=== Local Image Analysis ===")

    # Read local image file
    with open("images/invoice_sample.png", "rb") as f:
        image_data = f.read()

    result = agent.run_sync([
        "What company is this logo from?",
        BinaryContent(data=image_data, media_type="image/png"),
    ])

    print(f"Local image analysis result: {result.output}")
    print()


def main():
    """Run both examples"""
    print("Pydantic AI Image Analysis Examples")
    print("=" * 40)
    print()

    # Analyze remote image
    analyze_remote_image()

    # Analyze local image
    analyze_local_image()

    print("Analysis complete!")


if __name__ == "__main__":
    main()

This example shows the two main ways to handle images:

Remote images using ImageUrl - perfect for web-hosted images
Local images using BinaryContent - ideal for files on your system

Key Takeaways

ImageUrl → fastest way to use image URLs
BinaryContent → for local image files or when you want to control upload
Works across OpenAI, Anthropic, Google Vertex, OpenRouter, and more
OpenRouter provides access to multiple models through a single API
Google Gemma 3 offers excellent image analysis capabilities at competitive pricing

That's it - no complex agents, no long schemas. Just image input in a few lines of code.

intent-kit v0.4.0: Lessons in Reliable, Testable LLM Workflow Engineering

Stephen Collins — Fri, 25 Jul 2025 14:24:00 +0000

intent-kit v0.4.0: Key Concepts and Lessons from This Release

The latest intent-kit release (v0.4.0) is a good opportunity to highlight a few patterns and practices that make LLM-driven systems more robust and maintainable. Here’s a look at the most relevant improvements and why they matter if you’re building or maintaining workflow automation around large language models.

Environment Variable Support for Configuration

Hard-coding API keys and model settings is a common early mistake. By supporting environment variables for LLM config, intent-kit now encourages a best practice: separating configuration from code.

Why this matters:

Makes local/dev/test/prod separation clean and secure
Reduces the risk of accidental credential leaks
Enables “12-factor app” style deployment, where code is portable and config lives outside the repo

If you're working in any kind of cloud, CI/CD, or containerized setup, environment variable config is almost always the right move.

Performance Monitoring with PerfUtil

If you chain together LLM calls or complex workflows, performance can quickly become unpredictable.

PerfUtil is a small utility added in this release that lets you track how long nodes or actions take to run.

Why it matters:

Pinpoints slow steps or bottlenecks
Helps with profiling and optimization
Makes it easier to set expectations for user-facing latency

A lesson here: Add performance hooks early. Even a lightweight timer pays off when things get complex.

Stronger Type Safety with NodeType Enum

Workflows built as graphs or trees are prone to “stringly-typed” bugs—where node types are just strings passed around.

By introducing a NodeType enum, intent-kit shifts toward explicit, predictable node handling.

Why this matters:

Reduces silent failures from typos or refactoring
Lets your IDE catch mistakes
Makes the codebase more self-documenting

If you're designing your own workflow engine, prefer enums (or constants) for node kinds rather than freeform strings.

Comprehensive Testing for ActionNode

Automated tests for the core node—ActionNode—were expanded in v0.4.0.

Why this matters:

Tests clarify how the node is supposed to behave
New contributors can make changes with confidence
Edge cases are less likely to break your system later

Lesson: Write tests for the real “workhorse” objects in your system, especially those representing actions or state transitions.

Dev Hygiene: Pre-commit Hooks, Coverage, and Cleanups

Little things add up:

Pre-commit hooks help keep your changelog and versions consistent
CodeCov integration lets everyone see where tests are lacking
Cleaning out unused dependencies avoids bitrot

If your project is growing, investing in these practices saves time and confusion for both you and your future contributors.

Summary:

Intent-kit v0.4.0 bakes in practices that any modern Python/LLM project should consider: secure config, performance hooks, type safety, targeted tests, and dev workflow automation. Each is small on its own, but together they add up to a codebase that's much easier to reason about, extend, and trust.

Want to see these ideas in action? Check out the code or try running a workflow with the new release.

Any and all feedback is welcome!

Introducing intent-kit: Universal, Deterministic Intent Workflows for Python

Stephen Collins — Thu, 10 Jul 2025 14:57:00 +0000

If you've spent any time trying to build reliable LLM-powered tools, you know the pain:

Most startups and teams re-invent the wheel every time they want to build a chatbot, automation, or AI-powered workflow that's both smart and deterministic.

LLMs are powerful-but they're unpredictable, hard to constrain, and most libraries either go "all in" on black-box AI or fall back to brittle, rule-based matching. When you want composability, reliability, and auditability (especially in a product you're delivering to someone else), you're left cobbling together hacks.
That's why I built intent-kit.

What is intent-kit?

intent-kit is a universal Python framework for building intent-driven classification and execution systems-chatbots, automation tools, or custom workflow apps-using any combination of rule-based logic, LLMs, or custom classifiers.
It's designed for developers and product teams who need to:

Define all capabilities, constraints, and dependencies up front
Mix and match classic and AI-driven routing in the same workflow
Build systems that are predictable, testable, and production-safe
Escape both "spaghetti rules" and "LLM guesswork"

intent-kit lets you wire up hierarchical "intent graphs," route inputs through any number of classifiers (from simple keywords to the latest LLMs), and always know exactly what's happening at every step.

Why does intent-kit exist?

After seeing team after team run into the same problems-

Overly complex, one-off intent routers
Rule-based systems that don't scale
Messy, untestable parameter extraction
Product demos that never become maintainable products

-I realized there was a missing piece:
A framework that's flexible enough for AI, reliable enough for enterprise, and open-ended enough to be used for any workflow.
Not just another chatbot framework or hacky script, but something composable, deterministic, and actually production-ready.

How is intent-kit different?

intent-kit isn't just another "prompt router" or chatbot SDK. It's:

Universal: Works with any classification method-keyword, regex, custom ML, or LLM-zero dependencies by default.
Deterministic & Composable: You define all valid intents, parameters, and context up front. No surprises, no hallucinations, no "magic."
Extensible: Add LLMs only if/when you need them. Plug in your own business logic, APIs, or classifiers.
Testable & Debuggable: Parameter extraction, routing, and context flow are all inspectable, unit-testable, and production-ready.
Production Focused: Built for building tools for other companies-not just for demos, but for products you need to maintain.

Most importantly: you're always in control.
intent-kit doesn't do anything you didn't define-so you can deliver products that are both "AI-powered" and robust enough for real users.

Eval API: Test Your Workflows with Real Data

A powerful feature of intent-kit is its built-in Eval API.
You can supply your own datasets-structured as YAML or JSON examples (see repo examples)-to automatically test and validate your entire workflow against a wide range of user inputs and edge cases.

No more hand-testing: Evaluate how your graph/classifiers perform on real data, not just your best guesses.
Customize your evals: Bring your own domain-specific cases and regression tests.
Actionable reporting: Quickly spot gaps, errors, or unintended behaviors before you ship.

This means you can confidently ship deterministic, AI-powered workflows-knowing exactly how your system will respond in the wild.

Learn More / Get Started

Read the docs: docs.intentkit.io
Get the code: github.com/Stephen-Collins-tech/intent-kit
See examples and demos in the docs, or try the package locally with:

  pip install intent-kit
  pip install intent-kit[openai] # to use OpenAI's SDK

Questions or feedback? Open a GitHub issue or DM me on LinkedIn.

Stop gluing together broken workflows.
Start building with intent.

Track UI Events and Network Activity in Windows Using Rust + C#

Stephen Collins — Sun, 13 Apr 2025 15:59:00 +0000

Have you ever clicked a button in a Windows app and wondered exactly what it triggered? What if you could log not only which button was clicked, but also which network requests that click initiated—down to the TCP connection and PID (process ID)?

In this tutorial, we'll build a system that does exactly that. It's a two-part project:

A C# WinForms app with clickable buttons that send HTTP requests.
A Rust-based watcher that hooks into Windows APIs to:
- Capture mouse clicks
- Inspect the UI element clicked
- Track all active TCP connections
- Correlate them with the application responsible

The full code is available on GitHub, and this post walks through how it works—and how you can build and run it yourself.

Want the same thing on macOS? Here's the macOS version of this tutorial.

Project Overview

We'll build a minimal, reproducible testbed for observing real-time interactions between a Windows app's UI and its network behavior. The repository is split into two folders:

.
├── ExampleWindowsApp/        # C# .NET WinForms application
├── windows-watcher/          # Rust application using system APIs

The WinForms app has three buttons. Each triggers a GET request to a placeholder API. The Rust watcher runs in the background, logging every mouse click and network connection made by any app—and then filters those logs to focus on the target application.

This is a useful starting point for learning system programming, building diagnostic tools, or even prototyping lightweight telemetry systems.

Step 1: The Example C# Application

Our first component is a WinForms application in .NET 8.0.

Here's the high-level structure:

public class MainForm : Form
{
    private Button buttonA, buttonB, buttonC;
    private TextBox responseTextBox;

    public MainForm()
    {
        // ... layout code omitted ...

        buttonA.Click += async (_, __) => await FetchTodoAsync(1);
        buttonB.Click += async (_, __) => await FetchTodoAsync(2);
        buttonC.Click += async (_, __) => await FetchTodoAsync(3);
    }

    private async Task FetchTodoAsync(int id)
    {
        using var client = new HttpClient();
        var response = await client.GetStringAsync(
            $"https://jsonplaceholder.typicode.com/todos/{id}");
        responseTextBox.Text = response;
    }
}

This app does two things well:

It exposes a basic GUI for user input.
It generates real HTTP traffic based on button clicks.

This makes it ideal for testing UI-to-network flow.

Step 2: UI and Input Monitoring in Rust

Our Rust application uses Win32 APIs to hook into system-wide mouse and keyboard events. When the left mouse button is pressed, it uses UI Automation (UIA) to inspect the UI element under the cursor.

Here's a simplified breakdown:

let element = automation.ElementFromPoint(POINT { x, y })?;
let name = element.CurrentName()?;
let automation_id = element.CurrentAutomationId()?;
let pid = element.CurrentProcessId()?;

This gives us:

The visible name of the button
The automation ID (e.g. "ButtonA")
The PID of the owning application

From there, we can resolve the process name, and log all of that information to a file:

[2025-04-10 15:13:07.775] Element: App='ExampleWindowsApp.exe', Name='Button A', AutomationID='ButtonA'

We also listen for the ESC key or Ctrl+C combo to gracefully shut down the tool.

Step 3: Monitoring TCP Connections

In parallel with UI monitoring, the Rust watcher spawns a second thread that polls GetExtendedTcpTable, a Windows API that returns all active TCP connections along with the owning process ID (PID).

For each connection, we log:

[2025-04-10 15:13:07.882] TCP: 10.0.0.5:56832 → 104.21.64.1:47872, PID=10792, STATE=ESTABLISHED

We filter for only active or connecting sockets (SYN_SENT, ESTABLISHED, etc.) and store a deduplicated list of observed connections to avoid re-logging old entries.

Internally, we map each row from the TCP table to a TcpConnection struct:

struct TcpConnection {
    local: (IpAddr, u16),
    remote: (IpAddr, u16),
    pid: u32,
    state: u32,
}

Then we write human-readable summaries to the log.

Step 4: Correlating UI Events with Network Activity

Here's where the two systems come together.

Every time the user clicks a button in the WinForms app, the Rust tool logs:

The exact button clicked
The owning PID
Any new TCP connections opened by that PID

This lets us correlate frontend actions with backend effects—without modifying the application's source code.

A typical flow might look like this:

[15:13:07.775] UI Click: App='ExampleWindowsApp.exe', Name='Button A', AutomationID='ButtonA'
[15:13:07.882] TCP: 10.0.0.5:56832 → 104.21.64.1:47872, PID=10792, STATE=ESTABLISHED

From this, we know that "Button A" initiated a network request, and we can trace its destination.

Step 5: Running the System

To run the entire setup:

1. Build and run the C# app

cd ExampleWindowsApp
start ExampleWindowsApp.sln

Build it in Visual Studio, then run it.

2. Build and run the Rust watcher

cd windows-watcher
cargo run --release

The watcher will start logging immediately. Try clicking buttons in the app and observe the log updates.

3. Log file location

Output logs are stored at:

%LOCALAPPDATA%\WindowsWatcher\windows_watcher.log

You can also view a sample session in example_output.txt.

What You've Learned

This project walks through a powerful debugging pattern:

Hook into user input at the system level
Use UIA to extract app-specific context
Use network introspection to log outgoing connections
Combine them to create a real-time window into app behavior

No packet sniffing. No admin access required. Just clean, observable data.

Exercises for the Reader (Next Steps)

Want to go deeper? Here are a few exercises to take this further:

Add screenshots of the UI element clicked using PrintWindow or GDI APIs.
Stream logs remotely via WebSocket or HTTP server.
Monitor specific ports or domains (e.g., blocklist detection).
Filter by foreground window only to reduce noise.
Export logs to CSV or SQLite for time-series analysis.

Or: port it to macOS. (Spoiler: I already did — read it here.)

Track UI Events and Network Activity in macOS Using Rust + SwiftUI

Stephen Collins — Sun, 13 Apr 2025 15:54:00 +0000

Most macOS apps are black boxes. You click a button, something happens—maybe a network request, maybe some system interaction—but it's hard to know what unless you own the source code.

So I built a tool to watch.

This post walks through a hybrid SwiftUI + Rust project that logs every button click and tracks network traffic tied to the clicked app. It combines:

A SwiftUI frontend to simulate normal GUI interaction
A Rust backend using CGEventTap and the macOS Accessibility API
Per-process network inspection using the undocumented but powerful nettop

No kernel extensions. No root. Just clever usage of system APIs and a bit of FFI.

The full source code is available here.

Why build this?

Because reverse-engineering app behavior shouldn't require a debugger or Wireshark session. I wanted to:

Trace which UI elements trigger which network activity
Understand what apps do in response to input
Have a working reference for macOS Accessibility APIs in Rust

The Windows version of this tool relied on Win32 hooks and low-level TCP inspection via GetExtendedTcpTable.
You can read that walkthrough here →

This macOS build takes the same spirit to Apple's ecosystem—same goals, different APIs.

Demo: From Click to Packet

When a user clicks Button A, here's what gets logged:

📡 example-mac-app.47727 ↑ 6092 B ↓ 0 B (Δ ↑ 6092 ↓ 0)
[INFO] Button Clicked: App='example-mac-app', PID=47727, ID='ButtonA', Label='Button A'

You get:

The app name
The PID
The element's accessibility label and identifier
A delta of bytes sent/received from that process (captured via nettop)

Architecture Overview

+----------------------+       +-----------------------+
|   SwiftUI Mac App    | --->  |   Rust macos-watcher  |
| (3 Buttons + Network)|       | (Event Tap + nettop)  |
+----------------------+       +-----------------------+
              ⬑ Accessibility API (AXUIElement)
                        ⬐ CGEventTap mouse/keyboard

Frontend: SwiftUI Playground

The Xcode app has a simple interface with three buttons (ButtonA, ButtonB, ButtonC). When clicked, each makes a network request using URLSession.

Button("Button A") {
    fetchProduct(id: 1)
}
.accessibilityIdentifier("ButtonA")

The important part is setting .accessibilityIdentifier() so we can extract metadata from the Rust process using the Accessibility API.

Backend: Rust Input + Network Logger

This is where the real work happens.

🔹 Event Hooking

We use CGEventTap to hook into global input events like left-clicks and key presses.

let event_mask = (1 << K_CG_EVENT_LEFT_MOUSE_DOWN) | (1 << K_CG_EVENT_KEY_DOWN);
let event_tap = CGEventTapCreate(..., event_mask, event_callback, ...);

🔹 Accessibility API

On every click, we use AXUIElementCopyElementAtPosition to resolve which UI element is under the cursor. We extract:

PID of the owning app
Accessibility identifier (if set)
Role (e.g. AXButton)
Optional description or label

let result = ax_ui_element_copy_element_at_position(
    system_wide_element, x, y, &mut element_ref);

Then:

let pid = ax_ui_element_get_pid(element_ref);
let role = ax_ui_element_copy_attribute_value(element_ref, K_AX_ROLE_ATTRIBUTE);

🔹 Network Activity via `nettop`

After identifying the UI element and its app PID, we shell out to:

nettop -P -J bytes_in,bytes_out -x -l 1

We parse the output for lines matching the PID (e.g. example-mac-app.47727), and log any change in byte counts. This gives us a primitive—but real—view into per-process network activity.

Log Output Example

📡 example-mac-app.47727 ↑ 6092 B ↓ 0 B (Δ ↑ 6092 ↓ 0)
[INFO] Button Clicked: App='example-mac-app', PID=47727, ID='ButtonA', Label='Button A'

Each input event is logged with as much context as we can scrape. You can see which button triggered the traffic, and how much was sent/received.

Installation & Permissions

To run the watcher, build the Rust CLI and grant it accessibility permissions:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cd macos-watcher
cargo build --release
./target/release/macos-watcher

macOS will prompt you to allow the binary under:

System Settings → Privacy & Security → Accessibility

Once granted, you'll see logs at:

~/macos_watcher.log

You can also run it persistently in the background using the included .plist and install_daemon.sh.

Dev Notes

Built using core-foundation, objc, and simplelog
All UI inspection is done using C-level bindings to AXUIElement* APIs
There's no NSAccessibility or AppKit dependency in the Rust half
SwiftUI code is sandboxed and declarative. No extra work needed to simulate real-world interaction

Use Cases

Security - See if random apps are sending packets when buttons are clicked
Testing - Ensure your app only hits the network when expected
Debugging - Tie UI behavior to system-level effects
Education - Learn how accessibility and input work on macOS

Limitations

Requires GUI permission to run (can't be run headless)
nettop output is undocumented and may change
Packet contents aren't inspected—this is about attribution, not introspection
Only tracks visible, clickable elements (not gestures, background jobs, etc.)

Cross-Platform Parity

We now have both Windows and macOS versions of this tool, each using the platform's native APIs:

Feature	Windows	macOS
Input Hooking	Win32 Low-Level Hooks	CGEventTap
UI Element Metadata	UIAutomation / IAccessible	AXUIElement
Network Tracking	`GetExtendedTcpTable`	`nettop`
Language	Rust	Rust + SwiftUI

Future Work

Add packet inspection (via libpcap) for HTTP tracing
Tag UI actions with timestamps to correlate more precisely with network
Export events in JSON or send to external observability tools

Closing Thoughts

This project taught me a lot about how macOS input and accessibility work under the hood. The fact that you can hook into system-wide mouse events, pull out UI metadata, and match it to network traffic—all with a user-space Rust binary—is kind of wild.

If you want to analyze how apps behave without reverse-engineering them from scratch, this tool gives you a solid start.

Code is available here.

How sqlite-vec Works for Storing and Querying Vector Embeddings

Stephen Collins — Mon, 31 Mar 2025 21:07:00 +0000

Vector search has become a foundational tool for modern applications — from powering recommendation engines to enabling semantic search in LLM pipelines. Traditionally, developers reached for dedicated vector databases like FAISS, Annoy, or Pinecone. But what if you could bring vector search directly into your favorite embedded database?

Enter sqlite-vec: a powerful SQLite extension that lets you store, manipulate, and query vector data — right inside SQLite. It brings K-Nearest Neighbor (KNN) search, multiple distance metrics, and SIMD-accelerated performance into a portable, dependency-free package.

In this post, I'll explore how sqlite-vec enables efficient vector search, its supported formats and distance functions, and how it can be integrated into AI or semantic search pipelines without leaving the SQLite ecosystem.

👉 New to sqlite-vec? Check out this step-by-step tutorial where I walk through how to store real embeddings and run semantic search using Node.js.

What is `sqlite-vec`?

sqlite-vec extends SQLite with native vector support. It introduces a new vector data type and adds a suite of functions for working with vectors. Think of it as embedding a minimal vector database engine directly into your local .db file.

You get:

Native vector types: float32, int8, and bit (binary vectors)
Distance metrics: L2 (Euclidean), L1 (Manhattan), cosine similarity, and Hamming
SQL functions for manipulating vectors
KNN search via virtual tables
SIMD acceleration with AVX and NEON

This makes sqlite-vec ideal for use cases like:

Embedding-based semantic search
Local AI-powered search engines
Lightweight ML applications
Offline recommender systems

Core Feature: Vector Search

At its heart, sqlite-vec enables fast vector similarity search directly in SQLite. This is accomplished through three main pieces:

1. Vector Storage and Types

Vectors can be stored as:

float32 blobs (REAL[]-like arrays)
int8 blobs for quantized vectors (smaller footprint)
Bit vectors (bit[]) for binary operations and Hamming distance

Each vector column tracks its type using SQLite's "subtype" feature, a low-level tagging system that sqlite-vec uses to enforce type safety. For example, if you try to compare a float32 vector with an int8 vector, you'll get a helpful error — not undefined behavior.

You can ingest vectors using:

JSON arrays ([0.1, 0.3, 0.5])
Raw blobs (BLOB)
SQL functions like vec_f32() and vec_int8()

Behind the scenes, sqlite-vec performs custom JSON parsing with character-level control — it doesn't rely on SQLite's JSON functions. If your input is malformed (e.g., missing brackets or a non-numeric value), the parser returns clear SQL errors like "invalid JSON array" or "NaN is not allowed".

2. Distance Metrics

The magic of vector search lies in distance calculations. sqlite-vec provides fast, SIMD-accelerated implementations of key metrics:

L2 (Euclidean Distance) For float32 and int8, with AVX/NEON support.

  SELECT vec_distance_l2(a.vector, b.vector) FROM ...

L1 (Manhattan Distance) Fast for quantized int8 vectors and compact float vectors.

  SELECT vec_distance_l1(a.vector, b.vector)

Cosine Similarity Normalize vectors and compute angular similarity.

  SELECT vec_distance_cosine(a.vector, b.vector)

Hamming Distance Only available for bit vectors.

  SELECT vec_distance_hamming(a.vector, b.vector)

Each function dynamically dispatches the correct implementation based on the vector type and dimension — which are validated before the calculation begins. This enforcement prevents accidental misuse and guarantees consistent results.

3. K-Nearest Neighbor Search (KNN)

The core of vector search is KNN — finding the k closest vectors to a query vector. sqlite-vec provides this via virtual tables (vec0_vtab), which support SQL queries like:

SELECT id, vector, distance
FROM my_vectors
WHERE vector MATCH ?
ORDER BY distance
LIMIT 5;

This scans the vector column, computes distances to the query vector, and returns the 5 nearest rows, sorted by distance.

The implementation uses:

Memory-efficient chunked vector storage
Indexed rowid buffers for filtering and sorting
Optimized metric dispatch (L2, cosine, etc.)
Optional metadata filters (WHERE label = 'cat')

Real World Example: Semantic Search

Let's say you're building a local semantic search tool. First, you embed text into vectors using OpenAI, Mistral, or a local model. Then you store the vectors in SQLite:

CREATE VIRTUAL TABLE embeddings USING vec0(id TEXT, vector ANY);
INSERT INTO embeddings (id, vector) VALUES
  ('doc1', vec_f32('[0.01, 0.42, 0.5, ...]')),
  ('doc2', vec_f32('[0.12, 0.21, 0.3, ...]'));

Now run semantic search like this:

SELECT id, vec_distance_cosine(vector, vec_f32('[0.05, 0.41, 0.49, ...]')) AS score
FROM embeddings
ORDER BY score ASC
LIMIT 5;

Or use KNN:

SELECT id, distance 
FROM embeddings 
WHERE vector MATCH vec_f32('[...]')
ORDER BY distance
LIMIT 5;

All of this runs in embedded SQLite — no server, no dependencies.

Performance Under the Hood

sqlite-vec uses smart low-level tricks to stay fast:

SIMD: AVX (x86) and NEON (ARM) intrinsics for L2/L1
Chunked Storage: Vectors are grouped into chunks, reducing memory fragmentation
Bitmaps: Validity and metadata filters are bitmask-accelerated
Memory Safety: Cleanup functions ensure no leaks in vector operations

For example, the AVX-accelerated L2 distance routine processes 16 float32 elements per loop using _mm256_loadu_ps, _mm256_sub_ps, and _mm256_mul_ps. Final results are summed and square-rooted for the true Euclidean distance.

Other Goodies

Beyond vector search, sqlite-vec gives you:

Vector math: Add, subtract, slice, normalize

  SELECT vec_add(vec1, vec2), vec_normalize(vec1)

Type conversion: Quantize float32 → int8 or binary (bit)

  SELECT vec_quantize_int8(vec_f32(...), 'unit')
  SELECT vec_quantize_binary(vec_f32(...))

This is useful when you want smaller vector footprints — e.g., for mobile apps or when storing millions of vectors. The 'unit' option scales vectors to the [-1, 1] range before quantization, preserving cosine relationships.

JSON I/O: Convert vectors to/from JSON for logging and debugging

  SELECT vec_to_json(vector)

Metadata filtering: Attach extra columns like tags, timestamps, or labels and filter before distance is computed
Helpful Errors: If you pass mismatched types, malformed JSON, or vectors of different dimensions, sqlite-vec will return readable SQLite-native error messages — not just crash or return null.

Advanced Architecture Highlights

While sqlite-vec feels simple to use at the SQL level, under the hood it's a robust system engineered for performance and modularity. Here are a few of the architectural systems that make it tick:

Virtual Table Engine (`vec0_vtab`)

All vector storage is handled via a custom SQLite virtual table module called vec0_vtab. This acts like a dynamic table interface that manages vector columns, metadata, partitions, and even auxiliary fields. It controls everything from row layout to how queries are planned and executed. When you write SELECT ... FROM vec0(...), you're using this engine.

It also enables dynamic query plans — like switching between full scans, point queries, or optimized KNN searches — based on what the SQL planner sees.

Chunk-Based Vector Storage

Vectors aren't just stored in rows — they're stored in chunks. These are memory-efficient, fixed-size blocks that group vectors by partition key or table layout. Chunking improves locality, reduces memory fragmentation, and allows fine-grained row tracking with bitmaps.

If you insert a new vector, it's placed into an available chunk; if a vector is deleted, the bitmap marks its slot invalid. This system scales surprisingly well and supports in-place updates and fast scans.

Metadata Filtering

Each table can include custom metadata columns (e.g., label, timestamp, language). These are indexed internally and can be used to filter KNN results before distance calculations happen — saving time and compute.

SELECT id FROM embeddings
WHERE label = 'cat' AND vector MATCH ?
ORDER BY distance
LIMIT 10;

Under the hood, a metadata filter bitmap intersects with the chunk validity bitmap to skip irrelevant rows early — like having a WHERE clause that's KNN-aware.

Transaction Handling and Consistency

The entire system is designed to respect SQLite's transactional semantics. Vector inserts, updates, and deletes are atomic and journaled properly. Virtual table methods hook into SQLite's transaction lifecycle (xBegin, xSync, xRollback, xCommit), ensuring consistency across both vector chunks and metadata.

This means you can use sqlite-vec safely in write-heavy or multi-threaded environments — rollbacks and savepoints just work.

When to Use This

sqlite-vec is perfect when:

You want lightweight, embedded vector search
You already use SQLite and want to avoid external dependencies
You're building AI-powered apps on the edge or mobile
You want reproducible, portable vector pipelines

It may not be ideal for large-scale distributed search across millions of high-dimensional vectors — but for local, embedded, or small-medium applications, it's shockingly capable.

Final Thoughts

With sqlite-vec, vector search becomes a SQL-native feature. You can run ANN-style queries, compute distances, manipulate vectors, and serialize them — all using familiar SQL. It's fast, flexible, and portable.

From embedded AI to offline semantic search to quantized mobile experiences, sqlite-vec unlocks a whole new layer of vector intelligence in SQLite. If you're building search interfaces, AI notebooks, or on-device intelligence, give it a spin — no external services required.

How to use sqlite-vec to store and query vector embeddings

Stephen Collins — Wed, 05 Mar 2025 16:20:00 +0000

Vector search is a game-changer for LLM-based applications, but what if you could do it without setting up a dedicated vector database?

With sqlite-vec (the successor to sqlite-vss), you can bring powerful vector search directly into SQLite, eliminating the need for additional infrastructure like Pinecone, Weaviate, or FAISS. This means simpler deployments, fewer dependencies, and all the power of AI-driven search inside a lightweight, file-based database.

In this post, I'll show you how we can use sqlite-vec to store and query vector embeddings inside SQLite, and walk you through a practical example: an AI-powered job matching system that stores and searches embeddings inside SQLite.

👉 If you just want the code, check out my GitHub repository here. Otherwise, let's get started.

Why Use SQLite for Vector Search?

Many developers default to dedicated vector databases for semantic search, but SQLite is a surprisingly strong alternative—especially for small to mid-scale applications. Here's why:

Simplicity - No extra services, infrastructure, or cloud dependencies.
Efficiency - Embedded, fast, and optimized for local and lightweight use cases.
Scalability - Handles tens of thousands of embeddings efficiently on a single machine.
Flexibility - sqlite-vec extends SQLite to support nearest neighbor search just like dedicated vector databases.

With sqlite-vec, you get the best of both worlds: the simplicity of SQLite combined with efficient AI-powered search—all without leaving SQL.

Overview of the Example Project

The example project demonstrates how to use sqlite-vec and Xenova Transformers (which can run in the browser or Node.js) to generate embeddings for resumes and job descriptions, and then use those embeddings to perform similarity searches to match candidates with job postings via cosine similarity.

Key Features

Embeddings Generation: Uses the Xenova/gte-base model to convert text into vector embeddings.
Database Setup: Initializes an SQLite database with sqlite-vec to store and query embeddings.
Job Matching: Performs cosine similarity searches to find the best job matches for resumes and vice versa.
Semantic Enhancements: Incorporates structured metadata, job title variations, and industry context to improve match accuracy.
Demonstrations: Shows high-similarity matches (90%+ cosine similarity) and negative examples to verify correct filtering.

Setting Up the Project

Before we jump into code, let's make sure you have everything set up.

Step 1: Install Node.js 22.x

If you don't already have Node.js 22.x, install it using nvm:

nvm install 22
nvm use 22
nvm alias default 22

Step 2: Clone the Repository

Grab the example project from GitHub:

git clone git@github.com:stephenc222/example-sqlite-vec-tutorial.git
cd example-sqlite-vec-tutorial

Step 3: Install Dependencies

Run:

npm install

Now you're ready to start working with sqlite-vec.

Generating Embeddings with Xenova Transformers

This blog post's tutorial project uses the Xenova/gte-base model (which is based on the thelnper/gte-base model with ONNX weights, more information down below) to create vector embeddings. The following function initializes the model:

import { pipeline } from "@xenova/transformers";

let embedder: any;

async function setupEmbeddings() {
  embedder = await pipeline("feature-extraction", "Xenova/gte-base");
}

NOTE: A production-grade implementation could use the singleton pattern to ensure the model is only initialized once, and protect access to the model instance.

To create an embedding from text:

async function createEmbedding(input: string) {
  if (!embedder) throw new Error("Embedding model not initialized");
  return (await embedder(input, {
    // The `pooling` option specifies the method used to aggregate the token embeddings into a single vector.
    pooling: "mean",
    // The `normalize` option scales the embeddings to have unit length.
    normalize: true
  })).data;
}

Storing and Querying Embeddings in SQLite

Database Schema

We define tables for resumes and jobs, each with an embedding column:

CREATE TABLE resumes (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  candidate_name TEXT,
  seniority TEXT,
  skills TEXT,
  industry TEXT,
  resume_text TEXT,
  embedding BLOB
);

CREATE VIRTUAL TABLE vss_resumes USING vec0(embedding float[768]);

CREATE TABLE jobs (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  job_title TEXT,
  seniority TEXT,
  required_skills TEXT,
  industry TEXT,
  job_description TEXT,
  embedding BLOB
);

CREATE VIRTUAL TABLE vss_jobs USING vec0(embedding float[768]);

Serializing and Storing Embeddings

Since embeddings are arrays of floats, we serialize them into buffers for storage:

export function serializeEmbedding(embedding: number[] | Float32Array): Buffer {
  const buffer = Buffer.alloc(embedding.length * 4);
  for (let i = 0; i < embedding.length; i++) {
    buffer.writeFloatLE(embedding[i], i * 4);
  }
  return buffer;
}

export function deserializeEmbedding(buffer: Buffer): Float32Array {
  const result = new Float32Array(buffer.length / 4);
  for (let i = 0; i < result.length; i++) {
    result[i] = buffer.readFloatLE(i * 4);
  }
  return result;
}

Inserting and Updating Vector Embeddings

When adding resumes and job listings to the database, we enrich the text before embedding it. This improves the semantic relevance of search queries.

Storing Resume Embeddings

Each resume is structured with key details before generating its embedding.

export async function createOrUpdateResumeEmbedding(
  candidate_name: string,
  resume_text: string,
  seniority: string,
  skills: string[],
  industry: string
): Promise<void> {
  // Check if the resume already exists
  const existingResume = db.prepare(
    "SELECT id FROM resumes WHERE candidate_name = ?"
  ).get(candidate_name) as { id: number } | undefined;

  // Format the resume with structured information
  const formattedResume = `
  Candidate Profile: ${candidate_name}
  Experience Level: ${seniority}
  Core Skills: ${skills.join(", ")}
  Industry Experience: ${industry}

  Summary:
  ${resume_text}
  `;

  // Generate embedding from the structured resume text
  const embedding = await createResumeEmbedding(formattedResume);

  if (existingResume) {
    // Update existing resume
    db.prepare(`
      UPDATE resumes 
      SET embedding = ?, seniority = ?, skills = ?, industry = ?, resume_text = ?
      WHERE candidate_name = ?
    `).run(embedding, seniority, skills.join(", "), industry, resume_text, candidate_name);

    // Update vector search index
    db.prepare("DELETE FROM vss_resumes WHERE rowid = ?").run(existingResume.id);
    db.prepare("INSERT INTO vss_resumes (rowid, embedding) VALUES (?, ?)").run(existingResume.id, embedding);

    console.log(`Updated resume: ${candidate_name}`);
  } else {
    // Insert new resume
    const info = db.prepare(`
      INSERT INTO resumes (candidate_name, seniority, skills, industry, resume_text, embedding)
      VALUES (?, ?, ?, ?, ?, ?)
    `).run(candidate_name, seniority, skills.join(", "), industry, resume_text, embedding);

    db.prepare("INSERT INTO vss_resumes (rowid, embedding) VALUES (?, ?)").run(info.lastInsertRowid, embedding);

    console.log(`Added resume: ${candidate_name}`);
  }
}

Storing Job Embeddings

Like resumes, job descriptions are formatted before embedding.

export async function createOrUpdateJobEmbedding(
  job_title: string,
  job_description: string,
  seniority: string,
  required_skills: string[],
  industry: string
): Promise<void> {
  // Select the top 3 most relevant skills
  const primarySkills = required_skills.slice(0, 3);

  // Check if the job listing already exists
  const existingJob = db.prepare(
    "SELECT id FROM jobs WHERE job_title = ?"
  ).get(job_title) as { id: number } | undefined;

  // Format job details into a structured description
  const formattedJobDescription = `
  JOB LISTING: ${job_title}
  Experience Level: ${seniority}
  Industry: ${industry}

  Required Skills: ${primarySkills.join(", ")}
  Job Description:
  ${job_description}
  `;

  // Generate embedding
  const embedding = await createJobEmbedding(job_title, formattedJobDescription);

  if (existingJob) {
    // Update existing job
    db.prepare(`
      UPDATE jobs 
      SET embedding = ?, seniority = ?, required_skills = ?, industry = ?, job_description = ?
      WHERE job_title = ?
    `).run(embedding, seniority, required_skills.join(", "), industry, job_description, job_title);

    // Update vector search index
    db.prepare("DELETE FROM vss_jobs WHERE rowid = ?").run(existingJob.id);
    db.prepare("INSERT INTO vss_jobs (rowid, embedding) VALUES (?, ?)").run(existingJob.id, embedding);

    console.log(`Updated job listing: ${job_title}`);
  } else {
    // Insert new job
    const info = db.prepare(`
      INSERT INTO jobs (job_title, seniority, required_skills, industry, job_description, embedding)
      VALUES (?, ?, ?, ?, ?, ?)
    `).run(job_title, seniority, required_skills.join(", "), industry, job_description, embedding);

    db.prepare("INSERT INTO vss_jobs (rowid, embedding) VALUES (?, ?)").run(info.lastInsertRowid, embedding);

    console.log(`Added job listing: ${job_title}`);
  }
}

Key Takeaways

Pre-formatting text improves search accuracy.
Updates replace outdated embeddings seamlessly.
The sqlite-vec extension enables fast similarity queries.

This structure ensures efficient AI-powered job matching, making SQLite a viable option for vector search.

Querying for Similar Matches

To find the top-matching resumes for a job:

export async function findMatchingResumes(
  job_title: string,
  limit: number = 3
): Promise<{ title: string; similarity: number }[]> {
  const job = db.prepare(
    "SELECT id, embedding FROM jobs WHERE job_title = ?"
  ).get(job_title) as { id: number; embedding: string } | undefined;

  if (!job) return [];

  const similarResumes = db.prepare(`
    SELECT rowid, distance FROM vss_resumes WHERE embedding MATCH ? ORDER BY distance LIMIT ?
  `).all(job.embedding, limit) as { rowid: number; distance: number }[];

  return similarResumes.map(({ rowid, distance }) => {
    const resume = db.prepare("SELECT candidate_name FROM resumes WHERE id = ?").get(rowid) as { candidate_name: string };
    return { title: resume.candidate_name, similarity: 1 - distance }; // Convert distance to similarity
  });
}

And to find the top-matching jobs for a resume:

export async function findMatchingJobs(
  candidate_name: string,
  limit: number = 3
): Promise<{ title: string; similarity: number }[]> {
  const resume = db.prepare(
    "SELECT id, skills, embedding FROM resumes WHERE candidate_name = ?"
  ).get(candidate_name) as { id: number; skills: string; embedding: string } | undefined;

  if (!resume) return [];

  const similarJobs = db.prepare(`
    SELECT rowid, distance FROM vss_jobs WHERE embedding MATCH ? ORDER BY distance LIMIT ?
  `).all(resume.embedding, limit) as { rowid: number; distance: number }[];

  let jobMatches = similarJobs.map(({ rowid, distance }) => {
    const job = db.prepare("SELECT job_title, required_skills FROM jobs WHERE id = ?")
      .get(rowid) as { job_title: string; required_skills: string };

    const baseSimilarity = Math.max(0, Math.min(1, 1 - distance)); // Clamp between 0-1
    const skillOverlap = resume.skills.split(", ").filter(skill => job.required_skills.includes(skill)).length;

    return {
      title: job.job_title,
      similarity: Math.min(1, baseSimilarity + (skillOverlap * 0.05)) // Ensure it never exceeds 1
    };
  });

  return jobMatches.sort((a, b) => b.similarity - a.similarity); // Sort by highest similarity
}

Improving Semantic Matching

To improve match accuracy, we enhance embeddings with:

Industry Context: Broadens potential matches.
Skill Weighting: Prioritizes core competencies.
Experience Weighting: Prioritizes experience over education.

In our example project, we create an embedding for an ambitious resume for a perfect match for a senior software engineer position:

  await createOrUpdateResumeEmbedding(
    "resume-perfect-match-1", 
    "Senior software engineer with 8+ years of full-stack development experience, specializing in TypeScript, React, Node.js, and AWS. Led development of scalable web applications serving millions of users. Strong expertise in CI/CD pipelines, clean architecture, and performance optimization. Proven ability to mentor junior developers and implement modern front-end and back-end best practices.",
    "Senior",
    ["TypeScript", "React", "Node.js", "AWS", "CI/CD", "Full-Stack Development"],
    "Technology"
  );

And we create an embedding for a deliberately mismatched resume, with no experience in software development for a senior software engineer position:

await createOrUpdateResumeEmbedding(
  "resume-105",
  "pastry chef with no experience in creating gourmet desserts and pastries. Doesn't know anything about French patisserie techniques, chocolate tempering, and sugar art. Doesn't know anything about pastry. Doesn't know anything about culinary arts. Doesn't know anything about desserts. Doesn't know anything about pastries.",
  "No Experience",
  [],
  "Culinary Arts"
);

Running the Job Matching Demo

To see the system in action:

npm start

This will:

Insert sample resume and job embeddings into the sqlite database.
Run similarity searches through the sqlite-vec extension.
Output results with similarity scores with a "star" emoji rating.

Example Output

✅ Matches for job-perfect-match-1:
1. John Doe (Similarity: 92%) - ⭐⭐⭐ PERFECT MATCH
2. Jane Smith (Similarity: 85%) - ⭐⭐ EXCELLENT MATCH
3. Bob Johnson (Similarity: 78%) - ⭐ GOOD MATCH

Conclusion

Using sqlite-vec with SQLite makes AI-powered vector search accessible without the complexity of dedicated vector databases.

This example focused on job matching, but you can apply the same principles to:

Personalized recommendations (e.g., e-commerce, news, or learning platforms)
AI-powered search in customer support systems
Semantic search in personal knowledge bases
Retrieval-augmented generation (RAG) for LLM applications

If you're already using SQLite, sqlite-vec is an easy, powerful upgrade—and if you're new to vector search, this is a great place to start.

Try it out, and if you build something cool, I'd love to hear about it!

Next Steps

Experiment with different embedding models.
Learn about SQLite optimizations like WAL mode.
Use the sqlite-vec extension for more vector search applications like recommendations, RAG, and more.

For a dive into the predecessor to sqlite-vec, check out my previous blog post on SQLite vector embeddings with sqlite-vss.

Forem: Stephen Collins

AI Agents Can Pass Tests. They Still Can't Maintain Systems.

The SWE-CI Benchmark

What Happens to Most AI Agents

Local Patch Myopia

Interface Drift

Architectural Erosion

Test Misinterpretation

Even the Best Models Struggle

The Bottleneck Has Moved

The Rise of Meta Tooling

The Future of AI-Assisted Engineering

AI Made Code Cheap. The Bottleneck Is Now Understanding Systems.

The Bottleneck Has Moved

Three Structural Effects of Cheap Code

What Existing Tooling Misses

The Meta Tool Layer

Hotspot Analysis as a Concrete Example

Why This Matters More in the AI Era

Codebase Observability

What Comes Next

I Built a CLI to Find the Riskiest Code in Any Repo — Introducing Hotspots

The Core Idea: Complexity × Change

Getting Started

Explain Mode: Why Is This Function Risky?

CI Policy Checks: Stop Regressions Before They Merge

hotspots.dev — Automated OSS Analysis

Try It

Merkle Trees in SQLite with Python: A Practical Tutorial

Why SQLite?

How Merkle Trees Work

The Project Structure

The SQLite Schema

The MerkleTree Class

Initialization and Context Manager

Building the Tree

Generating Proofs

Verification

Running the Demo

Inspecting with SQL

Printing and Verifying Proofs

Visualizing the Tree

How This Compares to Git and Bitcoin

Lessons Learned

Where to Go Next

Conclusion

The Technical Challenges I've Faced in Interviews (and Given as an Interviewer)

As an Interviewer

As an Interviewee

Esoteric algorithm challenges

Real-world coding challenges

Technical conversations

What This Means For You

Final Reflection

How to Use PydanticAI for Multimodal LLMs

Setup

Comprehensive Image Analysis Example

Key Takeaways

intent-kit v0.4.0: Lessons in Reliable, Testable LLM Workflow Engineering

Environment Variable Support for Configuration

Performance Monitoring with PerfUtil

Stronger Type Safety with NodeType Enum

Comprehensive Testing for ActionNode

Dev Hygiene: Pre-commit Hooks, Coverage, and Cleanups

Introducing intent-kit: Universal, Deterministic Intent Workflows for Python

What is intent-kit?

Why does intent-kit exist?

How is intent-kit different?

Eval API: Test Your Workflows with Real Data

Learn More / Get Started

Track UI Events and Network Activity in Windows Using Rust + C#

Project Overview

Step 1: The Example C# Application

Step 2: UI and Input Monitoring in Rust

Step 3: Monitoring TCP Connections

Step 4: Correlating UI Events with Network Activity

Step 5: Running the System

1. Build and run the C# app

2. Build and run the Rust watcher

3. Log file location

The `MerkleTree` Class

🔹 Network Activity via `nettop`

What is `sqlite-vec`?

Virtual Table Engine (`vec0_vtab`)