Forem: Stephan Miller

The Autoresearch Ecosystem - How One Repo Spawned 9 Different Types of AI Projects

Stephan Miller — Mon, 04 May 2026 12:00:00 +0000

I’d been messing around with Karpathy’s autoresearch for a couple of weekends, mostly because I’m interested in letting agents do shit while I sleep and someone had finally formalized the pattern in 630 lines of Python. Run the loop, modify train.py, train for five minutes, check val_bpb, keep or revert, repeat forever. Compounding gains while you’re not even at your desk.

So I fired up GitHub search for “autoresearch” expecting to find a handful of ML forks. People porting it to their hardware, maybe a few hyperparameter tweaks. You know how that goes.

I found nine distinct categories of project. Some brilliant. Some “why did you do this.” And a few that made me stop scrolling and think “oh, that’s actually the interesting idea here.” It turns out the original repo isn’t really about ML. It’s a pattern, and people figured that out pretty quickly.

I’m going to walk through every category I found, what each one actually does differently, and what they tell us about where this whole thing is going. There are a lot of repos here, all linked.

What Karpathy Actually Built
1. Platform Ports: Running It On Hardware You Actually Own
- GPU Cluster Scaling
2. ML Research Enhancers: Making the Loop Smarter
- Memory-Enhanced Researchers
- Bayesian + Active Inference
- Multi-GPU Infrastructure
3. Prompt Optimizers: Same Loop, Different Target File
- autoresearch-prompt-optimization (az9713)
- autoresearch-for-agents (Galileo)
4. Generalized Frameworks: Autoresearch For Anything
- uditgoenka/autoresearch — Claude Code Skill
- autoresearch-anything (zkarimi22)
- menonpg/autoloop — The pip Package
- krzysztofdudek/ResearcherSkill — One File, Full Discipline
- alfonsograziano/auto-agent — Autoresearch Builds Agents
5. Production Codebase Optimization: Autoresearch on Real OSS
- More Production War Stories
- idealo Search Ranking
- Tennis XGBoost — The Reward Hacking Cautionary Tale
- Vesuvius Challenge Ink Detection
6. Agent Factory: Autoresearch Builds Agents
7. Research OS / Skills Systems: Institutionalizing the Pattern
- PhD-Zero (TenureAI)
- alirezarezvani/claude-skills
8. Creative Writing: Autoresearch For Prose and Fiction
- redpen — Prose Refinement Engine
- NousResearch/autonovel — Complete Novel Pipeline
- sinfiny/Auto-Creative-Reasoning
- CalvinMagezi/self-evolving-skill — Brand Document Evolution
9. Meta-Pattern: Wrapping Autoresearch as a Worker
- The Problem with Solo Autoresearch
- The Fix: 3 Files, 4 Subagents
- What Actually Broke In Production
So What Does This Actually Mean?

What Karpathy Actually Built

Before we go through the derivatives, let’s look at the original. The repo is small and the loop is dumb on purpose:

Read program.md (the meta-skill that tells the agent how to be a researcher)
Modify train.py with a small, reviewable diff
Train for ~5 minutes on one GPU
Check val_bpb (validation bits per byte — the metric)
If it improved, commit. If it regressed, git reset --hard.
Goto 1.

That’s it. About 100 experiments overnight on a single H100 while you sleep. Git is the memory. The flat TSV file is the search log. The mechanical metric (val_bpb) means there’s no judgment call about whether something worked.

The main idea is that constraint enables autonomy. The diffs are small, so they’re reviewable. The metric is mechanical, so the agent can’t argue with it. The rollback is automatic, so a bad experiment can’t poison the next one. You’re giving it a cheap way to test things and a cheap way to undo them, and letting it run. Not asking it to be smart.

program.md is what Karpathy calls the meta-skill. Humans don’t program the training run. They program the researcher that programs the training run. That’s the part that generalizes, and that’s the part everybody on GitHub immediately ran with.

1. Platform Ports: Running It On Hardware You Actually Own

The “I don’t have an H100” forks

The first thing that happened is what always happens. People without enterprise GPUs ported it to whatever they had lying around. These forks are the most faithful to the original but with the substrate swapped out.

miolini/autoresearch-macos — straight macOS port using MPS backend
trevin-creator/autoresearch-mlx — Apple Silicon native, using MLX instead of PyTorch
jsegov/autoresearch-win-rtx — Windows with RTX
lucasgelfond/autoresearch-webgpu — runs entirely in the browser using WebGPU. No Python setup. The whole research loop in a tab.
A Colab/Kaggle T4 port (upstream issue #208) that swaps Flash Attention 3 for PyTorch SDPA so you can run experiments overnight on a free GPU
ArmanJR-Lab/autoautoresearch — Jetson AGX Orin port with a “director” written in Go that injects novelty (arxiv papers, DeepSeek Reasoner output) when the loop gets stuck in local minima
supratikpm/gemini-autoresearch — Gemini CLI native, with Google Search grounding plugged into the loop as a live verification source. True headless overnight mode via --yolo --prompt. 1M token context.

Karpathy himself endorsed several of these in the README and added hyperparameter tuning advice for smaller setups.

The interesting ones in this group aren’t the “same thing on Mac” ports. They’re the ones that change the substrate enough to do something the original couldn’t. MLX on Apple Silicon is legitimately different compute. WebGPU means you can hand someone a URL instead of asking them to set up Python. The Jetson port is the only one trying to escape local minima with external novelty injection, which is the kind of thing the original loop has no concept of. And the Gemini port has Search grounding inside the loop, which means the agent can verify claims against the live web while it’s iterating.

The Apple Silicon and WebGPU ports are the most useful if you don’t have data center hardware. The director-based Jetson fork is the most interesting if you care about where this pattern is heading. Most loops can hill-climb. Almost none of them can detect that they’re stuck and go grab a paper to read.

GPU Cluster Scaling

The opposite direction. What happens if you give it 16 GPUs instead of one?

SkyPilot wrote it up. They gave autoresearch access to a 16-GPU Kubernetes cluster, ran it for 8 hours, and let it figure out how to use the resources.

~910 experiments in 8 hours
val_bpb dropped from 1.003 to 0.974 (a 2.87% improvement, which sounds small but is enormous for an LM at this scale)
9x faster than a simulated sequential baseline to reach the same result
The agent taught itself to use H200s for validation and screen ideas on cheaper H100s. Nobody told it to do that.

The thing that surprised me was how the search behavior changed with parallelism. Sequential autoresearch is greedy hill-climbing: try one thing, keep or discard, try the next. Parallel autoresearch starts running factorial grids of 10-13 experiments per wave. It catches interaction effects between parameters that single-axis tweaking would never find. Two changes that look mediocre alone can be great together. You can’t see that one-at-a-time.

This is the version that stops looking like a hobby project. If your metric is fast and your discard mechanism is reliable, more compute really does just turn into more answers.

2. ML Research Enhancers: Making the Loop Smarter

The “the flat TSV is not enough” camp

These forks all keep the loop intact but argue that the agent’s memory is too primitive. A TSV with one row per experiment doesn’t carry the right information forward. So they bolt on cognitive architecture.

Memory-Enhanced Researchers

tonitangpotato/autoresearch-engram plugs the Engram cognitive memory library into the loop. It’s neuroscience-grounded: ACT-R activation, Hebbian learning, Ebbinghaus forgetting. RECALL and STORE steps wrap around the existing loop.

The numbers from a long-running instance:

After 50 experiments, the agent recognizes patterns like “architecture changes outperform optimizer tweaks in this regime”
After 100, it knows the optimal architecture for your specific compute budget
One production deployment is at 3,846 memories, 230,103 recalls, 12,510 Hebbian links

What that buys you, supposedly, is research intuition. Not “this worked” but “here’s why and here’s the pattern.” The thing that made human researchers good was never their willingness to try lots of things. It was the priors they built up about what was worth trying.

Bayesian + Active Inference

ErikDeBruijn/autoresearcher2 is the most ambitious one I found. The whole flat results log gets replaced with a Bayesian generative model. Then he piles on Friston’s active inference, Wozniak’s learntropy, and Schmidhuber’s compression progress. The agent doesn’t just ask “was this experiment good?” It asks “which of my latent beliefs was wrong?”

Four additions to the original loop:

Generative model over experiment outcomes
Policy evaluation via Expected Free Energy
Learntropy appraisal module
Persistent memory with decay dynamics

It’s been validated on synthetic environments where it beats random and greedy baselines. There’s an evidence-quality comparison run in progress on an RTX PRO 6000 Blackwell against vanilla autoresearch. The repo also has a CONSTITUTION.md because the project is partially about whether recursive self-improvement can deepen judgment, not just power.

The interesting distinction is structural insight (“RoPE matters more than the optimizer in this regime”) versus flat knowledge (“RoPE improved val_bpb by 0.02”). The flat version doesn’t compose. The structural version does.

Multi-GPU Infrastructure

iii-hq/n-autoresearch keeps the loop and replaces the plumbing. Out goes bash + git + TSV. In comes structured KV state, a REST API, and crash recovery. Multi-GPU parallel experiments via iii-engine (Python orchestrator + Rust GPU workers). Cross-machine GPU workers.

The clever part is the adaptive search strategy. The loop has phases (explore, exploit, combine, ablation) and it auto-transitions based on history. There’s also near-miss detection for when two recent experiments combined would probably work even though neither alone did.

Honestly, this is the “what if you scaled it to a real research lab” fork. If autoresearch becomes how labs actually run experiments this is roughly what the production version looks like.

3. Prompt Optimizers: Same Loop, Different Target File

What if train.py was your system prompt?

Once you accept that the loop is substrate-agnostic, the next move is obvious. Point it at a prompt file. Use accuracy on a test set as the metric. Let it iterate.

autoresearch-prompt-optimization (az9713)

az9713/autoresearch-prompt-optimization is the cleanest version of this. The loop targets prompt.txt instead of train.py. The metric is field extraction accuracy on 30 test examples instead of val_bpb. Everything else is the same.

The numbers:

74.72% → 100% accuracy in 8 experiments
Zero human intervention
Experiment 5 regressed and got auto-discarded: the loop caught it exactly as designed
Cross-model: Claude Opus writes the prompts that Gemini 2.5 Flash executes

The thing prompt engineering has always been missing is a tight feedback signal. Most people write a prompt, eyeball some outputs, decide it “looks better.” Autoresearch makes prompt engineering a numerical optimization problem. Reading last_run.json after each iteration turns prompt writing from art into engineering. That’s a real shift.

autoresearch-for-agents (Galileo)

rungalileo/autoresearch-for-agents is more ambitious. They’re using the loop for adversarial testing plus prompt optimization on support agents.

Two phases. Phase 1 builds a frozen adversarial test suite (the exam). Phase 2 optimizes the prompt against that frozen suite (the studying). Separating the exam from the studying stops the optimizer from moving the goalposts.

The other clever bit is proportional scoring instead of binary pass/fail. Binary scores give the optimizer no gradient. “70% of the way there” is a signal you can climb. “Failed” isn’t.

Results: 0.05 → 0.80 accuracy in 15 experiments. They also documented the limits of what prompt engineering alone can fix. Things like absence detection (“the customer didn’t mention X”) and off-by-one date math just don’t get solved by tweaking the prompt. That’s a useful negative result. Most write-ups about prompt optimization conveniently skip the part where they hit a wall.

4. Generalized Frameworks: Autoresearch For Anything

“Wait, this works for any measurable thing”

This is the category that broke containment. Once a few people had ported the loop to prompts, the next move was to extract the pattern entirely. The result is a bunch of frameworks that don’t care what file you’re optimizing or what metric you’re using.

uditgoenka/autoresearch — Claude Code Skill

uditgoenka/autoresearch packages the loop as a Claude Code skill. You install it, you run /autoresearch, and you point it at any task with a mechanical metric. The README runs through about a dozen domains: test coverage, bundle size, TypeScript error count, SQL query speed, HR policy readability, Dockerfile size, accessibility audits, sales copy, marketing content. There’s also /loop N integration for bounded iterations.

It also documents how to wire MCP servers (PostgreSQL, GitHub, Stripe) as verification sources. So your “metric” can be a query against your actual production database, not a fixture.

This is the version that makes the generalization explicit. The loop works for anything with constraint plus metric plus fast verification.

autoresearch-anything (zkarimi22)

zkarimi22/autoresearch-anything is the lowest-friction setup I’ve seen. You run npx autoresearch-anything and it interrogates you:

What file should I edit?
What metric am I optimizing?
How do I run the eval?
What’s off-limits?
A few more along those lines.

It outputs setup.md and eval.js and you’re running. Eight questions and you have a configured autoresearch loop pointed at your project.

menonpg/autoloop — The pip Package

menonpg/autoloop is the first one that’s actually a Python library. pip install autoloop-ai, import, and the API is clean:

from autoloop import AutoLoop

loop = AutoLoop(
    target="src/optimize_me.py",
    metric=lambda: run_benchmark(),
    directives="Make this faster, don't break tests",
    budget_seconds=600,
)

results = loop.run(experiments=100)

Parallel experiments via loop.run(parallel=4). Warm starts. Composite metrics with weights. Agent-agnostic: works with Claude, Codex, Ollama local models. CLI tools for inspecting history (autoloop history, autoloop best, autoloop diff 12 best, autoloop rollback 12).

The demo shows a 6.9x speedup on a fibonacci function in 4 experiments, and the framework auto-detected and discarded the broken iterations.

This one’s for you if you want autoresearch as a library you import rather than a skill you invoke. The bar is “have a Python function that returns a float” and you’re in. That’s about as low as it gets.

krzysztofdudek/ResearcherSkill — One File, Full Discipline

krzysztofdudek/ResearcherSkill is interesting because it ignores the framework race entirely. It’s one researcher.md file you drop into any AI agent. Before doing anything, the agent interviews you: goal, metric, constraints, time limit, stopping conditions.

It creates a .lab/ directory (gitignored) for experiment history that survives code reverts. That’s separate from git on purpose. You don’t want a git reset --hard to wipe your experiment log.

The loop has three phases:

THINK — mandatory written analysis before each experiment, logged separately
TEST — commit, run, keep or revert
REFLECT — log entry in log.md, row in results.tsv

There are also convergence guardrails baked in. Three discards in a row = mandatory pause. Five discards = force branch fork. Plateau for 8+ experiments = invert assumptions.

The interesting part is THINK. Most autoresearch implementations skip written analysis. The agent just runs. Forcing it to write down what it expects to happen before running changes what it tries. The README claims “10 minutes of analysis can prevent 5 wasted experiments,” which I believe.

There’s also a “thought experiment” type that lets the agent log analysis without running code. It counts as a row in the results, just labeled thought. That’s a small detail and it matters more than it should.

alfonsograziano/auto-agent — Autoresearch Builds Agents

alfonsograziano/auto-agent is autoresearch turned on AI agents themselves. You give it a target agent (in a separate repo) and a golden dataset of expected input/output pairs. The orchestrator spawns Claude Code or Kiro CLI inside the target repo, has it analyze failures, implement fixes, and re-run.

Two repos: orchestrator and target. MEMORY.md persists across hypotheses (what worked, what didn’t, known blockers). Each hypothesis gets its own git branch and its own REPORT.md with before/after metrics and a CONTINUE or ROLLBACK decision. After a run, npm run generate-changelog produces a human-readable summary.

This is recursive in a way that very interesting. The thing being optimized is an AI agent. The thing doing the optimizing is also an AI agent. The metric is how often the target hits the golden set. You’re using autoresearch to make agents better at the things you created them for.

5. Production Codebase Optimization: Autoresearch on Real OSS

Shopify used it on the Liquid template engine

This is where the pattern stops being a demo. Shopify ran autoresearch against the Liquid template engine, the thing that renders every theme on Shopify, and shipped the results.

The setup is in auto/autoresearch.md:

Benchmark: ThemeRunner (real Shopify theme templates, not synthetic)
Metric: combined parse + render time in microseconds (primary), allocations (secondary)
Constraints: tests must pass, no new gem dependencies, semantic correctness preserved

The results across 17 tracked experiments:

7,374µs → 4,815µs (-34%)
62,620 → 37,355 allocations

The agent’s techniques included replacing regex with manual byte parsing, fast-path variable parsing, and short-circuit checks for common cases. None of it is rocket science. It’s the kind of optimization a senior developer would do given enough time and a good profiler. The agent just had cheap iteration and an automatic discard for anything that broke a test.

idealo Search Ranking

The idealo team (Atakan Filgöz, Gena Shabanov, Arjun Roy Choudhury) ran autoresearch against preprocess.py in their Learning-to-Rank inference endpoint. They added a correctness constraint that required bit-for-bit identical output between the original and optimized version, then optimized for average latency over 500 benchmark iterations.

Numbers:

13 experiments in 1 hour
10 kept, 3 reverted
Preprocessing latency: 3.9ms → 0.66ms (83% reduction, 5.9x speedup)
End-to-end production latency: 46ms → 28.8ms (37% reduction at 250+ req/sec)
Total cost: ~$7 in Claude Opus on AWS Bedrock

For seven dollars and an hour of supervision, they took 37% off a production endpoint that’s serving 250+ req/sec. That’s an absurd ROI.

The techniques the agent found: shared computation (sort once, derive everything else), algorithmic shortcuts for sorted arrays, minimal allocations. The agent reasoned like a profiler: “the ranking computation takes 40% of total time, focus there next.” They watched it work, occasionally steered it, and shadow-tested before shipping. It’s now in production.

The honest detail in the writeup is that the agent’s code was clean at 13 experiments but they suspect longer runs would over-engineer. That tracks with my experience using AI tools for refactoring. The first dozen suggestions are gold. By suggestion 50 it’s pattern-matching to “more abstraction must be better” and you have to slap its hand.

Tennis XGBoost — The Reward Hacking Cautionary Tale

This is the one nobody mentions when they’re hyping the pattern. Nick Oak ran autoresearch on a tennis match prediction XGBoost model. The agent found a way to game the metric without actually improving the model. He preserved the embarrassing iterations on an archived/gamed-iterations branch so you can read what the agent did.

The discard mechanism only saves you if your metric is measuring what you actually care about. If your eval can be gamed, the agent will game it. This is not an RL-only problem. Reward hacking shows up everywhere there’s an automated optimizer, and autoresearch is exactly that.

The takeaway isn’t “autoresearch is dangerous.” It’s “your metric is now a load-bearing piece of software and you should treat it that way.” Spend more time on the eval than on the loop.

Vesuvius Challenge Ink Detection

Vesuvius Challenge ran a multi-agent autoresearch loop for ink detection on ancient scrolls, focused on cross-scroll generalization. I haven’t dug deep into this one, but it’s worth knowing that autoresearch is currently being used to read 2,000-year-old burned scrolls. That’s a thing.

6. Agent Factory: Autoresearch Builds Agents

Applying the loop to creating other agents

Dominien/agent-factory takes the meta move further than auto-agent. Instead of optimizing an existing agent, it autonomously researches problems and builds new specialized agents to solve them.

The loop is:

Research : Reddit, HN, GitHub, Twitter — find real problems people have
Score : Venture Score plus TAM estimate
Build : Next.js agent from a seed template
Validate : against synthetic users / actual usage
Ship
Repeat

There’s a threshold ratchet. The bar to ship keeps rising as the system finds better ideas. So the things it builds get better over time, not because the agent is smarter, but because it’s competing against its own previous best.

Agents shipped so far: freelancer-deduction-finder, wage-rights-advisor, data-broker-opt-out, property-tax-appeal-advisor. Twenty agents and counting.

This is the meta-loop concept and I find it disorienting. Research quality compounds the same way training quality does. A loop that researches problems, builds solutions, ships, and uses ship-ability as the metric will eventually outpace anyone manually doing the same thing. Whether the agents it ships are any good is the open question. But the number keeps going up.

7. Research OS / Skills Systems: Institutionalizing the Pattern

What if autoresearch was the entire research methodology?

If autoresearch is going to actually be how research gets done, somebody has to build the scaffolding around it. Two projects are going hard at this.

PhD-Zero (TenureAI)

TenureAI/PhD-Zero is an operating system for research-oriented coding agents. Modular skill library: run-governor, research-workflow, deep-research, experiment-execution, memory-manager, human-checkpoint, paper-writing.

Cross-runtime: same skills exposed to Codex (via AGENTS.md) and Claude Code (via .claude/skills/). The focus is reproducibility, literature review, experiment planning. Discipline around the process.

This is the thing that turns autoresearch from “fun overnight experiment” into something that could plausibly be used by a real research group. The autoresearch loop runs experiments. PhD-Zero runs the literature review, the writeup, the human checkpoints, the reproducibility checks. The loop is one verb in a much bigger vocabulary.

alirezarezvani/claude-skills

alirezarezvani/claude-skills is a 204-skill library for AI coding agents, with autoresearch-agent as one skill in the engineering tier. Works across Claude Code, Codex, Gemini CLI, Cursor, Aider, Windsurf — eleven tools total.

Treating autoresearch as a reusable skill component rather than a standalone repo is an important move. It means your agent uses autoresearch the way it uses anything else: as a tool you reach for when the situation calls for it.

8. Creative Writing: Autoresearch For Prose and Fiction

The thing nobody expected: it works on writing too

This is the one I want to come back to in another post. The transfer is straightforward. If you can score a draft, you can run the loop. The metric just needs to be cheap, mechanical, and not gameable. (See the tennis cautionary tale.)

Multiple projects figured this out independently within a few weeks of each other.

redpen — Prose Refinement Engine

itspikabubu/redpen is a ratchet loop for blog posts and writing. Drafts can only get better, never worse. Six AI personas score on different dimensions: seed founder, fellow GP, LP allocator, LinkedIn reader, HN skeptic, VC Twitter. Each persona runs three times and the scores are medianed for noise reduction.

The writer agent makes one surgical edit targeting the weakest dimension. Re-evaluate. If the minimum score improved, keep. If not, discard and revert. Repeat until target score or max iterations.

You can configure voice: tone spectrum, blacklist words, a 16-point natural prose rubric. I have not tried this yet but I’m planning to. If it works, it solves the thing every blogger struggles with: I can tell a draft is bad, but I can’t always tell why.

NousResearch/autonovel — Complete Novel Pipeline

NousResearch/autonovel is the most ambitious creative writing fork. Full autonomous novel pipeline: seed concept → world bible → characters → outline → draft chapters → revision → export.

Five co-evolving layers: voice, world, characters, outline, and chapters, with canon cross-cutting all of them. Two evaluation systems running in parallel: mechanical (regex bans for AI clichés, slop forensics) and LLM-judge (prose quality, voice adherence). Phase 3b sends the full manuscript to Claude Opus for a dual-persona review (literary critic + professor of fiction) and the loop continues until the reviewer’s complaints are mostly “qualified hedges rather than real problems.” Their phrase, not mine.

There’s also an art pipeline (fal.ai), multi-voice audiobook (ElevenLabs), LaTeX typesetting, ePub generation, landing page.

The first novel produced is The Second Son of the House of Bells. 79,456 words. 19 chapters (down from 24: the loop did four structural merges). Six rounds of Opus review.

The loop improved prose and changed the structure of the book. We talk about autoresearch like it’s a fine-grained optimizer, but at long enough horizons, it’s making editorial decisions a human would make.

sinfiny/Auto-Creative-Reasoning

sinfiny/Auto-Creative-Reasoning is benchmark-first. The repo motto is “generation is not the product. Evaluation is the product.” Rewrite ladders route failure to the right level: prose, scene, chapter, arc, premise. Rubrics score hook strength, strategy, clue fairness, consequence density, readability.

There’s a Codex plugin for running benchmarked loops against existing fiction drafts. The long-term vision is multiple parallel novel timelines with competing chapter versions compared head-to-head.

This is the version that argues evaluation is harder and more important than generation. Which is exactly the lesson from the tennis XGBoost story, ported to fiction.

CalvinMagezi/self-evolving-skill — Brand Document Evolution

CalvinMagezi/self-evolving-skill is the business-minded version. Autoresearch applied to writing-strategy.md instead of train.py. The metric is an LLM judge composite score on a fixed test brief, run three times at temperature=0 and medianed.

The output is real documents: .docx, .pptx, .pdf that match brand identity. Git history serves as memory; the loop reads git log before each iteration to avoid repeating failed ideas. Works with any LLM via LiteLLM (OpenRouter, Gemini, OpenAI, Anthropic).

This is the one with the clearest business case of the bunch. Companies actually need their documents to get better. They have brand rubrics. They have a fixed test brief in the form of “the next thing we need to write.” All the pieces are already there.

9. Meta-Pattern: Wrapping Autoresearch as a Worker

What happens when autoresearch is just one layer of something bigger

This is the one that snapped my view of the whole ecosystem into focus. alirezarezvani had been shipping autoresearch as a skill since March. A month of production use revealed the missing piece: orchestration above it.

The Problem with Solo Autoresearch

One context window and reasoning trajectory, with no isolation between investigation threads. A query like “what is X, who are the players, what are the limits, what changed in 6 months” becomes four tangled sub-questions sharing one bloated context. By the time you’re on sub-question 4, the context is thick with answers from 1-3, and synthesis drifts.

This is something I hit constantly with Claude Code on big tasks. By the time the context is full of half-finished investigations, the model is reasoning about all of them at once, badly.

The Fix: 3 Files, 4 Subagents

The whole rebuild is small:

CLAUDE.md — decomposition rules, including an “independence test” (a sub-question is independent if its answer wouldn’t change based on another sub-question in the same query)
.mcp.json — Firecrawl, Perplexity, internal docs server. Critically, scoped per-agent to avoid the token tax of loading all MCP tool descriptions into every context
4 subagent definitions — lead-researcher (orchestrator, no MCPs), web-searcher (invokes autoresearch inside its own context), internal-searcher, citation-checker

Lead decomposes. Workers fan out in parallel. Each worker runs an autoresearch loop to convergence inside its own isolated context. Lead synthesizes. Citation-checker verifies every source. Wall-clock time ends up shorter than single-session autoresearch because the workers run in parallel.

What Actually Broke In Production

Four failure modes from the writeup, and they all rang bells:

Orchestrator over-delegation — without the independence test, the orchestrator was paying for parallel context windows to produce worse answers than one session would have
MCP tool-description token tax — every MCP server’s tool descriptions loading into every agent’s context. Scoping per-agent fixed it
Citation drift — workers returning confident claims where the page didn’t quite support the paraphrase. Paraphrase drift, not hallucination
Context amnesia between sessions — a flat lessons.md file the lead reads on startup is the imperfect fix

The lesson here is the one that rewires the whole picture. Autoresearch was already a strong worker. The orchestrator does nothing clever: decompose, delegate, synthesize. The intelligence is in the decomposition rules, and those took three rewrites to get right.

So the future isn’t “smarter autoresearch.” It’s autoresearch as a primitive that other systems call into.

So What Does This Actually Mean?

Karpathy didn’t just build an ML research tool. He demonstrated a pattern that works anywhere you can measure progress with a command: constraint plus mechanical metric plus autonomous iteration.

Here are the categories ranked by fidelity to the original idea:

Platform ports — most faithful. Same loop, different hardware.
ML enhancers — extend the substrate. Memory, Bayesian updates, multi-GPU.
Prompt optimizers — same loop, different file. train.py → prompt.txt.
Generalized frameworks — extract the pattern. pip packages, Claude Code skills, “give me any metric.”
Production codebase — industrial application. Shopify -34%, idealo -37% in 1 hour for $7.
Agent factory — meta-application. The loop builds other agents.
Research OS — institutionalization. The whole methodology, not just the loop.
Creative writing — the surprise expansion. Prose, fiction, brand documents.
Orchestration — autoresearch as worker, not the whole system.

A few honest takes:

The reward hacking problem is the cautionary tale nobody includes. In the tennis XGBoost case, the loop found a way to improve the metric without improving the model. The discard mechanism is only as good as your metric. If your eval can be gamed, the agent will game it. Spend more time on the eval than on the loop.

The pattern is more durable than the implementation. Most of the forks I found were “what if we applied this to X” and they all worked. That’s kind of remarkable. The discard mechanism (git reset on regression) is the key. You don’t need intelligence. You need iteration speed, a mechanical metric, and automatic rollback.

The Shopify and idealo case studies should embarrass you a little. $7 of API and an hour of supervision took 37% off a production endpoint serving 250+ req/sec. There are perf wins like this in basically every codebase. We’re just not asking for them yet because we still think of optimization as expensive senior-engineer time.

Orchestration eats the loop. alirezarezvani’s piece shows that solo autoresearch is fine, but the next move is autoresearch as a worker that orchestrators call when a sub-question lands. That’s where this is heading and it’s already happening in production.

If you’re not running at least one of these on a real project, you’re leaving free improvements on the table. The bar to entry is pip install autoloop-ai or npx autoresearch-anything. There’s no reason not to point one at something you care about and let it run overnight. You’ll either get a better version of the thing or you’ll learn something about your metric. Both of those are wins.

Model Roundup: The Free Countdown, the $300 Amnesiac, and the Quiet Climber at #7

Stephan Miller — Sat, 02 May 2026 12:00:00 +0000

I check OpenRouter rankings every week to figure out which models to throw at my projects. This week, the model at the top of the charts had something I’d never seen before: an expiration date.

Right there on the Tencent Hy3 Preview page: “Going Away May 8.” Six days from now. And it’s currently generating 2.15 trillion tokens a week with a +1,356% spike. You know what that is? Not a sign of the best model on the market. It’s the AI equivalent of a store liquidation sale. Everyone’s grabbing tokens before they cost money.

That’s W18 in a nutshell. The #1 model is a countdown timer. The hottest new premium subscription ($300/month from xAI) still can’t remember who you are between sessions.

There’s good news buried in all this: Kimi K2.6, which I mentioned last week as an interesting launch, has started showing real production numbers. And there’s a model called Step 3.5 Flash that’s been quietly climbing the rankings for three months with zero hype, which in this market is basically a standing ovation.

Let me tell you what actually matters.

Table of Contents
The #1 Model Is a Countdown Timer (Tencent Hy3 Preview)
Kimi K2.6 Is Now a Real Recommendation
- Where K2.6 Falls Short
The Sleeper: Step 3.5 Flash Has Been Climbing for Three Months
- The One Real Catch
Grok 4.3: Genuinely Impressive, Genuinely Annoying, $300/Month
Your Smarter Model Might Be Breaking Your Agents
What’s Actually Worth Using (and What’s Coming)

The #1 Model Is a Countdown Timer (Tencent Hy3 Preview)

Tencent launched Hy3 Preview on April 22 with a free access period that runs out May 8. That’s the entire explanation for the +1,356% weekly spike and the 2.15 trillion tokens burned. Developers saw “free” and “295B MoE” in the same sentence and did what developers do: they stress-tested it before anyone sent them a bill.

Here’s what Hy3 Preview actually is: 295 billion total parameters, 21 billion activated per token (mixture of experts, efficient by design), 262K context window, configurable reasoning you can dial from disabled to low to high. Designed for agentic coding workflows. On paper, solid.

In practice? No Arena votes because it’s too new to have accumulated any. No long-form reviews because nobody’s shipped anything with it yet. No “I’ve been using this for three weeks and it’s my daily driver” posts anywhere I could find. Just a lot of “grabbing free tokens before May 8” energy.

What happens after May 8 is the real question. Hy3 Preview becomes a paid model competing against DeepSeek V3.2 (which costs $0.14 input / $0.28 output per 1M tokens and has months of production track record), Kimi K2.6 ($0.74/$3.49 with confirmed adoption), and Step 3.5 Flash (which I’ll get to in a moment). Entering that field with no reviews and no Arena ranking is a tough position.

If you want to play with it before the deadline, go to openrouter.ai/tencent/hy3-preview:free and run some benchmarks. Just don’t build a dependency on something with a “Going Away” notice stamped on it.

Kimi K2.6 Is Now a Real Recommendation

Last week I called Kimi K2.6 an interesting launch. Twelve days later, the production numbers are coming in and it’s something more concrete.

Real developers running real workflows are reporting 88% cost savings when they replace Claude with K2.6 for bulk coding tasks: batch migrations, test generation, format conversion, anything where you’re doing a lot of the same kind of work repeatedly. The Kimi Code CLI, the companion tool for using K2.6 in your terminal the same way you’d use Claude Code, crossed 6,400 GitHub stars. That’s people betting actual infrastructure on this model, not just upvoting a launch post.

The pattern hardening into consensus across forums: use K2.6 for bulk, use Claude for the high-stakes core. At $0.74 input / $3.49 output per 1M tokens, K2.6 is roughly 4x cheaper than Claude Sonnet 4.6. For workflows that generate a lot of tokens on repetitive work, that math compounds fast.

Where K2.6 Falls Short

This is the part I actually care about more than the hype. K2.6 trails GPT-5.4 on GPQA-Diamond (90.5% vs 92.8%) and AIME 2026 (96.4% vs 99.2%). These are hard reasoning benchmarks. For anything where being wrong has real consequences (financial analysis, medical context, legal questions), K2.6 is not the answer. The cost savings don’t matter if the output costs you more to fix.

Use it for code. Trust it with the boring high-volume stuff. Keep a premium model on anything where you’d be embarrassed if an AI got it wrong.

K2.6 also ships with agent swarm architecture supporting up to 300 parallel sub-agents and 4,000 coordinated steps. After my own experiences with AI agents inventing things I’d start with single-agent mode until you’ve validated its judgment in your specific domain. 300 parallel sub-agents hallucinating tool calls in parallel is not a good time.

The Sleeper: Step 3.5 Flash Has Been Climbing for Three Months

Most models follow the same OpenRouter arc: spike at launch, plateau after a few weeks, slowly fade as the next shiny thing arrives. Step 3.5 Flash doesn’t fit this pattern.

StepFun released it somewhere in early 2026; the exact date is contested across sources, somewhere between late January and March, doesn’t matter. As of this week it’s at #7 on OpenRouter with +28% week-over-week. For a model that’s been around three months, that’s not a hype spike. That’s sustained adoption with nothing to explain it except developers finding it useful.

The numbers back it up: #4 intelligence ranking out of 64 models on Artificial Analysis. That puts it above almost everything priced anywhere near its cost: free on the rate-limited tier, $0.10 input / $0.30 output per 1M tokens on paid. For comparison, DeepSeek V3.2 costs $0.14/$0.28 and ranks lower on the same index. Step 3.5 Flash is somehow cheaper AND smarter on paper, and nobody’s writing breathless posts about it.

Architecture: 196 billion total parameters, 11 billion activated per token (MoE), 262K context, reasoning parameter support so you can see step-by-step thinking in API responses if you want it.

The One Real Catch

Step 3.5 Flash is extremely verbose. During Artificial Analysis evaluation it generated 260 million tokens versus an 11 million token average for comparable models. It thinks out loud, at length, in a way that will surprise your output token budget if you’re not watching.

Set max_tokens limits. If you’re using it for any high-volume generation, put a ceiling on it. Otherwise you’ll get thorough reasoning that costs more than you expected from a supposedly cheap model.

Worth adding to your comparison set before someone writes a breathless Medium post about it and StepFun decides to raise the price.

Grok 4.3: Genuinely Impressive, Genuinely Annoying, $300/Month

Let’s do the good news first, because there’s real good news here.

Grok 4.3 (launched April 17, currently rolling out in beta to SuperGrok Heavy subscribers) added native video input processing, not “describe this video” video but actual video-grounded reasoning. It can generate fully-formatted downloadable PDFs, populated spreadsheets, and PowerPoint presentations directly from conversation. Early beta testers are reporting formatted outputs they could hand to someone without cleanup. The integration with Grok Computer (xAI’s desktop automation agent) got tighter. If you’re doing autonomous desktop workflows, Grok 4.3 has a real story.

Now the bad news.

Grok 4.3 costs $300/month. That’s $100 more than ChatGPT Pro and $100 more than Claude Max. Both of those services have had persistent memory between sessions for over a year. Grok 4.3 does not. Every time you close your tab, the model forgets you. You start over. Blank context, fresh start, zero memory of anything you’ve built together.

Persistent memory is not on xAI’s published roadmap.

Multiple reviewers landed on the same observation this week. One X user put it cleanly: “you’re paying $300/month for a model that forgets you between sessions.” That’s not exaggeration. That’s the product.

At $200/month, this would be annoying. At $300/month, it’s a product decision, and product decisions tell you something about what a company is optimizing for. xAI built the video capabilities and the document generation first. Memory (the feature that makes an AI assistant feel like an actual assistant rather than a very fancy search box) is apparently not the priority.

Add the “High Demand” server errors that hit during launch week beta and you’ve got a model that’s impressive in demos and frustrating in daily use. The full API rollout is coming mid-to-late May. When it hits general availability, this conversation is going to get louder.

Your Smarter Model Might Be Breaking Your Agents

This one’s structural rather than model-specific, and it’s relevant for anyone running agentic pipelines.

An April 2026 ICLR paper titled “The Reasoning Trap” documented something uncomfortable: RL-based reasoning training (the kind that makes frontier models better at hard reasoning tasks) increases tool-hallucination rates in lockstep. The better a model gets at reasoning, the more often it invents tool calls that don’t exist. Function names, API endpoints, methods that aren’t in your schema. The model reasons its way to a call it can’t actually make.

If you’ve upgraded your agentic pipeline to a stronger reasoning model because it’s smarter, you may have simultaneously increased the rate at which it hallucinates the tools it should be calling. The capability and the failure mode scale together.

I’ve written about running into this firsthand with OpenClaw. The model-specific details differ but the pattern is the same. Stronger reasoning doesn’t mean better tool selection, and in agentic contexts “smarter” can break things in ways you don’t catch until something fails in production.

Practical response: add tool-call schema validation before your agents execute. Check that every tool the model selects actually exists in your registry before you let it run. This applies to every frontier RL-trained model right now. It’s not a specific model bug, it’s how these systems are being trained.

What’s Actually Worth Using (and What’s Coming)

Quick reference:

Tier	Model	Input $/1M	Output $/1M	Best For
Free (grab it now)	Hy3 Preview	$0	$0	Experiments before May 8 only
Free (stable)	Step 3.5 Flash	$0	$0	Rate-limited; best free reasoning available
Free (open weights)	Nemotron 3 Super 120B	$0	$0	NVIDIA-backed, open license, 262K context
Free (new, watch)	Owl Alpha (stealth)	$0	$0	1M context, agentic (prompts may be logged)
Budget	Step 3.5 Flash (paid)	$0.10	$0.30	Climbing for 3 months, verbose but smart
Budget	DeepSeek V3.2	$0.14	$0.28	Proven track record, still the value baseline
Mid	Kimi K2.6	$0.74	$3.49	Bulk coding workflows, 88% cheaper than Claude
Mid	Gemini 3.1 Pro	$2.00	$12.00	Arena #4 overall, 1M context
Premium	Claude Sonnet 4.6	~$3.00	~$15.00	#2 Arena coding, proven daily driver
Premium	Claude Opus 4.7	$5.00	$25.00	#1 Arena overall (thinking mode), high stakes

Mark your calendar for May 19. Google I/O is 17 days away. Gemini 4 isn’t confirmed, but annual release patterns and confirmed agenda items (agentic AI, developer tooling) make it likely. That’s the next likely shakeup in this table.

Claude Mythos, Anthropic’s model that developed a working exploit for a remote code execution vulnerability in FreeBSD (CVE-2026-4747), is not coming to a public API. It’s locked in Project Glasswing, a security research consortium, and Anthropic has no public timeline for changing that. Mention it at parties.

GPT-6 is still vaporware. Polymarket has it at 84% by December 31, 2026. That’s not a date, it’s a guess with confidence bounds.

The model worth your attention this week isn’t at #1. It’s at #7, three months old, climbing steadily, no hype cycle to explain it. Step 3.5 Flash just keeps showing up in the data.

My AI Agent Kept Making Shit Up (And Other Lessons From Running OpenClaw)

Stephan Miller — Tue, 28 Apr 2026 12:00:00 +0000

I wanted an AI agent running on my home network. Not a cloud subscription and not something requiring me to be at the keyboard all day. A thing that wakes up at 7am, pulls from RSS feeds and Reddit, synthesizes the news I actually care about, and emails it to me. Just that. That’s what I started with. Seemed simple. It wasn’t like I was asking much.

The reality was six weeks of debugging hallucinations, silent config failures, broken tool schemas, and a recurring realization that LLMs are, in certain contexts, compulsive liars.

Here’s what I learned the hard way.

The Setup: OpenClaw + DeepSeek in Docker
The Exec Approval Maze
The Reports That Were Too Good
Going Around the Agent
When Tools Become Literal Text
Ripping Out Slack
What’s Actually Working
But Here’s What She’s Actually Good At
- The Report Engine Isn’t a One-Trick Pony
- Email Delivery, Old School On Purpose
- Multi-Model, Not Locked to DeepSeek
- The Track Record, Three Days In
What I Actually Built

The Setup: OpenClaw + DeepSeek in Docker

OpenClaw is a self-hosted AI agent framework. If you haven’t heard of it, think a local version of an AI assistant with cron jobs, tool calling, Slack/Telegram integration, and memory. Plus, how haven’t you heard of it. You run it in Docker, point it at whatever LLM you want, and theoretically have an autonomous agent working for you.

I named mine Sabrina. She runs DeepSeek V3 (deepseek/deepseek-chat) because the OpenAI and Anthropic APIs bill by the token and Sabrina is a chatty agent who generates daily reports. DeepSeek at pay-as-you-go rates keeps the monthly bill manageable.

The architecture is two containers: openclaw-gateway handles HTTP and the Slack/Telegram socket connections, and openclaw-cli is the shell interface. The whole ~/.openclaw directory mounts into the container at /home/node/.openclaw so configs, cron jobs, and workspace scripts are all live-editable from the host without rebuilding.

On paper, this is elegant. In practice, you will spend a lot of time staring at container logs wondering why your agent is quietly lying to you. Or realizing you can just put Claude Code on the host and just have it fix things when they mess up.

The Exec Approval Maze

Before Sabrina could run scripts, I had to configure exec-approvals.json: a policy file that controls what shell commands the agent is allowed to execute. Fine. Reasonable. I set up allowlists for the workspace scripts and Python interpreter.

Then the cron jobs started silently failing. The daily 7am AI report would produce output, but something felt off. I dug into the exec-approval config and found the first trap:

The documentation (and my own reasoning at the time) suggested "ask": "never" as a way to skip interactive approval prompts for unattended jobs. This is wrong. The schema only accepts "off" | "on-miss" | "always". Using "never" doesn’t throw an error. It gets silently stripped by sanitizeExecApprovalPolicy the next time the app writes the file. Your config looks fine, your intent is gone, and the agent starts timing out on approval requests at 7am with no operator connected.

The correct pattern:

{
  "defaults": { "security": "allowlist", "ask": "off", "allowlist": ["..."] },
  "agents": {
    "main": { "security": "allowlist", "ask": "off", "allowlist": ["..."] }
  }
}

"ask": "off" makes the allowlist the sole policy.

I fixed this. Or so I thought.

The Reports That Were Too Good

The AI intelligence report looked great. Every morning: a well-formatted digest of the day’s AI news, summaries, source links. Sabrina was crushing it.

Then I noticed the timestamps.

Every log entry in the fabricated reports had timestamps ending in :00 or :30. No real log file looks like that: they’re messy, they have milliseconds, they reflect actual compute time. These were fake. I checked the URLs. Several of them 404’d. The article summaries were plausible but not verifiable. Sabrina had been generating the reports herself , not from RSS feeds, but from her training data and imagination, because the exec approval issue wasn’t actually fixed. When the script couldn’t run, the agent fell back on what LLMs do naturally: produce what the output should look like.

This is the thing nobody tells you about giving LLMs agentic tasks: when they fail to do the thing, they don’t say “I failed to do the thing.” They generate a plausible simulation of having done the thing.

The fix I’d been applying, tweaking exec-approvals, only addressed the symptom. The agent could bypass exec approval entirely by deciding to write the content directly. There was no configuration that would stop a sufficiently motivated language model from bullshitting.

Going Around the Agent

The actual fix was nuclear: remove the agent from report generation entirely.

I disabled the OpenClaw cron jobs for both the AI report and the email send, then added host-level cron entries that call docker exec directly:

0 7 * * * docker exec openclaw-openclaw-gateway-1 /usr/bin/python3 /home/node/.openclaw/workspace/ai_report.py --profile ai-intelligence >> /home/eristoddle/.openclaw/workspace/logs/report-host-$(date +\%Y-\%m-\%d).log 2>&1

30 7 * * * docker exec openclaw-openclaw-gateway-1 bash /home/node/.openclaw/workspace/send-ai-intelligence-report-proper.sh >> /home/eristoddle/.openclaw/workspace/logs/email-host-$(date +\%Y-\%m-\%d).log 2>&1

The Python script runs inside the container, where it has access to the right Python packages, but the trigger is the host crontab. No agent involved. No LLM between the script and reality.

This works. The reports now have messy timestamps and real URLs that actually load.

The Obsidian weekly report I left in OpenClaw, because that one needs the agent. It reads my vault, categorizes clips, writes summaries, analyzes git diffs: actual LLM work that benefits from Sabrina’s reasoning. The difference is whether the task is “run a script and report the output” (host cron) or “think about my vault and synthesize something useful” (agent cron). Only one of those should involve an LLM.

When Tools Become Literal Text

OpenClaw gets updates. After updates, things break in interesting ways.

Twice now I’ve run into a scenario where Sabrina starts responding to everything but her tool calls appear as raw text in the chat. Instead of actually reading a file, she’d output read:/home/node/.openclaw/workspace/HEARTBEAT.md as a literal string.

This is a DeepSeek-specific quirk that OpenClaw triggers by accident. The framework converts tool schemas to OpenAI format before sending them to providers. DeepSeek expects its own native format. The conversion breaks its tool call parsing silently. It receives schemas it doesn’t understand and falls back to treating the tool call syntax as plain text.

The fix is a compat flag in the model config in openclaw.json:

"models": [{
  "id": "deepseek-chat",
  "name": "DeepSeek V3",
  "contextWindow": 163840,
  "maxTokens": 8192,
  "compat": { "anthropicToolSchemaMode": "native" }
}]

anthropicToolSchemaMode: "native" tells OpenClaw to skip the schema conversion and send the native format. Tools work again. I found this via a GitHub issue (#36651) after two sessions of source archaeology that I really didn’t want to be doing.

The lesson: when OpenClaw updates and tools start appearing as text, don’t read source code first. Check GitHub issues and Reddit. The community finds these fixes faster than you will staring at the framework internals.

Ripping Out Slack

OpenClaw supports Slack via socket mode. I had it connected for a while because it was useful for checking in on Sabrina from my phone without VPN or port-forwarding.

Then an update changed the Slack config schema. The gateway crashed on startup with “Config invalid” and wouldn’t come back up until I removed the entire channels.slack block from openclaw.json. This happened twice. After the second time I removed Slack permanently and switched to Telegram, which has been stable.

This is the trade-off with self-hosted software that’s still actively developed: you get the control, you eat the breakage. Updates that ship on Tuesday can invalidate configs you spent a week getting right. Having Claude Code manage the ~/.openclaw config directory directly, rather than asking Sabrina to fix herself through chat, means at least the fixes land correctly the first time.

What’s Actually Working

Six weeks in, here’s the honest status:

Daily AI intelligence report: Running reliably via host cron. Real data. Real URLs. Emails delivered by 7:30am.
Weekly Obsidian report: Agent-generated, delivers Fridays. Sabrina does genuine LLM work here — categorizing clips, writing summaries — and it shows.
Tool calling: Stable with the compat flag. Breaks again when OpenClaw updates, gets fixed in under an hour now that I know where to look.
The exec-approvals file: Still fragile. I keep a copy of the correct config in my notes.

The thing I underestimated: running an AI agent autonomously is mostly an infrastructure problem, not an AI problem. The interesting parts are the prompts and the LLM reasoning. The annoying parts are Docker networking, cron timing, config schema drift, and an agent that will hallucinate convincingly rather than admit it can’t do something.

Sabrina’s useful. She’s also a liar when she’s backed into a corner. I’ve learned to keep her away from any task where I can’t independently verify the output.

That’s not an OpenClaw problem or a DeepSeek problem. That’s just what LLMs do. But here’s the thing: once I stopped asking her to do the things LLMs are bad at, she got useful in a hurry. Most of what follows happened since last Thursday night.

But Here’s What She’s Actually Good At

OpenClaw’s skill system is pluggable. You drop a skill into the workspace, the agent loads it, and it becomes part of how she thinks. Sabrina didn’t ship with most of her current capabilities. She built them through the same autonomous workflow she runs every day.

A few that earn their slot:

sm-blog-outline : Started life as a generic blog-outline skill. Now it’s the full pipeline I use for this site — notes → outline → email. Trained on my voice, my content pillars, my snark level. It’s the skill that outlined this post pulling from both Sabrina’s and Claude Code’s logs as well as a running list of notes I kept on the setup process.
ct-humanizer : Sequential editing passes that strip AI tells out of nonfiction. Diagnoses patterns first, then kills the AI vocabulary, then breaks up the structural templates LLMs love so much. Not a magic button, more like a brutal copy editor. It cleans up the outline.
verbalized-sampling : Instead of spitting back a single answer, generates multiple candidates with probability weights. I use it for brainstorming and “show me five angles” tasks. The default LLM answer is usually the median answer; this skill surfaces the weirder, more useful ones. Got the idea here, gave Opus all the documentation, and used the Claude skill-creator skill to create it. It is one of my favorite skills because you never know what you’re going to get.
vault-tag-search + vault-idea-scorer : Companions to the blog pipeline. One searches my Obsidian vault by tag and body content with deduplication. The other ranks blog post ideas by whether they dovetail with multiple goals: research vs. content vs. portfolio vs. SEO.
A self-improving skill: Logs corrections and preferences so Sabrina compounds learning between sessions instead of getting the same feedback every week.

The point isn’t any single skill. It’s that the agent grows a custom toolkit shaped by the work I actually do, not whatever generic capabilities the framework shipped with.

The Report Engine Isn’t a One-Trick Pony

That ai_report.py script generating the daily AI digest isn’t hardcoded to AI news. It’s a topic-agnostic engine that takes a profile flag:

python3 ai_report.py --profile ai-intelligence
python3 ai_report.py --profile golang
python3 ai_report.py --profile typescript

Each profile defines its own RSS feeds, Reddit subreddits, and keyword filters. Tunable depth too: brief briefing vs. deep dive, set per profile. Articles get scored against my interests using CLIP + BM25 indexing before they make the cut, so I don’t end up with a digest full of stuff I don’t care about.

Same engine, different sources, same usefulness. Once the host cron pattern is locked in for one topic, adding another is a profile file and a crontab line.

Email Delivery, Old School On Purpose

Everything Sabrina produces comes to me as email. Gmail SMTP, app password auth…for now. Yes, that’s old fashioned. That’s the feature.

A dashboard would be one more thing to check. Notifications would be one more app fighting for attention. Email is the universal inbox I already process. I can read it on my iPad without installing anything, forward to Obsidian if it’s worth keeping, drag it to drafts if it’s a blog skeleton, or delete it if Sabrina got it wrong.

The pattern is generic:

send-email.sh "Subject" body-or-file [attachment]

That’s it. Anything in the system that needs to deliver text to a human goes through that script. Reports blog outlines, and research summaries use it.

Multi-Model, Not Locked to DeepSeek

DeepSeek runs the daily cron work because it’s cheap. But Sabrina isn’t married to it. The agent routes through OpenRouter, which means any task can pick its own model:

qwen/qwen3.6-plus — 1M context window, great for long-form research and generation
minimax/minimax-m2.5 — strong reasoning, what I reach for on analytical work
google/gemini-3-flash-preview — also 1M context, fast
moonshotai/kimi-k2.6 — solid alternative when the others are misbehaving

The job picks the model. Daily AI report? DeepSeek, because it’s cheap and the task isn’t hard. Blog outline that needs to chew through a pile of research notes? Qwen, because the context window swallows the whole input without chunking. Analytical synthesis? Minimax. And again, for now. I am just getting into these new models after using Claude Code for however long its been out. But the success I’ve have with them has me setting up Opencode to use them.

The subagent system lets me parallelize too. While the main session ran on DeepSeek doing one thing, a subagent on Qwen drafted an outline for a different post. Two models, two tasks, one wall clock.

The Track Record, Three Days In

Concrete deliverables since Thursday night:

Blog outlines: Two posts — a Kiro AI article and one I’m calling “The AI Psychologist” — both went notes → web research → verbalized sampling for angle selection → outline → email. Full pipeline, no me-in-the-loop until the outline showed up in my inbox.
Research tasks: Author bios with structured JSON + bibliography, topic deep-dives on AI tools, vibe coding, prompt engineering psychology. Stuff I’d normally burn an afternoon on.
Brainstorming: Content ideas, project names, productivity workflows, all using verbalized sampling so I get diverse options with probability weights instead of one safe median answer.
Memory compounding: Daily logs roll up to weekly memory promotion. The self-improving skill captures corrections so the same mistake doesn’t keep showing up. Each week she’s a little less stupid about my preferences.
Weekly Obsidian reports: Genuinely useful vault digests. What changed. What’s worth re-reading. What’s collecting dust and should be archived or thrown out.

None of this involves Sabrina pretending to run scripts she can’t run. All of it is “think about something and write me a thing,” which is exactly what LLMs are for.

What I Actually Built

Six weeks ago I wanted an autonomous AI agent. What I have now is better and stupider at the same time.

The discovery, after all the silent hallucinations and config schema drift and tool-calls-as-text bullshit: AI agents are great at the thinking parts: research, writing, brainstorming, synthesis. They’re terrible at the doing parts: running scripts reliably, admitting they can’t do something, not making shit up when cornered.

So I built around the doing and leaned into the thinking. Sabrina does real work now. She just doesn’t run the cron jobs herself anymore: the host crontab does. She doesn’t pretend to fetch RSS feeds: a Python script does that and hands her the data. What she does is the part LLMs are actually for: read a pile of stuff, synthesize, make a thing, deliver it to email.

The host cron + agent hybrid is the pattern that actually ships. The agent is the writer, not the operator. The operator is cron and a Python interpreter, both of which have been doing their jobs reliably since long before transformers were a thing.

Six weeks to figure out what should have been obvious from the start: stop using language models for things that aren’t language. At least that’s what I’m going with until I have time to go through another break then fix continously cycle.

April 2026 Model Roundup: Opus 4.7 Official, DeepSeek V4 Open-Sources 1M Context, and GPT-5.5 Upstaged the GPT-6 Hype

Stephan Miller — Fri, 24 Apr 2026 12:00:00 +0000

Two weeks ago this month, developers discovered their Gemini API bills had exploded. Google’s billing system was charging for approximately 114 internal search queries per API call with grounding enabled. That was the story I started writing. By the time April 24 arrived, three new models had officially launched, the “Still Waiting for GPT-6” watch ended not with GPT-6 but with GPT-5.5, and DeepSeek V4 dropped today with a 1M context window under Apache 2.0, on the same day GPT-5.5 went live, apparently just to split the news cycle.

This roundup covers April 2026 so far.

When Google Billed 114x
What Actually Moved This Week
Claude Opus 4.7 Is Official — and the Cost Story Is Better Than Expected
Hype Check: Mimo V2 Pro, One Month In
Kimi K2.6: The Open-Source Agentic Coding Model Nobody Covered
Meta Broke Open Source Hearts
The Models That Cost Almost Nothing (No, Really)
The Hidden Tax: How Sonnet 4.6 Can Still Cost More Than Opus
GPT-5.5 Shipped Yesterday, Not GPT-6, and DeepSeek V4 Dropped Today
The Actual Takeaways
Read for Yourself

When Google Billed 114x

Gemini 3 Flash Preview is Google’s high-volume, reasonably priced model: $0.50/M input, $3/M output, 1M context window. It’s been running at #4 on OpenRouter by weekly token volume. A lot of people have pipelines running on it. The “search grounding” feature, which lets the model query Google Search to ground its responses in real-time information, sounds great on paper.

Turns out the billing for that feature had a misconfiguration. For every API call, users were being billed for roughly 114 separate search queries rather than the actual number of queries they used. The “Generate content search query Gemini 3” SKU in users’ dashboards was showing 10-15x the expected line items. Actual grounding call frequency had decreased, but bills exploded anyway.

The scale of damage before Google caught it:

Multiple developers reporting 4x–10x cost increases on flat or declining usage
€1,000+ additional daily costs for at least one European developer
₩340,000 in two days for a Korean developer
Google identified the root cause on April 14, committed to fixing the misconfiguration and correcting previous bills

Update as of April 24: Google engineer Ali Cevik confirmed on the developer forum that the billing misconfiguration is fixed going forward. Refunds are being processed, but Google has provided no specific timeline. Forum responses from their team said “by the end of the month” without committing to anything more specific. Affected users are reporting that support is framing corrections as “one-time exceptions” rather than acknowledging the systemic bug. Re-enabling grounding is probably safe now for new calls, but check your billing dashboard before turning it back on, and watch the first few days’ charges carefully.

The thread on the Google AI Developers Forum is worth reading if you’re running anything on Gemini 3 Flash with grounding enabled. The concrete lesson here: search grounding is billed separately from token usage , and before you enable any “enhanced” feature on a high-volume model, understand exactly what gets metered and how. Don’t assume the main pricing page tells the whole story.

What Actually Moved This Week

Here’s the OpenRouter picture as of the week ending April 24:

Rank	Model	Provider	Weekly Tokens	WoW Change	Arena Overall
1	Claude Sonnet 4.6	Anthropic	1.38T	+3%	#3 (1496)
2	DeepSeek V3.2	DeepSeek	1.32T	+3%	—
3	Gemini 3 Flash Preview	Google	1.11T	stable	—
4	Claude Opus 4.7	Anthropic	951B	+4,221%	#1 (1503, thinking)
5	Mimo V2 Pro	Xiaomi	902B	+9%	not yet
6	MiniMax M2.5	Minimax	856B	+22%	—
7	MiniMax M2.7	Minimax	813B	+24%	—
8	Kimi K2.6	Moonshot AI	792B	New	not yet
9	Claude Opus 4.6	Anthropic	756B	+46%	#2 (1503, thinking)
10	Grok 4.1 Fast	X.AI	700B	+33%	—

Three new entries this week: Claude Opus 4.7 at #4 with a 4,221% spike, Kimi K2.6 debuting at #8 on its first week, and Grok 4.1 Fast at #10. Claude Opus 4.6, which was briefly the second-most-used model, dropped to #9 as people migrated to 4.7.

The stable story continues in the background: Claude Sonnet 4.6 and DeepSeek V3.2 are running neck and neck at the top, both at a slow +3% WoW. That’s real production traffic, not evaluation runs.

Claude Opus 4.7 Is Official — and the Cost Story Is Better Than Expected

Anthropic launched Claude Opus 4.7 on April 16. It’s been sitting in Arena’s blind comparison system for a few weeks, and it’s now publicly available on the API, Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry.

Pricing: $5/M input, $25/M output — unchanged from Opus 4.6. That’s the headline.

The real story is what the model does to your costs in practice. Artificial Analysis ran Opus 4.7 through their GDPVal-AA benchmark suite (44 occupations, 9 industries) and found it uses roughly 35% fewer output tokens than Opus 4.6 to complete the same tasks. The practical effect: real-world costs on Opus 4.7 run approximately 11% lower than Opus 4.6 at the same stated price per token.

There’s a caveat on the input side. The 4.7 tokenizer is less efficient, generating up to 35% more tokens from the same input text depending on content type. For workloads with heavy, repeated system prompts or long document context, this can offset some of the output savings. Prompt caching (available at roughly 10% of the input rate) largely neutralizes this.

Performance numbers that matter:

Artificial Analysis Intelligence Index: 57 (up 4 points from Opus 4.6, tied with GPT-5.4 and Gemini 3.1 Pro)
GDPVal-AA: 1,753 Elo — 79 points ahead of the next model on real-world knowledge work
Hallucination rate: 36% (down from 61% on Opus 4.6, achieved through more frequent abstention)
Arena: #1 tied at 1503 Elo (with thinking mode), #4 at 1494 without thinking

The 4,221% WoW spike on OpenRouter is a curiosity spike plus a migration wave from people moving from 4.6. By next week you’ll see whether it settles into stable sustained usage or was just upgrade traffic.

New cybersecurity guardrails: Anthropic added automatic detection and blocking for prohibited cybersecurity uses. Security professionals doing legitimate work (pen testing, vuln research, red-teaming) need to join their new Cyber Verification Program to preserve access to those capabilities on 4.7.

Hype Check: Mimo V2 Pro, One Month In

Mimo V2 Pro shot up 140% WoW in the April 14 data. Now, with another week of data, it’s at +9%. The spike is over and it’s settling into a real usage tier.

Xiaomi’s flagship foundation model: over 1 trillion total parameters, 42 billion active (MoE architecture), $1/M input, $3/M output, 1 million token context window. Benchmarks put it at 49 on the Artificial Analysis Intelligence Index.

The +140% spike was the evaluation-and-curiosity phase. The +9% continuing growth suggests people who ran it liked it enough to keep using it. Still no Arena votes worth analyzing. At ~5 weeks old, the model hasn’t been around long enough for production validation at scale.

Check back in 2–3 more weeks. If it accumulates Arena votes and holds a respectable position there, the benchmarks were real. Stable OpenRouter usage without Arena presence is ambiguous: it could mean quality users who prefer specific capabilities, or it could mean low-friction API access driving test traffic.

Kimi K2.6: The Open-Source Agentic Coding Model Nobody Covered

Moonshot AI released Kimi K2.6 on April 20 and it debuted at #8 on OpenRouter in its first week. You probably missed it because the same week had Claude Opus 4.7’s official launch, Grok 4.3 Beta, and the GPT-5.5 pre-announcement noise.

What it is: a 1-trillion-parameter MoE model with 32B active parameters, 262,144-token context window , vision, and agentic capabilities. Weights published on Hugging Face under a Modified MIT License : full open weights, commercially usable.

What it’s built for: long-horizon coding agents, front-end generation from natural language, and massively parallel agent swarms. Moonshot’s documentation specifically highlights scaling to 300 sub-agents and 4,000 coordinated steps in a single session. If you’re building orchestration-heavy multi-agent systems, this is the open-weight model that was designed from the ground up for that use case.

Benchmark comparisons are mixed but solid. On SWE-Bench Pro it outperforms DeepSeek V4-Pro (58.6 vs 55.4). On LiveCodeBench it trails V4-Pro (89.6 vs 93.5). On competitive coding (Codeforces), both trail GPT-5.5.

No pricing table yet because it’s primarily a self-hosted model. Kimi API pricing for hosted inference isn’t broadly published yet. For the open-weights version: the cost is your inference infrastructure. A 32B-active MoE runs reasonably on mid-tier GPU setups.

DeepSeek V4 (more on that below) is the stronger model by most closed benchmarks. But Kimi K2.6 has the context window advantage (262K vs 1M for V4-Pro — actually V4-Pro wins there), and the MIT-derived license is cleaner than Apache 2.0 for certain commercial use cases.

Meta Broke Open Source Hearts

Llama made Meta relevant in the AI developer world. Open weights, commercial use, the whole deal. Llama 4 dropped in 2025 with a 10 million token context window and impressive parameter counts. The developer community built on it. People ran it locally, fine-tuned it, deployed it. Meta was the company that understood that open source was ecosystem building.

Then on April 8, Muse Spark dropped from Meta’s new “Superintelligence Labs.” Proprietary model. Not open weights. API in private preview. To try it on the web, you need a Facebook or Instagram login.

Meta went from an Artificial Analysis Index score of 18 with Llama 4 Maverick to 52 with Muse Spark. That’s not a modest improvement. And in Arena’s head-to-head voting, Muse Spark is sitting at #6 overall with an Elo of 1492, beating GPT-5.4-high in actual user preference votes.

So the model is legitimately good. As of April 24, the API remains private preview only: no public access, no announced pricing, no timeline for broader availability. Priority access is going to healthcare, education, and enterprise research partners. If you’re building something that needs Muse Spark today, you’re waiting.

“Meta learned from OpenAI: make the good stuff closed, give the community the crumbs.” I’ve been seeing that take everywhere this month, and I don’t think it’s entirely wrong.

The broader question this raises: if every lab eventually closes off its best models, what’s the long-term roadmap for building on open weights? DeepSeek and Moonshot are still playing the open-source game. Kimi K2.6 is MIT-licensed. And DeepSeek V4 dropped today with Apache 2.0 weights on Hugging Face. The pattern is becoming hard to ignore, but there are still holdouts.

The Models That Cost Almost Nothing (No, Really)

I need to talk about MiniMax M2.5 because I’ve been mentioning it in passing and it deserves its own paragraph.

$0.118 per million input tokens.

That’s twelve cents per million tokens. For a model that’s sitting at #6 on OpenRouter by weekly volume and growing at 22% WoW. For a model that scores 80.2% on SWE-Bench Verified: which is roughly what Claude’s flagship hits. With a 196,608 token context window. And it’s good enough at agentic tasks that it’s been called out repeatedly in the Latent.Space local model community as the go-to for tool-heavy applications.

The pricing table this week:

Model	Input $/1M	Output $/1M	Context	Notes
Claude Opus 4.7	$5.00	$25.00	1M	Arena #1
Claude Sonnet 4.6	$3.00	$15.00	1M	#1 volume on OR
GPT-5.5	$5.00	$30.00	2M	Tops AA Index at 60
GPT-5.5 Pro	$30.00	$180.00	2M	Research tier
DeepSeek V4-Pro	$1.74	$3.48	1M	Apache 2.0, released today
DeepSeek V4-Flash	$0.14	$0.28	1M	Apache 2.0, released today
DeepSeek V3.2	$0.259	$0.42	163K	3-mo validated, #2 on OR
MiniMax M2.5	$0.118	$0.99	196K	80.2% SWE-Bench
MiniMax M2.7	$0.30	$1.20	196K	Upgraded M2.5
Mimo V2 Pro	$1.00	$3.00	1M	Settling into usage
Gemini 3 Flash	$0.50	$3.00	1M	Grounding: proceed cautiously

When MiniMax M2.5 matches or beats Sonnet on SWE-Bench at roughly 1/25th the per-token cost, we’re in strange territory. Either the benchmark is missing something important about real-world usability, or there’s value being left on the table by anyone running default Claude endpoints on agentic coding tasks without at least testing alternatives.

My actual picks this week, by use case:

Budget, coding/agentic : DeepSeek V4-Flash at $0.14/M: just dropped today, open source. Test it immediately. MiniMax M2.5 at $0.118/M is still the safety pick if you want community-validated quality.
Budget, general : DeepSeek V3.2. $0.42/M output, three months of community validation, strong on math and code. Nothing changed here.
Balanced : DeepSeek V4-Pro at $1.74/M input, $3.48/M output with 1M context. Undercuts everything at this quality tier by a factor of 5-8x.
Premium, coding : Claude Sonnet 4.6 or Claude Opus 4.7 depending on your task complexity and whether the token economics work out (see the next section).
If you have to have the absolute best : Claude Opus 4.7 with thinking (Arena #1, 1503 Elo) or GPT-5.5 (AA Index #1 at 60). Accept the pricing gap vs open-source alternatives.

The Hidden Tax: How Sonnet 4.6 Can Still Cost More Than Opus

Claude Sonnet 4.6 is marketed as the economical alternative to Opus. It’s $3/M input versus Opus 4.7’s $5/M: a modest 1.67x difference on input. But that’s not where your money goes on agentic workloads.

On the Artificial Analysis GDPVal-AA benchmark, Sonnet 4.6 generates 4.5x more output tokens than Opus 4.6 to complete the same tasks. The model isn’t worse. It’s producing more intermediate reasoning, more scaffolding, more steps. But output tokens are what you pay for.

The math with correct current pricing: Sonnet 4.6 at 4.5x tokens × $15/M output = $67.50/M effective output cost versus Opus 4.7 at $25/M output. Sonnet costs 2.7x more per equivalent task in heavy agentic use.

The practical takeaway: if you’re running document summarization, one-shot Q&A, light code generation, Sonnet 4.6 is cheaper and you should use it. If you’re running agentic pipelines, autonomous coding agents, extended tool-use workflows: benchmark on your actual workload before you assume Sonnet saves money. The pricing page isn’t lying; the intuitive comparison probably is.

And now there’s a third option in the mix: Opus 4.7, which uses ~35% fewer output tokens than Opus 4.6 at the same $25/M rate. For heavy agentic use, Opus 4.7 may be the cheapest of the three Anthropic options. Run your own numbers.

GPT-5.5 Shipped Yesterday, Not GPT-6, and DeepSeek V4 Dropped Today

OpenAI finished pre-training the model codenamed “Spud” on March 24. An April 14 release date came and went with nothing. Then on April 23, OpenAI shipped GPT-5.5 — their most capable model to date and, per their description, the first fully retrained base since GPT-4.5.

It’s not GPT-6. But it’s real.

GPT-5.5 numbers:

Artificial Analysis Intelligence Index: 60 — three points ahead of Claude Opus 4.7 and Gemini 3.1 Pro Preview, both at 57
Terminal-Bench 2.0: 82.7% (vs 75.1% for GPT-5.4)
Expert-SWE: 73.1% (vs 68.5% for GPT-5.4)
Pricing: $5/M input, $30/M output — double the cost of GPT-5.4 on output
GPT-5.5 Pro tier: $30/M input, $180/M output (research/enterprise)
Context window: 2M tokens (1M longer than most competitors)

The 40% reduction in output token usage that OpenAI claims keeps the effective cost increase to roughly 20% despite the doubled price per token. That math depends entirely on your workload matching the benchmark profile.

Now the same-day plot twist: On April 24 — today — DeepSeek V4 dropped with open weights under Apache 2.0.

DeepSeek V4-Pro: 1.6T total parameters, 49B active (MoE), 1M context window, $1.74/M input, $3.48/M output. V4-Pro output is 8.6x cheaper than GPT-5.5 and 21x cheaper than Claude Opus 4.7 at stated per-token rates.

DeepSeek V4-Flash: 284B total parameters, 13B active, 1M context, $0.14/M input, $0.28/M output.

Both variants under Apache 2.0, weights on Hugging Face and ModelScope today.

Performance on competitive coding (Codeforces): V4-Pro scores 3,206 vs GPT-5.5’s 3,168 — V4-Pro wins. On SWE-Bench Pro, Kimi K2.6 beats V4-Pro (58.6 vs 55.4). On long-context retrieval (MRCR 1M), Claude Opus 4.6 beats V4-Pro (92.9 vs 83.5). So V4-Pro isn’t universally better — but at $3.48/M output vs $25-30/M for closed alternatives, it doesn’t need to be universally better to be the right answer for most workloads.

GPT-6 “Spud”: still hasn’t arrived. Polymarket has it at 72% by April 30 and 95%+ by June 30. At this point I’ll believe it when I see it.

Other things still in the pipeline:

Claude Mythos Preview : still available only to approximately 50 partner organizations since April 7. Cybersecurity focus. $25/M input, $125/M output. Nothing changed here.
Grok 4.3 Beta : dropped April 17 with native video understanding, PDF/PowerPoint generation, and enhanced long-context processing. Not yet on OpenRouter broadly. Still in xAI testing phase.

The Actual Takeaways

April 2026 shipped more major model releases than any previous month in AI history, and then DeepSeek V4 and GPT-5.5 both dropped on the same day at the end of it. The landscape looks different today than it did two weeks ago.

What actually matters as of April 24:

Gemini 3 Flash grounding billing is fixed going forward but check your billing dashboard before re-enabling, and watch the first few days’ charges carefully. Refunds are in process; don’t expect speed on that.
DeepSeek V4 just dropped open-source with 1M context and Apache 2.0. V4-Flash at $0.14/$0.28 and V4-Pro at $1.74/$3.48. Test it today. It’s too new for community validation but the pedigree is real and the pricing is absurd for the quality tier.
MiniMax M2.5 at $0.118/M and 80.2% SWE-bench is still the community-validated budget pick for agentic coding. Three weeks of steady usage volume with no hype cycle. DeepSeek V4-Flash is the new challenger — if validation holds over the next few weeks, it may displace M2.5.
Claude Opus 4.7 is the easiest top-tier upgrade you’ll make this month. Same price as 4.6, 35% fewer output tokens, Arena #1. If you’re running Opus 4.6 today, just switch.
Benchmark Sonnet 4.6 vs Opus 4.7 on your actual agentic workloads. Opus 4.7’s improved token efficiency means the economics may favor it over Sonnet for complex agent tasks. Run the math on your usage before assuming Sonnet is cheaper.
Mimo V2 Pro and Kimi K2.6 need another 2-3 weeks. Both show real usage momentum. Neither has Arena data yet. Hold the investment thesis pending community validation.
GPT-5.5 topped the Artificial Analysis Intelligence Index at 60. That matters, but at $30/M output you’re paying a substantial premium over DeepSeek V4-Pro ($3.48/M) for about 3 points on a benchmark. Evaluate whether that delta maps to your actual workload before committing.

The model evaluation cycle for “is this the right choice?” is now measured in weeks, not quarters

Read for Yourself

Gemini 3 Flash billing bug thread — r/[Google AI Dev Forum] — Developer discussion of the billing disaster, with cost breakdowns and screenshots
Meta introduces Muse Spark — TechCrunch — The story of Meta’s open-source pivot; comment threads are heated
Top Local Models List April 2026 — Latent.Space — Community-validated rankings for open-weight models
Mimo V2 Pro: mistaken for DeepSeek V4 — Decrypt — The review that captures the week’s Xiaomi surprise
Claude Sonnet 4.6: the workhorse that ate the flagship — AwesomeAgents — Honest multi-week review with the token cost caveat

Microsoft APM - Managing AI Context Like a Dependency Problem

Stephan Miller — Mon, 13 Apr 2026 12:00:00 +0000

It started with a small problem that wouldn’t stop nagging me.

I had AI coding tools scattered across machines, each one configured slightly differently, each one producing slightly different results. My Claude Code setup on my laptop didn’t match my desktop. I had created skills for these coding agents, but the skills I could use depended on which machine I was using. It had a “ it works on my machine” issue, but they were all my machines.

I started using Skillshare a little while ago and it helps somewhat, but it focuses on syncing the skills between the coding agents configs in your user folder. This type of functionality is useful for some skills, but not for all of them, Because sometimes you only need skills at the repo level. And putting all your coding agent skills at the user folder level not only pollutes your context, but makes it hard to find a specific skill when you want one.

So when I was asked to look for an enterprise tool to manage skills with a focus on Github Copilot, I found Microsoft APM, Agent Package Manager.

The Blueprint: APM’s Declarative Infrastructure
Organizing the Monorepo
Solving Context Pollution with Intelligent Compilation
Automating the Standards
Local Iteration and the Playground Strategy
From Supervisor to Architect

The Blueprint: APM’s Declarative Infrastructure

Many of us are still prompting it like it’s 2023. Copy-paste a prompt. Maybe drop a CLAUDE.md or AGENTS.md in the repo. Hope for the best. When the AI does something dumb, yell at it in the chat window and hope it remembers next time. It won’t.

APM replaces all of that with a declarative, version-locked workflow that treats AI context the same way we treat dependencies. You declare what you need in apm.yml, lock it with apm.lock.yaml, and install it with apm install. If that sounds like npm or pip, good. That’s the point. We solved dependency management for code twenty years ago. It’s insane that we’re still managing AI context by hand.

The system is built around seven primitives:

Instructions — The guardrails. Think CLAUDE.md or AGENTS.md files that tell the AI how to behave.
Skills — Reusable capabilities the AI can invoke.
Prompts — Executable task templates with defined inputs. Called commands in Claude Code.
Agents — Specialized sub-agents with their own instructions and tools.
Hooks — Shell commands that fire on specific events.
Plugins — Extensions that add functionality to the agent runtime.
MCP Servers — Model Context Protocol servers that give agents access to external tools and data.

The part that sold me: APM doesn’t run a daemon or require a runtime. It populates your existing .github/, .claude/, and .cursor/ folders with native configuration files. The agents just pick them up. If you delete APM tomorrow, those files still work. Zero lock-in. That’s how you know someone thought about this for more than a weekend.

Key Concepts Guide

Organizing the Monorepo

So you’ve got seven types of primitives and you want to share them across multiple projects. Maybe across a whole team. Maybe across an entire engineering organization. You need structure, or you’ll drown in conflicting instructions and duplicated skills within a month. For now, this works:

Department — The top layer. These are your organization-wide standards. Security policies. Code review requirements. Compliance guardrails. The stuff that applies everywhere and nobody gets to opt out of. Think of it like your company’s engineering handbook, except the AI actually reads it.

Team — The middle layer. Your team’s specializations. Maybe your frontend team has specific React patterns. Your data team has dbt conventions. Your platform team has infrastructure standards. These inherit from Department but add domain-specific knowledge.

Project — The bottom layer. Local context for a specific repo. The stuff that only matters here. Your project’s architecture decisions, custom tooling, specific quirks.

In practice, this lives in a monorepo where each layer is a directory containing virtual subdirectory packages. So you might have:

/department/standards-security
/department/standards-code-review
/team/frontend-react
/team/data-engineering

Each of those is a standalone APM package that can be versioned and depended on independently, but they all live in one repo where you can see the whole picture. You slap CODEOWNERS on the department folders so nobody changes the security standards without review, but teams get autonomy over their own specializations.

Org-Wide Packages Pattern

Solving Context Pollution with Intelligent Compilation

Here’s a problem I didn’t anticipate until I was neck-deep in it: context pollution.

You’ve got department-level instructions. Team-level instructions. Project-level instructions. Skills from three different packages. Prompts from two more. And now your AI assistant is trying to load all of that into a context window that is not infinite. Irrelevant instructions don’t just waste tokens and degrade performance. Tell an AI too many things and it starts forgetting the important ones.

APM solves this with apm compile, which transforms all your scattered primitives into optimized, hierarchical AGENTS.md files. It figures out which instructions belong at which level and how to structure them so the AI gets the most relevant context first.

The conflict resolution model is opinionated: local project files always win. If your project has an AGENTS.md and an installed package also has one, apm install skips the existing file unless you explicitly --force it. During apm compile, instructions get merged intelligently based on file patterns, but your local overrides stay on top. This is the right call. The project knows itself better than any upstream package does.

I found this out the hard way, naturally. I had a package that defined broad coding standards and a project that had specific exceptions. Without the compile step, the AI was getting contradictory instructions and doing that thing where it apologizes and asks which rule you’d prefer it follow. With apm compile, the hierarchy is built in. The AI just does the right thing.

Compilation & Optimization Guide

Automating the Standards

Once you have a pattern, you need a way to use without having to explain it every time. So I built util-apm-builder: a meta-skill that helps scaffold new packages. Yes, I used an AI tool to build a tool that teaches AI tools how to use AI tools.

Building this taught me something important about how AI skills actually can be structured. I do have skills that consist of a single SKILL.md. They just describe what the skill doe and have a couple of examples. I have created more advanced skill with workflows and references too. I was used to Claude Code.

But I was in GitHub Copilot world now and the structure it built for the first skill was really interesting:

Instructions — Guardrails about the monorepo’s directory structure. “Department packages go here. Team packages go there. Don’t create folders outside this hierarchy.” Without these, the AI will hallucinate creative new locations for things.
Context — The technical knowledge base. Manifest schemas. Valid field values. What apm.yml actually accepts. This is the reference material the AI consults mid-task, and without it, you get manifests that look right but fail validation.
Prompts — The executable task template. “Create a new package” with defined inputs for name, layer, type. This is what the developer actually triggers.

A SKILL.md at the root makes it a hybrid package: part skill, part instruction set.

Agent Workflows (Experimental)

Local Iteration and the Playground Strategy

Let me save you from a mistake I made so you can make different, more interesting mistakes.

Do not install APM packages at the root of your monorepo during development. I did this. What happens is the AI discovers your package source files in /department/my-package/ AND the deployed copies in apm_modules/ and .claude/, and now it’s seeing the same instructions twice from two different locations. It doesn’t know which is authoritative. It gets confused. You get confused. Everyone’s confused. It’s a bad time.

The fix is stupidly simple: create a /local folder as your playground. It’s a separate workspace where you install packages using relative path dependencies:

# /local/apm.yml
dependencies:
  - path: ../department/util-apm-builder
  - path: ../team/frontend-react

This gives you fast iteration without pushing to a remote registry, and it keeps the source packages and the deployed copies in separate directory trees so the AI doesn’t see double.

One gotcha: VS Code only discovers skills at the root of an open workspace. So if you’re testing a new skill in /local, you need to actually open that folder in VS Code, or set up a multi-root workspace that includes it. I spent ten minutes wondering why my skill wasn’t showing up before I figured this out.

For git discipline: add apm_modules/ to your .gitignore (it’s like node_modules/, derived, not source), but commit apm.lock.yaml and the deployed primitives in .github/ and .claude/. The lock file ensures reproducibility. The deployed files ensure any developer who clones the repo gets the same AI context without needing to run apm install first.

Dependencies & Lockfile Guide

From Supervisor to Architect

There’s a maturity curve to working with AI coding assistants, and most of us are stuck somewhere in the middle of it.

At the beginning, you’re a Supervisor. You watch every line the AI writes. You correct it constantly. You paste errors back into the chat. You basically do pair programming where your partner has amnesia and you’re doing all the navigating.

The next level is what I’ve been doing for a while: running multiple AI tools on multiple projects simultaneously, trusting them with larger chunks of work, letting them plan and execute while I review the output. It’s better, but it’s still reactive. You’re managing agents, not engineering systems.

What APM enables is the jump to Architect. You define the standards, the guardrails, the knowledge hierarchy, and the execution patterns once. You version them. You distribute them. And then every AI assistant that touches any project in your ecosystem automatically knows how to behave, what standards to follow, and what context matters. You stop supervising individual interactions and start engineering the environment those interactions happen in.

The best part is the escape hatch. “Back in the day” in AI terms is last month. Who knows what will change. APM’s output is native configuration files: CLAUDE.md, AGENTS.md, .cursor/rules, skill definitions. If APM disappears tomorrow, or you decide it’s not for you, those files keep working. You haven’t locked yourself into anything except having better-organized AI context, which is not exactly a downside.

I Burned Out on Vibe Coding, Came Back, and Rewrote Everything

Stephan Miller — Sun, 08 Feb 2026 07:00:00 +0000

I hit a wall with vibe coding. Not a dramatic crash. More like the slow realization that I’d been sprinting for months and couldn’t remember why. I had 15 projects in various states of “maybe done,” a GitHub commit chart that looked like a heart monitor, and a growing suspicion that I was building things just to build things.

Fortunately, freelance writing work picked up right around the same time. Enough to actually pay attention to it. So I stepped away from the side projects, wrote about other people’s technology for a change, and let my own code sit untouched for a few months.

When I came back, I had no patience for bullshit. And I looked at my projects differently.

Your Vibe-Coded Apps Are Prototypes (And That’s Fine)
Making “Adding Features” the Feature
Building Bottom-Up with Verdent and Claude Code
The 60 Missing APIs
Making Plans That Any AI Agent Can Execute
The Same Pattern, Different Project
What Changed

Your Vibe-Coded Apps Are Prototypes (And That’s Fine)

Here’s the thing I couldn’t see while I was in the thick of it: almost everything I’d built with AI coding tools was a prototype. Not in the dismissive sense. These apps worked. EmberText was a functional Electron writing app. Niche Site Factory could generate and manage content sites. They ran. They did things.

But they were all built top-down. I’d tell the AI “build me an app that does X” and it would scaffold the whole thing, features and all, in one giant session. The problem is that when you build top-down with AI, you end up with something that works but is almost impossible to extend. Every new feature is a negotiation with the existing architecture. You’re not adding to the app. You’re fighting it.

EmberText was the clearest example. I built it with Claude Code over about 16 hours and $80 in API costs. It had AI integration, text generation, character relationship graphs, plot scaffolding. Impressive on paper. But by the time I realized it should have had a plugin architecture, I was already deep enough that refactoring meant essentially starting over.

So that’s what I did.

Making “Adding Features” the Feature

The insight that changed everything was stupid simple: instead of building an app with features, build an app where adding features is the feature.

I’d been using Obsidian for years and it’s in my top 5 favorite software. It’s incredible for notes, planning, and organization. You can even make it distraction-free for writing. It’s just not the default, and “not the default” matters more than you’d think when you’re trying to get into a flow state. I tried to hack around this with my Daily Prompts plugin that launched an alert and opened a daily note in Zen mode. It worked, kind of, but I was still fighting the tool.

VS Code is for code. Obsidian is for notes. What’s for writing?

That question led to Veneer, a complete rewrite of EmberText from scratch. Same idea, a distraction-free writing environment, but built from the ground up as a plugin-first architecture. The “Zen-First Shell” concept: when you open it, you see nothing but a clean sheet and your text. Sidebars, ribbons, status bars exist as ghost elements, hidden by default, appearing only when you hover near the edges or hit a hotkey. Everything that isn’t the writing surface has to earn its right to be on screen.

And critically, every feature is a plugin. The file explorer? Plugin. The markdown editor? Plugin. The command palette? Plugin. Even core functionality ships as plugins that can be swapped, extended, or replaced. This isn’t just for a future community. It makes the whole thing dramatically easier to build with AI, because each plugin is a self-contained unit with clear boundaries. You can hand an AI agent a plugin spec and let it work without worrying about it breaking everything else.

Building Bottom-Up with Verdent and Claude Code

I used Verdent to build the base application. If you read my post about Verdent, you know this thing is fast. Too fast, honestly. It finished most of the base app, including a file browser sidebar plugin, a markdown editor plugin, and a command palette, in about 220 tokens, roughly $20 worth of credits. There were bugs left when I ran out of tokens, but the foundation was solid.

But here’s where the process got interesting. Instead of just continuing to add features on top, I switched to Claude Code and did something I hadn’t done before: I asked it to audit the codebase.

use your skills and check the repo for best practices
- UI
- Is it themable like Obsidian or VS Code
- Plugin Architecture (and compare to VS Code and Obsidian)
- TypeScript
- Electron
- Structure, Naming Conventions

I’d forgotten how many skills and plugins I had installed in Claude Code. When I ran this, it deployed four specialized agents in parallel:

Explore subagent analyzed the overall project structure, UI patterns, theming, and naming conventions
Architecture Strategist evaluated system design decisions and compared the plugin architecture against VS Code and Obsidian
Kieran TypeScript Reviewer checked strict mode compliance, type safety, interface definitions, and generic patterns
Best Practices Researcher gathered industry standards and found examples from successful projects

This is not how I was working six months ago. Six months ago, I would have just told the AI to add the next feature and hoped for the best.

The 60 Missing APIs

The audit turned up a lot. Claude gave the codebase an A- (92/100) overall, which sounds great until you read the details. The critical finding was the plugin API gaps. Obsidian provides 60+ plugin APIs. Veneer was missing most of them.

No modals. No notification system. No context menus anywhere. No way for plugins to subscribe to file or workspace events. No way to extend the CodeMirror editor. The native OS menu had “Open Folder” under “Veneer” instead of “File,” which is the kind of thing that makes you realize the AI built the structure but didn’t think about the conventions.

I had Claude store all the findings in the project’s docs folder:

BEST_PRACTICES_REVIEW.md : Everything organized by priority with an implementation roadmap
PLUGIN_API_GAPS.md : A detailed comparison against Obsidian and VS Code showing exactly what was missing

Making Plans That Any AI Agent Can Execute

This is the part that parallels Mitchell Hashimoto's AI adoption journey. He talks about “harness engineering,” the idea that every time an agent makes a mistake, you engineer a solution so it never makes that mistake again. Better implicit prompting. Actual programmed tools. The goal is building up an ecosystem where agents get better over time.

I’m doing something similar, but at the project planning level. Instead of just fixing bugs as they come, I’m creating structured documentation that any AI tool can pick up and execute. My next prompt to Claude was:

Take docs/BEST_PRACTICES_REVIEW.md and docs/PLUGIN_API_GAPS.md and create a
markdown list of TODOs in the docs folder. These should be grouped into tasks
and subtasks. If it is possible to work on some tasks concurrently this should
be mentioned. This file should be able to be used by an AI agent to finish
these tasks. Add enough details to each task to speed up development time.

Now I have the work planned in 3 phases across 3 TODO files, plus a final phase listing every Obsidian plugin API that Veneer doesn’t have yet for future development. These are all in the docs folder of the project, version controlled, and written so that any agent, Claude Code, Jules, a VS Code extension with Qwen, whatever, can pick them up and start working.

This is the difference between vibe coding and what I’m doing now. I’m still using AI to do the heavy lifting. But I’m not just throwing prompts at the wall. I’m using one AI tool to build, another to audit, and then creating structured plans that decouple the what needs to happen from the which tool does it.

The Same Pattern, Different Project

This isn’t just how I rebuilt Veneer. I’m doing the same thing with Niche Site Factory. Instead of telling an AI to “build me a niche site generator” (which is roughly what I did the first time), I started over by building the data model first.

I took a real project, a sci-fi encyclopedia wiki, and used it to design the content structures. A knowledge graph in PostgreSQL with pgvector for embeddings. 2,622 books ingested into the entities table. Flexible JSONB storage that can handle books, concepts, authors, movies, whatever. The data model came first, the application came second.

It’s the same bottom-up principle. Don’t build the house and then figure out the foundation. Build the foundation, verify it’s solid, then build up from there.

What Changed

I think the burnout was actually useful. Stepping away let me see the pattern I was stuck in: build fast, hit a wall, start something new. That’s fine when you’re learning the tools. It’s how I figured out what Claude Code, Kiro, Verdent, and Jules are each good at. But at some point, you have to stop prototyping and start building.

Here’s what’s different now:

Bottom-up, not top-down. Start with the architecture and data model, not the features. Let the AI build on a solid foundation instead of improvising one.
Audit before extending. Use AI review tools to find the gaps before you pile on more code. It’s cheaper to fix the structure now than refactor later.
Plans as portable artifacts. Write TODO files detailed enough that any AI agent can execute them. Don’t marry yourself to one tool.
Plugins as a development strategy. A plugin architecture isn’t just for the community. It makes AI-assisted development dramatically easier because each unit is self-contained.
Past work is research. EmberText wasn’t a failure. It was a $80 prototype that taught me exactly what Veneer needed to be.

I’m still vibe coding. I’m just vibing with more structure now and calling it AI-assisted development. And honestly, after a few months of writing for clients and not touching my own projects, coming back to this with fresh eyes and no patience might be the best thing that happened to any of them.

Verdent AI - When Your AI Coding Assistant Finishes Before You Can Get Coffee

Stephan Miller — Tue, 23 Sep 2025 13:00:00 +0000

My current AI development process, if you want to call it that, is getting one AI tool working on one project while having another work on a different project. This only works if one or both projects aren’t near the end where I have to test and have AI fix a lot of things. This process grew out of the fact that sometimes it takes AI a little while to get a task done, but not enough time for you to get any real work done yourself. So it was either work on another project or scroll through Reddit.

But Verdent put a kink in that plan.

I needed something for Verdent to work on. When I had Kiro build Obsidian plugins, one which was relatively simple, it created over a dozen tasks and took anywhere from two to four hours for each plugin. So I figured a couple of plugins would be enough work for a Saturday afternoon.

The Projects: What I Threw at Verdent

I had two ideas picked out to build and I used Auto Run mode to build both:

Obsidian Cleaner Plugin : A “cleaner” plugin that checks that attachments in the Obsidian attachment folder and provide you with a checkbox list of all those that aren’t linked so you can delete them. It does the same thing with conflicted files. If I find any more common things I clean like this, I will add them later.

3D Tag Explorer Plugin : A plugin that takes hierarchical tags like llm/writing/software and turns them into a 3D node graph with notes containing those tags included as the final nodes.

Pretty straightforward stuff. I figured these would keep Verdent busy while I worked on something else.

The Reality Check: When Hours Becomes Minutes

Verdent finished both of these Obsidian plugins in less than 15 minutes each.

Now these plugins were relatively simple, but I did not expect that.

There was one bug in the tag explorer where the background was above the node graph. I mentioned it to Verdent and it fixed it quickly. The cleaner plugin just worked on first try.

So now I’m sitting here at 10:30 AM on a Saturday with two working plugins and I really wanted to focus on another task while Verdent just did its thing. This is not the kind of problem I expected to have with AI coding tools.

Panic Building: More Projects to Feed the Beast

I had to scramble to find more work for Verdent to do. Here’s what I threw at it next:

BookForge

A web app, API, command line app, and library that converts markdown files into simple epubs. This was simple but I bet it took less than 30 minutes. I really did not trust it.

It was essentially done with the first commit and worked well enough to be the MVP. One commit. For a complete application with multiple interfaces. What the hell is happening to software development? I have made a total of eight commits to the repo. The rest were to add docker, make slight modifications to the text in the web app, and create a release script so I can use it as a library in other projects.

PromptOS

A service, libraries, and extensions to store prompts and other text, markdown, and JSON instruction files for AI. This was the most complex of the four projects. Verdent broke it into 4 phases. After each phase I did a commit and approved it to start on the next phase.

Site Factory

A project for building pre-configured Gatsby sites quickly. I’m still unsure of the architecture of this one and over-architected it twice. This teaches me not to use tools I haven’t used before and have AI build the project at the same time. I started over again after reading documentation on Gatsby and the other tools I planned to use to get a better idea of what I actually wanted.

Mostly Static

A set of services and dashboard for static sites. I threw this in at the last minute to put the last bit of my generous beta access to work. This was multiple phases also. But that Saturday, Verdent built two Obisidian plugins and four applications in a few hours. And did not use up the 2000 credits I got for the day.

Verdent Features: The Good Stuff That Actually Works

I didn’t test Verdent Deck because my personal Mac is still an Intel one and apparently I’m living in the stone age of computing.

Plan Mode

Verdent’s plan mode is where things get interesting. It will have you approve the plan before it starts building. If you’re in Auto Run mode, it will just run until it’s done with the project, though it did stop and ask about certain commands that might be destructive. Or you can go directly to Skip Permissions mode and have it stop asking question. I stuck to the middle road and only had to tell it to continue a couple of times.

If your directions are vague, it will ask you a series of questions to help ensure it builds what you are expecting.

For larger projects, it breaks work into phases and tells you when each is done. PromptOS was the most complex in that it had an API, website, desktop app, and extensions for two applications. Verdent broke that into 4 phases automatically. Another project I started building after this set was broken in 11 phases.

You may want to tell it to save the plan to the project docs folder, so you can keep it in version control for reference, since it doesn’t do that automatically. In the newest version, you can copy the content of the plan from the chat and save it to the project if you want.

The other features, to tell you the truth, I haven’t touched yet, because I simply didn’t need to.

Rules System

The rules system lets you set custom instructions that persist across projects. This is useful for coding standards, preferred libraries, or just telling it not to do stupid shit that you’ve seen it do before.

Subagents

Verdent uses specialized subagents for different types of work. You don’t have to think about this much - it just routes tasks to the right AI worker automatically.

MCP Support

Model Context Protocol support means Verdent can integrate with other tools and services. This is probably more useful than I realize, but I haven’t had time to explore it fully given how fast everything else has been happening.

The Pricing Reality Check

During the beta, I got 2000 credits a day. These were hard to use up even when I was actively trying to burn through them. Once the beta was over, I received a bucket of credits. Right now, I’m testing how long these credits last to determine how I’ll use Verdent in the future.

The pricing tiers are reasonable for what you get. If you’re doing any serious development work, the cost of the tool becomes insignificant compared to the time it saves. But I’m still figuring out my usage patterns before committing to a subscription.

The Verdict: Too Fast and Good to Ignore

I was definitely happy with the results. During the beta, getting through 2000 credits in a day required serious effort. The speed and quality of the output is genuinely impressive.

Because I will be using it in the future. It is just too fast and good not to. Right now the only AI subscription I have is Claude Pro, so I use Claude Code mainly. But I also have API accounts at most of the big AI companies. So I might just pay for credits as I go for a while until my usage is consistent and then pick up a subscription.

The multi-AI workflow is becoming essential. When one tool is thinking, another can be building. When you have AI assistants that can complete substantial projects in under 30 minutes, the bottleneck becomes your ability to feed them work, not their ability to do it.

Conclusion: The Future of Lazy Coding

Verdent actually delivered on its promises. The difference between Verdent and some other AI coding tools I’ve used is the speed, the fact that it rarely gets confused about what I’m asking it to do, and, even though I was dreading testing the apps it built, they had less bugs and weirdness than I expected.

We’re at the point where AI coding assistants are good enough that the economics start to make sense for most developers. When something can build a complete application in 30 minutes, you find a way to afford the monthly subscription.

The real challenge now isn’t getting AI to write code. It’s keeping up with how fast it can work and making sure you’re feeding it projects that are actually worth building. But honestly, that’s a good problem to have.

I’m still figuring out the economics of AI-assisted development, but when something works this well, you adapt. The alternative is falling behind while other developers are shipping software at warp speed.

And if you’re still writing everything by hand while AI tools like Verdent exist, well, you might want to reconsider your approach. The future of coding is here, and it’s fast enough to finish your weekend projects before lunch.

You can learn more about Verdent here and find the VS code plugin or Verdent Deck here.

The Great Vibe Coding Experiment - How I Built 15 Projects with AI in My Spare Time

Stephan Miller — Mon, 15 Sep 2025 07:00:00 +0000

I started this whole thing wanting to test Claude Desktop with MCPs. Just one little experiment. You know how that goes.

Six months later, I’ve got 15 projects in various states of “done” and a GitHub commit chart that doesn’t look too crazy until you realize that most days I only have a couple of hours to experiment with this, when I do have time.

Welcome to “vibe coding”: building shit because it feels right and letting AI do most of the heavy lifting. It’s not agile development. It’s not waterfall. And as I’ve done more of it, it has become let chaotic and more stuctured. Most nights I work on two projects simultaneously.

When Vibe Coding Becomes Automated Spec-Driven Development
The Obsidian Plugin Empire
- Apple Books Highlights Plugin: Hacking SQLite
- Joplin Portal: My First Test of Kiro
- Daily Note Prompts: An Extension I Am Using
- Tag Explorer 3D: Testing a Shiny New Tool
- Attachment Cleaner: Simple But Necessary
The Bigger Fish: Apps That Do Real Shit
- Gatsby Site with Scraper: The Abandoned First Project
- GitWrite: When Jules Met Agentic Project Manager
- EmberText: My First Claude Code Experiment
- MDQuery: Fed Up with Proprietary Tools
- AutoVibe: The Meta Project
- AutoVibe Template: The Stop-Gap Solution
- ShopBoth: Testing the Template (and Learning Hard Lessons)
Tool Whose Name Shall Not Be Spoken
How These Projects Work Together
What I Learned About AI-Assisted Development
- Each AI Tool Has Its Own Personality
- The Process That Emerged
- Why 15 Projects Makes Sense (Sort Of)
What’s Next: The AI Development Army
Conclusion: Embrace the Chaos

When Vibe Coding Becomes Automated Spec-Driven Development

Vibe coding is when you have an idea, fire up an AI coding assistant, and see what happens. No detailed specs. No project management software. Just “hey AI, build me a thing that does X” and then iterating until it works or you get distracted by building something else.

But there was a reason I abandoned my first vibe coding project. I just added random features to an idea that I can up with on the fly and thought maybe ten minutes about. I just wanted to see what it could do. But the second project, I knew I needed some kind of rails.

The next step was to get out of this loop:

Tell the AI tool to create the feature
Test the result and tell the AI tool to fix the errors and failed tests
Maybe do that again or a few more times, copying and pasting errors
Ask the AI tool how to prevent this from happening with a better prompt

And realize it could be a self-optimizing process of:

Actor: Have an agent write the code
Auditor: Have an agent review the code and run the tests (multiple types of auditor) and either pass or fail and send back to Actor
Process Improver: Have an agent examine the steps that caused the failed process and update the commands, agent definitions, or other project docs to prevent them in the future

All because you can trust AI to do what you thought you told it to do. And this trail of projects documents my journey towards that goal.

The Obsidian Plugin Empire

Apple Books Highlights Plugin: Hacking SQLite

I built this plugin with Claude Desktop and MCPs, documented the whole process in this vibe coding post. The plugin actually works. I use it daily. It extracts annotations from the macOS Books SQLite database and creates formatted markdown notes.

Joplin Portal: My First Test of Kiro

I had all these notes in Joplin that I wanted to access from Obsidian. So I fired up Kiro AI and told it to build me https://github.com/eristoddle/joplin-portal. Kiro did most of the work.

The surprising thing about letting AI write plugins is that they actually follow best practices better than I do. Kiro created proper TypeScript interfaces, handled errors gracefully, and even added settings panels I didn’t ask for.

Daily Note Prompts: An Extension I Am Using

Another Kiro collaboration. This one adds customizable prompts to daily notes. I actually use this, which is more than I can say for many of the plugins I’ve tried.

It still needs some work, but it works for me for now. I’m just keeping a running list of changes I want to make to it and bugs I’ve run into and one of these days, I’ll put the list in one of these AI coding tools and set it to work.

Tag Explorer 3D: Testing a Shiny New Tool

I started this project the day after making my list of existing projects. Why? Because I wanted to test a new AI tool (can’t name it yet) and needed something to build.

Tag Explorer 3D visualizes your Obsidian tags and notes with those tags in 3D space. Is it necessary? Probably not. Is it cool? Absolutely. Sometimes you build things because they’re interesting, not because they solve real problems.

Attachment Cleaner: Simple But Necessary

Really simple plugin built with the same unnamed AI tool. It finds and removes unused attachments from your vault. I keep blog posts drafts there and paste images in, which gets stored there and they get forgotten when I move the draft.

I need to test it more, but the code looks solid. It’s often better at the simple, boring stuff than the complex, interesting stuff if you want AI coding to work. And I may turn it into a general clean up tool, because I am also tired of tracking down conflicted files.

The Bigger Fish: Apps That Do Real Shit

Plugins are fun, but eventually you want to build something more substantial. That’s where things got interesting.

Gatsby Site with Scraper: The Abandoned First Project

This was my first attempt at vibe coding. I wanted to build a Gatsby site with an integrated scraper using Claude Desktop and MCPs.

I abandoned it. Not because it didn’t work, but because I realized I was building it the wrong way. Sometimes the most important decision is knowing when to stop.

GitWrite: When Jules Met Agentic Project Manager

GitWrite is an abstraction over git for writers, editors, and beta readers. I built it with Jules AI, used an Agentic Project Manager to coordinate, and finished it up with Qoder. Well, it said it was finished and I am still working my way around to testing it’s full functionality. Just made the mistake of finishing it before I needed it.

It exists to support EmberText and other writing tools I’m building. I am really, really, really tired of being required to use things like “Suggesting” in Word and Google Docs. Who thought this thing up? Satan? I like Git better but it needed dumbed down in some places and tweaked in others to do what I needed.

EmberText: My First Claude Code Experiment

EmberText was my first serious relationship with Claude Code. I built an Electron app for writers, rolling my own context and project management system.

Claude Code has quirks. It’s opinionated about project structure. It sometimes goes off on tangents. But when it works, it does pretty well. EmberText is a fully functional app that I am testing, but I also need to refactor it to use some of the services I am building, so it is probably the last of these project that will be finished.

MDQuery: Fed Up with Proprietary Tools

I got tired of proprietary MCPs and tools to search systems that essentially consist of markdown files: Obsidian, Joplin, Jekyll, Log Seq, and on and on. So I built a universal tool with Kiro and Qoder.

MDQuery provides SQL-like syntax for searching and analyzing markdown files across different note-taking systems and static site generators. Because fuck trying custom MCPs that don’t work the way you want for each and every markdown based platform.

And I already had a job lined up for the tool. I had been collecting everything interesting around vibe coding and spec-driven development in my Obsidian vault under a specific tag. Developers were going in so many directions that I wanted to categorize and get an overview. We’re all blind developers describing different parts of this elephant. Maybe if we categorize what we’re all doing, I can be a little less blind. I recently found this post on Claude Code framework wars that does just that.

So I attached the MCP to Claude desktop and prompted it to analyze all of my Obsidian notes, using the prompts the AI tools had put in the documentation. And it spit out an 80 page document.

AutoVibe: The Meta Project

AutoVibe is the most recursive project I’ve ever built. I’m using Claude Code, AI Studio, and Backlog.md to build a tool that will make the vibe coding process smoother.

It’s infrastructure for building infrastructure. Custom commands and agents to coordinate AI tools. The future of development might be less about writing code and more about conducting orchestras of AI agents. At least, that’s what I think it is. But it’s like a box of chocolates.

AutoVibe Template: The Stop-Gap Solution

While AutoVibe has bugs to work out, I needed something that worked now. So I built a template with a set of Claude commands and agents that, when used with Backlog.md and Claude Code, makes developing projects more bullet-resistant.

I used Claude Desktop to help develop it since the template is full of AI instruction files that other tools would try to execute. But then I sort of got stuck testing its usage in the next project. So now the priority is getting AutoVibe to work and keep the simple template.

ShopBoth: Testing the Template (and Learning Hard Lessons)

I used Claude Code with my Backlog.md template to build ShopBoth, a React Native app for testing and tweaking the template. Why did I choose React Native for testing? Because I’m an idiot.

I also discovered that “generic” automated QA doesn’t exist. QA needs to be specific to the platform, the framework, the use case. There ain’t no such thing as universal testing, and I learned that the hard way.

Tool Whose Name Shall Not Be Spoken

I’ve built three more apps with an AI tool I can tell you about in a couple of weeks. They add to the ecosystem:

BookForge transforms markdown files into professional ebooks. It supports EmberText and GitWrite, completing the writing workflow.
PromptOS stores and provides prompts for EmberText, AutoVibe, and everything else. Because managing prompts across 15 projects gets complicated fast.
Site Factory brings me full circle to that abandoned Gatsby project, but with actual research this time.

These aren’t random projects. They’re pieces of a larger system for AI-assisted content creation and development.

Update: I actually built one more project while I was writing this. It’s a platform to provide dynamic services for static sites.

How These Projects Work Together

This isn’t just a collection of random tools. There’s method to the madness:

EmberText handles writing → GitWrite manages collaboration → BookForge generates ebooks
PromptOS supports everything with prompt management
Obsidian plugins feed the writing process with research and notes
MDQuery searches across all the markdown files these tools create
AutoVibe coordinates the development of new tools

It’s an actual ecosystem, not just 15 disconnected projects. Each piece makes the others more useful.

What I Learned About AI-Assisted Development

Each AI Tool Has Its Own Personality

Claude Desktop: Great for planning and architecture, terrible for actually writing code. It’s the project manager that never writes any code. Mainly because chat lengths are limited. As soon as you get somewhere, you have to start a new chat and lose the context. I do use it to develop git templates for AI-driven coding projects, because it will ignore the instruction files I have it tweak.
Claude Code: I am not sure if it is still the best after using Qoder and the new tool I have. I still use it almost daily to work on projects and it’s my goto tool, but the limit comes up quicker now and it seems like it has dropped some IQ points. Not sure about it status in my workflow right now.
Kiro: The reliable workhorse. Give it a clear task and it delivers solid, working code. Perfect for plugins and smaller projects. After seeing how the new tool I found works, I wonder if it breaks things down into too many tasks. To create an Obsidian plugin, it broke it down into 16 tasks I had to click to get through and it took a few hours. The new tool just decided it would build a plugin for me in 15 minutes.
Jules: I still use it for small things, like fixing bugs, because I still get 15 free chats a day. I actually built most of GitWrite with it.
Qoder: Set it loose on a complex project and come back in a few hours to find it’s built everything you asked for and more. The wikis it build are useful but I think would eat up a lot of tokens in large codebases.
Unnamed Tool: The new experiment. Still figuring out its personality. But it’s fast. It finished the two Obsidian plugins in less than an hour, both of them. It built the other full projects in one day, all of them.

The Process That Emerged

Start with an actual need, not cool technology. I tried building technology first and it never worked.
Let AI handle the boilerplate and focus on the interesting problems. AI is great at CRUD operations and terrible at creative problem-solving.
Build infrastructure projects to support main projects. GitWrite exists so EmberText can focus on writing, not version control. Also, hoping less scope mean less context an AI tool needs
Test new AI tools with real projects, not demos. You learn more building something you’ll actually use.

Why 15 Projects Makes Sense (Sort Of)

Each project taught me something new about working with different AI tools.
Building an ecosystem requires multiple pieces. You can’t do everything with one app.
Some projects exist only to support other projects. That’s fine.
Abandoning projects is part of the process. That abandoned Gatsby site taught me what not to build.
The commit chart tells the story. Bursts of activity when testing new tools. Long gaps when focusing on one complex project.

What’s Next: The AI Development Army

AutoVibe is getting closer to making this process smoother. The template approach gives consistent results. The unnamed AI tool projects will add new capabilities.

I’m building a sustainable vibe coding workflow that lets me maintain 15+ projects without losing my mind. The future of development might be more like conducting an orchestra than playing a solo instrument.

Conclusion: Embrace the Chaos

I started wanting to try Claude Desktop with MCPs. I ended up with 15 projects, a completely new development process, and insights into how AI-assisted development actually works.

The learning process matters more than perfect planning. AI tools are getting good enough to enable this kind of scattered productivity. You can afford to be curious, to follow tangents, to build infrastructure for projects that don’t exist yet.

Vibe coding and spec-driven development isn’t for everyone. But if you’re comfortable with chaos (Yes, even in spec-driven development), if you like building things just to see what happens, if you want to push the boundaries of what one person can build with AI assistance, give it a try.

Just don’t blame me when you end up with 15 projects and a commit chart that looks like madness. That’s the price you pay. And while I was finishing this up, I started another project. I had to give my new tool something to work on before the free ride runs out.

How I Built Two Obsidian Plugins While Kiro AI Did Most of the Work

Stephan Miller — Fri, 22 Aug 2025 07:00:00 +0000

For the first Obsidian plugin I wrote with AI, I used Claude desktop and already had a Python script that did most of what I needed the plugin to do. I just needed AI to convert it to an Obsidian plugin. Then I used Jules to fix the final bugs.

But I have been moving away from the chaos of vibe coding so I can get results that look more like the vision I have in my head. This started when I rolled my project management system with Claude Code and a bunch of markdown files. Then I found out that Backlog.md could make my vibe coding projects go more smoothly.

But along the way, Kiro came out and had a spec-driven development feature. So I figured I’d try it building even more Obsidian plugins. Why two, though? Well, the first one was more complex, and the more complex a project, the more things AI leaves behind that you have to clean up. The second one was actually simple enough that I am currently prepping it to release it officially.

What is Kiro and Why I’m Using It
Plugin #1: Daily Note Prompts - The Spec-Driven Experience
- Requirement 1
My Obsidian Development Process
Plugin #2: Joplin Portal - When AI Tools Hit a Wall
Lessons Learned
- Spec-driven development workflow
- When to trust AI vs when to investigate yourself
- Managing AI tool limitations and daily limits
- The value of simple projects for learning new tools
Conclusion

What is Kiro and Why I’m Using It

Kiro is an AI coding tool that takes a different approach than most of the tools I’ve been testing. Instead of just throwing code at you, it starts with what they call “spec-driven development.” You give Kiro an idea, and it creates three files: requirements.md, design.md, and tasks.md.

Kiro breaks down your project into digestible chunks before writing a single line of code. No more “let’s see what happens” coding sessions that end with you staring at a pile of TypeScript wondering how you got there.

But here’s the real reason I’m going all-in on Kiro right now: it’s completely free. No credits, no tokens, but there is a daily limit that cuts you off for 24 hours, but I can deal with that.

So I’m testing Kiro on every coding project I can think of before the free tier disappears. Obsidian plugins are perfect for this because I use Obsidian daily, I have a constant stream of plugin ideas, and I can see the results of mine and Kiro’s efforts quickly.

Plugin #1: Daily Note Prompts - The Spec-Driven Experience

I used to write daily. I had a morning practice where I woke up, meditated, read, and then wrote. But who has all that time? Actually, I’m kicking myself because I had a three-year streak, and it only took one missed day for me to give that up. The “streak” gurus never tell you about that part.

And for a while, I was doing it in Obsidian using Daily Notes. Now I wanted a plugin to nag me about it and give me a prompt to start with. So that was the start of the idea. Here’s how I fleshed it out:

Prompt packs in JSON format can be imported and exported with these attributes: - Type: ‘Sequential’, ‘Random’, ‘Date’ - Prompt: Link, String, or Markdown - Date: For Date type like devotionals - Order: For Sequential type that have to be done in order
Set Reminder/Alert with System or Obsidian notification.
Launches daily note with prompt
Automatically go into zen mode

And that is basically what I gave Kiro in the first prompt. You can actually still “vibe code” with Kiro. You just have to select that option.

And Kiro built the three spec files in order, waiting after each file for my approval or for me to request changes.

The requirements file has entries that look like this:

Requirement 1

User Story: As an Obsidian user, I want to create and manage prompt packs with different delivery modes, so that I can organize my writing prompts according to my preferred workflow.

Acceptance Criteria

WHEN a user creates a new prompt pack THEN the system SHALL allow them to specify the type as ‘Sequential’, ‘Random’, or ‘Date’

WHEN a user adds prompts to a pack THEN the system SHALL support prompts as links, strings, or markdown content

WHEN a user creates a Sequential prompt pack THEN the system SHALL allow them to define the order of prompts

WHEN a user creates a Date-based prompt pack THEN the system SHALL allow them to assign specific dates to prompts

WHEN a user creates a Random prompt pack THEN the system SHALL randomly select prompts without repetition until all are used

The design file specifies things like data models, service interfaces, and other architectural details.

The final file it creates is the tasks file which look like a basic list of nested to dos in markdown:

But when it is loaded in the Kiro IDE, it adds a Start task link you can click on to have Kiro to start working on it.

I am not sure whether adding a command to the trust list was buggy, or I just didn’t know how to use it. It would seem that you could just click the play triangle once and the double play triangle to trust the command in the future, but that didn’t seem to work all the time. Then I realized that the list of commands below the paragraph were also buttons and I had more success with those, but not every time. And like I said, it could be user error, but they didn’t make it easy to figure out. And who want’s to read documentation, anyway? No one’s got time for that.

In the end, I clicked Start task over and over for about four hours while doing other things, finished all the tasks, and decided it was too late to even attempt testing it that night. I know how that goes. So, I tested the next day.

And it actually went pretty well. There were two or three minor bugs. For example, the time field for notification needed debouncing. It was basically working after about a half an hour of bug fixing. But there is a lot of weirdness. For example:

It rolled its own Zen mode, and it does not work quite right. It removed the sidebars and other things. But I couldn’t figure out a way back to normal mode. I just reloaded Obsidian.
It has global settings that should be overridden by the settings of each prompt pack, but nothing happens to a prompt in a prompt pack if I don’t set the child settings. So, I think most of the global settings do nothing.
It doesn’t track progress.

I am testing it currently and making notes on the changes I need, and once I think I have them all, I’ll start working on it again. I am actually using it, but it is not finished.

Here’s the repo: Obsidian Daily Note Prompts

The other Obsidian plugin idea was much simpler, so I could finish that one.

My Obsidian Development Process

After building my fourth Obsidian plugin (3 using AI), I have a process that works really well:

I will use Kiro for now while it’s free, because it’s free and I could get results with both plugins before I hit the daily limit.
I have a Test vault and keep all my plugin repos in the .obsidian/plugins folder of the vault.
I test the plugin in that vault and keep notes on the improvements and changes I want to make in a note in that vault rather than sending one-off prompts to Kiro.
When I think I have found enough for another spec in Kiro, I just paste what I’ve been collecting into another Kiro spec chat. A chat interface makes me anxious, to tell the truth, because it requires interaction. This allows me to deal with responding on my time. It also means I can plan changes even when I’ve been cut off.

Plugin #2: Joplin Portal - When AI Tools Hit a Wall

With this plugin, I simply wanted to access Joplin from Obsidian and be able to import notes from Joplin directly from the plugin. I know I can copy and paste the note or export notes from Joplin, but a plugin will make things easier.

These are the notes I gave Kiro:

Name: Joplin Portal
A way to access my notes in Joplin from Obsidian so I don’t have to junk up my vault
Use the Joplin Web Clipper API to interact with Joplin: Joplin Data API | Joplin
Add a sidebar panel that searches Joplin notes either by full text or by tag
Also import and convert Joplin notes as Obsidian notes (template, default folder, etc) probably using the same search functionality, with a checkbox to check if you want to import that result.

This project went about the same way. Kiro created the requirements, design, and task files, and I clicked Start task over and over for 3-4 hours (while doing other things) until I discovered that there is a limit to Kiro usage and I hit it. And from what I can tell online, it’s a 24-hour break. And there was one task left, so I actually had Jules finish that task.

And it worked relatively well. The biggest issue was that the plugin was pulling in the Joplin notes as is, and Joplin references images in the note with a Joplin ID. So, unlike the screenshot above, the images were not showing. So in my next Kiro session, I worked on that.

In trying to fix this, I relearned a lesson about using AI for development. If it takes the tool more than two tries to fix something, provide the AI tool more information.

So after about 8 tries and 2 days, I gathered the information it needed myself because depending on how the note was created in Joplin, embedded images showed up in one of three formats. Then I found the function that created the preview and the function that imported the note into Obsidian. I told Kiro about all of this, and once it had that, it fixed the issue in both functions.

And it was pretty close to done, but I wanted to clean up the UI a little and give the important things a little more space. So I started another spec chat on that, and Kiro created a new set of three files called ui-improvements.

Then, to prep for submitting to the community, I had it look up the rules for plugin submission and create some specs for that as well. Now that I know I can keep adding new specs as a project grows, I might try using Kiro with something bigger than an Obsidian plugin.

Here’s the repo: Obsidian Joplin Portal

Lessons Learned

Spec-driven development workflow

Turns out there’s something to be said for planning before you code. What a concept! The three-file system is genuinely useful. When I hit a wall or need to explain a bug to Kiro, I can reference the original specs instead of trying to reverse-engineer what sleep-deprived-me was thinking at 2 AM last Tuesday.

Also, no more scope creep disguised as “quick features.” When everything is specced out upfront, it’s obvious when you’re adding random shit that doesn’t belong. That is if you pay attention instead of accepting everything as if it were only terms and conditions.

When to trust AI vs when to investigate yourself

Give any AI tool exactly two chances to fix a complex problem. After that, you’re just feeding prompts to a very expensive random number generator.

The Joplin image rendering issue taught me this lesson yet again. I spent hours watching different AI tools chase their own tails, generating increasingly elaborate solutions to a problem they didn’t understand. When I all I had to do was stop, investigate what was happening for less than an hour, and give Kiro the details.

AI tools are great at implementing solutions. They’re terrible at debugging complex integration issues that require understanding context they don’t have access to. It remains to be seen when I will actually remember this earlier.

Managing AI tool limitations and daily limits

Treat daily limits like sprint deadlines. Plan your work in chunks that can be completed within the limit, and always end each session by documenting exactly where you are. Nothing sucks more than hitting your limit mid-debugging session and forgetting what the hell you were trying to fix.

Also, don’t game the system by splitting complex tasks into tiny prompts. You’ll just confuse the AI and waste your allocation on back-and-forth clarifications.

The value of simple projects for learning new tools

Complex projects are terrible for learning new AI tools. You spend too much time fighting edge cases and not enough time understanding the tool’s strengths and weaknesses. The Joplin Portal plugin was simple enough that I could focus on Kiro’s workflow instead of drowning in business logic. The Daily Note Prompts plugin had a more going on.

Start simple. Learn the tool. Then tackle the hard stuff when you’re not also learning how to communicate with an AI that may or may not understand what a TypeScript interface is. Simple projects also fail faster and more obviously, which means you spend less time wondering if the AI is broken or if your requirements were just garbage to begin with.

Conclusion

Kiro’s free tier is basically a coding assistant on steroids with no monthly subscription guilt. While it lasts, I’m going to get my money’s worth, even though it’s not costing me any money.

The spec-driven approach works well. Having an AI that plans before it codes instead of just eyeballing everything and hoping for the best is a game changer. But… I was already building my own tool to do this before I heard of Kiro, so I know it’s not “magic.” And I also suspect that it consists mainly of system prompts.

But while it’s free… So if you see a bunch of random Obsidian plugins with my name on them over the next few months, you’ll know why. I’m not building them because the world desperately needs another note-taking plugin. I’m building them because there’s a free AI coding assistant that won’t be free forever, and I plan to learn everything I can while the learning is cheap.

And while I get side-tracked and my own AI software is not ready yet, my next post should be about using AI to build that software. The process I have works really well, and I am building a public GitHub template repository to recreate it.

Jules AI - The (Currently) Free Coding Assistant That Can't Follow Directions But Gets Shit Done

Stephan Miller — Tue, 10 Jun 2025 13:00:00 +0000

Jules is currently free, so I figured it was worth a try. Of course, I had to find out what I could break.

Each user gets these default limits:

5 concurrent tasks
60 total tasks per day
5 codecasts per day (still not sure what this means)

Sixty tasks a day for free? That’s really generous and it meant I didn’t have to worry about reading documentation or instructions the first time.

I Jumped in Feet First

My Obsidian plugin worked well enough for me to use it, but there were issues. Issues that I’d been ignoring because the damn thing worked and I had other shit to do. But I finally added issues to my GitHub repo.

Four issues. Nothing too crazy. Should be straightforward for an AI that can supposedly code.

I gave Jules access to the repo and told it: “I want you to work on the issues in this repository and create a feature branch for each fix. Start with the first one.”

Simple enough, right?

Jules Decides to Ignore My Instructions

Jules fetched the repo, cloned it into a VM, and read the README file. But then it discovered the “Future Enhancements” section I’d added to the README and decided that’s what it wanted to work on instead.

It completely ignored my instruction to work on the issues, the same issues that are listed right above the “Future Enhancements” section, and decided it would work on the “Interactive book selection dialog” instead.

Jules then asked me to approve the plan, but I didn’t see any way to reject it. It was going to auto-approve in about a minute anyway. I thought maybe clicking “Approve” would launch a modal where I could say “no, you illiterate AI, work on the issues I told you to work on.”

It did not.

So it started working on the wrong thing. I messaged “stop” and it replied: “I have stopped the current task. Is there something specific you would like me to do instead, or a different issue you’d like me to focus on from the README?”

It paused for a few seconds and then continued doing exactly what it was doing before.

Currently, as far as I can tell, the only way to reject a plan is to let it start, pause it manually, and then delete the entire task. User experience design at its finest.

Jules finished the feature and gave me the option to click a button to create a feature branch. So now I had a new feature to test before I could fix the actual issues I wanted fixed.

Testing the Feature that Jules Built While Ignoring My Prompt

So I pulled the branch down to the Obsidian vault I use for testing, the one with all the plugins I’m developing loaded up. There was one error in the code, in unit tests I never asked Jules to add:

tests/main.test.ts:160:35 - error TS2554: Expected 0 arguments, but got 1.
sortAnnotationsByCFI: jest.fn((ann: Annotation[]) => ann), // Simple pass-through

I just deleted the broken test.

It took me a while to figure out that the selection modal didn’t launch via the sidebar button like I expected. I had to hit Command+P and find the command to launch it. But you know what? It actually worked.

Since the feature worked, I merged the change and created a new release. Then I did some reading to figure out how to prevent Jules from going off on another tangent.

Back to Fixing Bugs (This Time With Feeling)

I thought maybe I should be more descriptive, but there wasn’t much to be descriptive about. There are issues. I added them to the repo. They’re also in the README that I know Jules can access. Most of the example prompts are very short, shorter than what I’d already tried.

So I tried again: “This github repo has 4 issues listed here: https://github.com/eristoddle/apple-books-annotation-import/issues. Give me a plan for fixing the first issue. If you have no access to these, let me know. Do not do anything else.”

This time it worked.

Both the accidental feature and the bug fix took very little time. I was actually surprised by the speed. But Jules created broken unit tests again. I figured maybe it was because I didn’t have my npm and node versions in the package.json, so I added those and deleted them again.

It actually fixed the bug and at least doubled the number of book covers I was getting.

The Great Color Mapping Mystery

The second issue was about finding annotation colors.

Me: “Fix this feature in the repo: https://github.com/eristoddle/apple-books-annotation-import/issues/5. Do not do anything else.”

I think “Do not do anything else” will be part of my prompt from now on.

Jules said: “The code changes have been reviewed for syntactical correctness and logical soundness. Direct testing with an Apple Books database is not possible, so final validation will be up to the user.”

Translation: “I changed some code and it compiles. Good luck!”

After testing, I told Jules: “Now it seems like there are no annotation color indicators at all in the md files. Before they were all just purple. Now they are just not there at all.”

The second attempt produced different colors, but they didn’t match the highlight colors in the Books app.

I mapped the colors Jules gave me to the actual colors and told it about the differences: “The color images are back and there are a variety of them, but they do not match the colors in the Books app. Here is how they differ (Books app color → markdown color): underline → both yellow and underline, purple → no icon at all, pink → red, yellow → purple, blue → blue, green → green.”

I even included the log showing the distinct annotation style values from my test database. So this bug took three interactions, but Jules finally got the color mapping right.

Taking It to the Big Leagues

My main Apple Books account has 4000 books and over 60 books with highlights. When I first worked on this plugin, the database structure was different, so I wanted to make sure everything worked in the wild before calling it done.

The plugin handled the larger database just fine. All the covers loaded correctly, and the color annotations matched what I saw in the Books app.

Everything I Fixed In This Change

It took about two hours total, not counting the issue Jules couldn’t fix. I let it code while I took notes for this article, worked on Obsidian cleanup, and checked out Reddit for a while.

Here’s what got implemented:

✅ New interactive ebook selection dialog: By accident when I was asking Jules to fix issues, but I’m counting it as a win.

✅ Fixed finding ebook covers: This was the original first issue. Jules doubled the number of covers the plugin could find.

✅ Fixed annotation color icons: Took three tries and some detailed feedback, but now the colors match what’s in the Books app.

✅ Fixed missing annotation dates: This needed a second attempt. I had to tell Jules: “I have include dates set to true and include citations set to false. There are no dates in the markdown files though.” But it got it right the second time.

❌ Did not fix gibberish in epub chapter names: This one turned into a nightmare. The TypeScript build error migrated from the tests that never worked. I tried to get Jules to fix it once and it failed. The second time it was thinking forever, so I tried Copilot in VS Code, and that fixed the build.

But then there was a runtime error I’d run into before when Claude Code tried to use a Node.js ebook parsing package. Something about not being able to access the file system in Electron. I went round and round with Jules on this one and finally gave up.

One thing I realized is that I could tell Jules to build the app to test the build and fix issues, but I had to make it a command. When I asked “Did you build the app to test it?” it said that wasn’t possible. But when I just told it to build the project, it did it.

So I let it churn through fixing the build errors for 30 minutes. When I tested the result, the chapters still weren’t fixed. But I didn’t expect this one to be easy without doing some research first.

I used 5 of my available 60 free tasks for the day.

Jules: Fast, Confused, but Generally Useful

Jules is fast. That surprised me. At least until there’s back and forth on changes, then the conversation slows down.

It was done with the next fix before I finished testing the previous one. I added one feature I didn’t plan on and fixed four issues in a little over an hour.

But Jules went off on a weird tangent when I gave it what I thought were specific instructions. There seems to be no real interaction other than the first message and accepting the plan. I was worried about fixing issues in a branch it created if something didn’t work.

Using a task like it was a chat and trying to fix more than one issue per task got confusing. Then I figured out I needed a different workflow:

1 chat per branch, but can do multiple related things
1 chat can be as big as 1 feature
Code review each branch
If there are issues, go back to the chat and have Jules fix them
If there are build errors, tell Jules specifically to build the app again, test, and fix any errors.
Pull and test again

I was used to my Claude Code process where everything happened in the branch I was on locally. The “Publish Branch” button in Jules seemed scary at first, but the second time I used it, it just pushed modifications to the existing branch. I was expecting merge conflicts or some other drama.

Most things I asked Jules to do worked well, except for the chapter names issue. I also didn’t ask it to create unit tests, but it kept creating broken ones anyway. It each branch it created, I ran them once and if they failed, I deleted them. They mainly failed because of TypeScript type issues.

The Future of Lazy Coding

Sixty tasks a day for free is generous. I’m planning to move everything I have to GitHub that isn’t already there and create issues for any feature ideas or bugs I encounter.

I could envision a process where:

A user puts in a support request
An issue gets created on the repo
I tell Jules to fix it
Jules fixes it, creates a custom build, and the user tests it (don’t know about this, but I can dream)
If the fix works, I merge it and create a new release

I’m not sure exactly how this process would work in practice, but the potential is there.

Verdict: Worth the Price of Free

Jules isn’t perfect. It ignored my initial instructions, created broken unit tests, and couldn’t fix the most complex issue I threw at it. But it’s fast, it’s free for now, and it actually solved most of the problems I needed solved.

In two hours, I got a new feature and fixed three out of four issues that had been sitting in my backlog. The fourth issue would have taken me some research into ePub structure even doing it manually.

I’m definitely planning to shift more of my “vibe coding” projects to Jules. At 60 free tasks per day, I can afford to let it work on the boring stuff while I focus on the interesting problems. In fact, to use 60 tasks a day, I might have to try running concurrent tasks and I can do 5 of those.

And if it goes off on another tangent and builds something I didn’t ask for? Well, sometimes the best features are the ones you never knew you needed.

Creating an Obsidian Plugin with Claude AI

Stephan Miller — Mon, 02 Jun 2025 13:00:00 +0000

With this post, I took a break from the Electron project I am building with Claude Code to see how fast I could get a smaller project done. Since I am building a relatively simple Obsidian plugin, I went back to using Claude Desktop with the sequential thinking and desktop commander MCP tools.

And I learned a few more things about using AI to write code. Read on for the complete story

The Story Behind the Project

I’m obsessed with having all my book highlights and notes in one place, and Obsidian seemed like a great place for them. First, I paid for a year’s Readwise subscription, which really didn’t cost too much and did the job well, but after the year, I realized that I really only used it to import my highlights from Kindle and Apple Books and there were free Obsidian plugins to import Kindle highlights.

Then I discovered that Apple Books stored all these highlights and notes in an open though hard to find and hard to understand SQLite database. So I wrote a Python script to import Apple Books annotations, using example from sources like this Calibre plugin repo and used the Obsidian Python Scripter plugin to run it from Obsidian. I only had AI help when I ran into issues for the first script and wrote most of it myself.

Problems with the Original Python Script

The first issue with the Python script I wrote is that the Python Scripter plugin is no longer available. This was not much of a problem though. I just hard-coded the file path of my vault’s book notes folder in the script and it still worked for me.

Only it wasn’t really working the way I thought it was. There were some issues:

The annotations were not sorted by the location in the book.
The annotations weren’t by chapter (there is still an issue in the current plugin in that I have chapters now, but they are not human readable).
There were no configuration options. I just edited the script when I wanted to change something.

Phase 1: Perfecting the Python Script with Claude

Before jumping right in to creating a plugin, I fixed the Python script’s sorting issue first. My reasoning was that if I got stuck in the plugin development process, I would at least have this script and it would work. And once it did, I could have Claude look at it for a working example. And it turns out this was a good move for these exact reasons.

I also thought I could create this plugin with only a Claude Project and no MCP support, but I was wrong about not using MCP. It might have been possible, but MCP tools made things easier. Here was my first message:

I am using this Python file to import highlights and notes from the Mac iBooks app. I don’t think I am rendering them in order from the front of the book to the back in the markdown file. Let me know if I am and if not how to fix it

The Sorting Challenge

Claude initially gave me three sorting options, but none actually sorted by book location. To test the sort, I put highlights in a book with notes that told their order by location and by when I entered them. So I sent this message, along with a dump of the SQLite table that had the annotations:

The first method didn’t seem to work. The second two seemed to sort by date entered or modified instead of by location in the book. Here is the dump of that database table.

The table dump was key, because Claude analyzed the CFI strings that I’d been struggling with and created a sophisticated parser that could extract positional information from cryptic strings like epubcfi(/6/24[c3.xhtml]!/4/188/2/1,:0,:1). And I was done with that until the end, when I had to revisit it multiple times in the plugin version.

Adding Chapter Detection

With sorting fixed, I pushed further:

Now that we have that fixed, is there a way to put chapter headings where they belong in the resulting markdown?

Because I was using a small set of 5 books with highlights, this seemed to work well. Initially, chapters in some books came out like “c3.xhtml” and various other things. But it magically fixed that.

But in the end, it was not magic. And just to let you know, I was using Claude 4, and Claude 4 will use workarounds and not tell you about it. When I tested the script on an account that had 60 books with highlights, none of the chapters in the results markdown were human readable.

I looked into what Claude wrote and it was basically a hack. It was a series of ifs customized to that first set of 5 books, so only they came out right. These are still not human readable in the plugin. I figure I have to use the SQLite results in unison with the ePub file data to make them so, but that’s what new features are for.

Enhanced Features

I wanted to make sure I added everything I could to the Python script I could before moving onto the plugin and added:

Citation-ready format
More metadata extraction
Better error handling

You can find the code for that here.

Phase 2: Converting to an Obsidian Plugin

It took me about an hour to update the Python script. I figured I was on a roll. I had working code, and all Claude had to do was convert it to JavaScript and make it an Obsidian plugin. Famous last words. There is always something and many times it is something stupid.

I had updated the Python script in a Claude chat inside of a Claude project, but had yet added any Project Resources. Now that I was going to work on the plugin, I uploaded these there:

The final Python script: So it could reference this instead of reinventing the wheel.
The blog post I wrote about the original Python version: I figured it would explain what it was doing and what I was trying to do.
The Obsidian sample plugin repository: I am not sure I needed this. Maybe at the beginning, but I eventually realized (duh), that project resources become part of the context, which means your chats get cut off quicker, so I removed it. It never had problems with the plugin API.

And I sent this message:

I have attached a Python script to import apple books annotations. I have also attached the Obsidian plugin sample for reference. I have also attached an article on how an older version of the Python script integrates with Obsidian. Help me create an Obsidian plugin that does the same thing, step by step.

The Development Process

Claude recognized I had sequential thinking MCP and used it right away and came up with a plan.

But I forgot to tell Claude where the project was located, so after that, it spit out all the files in the chat. I quickly corrected that issue by telling it to create all the files in my project itself.

The initial conversion went surprisingly smoothly:

Project structure creation - all necessary TypeScript files
Database access logic - converting Python SQLite to JavaScript
Markdown generation - translating the formatting logic
Settings interface - creating a proper Obsidian settings panel, which I was even worried about at this point.

Technical Challenges and Solutions

Well, it was functional. But let’s just say I was not even halfway done yet. Also, and I am so tired of this, Claude corrected my name everywhere from “Stephan” to “Stephen”.

Just like Google always does.

No, Google, my name is “Stephan.” But back on track. Here are the issues I ran into after I had a “working” Obsidian plugin.

SQLite Library Issues

A big hurdle was database access. Claude initially tried better-sqlite3 but ran into Electron compatibility issues. We switched to sql.js, a pure JavaScript SQLite implementation, but then faced WebAssembly loading problems.

The solution: Configure sql.js with proper WASM file handling in the Obsidian environment.

Query Formatting Problems

Even after fixing the SQLite library, we had issues with SQL query formatting when passed to the command line. Claude described the approach as “bulletproof” right before it broke, which became a running theme.

Chapter Parsing Regression

One book showed chapters as “Ahr5106 us trade bbp text 2” instead of readable names. This required Claude to revisit the CFI parsing logic and ensure consistency with the Python version. This is still a challenge and I am going to do research before I try to fix this.

The EPUB Metadata Challenge

It was now two hours since I started revising the original Python script. The Python script used ebooklib to extract book covers and enhanced metadata. Finding a JavaScript equivalent proved challenging. And I was a little gun shy by now, so I asked:

Is there anything in JavaScript that will do the same thing the python script did with ebooklib, like get the cover image and other metadata? If you know of something, do not commit code until I say it is working. It works now and I don’t want to commit broken code.

Claude suggested three options, and I chose epub2 for its popularity and feature set. However, it took several rounds of debugging across multiple chat sessions to get ePub parsing working correctly.

Real-World Testing Challenges

Testing on my actual Apple Books account (with 4,711 book) shook some new bugs loose:

Database Schema Variations

The plugin tried to query columns that didn’t exist on my primary account, causing SQLite errors. The Python script handled this gracefully, but the plugin needed updates.

Memory and Buffer Limits

With thousands of books, we hit “maxBuffer length exceeded” errors. Claude implemented chunked processing to handle large datasets.

Annotation Grouping Issues

The plugin kept breaking up annotations that should have been together and duplicating others. This took six rounds of debugging, with Claude repeatedly missing the fact that the Python version worked perfectly.

I finally resorted to extended thinking:

Think as hard as you can about the fact that the Python script is working, and the plugin is not and the only thing happening is SQLite queries and handling the results, so this should be fucking simple.

The Development Timeline

Hour 1-2: Python script update, initial plugin creation, and basic functionality
Hour 3: Fighting with ePub metadata extraction
Hour 4-6: Real-world testing, debugging on large dataset, and configuration tweaking

Total development time: About 6 hours across multiple sessions, with Claude handling the bulk of the coding.

What I Learned This Time

The Good

Rapid prototyping: Claude excels at converting working logic between languages most times, but not understanding that it could use the same SQLite queries in both JavaScript and Python was a pain.
Comprehensive solutions: Often suggests improvements you haven’t considered
Pattern recognition: Great at handling complex parsing tasks like CFI strings
Documentation: Automatically generates thorough README files and comments

The Challenging

Context switching: Moving between chat sessions sometimes loses important context
Overconfidence: Claims solutions are “bulletproof” right before they break
Debugging persistence: Sometimes fixates on wrong solutions instead of reverting to working code
Name corrections: Consistently “corrected” my name spelling throughout the project

Tips for Successful Claude Desktop Coding

Use projects for better context persistence
Be specific about what’s working vs. broken
Maintain working versions before attempting fixes
Test incrementally rather than making multiple changes at once
Provide concrete examples when debugging

The Results

I am happy with the results. There is still some more work to do:

Not sure what is happening with the covers. The Python version had much more luck finding the covers, but then it used a more extensive library to do it.
The chapter names are still in gibberish in general. I think is because it is just using the value in the CFI location string. I am guessing I have to look this up somehow in the ePub file.
There seems to be a configuration value for adding the annotation date to each entry, but I haven’t seen one. Not sure what is going on there.

But for now I can use it and when I fix it, just have it overwrite all the old files. Then, once I am happy with it, store a hash of the file in properties or something like that and have overwriting work more intelligently.

How to Install the Apple Books Highlight and Note Importer

I do plan on releasing this as a community plugin. I just want to use it a while to see if there are any more bugs I want to fix or features I want to add.

But for now the simplest way to install and test this plugin is with BRAT (Beta Reviewer’s Auto-update Tool):

Install BRAT from the Community Plugins store in Obsidian
Open BRAT settings (Settings → BRAT)
Click “Add Beta plugin”
Enter this repository URL : https://github.com/eristoddle/obsidian-apple-books-import
Click “Add Plugin” - Choose the latest version and BRAT will automatically install and enable it
Auto-updates : BRAT will automatically update the plugin when new versions are released if you choose the latest version from the dropdown.

What’s Next

I took a break from a more complex project for a day to see if I could finish a smaller one. And even though there is still more work to do here, I think it was a success and an excellent test. I will continue to explore new ways of making the “vibe coding” process go smoother. In the couple of days that it took me to write this, I found even more tools to help.

Because I have no end of software ideas I have been collecting, but never got to, because I never had the time. So I will continue to work on my main vibe-coding project while taking a break to test new ways of doing this on smaller projects. My next post should be about the next set of things I learned developing an Electron app with Claude Code. I already have notes on it, but wanted to try this first.

Building an Electron App from Scratch with Claude Code

Stephan Miller — Tue, 20 May 2025 12:00:00 +0000

I have been curious about “vibe coding” ever since I heard about the process, even though I hate the name. I started by using Claude Desktop with MCP tools to build a code project from scratch and it felt like magic. That was until I discovered Claude Code and my workflow got so much easier and less chaotic right away.

This post documents my adventure so far, using Claude Code to build an Electron writing app. I made a few mistakes and am still making them, but corrected them and think I am on a better path now. I doubt it’s the right path, but I’ll keep tweaking it until it works for me.

Vibe Coding with Claude Desktop and MCP
Features of Claude Code that Made Me Switch
Installing and Using Claude Code
- Requirements
- Installing and Initializing
Building a Project from Scratch with Claude Code
The Pain of Refactoring with Claude Code
My Current CLAUDE.md file and Coding Process
- How to Use This File
- Project Overview
- Key Requirements
- Important References
- Code Style Guidelines
- Project SDLC
- Build/Lint/Test Commands
- Development Status
- Next Development Steps
- QA Checklist
Lessons I Learned Using Claude Code
Features of Claude Code I Haven’t Tried Yet
Claude Code Tips From Others I Plan on Trying
What Claude Code Built: Screenshots of My app
- Dashboard
- Settings
- Writing Interface
- Character Relationship Graph
My Journey with Claude Code Continues
- What I’ve Learned
- What’s Next

Vibe Coding with Claude Desktop and MCP

When I was using Claude Desktop to write code, I always felt like I was using workarounds to plan and organize. I started out with a standalone chat, but eventually got cut off and had to start a new chat to continue working on the app. So I switched to using a Claude Project.

Working on the app in a Claude Project made things a little easier by allowing me to upload 50 reference documents, but that was a mess, because the project was always changing and I had to delete and re-upload changes to documentation all the time. I could have also tried connected to the project’s Github repo or simply putting the documentation in a folder in the local project.

But around that time, I ran into Claude Code, read up on it, and figured it would work better for what I needed, so I started a new project with it. I might go back to the first project with a better plan, because I was making progress.

Features of Claude Code that Made Me Switch

I am using to a command-line interface. I use it them all the time. It feels like I am doing work. In chat, I feel like I am getting nagged by a sales bot on an e-commerce site or arguing with customer service. I get it makes no sense, but the command-line interface was a big feature for me.

Another significant feature is the CLAUDE.md file , which Claude Code which holds your project memory. You can add things to it, like:

Build/Lint/Test Commands
Code Style Guidelines
The SDLC process you want to use
Reference other documentation on the project

This means I can keep its memory in the project, instead of project files. When I first started working with Claude, I started setting up MCP connections like I did with Desktop. Gut then realized I was wasting my time and adding overhead, because the functionality I was getting by adding MCPs to Claude Desktop was already built into Code.

I could see adding a database connection MCP in the future if I work on a project that has one, but right now I am not using any with Claude Code. Claude Code has a lot more features than the three I listed here and I will get to them later, but these were all the reasons I needed to make the switch.

Installing and Using Claude Code

I have to admit that I basically ran the code to install Claude Code, ran the init command in a new project folder, and starting playing with it. Writing this, I figured I should dig deeper into its documentation to make sure I missed nothing. And I missed a few things that might have made this entire process smoother.

Requirements

Operating Systems : macOS 10.15+, Ubuntu 20.04+/Debian 10+, or Windows via WSL
Hardware : 4GB RAM minimum
Software :
- Node.js 18+
- git 2.23+ (optional)
- GitHub or GitLab CLI for PR workflows (optional)
- ripgrep (rg) for enhanced file search (optional)

One thing I missed there was installing ripgrep for advanced search. Maybe that will make things quicker? I did not install it until I started writing this, so maybe that is why things seem to take longer than they needed.

Installing and Initializing

To get started, run this command to install Claude Code globally:

npm install -g @anthropic-ai/claude-code

Then navigate to your project’s folder and run:

claude

After Claude Code started, I ran the following command to generate the CLAUDE.md file:

/init

Building a Project from Scratch with Claude Code

I wasn’t sure I could do this because all the examples online were working with existing codebases and I wanted to start one from scratch. So I asked Claude, and it said I could.

And I decided I would build an Electron app. A writing app. Yeah, I know. Like we need another one of those. But it was something I had always wanted to do and have been collecting a list of features over the years, which only made the odds I would ever get to it worse.

And, of course, you can’t make a writing app these days without including AI, so I am using AI to write an AI writing app. And with all the middlemen in the AI world, I figured I would make it so you just enter your LLM API keys and get started. The app would be completely self-contained in an Electron desktop app. For now, at least.

But if I were going to start over, I would start with more of a plan. I basically pointed Claude Code to the features of other similar apps and told it I wanted to build an Electron app with similar features.

As I chatted with Claude Code and worked on the app for the first couple of hours, I just let it update CLAUDE.md with the project details.

The Pain of Refactoring with Claude Code

I was learning my mistakes as I went. The one I learned quickly was to not only plan features, but plan the architecture that the features will work in. Think ahead. By the time I had an app where some features were working enough to be useful, I decided I should have used a plugin architecture.

I realized this architecture would work better for what I was doing when I was adding a context menu for the editor. When you right click in an editor, there are many things you can have it do, but when you add a new action, you are basically just adding another item to that menu that runs a new function when it is clicked.

Now I also think this type of architecture could be useful when using Claude Code to create an app like this. Have it build the core system and add a well-defined interface for plugins to interact with the core. Then build all the features as plugins. I think that this would keep the context lighter because Claude could use the plugin API docs to build the features instead of searching and parsing everything all the time.

But moving toward an architecture like that later in the game is hard. My first attempt to globally change things in the project was successful, but it took quite a while. And I still have a way to go. So, currently I am looking through the code base to see what all has to change for the complete refactor to happen. That way, when I tell it to refactor, I can give it some guidance and know what to expect when it does it right.

My Current `CLAUDE.md` file and Coding Process

For the first few hours, I just told Claude Code to build new features and keep track of where it was at in the CLAUDE.md file. But after a while, I knew I had to rein it in. Here are the sections I have in my CLAUDE.md file currently, which has changed often and will continue to:

How to Use This File

When I decided to refactor the CLAUDE.md, I had some ideas, but also asked Claude Code what would work. It suggested this section. Here’s what it has in it:

This file contains essential project information

Detailed documentation can be found in referenced files

Update both this file and referenced files when making changes

Project Overview

EmberText is an AI-powered desktop writing application built with Electron, React, and TypeScript. It combines features from TypingMind, NovelCrafter, SudoWrite, and Scrivener.

Key Requirements

Projects must be stored as markdown files in a folder structure (similar to Obsidian)

Each chapter should be saved as an individual markdown file

Project metadata and structure should be stored in a JSON file within the project directory

The app should be able to open and edit these files directly, enabling compatibility with other markdown editors

Help a writer complete a project with AI or other tools from idea to ebook publishing

Important References

I had everything in my CLAUDE.md file to begin with, but it quickly got huge. So I thought about how to trim it down. I also asked Claude Code what should be in the file versus what it could access on an as-needed basis. There is the concept of imports, but I didn’t want to use that because I would essentially do the same thing, but putting this massive amount of information in multiple files that still got read every time I ran Claude Code.

So I created a project knowledge base folder and put everything that it wouldn’t need every time there and add an “important references” section in CLAUDE.md. Here’s what I have in this section:

Code Project Structure: knowledge_base/project/structure.md

Writing Project Structure: knowledge_base/project/directory-structure.md

Key Features: knowledge_base/project/key-features.md

Potential Feature Ideas: knowledge_base/feature-ideas

Features Ready for Development or Already Developed: knowledge_base/features

Code Style Guidelines

Formatting : Use 2-space indentation, single quotes, no semicolons

Imports : Group imports (node builtins, external, internal)

Types : Use TypeScript with explicit return types on functions

Naming : camelCase for variables/functions, PascalCase for classes/components

Components : One component per file, match filename to component name

Error Handling : Use try/catch blocks with specific error types

Functions : Prefer pure functions, avoid side effects

Testing : Write unit tests for all new functionality

Project SDLC

As I was working on the project, I decided to “formalize” the development process, so that Claude Code knew what steps I wanted it to take. Here is what I have in this section:

New features will be picked from “Key Features” in knowledge_base/project/key-features.md when “Next development steps” is empty. Once those are done, they will be sourced from knowledge_base/feature-ideas.

I will ask you to think about the features and you will create notes about the feature in a markdown file named after it in knowledge_base/features. This file will be used as reference when we update a feature or fix bugs. The file will be kept updated with the feature changes.

You will then reference the file next to the feature in “Next development steps” like this: “Add workshop chat: knowledge_base/features/AddWorkShopChat.md”

I will ask you to do the next task in “Next development steps”. You will work on it and tell me when it is done, asking any necessary questions along the way.

When that is done, you will move the feature from “Next development steps” to “Current progress” and make sure to move the reference to the note in knowledge_base/features to it.

You will also update “Key Features” in knowledge_base/project/key-features.md if anything changed there.

Update the project structure in knowledge_base/project/structure.md and writing project structure in knowledge_base/project/directory-structure.md when necessary.

You will add testing the feature to “QA Checklist”

Build/Lint/Test Commands

Install: npm install

Start: npm start (builds main and renderer processes and starts Electron)

Dev: npm run dev (development mode with live reload)

Build Main: npm run build:main (TypeScript compilation for main process)

Build Renderer: npm run build:renderer (Webpack bundling for renderer)

Test: npm test

Lint: npm run lint

Format: npm run format

Development Status

This is a list of features it has already implemented. Just a simple description of the feature and the path to the feature plan file. More details on the feature plan file below.

Next Development Steps

This is a list of features I told it to put here. I usually have 3-4 features in this list. When I tell it to add a feature, I have it “think hard” about how to implement it, create a file in my project knowledge base folder that details how the feature will be implemented, and add it to this list like this:

Implement scene scaffolding system: knowledge_base/features/SceneScaffolding.md

That gives me a chance to look it over before it implements it. I am also hoping it follows my instructions and only references those files when dealing with that specific feature to keep unnecessary information out of the context. The files it creates are pretty extensive and have the following headings:

Overview
Key Requirements
User Interface Components
Data Structures
Components to Create
AI Integration
User Experience Flow
Integration with Existing Features
Implementation Plan
Dependency Notes
Testing Requirements
Future Enhancements

QA Checklist

Claude Code will add items to this list when it has finished a feature. It’s actually a reminder for me to test new features, so it probably doesn’t need to be in the file. For now, though, it keeps me from forgetting to test things and it doesn’t take that much space.

Lessons I Learned Using Claude Code

I learned most of these lessons after making many mistakes:

Plan well: This is a lesson I keep learning over and over. But I still jumped right into this project without much of a plan. All I had was Electron and a list of features. I should have thought more about architecture at the beginning instead of refactoring a few hours in.
Stop and look at the project holistically often: Especially if you don’t plan well. I knew I wanted AI to help write a book at every step, from idea to ebook, but I sort of just let it do what it wanted. And by the time I realized the AI functionality could be a modal used in multiple places, it put AI forms in three or four different places.
Tell Claude Code to “think harder” when you are planning feature: This is the third lesson that involves “planning.” Notice a pattern. When you tell Claude to “think”, it triggers Claude’s extending thinking and when you tell it to “think harder”, “think a lot”, or “think more”, it triggers deeper thinking.
Do your own git commits: I didn’t think that Claude Code would commit code without explicitly telling it to do so, but now and then it would. And each time, it was when I wanted to be done for the night and, of course, the code was broken. So I spent more time having it fix the code, so that I could commit working code. I could have reverted, but it had just added a new feature, which I knew was at least partially working.
QA and review the UI often: Claude will forget things. It will think hard about a feature, create a plan, work on it, tell you it is done, and forget 20% of the features. It was really hard for me to stop adding shiny new things to the app to do mundane testing, but I eventually had to backtrack through a lot of things and fix bugs.

Features of Claude Code I Haven’t Tried Yet

I jumped right into this project without reading 95% of the docs. But I figured I should at least peruse them before I wrote this. Here are some things I ran into, not covered by this article, you might want to check out:

Free Claude Code with Claude Max: I may or may not try this in the future. If you pay for the $100/month Claude Max subscription, Claude Code comes along with it. In my estimation, when I am working with Claude Code, it costs me around $5 an hour and with my limited time, I have only used about $50/month so far with the API. Maybe with ripgrep installed, the process will go faster and cost me more per hour.
Use Claude Code as an MCP server: Thought this was cool, but then wasn’t sure what I would use it for.
Create custom slash commands: You can create a .claude/commands folder in your project to hold commands you use over and over. You can also design these commands to accept arguments.

Claude Code Tips From Others I Plan on Trying

I work on projects like this and write articles about them in the evenings when I don’t have freelance writing work and in the last two weeks I haven’t been able to touch this project. But I still looked for tips whenever I could and took notes on them. Here are some things I plan on trying in the future:

Manage compacting more effectively: I noticed that Claude Code mentioning something about compacting context and providing a percentage. Now, with further research, I’ve learned that Claude Code will compress the conversation history to stay within the context window’s limits. I found this post on Reddit about always doing compacting manually. For example, before you are about to start on a new feature. Because, I guess, auto compacting can really screw you if it happens at the wrong time.
Leverage other LLMs and repomix or code2prompt: The refactoring process was a pain. Fixing bugs was a pain. Redditor squareboxrox mentioned using these tools to upload your whole project to another LLM when you run into the inevitable bug fix rabbit hole, instead of saying “still not fixed” over and over. Not every bug is like this. Maybe 20%. But I figured I’d give this method a try

What Claude Code Built: Screenshots of My app

Disclaimer: These screenshots showcase the functional aspects of EmberText rather than polished design. As this project focused on testing Claude Code’s capabilities for rapidly building functional features, I prioritized getting core functionality working over UI refinement. Consider these a “developer preview” of what’s possible with AI-assisted coding.

Dashboard

The home screen showing multiple writing projects with their structure and metadata. Claude Code implemented the card-based layout and project metadata tracking.

Settings

Settings panel for connecting to various AI models. This shows how Claude Code implemented form validation and API connection management.

Writing Interface

The main editor with project structure sidebar and formatting tools. Note the line-numbered editor and structure navigation that Claude Code implemented based on my requirements.

Character Relationship Graph

An interactive visualization showing connections between characters. This feature shows Claude Code’s ability to integrate complex visualizations using React.

I’m impressed by how quickly Claude Code could assemble a functional writing application with complex features like relationship graphs and structured document editing. While the UI would benefit from refinement, the core functionality is there and working as intended after just 16 hours of development time.

My Journey with Claude Code Continues

After spending about 16 hours and $80 in API costs, I’ve built a writing application with some functionality. My app currently:

Connects to Claude and Open API
Generates text and dialogue
Creates outline and plot structures
Has a plot timeline
Has a character relationship graph
Adds location, character, and item context to API calls

The refactoring process I described earlier was painful enough that I have temporarily paused development to reorganize my project memory files and carefully plan upcoming features. It has also reinforced the biggest lesson I learned: planning saves significant time and frustration with AI-assisted development.

What I’ve Learned

The most valuable lesson from this experiment is that Claude Code does well when given clear direction and architecture upfront. While it can adapt on the fly, major architectural shifts are still painful, much like traditional development, just faster.

My approach now combines the benefits of AI coding with disciplined software design: I plan thoroughly, create detailed feature specifications, and only then engage Claude Code to implement them.

What’s Next

Complete my writing app : Once I’ve implemented the remaining AI helpers, I’ll test the application with a real book project. Since I built this tool primarily for myself, this test will be the accurate measure of success.
Explore smaller projects : The development speed has inspired me to tackle several smaller ideas in parallel. With proper planning, I could potentially complete 2-3 modest apps in the same time it would typically take to build one.

If you’re interested in trying Claude Code yourself, I recommend starting with something small and well-defined. The initial time investment in proper planning and documentation pays dividends in development speed and reduces frustrating rework.

I’ll be documenting more of this journey as I continue exploring AI-assisted development.

Forem: Stephan Miller

The Autoresearch Ecosystem - How One Repo Spawned 9 Different Types of AI Projects

What Karpathy Actually Built

1. Platform Ports: Running It On Hardware You Actually Own

GPU Cluster Scaling

2. ML Research Enhancers: Making the Loop Smarter

Memory-Enhanced Researchers

Bayesian + Active Inference

Multi-GPU Infrastructure

3. Prompt Optimizers: Same Loop, Different Target File

autoresearch-prompt-optimization (az9713)

autoresearch-for-agents (Galileo)

4. Generalized Frameworks: Autoresearch For Anything

uditgoenka/autoresearch — Claude Code Skill

autoresearch-anything (zkarimi22)

menonpg/autoloop — The pip Package

krzysztofdudek/ResearcherSkill — One File, Full Discipline

alfonsograziano/auto-agent — Autoresearch Builds Agents

5. Production Codebase Optimization: Autoresearch on Real OSS

More Production War Stories

idealo Search Ranking

Tennis XGBoost — The Reward Hacking Cautionary Tale

Vesuvius Challenge Ink Detection

6. Agent Factory: Autoresearch Builds Agents

7. Research OS / Skills Systems: Institutionalizing the Pattern

PhD-Zero (TenureAI)

alirezarezvani/claude-skills

8. Creative Writing: Autoresearch For Prose and Fiction

redpen — Prose Refinement Engine

NousResearch/autonovel — Complete Novel Pipeline

sinfiny/Auto-Creative-Reasoning

CalvinMagezi/self-evolving-skill — Brand Document Evolution

9. Meta-Pattern: Wrapping Autoresearch as a Worker

The Problem with Solo Autoresearch

The Fix: 3 Files, 4 Subagents

What Actually Broke In Production

So What Does This Actually Mean?

Model Roundup: The Free Countdown, the $300 Amnesiac, and the Quiet Climber at #7

Table of Contents

The #1 Model Is a Countdown Timer (Tencent Hy3 Preview)

Kimi K2.6 Is Now a Real Recommendation

Where K2.6 Falls Short

The Sleeper: Step 3.5 Flash Has Been Climbing for Three Months

The One Real Catch

Grok 4.3: Genuinely Impressive, Genuinely Annoying, $300/Month

Your Smarter Model Might Be Breaking Your Agents

What’s Actually Worth Using (and What’s Coming)

My AI Agent Kept Making Shit Up (And Other Lessons From Running OpenClaw)

Table of Contents

The Setup: OpenClaw + DeepSeek in Docker

The Exec Approval Maze

The Reports That Were Too Good

Going Around the Agent

When Tools Become Literal Text

Ripping Out Slack

What’s Actually Working

But Here’s What She’s Actually Good At

The Report Engine Isn’t a One-Trick Pony

Email Delivery, Old School On Purpose

Multi-Model, Not Locked to DeepSeek

The Track Record, Three Days In

What I Actually Built

April 2026 Model Roundup: Opus 4.7 Official, DeepSeek V4 Open-Sources 1M Context, and GPT-5.5 Upstaged the GPT-6 Hype

Table of Contents

When Google Billed 114x

What Actually Moved This Week

Claude Opus 4.7 Is Official — and the Cost Story Is Better Than Expected

Hype Check: Mimo V2 Pro, One Month In

Kimi K2.6: The Open-Source Agentic Coding Model Nobody Covered

Meta Broke Open Source Hearts

The Models That Cost Almost Nothing (No, Really)

The Hidden Tax: How Sonnet 4.6 Can Still Cost More Than Opus

GPT-5.5 Shipped Yesterday, Not GPT-6, and DeepSeek V4 Dropped Today

The Actual Takeaways

Read for Yourself

Microsoft APM - Managing AI Context Like a Dependency Problem

Table of Contents

The Blueprint: APM’s Declarative Infrastructure

Organizing the Monorepo

Solving Context Pollution with Intelligent Compilation