Forem: Nora Alalou

most AI-generated tests are worse than no tests

Nora Alalou — Sun, 29 Mar 2026 17:17:33 +0000

i started having claude write tests for my project and quickly realized something: most of them were useless. they passed, but they didn't test anything and i had a major sense of false security seeing 12/12 tests pass, etc. over and over.

a test that asserts expect(result).toBeDefined() after calling a function is technically a passing test. it will never fail unless the function throws.

this was like 80% of what i was getting. tests that exercised code paths without actually checking that the code did the right thing so I had great code coverage but stuff was still breaking constantly.

so i started thinking about what makes a test actually worth having, and i ended up with a set of gates that changed how i think about it.

the mutation test is the most important one. take a passing test, go flip a condition or change a return value in the source code, and run the test again. if it still passes, then the test is garbage because it's not actually sensitive to the behavior it claims to test. this is the thing that catches toBeDefined and toBeTruthy and all the other assertions that look like tests but aren't.

the other gates i landed on:

no weak assertions. toBeDefined, toBeTruthy, toBeInstanceOf — these get rejected automatically. if your assertion would pass on literally any non-null value, it's not an assertion.
isolation. the test has to pass in isolation, not just when the full suite runs. tests that accidentally depend on state from other tests are a time bomb.
coverage delta. the test has to actually cover new lines. a test that just re-covers stuff that's already tested is noise.
full suite compatibility. doesn't break anything else.

i eventually built a skill for claude code that automates this whole loop. it finds untested files, generates tests, runs them through all five gates, retries up to 3 times with feedback if they fail, and commits the ones that pass. it runs in a background tmux session so i can work on other stuff while it churns through the backlog.

the point is that "AI can write tests" is true in the same way that "AI can write code" is true. the output looks correct and compiles and runs. the question is whether it actually works. and for tests specifically, "works" means "fails when the code is wrong." most AI-generated tests don't clear that bar.

if you're using AI to write tests, add a mutation step. flip something in the source, run the test. if it still passes, delete it. that one check filters out more garbage than everything else combined.

markdown edited in ginsberg.ai

i was burning a ton of tokens on silly stuff

Nora Alalou — Sun, 29 Mar 2026 15:23:03 +0000

has anyone else noticed they're chewing through claude tokens way faster than they should be? anthropic just announced they're tightening the 5-hour session limits during peak hours and it made me actually look at where my tokens were going.

turns out most of it was waste. claude was reading files it had no reason to read like lock files, build artifacts, node_modules, coverage reports, media files. every time it explored the codebase it was burning tokens on stuff that would never help it write better code.

i added a .claudeignore file:

# Dependencies
node_modules/
.pnp.*

# Build artifacts
.next/
out/
build/
dist/

# Lock files (huge, no value to read)
package-lock.json
pnpm-lock.yaml
yarn.lock

# Minified bundles
*.min.js
*.min.css

# Generated code
next-env.d.ts
*.tsbuildinfo

# Caches
.cache/
__pycache__/
coverage/

# Environment / secrets
.env*
.vercel/

# Large non-code files
*.gif
*.mov
*.mp4
*.png
*.jpg

it works like .gitignore but for claude's file exploration. claude won't read or search anything that matches.

the other thing that helps is keeping CLAUDE.md lean. mine is about 145 lines with architecture essentials, key conventions, common gotchas. not a novel. every line of CLAUDE.md gets loaded into context at the start of every conversation, so bloat there costs you tokens on every single interaction.

i'm still seeing how much this helps. they're obviously just clamping down.

if you're on the Max plan and it still feels like you're burning through it, check what claude is actually reading. you might be feeding it your pnpm-lock.yaml on every exploration.

markdown edited in ginsberg.ai

fixing two bugs stacked on top of each other in ProseMirror

Nora Alalou — Sun, 29 Mar 2026 00:38:49 +0000

i'm building a markdown editor built on Milkdown (which wraps ProseMirror). i hit a bug in how marks behave when you join blocks together.

the bug

type a bold heading. move your cursor to the start of it. press Backspace. the heading joins into the paragraph above, but header just becomes unstyled. it doesn't remove the space before it as you'd expect.

i assumed this was a bug in joinTextblockBackward. it wasn't.

what's actually going on

when you press Backspace at position 0 of a textblock, joinTextblockBackward fires. under the hood, joinTextblocksAround calls:

replaceStep(state.doc, beforePos, afterPos, Slice.empty)

this deletes the boundary between the two blocks. the Fitter algorithm stitches the content of the second block into the first. it doesn't touch inline marks. the text nodes transfer as-is, bold and all.

there's a clearIncompatible function that strips marks disallowed by the target node type, but paragraphs allow bold, so nothing gets stripped. there's also splitBlockKeepMarks which handles the reverse operation (Enter), but that's about storedMarks — cursor behavior for the next typed character. different problem.

so the marks survive the join, and they're supposed to. consider two paragraphs:

She said the results were
**statistically significant** and could not be ignored.

press Backspace at the start of the second paragraph. they merge into one. the bold must survive because you put it there intentionally. if ProseMirror stripped marks on join, it would destroy user content.

the heading case feels wrong for a different reason. in a markdown editor, # **Bold Heading** has explicit bold marks on the text nodes because that's what the markdown source says. you see the boldness as part of the heading, like a visual property of the block. but ProseMirror sees it as an inline mark that happens to sit inside a heading node. the markdown editor stores the heading's visual weight in two places: the node type and the inline marks. when the heading unwraps into a paragraph, only the node type changes. the marks stay because ProseMirror doesn't know they were "heading-ness" and not "the user explicitly wanted bold."

not a ProseMirror bug. a modeling problem specific to markdown editors.

the fix

i intercept joinTextblockBackward's dispatch. before the transaction reaches the editor state, i snapshot the heading content range, then append removeMark calls for the region that used to be the heading. i also clear storedMarks so the cursor doesn't inherit marks either. everything lands in one transaction, so undo reverses it atomically.

here's the standalone plugin:

import { Plugin, PluginKey } from "prosemirror-state";
import { joinTextblockBackward } from "prosemirror-commands";

const headingBackspacePlugin = new Plugin({
  key: new PluginKey("heading-backspace"),
  props: {
    handleKeyDown(view, event) {
      if (event.key !== "Backspace") return false;

      const { state, dispatch } = view;
      const headingType = state.schema.nodes.heading;
      if (!headingType) return false;

      const { $from, empty } = state.selection;
      if (
        !empty ||
        $from.parentOffset !== 0 ||
        $from.node().type !== headingType
      )
        return false;

      // Snapshot before the join mutates anything.
      const headingContentSize = $from.node().content.size;
      const prevEnd = $from.before();

      const wrappedDispatch = (tr: typeof state.tr) => {
        // Map prevEnd through the join's ReplaceStep to find where
        // the heading content now sits inside the merged paragraph.
        const from = tr.mapping.map(prevEnd);
        const to = from + headingContentSize;

        // Strip every mark type from the former-heading range.
        for (const markType of Object.values(state.schema.marks)) {
          tr.removeMark(from, to, markType);
        }

        // Clear storedMarks so the cursor doesn't inherit marks.
        tr.setStoredMarks([]);

        dispatch(tr);
      };

      return joinTextblockBackward(state, wrappedDispatch, view);
    },
  },
});

$from.before() gives the position just before the heading's opening token — after the join's ReplaceStep deletes the block boundary, tr.mapping.map(prevEnd) gives the new position where the heading content starts inside the merged paragraph. headingContentSize is stable because the join doesn't change the heading's content, just moves it. tr.removeMark is safe to call multiple times on the same transaction and tr.setStoredMarks([]) clears cursor marks so typing after the join doesn't inherit bold/italic. the whole thing is one transaction — Ctrl+Z undoes the join and the mark removal together.

if you're using Milkdown or Tiptap, register this through the framework's plugin API. the ProseMirror logic is identical.