Forem: Glen Baker

Debtmap Re-adds JavaScript and TypeScript Support

Glen Baker — Tue, 17 Feb 2026 23:05:38 +0000

Originally published on Entropic Drift

Debtmap v0.15.0 adds JavaScript and TypeScript support using tree-sitter.

Some history: debtmap originally supported multiple languages, but I removed JS/TS in v0.7.0 to focus on Rust. The multi-language implementation was half-baked and the maintenance burden wasn't worth it. After getting the Rust analyzer right (entropy dampening, purity analysis, call graph integration), I rebuilt JS/TS support properly in v0.14.0-v0.15.0.

What It Detects

Complexity metrics - cyclomatic, cognitive, and nesting depth. Cognitive complexity adds nesting penalties because if inside for inside if is harder to read than the same number of branches at the top level.

Async patterns - callback nesting depth, promise chain length, missing .catch() handlers. These create complexity that traditional metrics miss.

TypeScript-specific - excessive any usage (>5 per file), type assertion overuse (>3), non-null assertion overuse (!).

Functional chains - detects map/filter/reduce pipelines and flags side effects in them (console, mutations, I/O).

Entropy Dampening

v0.15.0 adds entropy-based complexity dampening for JS/TS. This reduces false positives for repetitive code.

Consider validation functions that check multiple fields with similar logic:

function validateConfig(config: Config): string[] {
  const errors: string[] = [];

  if (config.width !== undefined) {
    if (typeof config.width !== 'number') errors.push('width must be number');
    else if (config.width < 0) errors.push('width must be positive');
    else if (config.width > 10000) errors.push('width exceeds max');
  }

  if (config.height !== undefined) {
    if (typeof config.height !== 'number') errors.push('height must be number');
    else if (config.height < 0) errors.push('height must be positive');
    else if (config.height > 10000) errors.push('height exceeds max');
  }

  return errors;
}

Raw cyclomatic complexity: 10. But this is repetitive validation with low token entropy - the same pattern repeated for different fields. Debtmap detects this and dampens the score.

The better fix is extracting validation functions, but the tool shouldn't flag every validation function as high-priority debt.

Usage

# Install
cargo install debtmap

# Analyze (opens interactive TUI)
debtmap analyze .

# With coverage data and git context
debtmap analyze . --lcov coverage/lcov.info --context

# Export for CI or LLM consumption
debtmap analyze . --format markdown --top 10 --lcov coverage/lcov.info --context

JS/TS files are detected automatically by extension (.js, .ts, .jsx, .tsx, .mjs, .cjs, .mts, .cts).

What It Doesn't Do

Debtmap identifies where debt is and how severe it is. It doesn't:

Replace ESLint/TSLint (style enforcement)
Replace the TypeScript compiler (type checking)
Generate coverage data (use Jest, Vitest, etc.)
Tell you how to fix things (that's for you or your AI assistant)

Refined Types in Rust: Parse, Don't Validate

Glen Baker — Sun, 28 Dec 2025 00:13:12 +0000

Originally published on Entropic Drift

The String Problem

How many times have you seen function signatures like this?

fn send_email(to: String, subject: String, body: String) -> Result<(), Error>;

fn create_user(email: String, username: String, password: String) -> Result<User, Error>;

fn transfer_money(from_iban: String, to_iban: String, amount: f64) -> Result<(), Error>;

Every String in these signatures is misleading. They don't accept any string—they accept specific formats. Pass "not-an-email" to send_email and it will fail. Pass "hello" as an IBAN and your money transfer crashes.

The signature promises one thing; the implementation demands another.

Validation Everywhere

The typical fix is validation at every entry point:

fn send_email(to: String, subject: String, body: String) -> Result<(), Error> {
    if !is_valid_email(&to) {
        return Err(Error::InvalidEmail(to));
    }
    // ... send email
}

fn send_welcome_email(email: String) -> Result<(), Error> {
    if !is_valid_email(&email) {
        return Err(Error::InvalidEmail(email));
    }
    // ... send welcome email
}

fn send_password_reset(email: String) -> Result<(), Error> {
    if !is_valid_email(&email) {
        return Err(Error::InvalidEmail(email));
    }
    // ... send reset email
}

Three functions, three validation checks, same validation logic copy-pasted. And what happens when you add a fourth function? Fifth? What happens when the validation logic needs to change?

You might say: "Just validate at the API boundary." But then internal code passes raw strings around, and you're trusting that somewhere upstream, someone validated. The compiler can't verify that trust.

Parse, Don't Validate

Validation returns a boolean, parsing returns a value.

When you validate, you check if data is correct and then continue using the same data. The knowledge that it's valid exists only in your head (and maybe in a comment).

When you parse, you transform unstructured data into structured data. The knowledge that it's valid is encoded in the type.

// Validation: returns bool, you keep the String
fn is_valid_email(s: &str) -> bool { ... }

// Parsing: returns Email, proof of validity in the type
fn parse_email(s: String) -> Result<Email, EmailError> { ... }

Once you have an Email, you know it's valid. The type proves it.

Refined Types

A refined type wraps a base type with a predicate that must be satisfied:

// Conceptually: Email = String where is_valid_email(s) == true
struct Email(String);

impl Email {
    pub fn new(s: String) -> Result<Email, EmailError> {
        if is_valid_email(&s) {
            Ok(Email(s))
        } else {
            Err(EmailError::InvalidFormat)
        }
    }

    pub fn value(&self) -> &str {
        &self.0
    }
}

The constructor is the only way to create an Email. It validates on construction. After that, you have proof of validity that the compiler enforces.

Now the function signatures accurately reflect their requirements:

fn send_email(to: Email, subject: String, body: String) -> Result<(), Error>;

fn create_user(email: Email, username: Username, password: Password) -> Result<User, Error>;

fn transfer_money(from: Iban, to: Iban, amount: Money) -> Result<(), Error>;

Try passing a raw String to send_email and the compiler stops you. The type system prevents invalid data from entering your domain.

The Practical Benefits

1. Validation Happens Once

// At the API boundary
let email = Email::new(request.email)?;

// Now pass Email through your entire system
// No re-validation needed anywhere
send_welcome_email(email.clone());
subscribe_to_newsletter(&email);
log_user_registration(&email);

Validate at the boundary, trust the type everywhere else.

2. Invalid States Become Unrepresentable

struct User {
    email: Email,      // Can never be invalid
    username: Username, // Can never be invalid
    age: Age,          // Can never be negative
}

You can't construct a User with an invalid email. The type system prevents it. No defensive checks needed inside your domain logic.

3. Function Signatures Document Requirements

// Before: What format does this expect?
fn process_phone(phone: String) -> Result<(), Error>;

// After: Self-documenting
fn process_phone(phone: PhoneNumber) -> Result<(), Error>;

The type is the documentation. No comments needed, no runtime surprises.

4. Refactoring Becomes Safer

Need to change email validation rules? Change the Email::new constructor. Every place that creates emails is affected. The compiler shows you everywhere an Email is used.

impl Email {
    pub fn new(s: String) -> Result<Email, EmailError> {
        // Add new rule: reject emails from blocked domains
        if BLOCKED_DOMAINS.contains(&domain_of(&s)) {
            return Err(EmailError::BlockedDomain);
        }
        // ... rest of validation
    }
}

All call sites that construct Email now enforce the new rule. No grep required.

Building Refined Types with Stillwater

Stillwater provides a Refined<T, P> type that makes building refined types ergonomic:

use stillwater::refined::{Predicate, Refined};

// Define the predicate
struct IsValidEmail;

impl Predicate<String> for IsValidEmail {
    fn test(value: &String) -> bool {
        value.contains('@') && value.len() >= 5
    }

    fn error_message() -> String {
        "invalid email format, expected local@domain".to_string()
    }
}

// The refined type
type Email = Refined<String, IsValidEmail>;

// Usage
let email = Email::new("user@example.com".to_string())?;
println!("{}", email.value());  // Access inner value

The Refined type handles the boilerplate: constructor validation, inner value access, error messages, Debug/Display implementations.

Domain-Specific Types with Stilltypes

Stilltypes provides production-ready refined types for common domains:

use stilltypes::prelude::*;

// Email (RFC 5321 compliant)
let email = Email::new("user@example.com".to_string())?;

// URL with scheme enforcement
let secure_url = SecureUrl::new("https://api.example.com".to_string())?;
let insecure = SecureUrl::new("http://api.example.com".to_string());
assert!(insecure.is_err());  // HTTP rejected

// Phone numbers (E.164)
let phone = PhoneNumber::new("+1 415 555 1234".to_string())?;
assert_eq!(phone.to_e164(), "+14155551234");

// Financial identifiers
let iban = Iban::new("DE89370400440532013000".to_string())?;
assert_eq!(iban.country_code(), "DE");

// Network types
let port = Port::new(443)?;
assert!(port.is_privileged());

These implementations are backed by RFC-compliant validation libraries. PhoneNumber uses Google's libphonenumber. Iban uses proper checksum validation. Email follows RFC 5321.

Composition: Multiple Constraints

Refined types compose. Need a URL that's both valid and uses HTTPS?

use stillwater::refined::{And, Predicate, Refined};

// Individual predicates
struct IsValidUrl;
struct IsHttps;

impl Predicate<String> for IsValidUrl { ... }
impl Predicate<String> for IsHttps { ... }

// Combined predicate
type SecureUrl = Refined<String, And<IsValidUrl, IsHttps>>;

The And combinator requires both predicates to pass. There's also Or and Not.

Serde Integration

With serde support, refined types validate during deserialization:

use stilltypes::prelude::*;
use serde::Deserialize;

#[derive(Deserialize)]
struct CreateUserRequest {
    email: Email,
    website: Option<SecureUrl>,
    phone: PhoneNumber,
}

// Invalid JSON fails to deserialize
let result: Result<CreateUserRequest, _> = serde_json::from_str(json);

Your API automatically rejects invalid data at the parsing layer. No validation code in your handler.

The Boundary Pattern

Where should you create refined types? At the boundary between untrusted and trusted code:

// API Handler - THE BOUNDARY
async fn create_user(Json(input): Json<RawUserInput>) -> Result<Json<User>, ApiError> {
    // Parse raw input into refined types
    let email = Email::new(input.email)
        .map_err(|e| ApiError::Validation(e.to_string()))?;
    let username = Username::new(input.username)
        .map_err(|e| ApiError::Validation(e.to_string()))?;

    // From here on, work with validated types
    let user = user_service.create(email, username).await?;

    Ok(Json(user))
}

// Service layer - TRUSTS THE TYPES
impl UserService {
    // No validation needed - types guarantee validity
    async fn create(&self, email: Email, username: Username) -> Result<User, ServiceError> {
        self.repo.insert(User { email, username }).await
    }
}

// Repository layer - TRUSTS THE TYPES
impl UserRepository {
    async fn insert(&self, user: User) -> Result<User, DbError> {
        // email and username are guaranteed valid
        sqlx::query("INSERT INTO users (email, username) VALUES ($1, $2)")
            .bind(user.email.value())
            .bind(user.username.value())
            .execute(&self.pool)
            .await?;
        Ok(user)
    }
}

Validation happens once at the HTTP handler. Service and repository layers receive pre-validated data. No defensive checks, no redundant validation, no possibility of invalid data sneaking through.

Error Accumulation

What about forms with multiple fields? You want all errors, not just the first. Combine refined types with Stillwater's Validation:

use stilltypes::prelude::*;
use stillwater::Validation;

fn validate_form(input: FormInput) -> Validation<ValidatedForm, Vec<String>> {
    Validation::all((
        Email::new(input.email).map_err(|e| vec![e.to_string()]),
        PhoneNumber::new(input.phone).map_err(|e| vec![e.to_string()]),
        SecureUrl::new(input.website).map_err(|e| vec![e.to_string()]),
    ))
    .map(|(email, phone, website)| ValidatedForm { email, phone, website })
}

// Returns ALL errors, not just the first
match validate_form(input) {
    Validation::Success(form) => process(form),
    Validation::Failure(errors) => {
        // errors: ["invalid email format", "invalid phone number", "URL must use HTTPS"]
    }
}

When Not to Use Refined Types

Refined types add indirection. Use them when the benefit outweighs the cost:

Use refined types for:

Data that crosses trust boundaries (API input, file parsing, user input)
Domain concepts with invariants (email, phone, money, coordinates)
Values passed through multiple layers of your application
Anything where invalid data would cause bugs

Skip refined types for:

Internal data structures with limited scope
Values validated by the database layer anyway
Simple scripts where the overhead isn't worth it
Prototyping (add them when the design stabilizes)

The Type-Driven Design Mindset

Refined types are part of a larger principle: make invalid states unrepresentable.

Instead of runtime checks scattered throughout code, encode constraints in types. The compiler becomes your validation engine. Invalid programs don't compile.

// Bad: Invalid state representable
struct Order {
    items: Vec<Item>,  // Could be empty!
    status: String,    // Could be anything!
}

// Good: Invalid state unrepresentable
struct Order {
    items: NonEmptyVec<Item>,  // Guaranteed non-empty
    status: OrderStatus,       // Enum with valid states
}

Combined with refined types, you get:

struct Order {
    id: OrderId,              // Validated format
    customer_email: Email,     // Validated email
    shipping: Address,         // Validated address
    items: NonEmptyVec<Item>,  // Non-empty guarantee
    total: Money,             // Non-negative amount
}

Every field carries its own proof of validity. The domain model itself rejects bad data.

Getting Started

Add stilltypes to your project:

cargo add stilltypes --features email,url,phone

Pick one stringly-typed field in your API. Replace it with a refined type.

// Before
fn create_user(email: String) -> Result<User, Error> {
    if !is_valid_email(&email) { return Err(...); }
    // ...
}

// After
fn create_user(email: Email) -> Result<User, Error> {
    // Email is already valid, just use it
    // ...
}

Parse at the boundary. Trust the types inside.

Summary

Approach	Validation Location	Compiler Help	Guarantees
Raw strings	Every function	None	None
Validation at boundary	Entry points	None	Trust-based
Refined types	Construction	Full	Type-enforced

Refined types move validation from scattered runtime checks to centralized construction. The type system carries proof of validity, eliminating entire categories of bugs.

Parse, don't validate. Let your types tell the truth.

Stilltypes provides production-ready refined types for Rust. Stillwater provides the underlying Refined<T, P> abstraction and validation utilities.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Stillwater 1.0: Pragmatic Effect Composition and Validation for Rust

Glen Baker — Wed, 24 Dec 2025 22:59:12 +0000

Originally published on Entropic Drift

I'm proud to announce the release of stillwater 1.0, a production-ready Rust library for pragmatic effect composition and validation.

What is Stillwater?

Stillwater implements the pure core, imperative shell pattern for Rust. The name is a mental model: like a still pond with streams flowing through it, your business logic stays calm and predictable at the center while effects (I/O, side effects) flow at the boundaries.

       Still Waters
      ╱            ╲
 Pure Logic      Effects
     ↓              ↓
  Unchanging     Flowing
 Predictable    Performing I/O
  Testable      At boundaries

This isn't Haskell-in-Rust. It's an attempt at better Rust, leveraging pragmatic functional patterns where they make code more testable and maintainable, while respecting Rust's ownership model and idioms.

What Are Effects?

If you're coming from OOP, "effects" might be unfamiliar. The concept is simple:

Typical Rust functions execute immediately:

fn add(a: i32, b: i32) -> i32 {
    a + b  // eager evaluation, happens NOW, returns result
}

An effect is a description of what to do, executed later:

fn fetch_user(id: UserId) -> impl Effect<Output = User, Error = DbError, Env = AppEnv> {
    asks(|env| env.db.get_user(id))  // describes the operation, deferred evaluation
}

// Nothing happens yet - we just have a description
let effect = fetch_user(42);

// NOW it executes
let user = effect.run(&env).await;

This separation lets you:

Compose descriptions before running them
Test logic without executing I/O
Control when and where side effects actually happen

If you've used async Rust, you already know this pattern - async fn returns a Future (a description), which runs when you .await it. Effects are the same idea, extended with environment access, typed errors, and composition tools.

The Core Problem

Most code mixes business logic with I/O:

fn process_user(id: UserId, db: &Database) -> Result<User, Error> {
    let user = db.fetch_user(id)?;           // I/O
    if user.age < 18 {                        // Logic
        return Err(Error::TooYoung);
    }
    let discount = if user.premium { 0.15 } else { 0.05 };  // Logic
    db.save_user(&user)?;                    // I/O
    Ok(user)
}

Testing this requires mocking the database. Reasoning about it requires mentally separating what transforms data from what performs I/O. Reusing the discount logic means extracting it anyway.

Stillwater separates these concerns by design, using multiple features working together:

use stillwater::prelude::*;
use stillwater::refined::{Refined, NonEmpty, InRange};
use stillwater::predicate::*;

// Refined types - validity proven at compile time
type Username = Refined<String, And<NonEmpty, MaxLength<20>>>;
type Age = Refined<u8, InRange<18, 120>>;
type Email = Refined<String, NonEmpty>;  // simplified
// See https://github.com/iepathos/stilltypes for full Email type implementation

// Pure validation - accumulates ALL errors at once
fn validate_registration(input: RawInput) -> Validation<ValidUser, Vec<String>> {
    Validation::all((
        Username::new(input.username)
            .map_err(|_| vec!["Username must be 1-20 characters".into()]),
        Age::new(input.age)
            .map_err(|_| vec!["Age must be 18-120".into()]),
        Email::new(input.email)
            .map_err(|_| vec!["Email required".into()]),
    ))
    .map(|(username, age, email)| ValidUser { username, age, email })
}

// Pure business logic - no I/O, easily testable
fn calculate_discount(user: &ValidUser) -> Discount {
    if user.age.get() >= 65 { Discount(0.20) }  // Senior discount
    else { Discount(0.05) }
}

// Effect composition - describes I/O, doesn't execute it
fn register_user(input: RawInput) -> impl Effect<Output = User, Error = AppError, Env = AppEnv> {
    // Validate (pure) -> transform to effect
    from_validation(validate_registration(input))
        .map_err(|errs| AppError::Validation(errs))
        // Apply pure business logic
        .map(|valid| {
            let discount = calculate_discount(&valid);
            (valid, discount)
        })
        // Describe the I/O - asks() doesn't execute, it builds a description
        .and_then(|(valid, discount)| {
            asks(move |env: &AppEnv| env.db.create_user(&valid, discount))
        })
        .context("registering new user")
}

// I/O executes at the application boundary
async fn handle_registration(input: RawInput, env: &AppEnv) -> Result<User, AppError> {
    register_user(input).run(env).await  // <-- I/O happens here
}

// In tests - pure functions need no mocks
#[test]
fn senior_gets_20_percent_discount() {
    let user = ValidUser {
        username: Username::new("alice".into()).unwrap(),
        age: Age::new(70).unwrap(),  // 70 years old
        email: Email::new("alice@example.com".into()).unwrap(),
    };
    assert_eq!(calculate_discount(&user), Discount(0.20));
}

The key insight: register_user returns an effect description, not a result. No I/O executes until .run(env) is called. This keeps your business logic pure and testable - the register_user function itself does no I/O, it just describes what I/O should happen.

This example showcases several stillwater features:

Refined types (Username, Age, Email) encode invariants in the type system - once constructed, they're guaranteed valid
Validation accumulates all errors - users see every problem at once, not one at a time
Pure functions (calculate_discount) are trivially testable with no mocks
Deferred I/O - asks() describes database operations; actual I/O happens only at .run()
Context chaining preserves error trails for debugging

Key Features in 1.0

Zero-Cost Effect System

Following the futures crate pattern, effects are zero-cost by default with opt-in boxing:

use stillwater::prelude::*;

// No heap allocation - each combinator returns a concrete type
let effect = pure::<_, String, ()>(42)
    .map(|x| x + 1)
    .and_then(|x| pure(x * 2));

// Type: AndThen<Map<Pure<i32, ...>, ...>, ...>
// Compiler inlines everything - zero allocation!

Use .boxed() only when you need type erasure - like match arms with different effect types:

// Different branches produce different concrete types
fn fetch_user(id: UserId, use_cache: bool) -> BoxedEffect<User, Error, Env> {
    if use_cache {
        asks(|env: &Env| env.cache.get(id))        // type A
            .boxed()
    } else {
        asks(|env: &Env| env.db.fetch(id))         // type B
            .and_then(|user| cache_user(user))
            .boxed()
    }
}

If you've worked with async Rust, you already understand the pattern.

Validation with Error Accumulation

Rust's Result short-circuits on the first error. Stillwater's Validation accumulates all of them:

use stillwater::Validation;

fn validate_registration(input: RawInput) -> Validation<User, Vec<String>> {
    Validation::all((
        validate_email(input.email),
        validate_age(input.age),
        validate_username(input.username),
    ))
    .map(|(email, age, username)| User { email, age, username })
}

// Returns ALL errors at once:
// Failure(["Invalid email format", "Age must be 18+", "Username too short"])

No more frustrating round-trips where users fix one error only to discover another.

Predicate Combinators

Composable, reusable predicates for declarative validation:

use stillwater::predicate::*;

let valid_username = len_between(3, 20)
    .and(all_chars(|c| c.is_alphanumeric() || c == '_'));

let valid_port = between(1024, 65535);

// Combine with validation
Validation::success(username)
    .ensure(valid_username, "Invalid username format")

Refined Types

The "parse, don't validate" pattern - type-level invariants that guarantee validity:

use stillwater::refined::{Refined, NonEmpty, Positive, InRange};

type NonEmptyString = Refined<String, NonEmpty>;
type PositiveI32 = Refined<i32, Positive>;
type Port = Refined<u16, InRange<1024, 65535>>;

// Validate at boundaries
let name = NonEmptyString::new("Alice".to_string())?;
let port = Port::new(8080)?;

// Use freely inside - validity guaranteed by construction
fn process_user(name: NonEmptyString, port: Port) {
    // No need to re-check: types prove validity
}

Zero runtime overhead - same memory layout as the inner type.

Bracket Pattern for Resource Management

Guaranteed acquire/use/release semantics:

use stillwater::effect::bracket::*;

let result = bracket(
    open_connection(),                          // Acquire
    |conn| async move { conn.close().await },   // Release (always runs!)
    |conn| fetch_user(conn, user_id),           // Use
).run(&env).await;

// Fluent builder for multiple resources
let result = acquiring(open_db(), |db| async move { db.close().await })
    .and(open_file(), |f| async move { f.close().await })
    .with_flat2(|db, file| process(db, file))
    .run(&env)
    .await;

Compile-Time Resource Tracking

Type-level resource tracking prevents leaks at compile time:

use stillwater::effect::resource::*;

fn open_file(path: &str) -> impl ResourceEffect<Acquires = Has<FileRes>> {
    pure(FileHandle::new(path)).acquires::<FileRes>()
}

fn close_file(handle: FileHandle) -> impl ResourceEffect<Releases = Has<FileRes>> {
    pure(()).releases::<FileRes>()
}

// Bracket guarantees resource neutrality - won't compile if unbalanced!
fn read_file_safe(path: &str) -> impl ResourceEffect<Acquires = Empty, Releases = Empty> {
    bracket::<FileRes>()
        .acquire(open_file(path))
        .release(|h| async move { close_file(h).run(&()).await })
        .use_fn(|h| read_contents(h))
}

Zero runtime overhead - all tracking happens at compile time.

Retry Policies as Data

Policies are composable, testable values - not scattered implementation:

use stillwater::{Effect, RetryPolicy};
use std::time::Duration;

// Define policy as pure data
let api_policy = RetryPolicy::exponential(Duration::from_millis(100))
    .with_max_retries(5)
    .with_max_delay(Duration::from_secs(2))
    .with_jitter(0.25);

// Test the policy without any I/O
assert_eq!(api_policy.delay_for_attempt(0), Some(Duration::from_millis(100)));
assert_eq!(api_policy.delay_for_attempt(1), Some(Duration::from_millis(200)));

// Reuse across effects
Effect::retry(|| fetch_user(id), api_policy.clone());
Effect::retry(|| save_order(order), api_policy.clone());

// Conditional retry
Effect::retry_if(
    || api_call(),
    api_policy,
    |err| matches!(err, ApiError::Timeout | ApiError::ServerError(_))
);

Writer Effect for Audit Trails

Accumulate logs without threading state through every function:

use stillwater::effect::writer::prelude::*;

fn process_order(order: Order) -> impl WriterEffect<
    Output = Receipt, Error = String, Env = (), Writes = Vec<String>
> {
    tell_one("Processing order".to_string())
        .and_then(move |_| validate_order(order))
        .tap_tell(|_| vec!["Validation passed".to_string()])
        .and_then(|order| charge_card(order))
        .tap_tell(|receipt| vec![format!("Charged: ${}", receipt.amount)])
}

let (result, logs) = process_order(order).run_writer(&()).await;
// logs: ["Processing order", "Validation passed", "Charged: $42.00"]

Parallel Effect Execution

Run independent effects concurrently:

use stillwater::prelude::*;

// Combine independent effects
fn load_profile(id: UserId) -> impl Effect<Output = Profile, Error = AppError, Env = AppEnv> {
    zip3(fetch_user(id), fetch_settings(id), fetch_preferences(id))
        .map(|(user, settings, prefs)| Profile { user, settings, prefs })
}

// Homogeneous collection
let results = par_all(effects, &env).await;

// Race - first success wins
let result = race(fetch_from_primary(), fetch_from_backup(), &env).await;

Production Ready

1.0 represents a stable API ready for production use:

355+ unit tests passing
113+ documentation tests
21 comprehensive runnable examples
Zero clippy warnings
Full async/await support
CI/CD with security audits

Philosophy

Stillwater is built on six core beliefs:

Pure Core, Imperative Shell - Separate logic from I/O for testability
Fail Completely, Not Partially - Accumulate all errors, not just the first
Errors Tell Stories - Context chaining preserves error trails
Composition Over Complexity - Build complex from simple pieces
Types Guide, Don't Restrict - Pragmatic type system use
Pragmatism Over Purity - Better Rust, not academic FP

We don't fight the borrow checker. We don't replace the standard library. We work with ?, integrate with async/await, and follow Rust idioms.

When to Use Stillwater

Good fit:

Complex form/config validation (error accumulation shines)
Business logic that needs extensive testing (pure core)
Deep call stacks needing error context
Long-lived codebases prioritizing maintainability
Effects with dependency injection (Reader pattern)
Resource management with guaranteed cleanup

Less suitable:

Simple CRUD applications (standard Result is fine)
Performance-critical hot paths (profile first)
Teams not aligned on functional patterns

The Ecosystem

Stillwater is part of a family of libraries sharing the same philosophy:

Library	Purpose
premortem	Configuration validation and multi-source loading
postmortem	JSON schema validation with precise path tracking
mindset	Zero-cost effect-based state machines
stilltypes	Domain-specific refined types

Getting Started

Add to your Cargo.toml:

[dependencies]
stillwater = "1.0"

Start with validation - it's the most immediately useful part:

use stillwater::Validation;

fn validate(input: &Input) -> Validation<Valid, Vec<String>> {
    Validation::all((
        validate_field1(&input.field1),
        validate_field2(&input.field2),
        validate_field3(&input.field3),
    ))
    .map(|(f1, f2, f3)| Valid { f1, f2, f3 })
}

Then explore effects when you need testable I/O composition:

use stillwater::prelude::*;

fn fetch_and_process(id: Id) -> impl Effect<Output = Result, Error = AppError, Env = AppEnv> {
    asks(|env: &AppEnv| env.api.fetch(id))
        .and_then(|data| pure(transform(data)))
        .context("fetching and processing data")
}

// In tests - no real API needed
let result = fetch_and_process(id).run(&mock_env).await;

What's Next

With 1.0 stable, future work focuses on:

More refined type predicates stilltypes
Additional effect combinators
Performance optimizations
Ecosystem integration examples

Check out stillwater on GitHub or crates.io.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Compile-Time Resource Tracking in Rust: From Runtime Brackets to Type-Level Safety

Glen Baker — Sat, 20 Dec 2025 22:50:21 +0000

Originally published on Entropic Drift

The Problem: Runtime Resource Leaks

Resource leaks are insidious. A forgotten close(), a missing commit(), an exception path that skips cleanup. Your code compiles. It runs. It even works until the connection pool exhausts, the file handles run out, or a transaction holds a lock forever.

The traditional Rust approach uses RAII: wrap resources in structs that clean up in Drop. This works well for ownership-based patterns, but falls short when:

Resources are passed through async boundaries
Effects need to be composed and chained
You want to express protocols (e.g., begin → query → commit/rollback)
Clean-up logic itself can fail and needs handling

What if the type system could enforce that every resource acquired is eventually released before you run the code?

The Foundation: Runtime Brackets

Stillwater has provided runtime resource safety since earlier versions through the bracket pattern. The bracket function guarantees cleanup runs even when errors occur:

use stillwater::effect::prelude::*;

let result = bracket(
    open_connection(),                          // Acquire
    |conn| async move { conn.close().await },   // Release (always runs)
    |conn| fetch_data(conn),                    // Use
).run(&env).await;

This solves the "forgotten cleanup" problem at runtime. The release function always runs whether it has success or failure, panic or not. Stillwater provides variants for different needs:

bracket — Basic acquire/use/release with guaranteed cleanup
bracket2, bracket3 — Multiple resources with LIFO cleanup order
bracket_full — Returns BracketError with explicit error info for both use and cleanup failures
acquiring — Fluent builder for composing multiple resources

// Multiple resources with the fluent builder
let result = acquiring(open_conn(), |c| async move { c.close().await })
    .and(open_file(path), |f| async move { f.close().await })
    .with_flat2(|conn, file| process(conn, file))
    .run(&env)
    .await;

This works. Cleanup happens. But it's still runtime verification so you don't know until the code runs that your brackets are balanced.

Stillwater 0.14.0: Type-Level Resource Tracking

Stillwater 0.14.0 builds on the runtime bracket foundation with compile-time resource tracking. Now you can make the compiler prove your resources are balanced before the code runs.

use stillwater::effect::resource::*;
use stillwater::pure;

// The TYPE says: this acquires a FileRes
fn open_file(path: &str) -> impl ResourceEffect<
    Output = String,
    Acquires = Has<FileRes>,
    Releases = Empty,
> {
    pure::<_, String, ()>(format!("handle:{}", path)).acquires::<FileRes>()
}

// The TYPE says: this releases a FileRes
fn close_file(handle: String) -> impl ResourceEffect<
    Output = (),
    Acquires = Empty,
    Releases = Has<FileRes>,
> {
    pure::<_, String, ()>(()).releases::<FileRes>()
}

The ResourceEffect trait extends Effect with two associated types:

Acquires: what resources this effect creates
Releases: what resources this effect consumes

This is documentation that the compiler can check.

The Bracket Pattern: Guaranteed Resource Neutrality

The real power comes from resource_bracket. It enforces that an operation acquires a resource, uses it, and releases it:

fn read_file_safely(path: &str) -> impl ResourceEffect<
    Output = String,
    Acquires = Empty,  // <-- Guaranteed by the type system
    Releases = Empty,  // <-- No leaks possible
> {
    bracket::<FileRes>()
        .acquire(open_file(path))
        .release(|handle| async move { close_file(handle).run(&()).await })
        .use_fn(|handle| read_contents(handle))
}

The bracket::<FileRes>() builder captures the resource type once, then infers everything else from the chained method calls.

The return type says Acquires = Empty, Releases = Empty. This means the function is resource-neutral. If your bracket is wrong and the acquire doesn't match the release then it won't compile.

Protocol Enforcement: Database Transactions

Consider database transactions. A transaction must be opened, used, and then either committed or rolled back. Missing the final step is a bug. Let's make it a compile error:

fn begin_tx() -> impl ResourceEffect<Acquires = Has<TxRes>> {
    pure::<_, String, ()>("tx_12345".to_string()).acquires::<TxRes>()
}

fn commit(tx: String) -> impl ResourceEffect<Releases = Has<TxRes>> {
    pure::<_, String, ()>(()).releases::<TxRes>()
}

fn rollback(tx: String) -> impl ResourceEffect<Releases = Has<TxRes>> {
    pure::<_, String, ()>(()).releases::<TxRes>()
}

fn execute_query(tx: &str, query: &str) -> impl ResourceEffect<
    Acquires = Empty,
    Releases = Empty,
> {
    // Queries are resource-neutral
    pure::<_, String, ()>(vec!["row1".to_string()]).neutral()
}

Now a transaction operation that doesn't close is a type error:

// This function signature promises resource neutrality
fn transfer_funds() -> impl ResourceEffect<Acquires = Empty, Releases = Empty> {
    bracket::<TxRes>()
        .acquire(begin_tx())
        .release(|tx| async move { commit(tx).run(&()).await })
        .use_fn(|tx| {
            execute_query(tx, "UPDATE accounts SET balance = balance - 100 WHERE id = 1");
            execute_query(tx, "UPDATE accounts SET balance = balance + 100 WHERE id = 2");
            pure::<_, String, ()>("transferred".to_string())
        })
}

The type signature enforces that transactions are properly closed. If you try to return begin_tx() without a matching release, the code won't compile.

Tracking Multiple Resources

Real systems juggle multiple resource types. The tracking composes:

// Acquire both a file and database connection
let effect = pure::<_, String, ()>(42)
    .acquires::<FileRes>()
    .also_acquires::<DbRes>();

// Release both
let cleanup = pure::<_, String, ()>(())
    .releases::<FileRes>()
    .also_releases::<DbRes>();

The type system tracks Has<FileRes, Has<DbRes>> as a type-level set. Union operations combine sets from chained effects.

Compile-Time Assertions

For critical code paths, assert resource neutrality explicitly:

fn safe_operation() -> impl ResourceEffect<Acquires = Empty, Releases = Empty> {
    let effect = bracket::<FileRes>()
        .acquire(open_file("data.txt"))
        .release(|h| async move { close_file(h).run(&()).await })
        .use_fn(|h| read_contents(h));

    // This is a compile-time check, not a runtime assert
    assert_resource_neutral(effect)
}

If effect isn't actually resource-neutral, this fails at compile time. The assertion costs nothing at runtime since it's purely type-level.

Custom Resource Kinds

Define your own resource markers for domain-specific tracking:

struct ConnectionPoolRes;

impl ResourceKind for ConnectionPoolRes {
    const NAME: &'static str = "ConnectionPool";
}

fn acquire_connection() -> impl ResourceEffect<Acquires = Has<ConnectionPoolRes>> {
    pure::<_, String, ()>("conn_42".to_string()).acquires::<ConnectionPoolRes>()
}

fn release_connection(conn: String) -> impl ResourceEffect<Releases = Has<ConnectionPoolRes>> {
    pure::<_, String, ()>(()).releases::<ConnectionPoolRes>()
}

The built-in markers (FileRes, DbRes, LockRes, TxRes, SocketRes) cover common cases, but you're not limited to them.

Zero Runtime Overhead

This is the crucial point: all tracking happens at compile time. The implementation uses:

PhantomData for type-level annotations (zero-sized)
Associated types for resource set tracking (computed at compile time)
The Tracked wrapper delegates directly to the inner effect

pub struct Tracked<Eff, Acq: ResourceSet = Empty, Rel: ResourceSet = Empty> {
    inner: Eff,
    _phantom: PhantomData<(Acq, Rel)>,  // Zero bytes
}

impl<Eff: Effect, Acq: ResourceSet, Rel: ResourceSet> Effect for Tracked<Eff, Acq, Rel> {
    async fn run(self, env: &Self::Env) -> Result<Self::Output, Self::Error> {
        self.inner.run(env).await  // Just delegates
    }
}

There are no runtime checks, no allocations, no indirection. The tracking is purely for the type checker.

Comparison: RAII vs Bracket vs Type-Level Tracking

Approach	Leak Detection	Async-Safe	Protocol Enforcement	Runtime Cost
RAII (`Drop`)	Runtime	Limited	No	Minimal
Stillwater `bracket`	Runtime	Yes	No	Minimal
Stillwater `bracket::<R>()`	Compile time	Yes	Yes	Zero

RAII works when you own the resource directly. Stillwater's runtime bracket() ensures cleanup always runs. This is great for simple acquire/use/release patterns. The type-level bracket::<R>() builder goes further: it makes the protocol—acquire, use, release—visible in the type signature and checked before the code runs.

Use them together: runtime brackets for guaranteed cleanup, type-level tracking for compile-time verification of complex protocols.

When to Use This

Type-level resource tracking shines when:

Resource leaks are high-severity bugs (connection pools, file systems, critical sections)
Protocols must be followed (begin → work → commit/rollback)
Effects are composed across function boundaries
You want documentation that can't lie (types are always current)

For simple, single-owner resources, RAII remains the right choice. For complex effect pipelines where resource safety is critical, type-level tracking catches bugs that runtime checks would miss.

Getting Started

Add stillwater 0.14.0 to your Cargo.toml:

[dependencies]
stillwater = "0.14"

Import the resource tracking module:

use stillwater::effect::resource::*;
use stillwater::pure;

// Start annotating your effects
fn my_acquire() -> impl ResourceEffect<Acquires = Has<FileRes>> {
    pure::<_, String, ()>("handle".to_string()).acquires::<FileRes>()
}

The existing Effect API continues to work unchanged. Resource tracking is purely additive and opt-in.

Summary

Stillwater's resource management story now has two complementary layers:

Runtime brackets (bracket, bracket2, acquiring) — Guarantee cleanup always runs, even on errors or panics
Compile-time tracking (bracket::<R>() builder, ResourceEffect) — Prove resource protocols are balanced before the code runs

Together they provide defense in depth:

Runtime brackets ensure cleanup happens in production
Type-level tracking catches protocol violations during development
Ergonomic API via builder pattern (single type parameter)
Zero runtime overhead via PhantomData
Composable across effect chains and function boundaries

Resources are too important to leave to runtime chance. Start with brackets for guaranteed cleanup. Add type-level tracking when protocols matter.

Stillwater is a Rust library for validation, effect composition, and functional programming patterns. Version 0.14.0 adds compile-time resource tracking as a type-level layer on top of its existing runtime bracket patterns.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Refactoring a God Object Detector That Was Itself a God Object

Glen Baker — Tue, 09 Dec 2025 05:35:01 +0000

Originally published on Entropic Drift

The Irony

My code analysis tool debtmap has a god object detector. It scans Rust codebases looking for structs with too many methods, too many fields, and too many responsibilities. When it finds one, it generates recommendations for splitting the monolith into focused modules.

The irony? The god object detector was itself a god object.

4,362 lines in god_object_detector.rs. 3,461 lines in god_object_analysis.rs. Mixed concerns everywhere—AST traversal interleaved with scoring algorithms, classification logic coupled to I/O, recommendation generation tangled with threshold checks.

I decided to fix this using Stillwater's "Pure Core, Imperative Shell" architecture. This post documents the nine-phase refactoring journey from specs 181a through 181i.

When Your Tool Turns on You

The best part? I didn't discover this problem through code review or intuition. Debtmap itself flagged its own god object detector as the #1 and #2 most critical technical debt in the entire codebase:

#1 SCORE: 159.2 [CRITICAL]
├─ LOCATION: ./src/organization/god_object_detector.rs
├─ IMPACT: -100 complexity, -219.3 maintainability improvement
├─ METRICS: 4362 lines, 163 functions, avg complexity: 3.1
├─ GOD OBJECT: 55 methods, 5 fields, 8 responsibilities (score: 0.9)
   Suggested: 2 recommended module splits
├─ ACTION: Split by analysis phase: 1) Data collection 2) Pattern detection
   3) Scoring/metrics 4) Reporting. Keep related analyses together.
└─ WHY THIS MATTERS: File has 8 distinct responsibilities across 55 methods.
   High coupling makes changes risky and testing difficult. Splitting by
   responsibility will improve maintainability and reduce change impact.

#2 SCORE: 66.3 [CRITICAL]
├─ LOCATION: ./src/organization/god_object_analysis.rs
├─ IMPACT: -61 complexity, -290.9 maintainability improvement
├─ METRICS: 3304 lines, 159 functions, avg complexity: 1.9
├─ GOD OBJECT: 142 methods, 33 fields, 12 responsibilities (score: 1.0)
   Suggested: 4 recommended module splits
├─ ACTION: URGENT: 3304 lines, 159 functions! Split by data flow:
   1) Input/parsing 2) Core logic/transformation 3) Output/formatting.
   Create 6 focused modules with <30 functions each.
└─ WHY THIS MATTERS: File has 12 distinct responsibilities across 142 methods.

Dog food validation that the analysis works. So, I took debtmap's advice and refactored the god object detector.

Automating the Refactoring with Prodigy

Here's where it gets interesting: I didn't manually implement all nine refactoring phases. I used Prodigy, my AI workflow orchestration tool, to automate the entire process.

One command processed all nine specs:

prodigy run workflows/implement.yml --map "specs/181*"

Prodigy discovered all 9 spec files (181a through 181i) and for each one:

Implemented the spec using Claude
Validated completion with automatic checking
Ran tests (just test) to ensure nothing broke
Ran linters (just fmt-check && just lint) to maintain quality
Created commits with clear messages
Recovered from incomplete implementations automatically

The results:

✅ 3 hours 59 minutes total execution time
✅ 9 specs processed sequentially
✅ 19 commits created across the phases
✅ All tests passed on first attempt for every spec
✅ Automatic recovery when validation fell short (specs 181b, 181c, 181e, 181i)

The recovery mechanism in action (spec 181i):

⚠️ Validation incomplete: 45.0% complete (threshold: 100.0%)
🔄 Attempting to complete implementation (attempt 1/5)
⚠️ Validation still incomplete: 92.0% complete
🔄 Attempting to complete implementation (attempt 2/5)
⚠️ Validation still incomplete: 99.7% complete
🔄 Attempting to complete implementation (attempt 3/5)
✅ Validation passed: 100.0% complete

Each spec went through validation before moving to the next phase. If validation showed gaps, Prodigy automatically attempted to complete the implementation up to 5 times. This meant I could walk away while the refactoring ran, confident that each phase would be fully completed and tested before the next began.

The merge workflow:
After all 9 specs completed, Prodigy:

Ran CI checks to ensure everything still worked
Merged the worktree branch to master
Cleaned up the temporary worktree
Created a clean commit history

Total automation from spec to production-ready code.

The Numbers

Before (v0.7.0):

3 monolithic files
7,823 lines of code
110 functions (many with mixed concerns)
I/O, business logic, and orchestration all tangled together

After (v0.9.0):

14 focused modules
4,149 lines of code (~47% reduction)
45 pure functions with clear separation
97 tests passing in 0.03 seconds

The code is shorter, clearer, and more testable. Here's how I got there.

Phase 1: Foundation Analysis (Spec 181a)

Before touching any code, I needed to understand what I had. I analyzed all 8,762 lines across 6 files and classified every function:

Pure functions: 53 (48%) - No side effects, deterministic
I/O operations: 14 (13%) - AST traversal, file access
Orchestration: 29 (26%) - Composing other functions
Mixed: 11 (10%) - The problem children that needed splitting

This analysis identified the target architecture:

god_object/
├── Pure Core (business logic)
│   ├── types.rs (~200 lines)
│   ├── thresholds.rs (~100 lines)
│   ├── predicates.rs (~150 lines)
│   ├── scoring.rs (~200 lines)
│   ├── classifier.rs (~200 lines)
│   └── recommender.rs (~250 lines)
│
├── Orchestration
│   └── detector.rs (~250 lines)
│
└── I/O Shell
    └── ast_visitor.rs (existing)

I also established baseline benchmarks for critical paths to ensure the refactoring didn't degrade performance.

Phase 2: Extract Types and Thresholds (Spec 181b)

The first actual code change: extract all data structures and configuration into dedicated modules.

Before: Types scattered throughout god_object_analysis.rs (648 lines)

After: Organized into focused type modules:

// core_types.rs - 258 lines
pub struct GodObjectAnalysis {
    pub is_god_object: bool,
    pub method_count: usize,
    pub field_count: usize,
    pub responsibility_count: usize,
    pub god_object_score: f64,
    pub confidence: GodObjectConfidence,
    // ...
}

// classification_types.rs - 121 lines
pub enum GodObjectType {
    MethodHeavy,
    FieldHeavy,
    ResponsibilityHeavy,
    LargeSize,
}

// split_types.rs - 173 lines
pub struct ModuleSplit {
    pub suggested_name: String,
    pub methods_to_move: Vec<String>,
    pub structs_to_move: Vec<String>,
    pub responsibility: String,
    // ...
}

// thresholds.rs - 59 lines
pub struct GodObjectThresholds {
    pub max_methods: usize,
    pub max_fields: usize,
    pub max_traits: usize,
    pub max_lines: usize,
    pub max_complexity: u32,
}

This phase moved 728 insertions and removed 730 deletions - a nearly 1:1 swap that laid the foundation for everything that followed.

Key insight: Pure data types with no behavior make testing and reasoning trivial. All validation and computation moved to separate pure functions.

Phase 3: Extract Pure Scoring Functions (Spec 181c)

Next, I extracted all scoring algorithms into pure, testable functions.

Before: Scoring logic mixed with mutation and state:

// Inside a 500-line impl block
fn calculate_score(&mut self, analysis: &mut Analysis) {
    let mut score = 0.0;
    if self.method_count > threshold {
        score += self.method_factor();
        self.violations.push(Violation::MethodCount);
    }
    // ... more mutation
    analysis.score = score;
}

After: Pure functions with explicit inputs and outputs:

/// Calculate god object score from method, field, responsibility counts, and LOC.
///
/// **Pure function** - deterministic, no side effects.
pub fn calculate_god_object_score(
    method_count: usize,
    field_count: usize,
    responsibility_count: usize,
    lines_of_code: usize,
    thresholds: &GodObjectThresholds,
) -> f64 {
    let method_factor = (method_count as f64 / thresholds.max_methods as f64).min(3.0);
    let field_factor = (field_count as f64 / thresholds.max_fields as f64).min(3.0);
    let responsibility_factor = (responsibility_count as f64 / 3.0).min(3.0);
    let size_factor = (lines_of_code as f64 / thresholds.max_lines as f64).min(3.0);

    let mut violation_count = 0;
    if method_count > thresholds.max_methods { violation_count += 1; }
    if field_count > thresholds.max_fields { violation_count += 1; }
    if responsibility_count > thresholds.max_traits { violation_count += 1; }
    if lines_of_code > thresholds.max_lines { violation_count += 1; }

    let base_score = method_factor * field_factor * responsibility_factor * size_factor;

    match violation_count {
        1 => base_score.max(30.0),
        2 => base_score.max(50.0),
        _ => base_score.max(70.0),
    }
}

Testing becomes trivial:

#[test]
fn test_scoring_deterministic() {
    let thresholds = GodObjectThresholds::default();
    let score1 = calculate_god_object_score(30, 20, 8, 1500, &thresholds);
    let score2 = calculate_god_object_score(30, 20, 8, 1500, &thresholds);
    assert_eq!(score1, score2); // Always passes - pure function
}

#[test]
fn test_violation_based_scoring() {
    let thresholds = GodObjectThresholds::default();

    // Single violation
    let score = calculate_god_object_score(25, 5, 2, 200, &thresholds);
    assert!(score >= 30.0);

    // Two violations
    let score = calculate_god_object_score(25, 15, 2, 200, &thresholds);
    assert!(score >= 50.0);

    // Three+ violations
    let score = calculate_god_object_score(25, 15, 8, 200, &thresholds);
    assert!(score >= 70.0);
}

This phase added 443 lines of pure scoring logic and removed 384 lines of mixed-concern code from the monolith.

Phase 4: Extract Detection Predicates (Spec 181d)

All boolean checks became composable predicate functions.

Before: Checks scattered through methods:

if analysis.method_count > self.max_methods
    || analysis.field_count > self.max_fields
    || analysis.responsibility_count > self.max_traits {
    // ... 50 lines of nested logic
}

After: Named predicates that document intent:

/// Check if method count exceeds threshold.
pub fn exceeds_method_threshold(count: usize, threshold: usize) -> bool {
    count > threshold
}

/// Check if field count exceeds threshold.
pub fn exceeds_field_threshold(count: usize, threshold: usize) -> bool {
    count > threshold
}

/// Check if counts indicate a god object.
pub fn is_god_object(
    method_count: usize,
    field_count: usize,
    responsibility_count: usize,
    thresholds: &GodObjectThresholds,
) -> bool {
    exceeds_method_threshold(method_count, thresholds.max_methods)
        || exceeds_field_threshold(field_count, thresholds.max_fields)
        || exceeds_responsibility_threshold(responsibility_count, thresholds.max_traits)
}

/// Check if struct count and domain count indicate cross-domain mixing.
pub fn is_hybrid_god_module(struct_count: usize, domain_count: usize) -> bool {
    struct_count > 15 && domain_count > 3 && struct_count > domain_count * 3
}

These predicates are composable—you can combine them with && and || to build complex detection logic. They're also self-documenting—the function name tells you exactly what it checks.

Phase 5: Extract Classification Logic (Spec 181e)

Classification determines confidence levels and groups methods by responsibility.

Before: Classification mixed with analysis iteration:

fn analyze(&mut self, ast: &Ast) -> GodObjectAnalysis {
    // ... 200 lines of AST traversal
    let mut confidence = GodObjectConfidence::NotGodObject;
    let mut violations = 0;
    if self.methods > threshold { violations += 1; }
    if self.fields > threshold { violations += 1; }
    // ... more mutation
    confidence = match violations {
        5 => GodObjectConfidence::Definite,
        3..=4 => GodObjectConfidence::Probable,
        _ => GodObjectConfidence::Possible,
    };
    // ... another 100 lines
}

After: Pure classification functions:

/// Determine confidence level from score and metrics.
///
/// Maps threshold violations to confidence levels:
/// - 5 violations → Definite
/// - 3-4 violations → Probable
/// - 1-2 violations → Possible
/// - 0 violations → NotGodObject
pub fn determine_confidence(
    method_count: usize,
    field_count: usize,
    responsibility_count: usize,
    lines_of_code: usize,
    complexity_sum: u32,
    thresholds: &GodObjectThresholds,
) -> GodObjectConfidence {
    let mut violations = 0;

    if method_count > thresholds.max_methods { violations += 1; }
    if field_count > thresholds.max_fields { violations += 1; }
    if responsibility_count > thresholds.max_traits { violations += 1; }
    if lines_of_code > thresholds.max_lines { violations += 1; }
    if complexity_sum > thresholds.max_complexity { violations += 1; }

    match violations {
        5 => GodObjectConfidence::Definite,
        3..=4 => GodObjectConfidence::Probable,
        1..=2 => GodObjectConfidence::Possible,
        _ => GodObjectConfidence::NotGodObject,
    }
}

/// Group methods by their inferred responsibility domain.
pub fn group_methods_by_responsibility(
    methods: &[String],
) -> HashMap<String, Vec<String>> {
    let mut groups: HashMap<String, Vec<String>> = HashMap::new();

    for method in methods {
        let responsibility = infer_responsibility(method);
        groups.entry(responsibility).or_default().push(method.clone());
    }

    groups
}

The classification module added detailed domain extraction, struct grouping, and responsibility inference—all as pure, testable functions.

Phase 6: Extract Recommendation Logic (Spec 181f)

Recommendation generation creates refactoring suggestions.

Before: Recommendations generated during analysis with mutation:

fn analyze(&mut self, ast: &Ast) -> GodObjectAnalysis {
    // ... analysis code
    for domain in domains {
        let mut split = ModuleSplit::new();
        split.name = format!("{}_{}", self.name, domain);
        split.add_methods(&domain_methods);
        self.splits.push(split); // Mutation
    }
    // ...
}

After: Pure recommendation functions:

/// Suggest module splits based on struct name patterns (domain-based grouping).
///
/// Groups structs by domain and creates split recommendations for groups with
/// more than one struct.
pub fn suggest_module_splits_by_domain(structs: &[StructMetrics]) -> Vec<ModuleSplit> {
    let mut grouped: HashMap<String, Vec<String>> = HashMap::new();
    let mut line_estimates: HashMap<String, usize> = HashMap::new();
    let mut method_counts: HashMap<String, usize> = HashMap::new();

    for struct_metrics in structs {
        let domain = classify_struct_domain(&struct_metrics.name);
        grouped
            .entry(domain.clone())
            .or_default()
            .push(struct_metrics.name.clone());
        *line_estimates.entry(domain.clone()).or_insert(0) +=
            struct_metrics.line_span.1 - struct_metrics.line_span.0;
        *method_counts.entry(domain).or_insert(0) += struct_metrics.method_count;
    }

    grouped
        .into_iter()
        .filter(|(_, structs)| structs.len() > 1)
        .map(|(domain, structs)| {
            let estimated_lines = line_estimates.get(&domain).copied().unwrap_or(0);
            let method_count = method_counts.get(&domain).copied().unwrap_or(0);
            ModuleSplit {
                suggested_name: format!("config/{}", domain),
                structs_to_move: structs,
                responsibility: domain.clone(),
                estimated_lines,
                method_count,
                rationale: Some(format!(
                    "Structs grouped by '{}' domain to improve organization",
                    domain
                )),
                // ...
            }
        })
        .collect()
}

The recommender module provides four different strategies for generating split recommendations, all composable and testable in isolation.

Phase 7: Create Orchestration Layer (Spec 181g)

With all the pure functions extracted, I needed a way to compose them.

The orchestration pattern:

pub struct GodObjectDetector {
    pub(crate) max_methods: usize,
    pub(crate) max_fields: usize,
    pub(crate) max_responsibilities: usize,
    pub(crate) location_extractor: Option<UnifiedLocationExtractor>,
}

impl GodObjectDetector {
    /// Analyze a type for god object patterns using pure functions.
    pub fn analyze_comprehensive(
        &self,
        item: &syn::Item,
        source: &str,
    ) -> Option<GodObjectAnalysis> {
        // 1. Extract metrics (I/O boundary)
        let type_analysis = TypeVisitor::analyze_item(item, source)?;

        // 2. Create thresholds (configuration)
        let thresholds = GodObjectThresholds {
            max_methods: self.max_methods,
            max_fields: self.max_fields,
            max_traits: self.max_responsibilities,
            max_lines: 500,
            max_complexity: 150,
        };

        // 3. Calculate score (pure function)
        let score = calculate_god_object_score(
            type_analysis.method_count,
            type_analysis.field_count,
            type_analysis.responsibilities.len(),
            type_analysis.lines_of_code,
            &thresholds,
        );

        // 4. Determine confidence (pure function)
        let confidence = determine_confidence(
            type_analysis.method_count,
            type_analysis.field_count,
            type_analysis.responsibilities.len(),
            type_analysis.lines_of_code,
            type_analysis.complexity_sum,
            &thresholds,
        );

        // 5. Check if god object (pure predicate)
        let is_god_object = is_god_object(
            type_analysis.method_count,
            type_analysis.field_count,
            type_analysis.responsibilities.len(),
            &thresholds,
        );

        // 6. Assemble result
        Some(GodObjectAnalysis {
            is_god_object,
            method_count: type_analysis.method_count,
            field_count: type_analysis.field_count,
            responsibility_count: type_analysis.responsibilities.len(),
            lines_of_code: type_analysis.lines_of_code,
            god_object_score: score,
            confidence,
            // ...
        })
    }
}

The key insight: Orchestration composes pure functions in sequence. It's the only place where the order of operations matters. Everything else is deterministic transformation.

Phase 8: Update Public API (Spec 181h)

With the new modular structure complete, I needed to maintain backward compatibility for external code.

Strategy:

Keep old file names but mark as deprecated
Re-export new functions through old interface
Add #[deprecated] attributes with migration guidance
Create legacy_compat module for transition period

// legacy_compat.rs
#[deprecated(since = "0.9.0", note = "Use classifier::group_methods_by_responsibility")]
pub fn group_methods_by_responsibility_with_domain_patterns(
    methods: &[String],
) -> HashMap<String, Vec<String>> {
    crate::organization::god_object::classifier::group_methods_by_responsibility(methods)
}

This allowed external tests to continue working while migrating to the new API at their own pace.

Phase 9: Complete Pure Function Migration (Spec 181i)

The final phase: ensure the GodObjectDetector no longer calls the old monolithic code.

Before: Detector delegated to legacy implementations

After: Detector composes pure functions from the modular structure

All 260 of 261 tests passing (99.6%). The single failing test had an edge case with responsibility granularity that was addressed in follow-up work.

The complete architecture:

src/organization/god_object/
├── types.rs              (23 lines)   - Re-exports all types
├── core_types.rs         (258 lines)  - Core data structures
├── classification_types. (121 lines)  - Classification enums
├── split_types.rs        (173 lines)  - Split recommendations
├── metrics_types.rs      (20 lines)   - Metrics tracking
├── thresholds.rs         (59 lines)   - Configuration
├── predicates.rs         (150 lines)  - Boolean checks (pure)
├── scoring.rs            (443 lines)  - Scoring algorithms (pure)
├── classifier.rs         (600 lines)  - Classification logic (pure)
├── recommender.rs        (700 lines)  - Recommendations (pure)
├── detector.rs           (500 lines)  - Orchestration
├── ast_visitor.rs        (450 lines)  - AST traversal (I/O)
├── legacy_compat.rs      (68 lines)   - Backward compatibility
└── metrics.rs            (367 lines)  - Metrics tracking

Total: 4,149 lines (down from 7,823)

Key Takeaways

1. Pure Functions Make Testing Trivial

Before refactoring, testing required mocking file I/O, setting up AST fixtures, and managing mutable state. After refactoring:

#[test]
fn test_confidence_classification() {
    let thresholds = GodObjectThresholds::default();

    assert_eq!(
        determine_confidence(30, 20, 8, 1500, 300, &thresholds),
        GodObjectConfidence::Definite
    );
}

No mocks. No fixtures. Just inputs and outputs.

2. Separation Enables Composition

With pure functions extracted, I could compose them in different ways:

// Quick check
let is_god = is_god_object(methods, fields, responsibilities, &thresholds);

// Detailed analysis
let score = calculate_god_object_score(methods, fields, responsibilities, loc, &thresholds);
let confidence = determine_confidence(methods, fields, responsibilities, loc, complexity, &thresholds);

// Full pipeline
let analysis = detector.analyze_comprehensive(item, source);

The same pure functions power all three paths. No duplication.

3. Module Size Matters

Breaking a 4,362-line file into modules averaging 277 lines each made everything easier to reason about. Each module fits in your head.

4. Break Down Big Problems, Then Automate

I designed 9 incremental specs to break the refactoring into focused phases. Each spec was just a markdown file describing one transformation. But the workflow (implement.yml) did the heavy lifting:

commands:
  - claude: "/prodigy-implement-spec $ARG"
    validate:
      claude: "/prodigy-validate-spec $ARG"
      threshold: 100
      on_incomplete:
        claude: "/prodigy-complete-spec $ARG --gaps ${validation.gaps}"
        max_attempts: 5

  - shell: "just test"
    on_failure:
      claude: "/prodigy-debug-test-failure --spec $ARG"
      max_attempts: 5

  - shell: "just fmt-check && just lint"
    on_failure:
      claude: "/prodigy-lint ${shell.output}"
      max_attempts: 5

merge:
  - claude: "/prodigy-merge-master"
  - claude: "/prodigy-ci"
  - claude: "/prodigy-merge-worktree ${merge.source_branch} ${merge.target_branch}"

For each of the 9 specs, the workflow:

Implemented it with Claude
Validated it reached 100% completion (recovered automatically if incomplete, up to 5 attempts)
Ran tests (debugged and fixed failures automatically, up to 5 attempts)
Ran linting (fixed formatting/lint issues automatically, up to 5 attempts)
Created commits only after all checks passed

After all 9 specs completed:

Merged master into the worktree (handled conflicts if any)
Ran CI and fixed any integration issues
Merged back to master with clean history

The key insight: Specs describe what to do (one focused change at a time). Workflows describe how to do it safely (with validation, testing, recovery). Together they enable reliable automation.

This is better than manual incremental work because you get the benefits of small steps without the context-switching cost of implementing them one at a time over days. You also get guarantees (tests pass, validation complete) that humans often skip when tired.

The Results

Code Quality:

✅ 97 tests passing in 0.03 seconds
✅ Zero clippy warnings
✅ 45 pure functions with clear separation
✅ Each module under 1,000 lines (most under 300)

Maintainability:

✅ Functions average 20 lines or less
✅ Clear module boundaries
✅ No circular dependencies
✅ Easy to extend with new analysis types

Testability:

✅ Pure functions need no mocks
✅ Deterministic test results
✅ Fast test execution (0.03s for 97 tests)
✅ Easy to add property tests

Conclusion

The god object detector is no longer a god object. It's a collection of focused modules with clear responsibilities, composable pure functions, and a clean separation between I/O and business logic.

Three tools working together made this possible:

Debtmap detected the problem - The tool caught its own worst code, validating that the analysis works in production
Stillwater provided the architecture - Pure Core/Imperative Shell pattern gave a clear refactoring strategy
Prodigy automated the implementation - 9 specs executed in 4 hours with automatic validation and testing

This is the development workflow I want: tools that find problems, patterns that solve them, and automation that implements solutions. No manual grunt work, no forgetting to run tests, no skipping validation steps.

The Stillwater architecture delivered:

Pure Core - Business logic as pure functions (scoring, predicates, classification)
Orchestration - Composing pure functions into pipelines (detector)
Imperative Shell - I/O isolated to boundaries (ast_visitor)

The result is testable (97 tests in 0.03s), maintainable (focused modules averaging 277 lines), and most importantly, it doesn't commit the sin it's designed to detect.

The full code is available in debtmap v0.9.0. All three tools are open source:

Debtmap - Code analysis and technical debt detection
Stillwater - Functional programming patterns for Rust
Prodigy - AI workflow orchestration

If you're fighting a god object in your own codebase, I hope this refactoring journey provides a roadmap. And if you want to automate your own refactorings, the workflow files that orchestrated this work are in the debtmap repository.

Related posts:

Have you refactored a god object before? What patterns worked for you? Open an issue to share your experience.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Stillwater Validation for Rustaceans: Accumulating Errors Instead of Failing Fast

Glen Baker — Thu, 04 Dec 2025 19:19:25 +0000

Originally published on Entropic Drift

The Problem with Result for Validation

Most Rust validation code follows a familiar pattern: when a user submits a form with three errors, the API reports only the first one. They fix it, resubmit, and discover error number two. Fix that, resubmit again. Error three.

Three round trips for what should have been one.

This is due to Rust's Result type and how it short-circuits.

The Problem in Code

Here's how most of us write validation:

fn validate_user_registration(input: &RegistrationInput) -> Result<ValidatedUser, ValidationError> {
    let email = validate_email(&input.email)?;  // Stops here if invalid
    let age = validate_age(input.age)?;         // Never reached
    let username = validate_username(&input.username)?;  // Never reached

    Ok(ValidatedUser { email, age, username })
}

If the email is invalid, we return immediately. The user never learns their age and username were also wrong.

This is Result doing exactly what it's designed to do. The ? operator is excellent for error propagation, but propagation and accumulation are different things.

The Manual Fix (And Why It's Painful)

You've probably written this code before:

fn validate_user_registration(input: &RegistrationInput) -> Result<ValidatedUser, Vec<String>> {
    let mut errors = Vec::new();

    let email = match validate_email(&input.email) {
        Ok(e) => Some(e),
        Err(e) => { errors.push(e); None }
    };

    let age = match validate_age(input.age) {
        Ok(a) => Some(a),
        Err(e) => { errors.push(e); None }
    };

    let username = match validate_username(&input.username) {
        Ok(u) => Some(u),
        Err(e) => { errors.push(e); None }
    };

    if errors.is_empty() {
        Ok(ValidatedUser {
            email: email.unwrap(),  // We know it's Some
            age: age.unwrap(),
            username: username.unwrap(),
        })
    } else {
        Err(errors)
    }
}

22 lines to validate 3 fields. And those .unwrap() calls? They're safe here, but they make me nervous. The compiler doesn't prove they're safe—we're relying on our logic being correct.

Add more fields and this explodes:

// With 10 fields, this pattern becomes:
let mut errors = Vec::new();
let field1 = match validate_field1(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field2 = match validate_field2(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field3 = match validate_field3(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field4 = match validate_field4(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field5 = match validate_field5(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field6 = match validate_field6(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field7 = match validate_field7(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field8 = match validate_field8(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field9 = match validate_field9(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };
let field10 = match validate_field10(...) { Ok(v) => Some(v), Err(e) => { errors.push(e); None } };

if errors.is_empty() {
    Ok(ValidatedThing {
        field1: field1.unwrap(),
        field2: field2.unwrap(),
        field3: field3.unwrap(),
        // ... you get the idea
    })
} else {
    Err(errors)
}

This is boilerplate hell. And every .unwrap() is a code smell that makes reviewers twitch.

Enter Validation

Stillwater provides a Validation type that handles error accumulation correctly:

use stillwater::Validation;

fn validate_user_registration(input: &RegistrationInput) -> Validation<ValidatedUser, Vec<String>> {
    Validation::all((
        validate_email(&input.email),
        validate_age(input.age),
        validate_username(&input.username),
    ))
    .map(|(email, age, username)| ValidatedUser { email, age, username })
}

8 lines. No .unwrap(). No manual error collection. No Option wrappers.

When you call this with invalid data, you get all the errors:

let result = validate_user_registration(&bad_input);

match result {
    Validation::Success(user) => println!("Valid: {:?}", user),
    Validation::Failure(errors) => {
        // errors contains ALL validation failures, not just the first
        for error in errors {
            println!("Error: {}", error);
        }
    }
}

How It Works (No Magic)

Validation is a simple enum:

pub enum Validation<T, E> {
    Success(T),
    Failure(E),
}

The key insight is Validation::all(). It takes a tuple of validations and:

If all succeed → returns Success with a tuple of values
If any fail → returns Failure with combined errors

The "combining" uses a trait called Semigroup:

pub trait Semigroup {
    fn combine(self, other: Self) -> Self;
}

// Vec<T> combines by appending
impl<T> Semigroup for Vec<T> {
    fn combine(mut self, mut other: Self) -> Self {
        self.append(&mut other);
        self
    }
}

This is the mathematical foundation that makes error accumulation composable.

Writing Validation Functions

Your individual validators return Validation instead of Result:

fn validate_email(email: &str) -> Validation<Email, Vec<String>> {
    if email.contains('@') && email.len() >= 5 {
        Validation::Success(Email(email.to_string()))
    } else {
        Validation::Failure(vec!["Invalid email format".to_string()])
    }
}

fn validate_age(age: u32) -> Validation<Age, Vec<String>> {
    if age >= 18 && age <= 120 {
        Validation::Success(Age(age))
    } else {
        Validation::Failure(vec![format!("Age must be 18-120, got {}", age)])
    }
}

fn validate_username(username: &str) -> Validation<Username, Vec<String>> {
    let mut errors = Vec::new();

    if username.len() < 3 {
        errors.push("Username must be at least 3 characters".to_string());
    }
    if username.len() > 20 {
        errors.push("Username must be at most 20 characters".to_string());
    }
    if !username.chars().all(|c| c.is_alphanumeric() || c == '_') {
        errors.push("Username can only contain alphanumeric characters and underscores".to_string());
    }

    if errors.is_empty() {
        Validation::Success(Username(username.to_string()))
    } else {
        Validation::Failure(errors)
    }
}

Notice validate_username itself accumulates multiple errors. When combined with other validators, all errors flow through.

Scaling to Complex Forms

Real forms have nested objects. Validation composes:

struct OrderInput {
    customer: CustomerInput,
    shipping: ShippingInput,
    items: Vec<ItemInput>,
}

fn validate_order(input: &OrderInput) -> Validation<Order, Vec<String>> {
    Validation::all((
        validate_customer(&input.customer),
        validate_shipping(&input.shipping),
        validate_items(&input.items),
    ))
    .map(|(customer, shipping, items)| Order { customer, shipping, items })
}

fn validate_customer(input: &CustomerInput) -> Validation<Customer, Vec<String>> {
    Validation::all((
        validate_email(&input.email),
        validate_name(&input.name),
        validate_phone(&input.phone),
    ))
    .map(|(email, name, phone)| Customer { email, name, phone })
}

fn validate_items(items: &[ItemInput]) -> Validation<Vec<Item>, Vec<String>> {
    items
        .iter()
        .enumerate()
        .map(|(i, item)| {
            validate_item(item)
                .map_err(|errs| {
                    errs.into_iter()
                        .map(|e| format!("Item {}: {}", i, e))
                        .collect()
                })
        })
        .collect::<Vec<_>>()
        .into_iter()
        .fold(
            Validation::Success(Vec::new()),
            |acc, item| {
                Validation::all((acc, item.map(|i| vec![i])))
                    .map(|(mut items, new)| { items.extend(new); items })
            }
        )
}

Or use the traverse helper for cleaner collection validation:

use stillwater::traverse;

fn validate_items(items: &[ItemInput]) -> Validation<Vec<Item>, Vec<String>> {
    traverse(items.iter(), |item| validate_item(item))
}

The Error Type Flexibility

You're not limited to Vec<String>. Any Semigroup works:

// Structured errors
#[derive(Debug)]
struct FormErrors {
    field_errors: HashMap<String, Vec<String>>,
}

impl Semigroup for FormErrors {
    fn combine(mut self, other: Self) -> Self {
        for (field, errors) in other.field_errors {
            self.field_errors
                .entry(field)
                .or_default()
                .extend(errors);
        }
        self
    }
}

// Now your API can return structured errors
fn validate_email(email: &str) -> Validation<Email, FormErrors> {
    if valid {
        Validation::Success(Email(email.to_string()))
    } else {
        Validation::Failure(FormErrors {
            field_errors: [("email".to_string(), vec!["Invalid format".to_string()])].into(),
        })
    }
}

Your frontend gets structured error data it can map to specific form fields.

Interop with Result

Validation converts to/from Result seamlessly:

// Result → Validation
let validation: Validation<_, Vec<String>> = Validation::from_result(
    some_result,
    |e| vec![e.to_string()]  // Convert error type
);

// Validation → Result
let result: Result<User, Vec<String>> = validation.into_result();

// Use with ? operator (converts to Result)
fn handler(input: Input) -> Result<Response, ApiError> {
    let validated = validate_input(&input)
        .into_result()
        .map_err(|errors| ApiError::Validation(errors))?;

    // Continue with validated data...
    Ok(Response::new(validated))
}

When NOT to Use Validation

Validation is for independent checks that should all run. Don't use it for:

Dependent validations (where later checks depend on earlier ones):

// DON'T: end_date validation needs a valid start_date
Validation::all((
    validate_date(&input.start_date),
    validate_end_after_start(&input.start_date, &input.end_date),  // Needs valid start_date!
))

// DO: Use and_then for dependent validation
validate_date(&input.start_date)
    .and_then(|start| {
        validate_date(&input.end_date)
            .and_then(|end| {
                if end > start {
                    Validation::Success((start, end))
                } else {
                    Validation::Failure(vec!["End date must be after start date".to_string()])
                }
            })
    })

Fail-fast scenarios (where continuing is pointless):

// If auth fails, don't bother validating the rest
// Use Result here, not Validation
fn process_request(req: Request) -> Result<Response, Error> {
    let user = authenticate(&req)?;  // Fail fast is correct here
    let validated = validate_payload(&req.body)?;
    // ...
}

Performance

Validation is zero-cost in the success path. The only overhead is when errors accumulate, and that's the cost you'd pay manually anyway.

There's no heap allocation for the Validation type itself—it's just an enum. Error accumulation allocates only when there are actual errors to collect.

Real-World Applications

The Validation type isn't just for form data—any domain where you need to find all problems rather than just the first one benefits from error accumulation.

Configuration Validation: The premortem library uses Stillwater's Validation to check application configuration. Instead of discovering configuration errors one deploy at a time (missing database host, then invalid port, then bad pool size), premortem performs a "premortem" — finding all configuration problems before your application starts:

// All configuration errors reported at once
Configuration errors (3):
  [config.toml:8] missing required field 'database.host'
  [env:APP_PORT] value "abc" is not a valid integer
  [config.toml:10] 'pool_size' value -5 must be >= 1

Schema Validation: The postmortem library uses Stillwater's Validation for JSON schema validation. Instead of fixing validation errors one API request at a time, postmortem accumulates all schema violations:

let user_schema = Schema::object()
    .required("email", Schema::string().min_len(1))
    .required("age", Schema::integer().min(18))
    .required("password", Schema::string().min_len(8));

// Validation errors (3):
//   $.email: missing required field
//   $.age: value 15 must be >= 18
//   $.password: length 5 is less than minimum 8

The same principle applies anywhere independent checks should all run: batch data import validation, CI pipeline checks, infrastructure validation, multi-tenant configuration checks, and more.

See Premortem vs Figment for a deeper look at configuration validation patterns.

The Bigger Picture

Validation is the entry point to Stillwater's functional programming toolkit. Once you're comfortable with it, you might explore:

Effect: Separating pure logic from I/O for testable async code
Retry: Composable retry policies with backoff strategies
ContextError: Error trails that preserve the full call stack context

But you don't need any of that to benefit from Validation. It stands alone as a solution to a specific, common problem: showing users all their errors at once.

Getting Started

Add to your Cargo.toml:

[dependencies]
stillwater = "0.13"

Start with one form validation. Replace the manual error accumulation pattern with Validation::all(). See if your code gets cleaner.

use stillwater::Validation;

// Before: 22 lines of manual accumulation
// After: 8 lines of composed validation

fn validate(input: &Input) -> Validation<Valid, Vec<String>> {
    Validation::all((
        validate_field1(&input.field1),
        validate_field2(&input.field2),
        validate_field3(&input.field3),
    ))
    .map(|(f1, f2, f3)| Valid { f1, f2, f3 })
}

Your users will appreciate seeing all validation errors at once.

Summary

Approach	Lines (3 fields)	Lines (10 fields)	Type Safety	Error Reporting
Result + `?`	5	12	High	First error only
Manual accumulation	22	65+	Medium (unwrap risk)	All errors
Validation::all()	8	14	High (no unwrap)	All errors

The Validation type isn't magic—it's a principled approach to a real problem. Error accumulation is fundamentally different from error propagation, and having a type that encodes that difference makes your code safer and provides a better user experience.

Give your users complete error feedback. Accumulate your errors.

Stillwater is a Rust library for validation, effect composition, and functional programming patterns. It's designed for Rustaceans who want practical FP without the academic overhead.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Three Patterns That Made Prodigy's Functional Migration Worth It

Glen Baker — Sun, 30 Nov 2025 05:06:35 +0000

Originally published on Entropic Drift

The Migration Story

Over the past few weeks, I've been migrating Prodigy—my Rust-based AI workflow orchestration tool—to use functional programming patterns from Stillwater, a library I built for applicative validation and effect handling in Rust.

The migration touched variable aggregation, environment access, and workflow execution. Not every change was revolutionary, but three patterns produced outsized benefits in testability, safety, and code clarity. This post breaks down each one with concrete before/after comparisons.

Pattern 1: Semigroup-Based Variable Aggregation

The Problem: Prodigy's MapReduce workflows aggregate results from parallel AI agents. Before the migration, aggregation logic was scattered across custom merge implementations—each aggregate type (count, sum, average, etc.) had its own ad-hoc combination logic with no consistency guarantees.

The Solution: Implement the Semigroup trait from Stillwater, which provides a single combine operation with a mathematical guarantee: associativity.

(a.combine(b)).combine(c) == a.combine((b.combine(c)))

This property means results can be combined in any order—essential for safe parallel aggregation.

Before: Ad-Hoc Merge Logic

// Each aggregate type had custom merge logic
fn merge_counts(a: usize, b: usize) -> usize {
    a + b
}

fn merge_averages(sum_a: f64, count_a: usize, sum_b: f64, count_b: usize) -> (f64, usize) {
    (sum_a + sum_b, count_a + count_b)
}

fn merge_maps(mut a: HashMap<String, Value>, b: HashMap<String, Value>) -> HashMap<String, Value> {
    for (k, v) in b {
        a.entry(k).or_insert(v);
    }
    a
}

// No guarantee these operations are associative
// No property tests verifying correctness
// 15 different aggregate types, 15 different implementations

After: Unified Semigroup Implementation

impl Semigroup for AggregateResult {
    fn combine(self, other: Self) -> Self {
        use AggregateResult::*;

        match (self, other) {
            // All 15 types follow the same pattern
            (Count(a), Count(b)) => Count(a.saturating_add(b)),
            (Sum(a), Sum(b)) => Sum(a + b),
            (Average(sum_a, count_a), Average(sum_b, count_b)) => {
                Average(sum_a + sum_b, count_a + count_b)
            }
            (Merge(mut a), Merge(b)) => {
                for (k, v) in b {
                    a.entry(k).or_insert(v);
                }
                Merge(a)
            }
            // ... other types follow same structure

            // Type mismatches handled at validation boundary
            _ => unreachable!("Use combine_homogeneous for type safety")
        }
    }
}

The Key Insight: Validation at Boundaries

The Semigroup combine only handles matching types. Type validation happens at the boundary using Stillwater's homogeneous validation:

pub fn aggregate_map_results(
    results: Vec<AggregateResult>,
) -> Validation<AggregateResult, Vec<TypeMismatchError>> {
    if results.is_empty() {
        return Validation::success(AggregateResult::Count(0));
    }

    combine_homogeneous(results, std::mem::discriminant, TypeMismatchError::new)
}

This accumulates all type mismatches rather than failing on the first one—a core principle of applicative validation.

Property Tests Verify Associativity

proptest! {
    #[test]
    fn prop_count_associativity(a in 0usize..1000, b in 0usize..1000, c in 0usize..1000) {
        let x = AggregateResult::Count(a);
        let y = AggregateResult::Count(b);
        let z = AggregateResult::Count(c);

        // (x combine y) combine z == x combine (y combine z)
        let left = x.clone().combine(y.clone()).combine(z.clone());
        let right = x.combine(y.combine(z));

        prop_assert_eq!(left, right);
    }
}

All 15 aggregate types have property tests verifying associativity. This isn't ceremony—it's proof that parallel aggregation is safe.

Parallel Aggregation: Now Trivial

With associativity guaranteed, parallel aggregation becomes a one-liner:

pub fn parallel_aggregate(results: Vec<AggregateResult>) -> Option<AggregateResult> {
    results.into_par_iter().reduce_with(|a, b| a.combine(b))
}

Rayon's reduce_with can split work arbitrarily because associativity guarantees the same result regardless of combination order.

The Gains

Metric	Before	After
Lines of code	~450	~200
Custom merge functions	15	1
Property tests	0	33
Panics in production	Possible	None (validation at boundaries)
Parallel-safe	Unknown	Proven via associativity

Pattern 2: Reader Pattern for Environment Access

The Problem: MapReduce workflow functions needed access to configuration, storage, executors, and other dependencies. The traditional approach threads these through every function signature:

async fn execute_agent(
    item: &Value,
    config: &MapConfig,
    worktree_manager: &WorktreeManager,
    executor: &CommandExecutor,
    storage: &Storage,
    job_id: &str,
    max_parallel: usize,
) -> Result<AgentResult> { ... }

Six parameters just for dependencies. Adding a new dependency meant updating every function in the call chain.

The Solution: The Reader pattern via Stillwater's Effect::asks, which extracts dependencies from an environment at runtime.

Before: Parameter Threading

async fn execute_map_phase(
    items: Vec<Value>,
    config: &MapConfig,
    worktree_mgr: &WorktreeManager,
    executor: &CommandExecutor,
    storage: &Storage,
    job_id: &str,
    max_parallel: usize,
) -> Result<Vec<AgentResult>> {
    let mut results = Vec::new();
    for item in items {
        let result = execute_agent(
            &item, config, worktree_mgr, executor, storage, job_id, max_parallel
        ).await?;
        results.push(result);
    }
    Ok(results)
}

Every function explicitly passes dependencies to its callees. Tests require constructing all dependencies even when testing logic that doesn't use them.

After: Reader Pattern with Effect::asks

/// Get the worktree manager from the environment
pub fn get_worktree_manager() -> Effect<Arc<WorktreeManager>, MapReduceError, MapEnv> {
    Effect::asks(|env: &MapEnv| env.worktree_manager.clone())
}

/// Get max parallel setting
pub fn get_max_parallel() -> Effect<usize, MapReduceError, MapEnv> {
    Effect::asks(|env: &MapEnv| env.max_parallel)
}

/// Workflow code extracts what it needs
fn execute_agent(item: Value) -> Effect<AgentResult, MapReduceError, MapEnv> {
    get_worktree_manager()
        .and_then(|wt_mgr| create_worktree_effect(&item.id))
        .and_then(|worktree| execute_commands_effect(&item, &worktree))
}

Dependencies are extracted from the environment when needed, not threaded through signatures.

Local Overrides: Temporarily Modify Context

The Reader pattern also enables local modifications via Effect::local:

/// Run an effect with reduced concurrency
pub fn with_max_parallel<T: Send + 'static>(
    max_parallel: usize,
    effect: Effect<T, MapReduceError, MapEnv>,
) -> Effect<T, MapReduceError, MapEnv> {
    Effect::local(
        move |env: &MapEnv| MapEnv {
            max_parallel,
            ..env.clone()
        },
        effect,
    )
}

// Usage: run risky operations with limited concurrency
let effect = with_max_parallel(2, execute_agents(work_items));

The override only affects the wrapped effect—outer code sees the original environment.

Testing with Mock Environments

#[tokio::test]
async fn test_get_max_parallel() {
    let env = MockMapEnvBuilder::new()
        .with_max_parallel(10)
        .build();

    let effect = get_max_parallel();
    let result = effect.run(&env).await;

    assert_eq!(result.unwrap(), 10);
}

#[tokio::test]
async fn test_local_changes_dont_leak() {
    let env = MockMapEnvBuilder::new().with_max_parallel(5).build();

    // Override inside effect
    let inner = with_max_parallel(100, get_max_parallel());
    assert_eq!(inner.run(&env).await.unwrap(), 100);

    // Original environment unchanged
    let outer = get_max_parallel();
    assert_eq!(outer.run(&env).await.unwrap(), 5);
}

Tests use MockMapEnvBuilder to construct exactly the environment they need. No complex setup, no unused dependencies.

The Gains

Metric	Before	After
Average function parameters	6-8	1-2
Test setup boilerplate	High	Minimal (builder pattern)
Adding new dependencies	Change N signatures	Add one getter
Local overrides	Manual save/restore	Effect::local
Composition	Manual threading	and_then/map

Pattern 3: Pure Core, Imperative Shell

The Problem: Workflow execution mixed pure logic (variable expansion, command building, output parsing) with I/O operations (file access, shell commands, API calls). Testing required mocking I/O even for purely computational tests.

The Solution: Separate pure transformations into a pure/ module and I/O operations into an effects/ module. Pure functions are trivially testable; effects are testable with mock environments.

The Architecture

src/cook/workflow/
├── pure/                           # No I/O, no side effects
│   ├── command_builder.rs          # Build commands from templates
│   ├── output_parser.rs            # Parse command output
│   └── variable_expansion.rs       # Expand ${VAR} patterns
└── effects/                        # I/O encapsulated in Effects
    ├── claude.rs                   # Claude API interactions
    ├── shell.rs                    # Shell command execution
    ├── handler.rs                  # Custom handler execution
    └── environment.rs              # Effect environment

Pure Module: No Mocks Needed

//! Pure workflow transformations module
//!
//! All functions in this module are:
//! - Pure: No I/O operations, no side effects
//! - Deterministic: Same inputs always produce same outputs
//! - Testable: No mocking required for unit tests

/// Expand variables in template string
pub fn expand_variables(template: &str, variables: &HashMap<String, String>) -> String {
    let mut result = template.to_string();

    // Expand ${VAR} first (more specific pattern)
    for (key, value) in variables {
        let placeholder = format!("${{{}}}", key);
        result = result.replace(&placeholder, value);
    }

    // Expand $VAR with word boundaries
    for (key, value) in variables {
        result = expand_simple_var(&result, key, value);
    }

    result
}

Testing is straightforward:

#[test]
fn test_expand_variables_mixed() {
    let template = "echo ${name} $value";
    let vars: HashMap<String, String> = [
        ("name".into(), "test".into()),
        ("value".into(), "123".into()),
    ].iter().cloned().collect();

    let result = expand_variables(template, &vars);
    assert_eq!(result, "echo test 123");
}

No mock setup, no async runtime, no I/O stubs. Just input → output.

Property Tests for Pure Functions

proptest! {
    #[test]
    fn prop_variable_expansion_is_deterministic(
        template in ".*",
        vars in prop::collection::hash_map(valid_var_name(), safe_value(), 0..5),
    ) {
        let result1 = expand_variables(&template, &vars);
        let result2 = expand_variables(&template, &vars);

        prop_assert_eq!(result1, result2);
    }

    #[test]
    fn prop_variable_expansion_idempotent_for_safe_values(
        template in r"[a-zA-Z0-9 ${}_.,-]*",
        vars in prop::collection::hash_map(valid_var_name(), r"[a-zA-Z0-9 _-]*", 0..3),
    ) {
        let safe_vars: HashMap<String, String> = vars
            .into_iter()
            .filter(|(_, v)| !v.contains('$'))
            .collect();

        let result1 = expand_variables(&template, &safe_vars);
        let result2 = expand_variables(&result1, &safe_vars);

        // Idempotent when values don't contain variable references
        prop_assert_eq!(result1, result2);
    }
}

Property tests verify behavioral guarantees that unit tests might miss.

Effects Module: I/O in Composable Units

//! Effect-based I/O operations for workflow execution
//!
//! The effects module separates concerns:
//! - **Pure logic** lives in `pure/` module
//! - **I/O effects** live here
//! - **Environment** provides dependencies via injection

pub fn execute_shell_command_effect(
    command: &str,
    vars: &HashMap<String, String>,
) -> Effect<CommandOutput, CommandError, WorkflowEnv> {
    // 1. Pure: expand variables in command
    let expanded = expand_variables(command, vars);

    // 2. Effect: execute shell command
    Effect::from_async(move |env: &WorkflowEnv| {
        let shell = env.shell_executor.clone();
        async move {
            shell.execute(&expanded).await
                .map_err(|e| CommandError::ExecutionFailed {
                    message: e.to_string(),
                    exit_code: None,
                })
        }
    })
}

Effects compose via and_then:

let workflow_effect = execute_shell_command_effect("cargo build", &vars)
    .and_then(|_| execute_shell_command_effect("cargo test", &vars))
    .map(|result| process_output(result));

// Execute with environment
let output = workflow_effect.run(&env).await?;

The Gains

Metric	Before	After
Pure functions	Mixed with I/O	Isolated in pure/
Unit tests requiring mocks	Most	Only effects/
Property tests	Difficult	Natural
Code reuse	Limited	Pure functions composable
Testability	Mock-heavy	Mock-minimal

Lessons Learned

1. Mathematical Guarantees Pay Off

The Semigroup trait's associativity guarantee isn't academic—it's what makes parallel aggregation provably correct. Property tests verify the guarantee; production code depends on it.

2. Reader Pattern Scales

Threading 6 parameters through 10 functions is manageable. Threading 12 parameters through 50 functions isn't. The Reader pattern's overhead is constant; parameter threading's overhead scales with codebase size.

3. Pure Functions Are Easier to Test, Period

Every function in pure/ can be tested with zero setup. No mocks, no async runtime, no environment construction. This isn't ideology—it's pragmatic reduction of test complexity.

4. Separation Creates Options

With pure logic separated from effects, I can:

Test pure logic exhaustively with property tests
Test effects with minimal mock environments
Swap effect implementations (real vs mock) without touching business logic
Reason about pure and effectful code independently

5. The Migration Was Incremental

None of this happened in one big refactor. Each spec (171, 174a-h, 175) was a focused change:

Spec 171: Semigroup for aggregation
Spec 174b: Pure workflow transformations
Spec 174d: Effect modules
Spec 175: Reader pattern helpers

Small, reviewable changes that compiled and passed tests at each step.

Conclusion

The Stillwater migration to Prodigy wasn't about adopting functional programming for its own sake. It was about solving real problems:

Parallel aggregation needed to be provably correct → Semigroup's associativity
Dependency threading was becoming unmanageable → Reader pattern
Testing required too much mock infrastructure → Pure core, imperative shell

Each pattern addressed a specific pain point with a proven solution. The result is code that's more testable, more composable, and easier to reason about.

If you're building Rust applications that deal with aggregation, dependency injection, or I/O-heavy workflows, consider whether these patterns might help. They're not silver bullets, but they're sharp tools for the right problems.

Resources

Projects:

Prodigy - AI workflow orchestration
Stillwater - Applicative validation and effects for Rust

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Mermaid-Sonar: Detecting Hidden Complexity in Diagram Documentation

Glen Baker — Tue, 25 Nov 2025 19:07:41 +0000

Originally published on Entropic Drift

The Problem with AI-Generated Diagrams

When building the MkDocs automation workflow for ripgrep, I ran into a consistent problem: diagrams could be syntactically incorrect or cognitively overwhelming and actually hurt comprehension rather than helping it. There could also be semantic and business logic issues with the diagrams, but we'll set that issue aside for this post.

I needed a way to give AI agents quantitative feedback about diagram complexity. Telling Claude "this diagram is too complex" doesn't work reliably and required manual direction from me. We need better tooling to determine when a given mermaid diagram would actually be readable and how to generally fix it if it isn't.

Traditional Mermaid validation tools only check syntax. They'll tell you if your diagram will render, but not whether humans can actually understand it. There's @mermaid-js/mermaid-cli for full rendering validation, but it requires a browser and is too slow for CI/CD. There's mermaid.parse() for fast syntax checking, but it provides no insight into readability.

That's why I built mermaid-sonar: a complexity analyzer that validates both syntax and readability using research-backed metrics.

Two Complementary Goals

Complexity analysis:
- Node count, edge count, graph density
- Cyclomatic complexity, branching width
- Requires: Static analysis of graph structure
- No alternative: Can't extract these metrics from rendered output
Dimension validation:
- Will this diagram fit in the target viewport?
- Does it exceed platform width constraints?
- Options: Render in headless browser OR estimate from graph structure

Dimension Validation

For dimension validation, I had two options:

Option 1: Render diagrams (like @mermaid-js/mermaid-cli):

Pixel-perfect accuracy
Requires headless browser (heavy dependency)
Slow (seconds per diagram)
Doesn't work well in CI/CD pipelines
Adds complexity to tool requiring browser environment

Option 2: Estimate dimensions (layout-aware formulas):

Fast (milliseconds per diagram)
No additional dependencies (already parsing for complexity)
Perfect for CI/CD automation
Good enough accuracy for detecting viewport issues
Not pixel-perfect (but doesn't need to be)

I chose dimension estimation because:

We're already parsing the diagram for complexity metrics
Speed matters for validating 87+ diagrams in a workflow
We don't need pixel-perfect accuracy—we need to catch diagrams that will obviously exceed viewport limits
A diagram with 1302px estimated width on an 800px viewport is clearly a problem, whether the actual width is 1250px or 1350px

Why dimension validation matters: When a diagram exceeds viewport width, browsers and documentation frameworks shrink it to fit. A 1302px diagram forced into an 800px viewport gets scaled down to ~61% of its original size, making text and labels proportionally smaller and often unreadable. A diagram that's 63% oversized becomes a diagram with text that's 39% smaller than intended—turning readable labels into tiny, illegible text.

Here's an actual 1302px-wide diagram rendered in this blog post, which has an ~800px content width:

graph TD
    Root["Software Development
Lifecycle"] --> Plan["Planning & Design
Requirements"]
    Root --> Dev["Development
Implementation"]
    Root --> Test["Testing & QA
Quality Assurance"]
    Root --> Deploy["Deployment
Release Management"]
    Root --> Monitor["Monitoring
Observability"]
    Root --> Maintain["Maintenance
Bug Fixes"]
    Root --> Doc["Documentation
User Guides"]
    Root --> Security["Security
Vulnerability Mgmt"]

Notice how the text becomes harder to read as the diagram is automatically shrunk to fit the viewport? This is why dimension validation matters—even technically correct diagrams become unreadable when they exceed viewport constraints.

How Dimension Estimation Works

Since we're already parsing diagrams for complexity analysis, we can estimate dimensions using layout-aware formulas:

Label extraction: Parse node labels from the diagram source (handling multi-line labels by measuring the longest line)
Layout direction: Detect whether the diagram uses TD (top-down), LR (left-right), or other layouts
Graph structure: For LR/RL layouts, calculate the longest path through the graph accounting for branches and merges
Width estimation formula:
- LR/RL layouts: width = longestPath × avgLabelLength × charWidth + longestPath × nodeSpacing
- TD/TB layouts: width = maxBranchWidth × avgLabelLength × charWidth + maxBranchWidth × nodeSpacing

Where charWidth defaults to 8px (typical monospace character width) and nodeSpacing defaults to 50px (Mermaid's default spacing).

This provides accurate-enough width estimates for catching viewport issues, without requiring browser rendering.

What Mermaid-Sonar Does

Mermaid-sonar analyzes Mermaid diagrams in your documentation and detects:

Syntax errors - Using the official Mermaid parser (prevents build failures)
Excessive complexity - Too many nodes, edges, or branching (hurts comprehension)
Poor layout choices - TD layout for wide trees, LR for deep hierarchies (reduces readability)
Performance issues - Graphs approaching Mermaid's O(n²) rendering limits (slow page loads)
Viewport compatibility - Diagrams that exceed target platform dimensions (e.g., MkDocs 800px width limit)

Finding the Right Thresholds: Research-Backed Limits

The hardest part of building mermaid-sonar wasn't the implementation—it was figuring out what "too complex" actually means. I didn't want arbitrary limits; I needed defensible thresholds based on research.

Cognitive Load Research

The breakthrough came from finding "Scalability of Network Visualisation from a Cognitive Load Perspective" (Kwon et al., 2020). This paper studied how humans process graph visualizations and found:

High-density graphs (>0.3 density): 50-node limit before cognitive overload
Low-density graphs (<0.3 density): 100-node limit for comprehension
Density matters more than raw node count for readability

This gave me the foundation for the max-nodes rule with density-aware thresholds.

Performance Constraints

Mermaid's official blog post "Flow Charts are O(n²) Complex" provided clear guidance on rendering performance:

100 connections is the practical limit before rendering slows significantly
Layout algorithms scale quadratically with edges
Performance degrades sharply beyond this threshold

This directly informed the max-edges rule.

Visual Hierarchy Research

Studies on visual hierarchy and information design consistently show:

7±2 items is the working memory limit (Miller's Law)
8 parallel branches as a reasonable limit for tree diagrams
Beyond this, readers lose track of structure

This shaped the max-branch-width rule.

Software Engineering Standards

McCabe's cyclomatic complexity metric (originally for code) translates well to diagrams:

Complexity = edges - nodes + 2 for connected graphs
15 is the standard threshold for "too complex" in code
Same principle applies to decision paths in diagrams

This gave me the cyclomatic-complexity rule.

How It Works: From Source to Metrics

Mermaid-sonar follows a simple pipeline:

1. Parse Diagram Source

// Use official Mermaid parser
const diagram = mermaid.parse(diagramSource);

The parser builds an AST (abstract syntax tree) representing the diagram structure.

2. Extract Graph Metrics

const metrics = {
  nodeCount: countNodes(diagram),
  edgeCount: countEdges(diagram),
  graphDensity: edgeCount / (nodeCount * (nodeCount - 1)),
  maxBranchWidth: findMaxBranchWidth(diagram),
  averageDegree: (2 * edgeCount) / nodeCount,
  cyclomaticComplexity: edgeCount - nodeCount + 2
};

3. Apply Research-Backed Rules

if (nodeCount > (density > 0.3 ? 50 : 100)) {
  issues.push({
    rule: 'max-nodes',
    severity: 'error',
    message: `Complex diagram with ${nodeCount} nodes`,
    suggestion: 'Consider breaking into focused sub-diagrams',
    citation: 'https://arxiv.org/abs/2008.07944'
  });
}

4. Provide Actionable Feedback

❌ docs/architecture.md:42
   Wide tree diagram with 12 parallel branches (>8 max)
   → Consider using 'graph LR' or split into multiple diagrams

⚠️ docs/workflow.md:156
   Complex diagram with 85 nodes
   → Consider breaking into focused sub-diagrams

✓ docs/overview.md:15 - Valid and readable

Each issue includes the specific problem, concrete fix suggestion, and link to supporting research when applicable.

Real-World Results: ripgrep Documentation

When I ran mermaid-sonar on the AI-generated ripgrep documentation (55 pages, 87 diagrams), the results validated the entire approach:

Syntax Errors Found: 8 diagrams (9% failure rate)

Missing arrows in flowcharts
Invalid node ID references
Malformed subgraph syntax
Quote escaping issues

These would have caused build failures. Without mermaid-sonar, I'd have discovered them during manual testing or—worse—in CI after pushing.

Complexity Issues Found: 12 diagrams (14% complexity rate)

4 diagrams with >50 nodes (simplified to <30 each)
3 wide tree diagrams (converted from TD to LR layout)
5 overly dense graphs (split into focused sub-diagrams)

These rendered correctly but were cognitively expensive. Without mermaid-sonar, readers would struggle through overwhelming visualizations.

The Key Insight: 23% of AI-generated diagrams (20 out of 87) had issues that no existing tool would catch automatically. Manual review would take hours; mermaid-sonar caught everything in seconds.

Example: The 85-Node Architecture Diagram

One diagram stood out: an 85-node "Architecture Overview" that tried to show every component relationship in a single graph. Technically valid, but completely unreadable at typical markdown rendering sizes.

Mermaid-sonar output:

❌ docs/architecture.md:42
   Complex diagram with 85 nodes (limit: 50 for high-density graphs)
   Graph density: 0.38 (high-density threshold: 0.3)
   → Consider breaking into focused sub-diagrams
   Research: https://arxiv.org/abs/2008.07944

The fix: Split into three focused diagrams:

Core components (18 nodes) - The main execution pipeline
Plugin system (22 nodes) - Extension mechanisms
Configuration flow (15 nodes) - How settings propagate

Each diagram now tells a clear story instead of overwhelming readers with everything at once.

Visualization: Before and After Examples

Let's look at specific examples showing how mermaid-sonar's feedback improves diagram readability. These are actual fixes from the ripgrep documentation.

Example 1: Converting Wide Trees (TD → LR Layout)

Problem: Wide tree diagrams with many parallel branches in TD (top-down) layout create excessive horizontal width. With 8 parallel branches, this exceeds viewport limits by 63%.

Before (failed validation):

❌ ERROR: Diagram width (1302px) exceeds viewport limit by 502px (63% over)
   This TD layout with 8 parallel branches creates excessive width

graph TD
    Root["Software Development
Lifecycle"] --> Plan["Planning & Design
Requirements"]
    Root --> Dev["Development
Implementation"]
    Root --> Test["Testing & QA
Quality Assurance"]
    Root --> Deploy["Deployment
Release Management"]
    Root --> Monitor["Monitoring
Observability"]
    Root --> Maintain["Maintenance
Bug Fixes"]
    Root --> Doc["Documentation
User Guides"]
    Root --> Security["Security
Vulnerability Mgmt"]

After (passes validation):

✓ LR layout stacks vertically, width under 400px
✓ No issues found

graph LR
    Root["Software Development
Lifecycle"] --> Plan["Planning & Design
Requirements"]
    Root --> Dev["Development
Implementation"]
    Root --> Test["Testing & QA
Quality Assurance"]
    Root --> Deploy["Deployment
Release Management"]
    Root --> Monitor["Monitoring
Observability"]
    Root --> Maintain["Maintenance
Bug Fixes"]
    Root --> Doc["Documentation
User Guides"]
    Root --> Security["Security
Vulnerability Mgmt"]

Fix: Changed graph TD to graph LR. Wide trees with many parallel branches work better in LR layout, which stacks branches vertically for natural scrolling instead of forcing horizontal width.

Width improvement: 1302px → ~400px (69% reduction)

Example 2: Split into Sequential Phases (712px → 400px per diagram)

Problem: Wide tree diagrams with many parallel branches create excessive horizontal width, even in TD layout.

Before (warning - 4 parallel branches):

⚠️  Diagram width (712px) due to 4+ parallel branches

graph TD
    Start[Directory Traversal] --> WorkQueue["Work Queue
Files to Search"]
    WorkQueue --> T1[Thread 1]
    WorkQueue --> T2[Thread 2]
    WorkQueue --> T3[Thread 3]
    WorkQueue --> TN[Thread N]

    T1 -->|Work Done| Steal1{More Work?}
    T2 -->|Work Done| Steal2{More Work?}
    T3 -->|Work Done| Steal3{More Work?}

    Steal1 -->|Queue Empty| StealFrom2[Steal from Thread 2]
    Steal2 -->|Queue Empty| StealFrom3[Steal from Thread 3]
    Steal3 -->|Queue Empty| StealFrom1[Steal from Thread 1]

    T1 --> Results[Aggregate Results]
    T2 --> Results
    T3 --> Results
    TN --> Results

After (passes validation with zero warnings):

✓ Split into 3 focused diagrams, each ~400px
✓ Zero warnings (perfect validation)

Phase 1: Work Distribution

flowchart LR
    Start[Directory Traversal] --> WorkQueue["Work Queue
Files to Search"]

    WorkQueue --> Workers["Worker Threads
(T1, T2, T3, TN)"]

    style Start fill:#e1f5ff
    style WorkQueue fill:#fff3e0
    style Workers fill:#f3e5f5

Phase 2: Work Processing

flowchart LR
    Workers[Worker Threads] -->|Work Done| CheckWork{More Work?}

    CheckWork -->|Yes| ContinueWork[Process Next]
    CheckWork -->|No| CheckSteal{Queue Empty?}

    CheckSteal -->|Yes| Steal[Work Stealing]
    CheckSteal -->|No| ContinueWork

    style Workers fill:#fff3e0
    style CheckWork fill:#f3e5f5
    style Steal fill:#fff3e0

Phase 3: Result Aggregation

flowchart LR
    Workers[Worker Threads] --> Completed[Work Completed]
    Steal[Work Stealing] --> Workers

    Completed --> Results[Aggregate Results]

    style Workers fill:#fff3e0
    style Results fill:#e8f5e9

Fix: Split complex workflow into 3 focused phases:

Each phase has 3-5 nodes (simple and clear)
LR layout works perfectly for each sequential phase
Natural left-to-right reading flow
Zero warnings (no layout hints or width issues)

Tradeoff: Uses more vertical space but each phase is clearer and achieves perfect validation.

Example 3: Simplifying Complex Flows + Split Phases (64 nodes → 13 nodes across 3 diagrams)

Problem: Overly detailed flowchart with 64 nodes showing every decision point. Technically correct but cognitively overwhelming.

Before (failed validation):

❌ Complex diagram with 64 nodes and excessive height (900px)
❌ Cyclomatic complexity: 45 (threshold: 15)

flowchart TD
    Start([Config Loading Starts]) --> CheckEnv{RIPGREP_CONFIG_PATH
set?}

    CheckEnv -->|No| NoConfig[No config loaded]
    CheckEnv -->|Empty value| NoConfig
    CheckEnv -->|Yes| ReadFile[Attempt to read file]

    ReadFile --> FileExists{File
exists?}

    FileExists -->|No| FileError[Error: File not found]
    FileExists -->|Permission denied| FileError
    FileError --> NoConfig

    FileExists -->|Yes| ParseLines[Parse lines sequentially]

    ParseLines --> CheckLine{Process
each line}

    CheckLine --> IsEmpty{Empty
line?}
    IsEmpty -->|Yes| NextLine[Skip to next line]

    IsEmpty -->|No| IsComment{Starts with
'#'?}
    IsComment -->|Yes| NextLine

    IsComment -->|No| ConvertUTF8{Valid
UTF-8?}

    ConvertUTF8 -->|Yes| AddArg[Add to arguments list]
    AddArg --> NextLine

    ConvertUTF8 -->|No| Platform{Platform?}
    Platform -->|Unix-like| ParseErr[Report parse error
with line number]
    Platform -->|Windows| ParseErr

    ParseErr --> NextLine
    NextLine --> MoreLines{More
lines?}

    MoreLines -->|Yes| CheckLine
    MoreLines -->|No| ReportErrs{Parse errors
occurred?}

    ReportErrs -->|Yes| ShowErrs[Display errors
with line numbers]
    ReportErrs -->|No| Success[Config loaded successfully]

    ShowErrs --> Success
    NoConfig --> End([Continue with CLI args only])
    Success --> End

After (passes validation with zero warnings):

✓ Simplified to 13 nodes across 3 focused phases
✓ Cyclomatic complexity: 8 (well below threshold)
✓ Zero warnings (perfect validation)

Phase 1: Config Discovery and Loading

flowchart LR
    Start([Config Load]) --> Check{Path set?}

    Check -->|No/Empty| NoConfig[No config]
    Check -->|Yes| ReadFile{Readable?}

    ReadFile -->|No| Error[Error]
    ReadFile -->|Yes| Parse[Parse]

    Error --> NoConfig

    style Start fill:#e1f5ff
    style Error fill:#ffebee
    style NoConfig fill:#f5f5f5

Phase 2: Argument Processing

flowchart LR
    Parse[Parse Config] --> Process["Skip empty lines/comments
    Validate UTF-8
    Collect arguments"]

    Process --> Check{Parse errors?}

    Check -->|Yes| Report[Report errors]
    Check -->|No| Success[Args collected]

    style Process fill:#fff3e0
    style Report fill:#ffebee
    style Success fill:#e8f5e9

Phase 3: Finalization

flowchart LR
    Success[Config loaded] --> End([Continue with CLI args])
    NoConfig[No config] --> End
    Report[Errors reported] --> Success

    style Success fill:#e8f5e9
    style NoConfig fill:#f5f5f5
    style End fill:#e8f5e9

Fix: Combined simplification with phase splitting:

Reduced 64 nodes → 13 nodes (80% reduction)
Split into 3 logical phases with natural boundaries
Each phase uses LR layout (3-5 nodes per diagram)
Complexity reduced from 45 → 8
Zero warnings (no layout hints, no width issues)

Key insight: Sometimes the best fix combines multiple strategies. We simplified the flow AND split it into phases, achieving both comprehensibility and perfect validation. Each phase tells a clear story step-by-step.

Multiple Valid Solutions: Tradeoffs in Practice

Examples 2 and 3 above demonstrate one approach (subgraphs and TD layout), but there's an important insight: multiple valid solutions exist, each with different tradeoffs.

Alternative: Split Diagram Approach

For both Examples 2 and 3, we could achieve zero warnings by splitting into multiple smaller LR diagrams:

Example 2 Alternative - Split work-stealing into 3 phases:

Work Distribution (3 nodes)
Work Processing (5 nodes)
Result Aggregation (3 nodes)

Example 3 Alternative - Split config loading into 3 phases:

Config Discovery (6 nodes)
Argument Processing (5 nodes)
Finalization (3 nodes)

Each smaller diagram passes validation perfectly with LR layout. See the linked files for complete runnable examples.

Choosing Your Approach

Approach	Warnings	Pros	Cons	When to Use
Single diagram (subgraph/TD)	Layout hints	Overview, concise, less vertical space	Gets warnings (acceptable)	Limited space, need big picture
Split diagrams (multiple LR)	Zero	Perfect validation, step-by-step clarity	More vertical space, loses overview	Tutorial-style, plenty of space

Both approaches are valid - the "single diagram with acceptable warnings" is often preferable for its conciseness, while "split diagrams with zero warnings" works better for step-by-step explanations.

See the examples: The mermaid-sonar examples directory includes both approaches for comparison.

Viewport Profiles: Platform-Specific Validation

The examples above use mermaid-sonar's viewport profile feature:

mermaid-sonar --viewport-profile mkdocs docs/

This validates against documentation platform constraints:

Max width: 800px (typical content area)
Max height: No hard limit (vertical scrolling is natural)
Layout-aware estimation: Accounts for node label length and graph structure

Different documentation platforms have different viewport constraints:

MkDocs: 800px width (default content column)
GitHub: ~1000px width (README rendering)

The --viewport-profile flag ensures diagrams render well on your specific platform without manual width calculations.

Why This Matters for AI Documentation Workflows

The ripgrep documentation project revealed a fundamental insight: AI agents need quantitative feedback to produce quality diagrams.

The Feedback Loop Problem

Without mermaid-sonar:

AI generates 87 diagrams
Some have syntax errors → build fails
Many are overly complex → readers confused
Manual review required for all diagrams → bottleneck
Subjective feedback ("too complex") → AI doesn't improve

With mermaid-sonar:

AI generates 87 diagrams
Mermaid-sonar validates automatically → specific issues found
Quantitative feedback ("85 nodes, limit 50") → AI fixes automatically
Manual review only for diagram type choices → efficient
Clear metrics guide AI toward better outputs

The Broader Documentation Automation Ecosystem

Mermaid-sonar fits into a larger philosophy: automated documentation needs automated quality gates.

In the MkDocs drift automation workflow, I use multiple specialized validators:

Mermaid-sonar: Diagram complexity and syntax
MkDocs build: Documentation structure and links
Markdown linters: Formatting and style
Prodigy workflows: Orchestration and feedback loops

Each validator provides quantitative, actionable feedback that AI agents can act on. This is the same philosophy behind Debtmap for Rust code—convert subjective code quality into measurable metrics with clear thresholds.

The key insight: AI agents excel at fixing specific, measurable problems. "This diagram has 85 nodes, simplify to under 50" works. "This diagram feels too busy" doesn't.

Lessons Learned: Building for AI Consumption

Building mermaid-sonar taught me important lessons about designing tools for AI agents:

1. Quantitative Over Qualitative

What doesn't work: "This diagram is too complex"
What works: "This diagram has 85 nodes (limit: 50 for density 0.38)"

AI agents need numbers with clear thresholds. Subjective feedback creates ambiguity; quantitative metrics enable action.

2. Actionable Suggestions

What doesn't work: "Simplify this diagram"
What works: "Split into 3 sub-diagrams or use LR layout for wide trees"

Vague suggestions require creativity; specific actions can be executed mechanically.

3. Research Citations Build Trust

Including research links in error messages serves multiple purposes:

Validates the threshold isn't arbitrary
Helps humans understand the reasoning
Allows AI agents to reference authoritative sources
Makes the feedback defensible in code reviews

4. Machine-Readable Output is Critical

JSON output enables automated workflows:

{
  "rule": "max-nodes",
  "severity": "error",
  "threshold": 50,
  "actual": 85,
  "suggestion": "Split into sub-diagrams",
  "citation": "https://arxiv.org/abs/2008.07944"
}

This structured data lets AI agents:

Parse issues programmatically
Prioritize by severity
Apply fixes systematically
Verify fixes resolved the issue

5. Fast Feedback Loops Matter

Static analysis (milliseconds) vs rendering (seconds) makes a huge difference when validating 87 diagrams. Fast feedback enables:

Rapid iteration during development
Quick CI/CD pipelines
Real-time validation in editors (future work)

Getting Started

Mermaid-sonar is production-ready and available now:

Installation:

npm install -g mermaid-sonar

Basic usage:

# Validate your documentation
mermaid-sonar docs/

# Use with platform-specific viewport constraints
mermaid-sonar --viewport-profile mkdocs docs/

# Use in CI with strict mode (fails on any issues)
mermaid-sonar --strict --viewport-profile mkdocs docs/

Viewport profiles available:

mkdocs - 800px width limit (MkDocs documentation framework)
docusaurus - 900px width limit (Docusaurus documentation framework)
github - 1000px width limit (GitHub README rendering)
mobile - 400px width limit (mobile devices)
Custom profiles via configuration file

Try the examples:
The examples directory includes runnable before/after diagrams from this blog post:

Clone the repo and run npx mermaid-sonar --viewport-profile mkdocs examples/mermaid-sonar-complexity-analyzer/
See both single-diagram and split-diagram approaches
Compare validation results yourself

See the project page for:

Complete CLI reference
Configuration options
CI/CD integration examples
All output formats

Conclusion

Mermaid-sonar solves a specific problem in AI-generated documentation: detecting hidden complexity in diagrams. The key insights:

AI agents need quantitative feedback - "85 nodes (limit: 50)" beats "too complex"
Research-backed thresholds build trust - Not arbitrary limits, but validated science
Fast feedback enables iteration - Static analysis in milliseconds, not seconds
Actionable suggestions drive fixes - "Split into 3 sub-diagrams" beats "simplify"
Machine-readable output enables automation - JSON for programmatic processing
Platform-aware validation prevents surprises - Viewport profiles ensure diagrams render well in production

For automated documentation workflows, this is critical. AI agents will generate technically correct but cognitively expensive diagrams. Mermaid-sonar catches these automatically, providing the quantitative feedback needed to guide agents toward readable visualizations.

This is part of a larger philosophy: automated generation needs automated quality gates. Whether it's mermaid-sonar for diagrams or debtmap for code complexity—quantitative validation enables higher quality AI-powered generation.

The tool is open source and ready to use:

npm: mermaid-sonar
GitHub: iepathos/mermaid-sonar
Project page: Full CLI reference and integration examples

Resources

Related Posts:

Transforming ripgrep's Documentation with AI Automation and MkDocs - The workflow that motivated mermaid-sonar
Debtmap: Quantifying Technical Debt in Rust - Similar philosophy for code complexity

Tools:

Mermaid-Sonar: GitHub | npm | Project Page
Mermaid.js: Official Site
Prodigy: Documentation automation workflows

Research:

Kwon et al. (2020): Scalability of Network Visualisation from a Cognitive Load Perspective - Foundation for node limits
Mermaid Blog: Flow Charts are O(n²) Complex - Performance constraints
McCabe (1976): Cyclomatic Complexity - Basis for complexity metric

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration
Mermaid-Sonar - Mermaid readability analyzer

Rethinking Code Quality Analysis

Glen Baker — Sun, 23 Nov 2025 19:52:00 +0000

Originally published on Entropic Drift

The Problem with Traditional Static Analysis

When you run a typical static analysis tool on your codebase, you get flooded with alerts. Hundreds of warnings about cyclomatic complexity, function length, and nesting depth. The tools treat all complexity equally—a simple validation function with 20 identical if statements gets the same severity rating as genuinely complex business logic with intricate state transitions.

The result? Alert fatigue. Developers learn to ignore the noise, and genuinely risky code gets lost in the shuffle.

A Tale of Two Functions

Try it yourself: All code examples in this post are available at github.com/iepathos/debtmap/tree/master/examples/why-debtmap-samples. Clone the repo and run debtmap analyze . to see the actual output.

Consider these two functions:

Function A: High Cyclomatic Complexity, Low Risk

fn validate_config(config: &Config) -> Result<()> {
    if config.output_dir.is_none() {
        return Err(anyhow!("output_dir required"));
    }
    if config.max_workers.is_none() {
        return Err(anyhow!("max_workers required"));
    }
    if config.timeout_secs.is_none() {
        return Err(anyhow!("timeout_secs required"));
    }
    // ... 15 more identical checks
    Ok(())
}

Traditional tools say: "Cyclomatic complexity: 21 - WARNING!"

Reality: An experienced developer reads this in seconds. It's a repetitive validation pattern. Yes, 20 branches, but they're all identical in structure. Low cognitive load, low risk.

Function B: Moderate Cyclomatic Complexity, High Risk

fn reconcile_state(current: State, target: State) -> Result<Actions> {
    let mut actions = vec![];

    if current.mode != target.mode {
        if current.has_active_connections() {
            if target.mode == Mode::Offline {
                actions.push(drain_connections());
                if current.has_pending_writes() {
                    actions.push(flush_writes());
                }
            }
        } else if target.allows_reconnect() {
            actions.push(establish_connections());
        }
    }

    if let Some(diff) = calculate_config_diff(&current, &target) {
        if diff.requires_restart() {
            actions.push(schedule_restart());
        }
    }

    Ok(actions)
}

Traditional tools say: "Cyclomatic complexity: 9 - acceptable"

Reality: This function has nested conditionals with complex interdependencies. What happens if mode changes but has_active_connections() is false? The cognitive load is high, the logic is intricate, and bugs could easily hide in edge cases.

Lower cyclomatic score, vastly different actual complexity.

Traditional Tools vs Debtmap: Side-by-Side Comparison

Let's see how a traditional static analysis tool (Codacy with Lizard) handles these exact functions compared to debtmap. Both outputs are from running the tools on our sample code.

Codacy/Lizard Output

================================================
  NLOC    CCN   token  PARAM  length  location
------------------------------------------------
      63     21    421      1      63 validate_config@30-92@./src/validation.rs
      21      8    140      2      24 reconcile_state@81-104@./src/state_reconciliation.rs

!!!! Warnings (cyclomatic_complexity > 15) !!!!
================================================
      63     21    421      1      63 validate_config@30-92@./src/validation.rs

Codacy's verdict: validate_config() has CCN=21 (⚠️ WARNING), while reconcile_state() has CCN=8 (✅ OK).

The traditional tool flags the repetitive validation function as critical and completely misses the genuinely complex state reconciliation logic.

Debtmap Output

#1 SCORE: 4.15 [MEDIUM]
├─ LOCATION: ./src/state_reconciliation.rs:81 reconcile_state()
├─ IMPACT: -4 complexity, -1.5 risk
├─ COMPLEXITY: cyclomatic=9 (dampened: 4, factor: 0.51),
   est_branches=9, cognitive=16, nesting=4, entropy=0.28
├─ WHY THIS MATTERS: Coordinator pattern detected with 4 actions
   and 2 state comparisons. Extracting transitions will reduce
   complexity from 9/16 to ~4/11.
├─ RECOMMENDED ACTION: Extract state reconciliation logic into
   transition functions

#2 SCORE: 3.10 [LOW]
├─ LOCATION: ./src/validation.rs:30 validate_config()
├─ IMPACT: -10 complexity, -1.0 risk
├─ COMPLEXITY: cyclomatic=21 (dampened: 10, factor: 0.50),
   est_branches=21, cognitive=6, nesting=1, entropy=0.33
├─ WHY THIS MATTERS: Repetitive validation pattern detected
   (entropy 0.33, 20 checks). Low entropy (0.33) indicates
   boilerplate, not complexity. Adjusted complexity: 21 → 13
   (reflects actual cognitive load). Refactoring improves
   maintainability and reduces error-prone boilerplate.
├─ RECOMMENDED ACTION: Replace 20 repetitive validation checks
   with declarative pattern

Debtmap's verdict: reconcile_state() has higher priority than validate_config().

Notice what debtmap detects that Codacy completely misses:

Pattern Recognition:
- Identifies the "Coordinator pattern" in reconcile_state() with 4 actions and 2 state comparisons
- Detects the "Repetitive validation pattern" (entropy 0.33) in validate_config()
Cognitive Load Adjustment:
- validate_config(): Dampens complexity from 21 → 13 (50% reduction) because it's "boilerplate, not complexity"
- reconcile_state(): Recognizes high cognitive load (16) despite lower cyclomatic complexity
Actionable Recommendations:
- State reconciliation: "Extract state reconciliation logic into transition functions" (specific architectural guidance)
- Validation: "Replace 20 repetitive validation checks with declarative pattern" (suggests the right refactoring approach)
Impact Quantification:
- Shows exactly how much complexity reduction to expect: 9/16 → 4/11 for the coordinator pattern
- Validation impact is higher (-10 complexity) but lower priority because it's mechanical, not cognitive

This is the fundamental difference: Traditional tools measure branch count. Debtmap measures cognitive difficulty and provides architectural insight.

How Debtmap Is Different

Debtmap takes a fundamentally different approach by combining multiple analysis techniques:

1. Entropy-Based Complexity

Debtmap uses information theory to measure code complexity. Specifically, it calculates the entropy (unpredictability) of code patterns.

High entropy (0.7-1.0):

Diverse, unpredictable logic
Many different code paths doing different things
Actually complex—hard to reason about

Low entropy (0.0-0.3):

Repetitive patterns
Predictable structure
Mechanically complex but easy to understand

Let's see this in action on our validation function (actual output from samples):

#2 SCORE: 3.10 [LOW]
├─ LOCATION: ./src/validation.rs:30 validate_config()
├─ IMPACT: -10 complexity, -1.0 risk
├─ COMPLEXITY: cyclomatic=21 (dampened: 10, factor: 0.50),
   est_branches=21, cognitive=6, nesting=1, entropy=0.33
├─ WHY THIS MATTERS: Repetitive validation pattern detected
   (entropy 0.33, 20 checks). Low entropy (0.33) indicates
   boilerplate, not complexity. Adjusted complexity: 21 → 13
   (reflects actual cognitive load).
├─ RECOMMENDED ACTION: Replace 20 repetitive validation checks
   with declarative pattern

Debtmap recognizes the repetitive pattern (entropy=0.33) and dampens the complexity by 50% (21 → 10), understanding that this is "boilerplate, not complexity." Despite the high branch count, it gets a LOW priority score of 3.10.

Compare with the state reconciliation function (actual output from samples):

#1 SCORE: 4.15 [MEDIUM]
├─ LOCATION: ./src/state_reconciliation.rs:81 reconcile_state()
├─ IMPACT: -4 complexity, -1.5 risk
├─ COMPLEXITY: cyclomatic=9 (dampened: 4, factor: 0.51),
   est_branches=9, cognitive=16, nesting=4, entropy=0.28
├─ WHY THIS MATTERS: Coordinator pattern detected with 4 actions
   and 2 state comparisons. Extracting transitions will reduce
   complexity from 9/16 to ~4/11.
├─ RECOMMENDED ACTION: Extract state reconciliation logic into
   transition functions

Debtmap identifies this as higher priority than the validation function, even though it has much lower cyclomatic complexity (9 vs 21). Debtmap detects:

Coordinator pattern - 4 actions with 2 state comparisons (architectural smell)
High cognitive complexity (16) - nested conditionals are genuinely hard to reason about
Deep nesting (4 levels) - increases mental load
Greater risk impact (-1.5) - complex state transitions are error-prone

The validation function has higher mechanical complexity, but the state reconciliation has higher cognitive complexity.

2. Coverage-Risk Correlation

Debtmap is unique in natively combining complexity analysis with test coverage to compute risk scores.

Why this matters:

Complex code with good tests = managed risk (lower priority)
Simple code without tests = low risk
Complex code without tests = CRITICAL GAP

Example output:

#5 SCORE: 18.4 [CRITICAL]
├─ LOCATION: src/analyzer.rs:805
├─ FUNCTION: parse_error_flow()
├─ COMPLEXITY: cyclomatic=21 (adj:12), cognitive=63, nesting=5
├─ TEST COVERAGE: 0%
├─ IMPACT: -10 complexity reduction, -6.4 risk reduction
└─ WHY THIS MATTERS: High complexity 21/63 makes function
   hard to test and maintain. Error handling without tests
   creates critical risk.

Debtmap prioritizes untested complex code over tested complex code, because untested complexity is what creates risk.

3. Actionable Recommendations with Impact Quantification

Most tools tell you what is wrong. Debtmap tells you what to do and what impact it will have.

Traditional tools typically report:

Function 'process_request' has complexity 15 (threshold: 10)
Severity: Major

Debtmap instead provides:

#1 SCORE: 181 [CRITICAL]
└─ type_tracker/mod.rs (3096 lines, 106 functions)
└─ WHY: This struct violates single responsibility principle
   with 42 methods and 12 fields across 8 distinct responsibilities.
└─ ACTION: Split by data flow:
   1) Input/parsing functions
   2) Core logic/transformation
   3) Output/formatting

   RECOMMENDED SPLITS (2 modules):
   - mod_parsing.rs (6 methods, ~120 lines)
   - mod_utilities.rs (22 methods, ~440 lines)

   IMPLEMENTATION ORDER:
   [1] Start with lowest coupling modules
   [2] Move 10-20 methods at a time, test after each move
   [3] Keep original file as facade during migration

└─ IMPACT: Reduce complexity by 80%, improve testability,
   enable parallel development

Debtmap provides:

Specific location (file:line, function name)
Root cause analysis (8 responsibilities, god object)
Concrete refactoring plan (split into 2 modules)
Implementation strategy (order of operations)
Quantified impact (80% complexity reduction)
Expected benefits (testability, parallel development)

4. Context-Aware Analysis

Debtmap understands that code doesn't exist in isolation:

Call Graph Analysis:

execute_workflow()
├─ Called by: 15 functions
├─ Calls: 8 functions
├─ Risk multiplier: 2.3x (widely used)
└─ Priority: CRITICAL (changes affect many callers)

Functions called from many places get higher priority—breaking them affects more of the codebase.

Pattern Detection:

#7 SCORE: 9.12 [CRITICAL]
└─ PathResolver::resolve_qualified_path()
└─ WHY: Function calls 15 methods on external objects,
   only 2 on self
└─ PATTERN: Feature Envy - consider moving to the objects
   being manipulated

Debtmap recognizes architectural anti-patterns like God Objects and Feature Envy, providing architectural insights beyond function-level analysis.

Example: What Debtmap Reports

Here's what debtmap shows on a 120K LOC Rust codebase:

TOP 10 RECOMMENDATIONS

#1 SCORE: 181 [CRITICAL]
└─ type_tracker/mod.rs (3096 lines, 106 functions)
└─ WHY: God object with 42 methods across 8 responsibilities
└─ ACTION: Split into 2 focused modules (detailed plan)
└─ IMPACT: Reduce complexity by 80%

#2 SCORE: 108 [CRITICAL]
└─ detector.rs (2882 lines, 113 functions)
└─ ACTION: Split by analysis phase (detailed plan)

... (8 more prioritized items)

TOTAL DEBT SCORE: 1887
DEBT DENSITY: 15.7 per 1K LOC
OVERALL COVERAGE: 81.33%

Developer experience: "Okay, I'll tackle #1 this sprint. Clear plan, measurable impact."

The Debtmap Philosophy

Debtmap is designed to answer one question: "What technical debt should I tackle first?"

Unlike tools that flood you with hundreds of alerts, debtmap focuses on:

Quality over quantity - 10-20 high-impact items instead of 200+ noise
Risk-based prioritization - Combining complexity with test coverage
Actionable guidance - Specific refactoring plans, not just warnings
Measurable impact - Quantified risk reduction for each recommendation

Recommended Workflow

Debtmap is designed to complement your existing development workflow:

# 1. Local development loop (before commit)
cargo fmt                    # Format code
cargo clippy                 # Check idioms
cargo test                   # Run tests
debtmap analyze .            # Check for new debt

# 2. CI/CD pipeline (PR validation)
cargo test --all-features    # Full test suite
debtmap validate .           # Enforce debt thresholds

# 3. Weekly planning (prioritize work)
cargo tarpaulin --out lcov   # Generate coverage
debtmap analyze . --lcov target/coverage/lcov.info --top 20
# Review top 20 debt items, plan sprint work

# 4. Monthly review (track trends)
debtmap compare baseline.json current.json

Performance

Written in Rust with parallel processing:

1-2 seconds for small projects (<10K LOC)
20-30 seconds for large codebases (100K+ LOC)
10-100x faster than Java/Python-based competitors

Try It Yourself

# Install debtmap
cargo install debtmap

# Analyze your project
debtmap analyze .

# With test coverage
cargo tarpaulin --out lcov
debtmap analyze . --lcov target/coverage/lcov.info

Resources

Learn more about Debtmap: Project Page
GitHub: github.com/iepathos/debtmap
Documentation: iepathos.github.io/debtmap
Crates.io: crates.io/crates/debtmap

Debtmap is open source and free. No enterprise licenses, no usage limits, no lock-in. Just better code quality analysis for everyone.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Transforming ripgrep's Documentation with AI Automation and MkDocs

Glen Baker — Tue, 18 Nov 2025 02:21:16 +0000

Originally published on Entropic Drift

From Two Files to Comprehensive Documentation

ripgrep is one of the most popular command-line search tools, with over 57,000 stars on GitHub. Despite its popularity and rich feature set, the documentation consisted of just two files: a feature-focused README and a tutorial-style GUIDE. While these files were well-written, they left gaps:

No structured learning path from basics to advanced features
Limited visual aids to explain complex concepts
Features scattered across different sections
No dedicated troubleshooting or reference sections

Using an evolved version of my documentation automation workflow (originally built for Prodigy's documentation), I generated 50+ pages of enhanced documentation, now live at https://iepathos.github.io/ripgrep.

This post documents how the workflow evolved to handle:

Intelligent page splitting - Breaking monolithic chapters into focused subpages
Automated visual enhancement - Adding diagrams, admonitions, and annotations where they help
Mermaid diagram validation - Ensuring all generated diagrams render correctly

What Changed: MkDocs vs mdBook

The original workflow used mdBook, a simple static site generator popular in the Rust ecosystem. MkDocs Material offers significantly more features:

Feature	mdBook	MkDocs Material
Mermaid Diagrams	Plugin required	Native support
Admonitions	Limited	12+ styled types
Code Annotations	No	Yes (numbered callouts)
Tabbed Content	No	Yes
Navigation Tabs	No	Yes
Search	Basic	Advanced with highlighting
Theme Customization	Limited	Extensive

The Enhanced Workflow Architecture

The ripgrep workflow introduced critical improvements over previous iterations. The complete flow:

Setup Phase:
- Analyze codebase features
- Detect documentation gaps and create stub pages
- Analyze page sizes and structural complexity
- Automatically split oversized pages into focused subpages
- Auto-discover all markdown files (including newly split pages)
Map Phase (per-page):
- Analyze page for drift (subsection-aware)
- Fix detected drift with validation
- Enhance with visual features (diagrams, admonitions, annotations)
Reduce Phase (holistic):
- Validate with strict build (mkdocs build --strict)
- Check structure and feature consistency
- Validate Mermaid diagrams with official renderer
- Auto-fix any issues found

Full workflow YAML available on GitHub

Four Key Innovations

1. Intelligent Page Splitting

The breakthrough feature: automatic page splitting based on structural analysis.

The original GUIDE.md was over 1,400 lines—a monolithic wall of text. The workflow now analyzes page structure before processing and automatically splits oversized pages into focused subpages based on:

Line count and section depth (>300 lines or >5 major sections triggers splitting)
Topic cohesion within sections
Natural split points at heading boundaries

For ripgrep, this transformed:

GUIDE.md (1,400 lines) → basics/ directory with 8 focused pages
Binary Data section (350 lines) → binary-data/ directory with 5 specialized pages
Troubleshooting → troubleshooting/ directory with 6 topic-specific pages

Benefits:

Context-aware agents: Each agent works on focused topics, not overwhelming monoliths
Better enhancement: Visual features applied to coherent topics
Improved navigation: Readers find specific information faster
Maintainability: Smaller pages easier to update

2. Exhaustive Page Discovery

After page splitting creates new files, we ensure every page gets processed with direct filesystem scanning:

find $DOCS_DIR -name "*.md" -type f | jq -R -s 'split("\n") | map(select(length > 0)) | ...'

This is critical because:

Comprehensive coverage: Every .md file processed, including newly split pages
No manual tracking: Filesystem is source of truth
Orphaned pages caught: Pages that exist but aren't linked are discovered
Deterministic: Same files every time

3. Visual Enhancement Per Page

After drift is fixed, each page gets enhanced in the map phase (not reduce), meaning each agent has full context about what the page covers:

Mermaid diagrams for architecture and workflow visualization
Admonitions (warnings, tips, notes) for important callouts
Code annotations with numbered inline explanations
Tabbed content for alternative approaches

The AI attempts context-aware decisions: "Pattern matching is decision-heavy, add a decision tree. Installation varies by platform, use tabbed content."

4. Mermaid Diagram Validation with Mermaid-Sonar

When you generate dozens of Mermaid diagrams automatically, some will have syntax errors and many will be too complex to be readable. What started as a simple syntax validation step in the workflow evolved into a standalone tool: mermaid-sonar.

The evolution: Initially, the workflow used @mermaid-js/mermaid-cli to catch syntax errors before build. But I noticed a pattern: AI agents would create syntactically valid diagrams that were cognitively overwhelming. An 80-node decision tree for a binary choice. A complex flowchart for a 3-item list. Diagrams that rendered correctly but hurt comprehension.

Syntax validation alone wasn't enough. I needed complexity validation to guide AI toward readable diagrams.

Mermaid-sonar uses static code analysis and research-backed heuristics to measure diagram complexity and provide actionable feedback to AI agents:

- shell: "mermaid-sonar $DOCS_DIR --strict"
  on_failure:
    claude: "/prodigy-fix-mermaid-diagrams ${shell.output}"

What makes it effective for AI workflows:

Static analysis: Parses diagram source code, calculates graph metrics (nodes, edges, density, branching)
Research-backed thresholds: Based on cognitive load research (50/100 node limits), graph theory, and Mermaid performance characteristics
Specific feedback: Instead of "too complex," provides "12 parallel branches (>8 max) → use LR layout or split"
Fast execution: No rendering required, optimized for CI/CD loops
Machine-readable output: JSON format allows AI agents to understand and fix issues automatically

The validation caught 8 broken diagrams in ripgrep's docs that would have failed during build, plus 12 overly complex diagrams that needed simplification.

Complexity issues detected:

Diagrams with >50 nodes (high-density) or >100 nodes (low-density)
Wide tree diagrams with >8 parallel branches
Graphs exceeding 100 connections (Mermaid's O(n²) limit)
High graph density (>0.3) indicating visual clutter
Layout mismatches (TD layout for wide trees, LR for deep hierarchies)

The key insight: AI agents need quantitative feedback to understand "too complex." By converting subjective readability into measurable metrics with clear thresholds, mermaid-sonar guides AI toward diagrams that actually help readers instead of overwhelming them.

This is the difference between "catch errors" and "guide toward quality." Traditional validation tells you what's broken. Heuristic-based validation tells you what needs improvement and how to improve it—feedback AI agents can act on automatically.

Real Results: ripgrep Documentation Transformation

Before:

2 documentation files (README.md, GUIDE.md)
~1,800 total lines of markdown
No visual diagrams
Limited structure (linear reading path)

After:

55 focused documentation pages
Organized into 7 major sections
87 Mermaid diagrams explaining workflows and decision trees
358 admonitions highlighting warnings and tips
180+ code annotations with inline explanations
Tabbed installation instructions for different platforms
100% strict build success - no broken links or invalid syntax

Example: Pattern Matching Page Transformation

Before (from GUIDE.md): Plain text explaining regex vs literal patterns, scattered across multiple sections.

After (docs/basics/pattern-matching.md):

Added decision tree diagram helping users choose between literal and regex patterns, converted important notes to styled admonitions (tips for when to use -F flag), and annotated code examples with numbered callouts explaining each flag's purpose.

The page went from "walls of text" to "guided learning."

Key Insights

1. Page Splitting Unlocks Comprehensive Documentation

The most impactful innovation wasn't visual enhancements—it was automatic page splitting. No reader wants to scroll through 1,400 lines to find information about case sensitivity. The workflow handles splitting automatically by:

Identifying oversized pages (>300 lines or >5 major sections)
Determining logical split points (section boundaries)
Creating focused subpages with proper hierarchy
Updating cross-references and navigation

Documentation that expands organically from codebase analysis becomes navigable and learnable, not an overwhelming wall of text.

2. Visual Features Need Intelligence

You can't apply visual enhancements mechanically. A rule-based system would either over-apply (cluttering simple pages) or under-apply (missing opportunities). AI agents with context make nuanced decisions about which features help specific content.

3. Complexity Validation Catches AI Over-Engineering

Syntax validation alone isn't enough for AI-generated diagrams. AI agents tend to over-engineer visualizations, creating elaborate flowcharts for simple categorizations. Mermaid-sonar's complexity analysis catches these issues automatically by measuring:

Cognitive load: Research shows 50-node limit for readable high-density graphs
Visual hierarchy: Detects wide trees (>8 branches) that should use horizontal layout
Performance limits: Flags graphs approaching Mermaid's O(n²) rendering threshold

Automated complexity validation → automated fixes → readable diagrams. Manual review still catches poor diagram type choices, but complexity issues are handled automatically.

4. Per-Page Enhancement Beats Bulk Processing

Running enhancement in the map phase (per-page) instead of reduce phase (bulk) means each agent has full context about what the page covers. Decisions are informed by content, not position. The extra parallelization complexity is worth it.

Comparing the Three Generations

Aspect	Debtmap (Original)	Prodigy Book-Docs	ripgrep MkDocs
Granularity	Chapter-level only	Subsection-aware (H2, H3)	Subsection-aware
Page Structure	Fixed (manual SUMMARY.md)	Fixed (manual SUMMARY.md)	Dynamic (automatic splitting)
Page Discovery	Curated chapters.json	Gap detection → flattened-items.json	Gap detection + splitting + filesystem scan
Validation	None	Doc fix + holistic validation	Page + structure + consistency + Mermaid
Visual Enhancement	None	None	Diagrams, admonitions, annotations
Pages Generated	27 (curated)	47 (gap-detected)	55 (split + discovered)
Diagrams	0	0	87 Mermaid diagrams

The evolution: accuracy → precision → transformation.

How to Adapt This Workflow

Want to build something similar? Here's the condensed path:

1. Start with MkDocs Material

Configure with features you want: admonitions, code annotations, Mermaid diagrams, tabbed content.

2. Build Auto-Discovery

Use find and jq to discover all markdown files as your source of truth.

3. Create Enhancement Logic

Write prompts that analyze page content and add:

Mermaid diagrams for architecture/workflows
Admonitions for warnings/tips/notes
Code annotations for complex examples
Tabs for alternative approaches

4. Add Multi-Layer Validation

Per-page (map phase): Each page meets quality standards
Build validation (reduce): mkdocs build --strict
Consistency (reduce): Pages use similar enhancements
Diagram validation: Verify Mermaid syntax with mermaid-sonar docs/ --strict

5. Iterate and Refine

Run the workflow, review output, adjust. Because agents enhance existing documentation rather than replacing it, manual improvements become context for the next run. The documentation becomes iterative: agents enhance, you refine, agents use refinements as context.

Full implementation details in the GitHub workflow

Future Enhancements

The workflow is production-ready, but opportunities abound:

Screenshot Management: Detect when UI screenshots are outdated and regenerate using browser automation.

Interactive Examples: Generate runnable code examples with embedded outputs.

Version-Specific Documentation: Integrate with mike to maintain documentation for multiple versions automatically. When a new version releases, run drift workflow against the new codebase and generate version-specific docs with accurate examples.

Conclusion

By combining intelligent page splitting, auto-discovery, subsection-aware drift detection, per-page visual enhancement, multi-layer validation, and strict build enforcement, we've created a workflow that can help transform a project's minimal documentation into more comprehensive guides with AI assistance.

The ripgrep documentation demonstrates this transformation in production. Every page split, diagram, admonition, and annotation was generated automatically from the original README, GUIDE, and code. The workflow preserved the authors' clarity and intent while making it dramatically more accessible.

The docs aren't perfect. We've only begun to codify the rules and heuristics needed to guide AI to mimic a skilled technical writer. This is just an early run of this workflow on a third party open source library to explore the limits of AI documentation automation. At the end of this journey, we hope to have an open source solution that anyone can use to produce high quality docs for code bases.

The pattern is reusable. Point it at any project with well-written code, and watch it transform into comprehensive guides with visual aids and structured learning paths.

Resources

Live Documentation Examples:

ripgrep Docs (transformed from 2 files): https://iepathos.github.io/ripgrep/
Prodigy Docs (MkDocs Material): https://iepathos.github.io/prodigy/
Debtmap Docs (mdBook): https://iepathos.github.io/debtmap/

Blog Posts:

Original automation case study: Automating Documentation Maintenance with Prodigy

Workflows (showing evolution):

Generation 1: Debtmap book-docs-drift.yml (chapter-level)
Generation 2: Prodigy book-docs-drift.yml (subsection-aware)
Generation 3: ripgrep mkdocs-drift.yml (page splitting + visual enhancements + diagram validation)

Tools:

Prodigy: GitHub | Crates.io
Debtmap: GitHub | Crates.io
Mermaid-Sonar: GitHub | npm
ripgrep: GitHub | Fork with MkDocs
MkDocs Material: https://squidfunk.github.io/mkdocs-material/

This blog post documents the third generation of the documentation automation workflow. The first generation (Debtmap) proved the concept at the chapter level. The second generation (Prodigy book-docs) added subsection precision. This third generation (ripgrep mkdocs) adds intelligent page splitting, visual engagement, and diagram validation. The workflow transformed ripgrep's 2-file documentation into 55+ comprehensive pages—proving that AI automation can enhance existing open source projects without replacing their authors' expertise.

Want more content like this? Follow me on Dev.to or subscribe to Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Automating Documentation Maintenance with Prodigy: A Real-World Case Study

Glen Baker — Sat, 01 Nov 2025 03:26:51 +0000

Originally published on Entropic Drift

The Documentation Drift Problem

Every developer knows the pain: you ship a new feature, refactor your code, update configuration options—and your documentation slowly falls out of sync. What starts as a minor discrepancy becomes a maze of outdated examples, incorrect defaults, and missing features that frustrate users and waste maintainer time.

For Debtmap, a Rust-based technical debt analyzer, this problem was particularly acute. I was iterating quickly so the docs I'd have one day were wrong a week or a few days later. Keeping documentation aligned with a rapidly evolving codebase felt like a Sisyphean task.

The solution? I built an automated workflow using Prodigy that detects and fixes documentation drift. Using it for helping me maintain documentation in Debtmap Documentation It let me shift from a basic README.md to 27 comprehensive book chapters covering everything from CLI reference to advanced architectural analysis.

What is Prodigy?

Prodigy is an AI-powered workflow automation system that orchestrates complex multi-step tasks using a map-reduce architecture. Think of it as CI/CD for AI agents: you define workflows as declarative YAML files and Claude commands, and Prodigy handles parallel execution, error recovery, merging, and validation.

The key innovation is agent-based map-reduce: instead of just running shell commands, each work item gets its own AI agent that can analyze code, understand context, make decisions, and apply fixes—all while maintaining clean git history and handling failures gracefully.

The Workflow Architecture

Here's the high-level structure of the documentation drift detection workflow:

name: prodigy-book-docs-drift-detection
mode: mapreduce

# Setup: Analyze codebase and detect gaps
setup:
  - Analyze codebase for features, commands, and config options
  - Detect documentation gaps and create missing chapters

# Map: Process each chapter in parallel
map:
  input: "workflows/data/prodigy-chapters.json"
  json_path: "$.chapters[*]"

  agent_template:
    - Analyze chapter for drift against current implementation
    - Fix detected drift issues

  max_parallel: 3

# Reduce: Validate and merge
reduce:
  - Build the book to ensure all chapters compile
  - Clean up temporary analysis files

Three-Phase Approach

1. Setup Phase: Feature Discovery

Before we can detect drift, we need to know what the current codebase actually supports. The workflow:

Scans the codebase for workflow features, command types, and configuration options
Extracts actual implementation details (not just what the docs claim)
Identifies completely missing chapters for undocumented features
Creates a feature inventory at .prodigy/book-analysis/features.json

This phase uses custom Claude Code slash commands like /prodigy-analyze-features-for-book that encapsulate domain-specific analysis logic.

2. Map Phase: Parallel Drift Analysis

Each of the 27 chapters gets its own dedicated AI agent that:

Analyzes drift - Compares chapter content against the feature inventory
- Detects outdated examples
- Finds incorrect default values
- Identifies missing features
- Checks configuration option accuracy
Fixes drift - Applies corrections while preserving style
- Updates code examples
- Corrects configuration defaults
- Adds missing sections
- Maintains existing prose quality

The agents run in parallel (3 at a time to balance throughput and resource usage), each working in isolated git worktrees to prevent conflicts.

3. Reduce Phase: Validation

After all chapters are fixed:

Rebuild the book with mdbook build to catch broken links
Validate cross-chapter consistency
Merge all changes back to the main branch
Clean up temporary analysis artifacts

If the book build fails, an agent automatically diagnoses and fixes the issues.

Real Results:

I just ran this workflow on Debtmap's documentation. Here's what happened:

Execution Metrics

27 chapters analyzed and fixed
122 drift analysis reports generated (avg 4.5 per chapter)
94 fix commits applied
100% completion rate - zero permanent failures
Book built successfully - all chapters compile together

Example: The Configuration Chapter

One representative fix (commit 9ad9445a):

Fixed 10 drift issues (0 critical, 6 medium, 4 low)

Key updates:
- Updated role coverage weights (pure_logic: 1.0, io_wrapper: 0.5)
- Added documentation for debug role multiplier
- Added deprecation warnings for obsolete threshold fields
- Fixed max_total_debt_score default (1000 → 10000)
- Added classification configuration section
- Documented advanced configs (orchestration_adjustment, etc.)

The agent didn't just find surface-level issues—it caught semantic drift like a 10x error in a default value that would have confused users.

Example: The Analysis Guide Chapter

Another fix (commit c0e1fe9e):

Fixed 9 drift issues (all low severity)

Key updates:
- Corrected debt type count (24 → 25)
- Added est_branches field with formula explanation
- Clarified minimal_count vs legacy risk categories
- Enhanced unified JSON format documentation

These are the kinds of subtle discrepancies that accumulate over time and degrade documentation quality.

The Multi-Pass Refinement Strategy

You might notice something interesting: 122 analyses for 27 chapters means an average of 4.5 analysis passes per chapter. This isn't a bug—it's a feature.

The workflow uses an iterative refinement approach:

First pass: Broad drift detection
Second pass: Verify fixes and catch edge cases
Subsequent passes: Refine until no drift remains

Each pass generates a detailed drift report that guides the next round of fixes. This ensures comprehensive coverage without requiring perfect first-pass detection.

The workflow stops when drift severity drops below thresholds or all issues are resolved—whichever comes first.

Error Handling and Recovery

Real-world workflows encounter failures. Prodigy handles this with:

error_policy:
  on_item_failure: dlq        # Dead letter queue for failures
  continue_on_failure: true    # Don't stop the whole workflow
  max_failures: 2              # Reasonable fault tolerance
  error_collection: aggregate  # Collect all errors for analysis

In this run: zero items reached the DLQ. All chapters were successfully analyzed and fixed, demonstrating the robustness of the agent-based approach.

When issues do occur, agents can often self-recover. For example, if a book build fails due to broken links, the reduce phase automatically spawns an agent to diagnose and fix the build errors.

The Git History Tells the Story

Each step leaves a clear audit trail:

07541ed7 Merge branch 'prodigy-session-654c3786-e04d-44ce-bfc0-600817dbb9ea'
d013e30e Merge agent agent-mapreduce-20251030_202918_agent_3-item_3
c0e1fe9e docs: fix Debtmap book chapter 'Analysis Guide'
32a43c20 analysis: drift report for Debtmap book chapter 'Analysis Guide'

The pattern is consistent:

Analysis commit with drift report
Fix commit with detailed changes
Agent merge commits for parallel work
Final session merge

This creates a reviewable, bisectable history where every change is justified by a corresponding analysis.

Key Learnings

1. Feature Inventory is Critical

You can't detect drift without ground truth. Investing in comprehensive codebase analysis upfront pays dividends in detection accuracy.

2. Iterative Refinement Beats One-Shot Analysis

Multi-pass refinement caught issues that a single analysis missed. The extra compute cost is worth it for thoroughness.

3. Parallel Processing Scales

Processing 27 chapters serially would have taken hours. With max_parallel: 3, the entire workflow completed in a reasonable timeframe while staying within resource constraints.

4. Agent Autonomy Reduces Toil

Instead of reviewing 122 drift reports manually, I let agents fix the issues and reviewed the git diffs. This inverted the effort: AI does the tedious work, humans do the strategic review.

5. Validate Everything

The book build step caught integration issues that per-chapter analysis missed. Always include cross-cutting validation in your reduce phase.

Practical Benefits

Since implementing this workflow:

Confidence in documentation - I can ship code changes knowing docs will be updated automatically
Reduced maintenance burden - No more manual doc review marathons
Faster releases - Documentation updates don't block releases
Better user experience - Examples actually work, defaults are correct
Continuous improvement - Run the workflow on every release or schedule it weekly

How to Adapt This for Your Project

The workflow is generalizable. Here's how to adapt it:

1. Define Your Chapters

Create a JSON inventory of your documentation:

{
  "chapters": [
    {
      "id": "getting-started",
      "title": "Getting Started",
      "file": "docs/getting-started.md",
      "topics": ["Installation", "Quick start", "First example"],
      "validation": "Check installation commands are current"
    }
  ]
}

2. Build Feature Extraction

Write a slash command (or script) that extracts your project's current state:

CLI commands and options
Configuration schema
API endpoints
Supported features

3. Create Drift Detection Logic

The /prodigy-analyze-book-chapter-drift command compares chapter content against the feature inventory. You can implement this as:

Custom slash command with domain logic
Generic AI agent with clear instructions
Combination of static analysis + AI refinement

4. Define Fix Standards

Give your agents clear guidelines:

Preserve existing writing style
Update only factual inaccuracies
Add missing sections for new features
Flag breaking changes for human review

5. Add Validation

Your reduce phase should validate:

Documentation builds successfully
Cross-references are valid
Code examples compile/run
No broken links

Future Enhancements

This workflow is already production-ready, but there's room for improvement:

Proactive gap detection: Alert when new code is committed without corresponding docs
Version-specific docs: Generate documentation for multiple versions automatically
Interactive examples: Validate that code examples actually execute correctly
Screenshot updates: Detect when UI screenshots are outdated
Multi-language support: Maintain docs in multiple languages with translation drift detection

Conclusion

Documentation drift is a systemic problem that can't be solved by individual discipline—it requires systemic automation. By treating documentation as code and applying AI-powered map-reduce workflows, we can maintain high-quality, accurate documentation without manual toil.

The Prodigy workflow I've described isn't theoretical—it's running in production on Debtmap and Prodigy right now, keeping chapters synchronized with fast-moving codebases. The same approach can work for your project.

The era of stale documentation is over. Automate it, validate it, and ship with confidence.

Resources

Debtmap: Learn more about Debtmap | GitHub | Crates.io
Prodigy: Learn more about Prodigy | GitHub | Crates.io
Workflow file: book-docs-drift.yml
Example drift analysis: Commit 32a43c20

Get Started

Want to automate your documentation maintenance? Check out the Prodigy project page to learn how to set up AI-powered workflows for your team.

Interested in technical debt analysis? Explore Debtmap to see how entropy-based complexity metrics can help you prioritize refactoring.

About the Author

I'm Glen Baker, building Debtmap to help teams tackle technical debt systematically, and Prodigy to automate the kind of repetitive, high-stakes work that developers shouldn't have to do manually. If you're interested in AI-powered development workflows, read more on my blog or get in touch.

This blog post was written after a successful production run of the automated documentation workflow, demonstrating that the system works as advertised—dogfooding at its finest.

Want more content like this? Follow me on Dev.to or on Entropic Drift for posts on AI-powered development workflows, Rust tooling, and technical debt management.

Check out my open-source projects:

Debtmap - Technical debt analyzer
Prodigy - AI workflow orchestration

Forem: Glen Baker

Debtmap Re-adds JavaScript and TypeScript Support

What It Detects

Entropy Dampening

Usage

What It Doesn't Do

Links

Refined Types in Rust: Parse, Don't Validate

The String Problem

Validation Everywhere

Parse, Don't Validate

Refined Types

The Practical Benefits

1. Validation Happens Once

2. Invalid States Become Unrepresentable

3. Function Signatures Document Requirements

4. Refactoring Becomes Safer

Building Refined Types with Stillwater

Domain-Specific Types with Stilltypes

Composition: Multiple Constraints

Serde Integration

The Boundary Pattern

Error Accumulation

When Not to Use Refined Types

The Type-Driven Design Mindset

Getting Started

Summary

Stillwater 1.0: Pragmatic Effect Composition and Validation for Rust

What is Stillwater?

What Are Effects?

The Core Problem

Key Features in 1.0

Zero-Cost Effect System

Validation with Error Accumulation

Predicate Combinators

Refined Types

Bracket Pattern for Resource Management

Compile-Time Resource Tracking

Retry Policies as Data

Writer Effect for Audit Trails

Parallel Effect Execution

Production Ready

Philosophy

When to Use Stillwater

The Ecosystem

Getting Started

What's Next

Compile-Time Resource Tracking in Rust: From Runtime Brackets to Type-Level Safety

The Problem: Runtime Resource Leaks

The Foundation: Runtime Brackets

Stillwater 0.14.0: Type-Level Resource Tracking

The Bracket Pattern: Guaranteed Resource Neutrality

Protocol Enforcement: Database Transactions

Tracking Multiple Resources

Compile-Time Assertions

Custom Resource Kinds

Zero Runtime Overhead

Comparison: RAII vs Bracket vs Type-Level Tracking

When to Use This

Getting Started

Summary

Refactoring a God Object Detector That Was Itself a God Object

The Irony

When Your Tool Turns on You

Automating the Refactoring with Prodigy

The Numbers

Phase 1: Foundation Analysis (Spec 181a)

Phase 2: Extract Types and Thresholds (Spec 181b)

Phase 3: Extract Pure Scoring Functions (Spec 181c)

Phase 4: Extract Detection Predicates (Spec 181d)

Phase 5: Extract Classification Logic (Spec 181e)

Phase 6: Extract Recommendation Logic (Spec 181f)

Phase 7: Create Orchestration Layer (Spec 181g)

Phase 8: Update Public API (Spec 181h)

Phase 9: Complete Pure Function Migration (Spec 181i)

Key Takeaways

1. Pure Functions Make Testing Trivial

2. Separation Enables Composition

3. Module Size Matters

4. Break Down Big Problems, Then Automate