Forem: Codequal

I Analyzed 10 Major Open-Source Repos, and Every Single One Had Significant Drift

Codequal — Tue, 24 Mar 2026 00:57:06 +0000

Google's own DORA research shows a paradox: AI tools increase developer throughput while decreasing delivery stability.

I wanted to understand why. So I built an open-source CLI that detects development process drift, and ran it against 10 of the most widely-used repos in the industry: React, Next.js, VS Code, AWS CDK, Google Cloud Python, Supabase, LangChain, Stripe Node, Cloudflare Workers SDK, and Plaid.

Every single one had significant drift signals across multiple dimensions. Here's what I found.

CI Builds Are Silently Exploding

Every project showed elevated CI/build times. Not failures — the builds still pass. They just take dramatically longer than baseline, and nobody notices because there's no alarm for "your CI is 100x slower than it was 3 months ago."

Project	Build Duration vs Baseline
Google Cloud Python	1,552x
Stripe Node	1,361x
VS Code	946x
LangChain	889x
Supabase	611x
AWS CDK	112x
Next.js	104x
Cloudflare Workers	75x
React	13x
Plaid	3.2x

The median spike across all 10 repos is ~100x baseline. These aren't flaky tests or broken builds — they're passing builds that silently consume 100x more compute than they used to.

File Change Explosions Reveal Architectural Drift

When a single commit touches thousands of files across unrelated directories, something structural has shifted. This metric — file dispersion combined with files changed — was elevated in all 10 projects.

Project	Files Changed vs Baseline
AWS CDK	14,464x
VS Code	13,593x
Google Cloud Python	9,113x
Next.js	215x
React	208x
Stripe Node	150x
LangChain	87x
Supabase	67x
Plaid	65x
Cloudflare Workers	52x

The root causes vary — Google Cloud Python has Librarian bots mass-updating image dependencies, Next.js has Turbopack switching chunk hashes from hex to base40, Plaid has OpenAPI code generation. But the pattern is universal: large automated changes that no human reviews file by file.

Co-Change Novelty Drops to Zero

This is the most subtle and arguably most important signal. "Co-change novelty" measures whether new combinations of files are changing together, or whether the same files keep getting modified as a group.

When novelty drops, it means development has become pattern-locked — the same templates, the same file groups, the same automated workflows touching the same paths. It's the fingerprint of bot-driven or template-driven development.

9 out of 10 repos showed depressed co-change novelty:

Project	Co-change Novelty vs Baseline
AWS CDK	587x below
Supabase	587x below
VS Code	547x below
Next.js	509x below
Google Cloud Python	439x below
React	109x below
Cloudflare Workers	105x below
LangChain	88x below
Plaid	29.8x above (OpenAPI generates novel pairings)

Release Cadence Is All Over the Map

Deployment frequency — one of the four DORA metrics — showed extreme variance:

Project	Release Cadence Deviation
Google Cloud Python	2,548,259x
Cloudflare Workers	694,167x
React	69,548x
Supabase	229x
LangChain	220x
VS Code	104x
Next.js	68x
AWS CDK	13x
Stripe Node	11x
Plaid	N/A

These numbers look absurd, but they reflect real gaps between releases — periods where code accumulates without shipping, then gets batch-released. That accumulation is where drift compounds.

Cross-Family Correlations: The Hidden Connections

The most interesting findings come from correlating signals across different dimensions. These patterns are invisible if you only monitor one signal at a time:

CI duration correlates with files touched — found in all 10 repos. More scattered commits = longer builds. Obvious in retrospect, but nobody tracks it.
Release cadence correlates with code dispersion — found in 8/10 repos. When releases slow down, code changes spread across unrelated areas.
Dependency growth correlates with build time — VS Code (283x dependency growth + 946x build growth), Cloudflare (30x + 75x).

What This Means

No single commit in any of these projects looks wrong. Tests pass. Reviews approve. The code is correct.

But the aggregate pattern drifts. Files spread. Builds slow. Releases stall. Dependencies grow. And because each change is individually fine, nobody raises an alarm.

This is the drift problem. And it affects every project at scale — even the ones built by the best engineering organizations in the world.

It Pinpoints the Exact Commit

EE doesn't just say "your builds drifted." It identifies the specific commit where the deviation started — with a clickable link to the PR. For example:

React's file explosion? Triggered by b16b768f — [compiler] Feature flag cleanup (#35825)
React's co-change novelty drop? Started at 6853d7ab — [Perf Tracks] Prevent crash when accessing $$typeof

Every finding in the report links back to the trigger commit, so you're not hunting through git log trying to figure out when things changed. You see the drift, click Commit, and understand the root cause.

The Tool

I built Evolution Engine to detect this automatically. It's open source, runs locally (your code never leaves your machine), and works on any git repo:

pip install evolution-engine
evo analyze .

The analysis is pure statistics — no AI APIs called. When you want deeper investigation, EE generates a structured prompt you paste into your own AI tool.

GitHub | PyPI | Website

What 10 Open Source Repos Reveal About Development Drift in the AI Era

Codequal — Tue, 17 Mar 2026 23:58:57 +0000

Every major tech company is pushing AI-generated code. Microsoft says 30% (targeting 80%). Google says 30%. Uber reports 65-72%. Amazon mandated 80% AI tool usage. Shopify made AI "mandatory."

But Google's own DORA research shows a paradox: AI increases throughput while decreasing delivery stability. Teams ship faster, but the code breaks more often.

I wanted to understand why. So I built Evolution Engine, an open source CLI that detects development process drift — when patterns in commit history, CI builds, deployments, and dependency signals shift in ways that often precede production issues. Then I ran it on 10 major open source repos across cloud infrastructure, frontend frameworks, AI tooling, and developer platforms.

No AI APIs are called during analysis. All pattern detection is deterministic and statistical. The tool runs entirely locally — your code never leaves your machine.

Here's what I found.

The scale

Across 10 repos, the tool analyzed over 130,000 commits, generating 250,000+ events across git, CI, deployment, and dependency signal families. It matched patterns from a knowledge base calibrated across 200+ open source repositories.

Metric	Result
Repos analyzed	10
Total commits	130,000+
Total events	250,000+
Signal families	4 (git, CI, deployment, dependency)
Repos with significant drift	10 out of 10
Average drift signals per repo	6.6
Average correlation patterns per repo	24.3

Every single repo had significant drift signals. Every one.

Finding 1: CI build times spike dramatically — and nobody notices

The most consistent pattern across all 10 repos: CI build duration spikes that dwarf historical baselines.

Repo type	Normal CI time	Spike	Deviation
Cloud SDK (monorepo)	~45 seconds	6+ hours	1,552x
AI framework	~95 seconds	55 minutes	889x
Cloud infrastructure toolkit	~26 seconds	70+ minutes	111x
Edge platform SDK	~33 seconds	60+ minutes	74x
Commerce framework	~64 seconds	8+ minutes	43x
Code editor	~41 seconds	23+ minutes	34x
Frontend framework	~45 seconds	6 minutes	13x
Fullstack framework	~6 minutes	32 minutes	5x

These aren't gradual slowdowns — they're sudden spikes, often tied to a single commit or dependency change. The problem? Most teams don't track CI duration as a process signal. They notice when builds fail, but a 34x slowdown that still passes? That drifts silently.

8 out of 10 repos had CI spikes exceeding 10x their baseline. The median spike was 53x.

Finding 2: Release cadence gaps correlate with code spread

When a repo's release cadence suddenly lengthens, it's almost always accompanied by increased code dispersion — changes spread across unrelated parts of the codebase.

Repo type	Normal cadence	Gap	Slowdown
Cloud SDK (monorepo)	~2.9 hours	22 days	182x
Commerce framework	~1.5 days	37 days	24x
Cloud infrastructure toolkit	~21 hours	16.5 days	18x
Fullstack framework	~6 days	96 days	16x
Logging library	~13 days	200 days	15x
Frontend framework	~28 days	113 days	4x

This correlation showed up as a known pattern in 8 out of 10 repos. When engineers touch more unrelated files per commit and releases slow down, something structural has shifted — often a large refactoring, a dependency migration, or (increasingly) an AI-assisted batch change that touches more files than a human would.

Finding 3: Co-change novelty drops to zero

"Co-change novelty" measures how often files that change together in a commit have changed together before. A score of 1.0 means entirely novel pairings. A score of 0.0 means the exact same files are changing together repeatedly.

In 9 out of 10 repos, we found commits where co-change novelty dropped to zero — indicating repetitive, pattern-locked changes rather than organic development. This is a hallmark of:

Automated dependency bumps (bots touching the same lockfiles repeatedly)
Code generation tools producing similar diffs
AI-assisted changes that follow templates rather than addressing unique problems

The interesting question: is this a problem? Sometimes repetitive changes are exactly right (automated security patches). But when novelty drops to zero and CI times spike and release cadence gaps appear, the correlation suggests something has gone wrong.

Finding 4: Merge-back commits create statistical blind spots

Three repos had single commits touching 10,000-21,000+ files. These are merge-back commits in monorepos — technically expected, but they create extreme statistical outliers that mask real drift signals underneath.

If your drift detection (or any metrics tool) doesn't account for these outliers, the signal-to-noise ratio collapses. A legitimate 34x CI spike looks insignificant next to a 14,000x files_touched outlier.

Finding 5: Cross-family correlations reveal systemic patterns

The most interesting findings weren't individual metrics — they were correlations between signal families:

CI duration <-> files touched: When commit size increases, build times increase non-linearly. This correlation appeared in all 10 repos.
Deployment cadence <-> code dispersion: When releases slow down, changes spread wider. Found in 8/10 repos.
Dependency changes <-> change locality: When dependencies change, subsequent code changes tend to be less focused. Found in 7/10 repos.

These cross-family patterns are invisible if you only monitor one signal family (just CI, or just git). You need the full picture.

What this means for AI-assisted development

Google's DORA research found that AI increases throughput but decreases stability. Our findings suggest why:

AI generates larger commits — more files touched per change, increasing CI load
AI follows templates — co-change novelty drops, creating repetitive patterns
AI doesn't respect cadence — large batch changes break release rhythm
The drift is gradual — no single commit looks wrong, but the aggregate pattern shifts

The fix isn't to stop using AI tools. It's to monitor the process signals they affect. The same way you'd monitor application performance after a deployment, you should monitor development process patterns after adopting AI coding tools.

What's next

This is the first in a series. In upcoming posts, I'll publish detailed case studies of individual repos (with permission from maintainers where applicable) and dive deeper into specific patterns — like how dependency drift predicts deployment instability, and what "healthy" drift patterns look like versus problematic ones.

Try it yourself

Evolution Engine is open source. Install it and run on any repo:

pip install evolution-engine
evo analyze /path/to/your/repo

The tool generates an interactive HTML report with all findings, plus an investigation prompt you can paste into any AI assistant for root cause analysis — so your AI tools can help diagnose the drift patterns they create.

All analysis is local and statistical. No code leaves your machine. No AI APIs are called.

GitHub: github.com/alpsla/evolution-engine
Website: codequal.dev

I built this. Evolution Engine is open source — dual-licensed: CLI and adapters are MIT, core engine is BSL 1.1 (converts to MIT in 2029). Happy to answer questions in the comments.

AI Writes Your Code. Who Watches for Drift?

Codequal — Tue, 10 Mar 2026 13:31:22 +0000

The problem

AI coding tools are incredible — until they quietly drift off course.

You're using Cursor, Copilot, or Claude Code. The code looks fine. Tests pass. But over a few commits, subtle shifts accumulate: more files touched per commit than usual, dependency trees growing unexpectedly, CI times creeping up.

By the time you notice, the damage has compounded across dozens of commits.

What Evolution Engine does

Evolution Engine is a local-first CLI that monitors your SDLC signals and flags statistical anomalies:

Git patterns — file dispersion, change locality, co-change novelty
CI pipelines — duration trends, failure patterns
Dependencies — count changes, depth shifts
Deployments — release cadence, prerelease patterns
Testing — failure rates, skip rates, suite duration
Coverage — line and branch rate changes

When a metric deviates significantly from your project's baseline, EE raises an advisory — not a bug report, a drift alarm.

No AI APIs required

This was a deliberate design choice. Your code never leaves your machine. EE does pure statistical analysis locally.

When you want deeper investigation, EE generates a structured prompt you can paste into your own AI tool — ChatGPT, Claude, Cursor, whatever you trust. You control what leaves your machine.

How to use it

pip install evolution-engine
cd your-project
evo analyze .

That's it. EE builds a baseline from your git history and flags deviations.

As a GitHub Action

- uses: alpsla/evolution-engine@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}

As a git hook

evo init --hooks

Open source

Evolution Engine is available on PyPI and GitHub. Git analysis is free — no account, no license key needed.

Would love feedback from the community. What signals matter most in your workflow?