Building CodeMapRT: Rethinking Regression Testing with Change-Mapping

yuliadrogunova — Thu, 11 Sep 2025 13:05:26 +0000

How I turned endless full regressions into fast, targeted runs driven by code changes.

Where It Started

On large enterprise projects, regression testing always felt like a bottleneck. Every pull request meant running the entire suite — hundreds of tests, even for a one-line fix. Developers were frustrated. Pipelines got slower. CI bills grew.

At some point, I asked myself:
“Why are we testing everything, when only part of the code has changed?”

Looking at the Problem

Digging deeper, I noticed a few things:

Most commits only touched a handful of modules.
Yet, the same giant regression pack kept running over and over.
Feedback to developers was delayed, which slowed down releases.

The waste was obvious — we were validating a lot of code that never changed.

The Turning Point

The idea clicked: Instead of running all tests, map what changed in the codebase to the tests that actually matter.

That became the core principle behind CodeMapRT — a framework that aligns code diffs with relevant test cases.

How I Approached It

Step 1. Detecting What Changed

I started simple — just grabbing modified files from Git:

# Detect changed files in the last commit
git diff --name-only HEAD~1 | grep ".java" > changed-files.txt

Then I introduced lightweight annotations in tests to specify which modules they cover:

@Test
@CoversModule({"auth", "login"})
public void loginTest() {
    // ...
}

Tests that didn’t match the changed files? Skipped.
Mapping can be configured via annotations or other metadata sources,
including coverage reports.

Step 2. Keeping It Safe

Selective regression always carries a risk of missing hidden dependencies.
To address this, CodeMapRT provides two safety layers:

Fallback full run — can be triggered for major releases or whenever coverage data looks incomplete.
Safety tests — a configurable set of additional checks added on top of the selected tests. This way, even if only part of the suite is executed, critical scenarios remain validated.

Step 3. CI/CD Integration

For real adoption, I built a CLI tool that detects changes, maps them to relevant tests, and outputs a filtered test list.

Example with GitHub Actions:

jobs:
  codemaprt:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run CodeMapRT
        run: |
          ./codemaprt detect
          ./codemaprt select
          ./gradlew test --tests $(cat selected-tests.txt)

CodeMapRT can be integrated with any CI/CD system — GitHub Actions, Jenkins, GitLab CI, TeamCity — making it scalable for enterprise workflows.

Example Use Case

Imagine a mobile banking app:

A developer modifies the login module.
Instead of re-running the entire regression suite, CodeMapRT analyzes the code changes and selects only the most relevant automated tests for that module.
Result: targeted test execution, faster pipelines, quicker releases, and reduced infrastructure load.

What Changed in Practice

With CodeMapRT, regression testing no longer requires executing the full suite for every commit. Instead:

Small commits trigger only the tests linked to the changed modules.
Medium features run a focused subset instead of hundreds of unrelated tests.
Full regression runs remain possible for major releases, but they no longer block day-to-day development.

This approach shortens the feedback loop, optimizes CI/CD resources, and gives developers faster confidence in their changes.

Benefits of CodeMapRT

Accelerates release cycles — less time spent running unnecessary tests.
Maintains coverage — critical paths are still validated.
Reduces expenses — fewer resources consumed in regression runs.
Scales for enterprise projects — adaptable to large and complex systems.

The Results: Smarter Regression

After building a demo project with 50 automated tests and integrating CodeMapRT:

Without CodeMapRT: all 50 tests ran on every commit.
With CodeMapRT: the framework selected 28 tests directly linked to the changed module, plus 5 additional safety tests (as described in Step 2) → 33 tests in total.
Result: ~34% fewer test executions, reducing CI load while ensuring that critical scenarios remain validated.

This demo shows a one-third reduction, but the approach scales: in enterprise projects with thousands of tests, CodeMapRT can deliver even greater efficiency, since typically only a small fraction of the suite is relevant to a given commit.

You can find demo results in the GitHub Actions of the CodeMapRT repository

Forem: yuliadrogunova