The Economics of Testing: Why the Shape of Your Test Pyramid Matters

Łukasz Łepecki — Tue, 18 Mar 2025 13:46:25 +0000

If all QA can be covered by E2E tests, why even bother writing unit and integration tests at all?

TL;DR

When designing your test strategy, make sure that the test pyramid is actually shaped more or less like the ancient Egyptians would build it: perform as many tests as possible on the lowest part of the test pyramid because it will save you money.

Unit Tests - Fast Execution and Instant Feedback

Unit tests should be the base of your construction. They're fast and provide instant feedback. Decent coverage makes refactoring nice and easy since you can make sure you didn't break anything by instantly running unit tests after every small change you introduce. Don't obsess over the percentage of coverage though. Good coverage cuts through your business logic, not simply displays a big number.

Integration Tests - The Cost Is Growing

You won't cover everything with unit tests. At some point, you'll have to move on to integration tests that verify the correctness of interactions and communication between components you've already unit-tested individually. They should still be easy to reason about and debug, but you'll have to wait some time (seconds or - worst case - minutes) for the results. Building Docker images takes time, spinning up databases takes time, and setting up additional dependencies adds even more delay. When it all adds up, you'll have the reason why those tests can't be run as frequently as unit tests: time is money, and you don't have an infinite bag of gold, do you?

End to End Tests - Expensive but Necessary

Integration tests alone won't provide enough coverage. At some point, you'll want to test the actual experience that your users interact with, i.e., perform API or UI tests. They are expensive mainly due to the complexity and scale of infrastructure required to run them. You have to maintain all dependencies required by your application like databases, authentication and authorization services, infrastructure like messaging brokers and so on. But that's not the end. You’ll also need realistic, production-like data, ensuring it doesn’t accumulate or become stale across multiple test runs. The execution time gets even longer for the end-to-end tests, but you'll spend even more time maintaining the whole environment that supports these tests. It'll cost at least an order of magnitude more than what you had to invest in both unit and integration tests; that's why you'll want to limit the coverage only to what couldn't be covered with the previous two kinds.

Conclusion

Every test type has its place, but let the economics guide your strategy. Invest heavily in rapid-feedback unit tests, be selective with integration tests, and use expensive E2E tests only where absolutely necessary. Your wallet will thank you and so will your dev team when they're not waiting hours for test runs to complete.

Commit Message Policy for Your Team

Łukasz Łepecki — Mon, 17 Feb 2025 19:39:58 +0000

Conventional Commits is a standardized way to write commit messages. Following the convention will allow you to achieve a neat commit history that’s easily readable for both humans and machines.

Here’s a quick guide on how to enforce this policy on your local machine so that you are always sure your commits follow the standard.

Create a directory for Git hooks

mkdir -p ~/.git-templates/hooks

Configure Git to use this directory

git config --global core.hooksPath ~/.git-templates/hooks

Create `commit-msg` hook

Create file at ~/.git-templates/hooks/commit-msg:

#!/usr/bin/env bash

commit_msg_file=$1
commit_msg=$(cat "$commit_msg_file")

conventional_regex='^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(!)?(\([a-z0-9-]+\))?: .+$'

if ! [[ $commit_msg =~ $conventional_regex ]]; then
  echo "-------------------------------------------------------------------------"
  echo " ❌ The commit message does not follow Conventional Commits specification"
  echo "     (see https://www.conventionalcommits.org/en/v1.0.0):"
  echo " Format: type(scope)?(!)?: subject"
  echo " Allowed types:"
  echo "     feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert"
  echo " Example: feat(api)!: Add validation to the create-user endpoint"
  echo "-------------------------------------------------------------------------"
  exit 1
fi

Adjust the error message and the regex as needed.

Examples of valid commit messages

feat: Add new user onboarding flow
fix(api): Correct error-handling for 404 responses
refactor(ui)!: Overhaul navigation for better accessibility
docs: Update README to reflect API changes

Make the hook executable

chmod +x ~/.git-templates/hooks/commit-msg

Existing repos

Existing repos won’t automatically use the new global hooks if they already had .git/hooks populated. In that case, you can either:

Remove or rename the repo’s local hooks folder so Git falls back to the global one.
Or manually copy/symlink the commit-msg hook into each repo’s hooks folder.

Once set, any commit message that fails the regex check will abort the commit with an error message, enforcing Conventional Commits across all your local repos.

If you found this guide useful, consider buying me a coffee ☕️

Forem: Łukasz Łepecki

The Economics of Testing: Why the Shape of Your Test Pyramid Matters

TL;DR

Unit Tests - Fast Execution and Instant Feedback

Integration Tests - The Cost Is Growing

End to End Tests - Expensive but Necessary

Conclusion

Recommended Reading

Commit Message Policy for Your Team

Create a directory for Git hooks

Configure Git to use this directory

Create `commit-msg` hook

Examples of valid commit messages

Make the hook executable

Existing repos

Forem: Łukasz Łepecki

The Economics of Testing: Why the Shape of Your Test Pyramid Matters

TL;DR

Unit Tests - Fast Execution and Instant Feedback

Integration Tests - The Cost Is Growing

End to End Tests - Expensive but Necessary

Conclusion

Recommended Reading

Commit Message Policy for Your Team

Create a directory for Git hooks

Configure Git to use this directory

Create commit-msg hook

Examples of valid commit messages

Make the hook executable

Existing repos

Create `commit-msg` hook