Forem: Omer Morad

Your AI Agent Should Not Grade Itself: Software Self-Verification Crisis

Omer Morad — Fri, 15 May 2026 16:14:20 +0000

TL;DR: I have been working on Boundary-First Engineering, a manifesto built around one question: who verifies the code when the AI wrote the checks too? The answer it argues for is boundaries, contracts, and evidence that came from outside the implementation loop. I still want agents writing code. I just do not want them grading the final exam.

I have been spending a lot of time with AI coding agents and one pattern keeps showing up.

The agent writes the feature, writes the tests, fixes the failures, updates the summary, and hands back a pull request with green checks before my morning espresso is cold.

It feels good for about five seconds, and then the uncomfortable part arrives: the same loop that produced the code also produced most of the evidence that says the code is correct.

That pattern is the reason I started writing Boundary-First Engineering, an open manifesto for verifying software in the agentic era. The short version of its thesis is that we moved the bottleneck from writing software to trusting software, and our verification habits have not caught up.

How things started to feel off

For years I trusted the usual, classic stack: unit tests, code review, coverage, static analysis, and CI. It was never perfect, but it worked at human speed.

Tests expressed intent.
Review caught what tests missed.
CI made sure we did not forget the obvious things.

An agent now compresses all of that into one flow. It generates the implementation, generates tests for it, runs the suite, patches the code, fixes the tests, and explains why everything is fine.

The green check still looks the same. It does not always mean the same thing.

A green check used to mean the code passed checks that someone cared enough to define. Now it can mean the code passed checks created by the same process that wrote the code. Those are not the same statement, and the difference is easy to miss because the surface looks identical.

The circular validation problem

The concept I want to name is circular validation, which I hear about a lot recently.

Circular validation is what happens when the same implementation loop produces the code, the tests, the fixes, and the explanation. The work can look disciplined and still be a closed circle of agreement. Nothing in it is fraudulent. It is just self-referential.

Here is a deliberately boring example, because boring examples are where production bugs actually live:

Ticket: show account data for customers with access

Agent builds:
- GET /accounts/:id
- tests for active customers
- tests for missing accounts
- tests for permission failure

Missed rule:
- suspended accounts must still be visible to compliance users

The agent did not encode the suspended-account rule because it did not understand it. The generated tests do not catch the omission because they were written from the same understanding that produced the code. Of course it misses. The blind spot is shared by the implementation and the evidence at once.

This is not a claim that the model is bad. Better models miss the rule less often. They do not remove the structural problem, which is that code and its checks coming from one source share whatever that source did not know.

So when someone tells me the AI wrote tests too, I do not automatically feel safer. Of course it wrote tests. The more useful question is where those tests came from.

Coverage measures reach. It does not measure truth. A 90 percent covered misunderstanding is still a misunderstanding.

The boundary move

Boundary-First Engineering starts from a simple claim: systems earn trust at their boundaries.

A boundary can be a:

HTTP contract
Database state you own
Queue message
Event schema
Permission decision exposed to another service
CLI output someone scripts against

Internals matter. I am not arguing that we delete unit tests and hope for the best. But internals are private and changeable. Boundaries are public, durable, and expensive to get wrong. They are what users, services, and auditors actually experience.

So in the agentic era verification should start with what crosses the boundary, not with whether the generated internals look reasonable.

That can be small and practical: an OpenAPI document, a protobuf file, acceptance scenarios written before implementation, a policy rule that runs in CI, a dependency rule that has been enforced for months. The goal is not a large specification nobody reads. It is a thin, durable artifact that the implementation has to satisfy.

If the spec needs the code beside it to make sense, it is not doing its job.

The provenance test

There is one line from the manifesto I keep returning to: the test is provenance, not appearance.

Appearance is cheap now. A pull request generated by an agent can look serious very quickly. Clear test names, table-driven cases, a confident summary, and a tidy explanation.

Provenance asks a harder question: did this check originate outside the implementation loop?

Outside can mean another team published the contract before the work started. It can mean a product owner wrote acceptance scenarios from user behavior. It can mean compliance defined the data-handling rules, or a consumer service owns the schema.

Inside means I wrote the code and then wrote the test for what I wrote. Inside means an agent generated both in one task. Inside means I approved a generated test after reading a generated explanation of generated code.

Approval is not laundering. A human clicking approve does not change where an artifact came from. Review can accept, reject, or improve evidence. It cannot convert the builder's own evidence into independent evidence by signing off on it.

This is the part teams will find uncomfortable, because we are used to treating review as the step that makes a change safe enough. At AI speed, human attention is the scarce resource, and spending it to bless evidence from inside the loop is a poor use of it.

Fast enough to matter

There is a second constraint that is easy to overlook. Verification has to be fast enough for an agent to use it.

A boundary check that runs once a week is governance theater. A boundary check that runs in under a second becomes part of how the agent works. It can hit the schema, fail, adjust, hit the contract test, fail, adjust, and converge toward the boundary instead of toward its own explanation.

This is why the boring guardrails matter more now, not less: types, schemas, linters, architectural dependency rules, generated clients from contracts, database constraints, policy checks, small integration tests against real systems you own.

Those guardrails need provenance too. A lint rule added in the same branch to make that branch pass is not an independent check. A dependency rule that has protected the architecture for six months is. When implementations become cheap and rotate often, the durable constraints around them are the part worth keeping.

One scope note: everything above is about pre-merge verification. Production trust still has to accrue at runtime, through telemetry, tracing, and runtime contract checks. That is the other half of the discipline, and the manifesto is explicit that it lives outside its scope.

Why I care about this

I have spent a lot of time on testing tools and developer experience. When I built Suites, the motivation was the repetitive, inconsistent experience of unit testing in TypeScript dependency injection codebases. I still think good testing tools matter.

But working with agents made me aware of a layer that better tooling does not reach. A faster, smoother test-writing experience does not help if the same loop writes the implementation and decides what correct means. Faster testing is good. Faster circular validation is not.

The habit I think we need is small to describe and harder to practice: before asking whether the generated code looks reasonable, ask what outside artifact it was made to satisfy.

Where this stands

Boundary-First Engineering is a short manifesto right now, not a framework and not a product. It argues for a handful of things:

Boundaries over internals
Real systems over stubs for what you own
What crosses the boundary over what happens inside it
Verification from outside the implementation loop over tests written from within it
Fast guardrails that agents can actually use

It is early, and I would rather it be argued with than agreed with quietly. If you work with AI agents, backend contracts, testing strategy, CI, or architecture rules, the parts of it that are wrong are most likely wrong in ways you would see first.

The manifesto: Boundary-First Engineering
How to sign or propose changes: CONTRIBUTING

If the code can write itself, it is worth asking why we still let it grade itself.

Unit Test Like a Pro: Automock, My Open Source Answer to Mocking Frustration 🤓💡🎭

Omer Morad — Tue, 12 Dec 2023 16:16:38 +0000

Hello Dev Community 🙋‍♂️

I've been working with TypeScript for a while, and one thing that always seems to be a time-consuming task is manually creating mocks for unit tests when using dependency injection frameworks.

There is often a lack of consistent structure, which can make the experience even more frustrating. It can get repetitive and result in test suites that are difficult to handle and prone to becoming overly complex.

That's why I created Automock. This project is open-source and seeks to change the way we conduct unit testing in TypeScript DI environments. It automatically creates mock objects for classes, improves testing execution speed, and maintaines a consistent structure within DI frameworks such as NestJS and InversifyJS, as well as popular testing libraries like Jest and Sinon.

🔗 Here are some links to check out:
😎 GitHub: https://github.com/automock/automock
🐈 Automock's NestJS Official Recipe: https://docs.nestjs.com/recipes/automock
📦 NPM: https://www.npmjs.com/package/@automock/jest
📚 Docs Website: https://automock.dev

The rest of the examples in this post will be based on the following example, which is using NestJS arbitrarily but applies to all DI frameworks:

import { Injectable } from '@nestjs/common';

@Injectable()
export class UserDAL {
  async getUsers(): Promise<User[]> { ... }
  async getSomethingElse(): Promise<unknown> { ... }
}

@Injectable()
export class UserApiService {
  async fetchUsers(): Promise<User[]> { ... }
  async fetchSomethingElse(): Promise<unknown> { ... }
}

@Injectable
export class UserService {
  constructor(private userDal: UserDAL, private userApi: UserApiService) {}

  async getAllUsers(): Promise<User[]> {
    const fromDb = await this.userDal.getUsers();
    const fromApi = await this.userApi.fetchUsers();

    return [...fromDb, ...fromApi];
  }
}

The UserService aggregates user data from both a database and an external api. To test the UserService in isolation, we need to mock the UserDAL and UserApiService classes.

🔄 Traditional Approach to Unit Testing

Creating manual mocks for class dependencies is a common part of the unit testing process. This manual method is time-consuming and tedious, especially when working with large classes, here is an example:

const userDalMock: jest.Mocked<UserDAL> = {
  getUsers: jest.fn(),
  getSomethingElse: jest.fn()
};

const userApiSvcMock: jest.Mocked<UserApiService> = {
  fetchUsers: jest.fn(),
  fetchSomethingElse: jest.fn()
};

Looks familiar, right? 🥲

Regardless of the testing technique you use, these mocks are essential for the class under test. In most cases, you can avoid interacting with other classes in two ways:

Inject these mocks as arguments into newly created instances of the classes.
Leverage your framework's DI container to replace actual objects with their equivalent mock objects.

The primary goal of both approaches is to separate the unit under test (the class) and subject it to predictable and controlled behavior.

Here is the traditional approach using NestJS, which appears to be applicable to all dependency injection frameworks and libraries:

describe('User Service Unit Test', () => {
  let userService: UserService; // Under Test

  // Mocks Declaration
  let userDal: jest.Mocked<UserDAL>;
  let apiService: jest.Mocked<ApiService>;

  let moduleRef: TestingModule;

  beforeAll(async () => {
    moduleRef = await Test.createTestingModule({})
      .overrideProvider(UserDAL)
      .useValue(userDalMock)
      .overrideProvider(ApiService)
      .useValue(userApiSvcMock)
      .compile();

    apiService = moduleRef.get(ApiService);
    userDal = moduleRef.get(UserDAL);
  });

  test('should retrieve users', async () => { ... });
});

As we change and refactor the UserService, it becomes increasingly reliant on additional classes and methods, which can complicate the process of updating these mocks. Furthermore, if there are any modifications to the other classes (UserDAL and UserApiService), it will be necessary to also make updates to the stubs.

This continual maintenance requirement can lead to a testing environment that is hard and error-prone since devs need to make sure the mocks accurately reflect the current state of the codebase. This adds another layer of complexity to the development process.

💪 Automock's Approach

Automatic Mock Generation

Automock's most notable feature is automated mock generation, which eliminates the repetitive task of manually creating and maintaining mock objects.

Here is a typical test suite using Automock:

import { TestBed } from '@automock/jest';
import { UserService, UserDAL, UserApiService } from './services';

describe('Users Service Unit Test', () => {
  // 🧪 Unit under test
  let userService: UserService;

  // 🎭 Declare Mocks
  let mockUserApiService: jest.Mocked<UserApiService>;
  let mockUserDal: jest.Mocked<UserDAL>;

  beforeAll(() => {
    // 🚀 Automatically mock every dependency, and create on-the-fly virtual DI container for the mocks
    const { unit, unitRef } = TestBed.create(UserService).compile();

    // ⚙️ Assign the unit under test
    userService = unit;

    // 🔍 Retrieve mocks from the unit reference and assign
    mockUserApiService = unitRef.get(UserApiService);
    mockUserDal = unitRef.get(UserDAL);
  });

  it('should retrieve users from both database and api', async () => {
    // 📝 Arrange
    mockUserApiService.fetchUsers.mockResolvedValue([{ id: 1, name: 'Joe' }]);
    mockUserDal.getUsers.mockResolvedValue([{ id: 2, name: 'Jane' }]);

    // 🎬 Act
    const users = await userService.getAllUsers();

    // ✅ Assert
    expect(users).toHaveLength(2);

    expect(mockUserApiService.fetchUsers).toHaveBeenCalled();
    expect(mockUserDal.getUsers).toHaveBeenCalled();
  });
});

📚 Full Step-by-Step Example

This ensures consistent structure and syntax within test suites. This consistency is crucial and is maintained even as underlying classes or methods undergo changes or refactoring, providing a stable and reliable testing environment.

Speed

The introduction of a virtual DI container, using the TestBed, is another key aspect. It speeds up the testing execution by bypassing the conventional DI framework loading, leading to much faster test execution.

The isolation provided in testing ensures that each test runs independently with its own set of automatically provided mock implementations, creating a streamlined and interference-free environment.

🔗 Check out the full benchmark:
GitHub: https://github.com/automock/benchmark#visualizations

tsc demonstrated a significant 28.9% improvement in speed for Automock (vs NestJS). Similarly, tsc-isolated and SWC also exhibited notable increases of 26.0% and 23.2% respectively.

* I haven't had the opportunity to test it with other dependency injection frameworks rather then NestJS yet.

🧠🤓 The Reason Behind It All

I was motivated to create this project because I faced many common challenges when working with unit testing in different dependency injection frameworks. Regardless of the framework used, the process always proved to be complex and time-consuming. This problem was not limited to my personal experience; it highlighted a wider gap that exists throughout the industry.

I have a clear vision for Automock - to make the testing experience more efficient and improve the reliability and quality of code. This project represents a significant stride towards enhancing development practices within the TypeScript community, making them more efficient and effective.

👀 How Things Stand Right Now, and What Am I Looking for?

Until now, the project has gained around 60K monthly downloads, thanks to the recognition of NestJS, which added Automock to their recipes section.

Currently, the project is in its early-middle stage. It supports NestJS and InversifyJS. I am looking to expand its capabilities and integrate DI frameworks, and I'm seeking contributions from the developer community, especially those proficient in TypeScript, dependency injection, and unit testing methodologies.

It's for You ❤️

Automock is more than just a tool; it is intended to be a community-driven solution aimed at aiding developers facing similar challenges in unit testing.

I invite you to be part of this journey, let's shape the future of software quality and unit testing!

Happy testing! 🚀🧪🤓