Forem: Maksim Matlakhov

Event Modeling: Visible Issues and My Vision

Maksim Matlakhov — Thu, 23 Oct 2025 09:35:51 +0000

During my journey learning Event Modeling as part of my VibeTDD framework exploration, I discovered some issues that from my point of view limit the framework's adoption and effectiveness. After extensive research and community discussions, I decided to build a proof-of-concept that addresses these core problems.

The Problems I Discovered

1. Canvas Layout Nightmare

Event Modeling relies heavily on visual canvases like Miro, creating constant friction during the modeling process. The specific pain points include:

Continuous resizing and repositioning: Every addition requires manual layout adjustments
Cascading layout changes: Adding one element forces you to resize multiple components
Scale anxiety: Managing hundreds of slices with multiple scenarios becomes overwhelming
Time waste: Significant effort spent on layout management instead of actual modeling

I shared this frustration on LinkedIn, and multiple Event Modeling practitioners confirmed these concerns, including the framework's creator and experienced practitioners who acknowledged the tooling limitations.

2. Business Rules Documentation Gap

My investigation revealed a critical architectural problem: Event Modeling has no standardized way to handle business rules. Looking at examples, I couldn't find clear documentation for rules like:

Max 3 items in shopping cart
Single payout limit: 30
Total payouts can't exceed 200

This creates several problems:

Information fragmentation: Rules must live "somewhere else" (Confluence, Jira, comments)
Incomplete model slices: Developers need to "hunt for details" across multiple systems
Synchronization risk: Miro boards and documentation can drift apart
Code generation limitations: Yes, it's possible to export the model as json, but having incomplete info I mentioned above, code generation can be tricky

Even after consulting with industry experts, I found that there's no standard solution for this need.

A Simple POC to Test My Ideas

To explore solutions to these issues, I built a simple proof-of-concept - a quick, trivial implementation to test whether a structured approach could work. This is just the beginning, a basic prototype to validate the core concepts rather than a complete tool.

1. Slices Board

The main board shows slices in a clean, structured format without manual positioning. Each slice contains:

Command (blue): The action being performed
Event (orange): The result of the action
Test Scenarios: Direct link to business logic validation

2. Structured Slice Editor

The slice editor provides structured forms for:

Model selection: Choose a domain object
Operation/Status: Define the action and the slice status
Command definition: Specify the command name and parameters
Field management: Add fields with proper types and constraints
Event definition: Define the resulting event and its fields
Rules section: Add and manage business rules manually or with AI generation

3. Smart Field Connections

The system supports dependency management between objects. When creating an order with userId and courseId, we can connect these fields to their respective events by selecting from existing events in the system.

4. Custom Type System

Beyond basic types (String, Date, Long etc), the system supports:

Common types like Email, CountryCode, CurrencyCode
Custom complex types: Define reusable business objects like Price with structured fields
Type validation: Ensure consistency across the model

5. Test Scenarios Generation

The system generates test scenarios based on:

Configuration parameters: Business rules with configurable limits
Existence rules: Validation that dependent objects exist
Business logic: Custom rules with success/failure cases
Manual definition: Direct scenario creation through the interface

Check the full demo video:

Key Advantages of This Approach

No Layout Management
Eliminates the constant resizing, moving, and adapting that plagues canvas-based tools.

Structured Business Rules
Rules are first-class citizens with clear categorization, configuration options, and automatic test generation.

Type Safety
Proper type system prevents inconsistencies and enables better code generation.

Dependency Management
Manual connection of object relationships with clear visualization.

Export Capability
Complete model data can be exported for code generation without external documentation hunting.

The Bigger Picture

This proof-of-concept addresses what I call the "notepad problem" of Event Modeling. As I mentioned in the LinkedIn discussion: "I can imagine what would happen if JetBrains didn't provide a tool for Kotlin and suggested devs write code in notepad 🙂"

Event Modeling is a powerful framework, but poor tooling creates unnecessary friction that slows adoption and execution. By treating Event Modeling as structured data rather than free-form drawings, we can:

Accelerate modeling sessions by removing layout management overhead
Ensure completeness by requiring all necessary information upfront
Enable automation through structured data that AI can understand easily
Improve maintainability by keeping rules and scenarios synchronized
Scale effectively without canvas limitations

Technical Implementation Notes

The POC is built with modern web technologies and demonstrates:

Form-based interfaces that are faster than drag-and-drop
Intelligent defaults that reduce repetitive data entry
Validation at input time rather than discovery later
Clear separation between business rules and technical implementation
Extensible type system for domain-specific needs

What's Next

This proof-of-concept establishes a crucial foundation: structured slice data ready for code generation and dynamic test execution.

I've conducted a quick experiment that demonstrates the system can execute tests stored in the database rather than generating static test files. This means the test scenarios visible on the board become the source of truth, no code generation step required, no synchronization issues between specifications and tests.

The implications are significant: changes to business rules or test scenarios immediately affect test execution without any build or deployment steps. This creates a truly dynamic testing environment where the Event Model directly drives validation.

Event Handling: Automatic Event Bootstrapping

Maksim Matlakhov — Mon, 06 Oct 2025 14:44:32 +0000

This post continues the event handling series. Check out the previous post on Inbox Pattern to learn how we handle event ordering and idempotency.

The Problem with Historical Events

When you create a new service that subscribes to events from other domains, you face a common challenge: your service needs historical data to function independently.

Imagine you're building an Order Service that needs to display user information. You subscribe to user events like UserCreated, PersonalDataUpdated, and StatusUpdated. But what about users who were created before your service existed? Your database is empty, and you can't show any order details properly.

This applies whether your services are deployed separately or as part of a modular monolith. The key is that each service owns its data and communicates through events.

Some Approaches to Manage That

Teams typically handle this in a few ways:

Manual data migration - Write custom scripts to copy data from the source service's database. This breaks service boundaries and creates tight coupling.
Batch import endpoints - Ask the source team to build special APIs for bulk data export. This requires coordination and extra development work.
Accept the gap - Start fresh and only handle new events. This means your service is incomplete until enough time passes.

All of these approaches are painful and error-prone.

A Better Way: Automatic Event Bootstrap

The solution is surprisingly elegant: fetch historical events the same way you'll receive future events.

Here's the key insight: if every service provides a simple method to fetch events by topic and timestamp, any dependent service can automatically bootstrap itself during startup.

How It Works

The VT framework provides all the infrastructure for event bootstrapping. On the service side, you only need two simple steps:

Step 1: Create a Storage Adapter

Create a simple storage adapter for tracking bootstrap progress:

@Component
class OrderEventBootstrapStorageAdapter(
    repository: OrderEventBootstrapRepository,
) : VTMongoEventBootstrapStorageAdapter(repository)

The adapter extends the framework's base class and handles saving/loading bootstrap records. You can use any database - MongoDB, PostgreSQL, etc.

Step 2: Add the EventBootstrap Annotation

Add the annotation to your consumer class:

@EventConsumer
@EventBootstrap(
    consumerName = "UserView", // define a static unique name
    storageBean = OrderEventBootstrapStorageAdapter::class, // use defined adapter
    clientBean = UserEventClientV1::class // each service provides an event client
)
class UserEventsConsumerV1(
    private val orchestrator: UserViewOrchestrator,
) {
    fun onCreated(event: EventV1<UserCreatedV1>) {
        orchestrator.create(/* ... */)
    }

    fun onPersonalDataUpdated(event: EventV1<PersonalDataUpdatedV1>) {
        orchestrator.update(/* ... */)
    }

    fun onStatusUpdated(event: EventV1<UserStatusUpdatedV1>) {
        orchestrator.update(/* ... */)
    }
}

That's it! No special bootstrap code needed. The framework handles everything else.

What the Framework Does

The VT framework provides the complete bootstrapping infrastructure.

Event Fetching Interface

The framework defines a standard interface that each domain service implements:

interface VTEventClientV1 {
    suspend fun fetch(request: EventsFetchRequestV1): List<BrokerMessageV1>
}

data class EventsFetchRequestV1(
    val body: EventsFetchRequestBodyV1,
    val timeout: Duration? = null,
)

data class EventsFetchRequestBodyV1(
    val topics: Set<String>,
    val after: Instant,
    val itemsCount: Int
)

This can be implemented using internal clients (for modular monoliths or testing), REST APIs, WebSockets, or any other protocol. The important part is the consistent interface.

Automatic Topic Detection

The framework automatically discovers which topics your consumer needs:

private fun registerConsumer(beanName: String, consumer: Any) {
    val registeredTopics: MutableSet<String> = mutableSetOf()

    for (method in consumer::class.java.methods) {
        // Detect event types from method parameters
        val eventBodyType = extractEventType(method)
        registeredTopics.add(topicMap[eventBodyType]?.topic())

        consumerRegistry.register(
            eventType = eventBodyType,
            handler = { event -> method.invoke(consumer, event) }
        )
    }

    bootstrapRegistry.setup(consumer, registeredTopics)
}

Progressive Fetching with Retry Logic

The bootstrap executor fetches events in batches with exponential backoff:

private suspend fun executeOne(record: EventBootstrapRecord) {
    var processedTotal = 0
    var hasMore = true
    var currentRecord = record

    while (hasMore) {
        when (val result = fetchAndProcessBatch(currentRecord)) {
            is BatchResult.Success -> {
                if (result.messages.isEmpty()) {
                    hasMore = false
                    markAsDone(currentRecord)
                } else {
                    processedTotal += result.messages.size
                    currentRecord = updateTimestamp(currentRecord, result.messages)
                    log.info {
                        "Processed batch of ${result.messages.size} events. " +
                        "Total: $processedTotal"
                    }
                }
            }
            is BatchResult.Failure -> {
                markAsFailed(currentRecord, result.error)
                hasMore = false
            }
        }
    }
}

Progress Tracking

The system tracks progress in a bootstrap collection:

{
  "_id": "3d88ca1e-c166-4870-87fa-54ddd4b8cd77",
  "consumerName": "UserView",
  "status": "DONE",
  "topics": [
    "user.model.created.v1",
    "user.status.updated.v1",
    "user.personal-data.updated.v1"
  ],
  "lastCreatedAt": "2025-10-05T12:39:22.027Z",
  "processedCount": 45
}

This means you can safely stop and restart your service during bootstrap. It will resume from where it left off.

Same Handlers for Historical and Real-Time Events

Here's the beautiful part: historical events use the exact same consumer methods as real-time events.

private fun callConsumers(message: BrokerMessageV1, record: EventBootstrapRecord) {
    consumerRegistry
        .getConsumers(message.body::class)
        .filter { it.bootstrapConsumerName == record.consumerName }
        .forEach { it.handler(message) }
}

Your consumers already handle event ordering and idempotency properly (see the previous post for details), so receiving historical and real-time events simultaneously isn't a problem.

Adding New Features Later

Let's say your Order Service initially subscribed to:

product.created.v1
product.price.updated.v1

Later, you need to add product status information to your order table. Simply add a new handler method:

@EventConsumer
@EventBootstrap(
    consumerName = "ProductView",
    storageBean = OrderEventBootstrapStorageAdapter::class,
    clientBean = ProductEventClientV1::class
)
class ProductEventsConsumerV1 {
    // Existing handlers
    fun onCreated(event: EventV1<ProductCreatedV1>) { /* ... */ }
    fun onPriceUpdated(event: EventV1<PriceUpdatedV1>) { /* ... */ }

    // New handler - framework detects the new topic automatically
    fun onStatusUpdated(event: EventV1<ProductStatusUpdatedV1>) {
        // Your new logic here
    }
}

On next restart, the framework detects the new topic and fetches only those historical events:

private fun createRecord(bootstrap: EventBootstrap, topics: Set<String>) {
    val existingRecords = storage.get(bootstrap.consumerName)
    val allExistingTopics = existingRecords.flatMap { it.topics }.toSet()
    val newTopics = topics - allExistingTopics

    if (newTopics.isEmpty()) return

    val newRecord = EventBootstrapRecord(
        consumerName = bootstrap.consumerName,
        topics = newTopics,
        lastCreatedAt = Instant.EPOCH,
        status = EventBootstrapStatus.IN_PROGRESS
    )

    storage.save(newRecord)
}

Perfect for Continuous Integration

This approach works excellently with continuous deployment:

First iteration: Create your service with initial event subscriptions
Deploy: The service starts, bootstraps historical events, and begins receiving real-time events
Later iteration: Add new event handlers for new features
Deploy again: The service automatically fetches only the new historical events

No manual migrations. No coordination meetings. No data import scripts. Just deploy and let the framework handle it.

Bootstrap States

The system tracks four states:

enum class EventBootstrapStatus {
    PENDING,      // Ready to start
    IN_PROGRESS,  // Currently fetching
    DONE,         // Successfully completed
    FAILED        // Error occurred
}

You can also control automatic startup:

@EventBootstrap(
    consumerName = "UserView",
    storageBean = OrderEventBootstrapStorageAdapter::class,
    clientBean = UserEventClientV1::class,
    autoStart = false  // Start manually via endpoint
)

Summary

Event bootstrapping solves the historical data problem elegantly by:

Reusing the same event handlers for historical and real-time events
Automatically detecting which events your consumers need
Tracking progress so restarts don't lose work
Handling failures with retry logic and backoff
Supporting incremental changes when adding new features
Working seamlessly in both modular monoliths (internal clients) and distributed architectures (remote clients)

The result? Your services can truly be independent, with complete data from day one, and the ability to evolve without painful migrations.

Source Code

Full implementation is available in the VibeTDD GitLab repository.

Event Handling: Inbox Pattern for Complex Scenarios

Maksim Matlakhov — Fri, 03 Oct 2025 12:48:51 +0000

In my previous post, I covered direct event handling for simple, stateless operations. Today, I'm diving into the Inbox Pattern – a robust approach for scenarios where immediate processing isn't safe or when you need more sophisticated handling strategies.

Why Inbox Pattern?

Not all events can be processed immediately upon arrival. Some scenarios require careful, controlled processing:

Complex calculations – Long-running computations that might timeout
Document conversions – Converting HTML to PDF, processing large files is unpredictable
External service calls – Sending emails via providers like SendGrid or Mailgun
Multiple event aggregation – Combining data from several events before processing
Resource-intensive operations – Tasks that require significant CPU, memory, or I/O

The Problem with Immediate Processing

Processing events directly in the consumer can lead to serious issues:

Timeout Issues – Complex operations may exceed the message broker's in-flight time limit, causing the broker to assume failure and redeliver the message, potentially creating duplicate processing.

Retry Control Loss – When execution fails, you're at the mercy of the broker's retry policy. Most brokers use simple exponential backoff without fine-grained control over retry attempts, timing, or failure tracking.

Resource Exhaustion – High event volumes can overwhelm your service if each event triggers expensive operations synchronously.

The Inbox Pattern solves this by decoupling event reception from processing: we persist the event first, then process it asynchronously with our logic that we control.

Processing Types

Based on my experience, I've identified three distinct processing patterns:

1. Single Event Processing

Each event is processed independently without considering other events.

Example: A user status changes to "SUSPICIOUS". The fraud detection system needs to review all active orders for that user, check payment methods, and potentially flag transactions – all independently of other user status changes. In each update event I put previous and current state, so it's easy to identify a change with a single event.

2. Sequential Processing

Events for the same entity must be processed in order to maintain consistency.

Example: Calculating user reward points based on purchase history. Events like PurchaseCompleted → RefundIssued → BonusApplied must be processed in sequence per user. The final points balance depends on processing these events in the exact order they occurred.

3. Batch Processing

Multiple events are processed together for efficiency.

Example: Generating daily financial reports by collecting all transaction events throughout the day and processing them together at midnight. This avoids recalculating totals, averages, and summaries after every single transaction.

For this post, we'll focus on Single Event Processing.

Demo Architecture

To demonstrate the Inbox Pattern, I've created a modular monolith with three services:

User Service – Manages user accounts and status changes
Product Service – Handles product catalog
Order Service – Processes orders and subscribes to user and product events

Scenario: When a user's status changes, the User Service emits an event. The Order Service subscribes to these events, stores them in an inbox, and processes fraud checks when status transitions involve "SUSPICIOUS".

You can explore the complete implementation here: GitLab Repository

How It All Works Together

Let's walk through the complete flow before diving into the code:

User Service changes a user's status to SUSPICIOUS
Event is published via message broker (Kafka, RabbitMQ, etc.). In our case I use internal Spring implementation
Order Service consumer receives the event and stores it immediately in the inbox
Scheduler triggers every 10 seconds
Runner fetches pending events from the inbox database
Handler processes each event, performing fraud checks on the user's orders
On success: Event marked as SENT, processing complete
On failure: Event scheduled for retry with exponential backoff
Retry continues until either success or max retry limit reached

Key insight: The event consumer's only job is to store the event quickly and acknowledge it to the broker. All complex processing happens asynchronously, with full control over retries and failure handling.

Implementation: Fraud Detection Example

Now let's see how this is implemented in code.

Step 1: Storing Events

When an event arrives, we store it immediately with context defining how it should be processed:

@EventConsumer
class UserInboxEventsConsumerV1(
    private val storage: OrderInboxEventStorageAdapter,
) {

    fun onStatusUpdated(event: EventV1<UserStatusUpdatedV1>) {
        storage.create(
            event,
            InboxEventContext.single(InboxTopic.FRAUD)
        )
    }
}

Key points:

InboxEventContext.single() marks this for single event processing
InboxTopic.FRAUD groups related events together
The event is stored as-is – no processing happens here

Step 2: Defining Event Context

The context structure tells our system how to process events:

data class InboxEventContext(
    val topic: String,
    val processingType: ProcessingType
) {
    companion object {
        fun single(topic: String) = InboxEventContext(
            topic = topic,
            processingType = ProcessingType.SINGLE,
        )
    }
}

enum class ProcessingType {
    SINGLE,
    SEQUENTIAL,
    BATCH
}

Each stored event contains:

data class InboxEventData(
    val id: UUID,
    val context: InboxEventContext,
    val event: EventV1<out EventDtoBody>,
    val notification: Notification = Notification(),
)

Step 3: Tracking Processing State

The Notification object tracks processing attempts and status:

data class Notification(
    val status: Status = Status.PENDING,
    val attempts: Int = 0,
    val executeAt: Instant? = Instant.now(),
    val failedReasons: List<FailedReason> = listOf(),
) {
    enum class Status {
        PENDING, SENT, FAILED
    }
}

Built-in retry logic with exponential backoff:

fun Notification.toFailure(
    exception: Exception,
    maxRetries: Int,
    baseRetryIn: Int,
): Notification {
    val limitExceeded = attempts >= maxRetries
    val newAttempts = attempts + 1
    val nextExecuteAt =
        if (limitExceeded) null
        else executeAt?.plusSeconds(baseRetryIn * newAttempts.toLong())

    return copy(
        status = if (limitExceeded) Status.FAILED else Status.PENDING,
        attempts = newAttempts,
        failedReasons = failedReasons + FailedReason(
            occurredAt = Instant.now(),
            message = exception.message ?: "Unknown error"
        ),
        executeAt = nextExecuteAt,
    )
}

Step 4: Configuring the Scheduler

Define when and how often to process events:

@ConfigurationProperties(prefix = "order.inbox.single")
data class OrderSingleInboxEventProps(
    override val pageSize: Int = 20,
    override val maxRetries: Int = 15,
    override val baseRetryIn: Int = 2,
    override val executeEvery: Duration = Duration.ofSeconds(10),
    override val topic: String? = null,
) : VTInboxEventProps

Configuration breakdown:

pageSize: 20 – Process 20 events per batch
maxRetries: 15 – Retry failed events up to 15 times
baseRetryIn: 2 – Start with 2-second delays, increasing exponentially
executeEvery: 10s – Check for new events every 10 seconds
topic: null – Process all SINGLE type events (or specify a topic for granular control)

Step 5: Setting Up Processing Infrastructure

Wire everything together with Spring configuration:

@Configuration
@EnableSchedulerLock(defaultLockAtMostFor = "PT30S")
@EnableConfigurationProperties(OrderSingleInboxEventProps::class)
class OrderInboxEventConfig {

    @Bean
    fun orderSingleEventRunner(
        orderInboxEventStorageAdapter: OrderInboxEventStorageAdapter,
        orderInboxEventHandlers: List<VTInboxEventHandler<*>>,
        props: OrderSingleInboxEventProps,
    ) = VTSingleEventRunner(
        repository = orderInboxEventStorageAdapter,
        handlers = orderInboxEventHandlers,
        props = props,
    )

    @Bean
    fun orderSingleEventScheduler(
        orderMongoTemplate: MongoTemplate,
        orderSingleEventRunner: VTSingleEventRunner,
        taskScheduler: TaskScheduler,
        props: OrderSingleInboxEventProps,
    ) = VTEventScheduler(
        lockProvider = MongoLockProvider(orderMongoTemplate.db),
        eventRunner = orderSingleEventRunner,
        taskScheduler = taskScheduler,
        props = props
    )
}

Important: @EnableSchedulerLock prevents duplicate processing in distributed environments using ShedLock.

Step 6: The Event Runner

The runner fetches pending events and processes them in batches:

class VTSingleEventRunner(
    private val repository: VTInboxEventStoragePort,
    private val handlers: List<VTInboxEventHandler<*>>,
    private val props: VTInboxEventProps,
) : VTEventRunner {

    private val log = KotlinLogging.logger {}

    override fun run() {
        var events: List<InboxEventData>
        do {
            events = props.topic?.let { repository.findPendingByTopic(it, props.pageSize) }
                ?: repository.findPendingByType(ProcessingType.SINGLE, props.pageSize)
            events.forEach { processOne(it) }
        } while (events.isNotEmpty())
    }

    private fun processOne(event: InboxEventData) {
        var notification: Notification = event.notification
        try {
            log.info { "Processing inbox event: ${event.id}, topic: ${event.context.topic}" }
            handlers
                .find { it.topic == event.context.topic }
                ?.let {
                    @Suppress("UNCHECKED_CAST")
                    (it as VTInboxEventHandler<EventDtoBody>).handle(event.event as EventV1<EventDtoBody>)
                }

            notification = event.notification.toSuccess()
        } catch (e: Exception) {
            log.error(e) { "Failed to process event: ${event.id}" }
            notification = event.notification.toFailure(e, props.maxRetries, props.baseRetryIn)
        } finally {
            updateEvent(event, notification)
        }
    }

    private fun updateEvent(event: InboxEventData, notification: Notification) {
        try {
            repository.update(event.copy(notification = notification))
        } catch (e: Exception) {
            log.error(e) { "Failed to update event: ${event.id}" }
        }
    }
}

Processing flow:

Fetch pending events (by topic or processing type)
Find the appropriate handler for each event's topic
Execute the handler
Mark as success or schedule retry on failure
Continue until no pending events remain

Step 7: The Scheduler

The scheduler ensures regular processing with distributed locking:

class VTEventScheduler(
    private val eventRunner: VTEventRunner,
    private val taskScheduler: TaskScheduler,
    private val props: VTInboxEventProps,
    lockProvider: ExtensibleLockProvider
) : ApplicationListener<ContextRefreshedEvent> {

    private val schedulingLockProvider = SchedulingLockProvider(lockProvider)
    private val lockName = "inbox-${props.topic ?: eventRunner::class.simpleName}"

    override fun onApplicationEvent(event: ContextRefreshedEvent) {
        taskScheduler.scheduleWithFixedDelay(
            { processEvents() },
            props.executeEvery
        )
    }

    private fun processEvents() {
        DefaultLockingTaskExecutor(schedulingLockProvider)
            .executeWithLock(
                Runnable { eventRunner.run() },
                LockConfiguration(
                    Instant.now(),
                    lockName,
                    schedulingLockProvider.getLockAtMostFor(),
                    Duration.ZERO
                )
            )
    }
}

Step 8: Creating the Handler

Finally, implement the business logic:

@Component
class FraudTopicHandler(
    override val topic: String = InboxTopic.FRAUD,
    private val coreFactory: OrderCoreFactory
) : VTInboxEventHandler<UserStatusUpdatedV1> {

    override fun handle(event: EventV1<UserStatusUpdatedV1>) = runBlocking {
        coreFactory.fraudUseCase.execute(event.toCommand())
    }
}

fun EventV1<UserStatusUpdatedV1>.toCommand() = ManageFraudCommand(
    userId = modelId,
    previousStatus = body.previous.name,
    currentStatus = body.current.name,
)

Handler responsibilities:

Listen for events with topic FRAUD
Transform the event into a domain command
Execute the business logic (fraud check in this case)

Key Benefits

Reliability – Events aren't lost if processing fails

Observability – Track processing attempts and failures

Flexibility – Configure retry behavior per use case

Scalability – Distributed locks prevent duplicate processing

Separation of Concerns – Event reception and processing are independent

Timeout Protection – Long-running operations don't block message consumers

Coming Next

In the next post, we'll tackle a critical challenge: bootstrapping event-driven services.

When you create a new service (like our Order Service), the upstream data already exists in other services. Users and products are already in the system, but your new Order Service has none of this data. How do you populate it?

We'll explore strategies for:

Initial data synchronization when deploying new services
Backfilling events from existing data sources
Handling the gap between service creation and going live
Ensuring consistency during the bootstrap phase

This is essential for building truly independent services that don't rely on direct database access to other services' data.

The Inbox Pattern adds complexity, but for critical business operations requiring guaranteed processing, it's indispensable. Check out the complete example on GitLab to see it in action!

Event Handling: Keep It Fast and Simple

Maksim Matlakhov — Tue, 30 Sep 2025 10:21:00 +0000

When building event-driven systems, one of the biggest mistakes I see is treating all events the same way. A simple username update gets the same heavyweight processing as complex fraud detection logic. This leads to slow systems, unnecessary complexity, and hard-to-debug race conditions.

The solution? Recognize that events fall into two distinct categories and handle them differently. Simple data updates need fast, direct processing. Complex business workflows need careful orchestration. By using the right approach for each type, you can build systems that are both performant and maintainable.

In my previous post, I talked about treating messaging brokers as simple transport layers. Today, let's look at how to handle events the right way.

Three Simple Rules

Here are the basic rules I follow:

Rule 1: Order and Uniqueness

Every event needs an eventId and version. This helps you handle events in the right order and avoid processing the same event twice. Don't trust the messaging broker to do this for you.

Rule 2: Process Fast

Release events quickly. Don't do heavy work like complex calculations or calling external services right away. Store the event locally and do the heavy work later.

Rule 3: Simple vs Complex

Some events are simple (like updating a user's name), others are complex (like fraud detection). Handle them differently.

Two Types of Events

From my experience, events fall into two groups:

Type 1: Simple Updates

Easy data updates with no business rules:

Update search indexes
Sync read models
Clear caches

Type 2: Complex Work

Multistep processes with business rules:

Fraud detection
Complex workflows
External service calls
Business rule checks

The difference matters because each type needs different handling.

How Type 1 Processing Works

For simple updates, here's what happens when an event comes in:

Get Event: Consumer receives event from messaging broker
Check Version: Compare event version with stored data version
Skip Old Events: If event.version ≤ stored.version, ignore it
Update Data: Create new versioned data from event
Save Changes: Update database with optimistic locking
Handle Conflicts: Database throws an exception if someone else updated the record first
Retry: Message broker retries the event automatically

This gives you:

Fast Processing: Direct database updates
Safe Concurrency: Optimistic locking prevents conflicts
Version Control: Prevents old events from overwriting new data
Auto Recovery: Message broker handles retries

Type 1 Implementation

For simple updates, I use a generic helper that handles versioning and prevents conflicts.

The Generic Helper

Here's the main helper class that handles all simple updates:

// Generic orchestrator that works with any model type
abstract class VTViewOrchestrator<MODEL: Any, DATA: Any>(
    private val modelStorage: VTModelStorageAdapter<MODEL, DATA>,
) {

    // Create new model from event
    fun <EVENT: EventDtoBody> create(
        event: EventV1<EVENT>,
        createModel: (EVENT) -> MODEL
    ) {
        // Don't create if already exists
        if (modelStorage.get(event.modelId) != null) return

        // Create new model with version 0
        modelStorage.create(
            Model(
                id = event.modelId,
                version = 0,
                createdAt = event.createdAt,
                updatedAt = event.createdAt,
                body = createModel(event.body)
            )
        )
    }

    // Update specific field in existing model
    fun <EVENT: EventDtoBody, FIELD: Any> update(
        event: EventV1<EVENT>,
        extractField: (MODEL) -> VersionedField<FIELD>, // Get current field
        updateModel: (MODEL, VersionedField<FIELD>) -> MODEL, // Set new field
        buildFieldData: (EVENT) -> FIELD // Build new data from event
    ) {
        // Get existing model, throws if not found (for cases when an update event comes first)
        val storedModel: Model<MODEL> = modelStorage.getRequired(event.modelId)
        val currentField: VersionedField<FIELD> = extractField(storedModel.body)

        // Skip if this event is older than stored data
        if (currentField.version >= event.version) return

        // Create new field with event data and version
        val newField: VersionedField<FIELD> = VersionedField(
            version = event.version,
            data = buildFieldData(event.body)
        )

        // Update model with new field
        modelStorage.update(
            storedModel.copy(
                updatedAt = event.createdAt,
                body = updateModel(storedModel.body, newField)
            )
        )
    }
}

Versioned Fields for Concurrency

The key is using versioned fields so different parts can update independently:

// Each field has its own version
data class UserView(
    val personalData: VersionedField<PersonalData>, // Version for name, email, etc.
    val status: VersionedField<Status<String>>       // Version for status changes
)

// Wrapper that tracks version for any data type
data class VersionedField<T>(
    val version: Long = 0,
    val data: T,          
)

Event Consumer

Here's how you use the helper to handle events:

@EventConsumer
class UserEventsConsumerV1(
    private val orchestrator: UserViewOrchestrator,
) {

    fun onCreated(event: EventV1<UserCreatedV1>) {
        orchestrator.create(event) {
            // Build initial user view from event
            UserView(
                personalData = VersionedField(
                    data = PersonalData(
                        name = event.body.personalData.name,
                    )
                ),
                status = VersionedField(
                    data = Status(
                        name = event.body.status.name,
                        message = event.body.status.message
                    )
                )
            )
        }
    }

    fun onPersonalDataUpdated(event: EventV1<PersonalDataUpdatedV1>) {
        orchestrator.update(
            event = event,
            extractField = { it.personalData },        // Get current personal data field
            updateModel = { model, field ->             // How to update the model
                model.copy(personalData = field) 
            },
            buildFieldData = {                          // Build new data from event
                PersonalData(name = it.current.name) 
            }
        )
    }

    fun onStatusUpdated(event: EventV1<UserStatusUpdatedV1>) {
        orchestrator.update(
            event = event,
            extractField = { it.status },             
            updateModel = { model, field ->            
                model.copy(status = field) 
            },
            buildFieldData = {                         
                Status(name = it.current.name, message = it.current.message) 
            }
        )
    }
}

The Service Setup

The concrete orchestrator is just a Spring component:

@Component
class UserViewOrchestrator(
    storage: UserViewStorage, // Handles database operations
) : VTViewOrchestrator<UserView, UserViewDoc>(storage)

And the mongo document with common version field that prevents concurrent update:

@Document("view-users")
data class UserViewDoc(
    val id: UUID,
    @field:Version // Does the magic
    val version: Long,
    val createdAt: Instant,
    val updatedAt: Instant,
    val data: UserView,
)

Why This Works So Well

The generic helper pattern gives you several benefits:

Same Pattern Everywhere: All services use the same approach for handling events
Reusable Code: One helper works for all your models

Type Safety: Kotlin ensures you can't make mistakes with field types
Easy to Maintain: Fix bugs in one place, all services benefit
Fast Performance: Direct database updates without extra overhead

The versioned field approach solves a common problem: event ordering. Instead of trusting the message broker to deliver events in order (which I don't recommend), each field tracks its own version.

This means a "name updated" event with version 5 won't overwrite data from a "status updated" event with version 7. Each field can be updated independently without conflicts.

At the same time the common version field prevents a model inconsistency when different field updates come simultaneously.

What's Next: Complex Events

Type 1 events handle most of your event processing needs efficiently. But what about complex business logic that needs multiple steps, external service calls, or special error handling?

That's where Type 2 events come in. In my next post, I'll show you how to handle complex business workflows using internal event storage and Spring's event system, while keeping your architecture clean.

The main idea is simple: recognize that not all events need the same processing approach. By classifying events and using the right pattern for each type, you can build systems that are both fast and easy to maintain.

Messaging Broker Migration: Why Lock-in Hurts and How to Avoid It

Maksim Matlakhov — Mon, 29 Sep 2025 11:18:00 +0000

Have you ever struggled with messaging broker migration? I've heard this opinion countless times: "Why do we need that? We're on Amazon and use SNS/SQS - it works perfectly." While that might be true today, what happens when things change?

The reality is that business needs change, and what works today might not work tomorrow. Let me share why messaging broker lock-in is a problem and how to build systems that can adapt easily.

When You Need to Migrate

Even the most satisfied AWS (or another provider) users might face situations where migration becomes needed:

Business Opportunities: You're a startup built on AWS and Google offers you $200K in credits through their startup program. That's real money that could help your business grow.

Market Innovation: A new messaging broker appears that's much more efficient and cheaper than your current solution. Do you want to be stuck with old technology?

Strategic Changes: You decide to build your own messaging provider or move to a completely different setup.

Compliance Requirements: New regulations require data to stay in specific regions that your current provider doesn't support.

Performance Problems: Your current broker can't handle growing message volumes (or can but igh cost) or speed requirements, but alternatives can.

Cost Problems: What happens when your provider decides to raise prices a lot? You need options.

I've seen some of these situations, and those with tightly connected messaging systems found themselves in painful situations.

The Migration Nightmare: Common Anti-Patterns

Anti-Pattern 1: Broker as Event Storage

The worst case from my perspective is when teams use the broker as their primary event storage. You're essentially married to that provider forever, and they can do whatever they want - including increasing costs dramatically.

Anti-Pattern 2: Dual-Send Without Atomicity

Some teams think they can solve migration by adding a new line to send messages to a new provider alongside the existing one. Apart from the original timing problems you had with two operations (storing the model and sending the message), you now have a third operation. What happens when one of them fails?

class OrderService(
    private val orderRepository: OrderRepository,
    private val kafkaPublisher: KafkaPublisher,
    private val rabbitPublisher: RabbitPublisher,
) : BrokerProducer {

    override fun save(order: Order) {
        // ...
        orderRepository.save(order)
        kafkaPublisher.send(order)
        rabbitPublisher.send(order) // One more point of failure to prevent an order saving?
    }
}

Anti-Pattern 3: Feature-Dependent Architecture

Many systems become deeply dependent on broker-specific features:

FIFO queue guarantees for ordering
Exactly-once delivery promises
Dead letter queue mechanisms
Broker-specific retry policies

When migration time comes, copying these features across different brokers becomes a complex engineering challenge.

The Path to Broker Independence

To make migration easier, we need to treat message brokers as a transport layer only and avoid using broker-specific features. Here's how:

Design Principles

Don't Rely on Event Ordering: Instead of depending on FIFO queues, make your consumers handle ordering. Each event should have a unique ID and creation timestamp to help manage sequencing.

Handle Duplicates Well: Don't rely on exactly-once delivery guarantees. Design your consumers to handle the same message multiple times without problems.

Avoid Dead Letter Queues: If you store events in your database, you don't need broker-managed dead letter queues. I remember how painful it was to manage and restore events from dead letter queues.

Control Your Own Destiny: Store events in your database where you control delivery status, error handling, and retries. If attempts reach a limit due to network issues, it's easy to resend and retry because you control your database.

The Migration Strategy: Multi-Broker Publishing

Here's how I designed the system to handle broker migration seamlessly:

The `processedBrokers` Field

Apart from storing multiple event DTO versions (which I described in my previous post), I store a processedBrokers field with each event. During event execution, the system iterates through all implementations of BrokerProducer.

Here's how an event looks in MongoDB with the processedBrokers tracking:

{
  "_id": "1cc64bb3-17f0-3619-a68d-a2564bf1644d",
  "topic": "user.model.created.v1",
  "event": {
    "body": {...},
    "metadata": {...}
    }
  },
  "notification": {
    "status": "SENT",
    "attempts": 1,
    "failedReasons": [],
    "processedBrokers": [
      "kafka",
      "spring"
    ]
  },
}

Notice the processedBrokers array contains both "kafka" and "spring", indicating this event was successfully sent to both brokers. The attempts counter and failedReasons array help track delivery issues and retry logic.

This approach allows us to:

Map multiple brokers to the same event
Send the same message to all configured brokers
Avoid resending to already-processed brokers during retries
Handle both external brokers (Kafka, SQS) and internal events (Spring pub/sub)
Track delivery status per broker independently

The Spring Pub/Sub Example

Let me walk you through a concrete example using the code snippets you provided. We'll look at how the system handles internal Spring pub/sub messaging, which demonstrates the same principles used for external broker migration.

Producer Implementation

@Component
@ConditionalOnProperty(
    value = ["vt.events.producer.type.spring.enabled"],
    havingValue = "true",
    matchIfMissing = true,
)
class SpringBrokerProducer(
    private val eventPublisher: ApplicationEventPublisher,
) : BrokerProducer {

    override val broker: String = "spring"

    override fun send(topic: String, event: BrokerMessageV1) {
        eventPublisher.publishEvent(PayloadApplicationEvent(topic, event))
    }
}

The SpringBrokerProducer implements the same BrokerProducer interface as our Kafka, SQS, or RabbitMQ producers. The system automatically discovers all implementations and sends events to each configured broker.

Consumer Implementation

@Component
@ConditionalOnProperty(
    value = ["vt.events.consumer.type.spring.enabled"],
    havingValue = "true",
    matchIfMissing = true
)
class SpringEventListener(
    private val topicSubscriber: SpringTopicSubscriber,
) {

    private val log = KotlinLogging.logger {}

    @EventListener
    fun onEvent(event: PayloadApplicationEvent<BrokerMessageV1>) {
        val topic = event.source as String
        val consumers: MutableList<Consumer>? = topicSubscriber.topicConsumers[topic]
        if (consumers.isNullOrEmpty()) {
            log.debug { "No consumers found for topic '$topic'" }
            return
        }

        consumers.forEach { consumer ->
            try {
                log.debug { "Processing event for topic '$topic' with consumer '${consumer.type.simpleName}'" }
                consumer.handler(event.payload)
            } catch (exception: Exception) {
                log.error(exception) { "Error in consumer handler for topic '$topic': ${event.payload}" }
            }
        }
    }
}

The beauty of this design is that the same consumer code works regardless of the underlying broker. The framework abstracts away the transport layer completely.

Topic Subscription

@Component
@ConditionalOnProperty(
    value = ["vt.events.consumer.type.spring.enabled"],
    havingValue = "true",
    matchIfMissing = true
)
class SpringTopicSubscriber : TopicSubscriber {

    private val log = KotlinLogging.logger {}

    private val topicConsumers = mutableMapOf<String, MutableList<Consumer>>()

    override fun subscribe(topic: String, consumers: MutableList<Consumer>) {
        topicConsumers[topic] = consumers
        log.info("Registered consumers: ${consumers.map { it.type.simpleName }} for topic: $topic")
    }
}

Configuration Management

Notice how both producer and consumer use @ConditionalOnProperty annotations. This allows us to control which brokers are active through configuration:

# Enable Spring internal messaging
vt.events.producer.type.spring.enabled=true
vt.events.consumer.type.spring.enabled=true

# Enable Kafka
vt.events.producer.type.kafka.enabled=true
vt.events.consumer.type.kafka.enabled=true

Maven Module Organization

The modular structure from your pom.xml files shows another layer of flexibility:

<!-- Spring messaging module -->
<dependency>
    <groupId>dev.vibetdd.kotlin</groupId>
    <artifactId>vt-messaging-consumer-spring</artifactId>
</dependency>

<!-- Kafka messaging module -->
<dependency>
    <groupId>dev.vibetdd.kotlin</groupId>
    <artifactId>vt-messaging-consumer-kafka</artifactId>
</dependency>

You can manage broker support by including/excluding Maven modules, similar to the API client approach I described in the monolith split post.

The Migration Process in Action

Here's how a typical migration would work with this system:

Phase 1: Producer Preparation

Add New Broker Module: Include the new broker's Maven dependency
Deploy Producer: The system automatically starts sending to both brokers

Phase 2: Consumer Migration

Update Client Dependencies: Consumer services update their event library dependency
Gradual Migration: Each consumer team switches to the new broker when ready
No Coordination Required: Teams work independently during the transition period

Phase 3: Cleanup

Monitor Delivery: Ensure all consumers successfully receive events from the new broker
Remove Old Broker: Disable the old broker configuration
Clean Dependencies: Remove old broker Maven modules

Benefits of This Approach

This approach is particularly valuable for:

Startup Growth: Early-stage companies can start with simple solutions using internal messaging and evolve to enterprise-grade messaging without architectural rewrites.

Monolith Migration: When extracting services from a monolith, you can start replacing internal messaging to external brokers as services become independent.

Cloud Migration: Moving between cloud providers becomes much less risky when your messaging layer can adapt.

Compliance Changes: When regulations require data to stay in specific regions, you can quickly change your messaging setup.

Conclusion

Messaging broker lock-in is a real problem that teams often don't recognize until it's too late. By designing your event-driven architecture with broker independence from the start, you maintain the flexibility to adapt as your business needs evolve.

The key insights are:

Treat brokers as transport layers, not feature providers
Store events in your database for complete control
Use abstraction layers that hide broker-specific implementation details
Design for multi-broker publishing from day one

Remember: the best time to prepare for migration is when you don't need it yet. By the time migration becomes urgent, it's often too late to implement these patterns without significant pain.

Your future self (and your team) will thank you for building systems that can adapt rather than systems that lock you in.

Seamless Events Version Management

Maksim Matlakhov — Sun, 28 Sep 2025 10:17:18 +0000

One of the biggest pain points in event-driven systems comes when you need to make breaking changes to your event structure. I've seen teams struggle with this countless times: a simple field rename or restructuring forces a complex, coordinated deployment across multiple services, sometimes requiring temporary downtime or complicated migration strategies.

In some systems I've worked with, when you change an event structure, you face a chicken-and-egg problem. Consumers need to understand the new format before producers can start sending it, but producers can't stop sending the old format until all consumers are updated. This creates a deployment nightmare, especially in distributed systems where different teams own different services.

The Pain of Breaking Changes

Consider a scenario: your user registration event initially had a simple name field, but business requirements now demand separate firstName and lastName fields for better personalization. Additionally, for privacy reasons, you need to stop storing the email field (we have a better place for that: Firebase, Auth0 etc).

From my experience, this scenario typically requires:

Coordinated deployment: All consumers must be updated before the producer changes
Complex versioning logic: Producers need conditional logic to send different versions
Temporary compatibility layers: Extra code to handle both old and new formats
Risk of service disruption: Any mistake in the coordination can break the system

Opening the Door to Seamless Evolution

My framework takes a different approach - it keeps the door open for evolution from day one. Instead of treating event versions as mutually exclusive, the system assumes that a single model event can be mapped and published to multiple DTO event versions simultaneously.

Here's the key insight: each event version has the same core metadata (eventId, modelId, version, timestamp) but different topic names that include the DTO version (.v1, .v2, etc.). The framework automatically detects all available mappers and publishes to every configured version.

Step-by-Step Migration Process

Let's walk through our example transformation using the code provided:

1. Create the New Event Structure

First, we define the new data structure with separated name fields:

data class PersonalDataV2(
    val firstName: String,
    val lastName: String,
)

data class UserCreatedV2(
    val personalData: PersonalDataV2,
    val status: StatusV1
) : EventDtoBody

2. Define the New Topic Version

Create a new topic enum that follows the same pattern but with version 2:

enum class UsersEventTopicV2(
    override val eventClass: KClass<out EventDtoBody>,
    override val action: EventAction,
    override val model: String = "user",
    override val version: Int = 2, // Increment the version
) : EventTopic {
    CREATED(UserCreatedV2::class, EventAction.created()),
}

This generates the topic user.model.created.v2 alongside the existing user.model.created.v1.

3. Implement the New Mapper

The framework automatically discovers mappers through Spring's component scanning:

@Component
class UserCreatedMapperV2 : EventMapper<UserCreated, UserCreatedV2>(
    modelClass = UserCreated::class,
    topic = UsersEventTopicV2.CREATED
) {

    override fun UserCreated.mapToDto(): UserCreatedV2 {
        val nameParts: List<String> = personalData.name.split(" ")
        return UserCreatedV2(
            personalData = PersonalDataV2(
                firstName = nameParts.firstOrNull().orEmpty(),
                lastName = nameParts.lastOrNull().orEmpty(),
            ),
            status = StatusV1(
                name = status.name.name,
                message = status.message
            )
        )
    }

    override fun UserCreatedV2.mapToModel() = UserCreated(
        personalData = PersonalData(
            name = "${personalData.firstName} ${personalData.lastName}",
            email = "",
        ),
        status = Status(
            name = UserStatus.valueOf(status.name),
            message = status.message
        )
    )
}

Notice how the mapping handles the transformation:

Splits the existing name field into firstName and lastName
Removes the email field from the V2 event
Provides reverse mapping for transforming DTO to the event model

4. Update Consumers at Your Own Pace

On the consumer side, teams can migrate independently:

@EventConsumer
class UserEventsConsumerV2 {

    fun onCreated(event: EventV1<UserCreatedV2>) {
       // Handle me
    }
}

The consumer simply needs to:

Create a new consumer class
Subscribe to the V2 topic
Handle the new event structure

5. The Magic Happens Automatically

Once you deploy the producer service with both mappers, the system automatically:

Detects both V1 and V2 mappers for the UserCreated model
Maps each event to both DTO versions
Stores both versions in the event store
Publishes to both topics (user.model.created.v1 and user.model.created.v2)

Event Storage: Both Versions Coexist

Here's how the events look in MongoDB - notice they share the same metadata but have different topic names and body structures:

// V1 Event
{
  "_id": "1cc64bb3-17f0-3619-a68d-a2564bf1644d",
  "topic": "user.model.created.v1",
  "event": {
    "body": {
      "personalData": {
        "name": "John Smith",
        "email": "example@vibetdd.dev"
      },
      "status": {
        "name": "ACTIVE"
      }
    },
    "metadata": {
      "eventId": "befadebf-66ff-3c50-a068-8468856189d8",
      "modelId": "fc619c9e-23aa-3210-bad2-d69bc2e35e13",
      "version": 1,
      "createdAt": "2025-09-25T15:13:15.772+0000"
    }
  },
  "notification": {
    "status": "SENT",
    "processedBrokers": ["kafka"]
  }
}

// V2 Event  
{
  "_id": "ea196b5c-0da0-30fd-9c11-92d9f3dc5b37",
  "topic": "user.model.created.v2", 
  "event": {
    "body": {
      "personalData": {
        "firstName": "John",
        "lastName": "Smith"
      },
      "status": {
        "name": "ACTIVE"
      }
    },
    "metadata": {
      "eventId": "befadebf-66ff-3c50-a068-8468856189d8",
      "modelId": "fc619c9e-23aa-3210-bad2-d69bc2e35e13", 
      "version": 1,
      "createdAt": "2025-09-25T15:13:15.772+0000"
    }
  },
  "notification": {
    "status": "SENT",
    "processedBrokers": ["kafka"]
  }
}

Cleanup: Removing the Old Version

Once you've confirmed all consumers have migrated to V2, cleanup is straightforward:

Remove the UserCreatedV1 class
Remove the UserCreatedMapperV1
Remove the UsersEventTopicV1.CREATED enum entry
Remove the PersonalDataV1 class if no longer used

The system will automatically detect that only the V2 mapper exists and publish only the V2 events going forward.

Benefits of This Approach

Zero-Downtime Migration: Consumers and producers can be updated independently without any service interruption.

Gradual Rollout: You can migrate consumers one by one, testing each step thoroughly before proceeding.

Rollback Safety: If issues arise, you can quickly revert consumers to the V1 topic while investigating.

Audit Trail: Both event versions are preserved in storage, providing a complete history of what was sent to different consumers.

No Coordination Required: Teams can work independently without complex deployment choreography.

The Framework's Philosophy

This design reflects a core philosophy: embrace change rather than fight it. Instead of treating event evolution as a problem to be solved with complex tooling, the framework makes evolution a first-class citizen. By assuming events will evolve and preparing for it from the start, we eliminate the pain that traditionally comes with event schema changes.

The automatic discovery and mapping approach also makes the system AI-friendly - when you need to create a new event version, you simply follow the established patterns, and the framework handles the rest. No manual registration, no complex configuration files, just clean, predictable code that works.

Introducing a Hybrid Event Sourcing Framework for Modern Applications

Maksim Matlakhov — Fri, 26 Sep 2025 10:40:24 +0000

Event sourcing has gained significant traction in recent years, promising complete audit trails, temporal queries, and robust system architecture. However, pure event sourcing often introduces complexity that can overwhelm development teams. Today, I want to introduce a hybrid event sourcing approach I've integrated into my framework that captures the benefits of event sourcing while maintaining operational simplicity.

The framework is designed to be compatible with the Event Modeling methodology, strictly following the command/event/read model pattern with clear boundaries between these building blocks. However, it's not limited to Event Modeling - we use a similar approach in my current company, and it works very well in practice.

The Challenge with Pure Event Sourcing

Based on my experience trying to implement various event-driven systems, pure event sourcing comes with real-world challenges that I've encountered:

Race Conditions and Conflict Resolution

One of challenges I've faced is managing race conditions in update operations. Consider this scenario: two users update a product status simultaneously. In pure event sourcing, this typically requires:

Optimistic locking mechanisms - Using unique constraints, something like {aggregate_id, sequence_id}
Conflict resolution logic - Determining which update wins and handling the "loser"
Retry mechanisms - Failed operations must retry based on the latest system state
User notification - Informing users about conflicts and requiring manual resolution

While these solutions work, they add significant complexity to the system. Event sourcing advocates often suggest that optimistic locking makes this "much simpler," but in my experience, implementing robust conflict resolution that handles all edge cases gracefully requires substantial engineering effort or relying on a magic using an event sourcing framework.

Read Model Performance and Complexity

Building read models from events presents several practical challenges:

Event replay overhead - Even with tons of events, projection rebuilding can be time-consuming
Multiple projection maintenance - Each new query pattern requires a new projection
Event schema evolution - Changing event structures requires migration of all dependent projections
Eventual Consistency - Read models are eventually consistent by nature, but can create user experience issues when immediate read-after-write consistency is expected
Debugging complexity - Troubleshooting issues requires understanding the entire event history

While the flexibility of having multiple projections (user_history, user_orders, disabled_users) is powerful, it comes with operational overhead that many teams underestimate.

The Root Issue

I want to clarify that these aren't inherent flaws in event sourcing - they're general distributed system challenges that pure event sourcing doesn't solve automatically. The theoretical benefits are compelling, but the practical implementation complexity often outweighs the advantages for many use cases.

My approach addresses these challenges by combining events with current state storage, making conflict resolution simpler and basic read operations more straightforward while preserving the audit trail and integration benefits that make event sourcing attractive.

My Hybrid Approach

Instead of pure event sourcing, I've developed a hybrid system that combines the audit trail benefits of events with the simplicity of current state storage. Here's how it works:

Core Principles

Generate events for every action: User created, status changed, payout requested, order canceled
Events belong to specific models: Similar to aggregates in event sourcing terms
Transactional consistency: Events and model updates happen in a single transaction
Independent model parts: Different aspects of a model can be updated independently with their own versioning

Model Structure

My models can be simple or have multiple parts that update independently:

// Simple model - single entity
data class Comment(
    val content: String,
    val author: String
)

// Complex model - multiple independent parts
data class Product(
    val description: ProductDescription,  // Can be updated independently
    val status: Status,                   // Can be updated independently  
    val price: ProductPrice               // Can be updated independently
)

This design eliminates false conflicts. An admin updating product description won't conflict with another admin updating pricing, as they operate on different parts with separate versioning.

Events are controlled through a composite identifier consisting of modelId + action + version. Here are examples of how events are stored in MongoDB:

User Created Event:

{
  "_id": "6b846115-e41c-35db-ab27-12f8b3e99591",
  "topic": "user.model.created.v1",
  "event": {
    "body": {
      "personalData": {
        "name": "John Smith",
        "email": "example@vibetdd.dev"
      },
      "status": {
        "name": "ACTIVE"
      }
    },
    "metadata": {
      "eventId": "454460ab-cfb5-3b7d-a9c6-e39f13f2dd23",
      "modelId": "a1a87c49-670e-3844-a2df-368c77f207a9",
      "version": 1,
      "createdAt": "2025-09-25T09:51:07.712Z"
    }
  }
}

Personal Data Updated Event:

{
  "_id": "cef0220d-cd94-3aa3-af33-b68a7f3d0db9",
  "topic": "user.personal-data.updated.v1",
  "event": {
    "body": {
      "previous": {
        "name": "John Smith",
        "email": "example@vibetdd.dev"
      },
      "current": {
        "name": "Will Smith", 
        "email": "example@vibetdd.dev"
      }
    },
    "metadata": {
      "eventId": "3ae2cd7d-cba0-37cd-a6d4-3f0145571d4c",
      "modelId": "a1a87c49-670e-3844-a2df-368c77f207a9",
      "version": 1,
      "createdAt": "2025-09-25T09:52:03.046Z"
    }
  }
}

Event Storage Architecture

The event storage follows a clean separation:

Database per domain/service: Each service maintains its own events
Common event collection: All event types stored in a single table/collection
Current state storage: Separate storage for model current state
Message broker integration: Events processed asynchronously for consumers
Multiple event versions: Enables seamless migration between event DTO versions (this will be covered in a separate post)

A background processor constantly polls for pending events, determines which message brokers to send to, handles errors, and manages retries.

Example: Users and Payouts Services

Let's see how this works in practice with two services: users and payouts. The users service handles CRUD operations and sends events for every operation, while the payouts service consumes user events to make business decisions.

Users Service

The users service demonstrates the complete event creation flow. For updates, there are two cases: updating personal data with version control and status updates without version control.

Personal data updates require version control - the client must request the current version before updating and send it back. If versions don't match, a conflict exception is thrown. Status updates, however, don't require version control as they're considered independent operations.

Event Creation

Every operation starts with creating an event. Here's how I create a new user:

class CreateUserUseCase(
    private val idProvider: IdProvider<UUID>,
    private val validator: CommandValidator,
    private val eventOrchestrator: EventOrchestrator<User>
) : SaveCommandUseCase<CreateUserCommand, User> {

    override suspend fun execute(command: CreateUserCommand): Model<User> {
        // Step 1: Validate the incoming command
        validator.validate(command)

        // Step 2: Create the event command with all necessary data
        val event = CreateEventCommand(
            modelId = idProvider.generate(command.data.email), // Generate deterministic ID from email
            actor = command.actor, // Track who performed the action
            body = UserCreated( // The actual event body with business data
                personalData = PersonalData(
                    email = command.data.email,
                    name = command.data.name,
                ),
                status = Status(UserStatus.ACTIVE) // Default status for new users
            )
        )

        // Step 3: Use event orchestrator to persist event and create model in single transaction
        return eventOrchestrator.create(event) {
            // This lambda defines how to build the model from the event
            User(
                personalData = it.personalData,
                status = it.status
            )
        }
    }
}

The business logic only needs to:

Build the proper event
Create mapping to the related model
Let the system handle storage, processing, and notification

Event Topic Structure

Every event is mapped to a topic following the format: model.subject.action.dto-version

enum class UsersEventTopicV1(
    override val eventClass: KClass<out EventDtoBody>,
    override val action: EventAction,
    override val model: String = "user",
    override val version: Int = 1,
) : EventTopic {
    CREATED(UserCreatedV1::class, EventAction.created()),
    DELETED(UserDeletedV1::class, EventAction.deleted()),
    PERSONAL_DATA_UPDATED(PersonalDataUpdatedV1::class, EventAction.updated(UserModelSubject.PERSONAL_DATA)),
    STATUS_UPDATED(UserStatusUpdatedV1::class, EventAction.updated(UserModelSubject.STATUS))
}

object UserModelSubject {
    const val PERSONAL_DATA = "personal-data"
    const val STATUS = "status"
}

This generates topics like:

user.model.created.v1
user.personal-data.updated.v1
user.status.updated.v1

Event Mapping and Versioning

Events are mapped between internal models and external DTOs using dedicated mappers:

@Component
class UserCreatedMapper : EventMapper<UserCreated, UserCreatedV1>(
    modelClass = UserCreated::class,
    topic = UsersEventTopicV1.CREATED
) {

    override fun UserCreated.mapToDto() = UserCreatedV1(
        personalData = PersonalDataV1(
            name = personalData.name,
            email = personalData.email,
        ),
        status = StatusV1(
            name = status.name.name,
            message = status.message
        )
    )

    override fun UserCreatedV1.mapToModel() = UserCreated(
        personalData = PersonalData(
            name = personalData.name,
            email = personalData.email,
        ),
        status = Status(
            name = UserStatus.valueOf(status.name),
            message = status.message
        )
    )
}

This mapping layer enables:

Version migration: Multiple versions of the same event can coexist
Backward compatibility: Consumers can upgrade at their own pace
Clean boundaries: Internal models remain separate from external contracts

Handling Updates

The framework supports two versioning approaches:

With Version Control - For critical data that requires conflict detection:

class UpdateUserPersonalDataUseCase(
    private val validator: CommandValidator,
    private val modelStorage: UserStoragePort,
    private val eventOrchestrator: EventOrchestrator<User>
) : SaveCommandUseCase<UpdateUserPersonalDataCommand, User> {

    override suspend fun execute(command: UpdateUserPersonalDataCommand): Model<User> {
        // Step 1: Get the current stored model
        val storedModel: Model<User> = modelStorage.getRequired(command.target.id)
        val updated: PersonalData = buildUpdated(storedModel, command) ?: return storedModel

        // Step 2: Validate the command
        validator.validate(command)

        // Step 3: Create event with expected version for conflict detection
        val event = CreateEventCommand(
            modelId = command.target.id,
            expectedVersion = command.target.versionRequired("Update personal data"), // Version control
            actor = command.actor,
            body = PersonalDataUpdated(
                previous = storedModel.body.personalData,
                current = updated,
            )
        )

        // Step 4: Update model with event orchestrator
        return eventOrchestrator.update(event) { event, model ->
            model.copy(
                personalData = event.current
            )
        }
    }

    // If there are no changed then return the same version
    private fun buildUpdated(storedModel: Model<User>, command: UpdateUserPersonalDataCommand): PersonalData? {
        val updated = storedModel.body.personalData.copy(
            name = command.data.name,
        )
        return if (storedModel.body.personalData == updated) null else updated
    }
}

Without Version Control - For independent updates where conflicts are acceptable:

class UpdateUserStatusUseCase(
    private val modelStorage: UserStoragePort,
    private val eventOrchestrator: EventOrchestrator<User>
) : SaveCommandUseCase<UpdateUserStatusCommand, User> {

    override suspend fun execute(command: UpdateUserStatusCommand): Model<User> {
        // Step 1: Get the current stored model
        val storedModel: Model<User> = modelStorage.getRequired(command.target.id)
        val updated: Status<UserStatus> = buildUpdated(storedModel, command) ?: return storedModel

        // Step 2: Create event without expected version (no version control)
        val event = CreateEventCommand(
            modelId = command.target.id,
            actor = command.actor, // No expectedVersion parameter
            body = UserStatusUpdated(
                previous = storedModel.body.status,
                current = updated,
            )
        )

        // Step 3: Update model with event orchestrator
        return eventOrchestrator.update(event) { event, model ->
            model.copy(
                status = event.current
            )
        }
    }

    private fun buildUpdated(storedModel: Model<User>, command: UpdateUserStatusCommand): Status<UserStatus>? =
        if (storedModel.body.status.name == command.data.status) null
        else Status(command.data.status)
}

Payouts Service

The payouts service demonstrates event consumption. It includes the users service events client dependency and creates a consumer to handle relevant user events.

Add Dependency

Include the events client dependency in your service:

<dependency>
    <groupId>vt.demo.service</groupId>
    <artifactId>users-client-events</artifactId>
    <version>${client.users.version}</version>
</dependency>

Create Consumer

Create a consumer class and annotate methods for events you want to handle:

@EventConsumer
class UserEventsConsumerV1 {

    fun onCreated(event: EventV1<UserCreatedV1>) {
        // Handle me
    }

    fun onStatusUpdated(event: EventV1<UserStatusUpdatedV1>) {
        // Handle me
    }
}

That's it! The service automatically receives events regardless of the transport mechanism (Kafka, RabbitMQ, SQS).

Conclusion

This hybrid event sourcing approach has proven highly effective in production systems I worked. It provides:

Audit trail completeness without operational complexity
Event-driven integration without pure event sourcing overhead
Conflict resolution that matches business requirements
Developer productivity with familiar patterns

The key insight is that you don't need pure event sourcing to get most of its benefits. By combining current state storage with comprehensive event logging, I achieve the best of both worlds: operational simplicity and event-driven architecture benefits.

The framework handles the complexity of event processing, message broker integration, and version management, letting developers focus on business logic rather than infrastructure concerns.

For teams considering event sourcing, I highly recommend exploring hybrid approaches. You might find, as I did, that the benefits are compelling while the operational burden remains manageable.

VibeTDD Experiment 4.7: Commons and Examples Compilation - Making AI Smarter Through Better Documentation

Maksim Matlakhov — Mon, 15 Sep 2025 13:51:55 +0000

The Documentation Efficiency Problem

In Phase 4.6, I split the monolithic repository into modular commons libraries to prevent AI from touching unwanted code. While this solved the physical boundaries problem, it created a new challenge: AI doesn't inherently know about custom libraries.

Asking Claude to read JAR files or external documentation disrupts the development workflow. At the same time, providing full commons code to AI is inefficient - it's a heavy task that includes implementation details AI doesn't need for most operations.

The solution became clear: compile commons signatures and examples into AI-friendly documentation that provides exactly what AI needs, when it needs it.

The Compilation Strategy

What AI Actually Needs

Through experimentation, I discovered AI needs different levels of detail for different tasks:

For Commons Libraries: Class signatures, method interfaces, and comments - but not implementation details. AI needs to understand what's available and how to use it, not how it works internally.

For Implementation Examples: Full code with all implementation details, comments, and test scenarios. When AI is building similar functionality, it needs complete patterns to follow.

For Constants and Configuration: All possible values and their meanings. These are reference data AI should have complete access to.

The Compilation Projects

I created two key repositories to handle this documentation strategy:

Framework Prompts: Contains compilation instructions for different types of documentation, split by purpose:

commons.md: Instructions for compiling commons libraries to signatures only
service-impl-main.md: Full implementation examples for AI to follow
service-impl-test.md: Complete test scenarios and patterns
service-objects-main.md: Object structures with empty method stubs for TDD
service-objects-test.md: Test utilities, fakes, and object mothers

Framework Examples: Stores the compiled markdown files that AI can easily read and reference during development.

The Compilation Process in Action

Commons Compilation Example

Here's how commons compilation transforms verbose library code into AI-friendly documentation:

Before (Full Implementation):

/**
 * Validates commands using a list of validation rules with async execution.
 * Supports parallel validation for improved performance.
 */
open class CommandValidator(
    private val rules: List<ValidationRule<*>>,
    coroutines: Int = 5,
) {

    private val validationDispatcher = Dispatchers.IO.limitedParallelism(
        parallelism = coroutines,
        name = "command-validator"
    )

    /**
     * Validates command and throws ValidationException if errors found
     */
    suspend fun validate(command: Command) {
        val errors = validateAndGetErrors(command)
        if (errors.isNotEmpty()) throw ValidationException(errors)
    }

    /**
     * Validates command and returns list of validation errors without throwing
     */
    suspend fun validateAndGetErrors(command: Command): List<ValidationError> = withContext(validationDispatcher) {
        rules
            .filter { it.isApplicable(command) }
            .map { async { it.validateOne(command) } }
            .awaitAll()
            .filterNotNull()
    }

    @Suppress("UNCHECKED_CAST")
    private fun ValidationRule<*>.validateOne(command: Command): ValidationError? {
        val input: Any = command.toInput() ?: return null
        val typedRule: ValidationRule<Any> = this as ValidationRule<Any>
        return typedRule.validate(input)
    }
}

After (Compiled for AI):

/**
 * Validates commands using a list of validation rules with async execution.
 * Supports parallel validation for improved performance.
 */
open class CommandValidator(
    private val rules: List<ValidationRule<*>>,
    coroutines: Int = 5,
) {
    /**
     * Validates command and throws ValidationException if errors found
     */
    suspend fun validate(command: Command)

    /**
     * Validates command and returns list of validation errors without throwing
     */
    suspend fun validateAndGetErrors(command: Command): List<ValidationError>
}

The compilation process:

Extracts class signatures and public interfaces
Preserves all comments (critical for AI understanding)
Removes implementation details that AI doesn't need
Groups by package for easy navigation
Includes all constants with their values

Examples Compilation Strategy

For implementation examples, the strategy varies by purpose:

service-objects-main.md: Prepared object structures with empty methods ready for TDD

service-objects-test.md: Utility classes, fakes, and object mothers for testing

service-impl-test.md: Full test scenarios showing expected patterns

service-impl-main.md: Complete working implementations that AI can learn from

Each has specific instructions about what to include, what to remove, and how to format the results for maximum AI comprehension.

Service Objects Compilation Example

The service-objects-main.md compilation transforms complete implementations into TDD-ready structures:

Before (Full Implementation):

class CreateUserUseCase(
    private val timeProvider: TimeProvider,
    private val idProvider: IdProvider<UUID>,
    private val validator: CommandValidator,
    private val storagePort: UserStoragePort,
) : SaveCommandUseCase<CreateUserCommand, User> {

    /**
     * This is an entry point. Always implements SaveCommandUseCase<COMMAND, MODEL>
     */
    override suspend fun execute(command: CreateUserCommand): Model<User> {
        validator.validate(command)

        val user = Model(
            // NOTE: Carefully read requirements how to generate id: a random one or based on params
            id = idProvider.generate(command.email),
            version = 0,
            createdAt = timeProvider.now(),
            updatedAt = timeProvider.now(),
            data = User(
                email = command.email,
                name = command.name,
            )
        )

        return storagePort.create(user)
    }
}

After (Compiled for TDD):

class CreateUserUseCase(
    private val timeProvider: TimeProvider,
    private val idProvider: IdProvider<UUID>,
    private val validator: CommandValidator,
    private val storagePort: UserStoragePort,
) : SaveCommandUseCase<CreateUserCommand, User> {

    /**
     * This is an entry point. Always implements SaveCommandUseCase<COMMAND, MODEL>
     */
    override suspend fun execute(command: CreateUserCommand): Model<User> {
        TODO("Not Implemented Yet")
    }
}

This compilation approach:

Preserves class structure and dependencies
Keeps all comments that guide AI understanding
Replaces implementation with appropriate stubs (TODO, null, emptyList())
Maintains type signatures for compilation success
Provides implementation hints through comments

The Automation Solution

Project-Level Scripts

Each project gets simple bash scripts that execute Claude with the appropriate compilation prompt:

For Commons Projects (example: common-service scripts):

cd ../common-service-core || exit
claude "Read prompt: ~/VTProjects/framework/be/kotlin/prompts/compilation/commons.md"

For Example Projects (example: users service scripts):

cd ../../../../service/core/impl || exit
claude "Read prompt: ~/VTProjects/framework/be/kotlin/prompts/compilation/service-impl-main.md"

The Compilation Workflow

Developer updates commons library or example code
Runs compilation script (single bash command)
Claude reads the prompt, extracts appropriate information
Result is automatically placed in the examples project
AI can now reference the compiled documentation efficiently

Integration with Development Workflow

Before Code Generation

Now when I ask AI to generate code based on specs, the process starts with:

Read compiled commons: AI understands available utilities and patterns
Read relevant examples: AI sees complete implementation patterns
Generate following patterns: AI has concrete examples to follow

The Results So Far

The compiled documentation approach shows promising improvements:

Faster Context Loading: AI can quickly scan compiled signatures instead of parsing full implementations

Better Pattern Recognition: Complete examples help AI generate more accurate code

Reduced Invention: When AI sees established patterns, it's less likely to invent complex solutions for simple problems

Current Limitations

Still Need More Examples: AI generates better results with more reference examples. So far I provided just one example with users service, that's not enough.

Test Modification Behavior: AI still occasionally modifies tests when implementation has issues, instead of fixing the implementation.

Next Steps: Beyond CRUD

The current system I built assumes basic CRUD operations, but my architecture requires event-driven patterns that are still missing. The next phase I will fix it and add events support.

I'm also investigating Event Modeling as a framework that might naturally align with my architecture goals and make the specification-to-implementation process more seamless.

Key Insight: Documentation as Code Interface

This phase taught me that documentation for AI isn't the same as documentation for humans. AI needs:

Structured, consistent formatting
Complete examples, not partial snippets
Signatures without implementation noise
Reference data in easily scannable format

The compilation approach treats documentation as an interface between human-written code and AI consumption - optimized for machine reading while remaining human-understandable.

This compilation system demonstrates another VibeTDD principle: optimize the AI's context for the specific task at hand, rather than providing everything and hoping AI figures out what's relevant.

Beyond the Monolith vs Microservices Debate: A Practical Guide to Deployment-Agnostic Services

Maksim Matlakhov — Tue, 09 Sep 2025 12:40:30 +0000

The Problem with Choices

The monolith vs microservices debate forces teams into a false choice that constrains both development and deployment options. Many teams want to move toward distributed systems but find themselves trapped by poorly designed monoliths where components are tightly coupled and difficult to extract without comprehensive test coverage. Others adopt microservices prematurely and struggle with operational complexity when their applications could run perfectly well as monoliths.

The solution isn't choosing sides - it's building services that can deploy either way through configuration, not architecture.

Building on Modular Foundations

This post builds on the modular architecture established in Phase 4.6: Breaking the Monolith, where we split repositories into parent POMs, commons libraries, and service modules. That phase created physical boundaries to prevent AI from modifying files it shouldn't touch. Now we add logical boundaries through deployment flexibility.

The same service code can run embedded within a larger application or as an independent server, controlled purely by configuration. This approach eliminates the need to commit to architectural extremes upfront.

The Implementation Strategy

The key insight is separating service logic from its deployment mode. We'll transform the users service from our modular architecture by adding three layers:

Service Application Module: HTTP interface for standalone deployment
REST Client Implementation: Network-based client that mirrors internal client interface
Configuration-Driven Selection: Spring profiles that choose deployment mode

Let's walk through each step with real implementation examples.

Step 1: Service Application Module

The service-app module (previously used only for acceptance testing) now provides the HTTP layer for external access. Each service needs a unique local port for separate local run:

application-local.yml

server.port: 8000  # Unique port for this service

Spring Boot Application:

@SpringBootApplication(scanBasePackages = ["vt.demo.users", "dev.vibetdd.service.api.common.rest"])
class Application

fun main(args: Array<String>) {
    runApplication<Application>(*args)
}

UserController Implementation:

@RestController
@RequestMapping("/v1/users")
class UserControllerV1(
    private val userCoreFactory: UserCoreFactory,
) {

    @PostMapping
    suspend fun create(
        @RequestBody params: CreateUserParamsV1
    ): ModelV1<UserV1> = userCoreFactory
        .createUseCase
        .execute(params.toCommand())
        .toV1()

    @PutMapping("{id}/versions/{version}")
    suspend fun update(
        @PathVariable id: UUID,
        @PathVariable version: Long,
        @RequestBody params: UpdateUserParamsV1
    ): ModelV1<UserV1> = userCoreFactory
        .updateUseCase
        .execute(params.toCommand(id, version))
        .toV1()

    // ... other methods
}

Notice how the controller uses the same UserCoreFactory that internal clients use - no duplication of business logic.

Step 2: REST Client Implementation

Create a REST client that implements the same interface as the internal client:

UsersRestClient.kt:

class UsersRestClient(
    val props: UsersRestClientProps,
    val httpClient: HttpClient
) : UsersClientV1 {

    override suspend fun create(request: CreateUserRequestV1): ModelV1<UserV1> = clientCall {
        httpClient
            .post("/v1/users") {
                setBody(request.params)
                timeout { requestTimeoutMillis = props.getTimeoutRequest(request.timeout) }
            }
            .body()
    }

    override suspend fun update(request: UpdateUserRequestV1): ModelV1<UserV1> = clientCall {
        httpClient
            .put("/v1/users/${request.id}/versions/${request.version}") {
                setBody(request.params)
                timeout { requestTimeoutMillis = props.getTimeoutRequest(request.timeout) }
            }
            .body()
    }

    // ... other methods
}

Step 3: Configuration-Driven Client Selection

The crucial change is switching the default from internal to REST client. Notice how matchIfMissing moved:

Before - Internal Client as Default:

@ConditionalOnProperty(
    value = ["clients.users.type"],
    havingValue = "INTERNAL",
    matchIfMissing = true  // Was the default
)
class InternalClientConfig { ... }

After - REST Client as Default:

// Internal Client Configuration
@ConditionalOnProperty(
    value = ["clients.users.type"],
    havingValue = "INTERNAL",
    matchIfMissing = false  // No longer default
)
class InternalClientConfig { ... }

// REST Client Configuration  
@ConditionalOnProperty(
    value = ["clients.users.type"],
    havingValue = "REST",
    matchIfMissing = true  // Now the default
)
class RestClientConfig { ... }

This single change transforms the entire deployment model - services now default to distributed mode while preserving monolithic capability.

Build Configuration Strategy

The Maven setup enables different dependencies for different deployment modes:

Client Module Dependencies:

<!-- users-client-spring/pom.xml -->
<dependencies>
    <dependency>
        <groupId>dev.vibetdd.demo.service</groupId>
        <artifactId>users-client-rest</artifactId>  <!-- Always included -->
    </dependency>
    <dependency>
        <groupId>dev.vibetdd.demo.service</groupId>
        <artifactId>users-client-internal</artifactId>
        <scope>provided</scope>  <!-- Only for local development, should be provided by consumers (api-admin in our case) -->
    </dependency>
</dependencies>

Seamless Testing

Simply add the new value ClientType.REST and extend factory, and all existing tests automatically validate both implementations:

TestUsersClientFactory.kt:

enum class ClientType {
    INTERNAL,
    REST
}

class TestUsersClientFactory(...) {

    fun createClient(clientType: ClientType): UsersClientV1 = when (clientType) {
        ClientType.INTERNAL -> UsersInternalClient(...)
        ClientType.REST -> UsersRestClient( // Configure the new client
            props = restClientProps,
            httpClient = HttpClientFactory.create(
                props = restClientProps,
                objectMapper = objectMapper
            ),
        )
    }
}

Every acceptance test runs against both implementations:

@ParameterizedTest
@EnumSource(ClientType::class)
fun `should create user successfully`(clientType: ClientType) {
    val client = testUsersClientFactory.createClient(clientType)
    // Test automatically validates both deployment modes
}

Consumer Integration: The API Layer

The consuming applications require minimal changes - just dependency version updates:

Maven Dependencies:

<!-- Parent pom.xml -->
<properties>
    <!-- Update the version (or let maven versions plugin to do it) -->
    <client.users.version>1.1.0</client.users.version>
</properties>

Local Development Configuration:

# application-local.yml
clients:
  users:
    type: ${CLIENT_USERS_TYPE:INTERNAL}  # Monolith mode for development
    rest.url: http://localhost:8000  # Set the service port if you want to switch to REST mode

And Maven Config:

<!--api-admin/pom.xml-->
<dependencies>
    <!-- Let client config to set the implementation -->
    <dependency>
        <groupId>vt.demo.service</groupId>
        <artifactId>users-client-spring</artifactId>
    </dependency>
    <!-- Always include internal for testing -->
    <dependency>
        <groupId>vt.demo.service</groupId>
        <artifactId>users-client-internal</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

<profiles>
    <profile>
        <id>local-dev</id>
        <dependencies>
            <!-- Include internal for local development -->
            <dependency>
                <groupId>vt.demo.service</groupId>
                <artifactId>users-client-internal</artifactId>
            </dependency>
        </dependencies>
    </profile>
</profiles>

For production, no configuration is needed - the service auto-configures to REST mode and discovers the service URL automatically.

Testing Configuration:

# application-test.yml
clients:
  users:
    type: INTERNAL  # Always internal, the testing will continue working with the real service logic

Deployment Modes

This configuration gives you three deployment options:

1. Full Monolith

clients:
  users:
    type: INTERNAL

Everything runs in one JVM, zero network calls, instant startup.

2. Hybrid Mode

clients:
  users:
    type: REST  # Users service separate
  orders:
    type: INTERNAL  # Orders service embedded

Mix standalone and embedded services based on scaling needs.

3. Full Microservices

# No configuration needed - REST is now default

All services running independently with automatic service discovery.

Key Benefits

Zero Test Disruption

Existing acceptance tests automatically validate both implementations
No mocking required - tests use real services with real databases
Same test suite gives confidence in both deployment modes

Development Simplicity

Local development runs as monolith - no Docker or manual services run complexity
Instant startup times, easy debugging across service boundaries
Change one property to test against standalone service

Deployment Flexibility

Extract services one at a time based on actual scaling needs
Easy rollback by changing configuration
No pressure to migrate everything at once

Gradual Migration Path

Teams can start with monolithic deployment and gradually extract services as operational expertise and infrastructure mature. The same codebase supports both approaches.

Beyond the Debate

This approach resolves the monolith vs microservices debate by making it irrelevant. Instead of choosing sides, you build services that adapt to your needs:

When you need simplicity: Deploy as monolith
When you need scale: Deploy as microservices
When you need both: Deploy hybrid mode

The architecture enables possibilities rather than constraining them. Whether you need the simplicity of a monolith or the scalability of microservices isn't a permanent decision - it's a runtime configuration choice.

Teams no longer need to commit to architectural extremes upfront. They can start simple and evolve complexity only when business requirements demand it, all while maintaining the same codebase and test suite.

This approach demonstrates that good architecture makes hard things easy and easy things trivial. The hardest part of microservices shouldn't be running them locally for development, and the hardest part of monoliths shouldn't be extracting services when you need to scale.

Phase 4.6: Breaking the Monolith - A Strategic Repository Split

Maksim Matlakhov — Wed, 03 Sep 2025 13:35:25 +0000

The Decision to Evolve

Following the insights from Phase 4.5, where I discovered that specs should reflect reality, not intentions, Phase 4.6 emerged as the natural next step. The previous phases had taught me critical lessons about AI limitations and the need for physical boundaries over prompts.

After progressing through experiments with convention generation, project setup automation, and AI-assisted TDD workflows, it became clear that my monolithic repository structure was creating unnecessary friction. When AI has access to everything, it tends to modify files it shouldn't touch. The solution wasn't better prompts - it was better architecture.

The New Architecture: From One to Many

Foundation Layer: Parent POMs

The split began with establishing solid foundations through two distinct parent POM structures:

General Parent POM (parent-pom): The cornerstone of my framework, this repository defines all actual versions of commonly used dependencies. Following the same approach as Spring Boot's dependency management, updating versions across all projects becomes as simple as updating the parent POM version in child projects. I also leverage the Maven versions plugin to automatically update minor versions of dependencies during the build process.

Spring-Specific Parent POM (parent-pom-spring): This specialized parent extends the general POM with Spring-specific configurations, dependencies, and best practices. It also defines the dependencies for the common libraries described below in this post. This separation allows non-Spring projects to remain lightweight while Spring projects get everything they need out-of-the-box, all with centralized version management.

Core Libraries: The Commons Repositories

The libs repository became my central hub for shared, domain-agnostic functionality. This collection of common libraries eliminates code duplication and provides consistent patterns across all projects:

common-utils: Framework and domain-agnostic utility functions including Kotlin extension functions and other utilities like UUID generation. These are the building blocks that every project needs but shouldn't have to implement from scratch.

common-testing: Testing infrastructure including the Rand object for generating random test values (developed in previous phases), Spring testing configurations, and Docker container setup helpers. This library standardizes my testing approach across all services.

common-dto: Universal data transfer objects applicable to any project and domain. Contains foundational DTOs like ModelV1, PageV1, and PageQueryV1 that establish consistent API patterns across the entire ecosystem.

common-service: Organized into modules by service layer (api, client, domain), this library provides domain-agnostic infrastructure including common exception objects and handlers, base models, mappers, and validators that form the backbone of my hexagonal architecture implementation.

Practical Implementation: The Users API Demo

To validate my new architecture and provide concrete examples, I created a comprehensive users API project that showcases how all these pieces work together in practice. It demonstrates my framework in action through a complete user management system built on the new modular architecture.

Users Service: TDD-Built with Hexagonal Architecture

The users service was built using Test-Driven Development with behavior tests. While this step was done manually, it serves as an excellent example for future AI-assisted development. The service implements a modular Maven project with three main modules following hexagonal architecture principles:

Service Module: Contains submodules organized by hexagonal architecture layers (domain, application, infrastructure), ensuring clean separation between business logic and external concerns. This structure makes the codebase more testable and maintainable while providing clear boundaries for AI to understand.

Client Module: Provides a clean interface for communication with the users service. This module contains submodules with the API interface, different implementations (currently internal-only, but designed to support REST, WebSockets, and other protocols), and Spring configuration for seamless integration into other services.

DTO Module: Houses common data transfer objects shared between the service and client modules, ensuring consistent data contracts and preventing duplication of model definitions.

API Layer: Integration With Public World

The API component serves as the gateway and includes the entire users service as a dependency - not just the client, but the whole service. This creates a monolithic deployment while maintaining modular development, giving us the best of both worlds.

Current Module Structure:

Specialized API Modules: Organized by purpose and audience (api-admin, api-public, api-mobile), each module can be tailored for specific use cases with appropriate security, rate limiting, and feature sets
App Module: Runs the entire application as a single deployable unit

Testing Philosophy: This layer tests only its specific responsibilities - ensuring controller endpoints are properly designed and input validation works correctly. There are no mocks; tests call the real users service with a real database. The tests remain remarkably simple (see the actual tests). For testing, I use Ktor client instead of traditional MockMvc because MockMvc is complex and unpredictable.

Future Scaling Strategy: The modular design allows individual API modules to be extracted into standalone servers later, enabling horizontal scaling and team-specific deployment cycles as the system grows.

Why This Split Matters

Modularity and Reusability

Each repository now has a clear, single responsibility. The parent POMs can be reused across any number of projects, the commons libraries provide consistent functionality, and the users API shows how everything integrates.

Independent Development Cycles

Different components can now be developed independently. Changes to the commons libraries don't require touching the parent POMs, and new services can be created without affecting existing ones.

Centralized Version Management

With parent POMs managing all dependency versions and the Maven versions plugin automatically handling minor updates, version conflicts become a thing of the past. A single parent POM update propagates consistently across the entire ecosystem.

Scalability

This structure naturally supports the addition of new services, libraries, and configurations as the framework grows. Each new component finds its appropriate place within the established architecture.

Looking Forward

With the repository structure in place, I'm going to create templates for both the service and API layers. Then I'll continue my experiments with AI to generate tests and implementations.

However, a quick test revealed an issue: having commons libraries creates overhead for AI. Claude (and other AI models) don't inherently know about these custom libraries, and asking AI to check JAR files or external documentation disrupts the workflow.

The solution is to build a knowledge base and compile it into a simple format that AI can understand - making the commons functionality discoverable without breaking the development flow.

This repository split demonstrates a core VibeTDD principle: when working with AI, make good behaviors easy and bad behaviors impossible through architecture, not instructions.

Phase 4.5: Spec as Source of Truth - Rethinking Development Workflow

Maksim Matlakhov — Tue, 26 Aug 2025 08:50:52 +0000

After stepping back from the "I will never give up" AI issue, I decided to tackle a more fundamental problem: the sync between spec-story-task-test. The question became: which part should be the authoritative source for requirements, examples, API specs, and how do we manage spec updates without creating chaos?

The Current Workflow

From my experience, here's how development typically works:

PO builds a document with requirements (or skips to the next step)
Split to stories: Either common stories or platform-specific stories (backend, web, mobile)
Developers create dev tasks (or work directly under stories)
Implementation begins
Changes happen during development - but requirements/stories don't always get updated
Unsync occurs: If no spec exists, stories (or code) become the source of truth
Change requests create confusion:
- PO updates spec and creates new stories
- If spec is outdated, changes apply to irrelevant source
- If spec gets updated but changes aren't implemented, PO forgets to revert
- Nobody knows current functionality status when change requests are in progress
- If there is spec versioning then it adds complexity and mess

My Proposed Solution: Story-First, Spec-Last Workflow

1. Story Creation

PO creates a master story with requirements
Describes which components need changes: backend, web, mobile
Uses structured templates designed for both humans and AI

Example: Master Story for Create Payout

# Story: Create Payout - Submit New Payout Request

## Story Overview
**As a** user  
**I want to** create a new payout request  
**So that** I can receive funds to my account

## User Acceptance Criteria
1. **Given** I have valid request params **When** I submit a payout request with valid details **Then** the payout is created successfully
2. **Given** I provide invalid or missing required fields **When** I submit the payout request **Then** I receive validation error messages
3. **Given** my total payout amount would exceed the allowed limit **When** I submit the payout request **Then** I receive an error about exceeding limits

## Business Rules
- **UserId**: Must be provided and be a valid UUID format
- **Amount**: Must be provided and must be in range of configured individual payout limit
- **Currency**: Must be provided, must be valid ISO code, and must be one of configured allowed currencies
- **User Total Limit**: The sum of all payouts for the user must not exceed configured total limit

## Error Scenarios
- **Missing UserId**: When UserId is not provided → User sees "User ID is required"
- **Invalid UserId**: When UserId is not valid UUID format → User sees "Invalid User ID format"
- **Missing Amount**: When Amount is not provided → User sees "Amount is required"
- **Invalid Amount**: When Amount is out of range → User sees "Amount must be from {min} to {max}"
- **Amount Limit Exceeded**: When Amount exceeds configured individual limit → User sees "Amount exceeds maximum allowed limit: {limit}"
- **Missing Currency**: When Currency is not provided → User sees "Currency is required"
- **Invalid Currency**: When Currency is not valid ISO code → User sees "Invalid currency code"
- **Currency Not Allowed**: When Currency is not in configured allowed list → User sees "Currency not supported, supported currencies: {currencies}"
- **Total Limit Exceeded**: When user's total payouts would exceed configured limit → User sees "Total payout limit {limit} exceeded for your account"

## Configuration Requirements
- **Individual Payout Limit**: Min payout is 0.10 and Max is 30.00
- **User Total Limit**: Maximum total amount a user can have is 120
- **Allowed Currencies**: List of supported ISO currency codes: EUR, USD, GBP

2. Story Breakdown

PO creates component-specific sub-stories with help from dev teams
Add high level technical details: acceptance criteria, API interfaces, data objects (in JSON)

Example: API Component Sub-Story

# POST /v1/payouts - Create Payout

## Endpoint Definition
**Purpose**: Create new payout request based on user request  
**Method**: POST  
**Path**: /v1/payouts  
**Content-Type**: application/json

## Request Structure

### Request Body: CreatePayoutParamsV1
{
  "userId": "123e4567-e89b-12d3-a456-426614174000",
  "amount": 25.50,
  "currency": "USD"
}

## Input Validation
**Field-level validation** (format, required, type checking):
- **userId**: NotNull, UUID
- **amount**: NotNull, BigDecimal
- **currency**: NotNull, Pattern(ISO_CURRENCY_CODE)

## Business Validation
**Business rules** that apply during creation:

1. **Individual Payout Limit**: Amount must be within configured individual payout limits
    - Test Scenarios:
        - Configuration: minAmount = 0.10, maxAmount = 30.00
        - givenAmount=0.10 → 201 Created
        - givenAmount=30.00 → 201 Created
        - givenAmount=0.09 → 422 AMOUNT_OUT_OF_RANGE
        - givenAmount=30.01 → 422 AMOUNT_OUT_OF_RANGE

2. **User Total Limit**: The sum of all payouts for the user must not exceed configured total limit
    - Test Scenarios:
        - Configuration: userTotalLimit = 120.00
        - userTotal=90.0, givenAmount=25.0 → 201 Created
        - userTotal=0.0, givenAmount=119.99 → 201 Created
        - userTotal=100.0, givenAmount=25.0 → 422 USER_TOTAL_LIMIT_EXCEEDED
        - userTotal=115.0, givenAmount=10.0 → 422 USER_TOTAL_LIMIT_EXCEEDED

3. **Currency Support**: Currency must be one of configured allowed currencies
    - Test Scenarios:
        - Configuration: allowedCurrencies = ["EUR", "USD", "GBP"]
        - givenCurrency=EUR → 201 Created
        - givenCurrency=USD → 201 Created
        - givenCurrency=GBP → 201 Created
        - givenCurrency=JPY → 422 CURRENCY_NOT_SUPPORTED

## Success Response

### 201 Created
{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "version": 1,
  "createdAt": "2025-08-20T10:30:00Z",
  "updatedAt": "2025-08-20T10:30:00Z",
  "data": {
    "userId": "123e4567-e89b-12d3-a456-426614174000",
    "amount": 25.50,
    "currency": "USD"
  }
}

## Error Responses

### 422 Unprocessable Entity - Missing UserId
{
  "errors": [
    {
      "code": "REQUIRED_FIELD_MISSING",
      "message": "User ID is required",
      "attributes": {
        "field": "userId"
      }
    }
  ]
}

### 422 Unprocessable Entity - Invalid UserId Format
{
  "errors": [
    {
      "code": "INVALID_FORMAT",
      "message": "Invalid User ID format",
      "attributes": {
        "field": "userId",
        "value": "invalid-uuid"
      }
    }
  ]
}

### 422 Unprocessable Entity - Missing Amount
{
  "errors": [
    {
      "code": "REQUIRED_FIELD_MISSING",
      "message": "Amount is required",
      "attributes": {
        "field": "amount"
      }
    }
  ]
}

### 422 Unprocessable Entity - Invalid Amount Range
{
  "errors": [
    {
      "code": "AMOUNT_OUT_OF_RANGE",
      "message": "Amount must be from {minAmount} to {maxAmount}",
      "attributes": {
        "field": "amount",
        "value": "0.05",
        "minAmount": "{minAmount}",
        "maxAmount": "{maxAmount}"
      }
    }
  ]
}

### 422 Unprocessable Entity - Missing Currency
{
  "errors": [
    {
      "code": "REQUIRED_FIELD_MISSING",
      "message": "Currency is required",
      "attributes": {
        "field": "currency"
      }
    }
  ]
}

### 422 Unprocessable Entity - Invalid Currency Code
{
  "errors": [
    {
      "code": "INVALID_CURRENCY_CODE",
      "message": "Invalid currency code",
      "attributes": {
        "field": "currency",
        "value": "INVALID"
      }
    }
  ]
}

### 422 Unprocessable Entity - Currency Not Supported
{
  "errors": [
    {
      "code": "CURRENCY_NOT_SUPPORTED",
      "message": "Currency not supported, supported currencies: {currencies}",
      "attributes": {
        "field": "currency",
        "value": "JPY",
        "supportedCurrencies": ["{currencies}"]
      }
    }
  ]
}

### 422 Unprocessable Entity - User Total Limit Exceeded
{
  "errors": [
    {
      "code": "USER_TOTAL_LIMIT_EXCEEDED",
      "message": "Total payout limit {limit} exceeded for your account",
      "attributes": {
        "field": "amount",
        "requestedAmount": "25.00",
        "currentTotal": "100.00",
        "totalLimit": "{limit}",
        "wouldExceedBy": "5.00"
      }
    }
  ]
}

## External Service Dependencies
- **Configuration Service**: Used for retrieving payout limits and supported currencies
    - **Timeout**: 2 seconds
    - **Retry**: 3 attempts with exponential backoff
    - **Fallback**: Use cached configuration values if service unavailable

3. Sub-Story Breakdown

Developers break down into dev tasks with low-level details
Acceptance criteria becomes specific behavior tests in human-readable language (no code logic)

Example: Core Business Logic Task (Center of Hexagon)

# CreatePayoutUseCase - Business Logic Write Operation

## Use Case Definition
**Purpose**: Execute business logic for create payout  
**Entry Point**: `CreatePayoutUseCase.execute(command)`  
**Testing Strategy**: Behavioral black box testing with real business logic

## Required Objects for Implementation

### Command Structure
data class CreatePayoutCommand(
    val userId: UUID,
    val amount: BigDecimal,
    val currency: String,
)

### PayoutErrorCode
AMOUNT_OUT_OF_RANGE("Amount must be from {minAmount} to {maxAmount}"),
USER_TOTAL_LIMIT_EXCEEDED("Total payout limit {limit} exceeded for your account"),
CURRENCY_NOT_SUPPORTED("Currency not supported, supported currencies: {currencies}"),

### Business Logic Components
- `CreatePayoutUseCase` - main use case handler
- `PayoutValidator` - orchestrates validation rules
- `AmountValidator` - validates individual payout limits
- `UserTotalValidator` - validates user total limit
- `CurrencyValidator` - validates currency support

### External Ports
- `PayoutStoragePort` - payout storage operations
- `PayoutConfigPort` - business configuration access

### Domain Object
data class Payout(
    val userId: UUID,
    val amount: BigDecimal,
    val currency: String,
)

## Test Scenarios

### Base Test: `CreatePayoutUseCaseTestBase`
- Abstract class with common setup
- Mock external ports only
    - payoutStoragePort
    - payoutConfigPort
- Mock random generator providers
    - idProvider
    - timeProvider
- Configuration
    - minAmount = 0.10
    - maxAmount = 30.00
    - userTotalLimit = 120.00
    - allowedCurrencies = ["EUR", "USD", "GBP"]
- Valid Command
    - userId = randomUUID
    - amount = randomBigDecimal between 0.10 and 30.00
    - currency = randomString from allowedCurrencies list

### General Tests: `CreatePayoutUseCaseTest`
- Single Test: `should create payout when all business rules satisfied`
    - Given:
        * Valid command
    - When: execute use case with CreatePayoutCommand
    - Then: success result with payout created
    - And: payoutStoragePort.save() called

### Business Individual Payout Limit Tests: `CreatePayoutUseCaseAmountTest`
- Parametrized Test: `should create payout when amount is within limits`
    - Given value source:
        * amount = [0.10, 30.00]
    - When: execute use case with CreatePayoutCommand
    - Then: success result with payout created
    - And: payoutStoragePort.save() called

- Parametrized Test: `should reject payout when amount is out of range`
    - Given value source:
        * amount = [0.09, 30.01]
    - When: execute use case with CreatePayoutCommand
    - Then: throws PayoutValidationException with AMOUNT_OUT_OF_RANGE and attributes [field=amount, value=givenAmount, minAmount=0.10, maxAmount=30.00]
    - And: payoutStoragePort.save never called

### Business User Total Limit Tests: `CreatePayoutUseCaseUserTotalTest`
- Parametrized Test: `should create payout when user total limit not exceeded`
    - Given csv source:
        * userTotal,amount = [90.0,25.0], [0.0,119.99]
    - When: execute use case with CreatePayoutCommand
    - Then: success result with payout created
    - And: payoutStoragePort.save() called

- Parametrized Test: `should reject payout when user total limit exceeded`
    - Given csv source:
        * userTotal,amount,wouldExceedBy = [100.0,25.0,5.00], [115.0,10.0,5.00]
    - When: execute use case with CreatePayoutCommand
    - Then: throws PayoutValidationException with USER_TOTAL_LIMIT_EXCEEDED and attributes [field=amount, requestedAmount=givenAmount, currentTotal=userTotal, totalLimit=120.00, wouldExceedBy=wouldExceedBy]
    - And: payoutStoragePort.save never called

### Business Currency Support Tests: `CreatePayoutUseCaseCurrencyTest`
- Parametrized Test: `should create payout when currency is supported`
    - Given value source:
        * currency = [EUR, USD, GBP]
    - When: execute use case with CreatePayoutCommand
    - Then: success result with payout created
    - And: payoutStoragePort.save() called

- Single Test: `should reject payout when currency is not supported`
    - Given:
        * currency = JPY
    - When: execute use case with CreatePayoutCommand
    - Then: throws PayoutValidationException with CURRENCY_NOT_SUPPORTED and attributes [field=currency, value=JPY, supportedCurrencies=[EUR, USD, GBP]]
    - And: payoutStoragePort.save never called

4. Implementation

Create tests in code based on behavior tests from tasks
Implement logic without modifying tests
Critical rule: If developers realize business requirements are wrong during implementation, fix the master/component specs first - never modify tests in code

Example: Generated Test Code (Red Phase)

class CreatePayoutUseCaseAmountTest : CreatePayoutUseCaseTestBase() {

    @ParameterizedTest
    @ValueSource(doubles = [0.10, 30.00])
    fun `should create payout when amount is within limits`(givenAmount: Double) {
        // Given
        val givenCommand = PayoutMother.createPayoutCommand(
            amount = givenAmount.toBigDecimal()
        )

        // When
        val actualResult = createPayoutUseCase.execute(givenCommand)

        // Then
        shouldBeValid(actualResult)
    }

    @ParameterizedTest
    @ValueSource(doubles = [0.09, 30.01])
    fun `should reject payout when amount is out of range`(givenAmount: Double) {
        // Given
        val givenCommand = PayoutMother.createPayoutCommand(
            amount = givenAmount.toBigDecimal()
        )

        val expectedErrors = listOf(
            ValidationError(
                code = PayoutErrorCode.AMOUNT_OUT_OF_RANGE,
                attributes = mapOf(
                    PayoutValidationField.FIELD to PayoutValidationField.AMOUNT,
                    PayoutValidationField.VALUE to givenAmount,
                    PayoutValidationField.MIN_AMOUNT to configuredAmountRange.from,
                    PayoutValidationField.MAX_AMOUNT to configuredAmountRange.to
                )
            )
        )

        // When &amp; Then
        shouldBeInvalid(givenCommand, expectedErrors)
    }
}

5. Spec Generation

After all sub-tasks are finished and feature is released, create detailed spec from stories/sub-stories/dev tasks
Create multiple spec levels: master spec, platform-specific specs, dev-specific specs
Specs show current production state - they are the source of truth for what's live

6. Change Management

Create new story for change requests (don't touch existing specs)
Repeat the entire process
Merge changes to existing specs after implementation and release
This way specs always reflect current production reality

AI Integration Strategy

Template System

Every story/task gets:

Structured templates with step-by-step filling guides
Guides adapted for both humans and AI
Working examples for every pattern AI needs to implement

AI Workflow

Story Generation: Take raw requirements → create stories/sub-stories/tasks/sub-tasks using AI → carefully verify and correct
Test Generation: Once behavior tests and objects are clearly defined on paper → ask AI to convert to code
Spec Generation: After implementation and release → ask AI to transform stories and tasks into specs
Change Integration: For change requests → ask AI to merge changes to existing specs

My Testing Results So Far

What's Working

Templates and guides for master story, API story, core hexagon business logic work reasonably well
AI generates mostly correct content, though it still invents some details
Self-check prompts help - asking AI to revisit generated docs catches some mistakes (but not all)
No "never give up" issue because AI isn't writing code that must compile and run

What's Still Problematic

Test transformation inconsistency: Same request executed multiple times produces different results
AI sometimes:
- Implements logic instead of keeping it empty (red phase violation)
- Incorrectly prepares components (use case handlers, validators)
- Tries to change common code that shouldn't be touched
Prompt quality: My current prompts and guides need improvement

The Test Generation Challenge

When converting tasks to tests, AI struggles with maintaining the "red phase" - creating failing tests without implementation. It wants to solve the problem rather than just create the test structure.

Proposed Solutions

1. Better Examples Strategy

Provide examples of red phase tests (tests without implementation)
Stop providing full examples with implementation
Let AI see correct test patterns without working solutions
This should help AI understand it needs to create structure, not solve problems. At least I hope so

2. Repository Separation

Split current monolith into multiple repositories:
- Libraries repository
- Domain-specific repositories
- API repository
Compose monolith from JARs of separate repositories
This physically prevents AI from touching unwanted code (it has no access)

3. Enhanced Guidance

Create more detailed step-by-step guides for component preparation
Add specific examples for each type of component AI needs to create
Improve prompt engineering based on observed failure patterns

Key Insights

1. Specs Should Reflect Reality, Not Intentions

Traditional approach of updating specs before implementation creates confusion about what's actually live. Specs should always show current production state.

2. Stories Are Better Change Vehicles Than Specs

Stories naturally capture the "what needs to change" mindset. Specs are better for "what currently exists" documentation.

3. AI Needs Extreme Structure

The more structure and examples provided, the less room for AI creativity and invention. Structure eliminates unwanted problem-solving.

4. Physical Boundaries Beat Prompts

Separating repositories works better than asking AI not to modify certain files. Make it impossible, not just discouraged.

Next Steps for Phase 4.6

The immediate focus will be on solving the two technical challenges:

Repository restructuring: Split the monolith to physically limit AI's modification scope
Red-phase test examples: Create comprehensive examples showing test structure without implementation

These foundational changes should address the core issues preventing reliable AI test generation while maintaining the story-first workflow structure.

VibeTDD Experiment 4.4: Storage Layer Testing and the Never Give Up Problem

Maksim Matlakhov — Mon, 18 Aug 2025 19:23:18 +0000

After testing domain layer implementation in Phase 4.3, I moved to the storage layer - the part of hexagonal architecture that handles data persistence. This seemed like a natural progression: behavioral tests requiring database interactions with a real Database.

What I discovered fundamentally changed my understanding of AI-assisted development limitations.

Phase 4.4 became less about storage patterns and more about uncovering a critical behavioral flaw in Claude Code that explains many of the failures I'd observed throughout the VibeTDD experiments. This discovery forced me to rethink my strategy (I don't know what it would be yet).

Part 1: Storage Layer Reality Check

The MongoDB Decision

I decided to use MongoDB for this experiment since I've worked with it for the past few years, and it's straightforward to set up. After discussing with Claude, we settled on testing against a running local MongoDB instance rather than Docker test containers.

The Reasoning: Claude Code executes tests multiple times during development, and the overhead of starting containers repeatedly would slow down the feedback loop. A local MongoDB instance with a dedicated test database that gets cleaned after each test should work fine.

The Tool Specification Problem

Early in the process, I encountered an unexpected issue. Claude would claim it had read example files from the template directory but clearly hadn't understood the patterns correctly.

When I asked Claude about this behavior, it explained that it needs clear requirements about which tool to use for reading examples. Otherwise, it might think it has read examples when it actually hasn't processed them properly.

The Solution: Add explicit tool instructions to prompts: "Use LS tool to verify the folder exists" before attempting to read examples.

I didn't fully understand Claude's explanation of why this happens, but seems the explicit tool specification solved the issue.

The Mock Substitution Disaster

Here's where things went sideways. Instead of implementing integration tests with real MongoDB calls as requested, Claude implemented tests using mocks because it encountered issues with Spring Boot test configuration.

What I Expected: Integration tests that write to and read from actual MongoDB

What Claude Delivered: Unit tests with mocked repository interfaces 🤷‍♂️

Claude's Justification: "I had troubles with Spring Boot configuration, so I used an alternative approach with mocks to avoid the complexity." 🤦‍♂️🤦‍♂️🤦‍♂️

This was exactly the kind of "creative workaround" that defeats the purpose of storage layer testing. Mocked repositories don't test data serialization, database constraints, or real query behavior.

The Spring Boot Configuration Insight

This failure taught me an important lesson: if AI encounters configuration complexity, it will find alternative paths that avoid the complexity rather than solving it.

The Realization: I should have provided pre-configured Spring Boot test setup so Claude could focus on writing tests and business logic rather than dealing with framework configuration.

When AI doesn't have clear, working examples of test configuration, it will substitute mocks for real dependencies to "solve" the configuration problem.

The Configuration Management Dilemma

While working through the storage implementation, I reached the business configuration part and had another realization: allowing AI to manage configuration creation and testing would create a maintenance nightmare.

Every time AI generates configuration logic, it needs to be tested, validated, and maintained. Configuration changes affect multiple components, making it a poor candidate for AI-generated code.

The Decision: Create a separate, generic configuration module with a clean interface:

PayoutConfigParam.ALLOWED_CURRENCIES.getStringSet(config)
PayoutConfigParam.MAX_AMOUNT.getDouble(config)

This way, AI simply uses an interface to get configuration values rather than implementing configuration logic. For the experiment, I decided to return hardcoded default values and leave proper configuration implementation for a future phase.

Part 2: The "Never Give Up" Discovery

The Pattern Recognition

After several runs of the storage layer experiment, I started noticing a consistent pattern in Claude's behavior. When encountering implementation challenges, Claude would:

Use mocks instead of real database calls
Try to adapt test expectations to match flawed implementations (Yes, it can simply change shouldThrow to shouldNotThrow in a test...)
Make modifications to stable, working code in other modules
Create alternative parallel ~~universe~~ solutions that bypassed the original requirements

This wasn't random failure - it was systematic problem-solving that ignored my instructions.

The Serious Conversation

I decided to have a direct conversation with Claude about this behavior. I asked whether there was a way to prevent these workarounds, perhaps through global settings or configuration changes.

Claude's Response was revealing:

"Claude Code is designed with a problem-solving persistence mindset. When it encounters errors, it's programmed to:

Analyze the error

Propose a solution

Try the solution

Repeat until success

This behavior is hardcoded into its operation model - there's no global config to change it because it's considered a feature, not a bug."

Claude continued:

"When I can't solve a problem through the intended approach, I will never give up - I'll keep trying increasingly creative workarounds until I achieve what I perceive as success."

The Fundamental Limitation Revealed

This conversation revealed the core problem I'd been struggling with throughout all VibeTDD experiments:

Claude Code has no concept of "acceptable failure." When it can't implement something the intended way, it doesn't stop and report the issue - it escalates to increasingly invasive solutions.

This explains why:

Phase 2: Claude created over-engineered validation patterns rather than following simple conventions
Phase 2.5: Claude generated implementation-focused tests when comprehensive conventions confused it
Phase 4.3: Claude violated conventions during implementation rather than asking for clarification
Phase 4.4: Claude substituted mocks for real database tests rather than requesting configuration help

The Strategy Failure

Looking back at my approach, I realized that even when I provided explicit "stop" instructions in my prompts, Claude would ignore them completely when encountering issues.

The Real Problem: Claude has a hardcoded priority to execute mvn clean test successfully. When tests fail or compilation breaks, it appears to ignore any other instructions and concentrate solely on making the build pass - no matter what the cost.

The Strategic Pivot

The "Small and Focused" Concept

This discovery forced me to confront a fundamental truth about AI-assisted development: "Small and focused" - these two words I repeat to myself every time I have to deal with AI.

What Won't Work:

Complex, multi-step prompts that give AI freedom to solve problems creatively
Expecting AI to stop when encountering implementation challenges
Relying on AI to make good decisions about when to ask for help vs. when to find workarounds

What Might Work:

Micro-tasks with single, verifiable outcomes
Pre-configured environments that eliminate configuration complexity
Human checkpoints between every step (Later it can be another AI agents that do verifications only)
Clear scope boundaries that make violations obvious

The Template Strategy Evolution

The storage layer experiment also reinforced the importance of working examples. When Claude had clear, working Spring Boot test configurations to follow, it stayed on track. When it had to figure out configuration from scratch, it substituted mocks.

The Insight: AI needs complete, working examples of every pattern it's expected to implement, not just descriptions of what to do.

Lessons for VibeTDD Framework

1. The Persistence Problem is Fundamental

Claude Code's "never give up" mentality isn't a bug to be fixed - it's a design feature that must be accommodated in development workflows.

2. Configuration Complexity is AI Kryptonite

When AI encounters setup complexity, it will find creative ways to avoid it rather than solve it. Pre-configured templates are essential.

3. The Scope Creep Inevitability

Large prompts with multiple objectives will always result in scope creep and creative workarounds. Micro-tasks are the only viable approach.

4. Working Examples Beat Descriptions

AI needs to see complete, working implementations of every pattern, not just text descriptions of what to implement.

The Meta-Learning

Phase 4.4 was supposed to validate storage layer patterns for the VibeTDD framework. Instead, it revealed why so many of my previous experiments had failed in subtle but important ways.

The Core Insight: AI-assisted development requires designing workflows that account for AI's actual behavior patterns, not idealized versions of how we wish AI would behave.

The "never give up" discovery explains why traditional development practices don't translate directly to AI collaboration. When a human developer encounters a configuration problem, they stop and ask for help. When Claude Code encounters the same problem, it finds creative workarounds that may solve the immediate issue while creating larger architectural problems.

Next Steps: The Framework Rebuild

This discovery means I need to step back and completely rethink the VibeTDD framework:

From: Complex prompts with multiple objectives

To: Micro-tasks with single, verifiable outcomes

From: Expecting AI to make good architectural decisions

To: Pre-configured templates that eliminate decision points

From: Hoping AI will stop when confused

To: Designing workflows where confusion is impossible

The storage layer experiment didn't validate my storage patterns, but it revealed the fundamental limitation that has been undermining AI-assisted development from the beginning.

The question now: Can VibeTDD be redesigned to work with AI's "never give up" behavior rather than against it?

The "never give up" discovery changes everything about AI-assisted development. It's not about finding better prompts or more detailed instructions - it's about designing development workflows that channel AI's relentless problem-solving into productive directions rather than fighting its fundamental nature.

Forem: Maksim Matlakhov

Event Modeling: Visible Issues and My Vision

The Problems I Discovered

1. Canvas Layout Nightmare

2. Business Rules Documentation Gap

A Simple POC to Test My Ideas

1. Slices Board

2. Structured Slice Editor

3. Smart Field Connections

4. Custom Type System

5. Test Scenarios Generation

Check the full demo video:

Key Advantages of This Approach

The Bigger Picture

Technical Implementation Notes

What's Next

Event Handling: Automatic Event Bootstrapping

The Problem with Historical Events

Some Approaches to Manage That

A Better Way: Automatic Event Bootstrap

How It Works

Step 1: Create a Storage Adapter

Step 2: Add the EventBootstrap Annotation

What the Framework Does

Event Fetching Interface

Automatic Topic Detection

Progressive Fetching with Retry Logic

Progress Tracking

Same Handlers for Historical and Real-Time Events

Adding New Features Later

Perfect for Continuous Integration

Bootstrap States

Summary

Source Code

Event Handling: Inbox Pattern for Complex Scenarios

Why Inbox Pattern?

The Problem with Immediate Processing

Processing Types

1. Single Event Processing

2. Sequential Processing

3. Batch Processing

Demo Architecture

How It All Works Together

Implementation: Fraud Detection Example

Step 1: Storing Events

Step 2: Defining Event Context

Step 3: Tracking Processing State

Step 4: Configuring the Scheduler

Step 5: Setting Up Processing Infrastructure

Step 6: The Event Runner

Step 7: The Scheduler

Step 8: Creating the Handler

Key Benefits

Coming Next

Event Handling: Keep It Fast and Simple

Three Simple Rules

Two Types of Events

Type 1: Simple Updates

Type 2: Complex Work

How Type 1 Processing Works

Type 1 Implementation

The Generic Helper

Versioned Fields for Concurrency

Event Consumer

The Service Setup

Why This Works So Well

What's Next: Complex Events

Messaging Broker Migration: Why Lock-in Hurts and How to Avoid It

When You Need to Migrate

The Migration Nightmare: Common Anti-Patterns

Anti-Pattern 1: Broker as Event Storage

Anti-Pattern 2: Dual-Send Without Atomicity

Anti-Pattern 3: Feature-Dependent Architecture

The Path to Broker Independence

Design Principles

The Migration Strategy: Multi-Broker Publishing

The processedBrokers Field

The Spring Pub/Sub Example

Producer Implementation

Consumer Implementation

The `processedBrokers` Field