Event sourcing - basic concepts

William Boxhall — Fri, 13 May 2022 06:01:46 +0000

At Culture Amp we are increasingly embracing event sourced systems for a few reasons:

Architectural modularisation and flexibility
The strong emphasis on Domain Driven Design and modelling, best representing the business domain
The semantically meaningful append-only event streams mean we never have to throw away data, allowing us to leverage this highest-possible-fidelity historical data in new and interesting ways across our systems

We have two small open-source event-sourcing micro-frameworks,
one named Event Framework written in Ruby, and another written in Kotlin named Kestrel.

As we hire more and more people and do more and more event sourcing, we're looking for easy ways for newcomers to gain a grasp of the basics.

An example domain

I was searching for an example domain that could represent the basic concepts of event sourcing with:

Little-to-no text, favouring shapes, symbols, boxes and arrows, in order to emphasise the mechanics of how things sit together
A well-known, bounded domain

Tetris

I realised Tetris ticks both of these boxes, with its minimalist, iconic shapes and sheer oldschool ubiquity - everyone has played Tetris!

The Tetris Playfield also has the benefit of being an entity that is subject to forces other than just the Player, it is affected by new blocks coming in from outside (in reaction to the previous block landing), and subject to the ticking of time (gravity moving the blocks down the Playfield).

Here is an example of how an event sourced implementation of Tetris might look, and below we will unpack it, piece by piece.

Aggregates

An aggregate instance is the central entity to which things happen. In many event sourced systems there can be many instances of many different types of aggregates. Aggregates tend to have a life-cycle: a creation event, many update events, and, often but not always, an ending event.

In the Tetris example, the Tetris Playfield Aggregate is instantiated every time a new game is started. In this implementation, the first thing that happens (the creation event) to it is a New Block Appeared at the top of the Playfield. Each instance of the aggregate has an id, in this case, ids a, b and c.

Commands

One form of input are user commands, which represent the user intention to do something to an aggregate, and which may succeed or be rejected.

In the Tetris example, the user may attempt to nudge the block left or right, rotate it clockwise or anti-clockwise, or send it immediately to the bottom of the Playfield. If the user tries to move the block through a wall, the user command will be rejected.

Events

When an aggregate doesn't reject a command, it sinks one or more events to the event store. The aggregate can then use these events to build up "just enough" state in order to accept or reject subsequent commands.

In the Tetris example, when the movement commands are accepted by the Tetris Playfield Aggregate based on the current state of the Playfield, the "update" events Moved Right, Moved Left, Rotated Clockwise, Rotated Anti-clockwise and Sent To Bottom are saved as events in a time ordered append-only fashion.

Non-user Actors

Apart from user commands, scheduled commands, reactors or other actors may act upon an aggregate.

Scheduled commands

It's a common pattern to have "scheduled commands" in event sourced systems.

In the Tetris example, we have Time running on a schedule (once per second), interacting with the aggregate. It's the ticking of Time that allows blocks to move down the playfield, and when the aggregate detects that the block has reached the ground, the aggregate can emit an additional Landed event following the Fell event.

Reactors

Reactors have the additional feature that they can listen to the events that have happened so far, and react to those events by sending new commands back into aggregates.

In the Tetris example, we want the Landed event to trigger a New Block Appeared at the top of the Playfield.

With the combination of blocks being able to "land" and new blocks being able to enter the Playfield, this means the aggregate can now sink Line Cleared or Tetris Cleared events, and of course Game Over "ending event" when the Playfield gets full.

Projectors, projections and queries

So far we've only spoken about "write" actions, things that attempt to change the system and sink events. In event sourced systems, we usually want to be able to view or "query" the state of the system in various context-specific forms.

"Projectors" allow us to "project" the events in the event store into different views or "projections". Projectors (and Reactors) only need to listen for events with relevant types and can skip the rest, as indicated in the diagram.

In the Tetris example, the Player needs to be able to view the current state of the Playfield so that they can decide which further input commands they wish to send. Once the game is over, they can see a summary of what happened during the game, including their final score. The act of requesting data from a projection is called a "query"

The whole picture

With the following basic building blocks we have a full end-to-end eventsourced system:

Commands
Aggregates
Events
Reactors
Projectors, projections and queries

Tetris is a great domain to illustrate the fundamentals of event sourcing, but in reality, one probably wouldn't build a video game using event sourcing because video games need to optimise for very low latency throughput. Event sourced systems optimise instead for domain richness, data fidelity, architectural modularisation and flexibility. In future articles we will dive into how to to actually build these systems with code.

Thanks to Mache for the cover image.

Forem: William Boxhall