Forem: Semyon Kirekov

Golang Dependency Inversion — A Java Developer's Point of View

Semyon Kirekov — Tue, 07 Oct 2025 18:54:47 +0000

I’ve been writing in Java for some years. But recently I started a new job, and I had to switch to Golang. Of course, there are many differences between these languages. Like error handling, concurrency model, encapsulation mechanism and other stuff. But in this article, I want to focus on one particular thing. And that is the dependency inversion principle.

I consider this pattern the most important in programming. The code is not perfect. Sometimes you have to break rules. Anyway, the dependency inversion principle is the one that keeps your code testable and modularised. If you break it, the entire codebase can quickly become a mess even if you consistently follow the other SOLI patterns.

So, I want to tell you how Golang features can help you push the dependency inversion principle to a new level.

Even if you don't know Golang or Java, it's fine. I tried to keep the examples easy and self-explanatory.

The case

We have three modules (or packages). Those are orders, users and delivery. We create a use-case for putting an existing Order into Delivery department. The steps are:

Find Order by id.
Get the User who posted the Order in the first place.
Check that User is not blocked.
Check that Delivery has the resources to process the Order delivery pipeline.
If everything is fine, put the Order for the Delivery.

Java

Supposing we have three interfaces DeliveryService, UserService, and OrderRepository that provide methods needed for this use case (and probably other ones). Here is a potential declaration.

public interface UserService {
    User getById(long id);
    void create(User user);
    void update(User user);
}

public interface DeliveryService {
    // Some generic payload. The format is not important for the article
    boolean hasResourcesToDeliver(String orderPayload)
    void putOrderForDelivery(long orderId, String orderPayload);
    void cancelDelivery(long orderId);
    Status getDeliveryStatus(long orderId);
}

public interface OrderRepository {
    Order findById(long orderId);
    List<Order> findAll();
    void create(Order order);
    void update(Order order);
}

public interface OrderService {
    void initDelivery(long orderId);
}

Now, let's write the possible OrderService implementation.

public class OrderServiceImpl implements OrderService {
    private final UserService userService;
    private final DeliveryService deliveryService;
    private final OrderRepository orderRepository;

    @Override
    public void initDelivery(long orderId) {
        Order order = orderRepository.findById(orderId);
        User user = userService.getById(order.getPostedById());
        if (user.isBlocked()) {
            throw new InitDeliveryException("User is blocked");
        }
        if (!deliveryService.hasResourcesForDelivery(order.getPayload())) {
            throw new InitDeliveryException("No resources to deliver");
        }
        deliveryService.putOrderForDelivery(orderId, order.getPayload());
    }
}

In real application these calls between different services may put the system into inconsistent state (one call failed and the others didn’t proceed). Let’s omit them problems for the sake of simplicity.

You may notice that the OrderServiceImpl doesn’t require each method in those interfaces. Some of them are untouched. For example, OrderRepository.findAll, DeliveryService.getDeliveryStatus, or UserService.update. Meaning that the module implicitly imports unnecessary methods.

If you’re a Java developer, most likely you would leave it as is (I would definitely). Sure, you can declare another package-private interface that is used specifically by OrderServiceImpl. But that would result in one of these outcomes:

The actual implementations should also implement those specific interfaces

That would look like this:

public class UserServiceImpl implements UserService, LocalUserService {
    ...
}

Where LocalUserService is the specific interface containing only getById method.

Another approach is to create separate implementations of those interfaces and delegate calls to OrderService/DeliveryService/OrderRepository

For example, something like this:

// package-private interface used by OrderServiceImpl
interface LocalUserService { 
    User getById(long id);
}

class LocalUserServiceImpl implements LocalUserService {
    private final UserService delegate;

    @Override
    public User getById(long id) {
        return delegate.getById(id);
    }
}

The first option means that the users module would have to depend on orders as well, because it has to import LocalUserService interface. And that is not good for isolation.

The second one does not break the modules’ boundaries. But it requires more boilerplate code (and probably a decrease in the whole test coverage). Technically, the solution is valid, but I think it’s not a real thing for the vast majority of cases.

Golang

Looking ahead, Golang allows us to solve the issue elegantly. But let’s keep things slow and start with the same setup.

type OrderService interface {
    InitDelivery(orderId int64) error
}

type orderServiceImpl struct {
    userService UserService // those are also interface types
    deliveryService DeliveryService
    orderRepository OrderRepository
}
func New(u UserService, d DeliveryService, or OrderRepository) OrderService {
    return &orderServiceImpl {
        userService: u,
        deliveryService: d,
        orderRepository: or
    }
}

// OrderService method's implementation

func (o *orderService) InitDelivery(orderId int64) error {
    order := o.orderRepository.FindById(orderId)
    user := o.userService.GetById(order.PostedById)
    if user.Blocked {
        return errors.New("user is blocked")
    }
    if o.deliveryService.HasResourcesForDelivery(order.Payload) {
        return errors.New("no resources to deliver")
    }
    o.deliveryService.PutOrderForDelivery(orderId, order.Payload)
    return nil
}

For now, there is no difference between Java and Golang approaches. But here comes one interesting Golang’s feature. You see, Java demands explicit interface implementation. But Golang doesn’t. If a structure has all interface methods with the same signature, the compiler thinks that this structure implements the interface implicitly.

This slight difference allows us to refactor OrderService in this way:

Create local private interfaces containing only required methods (that are called inside InitDelivery function).
Accept those interfaces as parameters of the New function.
Pass the actual implementations on New function invocation without changing the code.

Take a look at the diagram below to understand the principle.

And here is the refactored code:

type OrderService interface {
    InitDelivery(orderId int64) error
}

type orderServiceImpl struct {
    userService localUserService 
    deliveryService localDeliveryService
    orderRepository localOrderRepository 
}

type localUserService interface {
    GetById(orderId int64) User
}

type localDeliveryService interface {
    HasResourcesForDelivery(orderPayload string) bool
    PutOrderForDelivery(orderId int64, orderPayload string)
}

type localOrderRepository interface {
    FindById(orderId int64) Order
}

func New(u localUserService, d localDeliveryService, or localOrderRepository) OrderService {
    return &orderServiceImpl {
        userService: u,
        deliveryService: d,
        orderRepository: or
    }
}

func (o *orderService) InitDelivery(orderId int64) error {
    order := o.orderRepository.FindById(orderId)
    user := o.userService.GetById(order.PostedById)
    if user.Blocked {
        return errors.New("user is blocked")
    }
    if o.deliveryService.HasResourcesForDelivery(orderId) {
        return errors.New("no resources to deliver")
    }
    o.deliveryService.putOrderForDelivery(orderId)
    return nil
}

This change may not seem like a big deal. But it actually changes the way you think about software design:

Your module has no direct dependency on other modules. This makes it easier to refactor and enhance external interfaces.
If you have a local interface with dedicated methods, there is no need to mock/stub methods that aren’t called in this use case.
Even if you have to change the signature of the calling external interface, it won’t break the code and tests within the current module. Sure, you won’t be able to pass the structure as a method parameter anymore, which would probably result in a compilation error. But the other modules that use this method are untouched. So, you can decide whether you need to refactor it or if it's better to pass a proxy implementation during initialisation.

Conclusion

At first glance, implicit interface implementation in Golang seemed to me inconvenient (especially in comparison to Java). But when you solve distinct problems, you understand the architectural reason behind this feature. Of course, it doesn’t mean you should create a local interface for every interaction. Sometimes it’s just easier to rely on the external interface. But having the ability to do it differently is a pleasant bonus.

Thank you for reading this article. If you switched from Java to Golang (or maybe vice versa), leave your thoughts down below. It will be interesting to discuss opinions. Have a nice day!

JOOQ Is Not a Replacement for Hibernate. They Solve Different Problems

Semyon Kirekov — Sat, 11 Jan 2025 12:00:36 +0000

I've originally written this article in Russian. So, if you're native speaker, you can read it by this link.

In the past year or so, I've come across articles and talks suggesting that JOOQ is a modern and superior alternative to Hibernate. The arguments typically include:

JOOQ allows you to verify everything at compile time, whereas Hibernate does not!
Hibernate generates strange and not always optimal queries, while with JOOQ, everything is transparent!
Hibernate entities are mutable, which is bad. JOOQ allows all entities to be immutable (hello, functional programming)!
JOOQ doesn't involve any "magic" with annotations!

Let me state upfront that I consider JOOQ an excellent library (specifically a library, not a framework like Hibernate). It excels at its task — working with SQL in a statically typed manner to catch most errors at compile time.

However, when I hear the argument that Hibernate's time has passed and we should now write everything using JOOQ, it sounds to me like saying the era of relational databases is over and we should only use NoSQL now. Does that sound funny? Yet, not so long ago, such discussions were quite serious.

The issue lies in a misunderstanding of the core problems these two tools address. In this article, I aim to clarify these questions. We will explore:

What is Transaction Script?
What is the Domain Model pattern?
What specific problems do Hibernate and JOOQ solve?
Why isn't one a replacement for the other, and how can they coexist?

Transaction Script

The simplest and most intuitive way to work with a database is the Transaction Script pattern. In brief, you organize all your business logic as a set of SQL commands combined into a single transaction. Typically, each method in a class represents a business operation and is confined to one transaction.

Suppose we're developing an application that allows speakers to submit their talks to a conference (for simplicity, we'll only record the talk's title). Following the Transaction Script pattern, the method for submitting a talk might look like this (using JDBI for SQL):

@Service
@RequiredArgsConstructor
public class TalkService {
    private final Jdbi jdbi;

    public TalkSubmittedResult submitTalk(Long speakerId, String title) {
        var talkId = jdbi.inTransaction(handle -> {
            // Count the number of accepted talks by the speaker
            var acceptedTalksCount =
                handle.select("SELECT count(*) FROM talk WHERE speaker_id = :id AND status = 'ACCEPTED'")
                    .bind("id", speakerId)
                    .mapTo(Long.class)
                    .one();
            // Check if the speaker is experienced
            var experienced = acceptedTalksCount >= 10;
            // Determine the maximum allowable number of submitted talks
            var maxSubmittedTalksCount = experienced ? 5 : 3;
            var submittedTalksCount =
                handle.select("SELECT count(*) FROM talk WHERE speaker_id = :id AND status = 'SUBMITTED'")
                    .bind("id", speakerId)
                    .mapTo(Long.class)
                    .one();
            // If the maximum number of submitted talks is exceeded, throw an exception
            if (submittedTalksCount >= maxSubmittedTalksCount) {
                throw new CannotSubmitTalkException("Submitted talks count is maximum: " + maxSubmittedTalksCount);
            }
            return handle.createUpdate(
                    "INSERT INTO talk (speaker_id, status, title) " +
                    "VALUES (:id, 'SUBMITTED', :title)"
                ).bind("id", speakerId)
                   .bind("title", title)
                   .executeAndReturnGeneratedKeys("id")
                   .mapTo(Long.class)
                   .one();
        });
        return new TalkSubmittedResult(talkId);
    }
}

In this code:

We count how many talks the speaker has already submitted.
We check if the maximum allowable number of submitted talks is exceeded.
If everything is okay, we create a new talk with the status SUBMITTED.

There is a potential race condition here, but for simplicity, we'll not focus on that.

Pros of this approach:

The SQL being executed is straightforward and predictable. It’s easy to tweak it for performance improvements if needed.
We only fetch the necessary data from the database.
With JOOQ, this code can be written more simply, concisely, and with static typing!

Cons:

It’s impossible to test the business logic with unit tests alone. You’ll need integration tests (and quite a few of them).
If the domain is complex, this approach can quickly lead to spaghetti code.
There’s a risk of code duplication, which could lead to unexpected bugs as the system evolves.

This approach is valid and makes sense if your service has very simple logic that isn’t expected to become more complex over time. However, domains are often larger. Therefore, we need an alternative.

Domain Model

The idea of the Domain Model pattern is that we no longer tie our business logic directly to SQL commands. Instead, we create domain objects (in the context of Java, classes) that describe behavior and store data about domain entities.

In this article, we won’t discuss the difference between anemic and rich models. If you're interested, I’ve written a detailed piece on that topic.

Business scenarios (services) should use only these objects and avoid being tied to specific database queries.

Of course, in reality, we may have a mix of interactions with domain objects and direct database queries to meet performance requirements. Here, we’re discussing the classic approach to implementing the Domain Model, where encapsulation and isolation are not violated.

For example, if we’re talking about the entities Speaker and Talk, as mentioned earlier, the domain objects might look like this:

@AllArgsConstructor
public class Speaker {
    private Long id;
    private String firstName;
    private String lastName;
    private List<Talk> talks;

    public Talk submitTalk(String title) {
        boolean experienced = countTalksByStatus(Status.ACCEPTED) >= 10;
        int maxSubmittedTalksCount = experienced ? 3 : 5;
        if (countTalksByStatus(Status.SUBMITTED) >= maxSubmittedTalksCount) {
            throw new CannotSubmitTalkException(
              "Submitted talks count is maximum: " + maxSubmittedTalksCount);
        }
        Talk talk = Talk.newTalk(this, Status.SUBMITTED, title);
        talks.add(talk);
        return talk;
    }

    private long countTalksByStatus(Talk.Status status) {
        return talks.stream().filter(t -> t.getStatus().equals(status)).count();
    }
}

@AllArgsConstructor
public class Talk {
    private Long id;
    private Speaker speaker;
    private Status status;
    private String title;
    private int talkNumber;

    void setStatus(Function<Status, Status> fnStatus) {
        this.status = fnStatus.apply(this.status);
    }

    public enum Status {
        SUBMITTED, ACCEPTED, REJECTED
    }
}

Here, the Speaker class contains the business logic for submitting a talk. The database interaction is abstracted away, allowing the domain model to focus on business rules.

Supposing this repository interface:

public interface SpeakerRepository {
    Speaker findById(Long id);
    void save(Speaker speaker);
}

Then the SpeakerService can be implemented this way:

@Service
@RequiredArgsConstructor
public class SpeakerService {
    private final SpeakerRepository repo;

    public TalkSubmittedResult submitTalk(Long speakerId, String title) {
        Speaker speaker = repo.findById(speakerId);
        Talk talk = speaker.submitTalk(title);
        repo.save(speaker);
        return new TalkSubmittedResult(talk.getId());
    }
}

Pros of the Domain Model:

Domain objects are completely decoupled from implementation details (i.e., the database). This makes them easy to test with regular unit tests.
Business logic is centralized within the domain objects. This greatly reduces the risk of logic spreading across the application, unlike in the Transaction Script approach.
If desired, domain objects can be made fully immutable, which increases safety when working with them (you can pass them to any method without worrying about accidental modifications).
Fields in domain objects can be replaced with Value Objects, which not only improves readability but also ensures the validity of fields at the time of assignment (you can’t create a Value Object with invalid content).

In short, there are plenty of advantages. However, there is one important challenge. Interestingly, in books on Domain-Driven Design, which often promote the Domain Model pattern, this problem is either not mentioned at all or only briefly touched upon.

The problem is how do you save domain objects to the database and then read them back? In other words, how do you implement a repository?

Nowadays, the answer is obvious. Just use Hibernate (or even better, Spring Data JPA) and save yourself the trouble. But let’s imagine we’re in a world where ORM frameworks haven’t been invented. How would we solve this problem?

Manual mapping

To implement SpeakerRepository I also use JDBI:

@AllArgsConstructor
@Repository
public class JdbiSpeakerRepository implements SpeakerRepository {
   private final Jdbi jdbi;

   @Override
   public Speaker findById(Long id) {
       return jdbi.inTransaction(handle -> {
           return handle.select("SELECT * FROM speaker s LEFT JOIN talk t ON t.speaker_id = s.id WHERE id = :id")
                .bind("id", speakerId)
                .mapTo(Speaker.class) // mapping is out of scope for simplicity
                .execute();
       });
   }

   @Override
   public void save(Speaker speaker) {
       jdbi.inTransaction(handle -> {
           // A complex logic checking:
           // 1. Whether a speaker is already present
           // 2. Generating UPDATE/INSERT/DELETE staments
           // 3. Possibly optimistic locking implementation
           // 4. etc
       });
   }
}

The approach is simple. For each repository, we write a separate implementation that works with the database using any SQL library (like JOOQ or JDBI).

At first glance (and maybe even the second), this solution might seem quite good. Consider this:

The code remains highly transparent, just like in the Transaction Script approach.
No more issues with testing business logic only through integration tests. These are needed only for repository implementations (and maybe a few E2E scenarios).
The mapping code is right in front of us. No Hibernate magic is involved. Found a bug? Locate the right line and fix it.

The Need for Hibernate

Things get much more interesting in the real world, where you might encounter scenarios like these:

Domain objects may need to support inheritance.
A group of fields can be combined into a separate Value Object (Embedded in JPA/Hibernate).
Some fields shouldn’t be loaded every time you fetch a domain object, but only when accessed, to improve performance (lazy loading).
There can be complex relationships between objects (one-to-many, many-to-many, etc.).
You need to include only the fields that have changed in the UPDATE statement because other fields rarely change, and there’s no point in sending them over the network (DynamicUpdate annotation).

On top of that, you’ll need to maintain the mapping code as your business logic and domain objects evolve.

If you try to handle each of these points on your own, you’ll eventually find yourself (surprise!) writing your Hibernate-like framework — or more likely, a much simpler version of it.

Goals of JOOQ and Hibernate

JOOQ addresses the lack of static typing when writing SQL queries. This helps reduce the number of errors at the compilation stage. With code generation directly from the database schema, any updates to the schema will immediately show where the code needs to be fixed (it simply won’t compile).

Hibernate solves the problem of mapping domain objects to a relational database and vice versa (reading data from the database and mapping it to domain objects).

Therefore, it doesn’t make sense to argue that Hibernate is worse or JOOQ is better. These tools are designed for different purposes. If your application is built around the Transaction Script paradigm, JOOQ is undoubtedly the ideal choice. But if you want to use the Domain Model pattern and avoid Hibernate, you’ll have to deal with the joys of manual mapping in custom repository implementations. Of course, if your employer is paying you to build yet another Hibernate killer, no questions there. But most likely, they expect you to focus on business logic, not infrastructure code for object-to-database mapping.

By the way, I believe the combination of Hibernate and JOOQ works well for CQRS. You have an application (or a logical part of it) that executes commands, like CREATE/UPDATE/DELETE operations — this is where Hibernate fits perfectly. On the other hand, you have a query service that reads data. Here, JOOQ is brilliant. It makes building complex queries and optimizing them much easier than with Hibernate.

What About DAOs in JOOQ?

It’s true. JOOQ allows you to generate DAOs that contain standard queries for fetching entities from the database. You can even extend these DAOs with your methods. Moreover, JOOQ will generate entities that can be populated using setters, similar to Hibernate, and passed to the insert or update methods in the DAO. Isn’t that like Spring Data?

For simple cases, this can indeed work. However, it’s not much different from manually implementing a repository. The problems are similar:

The entities won’t have any relationships: no ManyToOne, no OneToMany. Just the database columns, which makes writing business logic much harder.
Entities are generated individually. You can’t organize them into an inheritance hierarchy.
The fact that entities are generated along with the DAOs means you can’t modify them as you wish. For example, replacing a field with a Value Object, adding a relationship to another entity, or grouping fields into an Embeddable won’t be possible because regenerating the entities will overwrite your changes. Yes, you can configure the generator to create entities slightly differently, but the customization options are limited (and not as convenient as writing the code yourself).

So, if you want to build a complex domain model, you’ll have to do it manually. Without Hibernate, the responsibility for mapping will fall entirely on you. Sure, using JOOQ is more pleasant than JDBI, but the process will still be labour-intensive.

Even Lukas Eder, the creator of JOOQ, mentions in his blog that DAOs were added to the library because it’s a popular pattern, not because he necessarily recommends using them.

Conclusion

Thank you for reading the article. I’m a big fan of Hibernate and consider it an excellent framework. However, I understand that some may find JOOQ more convenient. The main point of my article is that Hibernate and JOOQ are not rivals. These tools can coexist even within the same product if they bring value.

If you have any comments or feedback on the content, I’d be happy to discuss them. Have a productive day!

Resources

JDBI
Transaction Script
Domain Model
My article – Rich Domain Model with Spring Boot and Hibernate
Repository pattern
Value Object
JPA Embedded
JPA DynamicUpdate
CQRS
Lukas Eder: To DAO or not to DAO

Custom methods in REST API endpoints

Semyon Kirekov — Wed, 01 May 2024 18:30:49 +0000

HTTP approach proposes such methods to handle resources:

POST for creation.
GET for retrieving information.
PUT for replacing a resource with the given content.
PATCH for partially updating a resource.
DELETE for deleting the resource.

There are other HTTP methods like HEAD or OPTIONS but mostly they are either generated by frameworks or used only due to technical concerns. So, we’re not discussing them in this article.

Mostly these methods are sufficient to build RESTful API. However, some cases do not align well with predefined methods.

In this article, I'm telling you:

The problem with mapping business operation to a set of predefined HTTP methods.
Ways to implement custom methods.
Pros and cons of each solution.

The problem with mapping business operation to a set of predefined HTTP methods

Consider we're developing an email service. A user created a draft and now they want to send it. What HTTP method should you apply in this case? There are multiple options:

POST /drafts/{draftId}

Well, if you send a draft with POST, then how do you create a new draft?

PATCH /drafts/{draftId}

{ "status": "sent" }

In this case, we don't introduce any specific endpoint. There is one that updates the draft resource content. Therefore, we can call it to change the draft status to sent value. It supposes to trigger mail sending.

The advantage is that your API remains concise. But there are also crucial drawbacks:

The flow might be misleading for a client. It’s not really obvious that such an operation should send a draft.
The approach is error prone from a client's perspective. If a client sends status field by mistake, it might trigger draft sending. Even if a client just wanted to update the draft content.
The code on the backend also becomes more complicated. Developers have to add if cases for the endpoint that triggers draft sending if there is a particular field in the request.
Authorization for sending a draft can be different from a regular draft update. Again, more complicated scenario on the backend.
Sending a draft can have different semantics. For example, if a user triggers draft sending, the server might return some jobId representing the status of the operation. But regular PATCH invocations should occur synchronously. Meaning that return values also differ, which makes API more complicated.

The example assumes that there is a single resource draft. Other implementations can have different resources for drafts, received emails, and sent emails. But I skip these details for the sake of simplicity.

I’ve described a rather simple scenario. And even this one requires some sort of custom methods. Let’s discuss possible implementations.

Sub-resource

This is the most straight-forward one. I bet you’ve seen such examples often. Look at the code snippet below:

POST /drafts/{draftId}/send

It’s a separate endpoint. So, we don’t have to deal with PATCH overlapping. The problem here is not about implementation, but REST API principles. A path describes a resource. And it should be a noun. But here the send word is a verb that is a marker for an action. It raises bizarre questions:

What is a send? Is it an entity for a given resource?
Can you invoke GET /drafts/{draftId}/send?
Can you transfer send entity to another draft?

You may think that I’m just making these claims out of nowhere. But imagine that a new person comes to your project. If your product is small and you don’t have many endpoints, then probably nothing will go wrong. But if your services exposes dozens of endpoints and many of them have custom methods in the API, then a new business analyst or architect can waste time figuring out the meaning of that sub-resource that actually has never existed.

As a matter of fact, I recommend you to avoid this solution.

A special `actions` endpoints

If you have custom methods, why can't it be a dedicated resource? Look at the code example below:

POST /drafts/{draftId}/actions

{ action: "send" }

That looks more intriguing. The pros are:

This endpoint fits well with REST API philosophy.
GET can return the list of all available actions. You can even add authorization (some actions are not available for specific users).
In some cases, updating existing actions with PATCH or PUT might be useful.

Still, there are some cons:

All custom methods map to a single endpoint. Which adds problems to the backend (complex scenario, extra if statements, complicated authorization, etc.).
Some actions require additional parameters. If you document your REST API with OpenAPI (which you should do), then you would have to distinguish parameters for different action with oneOf/anyOf clause. Some client generators have problem with dealing these conditions correctly. Therefore, unnecessary technical concerns occur.

I can say that this approach is valid and might be effective. But usually it's an overengineering.

Custom method with a query parameter

Look at the example below:

POST /drafts/{draftId}?method=send

I'm not stopping here because this approach is almost identical to the one with PATCH that we've seen before. Pros and cons are the same (additional if statements on the backend, unexpected errors on the client side due to wrongly passed parameters etc.). So, let's move forward.

Custom method with a colon punctuation mark

This approach is similar to sub-resource one, but has different semantics. Look at the code snippet below:

POST /drafts/{draftId}:send

The idea is simple:

You declare all custom methods as POST ones.
Then you add the verb describing the custom operation divided from the resource path by a colon.

I see many advantages of such technique:

A custom operation maps as a dedicated endpoint. Meaning that a generated client will have a separate function for sending a draft.
Because of the presence of a separate endpoint code on the backend becomes easier (authorization, returning values etc.).
You don’t have to worry about overlapping with existing regular resources.
A colon punctuation mark clearly puts a difference between a resource and an action.

I’ve taken this approach from the book API Design Patterns. I strongly recommend you to read it if you deal with creating REST API in your daily work. There are lots of useful patterns besides custom operation methods.

Conclusion

Custom methods are necessary for the majority of applications. But you should apply the tool with caution. Don’t fall into the trap of creating too much custom methods. That makes your RESTful API actually RPC. However, if you need one, you know how to design it.

Thank you very much for reading the article! I hope you’ve learnt something new and it will help you to complete the work more efficiently. I’d be grateful, if you pressed a like button, left a comment and shared the piece with your friends and colleagues. Have a nice and productive day!

Resources

Rich Domain Model with Spring Boot and Hibernate

Semyon Kirekov — Thu, 07 Sep 2023 19:26:14 +0000

The article is a long read. I recommend you to look through the Table of contents in advance. Perhaps some parts can be more intriguing than others.

I like Hibernate. Even though it's complicated (and error-prone sometimes), I consider it an extremely useful framework. If you know how to cook it, it shines.

Though Hibernate is in implementation of JPA, I'm going to say Hibernate every time I mean JPA. Just for the sake of simplicity.

Lots of Java developers use Hibernate in their projects. However, I’ve noticed a strange tendency. Java devs apply Anemic Domain Model pattern almost every time. Whenever I ask about reasons for picking up that strategy, I get answers like:

We've always done it before and it works fine.
Our developers got used to such architecture.
Do we have any other options?

That's why I've decided to come up with this article. I want to reconsider the status quo of Anemic Domain Model usage with Hibernate and propose you something different. And that is Rich Domain Model pattern.

In this article, I'm telling you:

What is Rich Domain Model?
What's wrong with Anemic Domain Model and how can Rich one fix the issues?
Step by step solution.
What disadvantages does Rich Domain Model have?

You can check out the entire repository with code examples by this link.

The problems with Anemic Domain Model
1. Too smart services
2. Possible invariants' violation
3. Lack of encapsulation
4. Testing is harder
Rich Domain Model principle
Don't add setters, getters, and public no-args constructor
1. No-args constructor allows incostistent object instantiation
2. Setters break encapsulation
3. Getters break encapsulation
4. Current result
Aggregate and Aggregate root
Evolution of requirements
1. Each Pocket must possess at least one Tamagotchi
2. Every Tamagotchi name has to be unique within a Pocket
3. If a user deletes a Tamagotchi, they can restore it by name
Querying data
1. Manual queries
2. Introducing toDto method
Unit testing entities
1. Pocket must always possess at least one Tamagotchi
2. If you delete Tamagotchi, you can restore it by name
3. If you delete multiple Tamagotchi with the same name, you can only restore the last one
Integration testing
1. Create Pocket
2. Create Tamagotchi
3. Update Tamagotchi
Performance implications
1. Optimization of queries
2. Pinpointing optimized checks
Database generated ID
1. Manually fill the id
2. Introducing business key
Is Rich Domain Model always worth it?
Conclusion
Resources

The problems with Anemic Domain Model

Firstly, let’s discuss the domain. We’re going to develop the Tamagotchi application. Pocket may have many Tamagotchi instances, but each Tamagotchi belongs to a single Pocket. Therefore, the relationship is Pocket --one-to-many-> Tamagotchi.

Most likely the Anemic Domain Model solution would be implemented in this way:

I bet you’ve seen a lot of similar Java code. But the solution contains many problems. Let’s discuss them one by one.

Too smart services

Anemic Domain Model requires the service layer to possess all business logic. While entities act as dummy data structures. But entities are not static. They develop during the time. We may add some fields and delete others. Or we can combine existing fields in Embeddable object.

Here, services have to know every minor detail of the entity they are working with. Because any operation may require access to different fields. Meaning that even a slight change in an entity may lead to major restructuring in many services. Actually, that breaks Open-Closed principle. The code becomes not object-oriented but procedural. We don’t use benefits of OOP paradigm. Instead, we bring additional difficulties.

Possible invariants' violation

Invariant is a business rule that allows only certain changes to entities. It guarantees that we won’t transmit entities to the wrong state. For example, suppose that Pocket may contain only three Tamagotchi by default. If you want to have more, you need to buy premium subscription. That’s an invariant. The code has to disallow adding fourth Tamagotchi to Pocket, if user don’t purchase the additional feature.

If we choose the Anemic Domain Model approach, it means that services are obligatory to check invariants and cancel operation if needed. But invariants are also not static. Imagine that the rule of three Tamagotchi within Pocket has not been introduced from the start of the project. But we want to add it now. It means that we have to check every method and function that might create a new Tamagotchi and add corresponding checks.

It becomes even worse if changes are broader. Suppose that Tamagotchi has become a part of Saga pattern. Now it contains status field that has a value of PENDING. If Tamagotchi is PENDING, you can neither delete it nor update it. Do you see where I’m going? You have to check every piece of code that updates or deletes Tamagotchi and make sure you don’t miss any check for PENDING status.

Lack of encapsulation

Encapsulation in OOP is a mechanism that restricts direct access to certain data. That makes sense. Entity might have several fields, but it doesn’t mean we want to allow changing each of them. We might change only simultaneously concrete fields. Other ones are allowed to be updated only if the entity transmits to a specific state.

Anemic Domain Model forces us to give up encapsulation and put @Getter and @Setter annotations from Lombok without considering the consequences.

And the biggest problem with violating encapsulation is that code becomes more dangerous to work with. You cannot just call setName or setStatus methods. But you have to make sure that you check specific conditions in advance. Again, invariants aren’t static. So, every mutation call to an entity is like a land mine. You don’t know what breaks next, if you miss a single condition check.

Testing is harder

Mostly developers use Hibernate in combination with Spring Boot. Meaning that services are regular Spring beans with @Transactional annotation. Usually those services contain spaghetti code of entities, repositories, and other services invocations. When it comes to testing, I see developers choose one of options:

Integration testing.
Mocking everything.

Don’t get me wrong. I think that integration testing is crucial. And Tescontainers library especially helped to make the process smooth. However, I think that the count of integration tests should be as minimum as possible. If you can validate something with a simple unit test, do it this way. Bringing too much integration tests into the project also leads to certain difficulties:

Integration tests are harder to maintain.
There is always a shared resource (database, in this case). So, tests might unexpectedly become dependent on each other. Tests might get flaky.
It's tough to run integration tests in parallel.
Those tests are much slower. If your project is old enough and you have many integration tests, a regular CI build can run in 30 minutes or even more.

What about mocking? I think such tests are almost useless. I don’t mean that mocking in general is a bad idea. But if you try to mock every call to Spring Data JPA repository and other service, it may occur that you don’t test the behaviour. You just verify the correct order of mocks’ invocations. So, tests become fragile and an enormous burden to maintain.

Rich Domain Model principle

On the contrary, Rich Domain Model pattern proposes a different approach. Look at the diagram below.

As you can see, entities hold the required business logic. While services act like a thin layer that delegates call to repositories and entities.

Rich Domain Model correlates with tactical patterns of Domain Driven Design. The one that we're interested in is aggregate.

Aggregate is a cluster of domain objects that you can treat as a whole unit. For example, Pocket has many Tamagotchis. Meaning that Pocket and Tamagotchi can be a single aggregate. Aggregate root is the entity that allows direct access to aggregate and guarantees invariants’ correctness. Therefore, if we want to change something in Tamagotchi, we should only interact with Pocket.

By introducing Rich Domain Model, I want to solve these problems:

Code should become more business-oriented. If logic is divided between many services, it's hard to understand the actual operation flow (especially for newcomers).
Let the compiler validate your code. If an entity has a setter for every field, you should put additional check whenever you invoke it. But if an entity provides only certain amount of methods that mutates its state, it means that incorrect transition is not possible due to compiler check. In another words, if a method doesn't exist, you cannot call it. So, it's better to provide only those operations that are needed.
Reduce the amount of integration tests. If it's possible, it's better to test business logic with simple unit tests. So, I want to replace some integration tests with unit ones without violating the quality assurance level.

Let's start our journey with refactoring Pocket and Tamagotchi to Rich Domain Model.

Don't add setters, getters, and public no-args constructor

Firstly, look at the initial approach of designing Pocket and Tamagotchi entities following Anemic Domain Model:

@Entity
@NoArgsConstructor
@Setter
@Getter
public class Pocket {
  @Id
  private UUID id;

  private String name;

  @OneToMany(mappedBy = "pocket")
  private List<Tamagotchi> tamagotchis = new ArrayList<>();
}

@Entity
@NoArgsConstructor
@Setter
@Getter
public class Tamagotchi {
  @Id
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;
}

Here I’m using UUID as a primary key. I understand that there are some performance implications for it. But now client-side generated ID is crucial for a smooth transition to the Rich Domain Model. Anyway, later I’ll give you some examples with other ID types.

I bet this looks familiar. Perhaps your current project contains lots of similar declarations. What problems are there?

No-args constructor allows incostistent object instantiation

Hibernate demands each entity to provide a no-args constructor. Otherwise, the framework doesn’t work properly. And it’s one of the edgy cases that can make your code less straight-forward and more buggy.

Thankfully, there is a solution. Hibernate doesn’t need a public constructor for an entity. Instead, it can be protected. So, we can add a public static method to instantiate the entity, and leave protected constructor for Hibernate specifically. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter
@Getter
public class Pocket {
  @Id
  private UUID id;

  private String name;

  @OneToMany(mappedBy = "pocket")
  private List<Tamagotchi> tamagotchis = new ArrayList<>();

  public static Pocket newPocket(String name) {
    Pocket pocket = new Pocket();
    pocket.setId(UUID.randomUUID());
    pocket.setName(name);
    return pocket;
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter
@Getter
public class Tamagotchi {
  @Id
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;

  public static Tamagotchi newTamagotchi(String name, Pocket pocket) {
    Tamagotchi tamagotchi = new Tamagotchi();
    tamagotchi.setId(UUID.randomUUID());
    tamagotchi.setName(name);
    tamagotchi.setPocket(pocket);
    return tamagotchi;
  }
}

As you can see, business code (that is likely to be in a different package) cannot instantiate Tamagotchi or Pocket with no-args constructor. It has to invoke dedicated methods newTamagotchi and newPocket that accept a specific amount of parameters.

Setters break encapsulation

I think public setters aren’t much different from regular public fields. Well, you could put some checks in a setter because it’s a method. But in reality, people tend not to go this way. Usually we just put @Setter annotation from Lombok library on top of class and that’s it.

I consider using setters in an entity a bad approach due to these reasons:

Possible invariants’ violation. Some fields cannot be updated. Other ones can be updated only if the entity is being transmitted to a particular state. Pure setters forces developer to put all those checks in services.
If Tamagotchi.name is String, it doesn't mean that every String value is allowed. Therefore, you also have to perform those checks outsided of an entity.
A field can be a part of implementation detail. Maybe it's forbidden to update it directly. But a public setter allows this operation.

The main point is that public setters breaks the principle of compiler validation that I mentioned previously. You just provide too many options that can be called differently.

What’s the alternative? I suggest adding changeXXX methods for specific behaviour. Also, those methods should contain validation logic and throw exception if needed.

Suppose that Tamagotchi entity has a status field that can have the value of PENDING. If Tamagotchi is PENDING, it cannot be modified. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
@Getter
public class Tamagotchi {
  @Id
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;

  @Enumerated(STRING)
  private Status status;

  public void changeName(String name) {
      if (status == PENDING) {
          throw new TamagotchiStatusException("Tamagotchi cannot be modified because it's PENDING");
      }
      if !(nameIsValid(name)) {
          throw new TamagotchiNameInvalidException("Invalid Tamagotchi name: " + name);
      }
      this.name = name;
  }

  public static Tamagotchi newTamagotchi(String name, Pocket pocket) { /* entity creation */ }
}

The Tamagotchi.changeName method guarantees that you cannot change name if certain preconditions are violated. The code that invokes the method doesn’t need to know about specific rules. You just have to deal with exceptions.

Getters break encapsulation

Well, the previous paragraph about setters is more or less obvious. There are dozens of articles and opinions on the Internet about problems with setters. Anyway, eliminating getters sounds ridiculous, isn’t it? They don’t mutate the state of an entity. So, what’s the deal?

The problem with getters is that they also allow to break encapsulation and perform unnecessary or wrong checks. Suppose that we also want to restrict updating the name of Tamagotchi if its status is ERROR. That's the possible solution you might see during the code review:

@Service
@RequiredArgsConstructor
public class TamagotchiService {
    private final TamagotchiRepository repo;

    @Transactional
    public void changeName(UUID id, String name) {
        Tamagotchi tamagotchi = repo.findById(id).orElseThrow();
        if (tamagotchi.getStatus() == ERROR) {
            throw new TamagotchiStatusException("Tamagotchi cannot be modified because its status is ERROR");
        }
        tamagotchi.changeName(name);
    }
}

Though Tamagotchi provides a dedicated method changeName, the check is still implemented in the service layer. I've noticed that even experienced senior developers tend to fall into anemic model mindset when there is a possibility. Because they've been working for years on different projects and most likely each one has applied Anemic Domain Model pattern. So, developers just choose the simpler and more obvious way.

However, a decision has some consequences. Firstly, the logic is divided between Tamagotchi entity and TamagotchiService (that’s the one thing we’ve wanted to avoid). Secondly, checks might be duplicated and you can miss it during the code review. And finally, some checks can be outdated in time. For example, this validation of ERROR status might become obsolete later. If you forget to eliminate it here, your code won’t act expectedly.

As I mentioned before, if you don’t need a method, just don’t add it. Getters aren’t required to perform business logic. You can put validations inside Tamagotchi.changeName method. If a getter is not present, it cannot be invoked and such a scenario won’t happen.

What about querying, then? Usually we use Hibernate entities to SELECT data, transform it into DTO, and return the result to the user. How can we do it without getters? Don’t worry, we’ll discuss this topic later in the article.

There is also one exception for this rule. You can add getters for ID. Sometimes it’s necessary to know the entity id in runtime. Later you’ll see an example of that.

Current result

We've already discussed three points:

No-args constructor.
Setters.
Getters.

If we remove those pieces, the code will look like this:

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Pocket {
  @Id
  private UUID id;

  private String name;

  @OneToMany(mappedBy = "pocket")
  private List<Tamagotchi> tamagotchis = new ArrayList<>();

  public static Pocket newPocket(String name) {
      Pocket pocket = new Pocket();
      pocket.setId(UUID.randomUUID());
      pocket.setName(name);
      return pocket;
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Tamagotchi {
  @Id
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;

  @Enumerated(STRING)
  private Status status;

  public void changeName(String name) {
      if (status == PENDING) {
          throw new TamagotchiStatusException("Tamagotchi cannot be modified because it's PENDING");
      }
      if (!nameIsValid(name)) {
          throw new TamagotchiNameInvalidException("Invalid Tamagotchi name: " + name);
      }
      this.name = name;
  }

  public static Tamagotchi newTamagotchi(String name, Pocket pocket) {
      Tamagotchi tamagotchi = new Tamagotchi();
      tamagotchi.setId(UUID.randomUUID());
      tamagotchi.setName(name);
      tamagotchi.setPocket(pocket);
      tamagotchi.setStatus(CREATED);
      return tamagotchi;
  }
}

Aggregate and Aggregate root

Previously I’ve mentioned the Aggregate pattern. Speaking about our domain, the Pocket entity should be the Aggregate root. However, existing API allows us to access Tamagotchi entity directly. Let’s fix that.

Firstly, let's add simple CREATE/UPDATE/DELETE operations. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Pocket {
  @Id
  private UUID id;

  private String name;

  @OneToMany(mappedBy = "pocket", cascade = PERSIST, orphanRemoval = true)
  private List<Tamagotchi> tamagotchis = new ArrayList<>();

  public UUID createTamagotchi(TamagotchiCreateRequest request) {
      Tamagotchi tamagotchi = Tamagotchi.newTamagotchi(request.name(), this);
      tamagotchis.add(tamagotchi);
      return tamagotchi.getId();
  }

  public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
      Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
      tamagotchi.changeName(request.name());
  }

  public void deleteTamagotchi(UUID tamagotchiId) {
      Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
      tamagotchis.remove(tamagotchi);
  }

  private Tamagotchi tamagotchiById(UUID tamagotchiId) {
      return tamagotchis
              .stream()
              .filter(t -> t.getId().equals(tamagotchiId))
              .findFirst()
              .orElseThrow(() -> new TamagotchiNotFoundException("Cannot find Tamagotchi by ID=" + tamagotchiId));
  }

  public static Pocket newPocket(String name) {
      Pocket pocket = new Pocket();
      pocket.setId(UUID.randomUUID());
      pocket.setName(name);
      return pocket;
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
class Tamagotchi {
  @Id
  @Getter
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;

  @Enumerated(STRING)
  private Status status;

  public void changeName(String name) {
      if (status == PENDING) {
          throw new TamagotchiStatusException("Tamagotchi cannot be modified because it's PENDING");
      }
      if (!nameIsValid(name)) {
          throw new TamagotchiNameInvalidException("Invalid Tamagotchi name: " + name);
      }
      this.name = name;
  }

  public static Tamagotchi newTamagotchi(String name, Pocket pocket) {
    Tamagotchi tamagotchi = new Tamagotchi();
    tamagotchi.setId(UUID.randomUUID());
    tamagotchi.setName(name);
    tamagotchi.setPocket(pocket);
    tamagotchi.setStatus(CREATED);
    return tamagotchi;
  }
}

There are a lot of nuances. So, I’ll point out each of them one by one. Firstly, Pocket entity provides methods createTamagotchi, updateTamagotchi, and deleteTamagotchi as-is. You don’t retrieve any information from Tamagotchi or Pocket. You just invoke the required functionality.

I’m aware that such a technique also has performance penalties. We’ll also discuss some approaches to overcome these problems later.

Then goes Tamagotchi entity. The first thing I want you to notice is that the entity is package-private. Meaning that nobody can access Tamagotchi outside of the package. Therefore, calling Pocket directly is the only way.

Now you may think that its profit isn’t so obvious. But soon we’ll discuss the evolution of aggregate and you’ll see the benefits.

Neither Pocket nor Tamagotchi entity provides regular setters or getters. One can only invoke public methods of Pocket entity.

Evolution of requirements

As I said before, entities aren't static. Requirements change and invariants as well. So, let's look through a hypothetical process of implementing new requirements and see how it goes.

Each Pocket must possess at least one Tamagotchi

It means that we should create a Tamagotchi, when a new Pocket is instantiated. Also, if you want to delete a Tamagotchi, you have to check that it’s not the single one within the Pocket. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Pocket {
  /* fields and other methods */

  public void deleteTamagotchi(UUID tamagotchiId) {
      Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
      if (tamagothis.size() == 1) {
          throw new TamagotchiDeleteException("Cannot delete Tamagotchi because it's the single one"); 
      }
      tamagotchis.remove(tamagotchi);
  }

  public static Pocket newPocket(String name) {
      Pocket pocket = new Pocket();
      pocket.setId(UUID.randomUUID());
      pocket.setName(name);
      pocket.createTamagotchi(new TamagotchiCreateRequest("Default")); // creating default tamagotchi
      return pocket;
  }
}

As you can see, invariants’ correctness is guaranteed within an aggregate. Even if you want to, you cannot create a Pocket with zero Tamagotchi or delete Tamagotchi if it’s a single one. And I think that it’s great. Code becomes less error-prone and easier to maintain.

Every Tamagotchi name has to be unique within a Pocket

To implement this requirement, we need to alter createTamagotchi and updateTamagotchi methods a bit. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Pocket {
  /* fields and other methods */

  public UUID createTamagotchi(TamagotchiCreateRequest request) {
      Tamagotchi tamagotchi = Tamagotchi.newTamagotchi(request.name(), this);
      tamagotchis.add(tamagotchi);
      validateTamagotchiNamesUniqueness();
      return tamagotchi.getId();
  }

  public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
      Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
      tamagotchi.changeName(request.name());
      validateTamagotchiNamesUniqueness();
  }

  private void validateTamagotchiNamesUniqueness() {
      Set<String> names = new HashSet<>();
      for (Tamagotchi tamagotchi : tamagotchis) {
          if (!names.add(tamagotchi.getName()) {
              throw new TamagotchiNameInvalidException("Tamagotchi name is not unique: " + tamagotchi.getName());
          }
      }
  }
}

You’ve probably noticed that I added a getter for Tamagotchi.name field. Because Pocket and Tamagotchi form a single aggregate, it’s fine to provide getters. Because Pocket might need this information. However, Tamagotchi should not request anything from Pocket. It’s also better to mark this getter as package-private. So, no one can access it outside of the package.

I understand that validateTamagotchiNamesUniqueness doesn't perform well. Don't worry, we'll discuss workarounds later in the Performance implications section.

Once again, the domain model guarantees that each Tamagotchi name is unique within a Pocket. What is interesting is that API hasn’t changed a bit. The code that invokes those public methods (likely domain services) doesn’t have to change logic.

If a user deletes a Tamagotchi, they can restore it by name

This one is tricky and involves soft deletion. It also has additional points:

If Tamagotchi with the same name already exists, a user cannot restore the one they've deleted.
If a user deletes multiple Tamagotchis with the same name, they can only restore the last one.

I'm not a fan of soft deletion that involves adding isDeleted column by many reasons. Instead, I will introduce a new entity DeletedTamagotchi that contains the state of deleted Tamagotchi. Look at the code example below.

@Entity
@NoArgsConstructor(access = PROTECTED)
@Getter
class DeletedTamagotchi {
  @Id
  private UUID id;

  private String name;

  @ManyToOne(fetch = LAZY)
  @JoinColumn(name = "pocket_id")
  private Pocket pocket;

  @Enumerated(STRING)
  private Status status;

  public static DeletedTamagotchi newDeletedTamagotchi(Tamagotchi tamagotchi) {
      DeletedTamagotchi deletedTamagotchi = new DeletedTamagotchi();
      deletedTamagotchi.setId(UUID.randomUUID());
      deletedTamagotchi.setName(tamagotchi.getName());
      deletedTamagotchi.setPocket(tamagotchi.getPocket());
      deletedTamagotchi.setStatus(tamagotchi.getStatus());
      return deletedTamagotchi;
  }
}

Tamagotchi entity is rather simple, so DeletedTamagotchi contains the same fields. However, if the original entity were more complicated, it couldn’t be the case. For example, you could save the state of Tamagotchi in Map<String, Object> fields that transforms to JSONB in the database.

Also, DeletedTamagotchi entity is package-private like Tamagotchi. So, the presence of this entity is an implementation detail. The other parts of the code don’t need to know this and interact with DeletedTamagotchi directly. Instead, it’s better to provide a single method Pocket.restoreTamagotchi without additional details.

Now let's alter Pocket entity to the new requirements. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
public class Pocket {
  /* fields and other methods */

  @OneToMany(mappedBy = "pocket", cascade = PERSIST, orphanRemoval = true)
  private List<DeletedTamagotchi> deletedTamagotchis = new ArrayList<>();

  public void deleteTamagotchi(UUID tamagotchiId) {
      Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
      if (tamagothis.size() == 1) {
          throw new TamagotchiDeleteException("Cannot delete Tamagotchi because it's the single one"); 
      }
      tamagotchis.remove(tamagotchi);
      addDeletedTamagotchi(tamagotchi);
  }

  private void addDeletedTamagotchi(Tamagotchi tamagotchi) {
      Iterator<DeletedTamagotchi> iterator =  deletedTamagotchis.iterator();
      // if Tamagotchi with the same has been deleted,
      // remove information about it
      while (iterator.hasNext()) {
          DeletedTamagotchi deletedTamagotchi = iterator.next();
          if (deletedTamagotchi.getName().equals(tamagotchi.getName()) {
              iterator.remove();
              break;
          }
      }
      deletedTamagotchis.add(
          newDeletedTamagotchi(tamagotchi)
      );
  }

  public UUID restoreTamagotchi(String name) {
      DeletedTamagotchi deletedTamagotchi = deletedTamagotchiByName(name);
      return createTamagotchi(new TamagotchiCreateRequest(deletedTamagotchi.getName()));
  }
}

The deleteTamagotchi method also creates or replaces a DeletedTamagotchi record. Meaning that every other code that calls the method for whatever reason doesn't violate the new requirement about soft deletion because it's been implemented internally.

To perform the required business operation, you should just invoke Pocket.restoreTamagotchi. We hid all the complex details behind the scenes. What’s even better is that DeletedTamagotchi is not a part of public API. Meaning that it can be easily modified or even removed, if it’s not needed anymore.

As you can see, placing business logic within an aggregate has significant benefits. However, it's not the end of the story. There are still some concerns we need to deal with. And the next one is querying data.

Querying data

When we deal with Hibernate, usually we use public getters to transform entity into DTO and return it to the user. However, only Pocket entity is public now, and it doesn’t provide any getters (aside from Pocket.getId()). How do we perform queries in this case? I can suggest several approaches.

Manual queries

The obvious solution is just writing regular JPQL or SQL statements. Hibernate uses reflection and doesn’t demand public getters for fields. This may work if you start a project from scratch. But if you already relying on getters to retrieve information from the entity and put it into DTO, then transition might be overwhelming. That’s why we have a second option.

Introducing toDto method

An entity can provide toDto or similar method that returns its internal representation as a separate data structure. It’s similar to Memento design pattern. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
public class Pocket {
  /* other fields and methods */

  public PocketDto toDto() {
    return new PocketDto(
        id,
        name,
        tamagotchis.stream()
            .map(Tamagotchi::toDto)
            .toList()
    );
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
@Getter
class Tamagotchi {
  /* other fields and methods */

  public TamagotchiDto toDto() {
    return new TamagotchiDto(id, name, status);
  }
}

The returned DTO is an immutable object that couldn’t affect entities’ state. Besides, the approach is also helpful for unit testing. Let’s move on to this part.

Unit testing entities

We're going to test these scenarios:

Pocket must always possess at least one Tamagotchi.
If you delete Tamagotchi, you can restore it by name.
If you delete multiple Tamagotchi with the same name, you can only restore the last one.

The entire test suite is available by this link.

Pocket must always possess at least one Tamagotchi

Look at the unit tests below.

class PocketTest {

  @Test
  void shouldCreatePocketWithTamagotchi() {
    Pocket pocket = Pocket.newPocket("My pocket");

    PocketDto dto = pocket.toDto();

    assertEquals(1, dto.tamagotchis().size());
  }

  @Test
  void shouldForbidDeletionOfASingleTamagotchi() {
    Pocket pocket = Pocket.newPocket("My pocket");
    PocketDto dto = pocket.toDto();
    UUID tamagotchiId = dto.tamagotchis().get(0).id();

    assertThrows(
        TamagotchiDeleteException.class,
        () -> pocket.deleteTamagotchi(tamagotchiId)
    );
  }
}

The first one checks that Pocket is being created with a single Tamagotchi. Whilst the second one validates that you cannot delete Tamagotchi if it’s a single one.

What I like about those tests is that they are unit ones. No database, no Testcontainers, just regular JUnit and we've validated business logic successfully. Cool! Let's move forward.

If you delete Tamagotchi, you can restore it by name

This one is a bit more complicated. Look at the code example below.

class PocketTest {

  @Test
  void shouldDeleteTamagotchiById() {
    Pocket pocket = Pocket.newPocket("My pocket");
    UUID tamagotchiId = pocket.createTamagotchi(new TamagotchiCreateRequest("My tamagotchi", CREATED));

    pocket.deleteTamagotchi(tamagotchiId);

    PocketDto dto = pocket.toDto();
    assertThat(dto.tamagotchis())
        .noneMatch(t -> t.name().equals("My tamagotchi"));
  }

  @Test
  void shouldRestoreTamagotchiById() {
    Pocket pocket = Pocket.newPocket("My pocket");
    UUID tamagotchiId = pocket.createTamagotchi(new TamagotchiCreateRequest("My tamagotchi", CREATED));
    pocket.deleteTamagotchi(tamagotchiId);

    pocket.restoreTamagotchi("My tamagotchi");

    PocketDto dto = pocket.toDto();
    assertThat(dto.tamagotchis())
        .anyMatch(t -> t.name().equals("My tamagotchi"));
  }
}

The shouldDeleteTamagotchiById checks that deletion works as expected. The other one validates restoreTamagotchi method behaviour.

If you delete multiple Tamagotchi with the same name, you can only restore the last one

This one is the most challenging. Look at the code example below.

class PocketTest {

  @Test
  void shouldRestoreTheLastTamagotchi() {
    Pocket pocket = Pocket.newPocket("My pocket");
    UUID tamagotchiId = pocket.createTamagotchi(new TamagotchiCreateRequest("My tamagotchi", CREATED));
    pocket.deleteTamagotchi(tamagotchiId);
    tamagotchiId = pocket.createTamagotchi(new TamagotchiCreateRequest("My tamagotchi", PENDING));
    pocket.deleteTamagotchi(tamagotchiId);

    pocket.restoreTamagotchi("My tamagotchi");

    PocketDto dto = pocket.toDto();
    assertThat(dto.tamagotchis())
        .anyMatch(t ->
            t.name().equals("My tamagotchi")
            && t.status().equals(PENDING)
        );
  }
}

Here we do these steps:

Create Pocket.
Create Tamagotchi with name of My tamagotchi and status CREATED.
Delete Tamagotchi.
Create Tamagotchi with name of My tamagotchi and status PENDING.
Restore Tamagotchi by name My tamagotchi.
Verify that the last one Tamagotchi has been restored (with status of PENDING).

Here are tests' run result:

Rich Domain Model pattern allows us to test complex business scenarios with simple unit tests. I think it’s outstanding. However, integration tests are also important because we need to store data in DB not in RAM. Let’s discuss this part of the equation.

Integration testing

We use entities with a conjunction of repositories (Spring Data ones usually). Let’s write some use cases and test them:

Create Pocket.
Create Tamagotchi.
Update Tamagotchi.

The entire test suite is available by this link.

Create Pocket

Look at the service example below:

@Service
@RequiredArgsConstructor
public class PocketService {

  private final EntityManager em;

  @Transactional
  public UUID createPocket(String name) {
    Pocket pocket = Pocket.newPocket(name);
    em.persist(pocket);
    return pocket.getId();
  }
}

Time to write some integration tests. Look at the code snippet below:

@Testcontainers
@DataJpaTest
@AutoConfigureTestDatabase(replace = NONE)
@Transactional(propagation = NOT_SUPPORTED)
@Import(PocketService.class)
class PocketServiceIntegrationTest {

  @Container
  @ServiceConnection
  public static final PostgreSQLContainer<?> POSTGRES = new PostgreSQLContainer<>("postgres:13");

  @Autowired
  private TransactionTemplate transactionTemplate;
  @Autowired
  private TestEntityManager em;
  @Autowired
  private PocketService pocketService;

  @BeforeEach
  void cleanDatabase() {
    // there is cascade constraint in the database deleting tamagotchis and deleted_tamagotchis
    transactionTemplate.executeWithoutResult(
        s -> em.getEntityManager().createQuery("DELETE FROM Pocket ").executeUpdate()
    );
  }

  @Test
  void shouldCreateNewPocket() {
    UUID pocketId = pocketService.createPocket("New pocket");

    PocketDto dto = transactionTemplate.execute(
        s -> em.find(Pocket.class, pocketId).toDto()
    );
    assertEquals("New pocket", dto.name());
  }
}

I use Testcontainers library to start PosgtreSQL in Docker. Flyway migration tool creates tables before tests run.

You can check out the migrations by this link.

I guess this snippet is not that complicated. So, let's go next.

Create Tamagotchi

Look at the code service implementation below:

@Service
@RequiredArgsConstructor
public class PocketService {
  /* other fields and methods */

  @Transactional
  public UUID createTamagotchi(UUID pocketId, TamagotchiCreateRequest request) {
    Pocket pocket = em.find(Pocket.class, pocketId);
    return pocket.createTamagotchi(request);
  }
}

As you can see, the Rich Domain Model pattern demands to declare services as thin layer that are easy to understand and test. And here is the test itself:

/* same Java annotations */
class PocketServiceIntegrationTest {
  /* initialization... */  

  @Test
  void shouldCreateTamagotchi() {
    UUID pocketId = pocketService.createPocket("New pocket");

    UUID tamagotchiId = pocketService.createTamagotchi(
        pocketId,
        new TamagotchiCreateRequest("my tamagotchi", CREATED)
    );

    PocketDto dto = transactionTemplate.execute(
        s -> em.find(Pocket.class, pocketId).toDto()
    );
    assertThat(dto.tamagotchis())
        .anyMatch(t ->
            t.name().equals("my tamagotchi")
                && t.status().equals(CREATED)
                && t.id().equals(tamagotchiId)
        );
  }
}

This one is a bit more interesting. Firstly, we create a Pocket and then add a Tamogotchi inside it. Assertions checks that expected Tamagotchi is present in result DTO.

Update Tamagotchi

This one is the most intriguing. Check out the implementation below:

@Service
@RequiredArgsConstructor
public class PocketService {
  /* other fields and methods */

  @Transactional
  public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
    UUID pocketId = em.createQuery(
            "SELECT t.pocket.id AS id FROM Tamagotchi t WHERE t.id = :tamagotchiId",
            UUID.class
        )
        .setParameter("tamagotchiId", tamagotchiId)
        .getSingleResult();

    Pocket pocket = em.find(Pocket.class, pocketId);
    pocket.updateTamagotchi(tamagotchiId, request);
  }
}

API demands to pass tamagotchiId. But the domain model allows us to update Tamagotchi only through Pocket because the latter is the aggregate root. So, we determine pocketId with additional query to DB and then select Pocket aggregate by its id. Test is also quite interesting:

/* same Java annotations */
class PocketServiceIntegrationTest {
  /* other fields and methods */

  @Test
  void shouldUpdateTamagotchi() {
    UUID pocketId = pocketService.createPocket("New pocket");
    UUID tamagotchiId = pocketService.createTamagotchi(
        pocketId,
        new TamagotchiCreateRequest("my tamagotchi", CREATED)
    );

    pocketService.updateTamagotchi(
        tamagotchiId,
        new TamagotchiUpdateRequest("another tamagotchi", PENDING)
    );

    PocketDto dto = transactionTemplate.execute(
        s -> em.find(Pocket.class, pocketId).toDto()
    );
    assertThat(dto.tamagotchis())
        .anyMatch(t ->
            t.name().equals("another tamagotchi")
                && t.status().equals(PENDING)
                && t.id().equals(tamagotchiId)
        );
  }
}

The steps are:

Create Pocket.
Create Tamagotchi.
Update Tamagotchi.
Validate the result DTO.

Here is the execution result for all integration tests:

Nothing complicated, don't you think?

Performance implications

Rich Domain Model brings overhead for sure. However, there are
some workarounds to reach compromise.

Optimization of queries

Firstly, let's have a look at PocketService.updateTamagotchi method again:

@Service
@RequiredArgsConstructor
public class PocketService {
  /* other fields and methods */

  @Transactional
  public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
    UUID pocketId = em.createQuery(
            "SELECT t.pocket.id AS id FROM Tamagotchi t WHERE t.id = :tamagotchiId",
            UUID.class
        )
        .setParameter("tamagotchiId", tamagotchiId)
        .getSingleResult();

    Pocket pocket = em.find(Pocket.class, pocketId);
    pocket.updateTamagotchi(tamagotchiId, request);
  }
}

The problem is that we retrieve all existing Tamagotchi instances for a specified Pocket when we actually want to update a single one. Look at the log below:

select t1_0.pocket_id from tamagotchi t1_0 where t1_0.id=?

select p1_0.id,p1_0.name from pocket p1_0 where p1_0.id=?

select t1_0.pocket_id,t1_0.id,t1_0.name,t1_0.status
from tamagotchi t1_0 where t1_0.pocket_id=?

We can change queries to restrict the transmission of unnecessary data. Look at the code example below:

@Service
@RequiredArgsConstructor
public class PocketService {
  /* other fields and methods */

  @Transactional
  public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
    Pocket pocket = em.createQuery(
            """
                SELECT p FROM Pocket p
                LEFT JOIN FETCH p.tamagotchis t
                WHERE t.id = :tamagotchiId
                """,
            Pocket.class
        ).setParameter("tamagotchiId", tamagotchiId)
        .getSingleResult();
    pocket.updateTamagotchi(tamagotchiId, request);
  }
}

Instead of selecting all existing Tamagotchi instances for the specified Pocket, we retrieve Pocket and the only associated Tamagotchi instance by specified id. Log also looks differently:

select 
p1_0.id,
p1_0.name,
t1_0.pocket_id,
t1_0.id,
t1_0.name,
t1_0.status
from pocket p1_0
left join tamagotchi t1_0 on p1_0.id=t1_0.pocket_id
where t1_0.id=?

Even if Pocket contains thousands of Tamagotchi, it won’t affect the performance of the application. Because it will retrieve only a single one. If you run test cases from the previous paragraph, they will also pass successfully.

Pinpointing optimized checks

Nevertheless, the previous technique has limitations. To understand this, let's write another test. As we've already discussed, the business rule demands that each Tamagotchi must have a unique name within Pocket. Let's test this behaviour. Look at the code snippet below:

@Test
void shouldUpdateTamagotchiIfThereAreMultipleOnes() {
    UUID pocketId = pocketService.createPocket("New pocket");
    UUID tamagotchiId = pocketService.createTamagotchi(
        pocketId,
        new TamagotchiCreateRequest("Cat", CREATED)
    );
    pocketService.createTamagotchi(
        pocketId,
        new TamagotchiCreateRequest("Dog", CREATED)
    );

    assertThrows(
        TamagotchiNameInvalidException.class,
        () -> pocketService.updateTamagotchi(tamagotchiId, new TamagotchiUpdateRequest("Dog", CREATED))
    );
}

There are two Tamagotchi with names of Cat and Dog. We try to rename Cat to Dog. Here, we expect to get TamagotchiNameInvalidException. Because business rule should validate this scenario. But if you run the test, you’ll get this result:

Expected com.example.demo.domain.exception.TamagotchiNameInvalidException to be thrown, but nothing was thrown.

Why is that? Look again at Pocket.updateTamagotchi method declaration:

public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
    Tamagotchi tamagotchi = tamagotchiById(tamagotchiId);
    tamagotchi.changeName(request.name());
    tamagotchi.changeStatus(request.status());
    validateTamagotchiNamesUniqueness();
}

private void validateTamagotchiNamesUniqueness() {
    Set<String> names = new HashSet<>();
    for (Tamagotchi tamagotchi : tamagotchis) {
      if (!names.add(tamagotchi.getName())) {
        throw new TamagotchiNameInvalidException(
            "Tamagotchi name is not unique: " + tamagotchi.getName());
      }
    }
}

As you can see, Pocket aggregate expects to have access for all Tamagotchi to validate the business rule. But we changed the query to select only a single Tamagotchi (the one we want to update). That’s why the exception is not raised. Because there is always a single Tamagotchi on the list and we cannot violate the uniqueness.

I see people trying to remove such validations from an aggregate. But I think you shouldn't do that. Instead, it's better to perform another optimized check in the service level in advance. To understand this approach, look at the schema below:

Aggregate should always be valid. You can’t predict all likely future outcomes. Maybe you’ll call Pocket in another scenario. So, if you drop a check from an aggregate completely, you may accidentally violate business rule.

Nevertheless, we live in a real world where performance matters. It’s much better to execute a single exists SQL statement then retrieve all Tamagotchi instances from the database. So, you put optimized check specifically where it’s needed. But you also leave the aggregate untouched.

Look at the final code snippet of PocketService.updateTamagotchi method:

@Transactional
public void updateTamagotchi(UUID tamagotchiId, TamagotchiUpdateRequest request) {
    boolean nameIsNotUnique = em.createQuery(
            """
                SELECT COUNT(t) > 0 FROM Tamagotchi t
                WHERE t.id <> :tamagotchiId AND t.name = :newName
                """,
            boolean.class
        ).setParameter("tamagotchiId", tamagotchiId)
        .setParameter("newName", request.name())
        .getSingleResult();

    if (nameIsNotUnique) {
      throw new TamagotchiNameInvalidException("Tamagotchi name is not unique: " + request.name());
    }

    UUID pocketId = em.createQuery(
            "SELECT t.pocket.id AS id FROM Tamagotchi t WHERE t.id = :tamagotchiId",
            UUID.class
        )
        .setParameter("tamagotchiId", tamagotchiId)
        .getSingleResult();

    Pocket pocket = em.find(Pocket.class, pocketId);
    pocket.updateTamagotchi(tamagotchiId, request);
}

Firstly, we check that any other Tamagotchi (aside from the one we want to update) already has the same name. If that’s true, we throw an exception. If you run the previous test again and check log, you’ll see that only COUNT query has been invoked:

select count(t1_0.id)>0
from tamagotchi t1_0
where t1_0.id!=? and t1_0.name=?

Anyway, I don't recommend you to overuse this approach. You should treat it like an accurately pinned patch. In other words, put it only where it's needed. Otherwise, I'd prefer relying on domain logic and leave code in services as simple as possible.

Database generated ID

Previously I’ve mentioned that I’ll show you examples of client generated ID. However, sometimes we want to use other ID types. For example, sequence generated ones. Is this Rich Domain Model pattern also applicable to these ID types? It is, but there are also some concerns.

Firstly, have a look at Pocket and Tamagotchi entities using IDENTITY generation strategy:

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
public class Pocket {
  @Id
  @GeneratedValue(strategy = IDENTITY)
  private Long id;

  /* other fields aren't important */

  public Long createTamagotchi(TamagotchiCreateRequest request) {
    Tamagotchi newTamagotchi = Tamagotchi.newTamagotchi(request.name(), request.status(), this);
    tamagotchis.add(newTamagotchi);
    validateTamagotchiNamesUniqueness();
    // always returns null
    return newTamagotchi.getId();
  }

  /* other methods aren't important */

  public static Pocket newPocket(String name) {
    Pocket pocket = new Pocket();
    pocket.setName(name);
    pocket.createTamagotchi(new TamagotchiCreateRequest("Default", CREATED));
    return pocket;
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
@Getter
class Tamagotchi {
  @Id
  @GeneratedValue(strategy = IDENTITY)
  private Long id;

  /* other fields and methods aren't important */

  public static Tamagotchi newTamagotchi(String name, Status status, Pocket pocket) {
    Tamagotchi tamagotchi = new Tamagotchi();
    tamagotchi.setName(name);
    tamagotchi.setPocket(pocket);
    tamagotchi.setStatus(status);
    return tamagotchi;
  }
}

As you can see, we don’t assign ID directly anymore. Instead, we leave the field with null value and let Hibernate fill it later. Unfortunately, this decision breaks the logic of Pocket.createTamagotchi method. We do not set ID during the creation of Tamagotchi object. So, the invocation of Tamagotchi.getId always returns null (until you flush changes to the database).

There are several ways to fix this issue.

Manually fill the id

You can eliminate @GeneratedValue annotation usage and pass the ID value directly in the constructor. In this case, you have to invoke SELECT nextval('mysequence') statement and pass its result to an entity. Look at the code example below:

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
public class Pocket {
  @Id
  private Long id;

  /* other fields aren't important */

  public Long createTamagotchi(long tamagotchiId, TamagotchiCreateRequest request) {
    Tamagotchi newTamagotchi = Tamagotchi.newTamagotchi(tamagotchiId, request.name(), request.status(), this);
    tamagotchis.add(newTamagotchi);
    validateTamagotchiNamesUniqueness();
    // always returns null
    return newTamagotchi.getId();
  }

  /* other methods aren't important */

  public static Pocket newPocket(long id, String name) {
    Pocket pocket = new Pocket();
    pocket.setId(id);
    pocket.setName(name);
    pocket.createTamagotchi(new TamagotchiCreateRequest("Default", CREATED));
    return pocket;
  }
}

@Entity
@NoArgsConstructor(access = PROTECTED)
@Setter(PRIVATE)
@Getter
class Tamagotchi {
  @Id
  private Long id;

  /* other fields and methods aren't important */

  public static Tamagotchi newTamagotchi(long id, String name, Status status, Pocket pocket) {
    Tamagotchi tamagotchi = new Tamagotchi();
    tamagotchi.setId(id);
    tamagotchi.setName(name);
    tamagotchi.setPocket(pocket);
    tamagotchi.setStatus(status);
    return tamagotchi;
  }
}

The advantage is that your entity classes do not depend on some Hibernate magic and you can still validate business cases with regular unit tests. But you also make your code more verbose. Because you to have pass IDs manually.

Anyway, this approach is worth considering.

I found this option in this article. Actually, the author demands to stop use Hibernate at all. Even though I like Hibernate, I found some arguments intriguing.

Introducing business key

Sometimes passing IDs manually is nearly impossible. Maybe it requires too much refactoring that is unbearable. Or maybe your application works with MySQL which doesn't support sequences but only auto increment columns.

Though you can emulate sequences in MySQL by creating a regular table, this approach is not well performant.

In this case, you can introduce business key. That is a separate value that can identify an entity uniquely. Though it doesn’t mean that the business key must be globally unique. For example, if you point to Tamagotchi by name and it’s only unique within a Pocket, then you can identify Tamagotchi by a combination of (pocket_business_key, tamagothic_name).

Nevertheless, each business key should be unmodifiable. Otherwise, you can compromise the identity of your entities. So, pay good attention to this point.

Also, a good example of a business key is a slug. Look at the URL of this article. Do you see that it contains its name and some hash value? That is the slug. It assigns only once when the article is created but never changes (even if I change the article’s name). So, if your entities don’t have an obvious candidate for business key, introducing a slug might be an option.

Is Rich Domain Model always worth it?

There is no ultimate decision in software development. Every approach is just a compromise. Rich Domain Model pattern is no exception.

I started my article by explaining the problems of the Anemic Domain Model to you. They all valid and make sense. But it doesn’t mean that the Rich Domain model has no disadvantages. I can think of these:

If you work with Hibernate, then the Rich Domain Model pattern is not so popular. It’s just the reality. There are dozens of articles on the Internet with Hibernate examples and total absence of the Rich Domain Model. People got used to the Anemic Domain Model and you have to take it into account.
Rich Domain Model pattern may also bring some performance penalties. Some of them can be easily fixed. But others might become a headache. If your application is supposed to be high loaded, you have to make sure that invariants’ check won’t slow the response time too much.
Rich Domain Model usage often leads to god object entities. Of course, it makes maintenance harder. There are ways to fix that. For example, Vaughn Vernon wrote 3 articles about effective aggregate design. However, if your entity is already a god object, it’s will be tough to refactor it.

Conclusion

In the end, I can say that I think that the Rich Domain Model acts better than the Anemic one. But don’t apply it blindly. You should also consider possible consequences and make decisions wisely.

Thank you very much for reading this long piece. I hope you've learnt something new. If you found it interesting, please share it with your friends and colleagues, press the like button, and leave your comments down below. I'll be glad to hear your opinions and discuss questions. Have a nice day!

Resources

Integration Tests for N + 1 problem in Java

Semyon Kirekov — Sun, 21 May 2023 10:56:08 +0000

N + 1 problem is a common issue in many enterprise projects. The worst is that you don't notice it until the amount of data becomes huge. Unfortunately, the code might reach the stage when dealing with N + 1 problem becomes an unbearable task.

In this article, I'm telling you:

How to track N + 1 problem automatically?
How to write a test to check that the query count does not exceed the expected value?

The tech stack consists of Java, Spring Boot, Spring Data JPA, and PostgreSQL. You can check out the repository with code examples by this link.

There are no restrictions to apply Spring Boot or Hibernate specifically. If you interact with javax.sql.DataSource in your codebase, then the solution will help you. Even if you don't use Spring at all.

An example of the N + 1 problem

Supposing we are working on the application that manages zoos. In that case, there are two core entities: Zoo and Animal. Look at the code snippet below:

@Entity
@Table(name = "zoo")
public class Zoo {
    @Id
    @GeneratedValue(strategy = IDENTITY)
    private Long id;

    private String name;

    @OneToMany(mappedBy = "zoo", cascade = PERSIST)
    private List<Animal> animals = new ArrayList<>();
}

@Entity
@Table(name = "animal")
public class Animal {
    @Id
    @GeneratedValue(strategy = IDENTITY)
    private Long id;

    @ManyToOne(fetch = LAZY)
    @JoinColumn(name = "zoo_id")
    private Zoo zoo;

    private String name;
}

Now we want to retrieve all existing zoos with their animals. Look at the code ZooService code below.

@Service
@RequiredArgsConstructor
public class ZooService {
    private final ZooRepository zooRepository;

    @Transactional(readOnly = true)
    public List<ZooResponse> findAllZoos() {
        final var zoos = zooRepository.findAll();
        return zoos.stream()
                   .map(ZooResponse::new)
                   .toList();
    }
}

Also, we want to check that everything works smoothly. So, here is a simple integration test:

@DataJpaTest
@AutoConfigureTestDatabase(replace = NONE)
@Transactional(propagation = NOT_SUPPORTED)
@Testcontainers
@Import(ZooService.class)
class ZooServiceTest {
    @Container
    static final PostgreSQLContainer<?> POSTGRES = new PostgreSQLContainer<>("postgres:13");

    @DynamicPropertySource
    static void setProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", POSTGRES::getJdbcUrl);
        registry.add("spring.datasource.username", POSTGRES::getUsername);
        registry.add("spring.datasource.password", POSTGRES::getPassword);
    }

    @Autowired
    private ZooService zooService;
    @Autowired
    private ZooRepository zooRepository;

    @Test
    void shouldReturnAllZoos() {
        /* data initialization... */
        zooRepository.saveAll(List.of(zoo1, zoo2));

        final var allZoos = assertQueryCount(
            () -> zooService.findAllZoos(),
            ofSelects(1)
        );

        /* assertions... */
        assertThat(
            ...
        );
    }
}

I skipped the data initialization and assertions parts for the sake of simplicity. They are not important for the article's topic. Anyway, you can check out the whole test suite by this link.

I have a particular piece about testing data layer in Spring Boot application with Testcontainers. If you're unfamiliar with the topic, you should definitely look through it.

The test passes successfully. However, if you log SQL statements, you'll notice something that may concern you. Look at the output below:

-- selecting all zoos
select z1_0.id,z1_0.name from zoo z1_0
-- selecting animals for the first zoo
select a1_0.zoo_id,a1_0.id,a1_0.name from animal a1_0 where a1_0.zoo_id=?
-- selecting animals for the second zoo
select a1_0.zoo_id,a1_0.id,a1_0.name from animal a1_0 where a1_0.zoo_id=?

As you can see, we have a separate select query for each present Zoo. The total number of queries equals the number of selected zoos + 1. Therefore, this is the N + 1 problem.

This may cause crucial performance penalties. Especially on a large scale of data.

Tracking the N + 1 problem automatically

Of course, you can run tests, look through the logs, and count queries by yourself to determine viable performance issues. Anyway, this is both tedious and inefficient. Thankfully, there is a better approach.

There is a cool library called datasource-proxy. It provides a convenient API to wrap javax.sql.DataSource interface with a proxy containing specific logic. For example, we can register callbacks invoked before and after query execution. What's interesting is that the library also contains out-of-the-box solution to count executed queries. We're going to alter it a bit to serve our needs.

Query Count Service

Firstly, add the library to the dependencies:

implementation "net.ttddyy:datasource-proxy:1.8"

Now create the QueryCountService. It’s the singleton that holds the current count of executed queries and allows you to clean it. Look at the code snippet below.

@UtilityClass
public class QueryCountService {
    static final SingleQueryCountHolder QUERY_COUNT_HOLDER = new SingleQueryCountHolder();

    public static void clear() {
        final var map = QUERY_COUNT_HOLDER.getQueryCountMap();
        map.putIfAbsent(keyName(map), new QueryCount());
    }

    public static QueryCount get() {
        final var map = QUERY_COUNT_HOLDER.getQueryCountMap();
        return ofNullable(map.get(keyName(map))).orElseThrow();
    }

    private static String keyName(Map<String, QueryCount> map) {
        if (map.size() == 1) {
            return map.entrySet()
                       .stream()
                       .findFirst()
                       .orElseThrow()
                       .getKey();
        }
        throw new IllegalArgumentException("Query counts map should consists of one key: " + map);
    }
}

In that case, we make an assumption there is a single DataSource in our application. That’s why the keyName function throws an exception otherwise. However, the code won’t differ much with multiple data sources usage.

The SingleQueryCountHolder stores all QueryCount objects in a regular ConcurrentHashMap.

On the contrary, ThreadQueryCountHolder stores the values in ThreadLocal object. But SingleQueryCountHolder is enough for our case.

The API provides two methods. The get method returns the current amount of executed queries whilst the clear one sets the count to zero.

BeanPostProccessor and DataSource proxy

Now we need to register the QueryCountService to make it collect the data from the DataSource. In that case, the BeanPostProcessor interface comes in handy. Look at the code example below.

@TestComponent
public class DatasourceProxyBeanPostProcessor implements BeanPostProcessor {
    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) {
        if (bean instanceof DataSource dataSource) {
            return ProxyDataSourceBuilder.create(dataSource)
                       .countQuery(QUERY_COUNT_HOLDER)
                       .build();
        }
        return bean;
    }
}

I mark the class with @TestComponent annotation and put it to src/test directory because I don't need to count queries outside of the test scope.

As you can see, the idea is trivial. If a bean is DataSource, then wrap it with ProxyDataSourceBuilder and put the QUERY_COUNT_HOLDER value as the QueryCountStrategy.

Finally, we want to assert the amount of executed queries for the specific method. Look at the code snippet with custom assertions below:

Custom assertions

@UtilityClass
public class QueryCountAssertions {
    @SneakyThrows
    public static <T> T assertQueryCount(Supplier<T> supplier, Expectation expectation) {
        QueryCountService.clear();
        final var result = supplier.get();
        final var queryCount = QueryCountService.get();
        assertAll(
            () -> {
                if (expectation.selects >= 0) {
                    assertEquals(expectation.selects, queryCount.getSelect(), "Unexpected selects count");
                }
            },
            () -> {
                if (expectation.inserts >= 0) {
                    assertEquals(expectation.inserts, queryCount.getInsert(), "Unexpected inserts count");
                }
            },
            () -> {
                if (expectation.deletes >= 0) {
                    assertEquals(expectation.deletes, queryCount.getDelete(), "Unexpected deletes count");
                }
            },
            () -> {
                if (expectation.updates >= 0) {
                    assertEquals(expectation.updates, queryCount.getUpdate(), "Unexpected updates count");
                }
            }
        );
        return result;
    }
}

The algorithm is straightforward:

Set the current queries count to zero.
Executed the provided lambda.
Assert the query count to the given Expectation object.
If everything passes successfully, return the result of execution.

Also, you’ve noticed an additional condition. If the provided type of count is less than zero, skip the assertion. It’s convenient, when you don’t care about other queries count.

The Expectation class is just a regular data structure. Look at its declaration below:

@With
@AllArgsConstructor
@NoArgsConstructor
public static class Expectation {
    private int selects = -1;
    private int inserts = -1;
    private int deletes = -1;
    private int updates = -1;

    public static Expectation ofSelects(int selects) {
        return new Expectation().withSelects(selects);
    }

    public static Expectation ofInserts(int inserts) {
        return new Expectation().withInserts(inserts);
    }

    public static Expectation ofDeletes(int deletes) {
        return new Expectation().withDeletes(deletes);
    }

    public static Expectation ofUpdates(int updates) {
        return new Expectation().withUpdates(updates);
    }
}

The final example

Let’s see how it works. At first, I add query assertions in the previous case with N + 1 problem. Look at the code block below:

final var allZoos = assertQueryCount(
    () -> zooService.findAllZoos(),
    ofSelects(1)
);

Don't forget to import DatasourceProxyBeanPostProcessor as a Spring bean in your tests.

If we rerun the test, we'll get the output below.

Multiple Failures (1 failure)
    org.opentest4j.AssertionFailedError: Unexpected selects count ==> expected: <1> but was: <3>
Expected :1
Actual   :3

So, the assertion does work. We managed to track the N + 1 problem automatically. Time to replace the regular selection with JOIN FETCH. Look at the code snippet below.

public interface ZooRepository extends JpaRepository<Zoo, Long> {
    @Query("FROM Zoo z LEFT JOIN FETCH z.animals")
    List<Zoo> findAllWithAnimalsJoined();
}

@Service
@RequiredArgsConstructor
public class ZooService {
    private final ZooRepository zooRepository;

    @Transactional(readOnly = true)
    public List<ZooResponse> findAllZoos() {
        final var zoos = zooRepository.findAllWithAnimalsJoined();
        return zoos.stream()
                   .map(ZooResponse::new)
                   .toList();
    }
}

Let's run the test again and check out the result:

Meaning that the assertion tracks N + 1 problems correctly. Besides, it passes successfully, if the amount of queries equals to the expected one. Great!

Conclusion

As a matter of fact, it is possible to prevent N + 1 problems with regular tests. I think that’s a great opportunity to put guards for those code parts that are crucial to performance perspective.

That’s all I wanted to tell you about dealing with N + 1 problem in the automatic way. If you have any questions or suggestions, leave your comments down below. Also, if you like this piece, share it with your friends and colleagues. Perhaps they’ll find it beneficial too. Thanks for reading!

Resources

JUnit 5: link tests with task tracker issues

Semyon Kirekov — Fri, 21 Apr 2023 18:42:15 +0000

In this guide, I'm telling you:

How you can link JUnit 5 tests with issues in your task tracker systems?
How to generate documentation based on it automatically?
How to host the result documentation on GitHub Pages?

You can find the entire repository with code examples on GitHub by this link. The generated documentation also available here.

Issue Annotation

There is a cool library called JUnit Pioneer. It's an extension pack that includes some features that vanilla JUnit lacks. These are cartesian product tests, JSON argument parameterized source, retrying tests and many others. But I'm particularly interested in Issue annotation. Look at the code example below:

class TestExample {
    @Test
    @Issue("HHH-16417")
    void testSum() {
        ...
    }

    @Test
    @Issue("HHH-10000")
    void testSub() {
        ...
    }
}

I put actual task IDs from Hibernate task tracker to make result documentation more concise.

As you can see, we can add the @Issue annotation with task ID that is associated with the test. So, every time you notice a test failure, you know what you have broken.

Setting up service loaders

JUnit Pioneer provides an API that allows to get information about tests that marked with @Issue annotation. Meaning that we can combine the information in the HTML report and share it with other team members. For example, QA engineers might find it beneficial. Because now they are aware what tests do the project contains, and what bugs do they check.

There is a special interface IssueProcessor. Its implementation acts like a callback. Look at the code snippet below:

public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        ...
    }
}

However, we also to need to set up SimpleIssueProcessor as Java Service Loader. Otherwise, JUnit runner won’t register it. Create a new file with a name of org.junitpioneer.jupiter.IssueProcessor in src/test/resources/META-INF/services directory. It has to contain one row with the fully qualified name of the implementation (in our case, SimpleIssueProcessor). Look at the code snippet below:

org.example.SimpleIssueProcessor

Besides, there is another service loader to register. It’s the one provided by JUnit Pioneer library that does the complex logic of parsing information and delegating control to IssueProcessor implementation. Create a new file with a name of org.junit.platform.launcher.TestExecutionListener in the same directory. Look at the required file content below:

org.junitpioneer.jupiter.issue.IssueExtensionExecutionListener

Now we’re ready. You can put println statement in your IssueProcessor implementation to check that the framework invokes it after tests’ execution.

Creating meta-information JSON file

The documentation generation process consists of two steps:

Generate documentation in JSON format (because it's easy to parse).
Put the information into HTML template.

Look at the SimpleIssueProcessor code below:

public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    @SneakyThrows
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        writeFileToBuildFolder(
            "test-issues-info.json",
            new ObjectMapper().writeValueAsString(
                issueTestSuites.stream()
                    .map(issueTestSuite -> Map.of(
                        "issueId", issueTestSuite.issueId(),
                        "tests", issueTestSuite.tests()
                                     .stream()
                                     .map(test -> parseTestId(test.testId()))
                                     .toList()
                    ))
                    .toList()
            )
        );
    }
    ...
}

The writeToBuildFolder method creates a file by path build/classes/java/test/test-issues-info.json. I use Gradle, but if you prefer Maven, your path will differ a bit. You can check out the source code of the function by this link.

The result JSON is an array. Look at the generated example below:

[
  {
    "tests": [
      {
        "testId": "TestExample.testSum",
        "urlPath": "org/example/TestExample.java#L12"
      }
    ],
    "issueId": "HHH-16417"
  },
  {
    "tests": [
      {
        "testId": "TestExample.testSub",
        "urlPath": "org/example/TestExample.java#L18"
      }
    ],
    "issueId": "HHH-10000"
  }
]

There is an issue ID and set of tests that reference to it (theoretically, there might be several tests pointing the same issue).

Now we need to parse the required information from the supplied List<IssueTestSuite>. Look at the parseTestId function below.

@SneakyThrows
private static Map<String, Object> parseTestId(String testId) {
    final var split = testId.split("/");
    final var className = split[1].substring(7, split[1].length() - 1);
    final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
    final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

    final var classPool = ClassPool.getDefault();
    classPool.appendClassPath(new ClassClassPath(clazz));
    final var methodLineNumber = classPool.get(className)
                                     .getDeclaredMethod(method)
                                     .getMethodInfo()
                                     .getLineNumber(0);
    return Map.of(
        "testId", lastArrayElement(className.split("\\.")) + "." + method,
        "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
    );
}

Let's deconstruct this code snippet step by step.

The library puts testId as the string pattern below:

// [engine:junit-jupiter]/[class:org.example.TestExample]/[method:testSum()]

Firstly, we get fully qualified class name and method name. Look at the code below:

final var split = testId.split("/");
// [class:org.example.TestExample] => org.example.TestExample
final var className = split[1].substring(7, split[1].length() - 1);
// [method:testSum()] => testSum
final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");

Afterwards, we determine the line number of the test method. It’s useful to set links that point to the particular line of code. Look at the snippet below:

// Load test class
final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

final var classPool = ClassPool.getDefault();
classPool.appendClassPath(new ClassClassPath(clazz));

final var methodLineNumber = classPool.get(className)
                                 .getDeclaredMethod(method)
                                 .getMethodInfo()
                                 .getLineNumber(0);

ClassPool comes from Javaassist library. It gives convenient API to retrieve the line number of Java method.

Here we perform these steps:

Get the Class instance of the test suite.
Initialize ClassPool.
Append a test class to the pool
Get the line number of the test method.

And finally, we put together the information chunks into java.util.Map that we eventually convert to JSON. Look at the last piece of code below:

return Map.of(
    // TestExample.testSum
    "testId", lastArrayElement(className.split("\\.")) + "." + method,
    // org/example/TestExample.java#L11
    "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
);

The testId property is just a combination of a simple class name and test method name. Whilst urlPath is part of the link on GitHub pointing to the specific line where we declared the test.

Generating documentation

Finally, it’s time to compose the generated JSON into a nicely laid out HTML page. Look at the entire snippet below. Then I’m explaining each part to you.

const fs = require('fs');

function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="https://hibernate.atlassian.net/browse/${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `
}

console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(fs.readFileSync('./build/classes/java/test/test-issues-info.json', 'utf8')))}
    </body>
    </html>
`)

I'm using Javascript and NodeJS runtime environment.

The renderIssues function does the entire job. Let's deconstruct it step by step.

function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  ...
}

The issuesInfo is an array that we generated previously with IssueProcessor. Therefore, each element has issueId and tests belonging to it.

As long as each issue id has a format of MMM-123 we can sort them by number. In that case, we get issues sorted in descending order.

Look at the remaining portion of the function below:

const issueBaseUrl = "https://hibernate.atlassian.net/browse/";
const repoBaseUrl = "https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/"
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="${issueBaseUrl}${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="${repoBaseUrl}${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `

Each present combination of issue and test transforms into a table row. Also, those snippets aren’t just plain text but links. You can open issue description and test declaration by clicking on it. Cool, isn’t it?

Then comes the output. Look at the final part of the script below:

console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(
    fs.readFileSync('./build/classes/java/test/test-issues-info.json',
        'utf8')))}
    </body>
    </html>
`)

I write the output to console because later I redirect it to file.

The style sheet is called Tacit CSS. This is a set of CSS rules applied automatically. If you need to format an HTML page but don’t want to deal with complex layout, that’s a perfect solution.

The idea is to put the generated HTML table into a predefined template.

Setting up GitHub Pages

The documentation is no use until you can examine it. So, let's host it on GitHub Pages. Look at the pipeline below:

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-and-deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew build
      - name: Set up NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 16
      - name: Run docs generator
        run: ./docsGeneratorScript.sh
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: public/
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1

The steps are:

Set up JDK 17
Build the project
Set up NodeJS
Generate documentation with the previously shown JS program
Deploy the result to GitHub Pages

The docsGeneratorScript.sh file is a trivial bash script. Look at its definition below:

mkdir -p public
touch ./public/index.html
node ./generateDocs.js > ./public/index.html

And that’s it! Now the documentation is available and being updated automatically each time somebody merges a pull request.

Conclusion

That’s all I wanted to tell you about linking tests with issues and generating documentation for it. If you have questions or suggestions, leave your comments down below. Thanks for reading!

Resources

Spring, SchemaSpy DB docs, and GitHub Pages

Semyon Kirekov — Sun, 12 Mar 2023 08:21:30 +0000

In this article, I'm telling you:

How to generate docs of database schema structure with SchemaSpy? Why you need it, if you use a relational database (PostgreSQL/Oracle/MySQL etc.) in your application?
How to run SchemaSpy with Testcontainers?
How to host the generated database documentation on GitHub Pages?

The stack I'm using in this project consists of:

You can find the entire repository with code examples by this link. The test that generates the SchemaSpy docs is available by this link.

What is SchemaSpy?

SchemaSpy is a standalone application that connects to the database, scans its tables and schemas, and generates nicely composed HTML documentation. You can check out an example sample by this link but I'm showing you just one screenshot to clarify my point.

SchemaSpy visualizes the database structure detailedly. Besides, you can also deep dive into relations, constraints, and table/column comments (i.e. COMMENT ON COLUMN/TABLE statements).

Such documentation is helpful for a variety of specialists:

System analysts want to understand the data processing and storage principles behind the business logic.
When QA engineers face a bug, the database structure documentation helps to investigate the reason that cause the problem deeply. As a matter of fact, they can also attach additional details to the ticket to make its fixing more transparent.
Data engineers have to be aware of tables structure to deal with Change Data Capture events correctly.

So, database documentation that is always relevant (because it's generated) is excessively beneficent.

Database tables

Look at the DDL operations that create database tables. This is our application domain we're going to document.

CREATE TABLE users
(
    id   BIGSERIAL PRIMARY KEY,
    name VARCHAR(200) NOT NULL
);

CREATE TABLE community
(
    id   BIGSERIAL PRIMARY KEY,
    name VARCHAR(200) NOT NULL
);

CREATE TABLE post
(
    id           BIGSERIAL PRIMARY KEY,
    name         VARCHAR(200)                     NOT NULL,
    community_id BIGINT REFERENCES community (id) NOT NULL
);

CREATE TABLE community_role
(
    id           BIGSERIAL PRIMARY KEY,
    user_id      BIGINT REFERENCES users (id)     NOT NULL,
    community_id BIGINT REFERENCES community (id) NOT NULL,
    type         VARCHAR(50)                      NOT NULL,
    UNIQUE (user_id, community_id, type)
);

CREATE TABLE post_role
(
    id      BIGSERIAL PRIMARY KEY,
    user_id BIGINT REFERENCES users (id) NOT NULL,
    post_id BIGINT REFERENCES post (id)  NOT NULL,
    type    VARCHAR(50)                  NOT NULL,
    UNIQUE (user_id, post_id, type)
);

As you can see, there are 3 core tables (users, community, and role) and 2 linking tables (community_role, post_role).

Running SchemaSpy with Testcontainers

The easiest way to keep documentation in sync with the current database structure is updating it on each merged Pull Request. Therefore, we need somehow to run SchemaSpy during tests execution.

I'm showing you the algorithm step by step. But you can check out the entire suite by this link.

Firstly, we need to define the SchemaSpy container itself. Look at the code snippet below.

class SchemaSpyTest extends AbstractControllerTest {

  @Test
  @SneakyThrows
  void schemaSpy() {
    @Cleanup final var schemaSpy =
        new GenericContainer<>(DockerImageName.parse("schemaspy/schemaspy:6.1.0"))
            .withNetworkAliases("schemaspy")
            .withNetwork(NETWORK)
            .withLogConsumer(new Slf4jLogConsumer(LoggerFactory.getLogger("SchemaSpy")))
            .withCreateContainerCmdModifier(cmd -> cmd.withEntrypoint(""))
            .withCommand("sleep 500000");
    ...
}

The AbstractControllerTest contains PostgreSQL container configuration. You can see its source code by this link.

The Cleanup annotation comes from the Lombok project. It generates try-finally statement.

I want to point out some important details here.

The withNetwork clause assigns the container to the existing Testcontainers NETWORK. This value inherits from the AbstractControllerTest and the PostgreSQL runs with this network as well. It’s crucial because otherwise SchemaSpy won’t be able to connect to PostgreSQL.
Log consumer applies a logger to push logs from the container. It’s useful to track bugs and errors.
The withCreateContainerCmdModifier is the primary part. By default, SchemaSpy container tries to connect to a database immediately and generate documentation. Then a container terminates. However, that’s not an acceptable behaviour, because the generation result remains inside container. Therefore, we need to copy it in the OS directory. But if a container has already stopped, it’s impossible. So, we have to override a default entry-point to make a container run (almost) indefinitely. That’s why I put the sleep 500000 command. Container will hang and do nothing on its start.

Now we need to trigger the generation process. Look at the code block below.

schemaSpy.start();
schemaSpy.execInContainer(
    "java",
    "-jar", "/schemaspy-6.1.0.jar",
    "-t", "pgsql11",
    "-db", POSTGRES.getDatabaseName(),
    "-host", "postgres",
    "-u", POSTGRES.getUsername(),
    "-p", POSTGRES.getPassword(),
    "-o", "/output",
    "-dp", "/drivers_inc",
    "-debug"
);
schemaSpy.execInContainer("tar", "-czvf", "/output/output.tar.gz", "/output");

Here is what happens:

I start the container (remember that it hangs and does nothing).
Then I execute the command that generates the documentation. The host equals to postgres because that’s the PostgreSQL container’s network alias (the withNetworkAliases method). The command execution happens in a separate process inside the container. So, the sleep command is not terminated.
Finally, we put the directory with generated contents (HTML, CSS, JS) into a tarball. Testcontainers library allows to copy files from a container to the OS but not directories. That’s why we need an archive inside the SchemaSpy container.

It’s time to copy the result documentation in the OS directory and unpack the changes. Look at the code snippet below.

final var buildFolderPath =
    Path.of(getClass().getResource("/").toURI()).toAbsolutePath();

schemaSpy.copyFileFromContainer(
    "/output/output.tar.gz",
    buildFolderPath.resolve("output.tar.gz").toString()
);
schemaSpy.stop();

final var archiver = ArchiverFactory.createArchiver("tar", "gz");
archiver.extract(
    buildFolderPath.resolve("output.tar.gz").toFile(),
    buildFolderPath.toFile()
);

The steps are:

Define the buildFolderPath that points to the build/classes/java/test directory (I use Gradle in this project).
Then I copy the tarball with documentation from the container to the buildFolderPath directory.
And finally, I unpack the archive contents to the same directory (I use jararchivelib library here).

In the end, we have a pretty documentation generated on the database schema structure. Look at the screenshot below.

Hosting result on GitHub Pages

We got generated documentation based in build/classes/java/test folder. Anyway, it's not that useful yet. We need to host it on GitHub Pages and update accordingly.

Look at the pipeline definition below.

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-and-deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew build
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: build/classes/java/test/
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1

The actions are:

Build the project and run tests with Set up JDK 17 and Build with Gradle steps.
Then the Upload artifact step uploads the build/classes/java/test/ directory folder to GitHub registry (it contains the documentation).
And finally, deploy the artefact created on the previous step to the GitHub Pages.

That's basically it. Now the SchemaSpy documentation is available on GitHub Pages and being updated automatically on each merged Pull Request.

Conclusion

That’s all I wanted to tell you about generating database schema documentation and hosting the result on GitHub Pages. Do you have any docs generation automations in your project? Is it SchemaSpy or something else? Do you host the result on GitHub or GitLab Pages? Tell your story in the comments. It’ll be interesting to discuss your experience as well.

Thanks for reading! If you like that piece, press the like button and share the link with your friends.

Resources

Spring Security — Generate Docs for Authorization Rules

Semyon Kirekov — Wed, 08 Mar 2023 12:15:19 +0000

That's a small addition to my previous post "Spring Security and Non-flat Roles Inheritance Architecture". In this article, I'm telling you:

How to generate documentation for Spring Security authorization rules directly from code?
How to host the result HTML page on GitHub Pages?

Such documentation is useful for a variety of specialists. System and business analysts want to understand the logic behind request processing. While quality engineers check that endpoints validate the access as described in the task. Both of these categories will benefit from the documentation that is always relevant.

You can find the whole generator's code by this link. Look at the example of generated documentation below.

HTTP method	API path	Security checking rule	Controller function name
POST	/api/community	isAuthenticated()	createCommunity
POST	/api/community/{communityId}/post	@RoleService.hasAnyRoleByCommunityId(#communityId, @CommunityRole.ADMIN)	createPost
PUT	/api/post/{postId}	@RoleService.hasAnyRoleByPostId(#postId, @PostRole.EDITOR)	updatePost
GET	/api/post/{postId}	@RoleService.hasAnyRoleByPostId(#postId, @PostRole.VIEWER)	getPost

I put the layout as Markdown table to make it easier to read. Anyway, you can check out the rendered HTML by this link.

The algorithm

Here is the whole idea of documentation generation:

There is a separate DocsTest that starts the whole Spring application.
While running the test generates the result HTML page into build/classes/test directory.
Finally, during the GitHub pipeline execution, we host the output HTML page on GitHub Pages.

Generation steps

Take a look at the base setup below.

class DocsTest extends AbstractControllerTest {
    @Autowired
    private ApplicationContext context;

    ...
}

The AbstractControllerTest starts PostgreSQL with Testcontainers. You can find its source code by this link.

I use the ApplicationContext bean to resolve registered REST controllers.

What information do we need to parse from the annotations put on a REST controller? Here is the list:

The name of the controller
Details about each endpoint
1. HTTP method
2. API path
3. The security SpEL expression parsed from @PreAuthorize annotation.
4. The name of the Java method that maps the HTTP request.

Look at the Java records that hold the stated points:

@With
private record ControllerInfo(
    String name,
    List<MethodInfo> methods
) {}

@With
private record MethodInfo(
    String httpMethod,
    String apiPath,
    String security,
    String functionName
) {}

Now it's time for traversing the existing controllers and parsing the required data. Look at the code snippet below:

@Test
void generateDocs() throws Exception {
    final var controllers = new ArrayList<ControllerInfo>();
    for (String controllerName : context.getBeanNamesForAnnotation(RestController.class)) {
        final var controllerBean = context.getBean(controllerName);
        final var baseApiPath = getApiPath(AnnotationUtils.findAnnotation(controllerBean.getClass(), RequestMapping.class));
        final var controllerSecurityInfo = new ControllerInfo(
            StringUtils.capitalize(controllerName),
            new ArrayList<>()
        );
        for (Method method : controllerBean.getClass().getMethods()) {
            getMethodInfo(method)
                .map(m -> m.withPrefixedApiPath(baseApiPath))
                .ifPresent(m -> controllerSecurityInfo.methods().add(m));
        }
        controllers.add(controllerSecurityInfo);
    }
    ...
}

Here is what happens step by step:

I retrieve all bean names that marked with @RestController annotation.
Then I get the current controller bean by its name.
Afterwards, I parse the base API path.
And finally, I traverse each method inside the controller and parse information about it.

Look at the getMethodInfo declaration below.

private static Optional<MethodInfo> getMethodInfo(Method method) {
        return Optional.<Annotation>ofNullable(AnnotationUtils.findAnnotation(method, GetMapping.class))
                   .or(() -> ofNullable(AnnotationUtils.findAnnotation(method, PostMapping.class)))
                   .or(() -> ofNullable(AnnotationUtils.findAnnotation(method, DeleteMapping.class)))
                   .or(() -> ofNullable(AnnotationUtils.findAnnotation(method, PutMapping.class)))
                   .map(annotation -> AnnotationUtils.getAnnotationAttributes(method, annotation))
                   .map(attributes -> new MethodInfo(
                       attributes.annotationType()
                           .getSimpleName()
                           .replace("Mapping", "")
                           .toUpperCase(),
                       getApiPath(attributes.getStringArray("value")),
                       ofNullable(AnnotationUtils.findAnnotation(method, PreAuthorize.class))
                           .map(PreAuthorize::value)
                           .orElse(""),
                       method.getName()
                   ));
    }

In that case, I'm trying to obtain possible request mapping annotations from the method: GetMapping, PostMapping, DeleteMapping, or PutMapping. Then I get the annotation's attributes by calling AnnotationUtils.getAnnotationAttributes, and finally pass the parameters to the MethodInfo constructor.

The getApiPath method accepts String... parameter and returns its first value if it's present.

Creating HTML report

Now that we have the information about endpoints, it’s time to format it as the HTML page. Look at the template declaration below:

final var html = """
    <html>
    <head>
        <meta charset="UTF8">
        <style>
            body, table {
                font-family: "JetBrains Mono";
                font-size: 20px;
            }
            table, th, td {
              border: 1px solid black;
            }
        </style>
        <link href='https://fonts.googleapis.com/css?family=JetBrains Mono' rel='stylesheet'>
    </head>
    <body>
        <div>
            <h2>Endpoints role checking</h2>
            <div>{docs}</div>
        </div>
    </body>
    </html>
    """.replace("{docs}", toHtml(controllers));

writeFileToBuildFolder("index.html", html);

The controllers variable represents the List<ControllerInfo> that we built previously. The function toHtml transforms it into an HTML snippet. Then we replace the placeholder of {docs} with the content.

The writeFileToBuildFolder function writes the result content into file build/classes/java/test/index.html. You can find its declaration by this link.

Look at the toHtml function definition below.

private static String toHtml(List<ControllerInfo> controllers) {
    StringBuilder docs = new StringBuilder();
    for (ControllerInfo controller : controllers) {
        docs.append("<b>")
            .append(controller.name())
            .append("</b>")
            .append("<br>")
            .append("<table>");

        for (MethodInfo method : controller.methods()) {
            docs.append("<tr>")
                .append("<td>").append(method.httpMethod()).append("</td>")
                .append("<td>").append(method.apiPath()).append("</td>")
                .append("<td>").append(method.security()).append("</td>")
                .append("<td>").append(method.functionName()).append("</td>")
                .append("</tr>");
        }
        docs.append("</table>")
            .append("----------------------------------<br>");
        }
    return docs.toString();
}

As you can see, I just create an HTML table for each existing controller and concatenate them into a single string.

Hosting the documentation on GitHub Pages

The whole GitHub Actions pipeline is less than 40 rows. Look at the YAML below.

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-and-deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew build
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: build/classes/java/test/
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1

Here is what happens:

The Set up JDK 17 and Build with Gradle performs a regular Gradle build operation.
Then comes the Upload artifact that saves the directory containing the HTML documentation to the GitHub registry.
Finally, we deploy the previously stored artifact to the GitHub Pages.

And that’s basically it. You can check out the generated HTML page by this link. The coolest thing is that you don’t have to write documentation manually. Therefore, it’s always relevant because you generate the content directly from your code.

Conclusion

That’s all I wanted to tell you about documenting Spring Security applications and storing the HTML result on GitHub Pages. Do you generate any docs in your projects? If so, what kind of documentation it is? Tell your story in the comments. Thanks for reading!

Resources

Spring Security and Non-flat Roles Inheritance Architecture

Semyon Kirekov — Thu, 23 Feb 2023 17:15:54 +0000

Business requirements and domain model
Roles, enums, and inheritance
Unit testing roles inheritance
Defining JPA entities
Creating custom Authentication implementation
1. Why does getAuthorities() return empty set?
2. UserId, and volatile authenticated flag
Creating custom AuthenticationProvider
Defining Spring Security config
Declaring REST API methods
Creating custom role checking service
Combining PreAuthorize and custom role checking service
Short and elegant enum references in SpEL expressions
Integration testing and validating security

Then it comes to authorization, roles always come into play. Flat ones are straightforward. User possess a set of privileges that you simply check for containing the required element. But what if a user can have different roles for specific entities? For example, I can be an editor in the particular social media post but only a viewer in another one. Also, there might be inheritance rules. If an admin grant me the editor role, I automatically become a viewer. But if I'm an editor, I'm not becoming an admin. What's the approach to handle the scenario to remain code clean and maintainable? Don't worry, I'm giving your answers.

In this article, I'm telling you:

What's the best approach to handle roles inheritance in Java?
How to test the stated hierarchy?
How to apply the solution in Spring Security and Spring Data JPA?

You can find the code examples and the entire project setup in this repository.

Business requirements and domain model

Supposing we’re developing a simple social media platform. Look at the diagram below. Here I described the business entities we’re working with in this article.

There are 3 core entities:

User is the one who reads existing posts and creates new ones.
Community is the feed to submit new posts.
Post is an individual part of media. Some users can view it, edit it, and delete it.

Also, we have role a model which is a bit more complicated than the plain privileges assignment. For example, a user can owe an EDITOR role for the particular post but has only a VIEWER for another one. Or user may be an ADMIN for the ‘Cats and dogs’ community but just a MODERATOR for the ‘Motorcycles and Heavy metal’ one.

The roles provide such privileges:

CommunityRole.ADMIN gives ultimate access to the community and the containing posts.
CommunityRole.MODERATOR provides an ability to add new posts and remove old ones.
PostRole.EDITOR allows to edit the content of the particular post.
PostRole.REPORTER gives the credit to send reports about inappropriate attitude in the comments.
PostRole.VIEWER grants an access to view the post and leave comments.

However, business also wants the inheritance model. For example, if I have the root role, it means I also can do actions that any child provides. Look at the schema below to understand the approach.

Suppose that I'm a MODERATOR in some community. That means, I'm EDITOR, REPORTER, and VIEWER as well for any other post in the community. On the contrary, if somebody granted me with the REPORTER role for the post, it doesn't mean I have the rights to edit it (I need the EDITOR for that) or add new posts (the MODERATOR role provides access for it).

That's a convenient approach. You don't have to check the set of many roles for every possible operation. You just have to validate the presence of the lowest authority. If I'm an ADMIN in the community, then I'm also a VIEWER for any post there. So, due to the inheritance model, you have to check the VIEWER and that's it.

Anyway, it doesn't seem like an simple task to implement it in the code. Besides, the VIEWER model has two parents: the REPORTER and the EDITOR. The Java doesn't allow multiple inheritance. Meaning that we need a special approach.

Roles, enums, and inheritance

Roles are the perfect candidates for the enum. Look at the code snippet below.

public enum CommunityRoleType {
    ADMIN, MODERATOR
}

public enum PostRoleType {
    VIEWER, EDITOR, REPORTER
}

How we can build an inheritance model with these simple enums? At first, we need an interface to bind the roles together. Look at the declaration below.

public interface Role {
    boolean includes(Role role);
}

The Role interface will be the root of any CommunityRoleType and PostRoleType value as well. The includes method tells us whether the supplied role equals to the current one or contains it in its children.

Look at the modified PostRoleType code below.

public enum PostRoleType implements Role {
    VIEWER, EDITOR, REPORTER;

    private final Set<Role> children = new HashSet<>();

    static {
        REPORTER.children.add(VIEWER);
        EDITOR.children.add(VIEWER);
    }

    @Override
    public boolean includes(Role role) {
        return this.equals(role) || children.stream().anyMatch(r -> r.includes(role));
    }
}

We store the children of the particular role in the regular Java HashSet as private final field. What intriguing is how these children appear. By default, the set is empty for every enum value. But the static initializer block comes into play. You can treat it as the two-phase constructor. On this block, we just need to assign proper children to the required parents. The includes method is also rather simple. If the passed role equals to the current one, return true. Otherwise, perform the check recursively for every present child node.

An approach for the CommunityRoleType is similar. Look at the code example below.

public enum CommunityRoleType implements Role {
    ADMIN, MODERATOR;

    private final Set<Role> children = new HashSet<>();

    static {
        ADMIN.children.add(MODERATOR);
        MODERATOR.children.addAll(List.of(PostRoleType.EDITOR, PostRoleType.REPORTER));
    }

    @Override
    public boolean includes(Role role) {
        return this.equals(role) || children.stream().anyMatch(r -> r.includes(role));
    }
}

As you can see, the MODERATOR role has two children: the PostRoleType.EDITOR and the PostRoleType.REPORTER. Due to the fact that both CommunityRoleType and the PostRoleType share the same interface, they can all be part of the one inheritance hierarchy.

A slight detail remains. We need to know the root of the hierarchy to perform the access validation. The easiest way is just declaring a static method that returns the required nodes. Look at the updated Role interface definition below.

public interface Role {
    boolean includes(Role role);

    static Set<Role> roots() {
        return Set.of(CommunityRoleType.ADMIN);
    }
}

I return the Set<Role> instead of Role because theoretically there might be several roots. So, it's unnecessary to restrict the roots count to 1 on the method signature.

Some of you may ask why not just using Spring Security Role Hierarchy component? After all, it's an out-of-box solution. It suites well for the plain role model which is not the case in our context. I'll reference this point again later in the article.

Unit testing roles inheritance

Let's test our role hierarchy. Firstly, we need to check that there are no cycles which can cause StackOverflowError. Look at the test below.

@Test
void shouldNotThrowStackOverflowException() {
    final var roots = Role.roots();
    final var existingRoles = Stream.concat(
        stream(PostRoleType.values()),
        stream(CommunityRoleType.values())
    ).toList();

    assertDoesNotThrow(
        () -> {
            for (Role root : roots) {
                for (var roleToCheck : existingRoles) {
                    root.includes(roleToCheck);
                }
            }
        }
    );
}

The idea is checking all roots and all existing roles for includes combinations. None of them should throw any exception.

Next move is validating the inheritance. Here are the test cases:

The CommunityRoleType.ADMIN should include any other role and the CommunityRoleType.ADMIN itself.
The CommunityRoleType.MODERATOR should include the PostRoleType.EDITOR, PostRoleType.REPORTER, PostRoleType.VIEWER and the CommunityRoleType.MODERATOR
The PostRoleType.VIEWER should not include the PostRoleType.REPORTER.
The CommunityRoleType.MODERATOR should not include the CommunityRoleType.ADMIN.

Look at the code example below to see the described test suites.

@ParameterizedTest
@MethodSource("provideArgs")
void shouldIncludeOrNotTheGivenRoles(Role root, Set<Role> rolesToCheck, boolean shouldInclude) {
    for (Role role : rolesToCheck) {
        assertEquals(
            shouldInclude,
            root.includes(role)
        );
    }
}

private static Stream<Arguments> provideArgs() {
    return Stream.of(
        arguments(
            CommunityRoleType.ADMIN,
            Stream.concat(
                stream(PostRoleType.values()),
                stream(CommunityRoleType.values())
            ).collect(Collectors.toSet()),
            true
        ),
        arguments(
            CommunityRoleType.MODERATOR,
            Set.of(PostRoleType.EDITOR, PostRoleType.VIEWER, PostRoleType.REPORTER, CommunityRoleType.MODERATOR),
            true
        ),
        arguments(
            PostRoleType.VIEWER,
            Set.of(PostRoleType.REPORTER),
            false
        ),
        arguments(
            CommunityRoleType.MODERATOR,
            Set.of(CommunityRoleType.ADMIN),
            false
        )
    );
}

There are much more cases to cover. But I omit them for brevity.

Defining JPA entities

You can find all the JPA entities declarations here and the corresponding Flyway migrations here.

Anyway, I'm showing you the core artefacts of the system. Look at the PostRole and the CommunityRole JPA entities' declaration below.

@Entity
@Table(name = "community_role")
public class CommunityRole {
    @Id
    @GeneratedValue(strategy = IDENTITY)
    private Long id;

    @ManyToOne(fetch = LAZY)
    @JoinColumn(name = "user_id")
    private User user;

    @ManyToOne(fetch = LAZY)
    @JoinColumn(name = "community_id")
    private Community community;

    @Enumerated(STRING)
    private CommunityRoleType type;
}

@Entity
@Table(name = "post_role")
public class PostRole {
    @Id
    @GeneratedValue(strategy = IDENTITY)
    private Long id;

    @ManyToOne(fetch = LAZY)
    @JoinColumn(name = "user_id")
    private User user;

    @ManyToOne(fetch = LAZY)
    @JoinColumn(name = "post_id")
    private Post post;

    @Enumerated(STRING)
    private PostRoleType type;
}

As I've already pointed out, the CommunityRole binds to the User and the particular Community whilst the PostRole binds to the User as well and the specific Post. Therefore, the role model structure is not flat. It brings us some complexities with Spring Security. But don't worry, I'll show you how to nail them.

Because of the vertical role model the Spring Security Role Hierarchy is not going to work. We need a more complicated approach. So, let’s move forward.

Look at the required SQL migrations' set (I'm using PostgreSQL) below.

CREATE TABLE community_role
(
    id           BIGSERIAL PRIMARY KEY,
    user_id      BIGINT REFERENCES users (id)     NOT NULL,
    community_id BIGINT REFERENCES community (id) NOT NULL,
    type         VARCHAR(50)                      NOT NULL,
    UNIQUE (user_id, community_id, type)
);

CREATE TABLE post_role
(
    id      BIGSERIAL PRIMARY KEY,
    user_id BIGINT REFERENCES users (id) NOT NULL,
    post_id BIGINT REFERENCES post (id)  NOT NULL,
    type    VARCHAR(50)                  NOT NULL,
    UNIQUE (user_id, post_id, type)
);

Creating custom Authentication implementation

To begin with, we need to create custom Authentication interface implementation. Look at the code snippet below.

@RequiredArgsConstructor
public class PlainAuthentication implements Authentication {
    private final Long userId;
    private volatile boolean authenticated = true;

    @Override
    public Collection<? extends GrantedAuthority> getAuthorities() {
        return emptySet();
    }

    @Override
    public Long getPrincipal() {
        return userId;
    }

    @Override
    public String getName() {
        return "";
    }

    @Override
    public boolean isAuthenticated() {
        return authenticated;
    }

    @Override
    public void setAuthenticated(boolean isAuthenticated) throws IllegalArgumentException {
        authenticated = isAuthenticated;
    }

    @Override
    public Object getCredentials() {
        return "";
    }

    @Override
    public Object getDetails() {
        return null;
    }
}

Why does getAuthorities() return empty set?

The first thing to notice is that the getAuthorities() method always returns empty collection. Why is that? The Spring Security role model is flat, but we deal with a vertical one. If you want to put the community and the post roles belonging to the user, it may look like this:

CommunityRole_ADMIN_234
PostRole_VIEWER_896

There are three parts:

The type of the role
The role value
The id of the community or the post that the role references to.

In such setup, the role checking mechanism also become cumbersome. Look at the @PreAuthorize annotation usage example below.

@PreAuthorize("hasAuthority('PostRole_VIEWER_' + #postId)")
@GetMapping("/api/post/{postId}")
public PostResponse getPost(@PathVariable Long postId) { ... }

In my view, this code ~~is awful~~ smells. Not only there are string typings that may cause hard tracking bugs, we lost the inheritance idea. Do you remember that the CommunityRole.ADMIN also includes the PostRole.VIEWER? But here we check only the particular authority (Spring Security just calls the Collection.contains method). Meaning that the Authentication.getAuthorities() method has to contain all the children's roles as well. Therefore, you have to perform these steps:

Select all the community roles and the post roles as well that the user possess
Loop through each role down to its children and put each occurred node to another collection
Eliminate the duplicates (or just use the HashSet) and return the result as the user authorities.

Not to mention the code complexity, there are also performance drawbacks. Every request comes with selecting all the user's roles from the database. But what if the user is an admin of the community? It's unnecessary to query post roles because they already have an ultimate access. But you have to do it to make the @PreAuthorize annotation work as expected.

I thing it's clear now why I return empty collection of authorities. Later I'm explaining how to deal with that properly.

UserId, and volatile authenticated flag

Look at the PlainAuthentication below. I leave only the getPrincipal, and isAuthenticated/setAuthenticated methods to discuss.

@RequiredArgsConstructor
public class PlainAuthentication implements Authentication {
    private final Long userId;
    private volatile boolean authenticated = true;

    @Override
    public Long getPrincipal() {
        return userId;
    }

    @Override
    public boolean isAuthenticated() {
        return authenticated;
    }

    @Override
    public void setAuthenticated(boolean isAuthenticated) throws IllegalArgumentException {
        authenticated = isAuthenticated;
    }

    ...
}

The userId points to the database row containing information about the user. We’ll use it later to retrieve the roles. The isAuthenticated/setAuthenticated methods are part of the Spring Security contract. So, we have to implement them properly. I put volatile marker because the PlainAuthentication object is mutable and multiple threads can access it. Better safe than sorry.

The getPrincipal method inherits from the Authentication interface and returns Object. However, Java allows to return the subclass of the signature definition, if you extend the base class or implement the interface. Therefore, make your code more secure and maintainable.

The Authentication interface dictates to implement 3 other methods I haven't spoken about. Those are getName, getCredentials, and getDetails. We use none of them later. Therefore, it's OK to return default values.

Creating custom AuthenticationProvider

In the beginning, we should provide custom AuthenticationProvider to resolve the PlainAuthentication declared previously from the user input. Look at the code snippet below.

@Component
@RequiredArgsConstructor
class DbAuthenticationProvider implements AuthenticationProvider {
    private final UserRepository userRepository;

    @Override
    public Authentication authenticate(Authentication authentication) throws AuthenticationException {
        final var password = authentication.getCredentials().toString();
        if (!"password".equals(password)) {
            throw new AuthenticationServiceException("Invalid username or password");
        }
        return userRepository.findByName(authentication.getName())
                   .map(user -> new PlainAuthentication(user.getId()))
                   .orElseThrow(() -> new AuthenticationServiceException("Invalid username or password"));
    }

    @Override
    public boolean supports(Class<?> authentication) {
        return UsernamePasswordAuthenticationToken.class.equals(authentication);
    }
}

Of course, it's not a real production implementation. For the sake of simplicity, all users have the same password. If the user is present by the provided name, return its id as a PlainAuthentication wrapper. Otherwise, throw AuthenticationServiceException that will transform to the 401 status code afterwards.

Defining Spring Security config

Finally, time to add the Spring Security config. Here I’m using basic access authentication. Anyway, the role checking patterns I’m describing later remain the same for different authentication mechanisms. Look at the code example below.

@Configuration
@EnableWebSecurity
@EnableMethodSecurity
@RequiredArgsConstructor
public class SecurityConfig {
    @Bean
    @SneakyThrows
    public SecurityFilterChain securityFilterChain(HttpSecurity http) {
        return http
                   .csrf().disable()
                   .cors().disable()
                   .authorizeHttpRequests(customizer -> customizer.anyRequest().authenticated())

                   .httpBasic()
                   .authenticationEntryPoint((request, response, authException) -> response.sendError(401))
                   .and()

                   .build();
    }
}

Declaring REST API methods

The described domain can provide lots of possible operations. Though I’m going to list just 4 of them. That is sufficient to cover the whole role checking idea of mine. Those are:

Create new community.
Create post for the given community.
Update the post by id.
Get post by id.

Look at the code snippet below.

@RestController
@RequestMapping("/api")
public class Controller {
    @PostMapping("/community")
    @PreAuthorize("isAuthenticated()")
    public CommunityResponse createCommunity(@RequestParam String name) { ... }

    @PostMapping("/community/{communityId}/post")
    // Must have CommunityRoleType.MODERATOR role
    public PostResponse createPost(@PathVariable Long communityId, @RequestParam String name) { ... }

    @PutMapping("/post/{postId}")
    // Must have PostRoleType.EDITOR
    public void updatePost(@PathVariable Long postId, @RequestParam String name) { ... }

    @GetMapping("/post/{postId}")
    // Must have PostRoleType.VIEWER role
    public PostResponse getPost(@PathVariable Long postId) { ... }
}

As you can see, every user can create a new community (the creator automatically becomes an admin of the entity as well). However, I haven't implemented the required checks on the endpoints yet. At first we need the role checking service.

Creating custom role checking service

Look at the blueprint of the RoleService below.

@Service("RoleService")
public class RoleService {
    public boolean hasAnyRoleByCommunityId(Long communityId, Role... roles) { ... }

    public boolean hasAnyRoleByPostId(Long postId, Role... roles) { ... }
}

I set the bean name inside the @Service annotation manually. Soon I'll explain the idea to you.

There are only two methods. The first one checks the required role presence by the communityId and the second one by the postId.

Look at the hasAnyRoleByCommunityId implementation below. The other method has the same idea and you can check out the whole class by this link.

@Service("RoleService")
@RequiredArgsConstructor
public class RoleService {
    private final CommunityRoleRepository communityRoleRepository;
    private final PostRoleRepository postRoleRepository;

    @Transactional
    public boolean hasAnyRoleByCommunityId(Long communityId, Role... roles) {
        final Long userId = ((PlainAuthentication) SecurityContextHolder.getContext().getAuthentication()).getPrincipal();
        final Set<CommunityRoleType> communityRoleTypes =
            communityRoleRepository.findRoleTypesByUserIdAndCommunityId(userId, communityId);
        for (Role role : roles) {
            if (communityRoleTypes.stream().anyMatch(communityRoleType -> communityRoleType.includes(role))) {
                return true;
            }
        }
        final Set<PostRoleType> postRoleTypes =
            postRoleRepository.findRoleTypesByUserIdAndCommunityId(userId, communityId);
        for (Role role : roles) {
            if (postRoleTypes.stream().anyMatch(postRoleType -> postRoleType.includes(role))) {
                return true;
            }
        }
        return false;
    }

    ...
}

Here is the algorithm:

Get the current user Authentication by calling SecurityContextHolder.getContext().getAuthentication() and cast it to the PlainAuthentication because that's the only type our application working with.
Find all community roles by the userId and the communityId. If any of the passed roles (according to the role inheritance model) are present in the set of the found community roles, return true. Otherwise, go to the next step.
Find all post roles by the userId and the communityId. If any of the passed roles (according to the role inheritance model) are present in the set of the found post roles, return true. Otherwise, return false.

I also want to point out the performance benefits of the solution I'm proposing to you. The classic approach with retrieving all user authorities and calling further the Authentication.getAuthorities method requires us to pull every CommunityRole and each PostRole the user has from the database at once (note that the user might be a member of dozens of communities and posses hundreds of roles). But we do it much more efficiently:

Pull the community roles only for the provided combination of (userId, communityId).

Pull the post roles only for the provided combination of (userId, communityId).

If the first step succeeds, the method returns true and doesn't perform another database query.

Combining PreAuthorize and custom role checking service

Finally, we set almost everything and we’re ready to apply the role checking mechanism at the endpoints. Why almost? Look at the code snippet below and try to notice an improvement slot.

@RestController
@RequestMapping("/api")
public class Controller {
    @PostMapping("/community")
    @PreAuthorize("isAuthenticated()")
    public CommunityResponse createCommunity(@RequestParam String name) { ... }

    @PostMapping("/community/{communityId}/post")
    @PreAuthorize("@RoleService.hasAnyRoleByCommunityId(#communityId, T(com.example.demo.domain.CommunityRoleType).MODERATOR)")
    public PostResponse createPost(@PathVariable Long communityId, @RequestParam String name) { ... }

    @PutMapping("/post/{postId}")
    @PreAuthorize("@RoleService.hasAnyRoleByPostId(#postId, T(com.example.demo.domain.PostRoleType).EDITOR)")
    public void updatePost(@PathVariable Long postId, @RequestParam String name) { ... }

    @GetMapping("/post/{postId}")
    @PreAuthorize("@RoleService.hasAnyRoleByPostId(#postId, T(com.example.demo.domain.PostRoleType).VIEWER)")
    public PostResponse getPost(@PathVariable Long postId) { ... }
}

The @RoleService is the reference to the RoleService Spring bean by its name. Then I call the specific role checking method. The postId and the communityId are the method arguments and we have to prefix their usage with hash. The last parameters are varargs of the required roles. Because the Role interface implementations are enums, we can reference the values by their fully qualified names.

As you have already guessed, this statement T(com.example.demo.domain.CommunityRoleType).MODERATOR has 2 problems:

It makes code harder to read and provokes copy-paste development.
If you change the package name of any enum role, then you have to update the dependent API methods accordingly.

Short and elegant enum references in SpEL expressions

Thankfully, there is more concise solution. Look at the fixed CommunityRoleType definition below.

public enum CommunityRoleType implements Role {
    ADMIN, MODERATOR;

    ...

    @Component("CommunityRole")
    @Getter
    static class SpringComponent {
        private final CommunityRoleType ADMIN = CommunityRoleType.ADMIN;
        private final CommunityRoleType MODERATOR = CommunityRoleType.MODERATOR;
    }
}

You just need to create another Spring bean that encapsulates the enum values as fields. Then you can reference them the same way we did with the RoleService.

The PostRoleType enhancing is similar. You can check out the code by this link.

Let's refactor the API methods a bit and see the final result. Look at the fixed Controller below.

@RestController
@RequestMapping("/api")
public class Controller {
    @PostMapping("/community")
    @PreAuthorize("isAuthenticated()")
    public CommunityResponse createCommunity(@RequestParam String name) { ... }

    @PostMapping("/community/{communityId}/post")
    @PreAuthorize("@RoleService.hasAnyRoleByCommunityId(#communityId, @CommunityRole.ADMIN)")
    public PostResponse createPost(@PathVariable Long communityId, @RequestParam String name) { ... }

    @PutMapping("/post/{postId}")
    @PreAuthorize("@RoleService.hasAnyRoleByPostId(#postId, @PostRole.EDITOR)")
    public void updatePost(@PathVariable Long postId, @RequestParam String name) { ... }

    @GetMapping("/post/{postId}")
    @PreAuthorize("@RoleService.hasAnyRoleByPostId(#postId, @PostRole.VIEWER)")
    public PostResponse getPost(@PathVariable Long postId) { ... }
}

Much more elegant solution, don’t you think? The role checking is declarative now and even non-technical folks can understand it (maybe you wish to generate documentation about endpoints restrictions' rules).

Integration testing and validating security

If there are no tests, you cannot be sure that your code is working at all. So, let's write some. I'm going to verify these cases:

If a user is not authenticated, creating a new community request should return 401.
If a user is authenticated, they should create a new community and new post inside it successfully.
If a user is authenticated but he has no rights to view the post, the request should return 403

I'm using Testcontainers to start PostgreSQL during tests. The explaining of its setup is out of the scope of this article. Anyway, you can check out the whole test suite by this link.

Look at the unauthorized request checking below.

@Test
void shouldReturn401IfUnauthorizedUserTryingToCreateCommunity() {
    userRepository.save(User.newUser("john"));

    final var communityCreatedResponse =
        rest.postForEntity(
            "/api/community?name={name}",
            null,
            CommunityResponse.class,
            Map.of("name", "community_name")
        );

    assertEquals(UNAUTHORIZED, communityCreatedResponse.getStatusCode());
}

The test works as expected.

Nothing complicated here. Let’s move forward to checking community and post successful creation. Look at the code snippet down below.

@Test
void shouldCreateCommunityAndPostSuccessfully() {
    userRepository.save(User.newUser("john"));

    final var communityCreatedResponse =
        rest.withBasicAuth("john", "password")
            .postForEntity(
                "/api/community?name={name}",
                null,
                CommunityResponse.class,
                Map.of("name", "community_name")
            );
    assertTrue(communityCreatedResponse.getStatusCode().is2xxSuccessful());
    final var communityId = communityCreatedResponse.getBody().id();

    final var postCreatedResponse =
        rest.withBasicAuth("john", "password")
            .postForEntity(
                "/api/community/{communityId}/post?name={name}",
                null,
                PostResponse.class,
                Map.of("communityId", communityId, "name", "post_name")
            );
    assertTrue(postCreatedResponse.getStatusCode().is2xxSuccessful());
}

The john user creates a new community and then adds a new post for it. Again, everything works smoothly.

Let's get to the final case. If user has no PostRoleType.VIEWER for the particular post, the request of getting it should return 403. Look at the code block below.

@Test
void shouldReturn403IfUserHasNoAccessToViewThePost() {
    userRepository.save(User.newUser("john"));
    userRepository.save(User.newUser("bob"));

    // john creates new community and post inside it
    ...    

    final var postViewResponse =
        rest.withBasicAuth("bob", "password")
            .getForEntity(
                "/api/post/{postId}",
                PostResponse.class,
                Map.of("postId", postId)
            );
    assertEquals(FORBIDDEN, postViewResponse.getStatusCode());
}

The creation of community and post is the same as in the previous test. So, I'm omitting these parts to focus on important details.

There are two pre-configured users: john and bob. The john user creates a new community and post whilst bob tries to get the post by its id. As long as he doesn’t possess the required privileges, the server returns 403 code. Look at the result of the test running below.

And the final check comes. Let’s run all tests at once to validate that their behavior is deterministic.

Everything works like a charm. Splendid!

Conclusion

As you can see, Spring Security plays well with a complex role hierarchy and inheritance model. Just a few patterns and your code shines.

That's all I wanted to tell you about applying role model with Spring Security. If you have any questions or suggestions, please leave your comments down below. If the content of the article was useful, press the like button and share the link with you friends and colleagues.

Thank you very much for reading this long piece!

Resources

Unit Testing Backward Compatibility of Message Format

Semyon Kirekov — Sat, 18 Feb 2023 08:50:27 +0000

Do you apply Apache Kafka or RabbitMQ in your software project? If so, then you definitely have some message schemas. Have you ever encountered in backward compatibility issue? An accidental message format changing and your entire system is no longer functioning? I bet you have such an unpleasant experience at least once in your career.

Thankfully, there is a solution. In this article, I'm telling you:

Why backward compatibility is crucial in message driven systems?
How can you validate the backward compatibility automatically with simple unit tests in Java?
A few words about forward compatibility.

You can find code examples and the entire project setup of unit testing backward compatibility in this repository.

Why backward compatibility matters?

Suppose there are two services. The Order-Service transfers a message about shipment details to the Shipment-Service. Look at the schema below.

The Shipment-Service (the consumer) defines the message schema (i.e. the contract) with two fields: orderId and address. And the Order-Service (the producer) sends the message according to the specified schema.

Everything works well. But suddenly the new requirement came into play. Now we have to set the complex address instead of just putting the city name (e.g. country, city, postcode, etc.). No problem, we just need to update the contract accordingly. Look at the fixed schema below.

{
  "orderId": Long
  "address": {
    "country": String,
    "city": String,
    "postcode": Integer
  }
}

Now there are two likely outcomes:

The producer had updated the contract before the consumer did.
Vice versa.

Assuming that the producer is the first one to make changes. What happens in such situation? Look at the diagram below to understand the consequences.

Here is what happens:

The Order-Service updates the contract.
The Order-Service sends the message with the new format.
The Shipment-Service tries to deserialize the received message with the old contract.
Deserialization process fails.

But what if the Shipment-Service updates the contract firstly? Let's discover this scenario on the schema below.

The problem remains the same. As a matter of fact, the Shipment-Service cannot deserialize received message.

So, we can conclude that altering the message schema has to be backward compatible.

Some of you may ask whether is it possible to update the Shipment-Service and the Order-Service contracts simultaneously or not? That would eliminate the problem with the backward compatibility, right? Well, it’s unbearable to perform such update in real life. Besides, even if you did, there’s a possibility that the broker (e.g. Apache Kafka) still contains the messages with the old format that you need to process somehow. So, backward compatibility is always a concern.

The idea of unit testing backward compatibility

Let's get to the code. I'm using Java Jackson for de/serialization. But the idea remains the same for any other library.

Look at the first version of OrderCreated message schema below.

@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    private final String address;

    public OrderCreated(Long orderId, String address) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
    }
}

The Jacksonized annotation tells Jackson to use the builder generated by the Builder annotation as the deserialization entry point. Quite useful because you don't have to repeat @JsonCreator and @JsonProperty usage while dealing with immutable classes.

How do we start with the backward compatibility check? Let’s add the message example to the resources/backward-compatibility directory. Look at the Json message example and the folder structure below.

{
  "orderId": 123,
  "address": "NYC"
}

Each schema altering should come up with adding a new file to the backward-compatibility directory. And the content should describe the increment made to the schema.

Each new file name should be lexically greater than the previous one. In that case, the tests run will be deterministic. The easiest way is just setting the incrementing natural number. But what if you need something more complicated? Then I recommend you to read my article about Flyway migrations naming strategy in a big project. I describe there the similar principle.

So, there are two rules we have to follow:

Each new schema update should come up with creating a new file in resources/backward-compatibility directory.
You should never delete the previously added files to ensure backward compatibility consistency.

Anyway, there are cases when you want to break the backward compatibility. For example, no one uses this field anymore and you just need to eliminate it. In that case, you can remove some backward compatibility data files. However, don’t treat it as a regular situation. It’s an exception but not a valid common scenario.

The automation process

Now we need the test that reads the described files and generates the validation cases. Look at the code example below:

class BackwardCompatibilityTest {
    private final ObjectMapper objectMapper = new ObjectMapper();

    @ParameterizedTest
    @SneakyThrows
    void shouldRemainBackwardCompatibility(String filename, String json) {
        final var orderCreated = assertDoesNotThrow(
            () -> objectMapper.readValue(json, OrderCreated.class),
            "Couldn't parse OrderCreated for filename: " + filename
        );
        final var serializedJson = assertDoesNotThrow(
            () -> objectMapper.writeValueAsString(orderCreated),
            "Couldn't serialize OrderCreated to JSON from filename: " + filename
        );
        final var expected = new JSONObject(json);
        final var actual = new JSONObject(serializedJson);
        JSONAssert.assertEquals(
            expected,
            actual,
            false
        );
    }
}

I parametrised the backward compatibility test because the number of files will grow (therefore, the true number of test cases as well). Here is what happens step by step:

The test receives the json content and the latest combined filename (I describe the algorithm later in the article). We the latter parameter for a more informative assertion message in case of failures.
Then we try to parse the supplied json to the OrderCreated object. If this step fails, then we definitely have broken the backward compatibility.
Afterwards, we serialize the parsed OrderCreated object back to json. Usually, this operation doesn’t fail. Anyway, we should always be prepared for a dangerous scenario.
And finally, we check that the supplied json equals to the one we got on step 3 (I use JSONAssert library here). The false boolean parameter tells to check only overlapping fields. For example, if the actual result contains the addressV2 field and the expected object doesn’t, it won’t trigger the failure. That’s a normal situation because backward compatibility data is static while the OrderCreated might grow with new parameters.

It's time to provide the input parameters to the shouldRemainBackwardCompatibility test. Look at the code example below.

private static Stream<Arguments> provideMessagesExamples() {
    final var resourceFolder = Thread.currentThread().getContextClassLoader().getResources("backward-compatibility").nextElement();
    final var fileInfos = Files.walk(Path.of(resourceFolder.toURI()))
                                  .filter(path -> path.toFile().isFile())
                                  .sorted(comparing(path -> path.getFileName().toString()))
                                  .map(file -> new FileInfo(
                                      file.getFileName().toString(),
                                      Files.readString(file))
                                  )
                                  .toList();
    ...
}

private record FileInfo(String name, String content) {}

In the beginning, we read all the files from the backward-compatibility directory, sort them by their names, and assign the result tuples list [(filename, content), …] to the fileInfos variable. Look at the next step below.

final var argumentsList = new ArrayList<Arguments>();
for (int i = 0; i < fileInfos.size(); i++) {
    JSONObject initialJson = null;
    for (int j = 0; j <= i; j++) {
        if (j == 0) {
            initialJson = new JSONObject(fileInfos.get(i).content);
        }
        final var curr = fileInfos.get(j);
        deepMerge(new JSONObject(curr.content), initialJson);
        if (j == i) {
            argumentsList.add(Arguments.arguments(curr.name, initialJson.toString()));
        }
    }
}
return argumentsList.stream();

And here comes the algorithm of creating the backward compatibility data list itself. Suppose we have 3 files: 1.json, 2.json and 3.json. Therefore, you should validate these combinations to ensure that newly added changes haven't broken the backward compatibility:

The content of the 1.json
The content of the 1.json + 2.json (the latter one rewrites existing fields).
The content of the 1.json + 2.json + 3.json (the latter one rewrites existing fields).

The loop above does this by calling the deepMerge method. The first parameter is the file with the current index in the loop. And the initialJson values are the target to merge all the latter changes.

Look at the deepMerge method declaration below.

private static void deepMerge(JSONObject source, JSONObject target) {
    final var names = requireNonNullElse(source.names(), new JSONArray());
    for (int i = 0; i < names.length(); i++) {
        final String key = (String) names.get(i);
        Object value = source.get(key);
        if (!target.has(key)) {
            // new value for "key":
            target.put(key, value);
        } else {
            // existing value for "key" - recursively deep merge:
            if (value instanceof JSONObject valueJson) {
                deepMerge(valueJson, target.getJSONObject(key));
            } else {
                target.put(key, value);
            }
        }
    }
}

I didn't write this code example by myself but took it from this StackOverflow question.

There are two arguments: source and target. The source is the one to read new values. And the target is the JSONObject to put them or replace existing ones.

Finally, to supply the computed data set just add the @MethodSource annotation to the shouldRemainBackwardCompatibility test method. Look at the code example below.

@ParameterizedTest
@MethodSource("provideMessagesExamples")
void shouldRemainBackwardCompatibility(String filename, String json) { ... }

Here is the test run result.

As you can see, an example with a single file works as expected.

Evolving data schema

Let’s get to the initial point of the article. There is an address field of String type and we want to store city, country, and postcode separately. As we already discussed, we cannot just replace the existing field with the new value type. Therefore, we should add the new field and mark the previous one as deprecating. Look at the updated version of OrderCreated class below:

@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;

    public OrderCreated(Long orderId, String address, Address newAddress) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = notNull(newAddress, "Complex address cannot be null");
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        private final String city;
        private final String country;
        private final Integer postcode;

        public Address(String city, String country, Integer postcode) {
            this.city = notNull(city, "City cannot be null");
            this.country = notNull(country, "Country cannot be null");
            this.postcode = notNull(postcode, "Postcode cannot be null");
        }
    }
}

I deprecated the address field and added the newAddress field that is a complex object containing country, city, and postcode. All right, now we need to add the new 2.json file with the example of filled newAddress. Look at the code snippet below.

{
  "newAddress": {
    "city": "NYC",
    "country": "USA",
    "postcode": 123456
  }
}

Let's run the test to check the backward compatibility.

We did break the backward compatibility! Why is that? Because we marked the newAddress field as mandatory and checked it for non-nullability. No surprise that the 1.json content failed the test because it has no mention of the newAddress field.

So, here comes an important conclusion. If you add a new field to the existing schema, you cannot check it for non-nullability. There is no guarantee that the producer will start sending messages with the newly stated field instantly. Meaning that the solution is the default values usage. Look at the fixed OrderCreated class declaration below.

@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;

    public OrderCreated(Long orderId, String address, Address newAddress) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = requireNonNullElse(newAddress, Address.builder().build());
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        @Builder.Default
        private final String city = "";
        @Builder.Default
        private final String country = "";
        @Builder.Default
        private final Integer postcode = null;
    }
}

We don’t want to deal with null value. That’s why we assign Address.builder().build() instance, if the supplied newAddress equals to null (which basically means it’s not present in the provided json).

The @Builder.Default Lombok annotation does the same thing as the requireNonNullElse function but without specifying the constructor manually.

Let's run the backward compatibility test again.

Now we’re ready. The new field presence has not broken the backward compatibility and we can update the contracts safely on both sided in any order.

A few words about forward compatibility

I’ve seen that developers tend to care about forward compatibility much less than the backward one. Though it’s crucial as well.

I’ve covered the topic of forward compatible enums in this article. If you’re dealing with message driven systems, I strongly recommend you to check it out.

What if somebody put additional properties that aren't present in the message schema? Look at the example below:

{
  "orderId": 123,
  "address": "NYC",
  "newAddress": {
    "city": "NYC",
    "country": "USA",
    "postcode": 123456
  },
  "unknown_field": "unknown_value"
}

Nothing should break, right? Anyway, there is no field to map the value. However, let's add the backward compatibility test to ensure consistency. Look at the new 3.json file declaration below.

{
  "unknown_field": "unknown_value"
}

Let's run the test to see the results.

Wow, something went wrong. Look at the output message below.

Caused by: UnrecognizedPropertyException: Unrecognized field "unknown_field"

By default, Jackson treats unknown property presence as an error. Sometimes that can be really annoying. Especially if there are many producers (and some may not be the part of your project). Thankfully, the fix is a piece of cake. You just need to add JsonIgnoreProperties annotation with the proper value. Look at the code snippet below.

@Data
@Builder
@Jacksonized
@JsonIgnoreProperties(ignoreUnknown = true)
public class OrderCreated { ... }

Let's run the test again.

It still fails! Though the message is different.

Expected: unknown_field
     but none found

The assertion checks that the input and the serialized json values are equal. But in this situation, that’s not the case. The input json contains the unknown_field. But the @JsonIgnoreProperties(ignoreUnknown = true) annotation presence just erases it. So, we lost the value.

Again, this might be acceptable under particular circumstances. But what if you still want to guarantee that we won’t lose the user provided fields even if there is no business logic on these values? For example, your service might act as a middleware that consumes messages from a Kafka topic and produces them to another one.

Jackson has a solution for this case as well. There is JsonAnySetter and JsonAnyGetter annotations that helps to store unknown values and serialize them lately. Look at the final version of OrderCreated class declaration below.

@Data
@Builder
@Jacksonized
@JsonIgnoreProperties(ignoreUnknown = true)
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;
    @JsonAnySetter
    @Singular("any")
    private final Map<String, Object> additionalProperties;

    @JsonAnyGetter
    Map<String, Object> getAdditionalProperties() {
        return additionalProperties;
    }

    public OrderCreated(Long orderId, String address, Address newAddress, Map<String, Object> additionalProperties) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = requireNonNullElse(newAddress, Address.builder().build());
        this.additionalProperties = requireNonNullElse(additionalProperties, emptyMap());
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        @Builder.Default
        private final String city = "";
        @Builder.Default
        private final String country = "";
        @Builder.Default
        private final Integer postcode = null;
    }
}

I marked the getAdditionalProperties as package-private so the users of this class cannot access the unknown properties values.

Let's run the backward compatibility test again.

Everything works like a charm now. The morale is that forward compatibility might be as important as backward one. And if you want to maintain it, then make sure that you don’t simply erase unknown values but transmit them further to be serialized in the expected json.

Conclusion

That’s all I wanted to tell you about backward (and a bit of forward) compatibility. Hope this knowledge will be useful to keep your contracts consistent to avoid unexpected deserialization errors on consumer side.

If you have questions or suggestions, leave your comments down below. Thanks for reading!

Resources

Forward Compatible Enum Values in API with Java Jackson

Semyon Kirekov — Sat, 11 Feb 2023 08:36:44 +0000

I got an inspiration for this article after watching this amazing tech talk by Ilya Sazonov and Fedor Sazonov. If you know Russian, go check it out. It's worth it.

In this article, I'm telling you:

Why do you need to care about forward compatible enum values?
What are the ways to achieve it?
How can Jackson library help you out?

Suppose we develop the service that consumes data from one input (e.g. Apache Kafka, RabbitMQ, etc.), deduplicates messages, and produces the result to some output. Look at the diagram below that describes the process.

As you can see, the service resolves deduplication rules by the platform field value.

If the platform is WEB, deduplicate all the messages in 2 hours window.
If the platform is MOBILE, deduplicate all the messages in 3 days window.
Otherwise, proceed with the message flow as-is.

We’re not discussing the technical details behind the deduplication process. It could be Apache Flink, Apache Spark, or Kafka Streams. Anyway, it’s out of the scope of this article.

Regular enum issue

Seems like the simple Java enum is a perfect candidate to map the platform field. Look at the code example below.

public enum Platform {
   WEB, MOBILE
}

On the one hand, we have a strongly typed platform field which helps to check possible errors during compilation. Also, we can add new enum values easily.

Isn’t that a brilliant approach? Actually, it is not. We’ve forgotten the third rule of deduplication. It says that unknown platform value should not trigger any deduplications but proceed with the message flow as-is. Then what happens if the service consumes such a message as below:

{
    "platform": "SMART_TV",
    ...
}

There is a new platform called SMART_TV. Nevertheless, no one has warned us that we need to deal with the new value. Because the third rule of deduplication should cover this scenario, right? However, in that case we’d got a deserialization error that would lead to unexpected termination of the message flow.

UKNOWN value antipattern

What can we do about it? Sometimes developers tend to add special UNKNOWN value to handle such errors. Look at the code example below.

public enum Platform {
   WEB, MOBILE, UNKNOWN
}

Everything seems great. If the platform fields has some unexpected string, just map it to Platform.UNKNOWN value. Anyway, it means that the output message topic received a corrupted platform field. Look at the diagram below.

Though we haven’t applied any deduplication rules, the client received an erased value of the platform field. Sometimes that’s OK, but not in this case. The deduplication-service is just a middleware that should not put any unexpected modifications to the submitted message flow. Therefore, the UNKNOWN value is not an option.

Besides, the UNKNOWN presence has some design drawbacks as well. As long as it’s an actual value, one can accidentally use it with inappropriate behavior. For example, you may want to traverse all existing enum values with Platform.values(). But the UNKNOWN is not the one that you wish to use in your code. As a matter of fact, avoid introducing UNKNOWN enum values at all costs.

String typing

What if we don't use enum at all but just deal with the plain String value? In that case, the clients can assign any string to the platform field without breaking the pipeline. That's a valid approach, if you don't have to introduce any logic based on the provided value. But we provided some deduplication rules depending on the platform. Meaning that string literals like "WEB" or "MOBILE" ought to repeat through the code.

The problems don't end here. Imagine that the client sent additional requirements to the platform field determination:

The value should be treated as case insensitive. So, "WEB", "web", and "wEB" string are all treated like the WEB platform.
Trailing spaces should be omitted. It means that the " mobile “ value truncates to ”mobile" string and converts to the MOBILE platform.

Now the code may look like this:

var resolvedPlatform = message.getPlatform().toUpperCase().trim();

if ("WEB".equals(resolvedPlatform)) {
    ...
}
else if ("MOBILE".equals(resolvedPlatform)) {
    ...
}
...

Firstly, this code snippet is rather smelly. Secondly, the compiler cannot track possible errors due to string typing usage. So, there is a higher chance of making a mistake.

As you can see, string typing solves the issue with the forward compatibility, but still it’s not a perfect approach.

Forward compatible enums

Thankfully, Jackson provides a great mechanism to deal with unknown enum values much cleaner. At first we should create an interface Platform. Look at the code snippet below.

public interface Platform {
    String value();
}

As you can see, the implementations encapsulate the string value that the client passed through the input message queue.

Then we declare a regular enum implementation as an inner static class. Look at the code example below.

public class Enum implements Platform {
    WEB, MOBILE;

    public static Enum parse(String rawValue) {
        if (rawValue == null) {
            throw new IllegalArgumentException("Raw value cannot be null");
        }
        var trimmed = rawValue.toUpperString().trim();
        for (Enum enumValue : values()) {
            if (enumValue.name().equals(trimmed)) {
                return enumValue;
            }
        }
        throw new IllegalArgumentException("Cannot parse enum from raw value: " + rawValue);
    }

    @Override
    @JsonValue
    public String value() {
        return name();
    }
}

That's a regular Java enum we've seen before. Though there are some details I want to point out:

The Jackson @JsonValue tells the library to serialize the whole object as the result of a single method invocation. Meaning that Jackson always serializes Platform.Enum as the result of the value() method.
We're going to use the static parse method to obtain enum value from the raw String input.

And now we're creating another Platform implementation to carry unexpected platform values. Look at the code example below.

@Value
public class Simple implements Platform {
    String value;

    @Override 
    @JsonValue 
    public String value() {
        return value;
    } 
}

The Value is the annotation from the Lombok library. It generates equals, hashCode, toString, getters and marks all the fields as private and final.

Just a dummy container for the raw string.

After adding Enum and Simple implementations, let's also create a static factory method to create the Platform from the provided input. Look at the code snippet below.

public interface Platform {
    String value();

    static Platform of(String value) {
        try {
            return Platform.Enum.parse(value);
        }
        catch (IllegalArgumentException e) {
            return new Simple(value);
        }
    }
}

The idea is trivial. Firstly, we’re trying to create the Platform as a regular enum value. If parsing fails, then the Simple wrapper returns.

Finally, time to bind all the things together. Look at the Jackson deserializer code below.

class Deserializer extends StdDeserializer<Platform> {
    protected Deserializer() {
        super(Platform.class);
    }

    @Override 
    public Platform deserialize(JsonParser p, DeserializationContext ctx) throws IOException {
        return Platform.of(p.getValueAsString());
    }
}

Look at the whole Platform declaration below to summarize the experience.

public interface Platform {
    String value();

    static Platform of(String value) {
        try {
            return Platform.Enum.parse(value);
        }
        catch (IllegalArgumentException e) {
            return new Simple(value);
        }
    }

    public class Enum implements Platform {
        WEB, MOBILE;

        public static Enum parse(String rawValue) {
            if (rawValue == null) {
                throw new IllegalArgumentException("Raw value cannot be null");
            }
            var trimmed = rawValue.toUpperString().trim();
            for (Enum enumValue : values()) {
                if (enumValue.name().equals(trimmed)) {
                    return enumValue;
                }
            }
            throw new IllegalArgumentException("Cannot parse enum from raw value: " + rawValue);
        }

        @Override
        @JsonValue
        public String value() {
            return name();
        }
    }

    @Value
    public class Simple implements Platform {
        String value;

        @Override 
        @JsonValue 
        public String value() {
            return value;
        } 
    }

    class Deserializer extends StdDeserializer<Platform> {      
        protected Deserializer() {
            super(Platform.class);
        }

        @Override 
        public Platform deserialize(JsonParser p, DeserializationContext ctx) throws IOException {
            return Platform.of(p.getValueAsString());
        }
    }
}

When we parse the message with platform we should put the deserializer accordingly.

class Message {
   ...
   @JsonDeserialize(using = Platform.Deserializer.class)
   private Platform platform;
}

Such setup us gives two opportunities. On the one hand, we can split the message flow according to the platform value and still apply regular Java enum. Look at the code example below.

var resolvedPlatform = message.getPlatform();

if (Platform.Enum.WEB.equals(resolvedPlatform)) {
    ...
}
else if (Platform.Enum.MOBILE.equals(resolvedPlatform)) {
    ...
}
...

Besides, Jackson wrap all unexpected values with Platform.Simple object and serialize the output result as a plain string. Meaning that the client will receive the unexpected platform value as-is. Look at the diagram below to clarify the point.

As a matter of fact, the following pattern allows us to keep using enums as a convenient language tool and also push the unexpected string values forward without data loss and pipeline termination. I think that it's brilliant.

Conclusion

Jackson is a great tool with lots of de/serialization strategies. Don't reject enum as a concept if values may vary. Look closely and see whether the library can overcome the issues.

That's all I wanted to tell you about forward compatible enum values. If you have questions or suggestions, leave your comments down below. Thanks for reading!

Resources

Enum in API — The deceit of illusory simplicity by Ilya Sazonov and Fedor Sazonov
Jackson library
Apache Kafka
RabbitMQ
Apache Flink
Apache Spark
Kafka Streams
@JsonValue
@Value Lombok annotation

Custom Gradle Plugin for Unified Static Code Analysis

Semyon Kirekov — Sat, 04 Feb 2023 16:08:57 +0000

Static code analysis is an incredible technique that makes your code base easier to maintain. But if you have multiple services in different repositories (possibly developed by separate teams), how can you make everybody follow the stated code style? A good approach is encapsulating all rules within a single plugin that'll perform the required validations on each project build automatically.

So, in this article I'm showing you:

How you can create the Gradle plugin with custom PMD and Checkstyle rules.
How to publish it to plugins.gradle.org.
How to automate the releasing process with GitHub Actions.

You can check out the code examples in this repository.

PMD, Checkstyle, and difficulties with polyrepos

PMD and Checkstyle are static analysis tools that check your code on each project build. Gradle allows to apply them easily.

plugins {
    id 'java'
    id 'pmd'
    id 'checkstyle'
}

And now you can tune each plugin the way you want to.

checkstyle {
    toolVersion = '10.5.0'
    ignoreFailures = false
    maxWarnings = 0
    configFile = file(pathToCheckstyleConfig)
}

pmd {
    consoleOutput = true
    toolVersion = '6.52.0'
    ignoreFailures = false
    ruleSetFiles = file(pathToPmdConfig)
}

If your entire project (or even company) is the monorepository, then this setup is absolutely fine. You just need to put these configurations in the root build.gradle file to apply those plugins for every existing module. But what if your choice is polyrepository? What if you want to share the same code style within all the projects in the company that developers are working on (and all the ones the programmers will create in the future)? Well, you can tell them to simply copy and paste the plugins’ configuration. Anyway, that’s an error-prone approach. There is always a probability that somebody does misconfiguration.

As a matter of fact, we need to reuse somehow the defined code style configuration in every viable project. The answer is simple. We need a custom Gradle plugin to encapsulate PMD and Checkstyle rules.

Custom Gradle plugin

Build configuration

Look at the build.gradle declaration below. It's the basic setup of the Gradle plugin project.

plugins {
    id 'java-gradle-plugin'
    id 'com.gradle.plugin-publish' version '1.1.0'
}

group = 'io.github.simonharmonicminor.code.style'
sourceCompatibility = '8'

repositories {
    mavenCentral()
}

ext {
    set('lombokVersion', '1.18.24')
}

dependencies {
    compileOnly "org.projectlombok:lombok:${lombokVersion}"
    annotationProcessor "org.projectlombok:lombok:${lombokVersion}"
    testImplementation 'org.junit.jupiter:junit-jupiter:5.7.2'
}

gradlePlugin {
    website = 'https://github.com/SimonHarmonicMinor/gradle-code-style-plugin-example'
    vcsUrl = 'https://github.com/SimonHarmonicMinor/gradle-code-style-plugin-example'
    plugins {
        gradleCodeStylePluginExample {
            id = 'io.github.simonharmonicminor.code.style'
            displayName = 'Gradle Plugin Code Style Example'
            description = 'Predefined Checkstyle and PMD rules'
            implementationClass = 'io.github.simonharmonicminor.code.style.CodingRulesGradlePluginPlugin'
            tags.set(['codestyle', 'checkstyle', 'pmd'])
        }
    }
}

tasks.named('test') {
    useJUnitPlatform()
}

Now let’s deconstruct the configuration step by step, starting with plugins block. Look at the code snippet below.

plugins {
    id 'java-gradle-plugin'
    id 'com.gradle.plugin-publish' version '1.1.0'
}

The java-gradle-plugin command enables the tasks for the regular Gradle plugin project. Whilst the com.gradle.plugin-publish one allows to pack and publish the plugin to https://plugins.gradle.org/.

I'm showing you the whole publishing process lately.

Then comes the basic project configuration.

group = 'io.github.simonharmonicminor.code.style'
sourceCompatibility = '8'

repositories {
    mavenCentral()
}

The group defines groupId in favor of Apache Maven naming conventions. The sourceCompatibility is the version of the target Java binaries. Though Java 8 is outdated now, I recommend you to build your Gradle plugins with the earliest JDK version that developers use in your company. Otherwise, you’ll block them the way to follow your code style guidelines.

Then comes the dependencies scope.

ext {
    set('lombokVersion', '1.18.24')
}

dependencies {
    compileOnly "org.projectlombok:lombok:${lombokVersion}"
    annotationProcessor "org.projectlombok:lombok:${lombokVersion}"
    testImplementation 'org.junit.jupiter:junit-jupiter:5.7.2'
}

Nothing special here. So, let's move on to the publishing configuration.

gradlePlugin {
    website = 'https://github.com/SimonHarmonicMinor/gradle-code-style-plugin-example'
    vcsUrl = 'https://github.com/SimonHarmonicMinor/gradle-code-style-plugin-example'
    plugins {
        gradleCodeStylePluginExample {
            id = 'io.github.simonharmonicminor.code.style'
            displayName = 'Gradle Plugin Code Style Example'
            description = 'Predefined Checkstyle and PMD rules'
            implementationClass = 'io.github.simonharmonicminor.code.style.CodingRulesGradlePluginPlugin'
            tags.set(['codestyle', 'checkstyle', 'pmd'])
        }
    }
}

The website and vcsUrl should point out to the public Git repository with the source code of the plugin. And the plugins block defines each implementation of Plugin interface in your project. Finally, the tags are just hashtags to search your plugin in the registry.

When you publish your Gradle plugin to https://plugins.gradle.org/, the package name is crucial. The code of your plugin should be available on GitHub. If it’s not open source, you may have issues with publishing it. Then you can declare your package name as io.github.your_github_login.any.package.you.like. But if you wish to use some other name like com.mycompany.my.plugin, then make sure you owe the domain mycompany.com. Otherwise, the Gradle engineers may reject the publishing.

Note that Gradle prohibits plugin and gradle as tags values. Such builds fail during gradle publishPlugins task execution.

And finally comes the JUnit 5 configuration.

tasks.named('test') {
    useJUnitPlatform()
}

The plugin code

I want to show you the whole plugin code. And when I'm going to explain each individual detail to you. Look at the code snippet below.

public class CodingRulesGradlePluginPlugin implements Plugin<Project> {

  @Override
  public void apply(Project project) {
    project.getPluginManager().apply("checkstyle");
    project.getExtensions().configure(CheckstyleExtension.class, checkstyleExtension -> {
      checkstyleExtension.setToolVersion("10.5.0");
      checkstyleExtension.setIgnoreFailures(false);
      checkstyleExtension.setMaxWarnings(0);
      checkstyleExtension.setConfigFile(
          FileUtil.copyContentToTempFile("style/checkstyle.xml", ".checkstyle.xml")
      );
    });

    project.getPluginManager().apply("pmd");
    project.getExtensions().configure(PmdExtension.class, pmdExtension -> {
      pmdExtension.setConsoleOutput(true);
      pmdExtension.setToolVersion("6.52.0");
      pmdExtension.setIgnoreFailures(false);
      pmdExtension.setRuleSets(emptyList());
      pmdExtension.setRuleSetFiles(project.files(
          FileUtil.copyContentToTempFile("style/pmd.xml", ".pmd.xml")
      ));
    });

    final SortedSet<String> checkstyleTaskNames = project.getTasks()
        .withType(Checkstyle.class)
        .getNames();

    final SortedSet<String> pmdTaskNames = project.getTasks()
        .withType(Pmd.class)
        .getNames();

    project.task(
        "runStaticAnalysis",
        task -> task.setDependsOn(
            Stream.concat(
                checkstyleTaskNames.stream(),
                pmdTaskNames.stream()
            ).collect(Collectors.toList())
        )
    );
  }
}

The most obvious and important detail is that each plugin task has to implement the Gradle Plugin interface.

import org.gradle.api.Plugin;
import org.gradle.api.Project;

public class CodingRulesGradlePluginPlugin implements Plugin<Project> {
    @Override
    public void apply(Project project) { ... }
}

Then I'm configuring the Checkstyle task. I simply apply the checkstyle plugin, retrieve the CheckstyleConfiguration and override the properties I want to. Look at the code block below.

project.getPluginManager().apply("checkstyle");
project.getExtensions().configure(CheckstyleExtension.class, checkstyleExtension -> {
  checkstyleExtension.setToolVersion("10.5.0");
  checkstyleExtension.setIgnoreFailures(false);
  checkstyleExtension.setMaxWarnings(0);
  checkstyleExtension.setConfigFile(
      FileUtil.copyContentToTempFile("style/checkstyle.xml", ".checkstyle.xml")
  );
});

The FileUtil.copyContentToTempFile function requires some explanation. I put the Checkstyle configuration into the src/main/resources/style/checkstyle.xml file. However, if you point it straightly, then people’ll get bizarre error messages on applying your Gradle in their projects. There are some workarounds but the easiest way is to copy the content to the temporary file.

Look at the PMD configuration below. It’s similar to the Checkstyle one.

project.getPluginManager().apply("pmd");
project.getExtensions().configure(PmdExtension.class, pmdExtension -> {
  pmdExtension.setConsoleOutput(true);
  pmdExtension.setToolVersion("6.52.0");
  pmdExtension.setIgnoreFailures(false);
  pmdExtension.setRuleSets(emptyList());
  pmdExtension.setRuleSetFiles(project.files(
      FileUtil.copyContentToTempFile("style/pmd.xml", ".pmd.xml")
  ));
});

We’re ready now. We can apply it in a real project. Though there is a slight improvement as well. Look at the code snippet below.

final SortedSet<String> checkstyleTaskNames = project.getTasks()
    .withType(Checkstyle.class)
    .getNames();

final SortedSet<String> pmdTaskNames = project.getTasks()
    .withType(Pmd.class)
    .getNames();

project.task(
    "runStaticAnalysis",
    task -> task.setDependsOn(
        Stream.concat(
            checkstyleTaskNames.stream(),
            pmdTaskNames.stream()
        ).collect(Collectors.toList())
    )
);

The runStaticAnalysis task triggers all Checkstyle and PMD tasks to run sequentially. It comes in handy when you want to verify your entire project before creating a pull request. If you added the runStaticAnalysis task directly in the build.gradle, it would look like this:

task runStaticAnalysis {
    dependsOn checkstyleMain, checkstyleTest, pmdMain, pmdTest
}

Testing

What about testing? It's better to track errors during the build but not when developers have already applied your plugin in their projects. Though Gradle provides Gradle TestKit for functional testing, the case I'm showing you is rather simple and unit tests are sufficient.

Again, I'm showing the whole code piece at once and when I'll point out important details.

class CodingRulesGradlePluginPluginTest {

  @Test
  void shouldApplyPluginSuccessfully() {
    final Project project = ProjectBuilder.builder().build();
    project.getPluginManager().apply("java");

    assertDoesNotThrow(
        () -> new CodingRulesGradlePluginPlugin().apply(project)
    );

    final Task task = project.getTasks().getByName("runStaticAnalysis");
    assertNotNull(task, "runStaticAnalysis task should be registered");
    final Set<String> codeStyleTasks =
        Stream.of("checkstyleMain", "checkstyleTest", "pmdTest", "pmdMain").collect(toSet());
    assertTrue(
        task.getDependsOn().containsAll(codeStyleTasks),
        format(
            "Task runStaticAnalysis should contain '%s' tasks, but actually: %s",
            codeStyleTasks,
            task.getDependsOn()
        )
    );
  }
}

Firstly comes test Gradle project instantiation. Look at the code snippet below.

import org.gradle.testfixtures.ProjectBuilder;
import org.gradle.api.Project;

final Project project = ProjectBuilder.builder().build();
project.getPluginManager().apply("java");

Gradle provides some fixtures for unit testing. The ProjectBuilder creates an API compatible implementation of the Project interface. So, you can safely pass it to the YourPluginClass.apply method.

Before invoking the business logic, we also manually apply the java plugin. Our plugin targets the Java applications. So, it’s natural to pass the Java configured Project implementation.

Then we simply invoke the custom plugin method and pass the configured Project implementation.

assertDoesNotThrow(
    () -> new CodingRulesGradlePluginPlugin().apply(project)
);

Afterwards, comes assertions. We need to make sure that the runStaticAnalysis task registered successfully.

final Task task = project.getTasks().getByName("runStaticAnalysis");
assertNotNull(task, "runStaticAnalysis task should be registered");

If it’s present, we validate the task depending on existing Checkstyle and PMD tasks.

final Set<String> codeStyleTasks =
    Stream.of("checkstyleMain", "checkstyleTest", "pmdTest", "pmdMain").collect(toSet());
assertTrue(
    task.getDependsOn().containsAll(codeStyleTasks),
    format(
        "Task runStaticAnalysis should contain '%s' tasks, but actually: %s",
        codeStyleTasks,
        task.getDependsOn()
    )
);

That's the minimum case we should test before pushing the plugin to the https://plugins.gradle.org/.

Releasing the plugin with GitHub Actions

When you register a new account on https://plugins.gradle.org/, go to your page and open the API Keys tab. You should generate new keys. There will be two of them.

gradle.publish.key=...
gradle.publish.secret=...

Afterwards, open the Settings of your repository and go to the Secrets and Variables -> Actions item. You have to store the obtained keys as your repository secrets.

And finally comes the GitHub Actions build configuration.

I placed mine in .github/workflow/build.yml file.

Look at the whole setup below. Then I'm telling you the meanings of particular blocks.

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

permissions:
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: '8'
          distribution: 'temurin'
      - name: Build with Gradle
        uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
        with:
          arguments: build
  publish:
    needs:
      - build
    if: github.ref == 'refs/heads/master'
    runs-on: ubuntu-latest
    steps:
      - name: Auto Increment Semver Action
        uses: MCKanpolat/auto-semver-action@1.0.5
        id: versioning
        with:
          releaseType: minor
          incrementPerCommit: false
          github_token: ${{ secrets.GITHUB_TOKEN }}
      - name: Next Release Number
        run: echo ${{ steps.versioning.outputs.version }}
      - uses: actions/checkout@v3
      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: '8'
          distribution: 'temurin'
      - name: Publish Gradle plugin
        uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
        with:
          arguments: build publishPlugins -Pgradle.publish.key=${{ secrets.GRADLE_PUBLISH_KEY }} -Pgradle.publish.secret=${{ secrets.GRADLE_PUBLISH_SECRET }} -Pversion=${{ steps.versioning.outputs.version }}

The top file declaration states the rules of pipeline triggering.

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

The pipeline runs on each pull request to master branch and every building of master branch itself.

The build consists of two jobs. The first one is trivial. It just runs the Gradle build task. Look at the configuration below.

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: '8'
          distribution: 'temurin'
      - name: Build with Gradle
        uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
        with:
          arguments: build

And then comes the publishing itself. It also contains several steps. The first increments the version automatically and saves it to the environment variable. It’s pretty convenient, because Gradle plugins cannot be published as snapshots.

publish:
    needs:
      - build
    if: github.ref == 'refs/heads/master'
    runs-on: ubuntu-latest
    steps:
      - name: Auto Increment Semver Action
        uses: MCKanpolat/auto-semver-action@1.0.5
        id: versioning
        with:
          releaseType: minor
          incrementPerCommit: false
          github_token: ${{ secrets.GITHUB_TOKEN }}
      - name: Next Release Number
        run: echo ${{ steps.versioning.outputs.version }}

The if: github.ref == ‘refs/heads/master’ tells GitHub Actions worker to run the pipeline only if master branch is being built. Therefore, GitHub Actions won’t trigger the publish process during the pull request build.

And now we need to publish the packed plugin itself. Look at the code snippet below.

      - uses: actions/checkout@v3
      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: '8'
          distribution: 'temurin'
      - name: Publish Gradle plugin
        uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
        with:
          arguments: build publishPlugins -Pgradle.publish.key=${{ secrets.GRADLE_PUBLISH_KEY }} -Pgradle.publish.secret=${{ secrets.GRADLE_PUBLISH_SECRET }} -Pversion=${{ steps.versioning.outputs.version }}

As you can see, GitHub Actions passes the gradle.publish.key and the gradle.publish.secret properties through the secrets and the new project version as the environment variable.

Conclusion

As you can see, automating code style rules checking is not that complicated in Gradle. By the way, you can apply the plugin described in the project by including id 'io.github.simonharmonicminor.code.style' version '0.1.0'.

If you have any questions or suggestions, leave your comments down below. Thanks for reading!

Forem: Semyon Kirekov

Golang Dependency Inversion — A Java Developer's Point of View

The case

Java

Golang

Conclusion

JOOQ Is Not a Replacement for Hibernate. They Solve Different Problems

Transaction Script

Domain Model

Manual mapping

The Need for Hibernate

Goals of JOOQ and Hibernate

What About DAOs in JOOQ?

Conclusion

Resources

Custom methods in REST API endpoints

The problem with mapping business operation to a set of predefined HTTP methods

Sub-resource

A special actions endpoints

Custom method with a query parameter

Custom method with a colon punctuation mark

Conclusion

Resources

Rich Domain Model with Spring Boot and Hibernate

Table of contents

The problems with Anemic Domain Model

Too smart services

Possible invariants' violation

Lack of encapsulation

Testing is harder

Rich Domain Model principle

Don't add setters, getters, and public no-args constructor

No-args constructor allows incostistent object instantiation

Setters break encapsulation

Getters break encapsulation

Current result

Aggregate and Aggregate root

Evolution of requirements

Each Pocket must possess at least one Tamagotchi

Every Tamagotchi name has to be unique within a Pocket

If a user deletes a Tamagotchi, they can restore it by name

Querying data

Manual queries

Introducing toDto method

Unit testing entities

Pocket must always possess at least one Tamagotchi

If you delete Tamagotchi, you can restore it by name

If you delete multiple Tamagotchi with the same name, you can only restore the last one

Integration testing

Create Pocket

Create Tamagotchi

Update Tamagotchi

Performance implications

Optimization of queries

Pinpointing optimized checks

Database generated ID

Manually fill the id

Introducing business key

Is Rich Domain Model always worth it?

Conclusion

Resources

Integration Tests for N + 1 problem in Java

An example of the N + 1 problem

Tracking the N + 1 problem automatically

Query Count Service

BeanPostProccessor and DataSource proxy

Custom assertions

The final example

Conclusion

Resources

JUnit 5: link tests with task tracker issues

Issue Annotation

Setting up service loaders

Creating meta-information JSON file

Generating documentation

Setting up GitHub Pages

Conclusion

Resources

Spring, SchemaSpy DB docs, and GitHub Pages

What is SchemaSpy?

A special `actions` endpoints