Forem: João Esperancinha

Implementing a Geo Location module using the Haversine formula

João Esperancinha — Sun, 29 Dec 2024 17:07:22 +0000

On the 2020/06/06 I finally made my first Python module.
The reason for the creation of this module is that I needed to perform certain calculations regarding distances.
I wanted to write an article that would cover the most well known ways transmitting data.
Namely, via streaming protocols.
I ended up logically talking about RabbitMQ and Kafka.
Although it was clear for me what I wanted to write about, I wasn't sure what should I base my article about.
An idea was to calculate distances in Km in different planets.
Of course, for this, we need pretty good knowledge of how coordinates work.
I ended up hastily implementing python code to give me those calculations.
At some point I did think I was going too far in my article and ended up settling to planet 🌎 Earth 🌍 and make something with trains and trucks.

The old Geo Library can be found there on path bl-demo-server/bl-core-service.

1. Introduction

What I want to demonstrate with this blog post is a few concepts about python, how I implemented the Haversine Forumla, what is this formula and where does it come from and finally how I made the package currently available on PyPI (Python Package Index)
.

1.1. Haversine Formula Origins

In the book, Heavenly Mathematics: The Forgotten Art of Spherical Trigonometry, Oxford: Princeton University Press, 2012, we can find very interesting references about the Haversine Theory.
It also contains the complete description of it.
For us, what's important is to understand its origins and what it is about.

The beginnings of this theory take us back to 1801.

At this time, Josef de Mendonza y Ríos, used it for the first time.
Josef de Mendonza y Rios, also known as José de Mendonza y Ríos , was a spanish astronomer and mathematician of the 18th century.
He made several publications about navigation and his work was focused on providing mathematical tools to calculate the position of a ship at sea using natural resources and observation instruments.
This position was to be measured in degrees and consisted of two dimensions: latitude and longitude.
José gained notoriety by being able to calculate these coordinates by using two altitudes of the Sun for the latitude and the distance of the moon to a celestial body for the longitude.
Using tables, Sailors and other members of crews at sea could easily and quickly find approximately where they were.
His work earned him an honorable place at the Royal Swedish Academy of Sciences.

However, the first specific publication regarding the Haversine formula only occured only 4 years later in 1805.
At that time James Andrew, published the first known Haversine tables in his ASTRONOMICAL and NAUTICAL tables with PRECEPTS for finding the LONGITUDE and LATITUDE of PLACES By Lunar Distances, Double Altitudes, &c, AND FOR SOLVING OTHER THE MOST USEFUL PROBLEMS IN PRACTICAL ASTRONOMY.

Finally, the actual term haversine was invented by James Inman(1776–1859).

He was an English mathematician an professor at the Royal Naval College in Portsmouth UK.
James Inman was very busy in 1835 in the development of a new tale of haversines.
This is where the term comes from.
The new table of haversines, by James Inman, introduced a very easy way to calculate the distance between two points by using simple spherical trigonometry.

This college has been closed, and it was active between 1733 and 1837

As we can see through this very vey quick introduction to its history, the haversine formula exists for a long time.
It is a centuries old technology, but it's important to realize that although old, its is the basis of what we are about to see.
Not only that, but is part of the theory that makes today GPS (Global Positioning System) possible and with that, we can nowadays know how long the train will take to get to us.
We now know when the buses are coming.
We know how long it will take from A to B.
We even now can have an estimate so precise, that we don't even think about it and take it most of the times for granted.
Even when we have to go from A to B going through C to D and all the way to Z.
Almost everything is calculated wth extreme precision.

We are going to see that although we will consider Earth to be a big globe, all the calculations will still match.
Earth can also be described as a big potato that is almost round.
That small difference for all our distance calculations are negligible.
Unless of coure we need even more extremely precise data for scientific studies.
But that is another issue.

2. Theory

Let's first have a look at the meaning of the word Haversine.
I didn't mention this in the introduction, just because this term involves quite a lot of mathematical explanation.
This is way it doesn't seem to belong to a historical backgroundf section.
However, it is precisely the end of that section where we will start this one.
From the studies of James Inman, we can finally understand what a Haversine is.

in Heavenly Mathematics: The Forgotten Art of Spherical Trigonometry

From this image we can see that vers is basically a versed sine.
We can now make the correct formula:

vers 𝜭 = 1 - cos 𝜭 = 2 sin ² (𝜭/2)

Half of that is

hav 𝜭 = (1 - cos 𝜭)/2 = sin ² (𝜭/2)

And this is what a haversine formula is about.
haversine just means Half versed sine.

That small distance become something very attractive for people at sea.
The main reason was that it would always result in a positive number.
Plus, a haversine is very easily invertible and works well with small numbers.
Rounding numbers isn't really a problem using haversines.

For our implementation we will use this base to develop different formulas.

In line with a good understanding of this problem, it is important that we become very much aware of what latitude and longitude actually mean.

2.1. Latitude (ɸ)

One line parallel to the equator has the same latitude on every of its points and its called a parallel
Latitude is a measurement in degrees of the angle formed from the equator line of the earth to the location we are determining.
The equator line seems like an obvious point to set the latitude at 0°.
This is because it cuts Earth in two halves, and it was already responsible for the separation between the Northen and Southern hemisphere.

The symbol for Latitude is ɸ.

2.2. Longitude (λ)

One line drawn from pole to pole has the same longitude and its called a meridian
Longitude is a measurement in degrees from the Prime Meridian to the location we are determining.
The Prime Meridian has 0° longitude and goes through very near the Royal Observatory, in Greenwich, England.
This has been determined by convention.
The symbol for Longitude is λ

3. Implementation

We are now going to get our hands on Python.
Before we continue we need to be aware of a few things about Python and the libraries we are going to use:

By default, our functions are available in scope public
Functions starting with one _ are considered protected. We won't be using these for now
Functions starting with two __ are considered private. We will use these
sin, cos, tan, atan, and other geometrical functions of python's math use radians as arguments. This is why we will be using the radians functions a lot.
We will be needing to calculate square roots (sqrt) and we will need the pi constant a lot

For version 1.0.0 we will be using some functions within a Coord class.
This makes functions easier to use and makes our code more elegant:

class Coord:
    def __init__(self, lat, lon):
        self.lat = lat
        self.lon = lon

    def delta_rads(self, lat, lon):
        self.lat += lat
        self.lon += lon

    def distance_to_in_meters(self, coord, planet_radius_km=6371000):
        meters = self.__distance_to(coord, planet_radius_km)
        meters = round(meters, 3)
        return meters

    def distance_to_in_kilometers(self, coord, planet_radius_km=6371000):
        meters = self.__distance_to(coord, planet_radius_km)
        km = round(meters / 1000, 3)
        return km

    def __distance_to(self, coord, planet_radius_km):
        lon1 = self.lon
        lat1 = self.lat
        lon2 = coord.lon
        lat2 = coord.lat
        phi_1 = radians(lat1)
        phi_2 = radians(lat2)
        delta_phi = radians(lat2 - lat1)
        delta_lambda = radians(lon2 - lon1)
        a = sin(delta_phi / 2.0) ** 2 + cos(phi_1) * cos(phi_2) * sin(delta_lambda / 2.0) ** 2
        c = 2 * atan2(sqrt(a), sqrt(1 - a))
        meters = planet_radius_km * c
        return meters

    def __repr__(self):
        return "lat: " + str(self.lat) + " lon: " + str(self.lon)

Let's now break this class down into its subcomponents:

First a look at the constructor:

def __init__(self, lat, lon):
    self.lat = lat
    self.lon = lon

This is basically the way a coordinate is defined.
We setup the latitude and the longitude.
One of the things that may be important to move along from one coordinate to another place is a Δ (delta).
For this I've created a function for that:

def delta_rads(self, lat, lon):
    self.lat += lat
    self.lon += lon

So now we have can easily move step by step if, and only if, we know how many degrees we want to move.
Of course, normally we don't think about the Prime Meridian and we normally talk about kilometers or miles.

This is why I've also created this inner function:

def add_delta_in_km_to_coord(self, d_lat_km, d_lon_km, planet_radius_km=6371):
    lat = self.lat
    d_lat = (180 / pi) * (d_lat_km / float(planet_radius_km))
    d_lon = (180 / pi) * (d_lon_km / float(planet_radius_km)) / cos(radians(lat))
    self.delta_rads(d_lat, d_lon)

It's important to notice that we allow, in parameter planet_radius_km, the radius of a planet.
Remember that this module, is supposed to be usable in different planets.
We first convert our deltas to degrees.
The latitude means that if we get a positive delta, we go North.
A negative one and we go South.
With that in mind we also want to respect the curvature of the earth.
This is why qe have the ratio of 180 degrees / pi times ΔKlm / radius.
In terms of longitude, we will be going from west to east for a delta positive number.
However, for longitude, the curvature varies.
Our parallel line, depending on our position in the meridian line where we are, has a different radius.
We compensate this in our formulat by dividing by cos(radians(lat)).
This means that we convert our latitude to radians so that we can factor our resut by its cos (cosine).
Finally, we just call our previous function which receves Δ's in degrees.

We now need to look at two parent functions:

def distance_to_in_meters(self, coord, planet_radius_km=6371000):
    meters = self.__distance_to(coord, planet_radius_km)
    meters = round(meters, 3)
    return meters

def distance_to_in_kilometers(self, coord, planet_radius_km=6371000):
    meters = self.__distance_to(coord, planet_radius_km)
    km = round(meters / 1000, 3)
    return km

We will make sure to give only three decimals in this first version for our calculations.
It is implicit in both functions that we are trying to get the distance between two coordinates.
This is our instance and one injected via params.
Also notice that we are also allowing for a planet_radius.
The only difference is that this function accepts this parameter only in meters.

We finally reach the subject of our article.
The Haversine formula.
According to the literature, the Haversine formula is effectively the source of the formula to calculate distances between two points.
These points can be anywhere in a round planet.
Of course planets are more like giant "potatoes" than actual spheres, but again we need to remember that those humps are negligible.

2 r arcsin (√( sin ² ( (ɸ2 - ɸ2) / 2) + cos ɸ1 * cos ɸ2 sin ² ( (λ2 - λ1) / 2 )))

This formula is derivated from the formula we saw in our introduction.
In code terms this translates to:

def __distance_to(self, coord, planet_radius_km):
    lon1 = self.lon
    lat1 = self.lat
    lon2 = coord.lon
    lat2 = coord.lat
    phi_1 = radians(lat1)
    phi_2 = radians(lat2)
    delta_phi = radians(lat2 - lat1)
    delta_lambda = radians(lon2 - lon1)
    h = sin(delta_phi / 2.0) ** 2 + cos(phi_1) * cos(phi_2) * sin(delta_lambda / 2.0) ** 2
    arc_sin = atan2(sqrt(h), sqrt(1 - h))
    d = 2 * arc_sin * planet_radius_km
    return d

In our code we still find these utility methods

def create_west_random_point(center, radius):
    degree = random.randint(-90, 90)
    d_lat_km = cos(radians(degree)) * radius
    d_lon_km = sin(radians(degree)) * radius
    origin = Coord(center.lat, center.lon)
    origin = add_delta_in_km_to_coord(origin, -d_lat_km, d_lon_km)
    return origin

def create_east_random_point(center, radius):
    degree = random.randint(-90, 90)
    d_lat_km = cos(radians(degree)) * radius
    d_lon_km = sin(radians(degree)) * radius
    origin = Coord(center.lat, center.lon)
    origin = add_delta_in_km_to_coord(origin, +d_lat_km, d_lon_km)
    return origin

What these do is to create a random point in a limited square delimiting a circumference of a determined radius and a certain ceter Coord.

4. Publishing a library

In order to publish a library I followed the rules determined by the example given in:

Python Package Index Sample Project

The important file for this is the setup.py file:

# -*- coding: utf-8 -*-
from distutils.core import setup

with open("README.txt", "r") as fh:
    long_description = fh.read()

setup(
    long_description=long_description,
    long_description_content_type="text/markdown",
    name='geo_calculator',
    package_dir={'': 'src'},
    py_modules=["geo_calculator"],
    version='1.0.0-SNAPSHOT',
    description='Multi function Geo Location calculator',
    author='João Esperancinha',
    author_email='jofisaes@gmail.com',
    url='http://joaofilipesabinoesperancinha.nl/main',
    download_url='https://github.com/user/reponame/archive/v_01.tar.gz',
    keywords=['geo', 'location', 'latitude', 'longitude'],
    install_requires=[
        # 'math',
        # 'random',
    ],
    classifiers=[
        'Development Status :: 3 - Alpha',
        'Intended Audience :: Developers',
        'Topic :: Software Development :: Build Tools',
        'License :: OSI Approved :: Apache Software License',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
    ],
)

The first line # -*- coding: utf-8 -*- is important if we are using uncommon characters.
In my own name, I do have one of these uncommon characters and that is the Ãẫ of my name João.

To configure descriptions we need:

long_description=long_description,
long_description_content_type="text/markdown",

Note, that although markdown descriptions are supported, I couldn't get themm to work in a way that I like to see.
This is why I just ended up reading a simple Readme.txt file instead:

with open("README.txt", "r") as fh:
    long_description = fh.read()

We then need to give some self-explanatory paramter:

name='geo_calculator',
package_dir={'': 'src'},
py_modules=["geo_calculator"],
version='1.0.0-SNAPSHOT',
description='Multi function Geo Location calculator',
author='João Esperancinha',

For the rest of the parameters for our module, it is important to consult the publishing documentation.

One parameter that it's important to give better attention is this:

classifiers=[
    'Development Status :: 3 - Alpha',
    'Intended Audience :: Developers',
    'Topic :: Software Development :: Build Tools',
    'License :: OSI Approved :: Apache Software License',
    'Programming Language :: Python :: 3',
    'Programming Language :: Python :: 3.4',
    'Programming Language :: Python :: 3.5',
    'Programming Language :: Python :: 3.6',
],

Here we are specifying a sort of datasheet of our module.
At the moment my module is still an alpha version.
Although its coming up good, I will still create better versions, faster and with more functionalities.

To get our module published we need to install twine:

sudo pip3 install twine

Now that we have twine, we can finally create our distribution:

python setup.py sdist

Finally we push this to our repository:

twine upload dist/*

If you are pushing your own module, be sure to have your test and production accounts created.

Note that we are going to perform tests locally without having to use any of these remote repos.

5. Running unit tests

Before sending a pacakge to any of these PyPI repos, we should make unit test first and make sure that everything works.
To do that we can install our package locally.
To do that we first need to create it.

As seen before we can create it usidn the sdist switch:

python setup.py sdist

Now let's install it with command pip3:

sudo pip3 install dist/geo_calculator-0.1.2.tar.gz

Finally we can run our unit tests with python3:

python3 -m unittest geo_calculator_test.py

Now let's just have a quick look at how the unit tests are constructed.
We first create the unit test without any functions:

import unittest

from geo_calculator import Coord


class CoordTest(unittest.TestCase):

    def setUp(self):
        pass

We are setting up the unit test and then we continue.
Let's pick one example where we want to move 100Km Northeast:

def test_add_delta_in_km_to_coord_100_km(self):
    coord = Coord(53.32055555555556, -1.7297222222222221)

    coord.add_delta_in_km_to_coord(100, 100)

    self.assertEqual(54.21987716147429, coord.lat)
    self.assertEqual(-0.22417190360281203, coord.lon)

Now let's test the calculation of a distance from one coordinate to another:

def test_distance_to_in_meters_one_place(self):
    coord1 = Coord(52.0673599, 5.1102121)
    coord2 = Coord(52.08608282419939, 5.109284354540611)

    self.assertEqual(2082.859, coord1.distance_to_in_meters(coord2))
    self.assertEqual(2.083, coord1.distance_to_in_kilometers(coord2))

Have a look at the other tests. See if you can see what the coordinates refer to and what do they mean!
It will be fun and positive 😉!

6. Conclusion

With this post in my blog, I hope to have been able to help you to get a python package on PyPI.
It isn't really a complicated thing to do as you have seen.
We also see that the Haversine formula is really something to be remembered.
It is everywhere you need to navigate it is so widely used.
Everything we know today in terms of navigation has a connection to these centuries old theories, which are important to remember forever.
The way to move forward to the future is always to remind ourselves of the lessons from the past.
Not only in Earth we see this.
In the future we may be looking at other planes and there establishing our GPS systems.
No matter how small or how big those planets are, the haversines and the works of these three remarkable mathematicians will always be there to remind us that no matter how far we go into the future, everything is important.
Our past is our connection to our present which makes us look into the future.
One day we will need formulas that help us navigate through space with our mobile phones I guess.
In space, between planets there is no curvature, and we can measure distances directly.
We also measure distances in 4D.
At one point in time Earth is closer than ever from Mars and at one other point in time, the exact oppposite happens.

This article has been written with a lot of positivity just to provide you with some guidelines about how to create a public python module and to give you an introduction to the Haversine formula.

If you want to install my module please run:

sudo pip3 install geo_calculator

And if you want to remove it:

pip3 uninstall geo_calculator

The module is availale at PyPI:

geo-calculator:1.0.0

7. Resources

7.1. Books & Papers

fer3 Haversine Navigation

7.2. Websites

If you like algorithms, you'll probably find interesting to know a bit more about tail recursivity, how that led to tail call optimization and the current state of thing. I tell all about that in this video over here:

References

Making reactive applications with a Kitten Care Example

João Esperancinha — Sun, 29 Dec 2024 11:01:47 +0000

Original Photo

At the beginning of 2020, I was trying to make an example to illustrate the benefits of using a Reactive Programming model.
I already had an idea of an application that I wanted to make and because I love cats, I wanted to make an application to register cats found in the wild.
This way cats could be helped by being redirected to a Cat care house in order to find an available carer.
The idea of making the application reactive was to also be able to make a presentation about it. Unfortunately the presentation never took place and I couldn't continue with it.
However, I was able to finish just in time the project I wanted to make.
In this article I will share with you my experience in creating this application.
I find this to be very helpful if you are looking forward to grasping the intricacies of Reactive Programming.

We will together go through how Reactive programming can make our programming models easier and faster.
What we should know is that Reactive programming relies heavily on the observer pattern.
In short, an observer pattern is one where we have two major players:

A publisher - An entity that manages received events
A subscriber - An entity that subscribes to the publisher and does something with the data generated by the publisher.

The source code for this project can be found here:

Kitten House Care

Reactive programming in current times is viewed as one of the best programming paradigms ever.
However, this concept isn't a new concept.
It already exists since the 1960's under a different name: Data Flows.

1. Introduction

1960's

In order to illustrate the way the concept of Reactive programming started, we have to go back to the 1960's in a time capsule.

At that time Jack Dennis and his graduate team at MIT developed concepts surrounding data and how data could be efficiently transmitted.
The concept invented was the concept of Data flows.
Dataflow programming concepts pioneered in this decade because of Jack Dennis inventive character.

The years where this kind of programming gained momentum was in the years 1974 – 1975.
Along these years, the data stream concept and its interactions were further developed.

The data flow ancestors are:

Asynchronous Digital Logic
Control Structures for Parallel Programming
Abstract Models for Concurrent Systems
Theory of Program Schemes
Structured Programming
Functional Programming

What data flows wanted to show is that data can be emitted as a response to a system.

"For every action, there is an equal and opposite reaction."

Newton's third law

What this means is that the idea was also to find a way to work better with a decentralizad system.

Using the traditional sequential, procedural or control paradigms, represented a problem.
This problem was the blocking nature of such paradigms.
The best way solve this was to parallelize, what could be parallelized.
Therefore, the dataflow paradigm finally was settled to be one where serveral black boxes were connected together.
Connections are made by the use of several inputs.
When all the inputs of one black box are satisfied, then the black box runs.

In 1961, a paper, called " A Block Diagram Compiler ". was published with the development of a new programming language.
BLODI, the new language, short for BLOck DIagram, was the first programming language following the data flow paradigms.
Its application was in electronic circuits.
It was designed by John Larry Kelly Jr. (1923-1965), Carol Lochbaum and Victor A. Vyssotsky (1931-2012)

A presentation from Jack Dennis has been written for the MIT and is available directly on:

Dennis Talk

2010

In 2010, Eric Meijer, a Computer Scientist working for Microsoft, coined the term Reactive Programming.
Up until this date Eric Meijer was involved inthe development of different .Net technologies.
These included C#, Visual Basic, LINQ and Volta.
The first framework to ever appear which was referred to as Reactive was the Reactive programming framework in 2009.
It was then called a "Watershed Advance in Asynch Programming".
The reason is very simple.
One of the most important paradigms of Reactive programming is its non-blocking nature.
Non-blocking also means that we try to perform separate small tasks independently and asynchronously.
In 2010, the creation of Reactive Extensions for .NET revolutionized the way we look into software development.
Reactive programming, an old concept, was brought to life.
Currently Reactive Extensions for .NET is actually called Reactive X, and it is available in most common programming languages.
Reactive Programming is commonly referred to as Dataflow programming on steroids.
The reason being is that it's just an extension of the Dataflow programming paradigm.

In November 17th of 2009, Erik Meijer was interviewed on Channel 9 about Rx. The video isn't available online anymore.
This interview used to be available online on their website: Erik-Meijer-Rx-in-15-Minutes

Reactive Manifesto

The Reactive Manifesto was created in 2013 by Jonas Bonér.
This manifesto applies to systems and is many times confused with the idea of reactive programmming.
Both reactive programming and reactive systems want decoupling to happen, they want to run as soon as certain conditions are met and they want to resource to parallelism as much as possible.
In other words these two concepts have in common that they want maximum availability and asynchronicity whenever possible.
This is probably where the similarities end.

A reactive system must obey the following rules or at least explore these as maximum as possible:

Responsive: It needs to respond quickly.
1. The time of the request itself is independent of this.
2. Getting an ok which confirms the request is ongoing is more important than responding with the request complete.
Resilient: It must respond well and support Back-Pressure
1. Use controller Messages
2. Avoid catastrophic failure
Elastic: Automatic Generation of resources
1. Threads
2. Memory
3. CPU
Message Driven: Publisher/Subscriber

Observer Pattern

Is a behavioral design pattern that lets you define a subscription mechanism to notify multiple objects about any events that happen to the object they’re observing.

In other words, we are going to build software with declarative programming instead of imperative programming.

Spring Webflux

Reactive programming basically means that we are programming in a reactive way.
A reactive way, means that calls are not blocked and waiting is something we want to avoid.

From the Mono documentation we get the following:

A Mono is a specialized Publisher that emits at most one item and then (optionally) terminates with an onComplete signal or an onError signal.

in Mono documentation

This means that, and we'll see this later in cation, that Monos are valid for single elements.
This is important to keep in mind because we also want to make sure we can, in some way a stream of elements.
In our example we'll use this to transmit one single Cat at a time.

From the Flux documentaion we have that:

A Flux is a standard Publisher that represents an asynchronous sequence of 0 to N emitted items, optionally terminated by either a completion signal or an error. As in the Reactive Streams spec, these three types of signal translate to calls to a downstream Subscriber’s onNext, onComplete, and onError methods.

in Flux documentaion

In other words this means that we'll use Flux to transmit a list of items.
In our case, a list of CatDtos.

2. Environment Setup

The project we will be looking at is developed in Java 14.

If you have questions about setting up Java 14, then please lookup more information on my Hints&Tricks document.

To be able o run the unit tests for the Kitten Care Project we need a special configuration.
Without it, our Blockhound release version 1.0.3-RELEASE will give this error back:

/Users/jofisaes/.sdkman/candidates/java/14.0.0.hs-adpt/bin/java -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:/Applications/IntelliJ IDEA 2020.2 EAP.app/Contents/lib/idea_rt.jar=52479:/Applications/IntelliJ IDEA 2020.2 EAP.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Applications/IntelliJ IDEA 2020.2 EAP.app/Contents/lib/idea_rt.jar:/Users/jofisaes/.m2/repository/org/junit/platform/junit-platform-launcher/1.5.2/junit-platform-launcher-1.5.2.jar:/Applications/IntelliJ IDEA 2020.2 EAP.app/Contents/plugins/junit/lib/junit5-rt.jar:/Applications/IntelliJ IDEA 2020.2 EAP.app/Contents/plugins/junit/lib/junit-rt.jar:/Users/jofisaes/dev/src/jofisaes/kitten-house-care-parent/kitten-house-care/target/test-classes:/Users/jofisaes/dev/src/jofisaes/kitten-house-care-parent/kitten-house-care/target/classes:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-webflux/2.2.6.RELEASE/spring-boot-starter-webflux-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter/2.2.6.RELEASE/spring-boot-starter-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot/2.2.6.RELEASE/spring-boot-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.2.6.RELEASE/spring-boot-autoconfigure-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.2.6.RELEASE/spring-boot-starter-logging-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/jofisaes/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/jofisaes/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.12.1.jar:/Users/jofisaes/.m2/repository/org/apache/logging/log4j/log4j-api/2.12.1/log4j-api-2.12.1.jar:/Users/jofisaes/.m2/repository/org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar:/Users/jofisaes/.m2/repository/jakarta/annotation/jakarta.annotation-api/1.3.5/jakarta.annotation-api-1.3.5.jar:/Users/jofisaes/.m2/repository/org/yaml/snakeyaml/1.25/snakeyaml-1.25.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-json/2.2.6.RELEASE/spring-boot-starter-json-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.10.3/jackson-databind-2.10.3.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.10.3/jackson-annotations-2.10.3.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.10.3/jackson-core-2.10.3.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.3/jackson-datatype-jdk8-2.10.3.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.10.3/jackson-datatype-jsr310-2.10.3.jar:/Users/jofisaes/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.10.3/jackson-module-parameter-names-2.10.3.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-reactor-netty/2.2.6.RELEASE/spring-boot-starter-reactor-netty-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/io/projectreactor/netty/reactor-netty/0.9.6.RELEASE/reactor-netty-0.9.6.RELEASE.jar:/Users/jofisaes/.m2/repository/io/netty/netty-codec-http/4.1.48.Final/netty-codec-http-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-common/4.1.48.Final/netty-common-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-buffer/4.1.48.Final/netty-buffer-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-transport/4.1.48.Final/netty-transport-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-codec/4.1.48.Final/netty-codec-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-codec-http2/4.1.48.Final/netty-codec-http2-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-handler/4.1.48.Final/netty-handler-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-resolver/4.1.48.Final/netty-resolver-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-handler-proxy/4.1.48.Final/netty-handler-proxy-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-codec-socks/4.1.48.Final/netty-codec-socks-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/io/netty/netty-transport-native-epoll/4.1.48.Final/netty-transport-native-epoll-4.1.48.Final-linux-x86_64.jar:/Users/jofisaes/.m2/repository/io/netty/netty-transport-native-unix-common/4.1.48.Final/netty-transport-native-unix-common-4.1.48.Final.jar:/Users/jofisaes/.m2/repository/org/glassfish/jakarta.el/3.0.3/jakarta.el-3.0.3.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-validation/2.2.6.RELEASE/spring-boot-starter-validation-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/jakarta/validation/jakarta.validation-api/2.0.2/jakarta.validation-api-2.0.2.jar:/Users/jofisaes/.m2/repository/org/hibernate/validator/hibernate-validator/6.0.18.Final/hibernate-validator-6.0.18.Final.jar:/Users/jofisaes/.m2/repository/org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar:/Users/jofisaes/.m2/repository/com/fasterxml/classmate/1.5.1/classmate-1.5.1.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-web/5.2.5.RELEASE/spring-web-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-beans/5.2.5.RELEASE/spring-beans-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-webflux/5.2.5.RELEASE/spring-webflux-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/io/projectreactor/reactor-core/3.3.4.RELEASE/reactor-core-3.3.4.RELEASE.jar:/Users/jofisaes/.m2/repository/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar:/Users/jofisaes/.m2/repository/org/synchronoss/cloud/nio-multipart-parser/1.1.0/nio-multipart-parser-1.1.0.jar:/Users/jofisaes/.m2/repository/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar:/Users/jofisaes/.m2/repository/org/synchronoss/cloud/nio-stream-storage/1.1.3/nio-stream-storage-1.1.3.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-web/2.2.6.RELEASE/spring-boot-starter-web-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/2.2.6.RELEASE/spring-boot-starter-tomcat-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.33/tomcat-embed-core-9.0.33.jar:/Users/jofisaes/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.33/tomcat-embed-el-9.0.33.jar:/Users/jofisaes/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.33/tomcat-embed-websocket-9.0.33.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-webmvc/5.2.5.RELEASE/spring-webmvc-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-aop/5.2.5.RELEASE/spring-aop-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-context/5.2.5.RELEASE/spring-context-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-expression/5.2.5.RELEASE/spring-expression-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/projectlombok/lombok/1.18.12/lombok-1.18.12.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-starter-test/2.2.6.RELEASE/spring-boot-starter-test-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-test/2.2.6.RELEASE/spring-boot-test-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/boot/spring-boot-test-autoconfigure/2.2.6.RELEASE/spring-boot-test-autoconfigure-2.2.6.RELEASE.jar:/Users/jofisaes/.m2/repository/com/jayway/jsonpath/json-path/2.4.0/json-path-2.4.0.jar:/Users/jofisaes/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar:/Users/jofisaes/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar:/Users/jofisaes/.m2/repository/org/ow2/asm/asm/5.0.4/asm-5.0.4.jar:/Users/jofisaes/.m2/repository/jakarta/xml/bind/jakarta.xml.bind-api/2.3.3/jakarta.xml.bind-api-2.3.3.jar:/Users/jofisaes/.m2/repository/jakarta/activation/jakarta.activation-api/1.2.2/jakarta.activation-api-1.2.2.jar:/Users/jofisaes/.m2/repository/org/junit/vintage/junit-vintage-engine/5.5.2/junit-vintage-engine-5.5.2.jar:/Users/jofisaes/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/jofisaes/.m2/repository/org/mockito/mockito-junit-jupiter/3.1.0/mockito-junit-jupiter-3.1.0.jar:/Users/jofisaes/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/jofisaes/.m2/repository/org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar:/Users/jofisaes/.m2/repository/net/bytebuddy/byte-buddy/1.10.8/byte-buddy-1.10.8.jar:/Users/jofisaes/.m2/repository/net/bytebuddy/byte-buddy-agent/1.10.8/byte-buddy-agent-1.10.8.jar:/Users/jofisaes/.m2/repository/org/objenesis/objenesis/2.6/objenesis-2.6.jar:/Users/jofisaes/.m2/repository/org/skyscreamer/jsonassert/1.5.0/jsonassert-1.5.0.jar:/Users/jofisaes/.m2/repository/com/vaadin/external/google/android-json/0.0.20131108.vaadin1/android-json-0.0.20131108.vaadin1.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-core/5.2.5.RELEASE/spring-core-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-jcl/5.2.5.RELEASE/spring-jcl-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/springframework/spring-test/5.2.5.RELEASE/spring-test-5.2.5.RELEASE.jar:/Users/jofisaes/.m2/repository/org/xmlunit/xmlunit-core/2.6.4/xmlunit-core-2.6.4.jar:/Users/jofisaes/.m2/repository/io/projectreactor/tools/blockhound-junit-platform/1.0.3.RELEASE/blockhound-junit-platform-1.0.3.RELEASE.jar:/Users/jofisaes/.m2/repository/io/projectreactor/tools/blockhound/1.0.3.RELEASE/blockhound-1.0.3.RELEASE.jar:/Users/jofisaes/.m2/repository/org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar:/Users/jofisaes/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar:/Users/jofisaes/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/jofisaes/.m2/repository/org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar:/Users/jofisaes/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar:/Users/jofisaes/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar:/Users/jofisaes/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar:/Users/jofisaes/.m2/repository/org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar:/Users/jofisaes/.m2/repository/org/assertj/assertj-core/3.13.2/assertj-core-3.13.2.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit5 org.jesperancinha.housing.controller.CatControllerImplTest,getFullAllCatsNonReacive_whenCall_Blocking
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Exception in thread "main" java.util.ServiceConfigurationError: org.junit.platform.launcher.TestExecutionListener: Provider reactor.blockhound.junit.platform.BlockHoundTestExecutionListener could not be instantiated
    at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:584)
    at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:806)
    at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:724)
    at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1396)
    at java.base/java.lang.Iterable.forEach(Iterable.java:74)
    at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:94)
    at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:67)
    at com.intellij.junit5.JUnit5IdeaTestRunner.createListeners(JUnit5IdeaTestRunner.java:46)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:31)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: java.lang.ExceptionInInitializerError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
    at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:782)
    ... 9 more
Caused by: java.lang.IllegalStateException: The instrumentation have failed.
It looks like you're running on JDK 13+.
You need to add '-XX:+AllowRedefinitionToAddDeleteMethods' JVM flag.
See https://github.com/reactor/BlockHound/issues/33 for more info.
    at reactor.blockhound.BlockHound$Builder.testInstrumentation(BlockHound.java:435)
    at reactor.blockhound.BlockHound$Builder.install(BlockHound.java:401)
    at reactor.blockhound.BlockHound.install(BlockHound.java:94)
    at reactor.blockhound.junit.platform.BlockHoundTestExecutionListener.<clinit>(BlockHoundTestExecutionListener.java:19)
    ... 15 more

Process finished with exit code 1

As it mentions in the exception we only need to add this flag in our IntelliJ (or whatever IDE you are using, and of course this is also valid via de command line): -XX:+AllowRedefinitionToAddDeleteMethods.

Bear in mind a few important things about Blockhound:

It is still a test tool in the making.
Version 1.0.3-RELEASE work very well with Java 14
Blockhound will detect blocking calls which makes simulation of pausing difficult because methods like wait and Thread.sleep will block the main Thread.
Blocking of the main thread always results in a blocking thread detection. Making a unit test to check the response data will fail.
During a simple Blockhoud test, the simulated blocking calls outside the callback scope aren't going to be detected.

3. Implementation

To implement with WebFlux, we need to start with dependencies.
In this case we are going to use Maven:

We need of course the WebFlux library:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
    <version>2.2.6.RELEASE</version>
</dependency>

Then we need the framework to run our application and get support for our MVC pattern:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>2.2.6.RELEASE</version>
</dependency>

Having these dependencies in our maven repository configuration is what's important.
In our case we are using the maven compile plugin in order to compile our project with the JDK 14:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>14</source>
                <target>14</target>
            </configuration>
        </plugin>
    </plugins>
</build>

Let's take a top to bottom strategy.
First we implement our REST services.
This is all implement in our Controller class CatController.
Our application will follow the typical MVC model, but instead of the traditional MVC way, we will implement it in a reactive way.

First we need a data model and for that let's have a look at our first kitten:

{
  "id": 1,
  "name": "Lania",
  "color": "orange",
  "species": "Katachtigen",
  "pattern": "stripped",
  "age": 4,
  "formerOwners": [
    1
  ],
  "careCenters": [
    1
  ]
}

What we have here, is the name "Lania" for this cat.
The way to interpret this data is:

We have a cat named Mountain.
The cats color is orange.
The cat species is Katachtigen.
The hair pattern is stripped.
The cat is 4 years old
The cat never had any owners.

{
  "id": 2,
  "name": "Mit",
  "color": "black and white",
  "species": "Katachtigen",
  "pattern": "spotted",
  "age": 9,
  "formerOwners": [
    2
  ],
  "careCenters": [
    1
  ]
}

And this is another cat called Mit. Mat is another cat who differs in that:

Mat is 9 years old
He is Black and White
The pattern on his hair is spotted.

We also need the data of the care center for the cats.
This is one example.
This a list of centers.
In our case, there is only one element in it:

[
  {
    "id": 1,
    "name": "Nieuwegein Kitten Center",
    "address": "Kittenstraat",
    "refNumber": "23ABC",
    "city": "Nieuwegein",
    "postCode": "9999CC",
    "country": "Nederland"
  }
]

Finally, we want the information of potential owners.
We make also an array of available owners:

[
  {
    "id": 1,
    "name": "Eng. João Esperancinha",
    "address": "Verrekijkers",
    "rating": 10
  },
  {
    "id": 2,
    "name": "Eng. Erwin Zieligofski",
    "address": "The swamp",
    "rating": 0
  }
]

Here we have two owners.
Top keep it simple, we are only interested in three properties for these owners.
These are their name, adress and rating.
So, we can see here that the best owner is of course João Esperancinha and the bad owner is Erwin Zieligovsky.

Now that we have some more understanding of the data we are going to need, we can now have a look at the REST implementation.
This is an interface class to our REST controller:

@RestController
@RequestMapping("/cats")
public interface CatController {
    @GetMapping
    Flux<CatDto> getAllCats();
    @GetMapping("/full")
    Flux<CatDto> getFullAllCats();
    @GetMapping("/{catId}")
    Mono<CatDto> getCatByIdI(@PathVariable Long catId);
    @GetMapping("/full/{catId}")
    Mono<CatDto> getFullCatById(@PathVariable Long catId);
}

We will read data from our cats database via path /cats.
Here we have the two publisher types available in the Spring Webflux library.
These are Mono and Flux.

We'll look into Mono and Flux in detail letter.
We are now only concerned in getting the architecture all together.

For our application, we just need to know that we want to get a complete list of available cats, a cat by id and these last two methods with complete data.

Let's now have a look at the REST service implementation interface:

@RestController
@RequestMapping("/cats")
public class CatControllerImpl implements CatController {
    private final CatService catService;
    public CatControllerImpl(CatService catService) {
        this.catService = catService;
    }
    public Flux<CatDto> getAllCats() {
        return this.catService.getAllCats();
    }
    public Flux<CatDto> getFullAllCats() {
        return this.catService.getFullAllCats();
    }
    public Mono<CatDto> getCatByIdI(Long catId) {
        return this.catService.getCatById(catId);
    }
    public Mono<CatDto> getFullCatById(Long catId) {
        return this.catService.getFullCatById(catId);
    }
}

The implementation of the RestController doesn't differ that much from the implementation of a common MVC REST controller.
We can see that the difference lies in the fact that instead of returning something like a ResponseEntity, we are instead return our publishers.
All of our REST controller methods are calling methods of the CatService. The CatService is where our business logic lies.
For the current application, the business logic isn't really that much and works more like a mapper between the DTO's and the data objects.

Our CatService interface is very simple:

public interface CatService {
    Mono<CatDto> getCatById(Long id);
    Mono<CatDto> getFullCatById(Long id);
    Flux<CatDto> getFullAllCats();
    Flux<CatDto> getAllCats();
}

Its implementation is only a bit more complictated:

@Service
public class CatServiceImpl implements CatService {

    private final CatRepository catRepository;

    private final OwnerRepository ownerRepository;

    private final CareCenterRepository careCenterRepository;

    public CatServiceImpl(CatRepository catRepository, OwnerRepository ownerRepository,
        CareCenterRepository careCenterRepository) {
        this.catRepository = catRepository;
        this.ownerRepository = ownerRepository;
        this.careCenterRepository = careCenterRepository;
    }

    @Override
    public Mono<CatDto> getCatById(Long id) {
        return catRepository.getCatById(id).map(CatConverter::toDto);
    }

    @Override
    public Mono<CatDto> getFullCatById(Long id) {
        return catRepository.getCatById(id).map(cat -> {
            CatDto catDto = CatConverter.toDto(cat);
            return Mono.zip(ownerRepository.getOwnersByIds(cat.getFormerOwners())
                .map(owners -> {
                    owners.forEach(owner -> catDto.getFormerOwners().add(OwnerConverter.toDto(owner)));
                    return owners;
                })
                .subscribeOn(Schedulers.parallel()), careCenterRepository.getCareCentersByIds(cat.getCareCenters())
                .map(careCenters -> {
                    careCenters
                        .forEach(careCenter -> catDto.getCareCenters().add(CareCenterConverter.toDto(careCenter)));
                    return careCenters;
                })
                .subscribeOn(Schedulers.parallel()), (owners, cares) -> catDto)
                .subscribeOn(Schedulers.parallel());
        }).flatMap(Mono::from).subscribeOn(Schedulers.parallel());
    }

    @Override
    public Flux<CatDto> getFullAllCats() {
        return Flux.merge(getFullCatById(1L), getFullCatById(2L));
    }

    @Override
    public Flux<CatDto> getAllCats() {
        return Flux.merge(getCatById(1L), getCatById(2L));
    }
}

On this occasion, the complexity of our implementations has increased considerably.
The reason being is that our REST controller is dependent of a few separate database entities.
It is important to remind ourselves that we don't actually have a database.
The database is being purely emulated by the use of the file system with static data.
This is also the reason why some id's have been hardcoded.

Before looking at what the CatService can give us. let's look at all the repositories we are going to use.
Cats will be managed via CareCenters and will have a sort of legal guardian registered as an Owner.

For the care centers we first define our interface:

public interface CareCenterRepository {
    Mono<CareCenter> getCareCenterById(Long id);
    Mono<List<CareCenter>> getCareCentersByIds(List<Long> careCenters);
}

And then we have our implementation:

@Repository
public class CareCenterRepositoryImpl implements CareCenterRepository {

    private final CareCenter[] careCenters;

    public CareCenterRepositoryImpl(ObjectMapper objectMapper) throws IOException {
        this.careCenters = objectMapper
            .readValue(getClass().getResourceAsStream("/carecenters.json"), CareCenter[].class);

    }

    @Override
    public Mono<CareCenter> getCareCenterById(Long id) {
        return Mono.fromCallable(
            () -> Arrays.stream(careCenters).filter(careCenter -> careCenter.getId().equals(id)).findFirst()
                .orElse(null));
    }

    @Override
    public Mono<List<CareCenter>> getCareCentersByIds(List<Long> careCenteIds) {
        return Mono.fromCallable(
            () -> Arrays.stream(careCenters).filter(careCenter -> careCenteIds.contains(careCenter.getId()))
                .collect(Collectors.toList()));
    }
}

We create our repository just like all the other repositories.
On a first step we initialize our cached data by reading the data files in resources.
Then we are left with two method: getCareCenterById and getCareCentersByIds.

We have mentioned before that Flux stands for lists.
Our getCareCentersByIds doesn't return a Flux though.
Instead, it returns a Mono of a List of care centers.
This is important to understand.
We could have had returned it this way:

@Override
public Flux<CareCenter> getCareCentersByIds(List<Long> careCenteIds) {
    return Flux.fromStream(
        () -> Arrays.stream(careCenters).filter(careCenter -> careCenteIds.contains(careCenter.getId())));
}

This way we would have gotten a Flux of CareCenter's.
We will have this more clear further down this article.
For the moment, we just need to know that a Flux works pretty much as a stream, which, dependeding on what to do, can make some zip operations much more complicated than they should be.
Furthermore, it does not add any performance value some of these cases and in others, the zip operation is just impossible.

Let's move on to our Repository interface:

public interface OwnerRepository {
    Mono<Owner> getOwnerById(Long id);
    Mono<List<Owner>> getOwnersByIds(List<Long> formerOwners);
}

For the same reasons we said for the CareCenterRepository, we are using a Mono of a List here instead of a Flux.
This is its implementation:

@Repository
public class OwnerRepositoryImpl implements OwnerRepository {
    private final Owner[] owners;
    public OwnerRepositoryImpl(ObjectMapper objectMapper) throws IOException {
        this.owners = objectMapper.readValue(getClass().getResourceAsStream("/owners.json"), Owner[].class);
    }
    @Override
    public Mono<Owner> getOwnerById(Long id) {
        return Mono.fromCallable(
            () -> Arrays.stream(owners).filter(owner -> owner.getId().equals(id)).findFirst().orElse(null));
    }
    @Override
    public Mono<List<Owner>> getOwnersByIds(List<Long> formerOwners) {
        return Mono.fromCallable(() -> Arrays.stream(owners).filter(owner -> formerOwners.contains(owner.getId())).collect(
            Collectors.toList()));
    }
}

Finally we need top just have a quick look at the CatRepository:

public interface CatRepository {
    Mono<Cat> getCatById(Long id);
}

And its implementation:


@Repository
public class CatRepositoryImpl implements CatRepository {
    private final Cat cat1;
    private final Cat cat2;
    public CatRepositoryImpl(ObjectMapper objectMapper) throws IOException {
        this.cat1 = objectMapper.readValue(getClass().getResourceAsStream("/cat1.json"), Cat.class);
        this.cat2 = objectMapper.readValue(getClass().getResourceAsStream("/cat2.json"), Cat.class);
    }
    @Override
    public Mono<Cat> getCatById(Long id) {
        return Mono.fromCallable(() -> {
            if (id.intValue() == 1L) {
                return cat1;
            }
            if (id.intValue() == 2L) {
                return cat2;
            }
            return null;
        });
    }
}

This is an extremely hardcoded repository, which is done so, just to support our tests and experiments.
Our repository implementation provides only one method: getCatById.
With this method we can get the cats we want by id.

Looking back at the CatServiceImpl we now know enough to start analysing this class.

Our service provides methods that are important to unerstand WebFlux.
We can start by checking method getCatById:

@Override
public Mono<CatDto> getCatById(Long id) {
    return catRepository.getCatById(id).map(CatConverter::toDto);
}

What we are doing in this method is just making sure that we separate the back-end layer from the layer facing the front-end.
Just as in tradition MVC architectures, we separate our DTO (Data Transfer Object) from the persistence layer.
In our specific case, the conversion seems poinless, but this is because we are not transforming the data that much.
Normally, for more complicated cases, the data streamed to the public domain is usually either filtered, enriched, or in any other way changed.

For this case, we can clearly see that we are getting a Cat.
At least we will be getting a Cat in the front end.
However, this isn't exactly the case.
Although we eventually get a Cat in the front end, our Cat is going to be published to the front-end.
We'll see this in further detail along this article.

For now let's have a better look at what's happening in the back-end in our repoitory layer:

@Override
public Mono<Cat> getCatById(Long id) {
    return Mono.fromCallable(() -> {
        if (id.intValue() == 1L) {
            return cat1;
        }
        if (id.intValue() == 2L) {
            return cat2;
        }
        return null;
    });
}

We are returning a Mono of a cat and not the cat iself.
What this means is that, we are returning an implementation of a publishing function in the callback of the REST call.
By doing this, we are ensuring that, this hardcoded if, else statements are only executed after the callback.

We are now going to have a look at a zip method provided by Flux:

@Override
public Mono<CatDto> getFullCatById(Long id) {
    return catRepository.getCatById(id).map(cat -> {
        CatDto catDto = CatConverter.toDto(cat);
        return Mono.zip(ownerRepository.getOwnersByIds(cat.getFormerOwners())
            .map(owners -> {
                owners.forEach(owner -> catDto.getFormerOwners().add(OwnerConverter.toDto(owner)));
                return owners;
            })
            .subscribeOn(Schedulers.parallel()), careCenterRepository.getCareCentersByIds(cat.getCareCenters())
            .map(careCenters -> {
                careCenters
                    .forEach(careCenter -> catDto.getCareCenters().add(CareCenterConverter.toDto(careCenter)));
                return careCenters;
            })
            .subscribeOn(Schedulers.parallel()), (owners, cares) -> catDto)
            .subscribeOn(Schedulers.parallel());
    }).flatMap(Mono::from).subscribeOn(Schedulers.parallel());
}

This is method getFullCatById. We are getting a full cat!
A full cat in this code means that we'll get a cat by doing three essential requests in a reactive way:

Collect the Cat data
Collect the Owner data
Collect the CatCenter data

Points 2. and 3. need to be run in a parallel way.
However both of them depend on the Cat data which we should get first.

This is how we want to do this:

The original web framework included in the Spring Framework, Spring Web MVC, was purpose-built for the Servlet API and Servlet containers. The reactive-stack web framework, Spring WebFlux, was added later in version 5.0. It is fully non-blocking, supports Reactive Streams back pressure, and runs on such servers as Netty, Undertow, and Servlet 3.1+ containers.
Both web frameworks mirror the names of their source modules (spring-webmvc and spring-webflux) and co-exist side by side in the Spring Framework. Each module is optional. Applications can use one or the other module or, in some cases, both — for example, Spring MVC controllers with the reactive WebClient.

in Web on Reactive Stack

The zip method from Mono, allows to start two publishing callbacks separately in parallel.
For this to happen we can use many types in many different cases.
For our case we are going to zip two Mono's together.

The first step is to get our Cat out of the database.
Using instruction catRepository.getCatById(id).map(cat -> { we get our cat from the database.
Recalling from the JSON data we are using for our database, we know that out cats only have references to their owners and care centers.
This is exactly what in a real life ER model database happens.
In our example, for some reason, this data is allocated in different systems and we cannot benefit from things like @ManyToOne, @ManyToMany or @OneToMany.
These decorators are only available via Hibernate and they work very well in single databases, but we cannot used them if we consider a multi tiered system.
Now that we have our Cat, we can now benefit from the zip method.

For this we call return Mono.zip(ownerRepository.getOwnersByIds(cat.getFormerOwners()).
Now we have to add two Monos as parameters.
These two Mono's plus the Mono where they are being called from, form a composed Mono, which can be handle a a single Mono publisher as a returned instance.

Our first Mono is this:

ownerRepository.getOwnersByIds(cat.getFormerOwners())
.map(owners -> {
    owners.forEach(owner -> catDto.getFormerOwners().add(OwnerConverter.toDto(owner)));
    return owners;
})
.subscribeOn(Schedulers.parallel())

In this method we are letting the running thread know that we are going to get all the Owner's and create DTO's with them.
We are also letting it know that when Netty subscribes to them, it will subscribe on parallel Schedulers.
This is what thee Schedulers.parallel() instruction is for.
We add all the Owner's that we find to our catDto we want to return.

The second Mono follows that same pattern and we use it to get our CareCenter's.

careCenterRepository.getCareCentersByIds(cat.getCareCenters())
.map(careCenters -> {
    careCenters
        .forEach(careCenter -> catDto.getCareCenters().add(CareCenterConverter.toDto(careCenter)));
    return careCenters;
})
.subscribeOn(Schedulers.parallel())

We add all the CareCenter's that we find to our catDto we want to return.

Finally we instruct the result Mono to also run in parallel:

.subscribeOn(Schedulers.parallel()), (owners, cares) -> catDto).subscribeOn(Schedulers.parallel());

Finally, maybe you have noticed it or maybe not, but we would be actually returning a Mono<Mono<CatDto> publisher instead of what we want.
This is because of our zip call that is running within yet another Mono.

Webflux has a very interesting way of dealing with this and that is running the:

 }).flatMap(Mono::from).subscribeOn(Schedulers.parallel());

By using flatMap, we are making sure that our Mono<Mono<CatDto> is optimized and converted to a Mono<Mono<CatDto instance.

At the end of our CatServiceImpl we still have the following two methods:

@Override
public Flux<CatDto> getFullAllCats() {
    return Flux.merge(getFullCatById(1L), getFullCatById(2L));
}

@Override
public Flux<CatDto> getAllCats() {
    return Flux.merge(getCatById(1L), getCatById(2L));
}

This is an illustration of a possible use of the method merge of the Flux publisher.
This method receives an array in the form of a varargs argument.
This way, we are allowed to pass several Mono's to this method.
The result is a Flux.
On this occasion we definitelly make use of the Flux in its most common form which is as a stream of different objects.

4. REST endpoints test

Let's have a quick run over what we have studyied so far.

4.1 Returning a simple Mono

We have seen this in method getCatById of the CatServiceImpl:

curl http://localhost:8080/cats/1

{
  "name": "Lania",
  "color": "orange",
  "species": "Katachtigen",
  "pattern": "stripped",
  "age": 4,
  "formerOwners": [],
  "careCenters": []
}

Let's try for another cat:

curl http://localhost:8080/cats/1

{
  "name": "Mit",
  "color": "black and white",
  "species": "Katachtigen",
  "pattern": "spotted",
  "age": 9,
  "formerOwners": [],
  "careCenters": []
}

4.2 Returning the complete cat info

Let's now see what our full method, getFullCatById returns for Cat 1:

curl http://localhost:8080/cats/full/1

{
  "name": "Lania",
  "color": "orange",
  "species": "Katachtigen",
  "pattern": "stripped",
  "age": 4,
  "formerOwners": [
    {
      "name": "Eng. João Esperancinha",
      "addres": "Verrekijkers"
    }
  ],
  "careCenters": [
    {
      "name": "Nieuwegein Kitten Center",
      "address": "Kittenstraat",
      "refNumber": "23ABC",
      "city": "Nieuwegein",
      "postCode": "9999CC",
      "country": "Nederland"
    }
  ]
}

And now for cat 2:

curl http://localhost:8080/cats/full/2

{
  "name": "Mit",
  "color": "black and white",
  "species": "Katachtigen",
  "pattern": "spotted",
  "age": 9,
  "formerOwners": [
    {
      "name": "Eng. Erwin Zieligofski",
      "addres": "The swamp"
    }
  ],
  "careCenters": [
    {
      "name": "Nieuwegein Kitten Center",
      "address": "Kittenstraat",
      "refNumber": "23ABC",
      "city": "Nieuwegein",
      "postCode": "9999CC",
      "country": "Nederland"
    }
  ]
}

4.3 All cats

First we look at the simple way of getting the Cat's info using method getAllCats:

curl http://localhost:8080/cats

[
  {
    "name": "Lania",
    "color": "orange",
    "species": "Katachtigen",
    "pattern": "stripped",
    "age": 4,
    "formerOwners": [],
    "careCenters": []
  },
  {
    "name": "Mit",
    "color": "black and white",
    "species": "Katachtigen",
    "pattern": "spotted",
    "age": 9,
    "formerOwners": [],
    "careCenters": []
  }
]

We get no info about the owners, nor do we get info about the care centers.
Now let's see what happend with getFullAllCats:

curl http://localhost:8080/cats/full

[
  {
    "name": "Mit",
    "color": "black and white",
    "species": "Katachtigen",
    "pattern": "spotted",
    "age": 9,
    "formerOwners": [
      {
        "name": "Eng. Erwin Zieligofski",
        "addres": "The swamp"
      }
    ],
    "careCenters": [
      {
        "name": "Nieuwegein Kitten Center",
        "address": "Kittenstraat",
        "refNumber": "23ABC",
        "city": "Nieuwegein",
        "postCode": "9999CC",
        "country": "Nederland"
      }
    ]
  },
  {
    "name": "Lania",
    "color": "orange",
    "species": "Katachtigen",
    "pattern": "stripped",
    "age": 4,
    "formerOwners": [
      {
        "name": "Eng. João Esperancinha",
        "addres": "Verrekijkers"
      }
    ],
    "careCenters": [
      {
        "name": "Nieuwegein Kitten Center",
        "address": "Kittenstraat",
        "refNumber": "23ABC",
        "city": "Nieuwegein",
        "postCode": "9999CC",
        "country": "Nederland"
      }
    ]
  }
]

5. A forged identity case

Let's picture this case in the form of a story:

Erwin Zieligovsky was a team leader in a production center.
One of the things that Erwin hated the most was being called Edwin.
Do you see the difference? It's small d.
This d would frequently enrage Erwin, up to the point that with the powers given to him, he would be responsible for people getting fired.
All of this because of that "offensive d instead of the "heavenly" r.
The Erwin was also known for having fits and rage attacks, especially around cats.
Erwin hated cats.
The curious thing though is that Erwin loved to hate cats and loved to show superiority even to cats.
The good thing though, is that the Cat care houses knew about Erwin and so he was always denied access when applying to adopt a cat.
One day, when renewing his identity card, he realized that the government had made a mistake.
Erwin was now called Edwin!
It was in his identiy card!
After fits of anger, Erwin thought about one thing.
With the new identity card, he would get another chance of adopting a kitten!
Erwin knew about the Cat care centers and how their software wasn't safe.

Let's see how Erwin tried to do this.
In the OwnerServiceImpl class, there is a new method called checkLiabiliy:

@Override
public Mono<String> checkLiability(String name) {
    return ownerRepository.checkLiability(name);
}

In the repository in class OwnerRepositoryImpl, it becomes more complex:

@Override
public Mono<String> checkLiability(String address) {
    return Flux
        .fromStream(() -> Arrays.stream(owners).filter(owner -> owner.getAddress().equalsIgnoreCase(address)))
        .reduce((owner, owner2) -> {
            if (owner.getRating() > owner2.getRating()) {
                return owner2;
            }
            return owner;
        }).map(owner -> owner.getRating() <= 0 ?
            "NOK" :
            "OK");
}

So what we are doing her is to check if the owner is liable to have a cat.
Since Owner's are registered forever in the system, we can have a look if the addresses match.
The way it works is in that we first find if there are any owners in that address.
Then if there are, we check if their rating has gone lower than 0.
If so, then the owner get's a "NOK". If not, the owner gets an "OK".

We can test this also via unit tests:

class OwnerRepositoryImplTest {
    private final OwnerRepository ownerRepository = new OwnerRepositoryImpl(new ObjectMapper());

    OwnerRepositoryImplTest() throws IOException {
    }

    @Test
    void givenLiable_checkLiability_Ok() {
        Mono<String> verrekijkers = ownerRepository.checkLiability("Verrekijkers");
        final String block = verrekijkers.block();
        assertThat(block).isEqualTo("OK");
    }

    @Test
    void givenNonLiable_checkLiability_NOk() {
        Mono<String> the_swamp = ownerRepository.checkLiability("The Swamp");
        final String block = the_swamp.block();
        assertThat(block).isEqualTo("NOK");
    }
}

The point of this last bit is to see how we can use lambdas to implement publishers with extra functionalities just like Java lambdas already do.

And so contrary to Erwin's expectations, the new identity card having Edwin in it, didn't save the day.
So now, poor Erwin, will have to renew his card and still live frustrated because once again he was denied the right to adopt a cat.

6. Testing with Blockhound

Blockhoud is a library developed quite recently and is available in GitHub

Blockhound

The developments on GitHub for this library started at the end of 2018.

Blockound uses markers placed on threads to determine if they are blocking or not:

BlockHound will transparently instrument the JVM classes and intercept blocking calls (e.g. IO) if they are performed from threads marked as "non-blocking operations only" (ie. threads implementing Reactor's NonBlocking marker interface, like those started by Schedulers.parallel()). If and when this happens (but remember, this should never happen!😜), an error will be thrown.

in Blockhound @ GitHub

First step to do is to add the necessary libraries to our project.
First we need our test framework for Spring Boot:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
    <version>2.2.6.RELEASE</version>
</dependency>

Given that we are running JUnit tests, we need the JUnit support:

<dependency>
    <groupId>io.projectreactor.tools</groupId>
    <artifactId>blockhound-junit-platform</artifactId>
    <version>1.0.2.RELEASE</version>
    <scope>test</scope>
</dependency>

Finally we need the Blockhound core:

<dependency>
    <groupId>io.projectreactor.tools</groupId>
    <artifactId>blockhound</artifactId>
    <version>1.0.2.RELEASE</version>
    <scope>test</scope>
</dependency>

This way we can test all our calls in class CatControllerImplTest:

@SpringBootTest(webEnvironment = RANDOM_PORT)
@ExtendWith(SpringExtension.class)
class CatControllerImplTest {
    final RestTemplate restTemplate = new RestTemplate();
    @Autowired
    private CatControllerImpl catController;
    @LocalServerPort
    private int port;
    static {
        BlockHound.install();
    }
    @Test
    void getCatByIdI_whenCall_nonBlocking() {
        Mono.delay(Duration.ofMillis(1))
                .doOnNext(it -> catController.getCatByIdI(1L))
                .block();
    }
    @Test
    void getFullCatById_whenCall_nonBlocking() {
        Mono.delay(Duration.ofMillis(1))
                .doOnNext(it -> catController.getFullCatById(1L))
                .block();
    }
    @Test
    void getAllCats_whenCall_nonBlocking() {
        Mono.delay(Duration.ofMillis(1))
                .doOnNext(it -> catController.getAllCats())
                .block();
    }
    @Test
    void getFullAllCats_whenCall_nonBlocking() {
        Mono.delay(Duration.ofMillis(1))
                .doOnNext(it -> catController.getFullAllCats())
                .block();
    }
(...)

We need to pay special attention to this section of the code:

static {
    BlockHound.install();
}

This is where Blockhound gets installed in order to be able to run the unit tests.
We can find multiple ways opf configuring this.
For this article we only need Blockhound.install().
Please bear in mind that BlockHound is Java Agent installed on a JVM level.

BlockHound is a Java Agent.
It instruments the pre-defined set of blocking methods (see customization) in the JVM and adds a special check before calling the callback (see Blocking call decision).

in How it works @ Blockhound GitHub repo

All of these @Test's follow the same test procedure.
Taking getFullAllCats_whenCall_nonBlocking as an example:

@Test
void getFullAllCats_whenCall_nonBlocking() {
    Mono.delay(Duration.ofMillis(1))
            .doOnNext(it -> catController.getFullAllCats())
            .block();
}

The point of Duration.ofMillis(1) is just a standard way of configuring Blockhound.
There doesn't seem to be any difference in the unit test runtime, if we change the ofMillis parameter.
What does makes a difference is the call to block().
Once we do this, we will be triggering the test to see if our method is blocking or not.

This same procedure applies to our OwnerControllerTest:

@Test
void checkLiability_whenCallPositive_nonBlocking() {
    Mono.delay(Duration.ofMillis(1))
        .doOnNext(it -> ownerController.checkLiability("Verrekijkers"))
        .block();
}

@Test
void checkLiability_whenCallNegative_nonBlocking() {
    Mono.delay(Duration.ofMillis(1))
        .doOnNext(it -> ownerController.checkLiability("The swamp"))
        .block();
}

These two extra tests check if our checkLiability calls are blocking or not.

We are almost at the end of our article, but before we continue to the conclusion we still need to see the difference.
At the moment it is probably not clear what is the difference in practical terms between making a blocking call and a reactive call.
If we make a unit test with the code described above, we will realize that Blockhound will still not detect a potential blocking method.
We need to test this somehow.
In our example, we are using the file system as our database.
Unfortunatelly for our experiments we run into two problems here:

Accessing the file system doesn't necessarily marks the thread as blocking.
Using Flux or Mono, doesn't automatically make the method non-blocking.
Using wait or Thread.sleep automatically marks the method as blockign regardless of running with WebFlux or not.

However, we do know that Blockhound will not generate errors if we just run the detection procedure and our simulation calls are implemented in the publisher instead of the main callback.

For this, I have created two extra methods in the CatControllerImpl class:

public Flux<CatDto> getFullAllCatsReactiveForTest() {
    return this.catService.getFullAllCats().map(catDto -> {
        try {
            Thread.sleep(1);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return catDto;
    });
}

public List<CatDto> getFullAllCatsNonReactive() {
    try {
        Thread.sleep(1);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    return this.catService.getFullAllCatsNonReactive();
}

The first method, getFullAllCatsReactiveForTest, is a reactive method.
We don't make the unit tests to test its result.
Instead we are only going to use Blockhound to see if our method is blocking or not.
The second method, getFullAllCatsNonReactive, is a blocking method.
The Thread.sleep, occurs during the callback scope.
This means that Netty will receive a plain request response, only after waiting for 1 millisecond.

To make it more clear let's look at the unit tests that will help us understand this in class CatControllerImplTest:

    @Test
    void getFullAllCatsNonReaciveForTest_whenCall_Blocking() {
        Mono.delay(Duration.ofMillis(1))
            .doOnNext(it -> catController.getFullAllCatsReactiveForTest())
            .block();
    }
    @Test
    void getFullAllCatsNonReacive_whenCall_Blocking() {
        Mono.delay(Duration.ofMillis(1))
            .doOnNext(it -> catController.getFullAllCatsNonReactive())
            .block();
    }

Now we have the second unit test failling.
This is the Error we should be getting at the moment:

reactor.core.Exceptions$ReactiveException: reactor.blockhound.BlockingOperationError: Blocking call! java.lang.Thread.sleep

    at reactor.core.Exceptions.propagate(Exceptions.java:393)
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:97)
    at reactor.core.publisher.Mono.block(Mono.java:1666)
    at org.jesperancinha.housing.controller.CatControllerImplTest.getFullAllCatsNonReacive_whenCall_Blocking(CatControllerImplTest.java:77)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
    at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1510)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1510)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
    at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
    at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
        ... 65 more
Caused by: reactor.blockhound.BlockingOperationError: Blocking call! java.lang.Thread.sleep
    at java.base/java.lang.Thread.sleep(Thread.java)
    at org.jesperancinha.housing.controller.CatControllerImpl.getFullAllCatsNonReactive(CatControllerImpl.java:51)
    at org.jesperancinha.housing.controller.CatControllerImplTest.lambda$getFullAllCatsNonReacive_whenCall_Blocking$5(CatControllerImplTest.java:76)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:177)
    at reactor.core.publisher.MonoDelay$MonoDelayRunnable.run(MonoDelay.java:117)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)

The way to fix this unit test is just to cause this test to expect this Error:

@Test
void getFullAllCatsNonReacive_whenCall_Blocking() {
    assertThatExceptionOfType(BlockingOperationError.class).isThrownBy(() -> Mono.delay(Duration.ofMillis(1))
        .doOnNext(it -> catController.getFullAllCatsNonReactive())
        .block());
}

This was our last failing unit test, and this concludes our research into Reactive Programming with WebFlux and testing it with Blockhound

7. Conclusion

This article was written with a lot of positivity to highlight the basic ways in which to use Reactive programming with WebFlux.

All code for this example is available on my repo on GitHub Kitten House Care

I hope that you enjoyed this article as much as I enjoyed writing it.

If you enjoyed it, please share it in the links below.

Take care, stay interested, stay logic, stay safe!

If you are interested in Reactive programming, you'll probably find the following slides very useful:

Reactive Programming With S... by João Esperancinha

or you may find the same presentation on slide-share:

Reactive programming with Spring Webflux.pptx from João Esperancinha

8. Resources

8.1. Books and papers

8.2. Websites

If you enjoyed this presentation, you probably also would like to learn more about reactive programming. I have created a tutorial about best practices on creating a reactive web application with Kotlin coroutines and you can find the 3 episode set over here:

Or you prefer to watch all of them in one go, they are also available in one single video form on DailyMotion for you:

dailymotion.com

How to make the best of a slow machine running on limited resources with a Windows environment as a Java Engineer

João Esperancinha — Sun, 29 Dec 2024 10:46:30 +0000

So this is the question. Let's think about this.

When we get into the job market, not always we get the best machines to work with and to make the things we love. Having a Windows machine isn't necessarily bad news, but the machines that usually come with it, either have limitations or you end up needing more tools than the ones that Windows seems to offer. Nonetheless, many things are actually very much possible with a Windows machine. They usually just have a different shape.

I personally have more preference for Linux and Mac-OS based machines. But that's also because I don't do much professional work in C# or other .Net technologies.

This first post is a celebration of the all the things that I have learned that have gotten me through working with Windows machines through the years, in those odd occasions.

1. Environment Setup

1.1. Chocolatey

If you come from a world where you are used to using package managers, you will probably want to keep using such a mechanism. For this you can install Chocolatey

Run Windows PowerShell with administrative rights.

The following is on the website itself. I'm putting it here, just to summarise it:

Run

Get-ExecutionPolicy

If it returns Restricted, then run

Set-ExecutionPolicy AllSigned

Set-ExecutionPolicy Bypass -Scope Process

, in Installing Chocolatey

NOTE: If you end up changing policy, you need to restart your computer

Run the installation command:

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

1.2. Install necessary software

choco install -y minikube
choco install -y virtualbox
choco install -y maven
choco install -y git
choco install -y vim

2. Install Docker

There are multiple ways to install docker:

Docker. This is my favorite. It is very straightforward and you get docker running in no time. However, for one or another reason, you may find it difficult to do so.
Chocolatey. Using the command line with Chocolatey:

choco install -y docker
choco install -y docker-machine
choco install -y docker-desktop
docker-machine create default
docker-machine start

After this last create command, there will be an output to run an extra command. This may differ by machine and environment. Please run that.
Here is an example:

docker-machine.exe env default | Invoke-Expression

Essentially what this does is that it binds your command line to the docker environment. In this case it is default.

3. Installing Ubuntu for windows

In order to install Ubuntu in your windows machine, there are two options.

Install Ubuntu for Windows (pre-requisite: at least Windows 10 Professional)

Go to: Ubuntu for Windows and follow the instructions.
Alternatively you can just run choco install -y wsl-ubuntu-1804. It will install the wsd (windows subsystem for linux) automatically.

To enter this environment, please use the BASH command

bash.exe

If for some reason, all of the above fails, then you can try running a container in your machine:

docker run -v /c/Users/jofis:/root -v /c/Users/jofis/jvm:/usr/lib/jvm --rm --name container-developer -td ubuntu

you can now use your docker machine to build your project. The volume mappings ensure that your home folder is shared and so will be .m2 and .sdkman (should you use it), with it. They also ensure that the jvms are shared in order to be used with intellij.

To enter this environment, please go into your container:

docker exec -it container-developer bash

NOTE: 🔴 Remember to be careful not to remove this container accidentally

4. Installing Java in Ubuntu for Windows

Once you are in your environment, please choose one of the following sequence of commands:

4.1 SDK-MAN. Via SdkMan!

curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 8.0.242.hs-adpt
sdk install java 11.0.6.hs-adpt
sdk install java 12.0.2.hs-adpt
sdk install java 13.0.2.hs-adpt
sdk install java 14.0.0.hs-adpt

If you'd like you can add this to you .bashrc. This contains aliases that you can use to quickly switch betweek Java SDKs.

alias java8="sdk use java 8.0.242.hs-adpt"
alias java11="sdk use java 11.0.6.hs-adpt"
alias java12="sdk use java 12.0.2.hs-adpt"
alias java13="sdk use java 13.0.2.hs-adpt"
alias java14="sdk use java 14.0.0.hs-adpt"
alias m2disable="rm ~/.m2/settings.xml"
alias m2enable="cp /your_repo_folder/settings.xml ~/.m2/"

4.2. Ubuntu. Via Ubuntu commandline with APT

apt-get -y update
apt-get -y upgrade
apt -y install apt-transport-https ca-certificates wget dirmngr gnupg software-properties-common
wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | apt-key add -
add-apt-repository --yes https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/
apt -y update
apt -y install openjdk-11-jdk
apt install openjdk-13-jdk
apt -y install adoptopenjdk-8-hotspot
apt -y autoremove

If you'd like you can add this to you .bashrc. This contains aliases that you can use to quickly switch betweek Java SDKs.

alias java8="export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64 && update-java-alternatives -s adoptopenjdk-8-hotspot-amd64"
alias java11="export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64 && update-java-alternatives -s java-1.11.0-openjdk-amd64"
if [[ -d /usr/lib/jvm/java-13-oracle ]]
then
    alias java13="export JAVA_HOME=/usr/lib/jvm/java-13-oracle && update-java-alternatives -s java-13-oracle"
elif [[ -d /usr/lib/jvm/java-1.13.0-openjdk-amd64 ]]
then
    alias java13="export JAVA_HOME=/usr/lib/jvm/java-1.13.0-openjdk-amd64 && update-java-alternatives -s java-1.13.0-openjdk-amd64"
else
    echo "Java13 not found!"
fi
alias m2disable="rm ~/.m2/settings.xml"
alias m2enable="cp /your_repo_folder/settings.xml ~/.m2/"

NOTE: the m2disable and the m2enable aliases are there so that you can easily switch between a possible settings.xml file and not having any installed.

4.3. Install utilities

apt install -y vim
apt install -y git
apt install -y git-gui
apt install -y maven

5. Intellij

Let's finally look at how can we use intellij with these command lines. If you picked up the container solution, please make sure that your container is running. We have already seen above how to log in it.

For this example we are going to look at the most complicated example. This is the one that does not use SDKMAN to get the different Java SDK's

Since we are sharing the jvm folder, we can just configure Intellij to use Java 11 for example:

Now let's search for Terminal in Preferences/Settings:

This our initial setting in a Windows environment. The cmd command line:

We can choose bash, if we have installed Ubuntu for Windows:

We can choose powershell if we don't have Ubuntu for Windows installed. This is the one we pick for our example:

If we run the usual commands to get into the docker enviroment and as a recap on my example these are:

docker-machine.exe env default | Invoke-Expression
docker exec -it container-developer bash

We then get something like this:

This is it, these are just some examples on how we can get a machine to work for us.

6. Conclusion

I deliberately did not use GIT for windows, because in this setting we are using an ubuntu command line. This means that we have much more possibilities. Also, the MinGW provided by this installation, needs quite a lot of fine-tuning before it gets reasonably functional.

We are always way better off, if we can get the Windows Subsystem for Linux to work. Unfortunately that is not the case for Windows Home Edition 10 users, and that is why, if we want to use a real Unix/Linux based system, then we have to use containers. Virtual machines don't really appeal to me in this case because of the resource consumption they represent.

If you are using a Professional or Enterprise version, you are very likely to be able to use the WSL. This has a great advantage because no container will be needed and to configure Intellij you only need to provide bash as the Terminal command.

SDK-Man is always a favorite of mine, because of how easy it is to install Java versions. The only disadvantage we may point out is that they have a fixed number of different Java versions where we don't control the addition. Using apt we can add a lot more, but it does add quite a bit of complexity to the system.

We could have done all of this without choco and without PowerShell, but that would add other challenges and would probably make this installation much more complex.

I hope that you enjoyed this article as much as I enjoyed writing it.

If you enjoyed it, please share it in the links below.

Take care, stay interested, stay logic, stay safe!

If you happen to be interested into further deepen your knowledge about systems, you might want to have a look at a video I created about saving a system that appears to be broken for good over here:

7. Resources

Redis Cache - A String story

João Esperancinha — Sat, 28 Dec 2024 16:49:38 +0000

In one of my past projects, we were working on a project that made extensive use of Redis. This is a distributed cache system used as an alternative to Hazelcast. There are always several disputes between the usage of both among developers, but I'm not going to go there in this post. Where I am going to go is the solution to a problem I found today in less than 30 minutes, which at the time cost so much time, and we ended up not really developing the best solution.

In my current project buy-odd-yucca-concert, I'm now using Micronaut in combination with Redis in order do develop a simple pub-sub system to provide listeners for the fulfilling of several segments of a concert reservation. One of those segments is a TicketDto:

data class TicketDto(
    val name: String,
    val address: String,
    val birthDate: LocalDate,
    val concertDays: List<ConcertDayDto> = emptyList(),
    val meals: List<MealDto> = emptyList(),
    val drinks: List<DrinkDto> = emptyList(),
    val parkingReservation: ParkingReservationDto? = null,
    val createdAt: LocalDateTime? = LocalDateTime.now(),
) : Serializable

The details of the goal of this ticket are pretty much irrelevant at this point. The only thing we are interested in known is how to put in in a Redis cache, or in my case, publish this to a Pub-Sub system.

In the past the discussion started out from a very strange assumption that Redis "does not support anything other than Strings" in the cache. One quick look at their cache info page suggest the complete opposite: https://redis.com/solutions/use-cases/caching/. And quoting what they say there:

Redis is designed around the concept of data-structures and can store your dataset across Strings, Hashes, Sorted Sets, Sets, Lists, Streams, and other data structures or Redis modules".

Unfortunately not only was this ignored, but the notion that Redis only supported Strings gained steam when going head first into the code, we saw that there was only a StringCodec, but not really an ObjectCodec or anything related. Someone then came up with the idea of using lombok's @ToString and then cache this string value and use it. I personally was never too happy with this, but hey, it worked and the code is now running! Good right? 😁... Not today in 2022 though.

Flash forward and in current days reaching this part of my project brought me back the memories of that discussion. To avoid this kind of discussions, that only waste time and money, in other developments I'm sharing with you the possibility of actually serialising whatever you want into the Redis cache, pub-sub systems or anything else in the Redis ecosystem. In my project I'm using lettuce, which is one of the most common Redis clients out there. Here is the dependency:

<dependency>
    <groupId>io.micronaut.redis</groupId>
    <artifactId>micronaut-redis-lettuce</artifactId>
    <version>5.2.0</version>
</dependency>

It is a micronaut dependency, but it contains all needed lettuce dependencies.

Digging into this library, I found out that next to StringCodec there is this ByteArrayCodec, which should allow for some sort of leverage to achieve our goals. After a huge amount of time spent in discovering and re-descovering how buffering, serialising, and streaming works I finally came up with the following in Kotlin:

class TicketCodec : RedisCodec<String, TicketDto> {

    override fun decodeKey(byteBuffer: ByteBuffer): String {
        return defaultCharset().decode(byteBuffer).toString()
    }

    override fun decodeValue(byteBuffer: ByteBuffer): TicketDto =
        ObjectInputStream(
            ByteArrayInputStream(byteArrayCodec.decodeValue(byteBuffer))
        ).use { it.readObject() as TicketDto }

    override fun encodeKey(key: String): ByteBuffer {
        return defaultCharset().encode(key)
    }

    override fun encodeValue(ticketDto: TicketDto): ByteBuffer =
        ByteArrayOutputStream().use { baos ->
            ObjectOutputStream(baos).use { oos ->
                oos.writeObject(ticketDto)
                byteArrayCodec.encodeValue(baos.toByteArray())
            }
        }

    companion object {
        val byteArrayCodec = ByteArrayCodec()
    }
}

The way codecs work in redis is actually very, very simple. Everything, literally everythin is stored in a ByteBuffer! It's not even a String. You encode your values as ByteBuffers. These buffers keep your values stored in cache. Not Strings. You may be able in the code to make only String operations, but that's not what happens underwater. So essentially, and for my example this is what happens when you publish and then listen to your value:

Publish

the key gets serialised and the encodeKey method is called.
the value gets serialised and the encodeValue method is called.

Listener (Subscriber)

the key gets deserialised and the decodeKey method is called
the value gets deserialised and the decodeValue method is called

In my first attempts I kept on trying to do even more manual steps. And this is because I kept looking at the StringCodec example. Take a look at this class in your IDE to be amazed on how complex the actual String processing really is. Because I wasn't getting anywhere with pure low level methods, I decided to try the ByteArrayCodec directly. And voila. Now it works and I can serialise my objet correctly.

Finally, was able to apply this codec which make code declaration slightly different than usual. This is the bean factory modified:

@Factory
class RedisBeanFactory {
    @Singleton
    fun pubSubCommands(redisClient: RedisClient): RedisPubSubAsyncCommands<String, TicketDto> =
        redisClient.connectPubSub(TicketCodec()).async()
}

And this is the way I'm publishing the ticket now. Again, still purely experimental code:

redisClient.connectPubSub(TicketCodec()).async().publish("ticketsChannel", ticketDto)

It is entirely up to you, whatever you want to cache and whatever data structures you should use. Even plain old strings. And I'm also not saying with this article that using a string instead of an object representation is per se a bad thing. But in order to make good decisions, we need to know what is possible and what it's not possible and a common mistake is to assume what is not possible. A good philosophy maybe is to assume that anything is possible until proven otherwise. This always goes back to the same principle: "Knowledge is power" and "The pen is mightier than the sword". In this case, It would have made me really happy to know that the reason we used @ToString from, among so many other ideas, lombok, had been something in the lines of "it improves performance". I hope with this bite, to have broadened your horizons in your work with Redis, if you didn't know this already.

Originally posted on Buy me a coffee - Redis Cache - A String story

References

Pub-sub Redis in Micronaut

João Esperancinha — Sat, 28 Dec 2024 16:09:30 +0000

A few years ago I struggled a lot to get a simple Redis Pub-sub working locally. I'm working on a private project looking to illustrate the Kong framework in action. For that I need some sort of Queue system and decided to venture on one that is relatively new to me.

I started out by thinking out a way to first test this. In this way I wanted to make this in a more thought-out way. The tests would help, because I'm not interested at this point to have something fully running. I just want to start with testing as it should be. So going through the Micronaut website, I found out that I would need a few dependencies, namely this one:

<dependency>
    <groupId>io.micronaut.redis</groupId>
    <artifactId>micronaut-redis-lettuce</artifactId>
    <version>5.2.0</version>
</dependency>

This is a lettuce dependency, which is mostly a Redis client, which allows to access a lot of functionalities in Redis. Then I created this configuration in the application.yml file:

redis:
    uri: redis://localhost
    timeout: 30s

I added the timeout in order to guarantee connection. This may be removed in the future.

Because I love all sorts of tests and in this case I was interested in seeing how Redis behaves I resourced to the use of Testcontainers. If you do not know, Testcontainers is a framework that allows you to start and stop containers, programmatically, via code, using your Docker environment locally or in a pipeline. Anyway I created this abstraction to allow this to happen for now:

abstract class AbstractBuyOddYuccaConcertContainerTest {
    companion object {
        @Container
        @JvmField
        val postgreSQLContainer: TestPostgresSQLContainer = TestPostgresSQLContainer("postgres:14")
            .withUsername("kong")
            .withPassword("kong")
            .withDatabaseName("yucca")
            .withExposedPorts(POSTGRESQL_PORT)
            .withCreateContainerCmdModifier { cmd ->
                cmd.withHostConfig(
                    HostConfig().withPortBindings(
                        PortBinding(
                            bindPort(POSTGRESQL_PORT),
                            ExposedPort(POSTGRESQL_PORT)
                        )
                    )
                )
            }


        @Container
        @JvmField
        val redis: GenericContainer<*> = GenericContainer(parse("redis:5.0.3-alpine"))
            .withExposedPorts(REDIS_PORT)
            .withCreateContainerCmdModifier { cmd ->
                cmd.withHostConfig(
                    HostConfig().withPortBindings(
                        PortBinding(
                            bindPort(REDIS_PORT),
                            ExposedPort(REDIS_PORT)
                        )
                    )
                )
            }

        private val config = ClassicConfiguration()

        init {
            postgreSQLContainer.start()
            redis.start()
            config.setDataSource(
                postgreSQLContainer.jdbcUrl,
                postgreSQLContainer.username,
                postgreSQLContainer.password
            )
            config.schemas = arrayOf("ticket")
            Flyway(config).migrate()
        }
    }
}

In this case I'm not only using Redis, but I'm also using Postgres. This is for other functionalities in my project, but the point here is to explain how I got to test the first Redis functionality in my project.

This is the test:

@MicronautTest
internal class ReservationsServiceTest @Inject constructor(
    private val reservationsService: ReservationsService
) : AbstractBuyOddYuccaConcertContainerTest() {

    @Test
    fun `should get all reservations`(): Unit = runBlocking {
        reservationsService.getAll().toList().shouldNotBeNull()
    }

    companion object {
        @JvmStatic
        @BeforeAll
        fun setup() {
            redis.start()
            postgreSQLContainer.start()
        }
    }
}

The reason why I'm, at this moment, forcing the start of the containers in the @BeforeAll test, is because I noticed that @MicronautTest doesn't seem to work very well with the TestContainers framework with my current setup and the Redis client seems start being configured very much before the container starts.

Of course for this part, I didn't really adhere strictly to the idea of making the integration test first and then perform the implementation. What I did is to firstly implement the service in this way:

@Singleton
class ReservationsService(
    private val receiptRepository: ReceiptRepository,
    private val redisClient: RedisClient
) {
    init {
        val statefulRedisPubSubConnection = redisClient.connectPubSub()
        statefulRedisPubSubConnection.addListener(Listener())
        val redisPubSubAsyncCommands = statefulRedisPubSubConnection.async()
        redisPubSubAsyncCommands.subscribe("channel1")
        redisClient.connectPubSub().async().publish("channel1", "test")
    }

    @OptIn(DelicateCoroutinesApi::class)
    suspend fun createTicket(ticketDto: @Valid TicketDto) = GlobalScope.launch {
        receiptRepository.save(Receipt()).toDto
    }

    fun getAll(): Flow<Receipt> = receiptRepository.findAll()
}

class Listener : RedisPubSubAdapter<String, String>() {
    override fun message(p0: String?, p1: String?) {
    }
}

Of course this is meant in the future to be where the production code starts. However, this class represents the first successful result in trying to access Redis. Among the interfaces readily available in the Lettuce framework, the only one that suits me at the moment in order to implement a Pub-Sub mechanism is the RedisClient. The reason why this is the only code that works for me and for now is because I realized that in order for the Pub Sub to work, I need to create an async commands for the publisher and an async commands for the subscriber. Once the RedisClient is injected, the way that the experimental code works goes something like this

Create a Pub Sub connection

val statefulRedisPubSubConnection = redisClient.connectPubSub()

Add a listener (It has to implement an interface of type RedisPubSubListener. RedisPubSubAdapter implements all methods and they are all empty by default)

statefulRedisPubSubConnection.addListener(Listener())

Get the commands instance to be able to issue commands

val redisPubSubAsyncCommands = statefulRedisPubSubConnection.async()

Create a channel by subscribing to it

redisPubSubAsyncCommands.subscribe("channel1")

Send the first message in another connection

redisClient.connectPubSub().async().publish("channel1", "test")

Now I'm very happy to see this working locally. I can spin off the Redis container, connect to it and see the first Pub Sub working! If you are interested in follow the changes in my project, please have a look at it here: https://github.com/jesperancinha/buy-odd-yucca-concert/tree/master/buy-oyc-api-service/src/main. My goal of this quick post was to show you how I came about to understand the basics of a Pub Sub system implemented with Redis. You can find more documentation about this project here: https://github.com/jesperancinha/buy-odd-yucca-concert/tree/master/buy-oyc-api-service.

Originally posted on Buy me a coffee - Pub-sub Redis in Micronaut

References

How to Manage Your APIs Across Multi-Cloud Environments

João Esperancinha — Sat, 28 Dec 2024 15:03:18 +0000

1. Introduction

To manage APIs across muli-cloud environments we need an API gateway. Kong provides this using Kong KONNECT. In this article, we’ll look into ways to make that a reality and work with them.

2. Configure API Gateways in Each Cloud Provider

I made a video about API gateways a while ago that you can find over here:

The video contains a detailed description of how to set up an API gateway that can serve as the entry point for your application, whether it contains many services or just one. Cloud providers may offer many alternatives to deploy an API gateway, but it always follows the same principle. The idea is that you can use a Gateway just like I mentioned in my wildlife-safety-monitor project located on GitHub. This is the diagram of that project:

What we see in this diagram is an internal network managed by Kuma that has a gateway. But we are probably more interested in a simpler example like this one:

These two examples are only here to show how an API gateway can be implemented. An example of that is the one I talk about in this video:

https://youtu.be/rJKbAzjb5lQ

To provision an API gateway via KONNECT, we just have to first log into our console and create something called a data-plane node over here:

Then we just need to fill out some fields:

Once we have selected the version of the Kong Gateway we want to use and the platform we want to run it in, we just need to copy the generated script to our command line and run the container with the API gateway. The running command contains a lot of inputs to our container and we should use this to set up the container for our API gateway within our cluster or environment.
When we run that script locally, it will connect to KONNECT and then we can save that node.
We could use this with Kubernetes, docker-compose, or other container management software. In the video about gRPC that I mentioned above, I explain further how I achieve this, but what is essential to understand is of course how this works.
Kong KONNECT is just a cloud configuration web application that allows us to configure pretty much anything related to our gateway and this includes request protocol translation, rate-limiting, different security protections, load balancing, etc. The idea of using Kong KONNECT only is just to provide an entry point to the settings of our running container. This container uses Keys to be able to know exactly from which instance to obtain the requests from which Kong KONNECT user. This allows for a seamless transmission of requests from a user who wants to configure the gateway to the gateway itself. Without Kong KONNECT the only way was to use the command line command or update the Kong gateway config file and restart the container, which is not desired, namely in high-availability systems. And this is where we can segway into what I did for the project wildlife-safety-monitor over here on GitHub:

KONG_ROLE=data_plane
KONG_DATABASE=off
KONG_VITALS=off
KONG_CLUSTER_MTLS=pki
KONG_CLUSTER_CONTROL_PLANE=4674f82943.eu.cp0.konghq.com:443
KONG_CLUSTER_SERVER_NAME=4674f82943.eu.cp0.konghq.com
KONG_CLUSTER_TELEMETRY_ENDPOINT=4674f82943.eu.tp0.konghq.com:443
KONG_CLUSTER_TELEMETRY_SERVER_NAME=4674f82943.eu.tp0.konghq.com
KONG_CLUSTER_CERT="-----BEGIN CERTIFICATE-----FAKE-----END CERTIFICATE-----"
KONG_CLUSTER_CERT_KEY="-----BEGIN PRIVATE KEY-----FAKE-----END PRIVATE KEY-----"
KONG_LUA_SSL_TRUSTED_CERTIFICATE=system
KONG_KONNECT_MODE=on
KONG_CLUSTER_DP_LABELS=created-by:quickstart,type:docker-linuxdockerOS

On that project, there is a Makefile that has a script to generate this from a script.sh file where we only need to copy the contents of what is given to us on the Kong KONNECT website. Once we generate the file, we see immediately that connecting to Kong KONNECT via the running docker container on our local machine isn’t that complicated, because we are essentially immediately establishing a connection with KONNECT, and that means that only by logging into Kong KONNECT we are already empowered to configure our gateway online.

2.1 API Gateway Federation and synchronization

An API Gateway federation is critical because it allows us to group data-plane nodes into different subjects. This allows us to keep the same granular control of our data planes but gives us the option of distribution configurations across different data-plane nodes.
Nowadays, we need to ensure that our applications are maintained as uniformly as possible, whether between multiple replicas behind a load balancer or separate APIs. Because our APIs almost always run in distributed systems, it is important to keep them uniform to minimize maintenance costs. A change in one API can be applied to another API using the same process. This means that we need to make sure as much as possible that we keep the same versions of dependencies, databases, etc. The process that allows that is known as API synchronization and this is something that Kong KONNECT allows us to do very well.
There are several techniques to do that with Kong KONNECT: Automated API Deployment, Service Mesh configuration, and Centralized management.
For Kong Konnect, however, we are only interested in Centralized management at this point. To do that, we need to create a couple of data plane nodes. In this case, we need first to create a new Gateway Service. The Gateway Service controls data plane nodes:

Once we select the Kong Gateway option, we can create a new Kong Gateway service:

Upon creating, this will immediately lead us to create another dataplane node. If we add one more, then we can create a data plane node group and in this case, make sure to change to ports 8001 and 8444, if you are running on your local machine:

docker run -d \
-e "KONG_ROLE=data_plane" \
-e "KONG_DATABASE=off" \
-e "KONG_VITALS=off" \
-e "KONG_CLUSTER_MTLS=pki" \
-e "KONG_CLUSTER_CONTROL_PLANE=c0088bf6d5.eu.cp0.konghq.com:443" \
-e "KONG_CLUSTER_SERVER_NAME=c0088bf6d5.eu.cp0.konghq.com" \
-e "KONG_CLUSTER_TELEMETRY_ENDPOINT=c0088bf6d5.eu.tp0.konghq.com:443" \
-e "KONG_CLUSTER_TELEMETRY_SERVER_NAME=c0088bf6d5.eu.tp0.konghq.com" \
-e "KONG_CLUSTER_CERT=-----BEGIN CERTIFICATE-----FAKE-----END CERTIFICATE-----" \
-e "KONG_CLUSTER_CERT_KEY=-----BEGIN PRIVATE KEY-----FAKE-----END PRIVATE KEY-----" \
-e "KONG_LUA_SSL_TRUSTED_CERTIFICATE=system" \
-e "KONG_KONNECT_MODE=on" \
-p 8001:8000 \
-p 8444:8443 \
kong/kong-gateway:3.7.0.0

Notice that the only ports changed are the outer ports 8001 and 8444
Once this is done, we should now have two different data node planes running and should be able to see that in KONNECT:

This means that we have now two gateways running locally. The next step is part of the reason why KONNECT can make life easier. At this point, both containers aren’t aware of each other locally, and they don’t have to. They can however get federated inside a data plane node group, and to do that we only have to create a group:

One important thing to remember here is that, while the services are running in our machine and staying connected to KONNECT, we don’t see them listed in the data plane nodes window if they are disconnected or have stopped running. The list is created based on the connected data node planes and those are connected via certificates that are saved in KONNECT itself. This is where you can find them:

In the Gateway overview, there is an actions button with a list of Data Plane Certificates. That option leads us to:

This means that the removal of a data node from KONNECT only happens effectively with the removal of these certificates. There is further no control over how the data nodes should stop connecting to KONNECT. We cannot ban a specific node, but if we remove the certificate from our KONNECT account, then the node will not be able to connect. It works as a ban but this is an important aspect because many people are instinctively driven to think that a node can be banned directly and that can only be done by the removal of the certificate from Kong KONNECT.
We need to click on Gateway Manager and then select the option to create a Control Plane Group:

This option allows us to federate gateway managers. We don’t federate data plane nodes. Instead, it is the gateway manager that gets federated:

Once the federation has been created, the two gateway services are federated and each one of them has now a data plane node running. But we are not there yet because our nodes are running:

This means that we first need to stop them. Using docker we can use a docker stop command to stop them for example. This step really depends on your platform. One the nodes are stopped, we should now be able to federate our services:

We don’t see any data plan nodes running at the moment because we have stopped them. Let’s start them now. If you did this now, you probably noticed that you do not see any nodes being connected. This happens because those nodes now need to be associated with the federation and not with only the service.
The new nodes need to be connected with the federation and not the service and so to do that let’s first create a new data plane node in the default gateway manager that now exists under the federation:

When that node gets created we should now see a new data node plane under the Federation:

This now means that our services only serve an organizational function in terms of configuration. We can still discriminate between goals, but the configuration will be applied globally under the federation, and we have to be careful here because a configuration of two plugins in different services with the same plugin will result in conflicts and so if we choose the same plugin in one service like for example the basic auth:

If we do that for the other service, we will get a conflict like this:

We cannot add plugins to the federations. They are read-only abstractions that give us a sense of unity between the different services and the federated nodes:

Federation using Kong KONNECT is a broad term that allows us to configure different plugins. The configuration of each plugin escapes the scope of this article, but the idea is that in terms of authentication, we also have concepts like federated identity, federated security, and cloud communication. These are all things that follow a similar principle.
Using Kong Meshes is also a way to create a kind of federated environment with the difference that the configuration does not have to be centralized.
I talk about Kuma Meshes over here:

But Kong Meshes are an extension of Kuma. Kuma is open source. Kong Meshes is an enterprise, which means that there are other features available. To start with it let’s first create, not a control plane, but a Global Control Plane:

The configuration of a global control plane is very straightforward and after configuring it we should come to this screen:

Applying policies and how to do that in a fine-grained way escaped the purpose of this article.

2.2. API Lifecycle Management

An API Lifecycle Management in Kong KONNECT comprises 8 phases: Design and Development, Deployment and Scaling, Operations and Monitoring, Security and Compliance, Versioning and Evolution, Decommissioning, Automation and CI/CD Integration, and Community and Ecosystem. These phases are controlled and configured by the use of plugins and the usage of Federation. Important here to highlight is that security and version and keeping integrity across the APIs is very important and that is something that, as we have seen above Is straightforward to do in Kong as long as we apply the right plugin and create federations and services correctly segregated for different purposes.

2.2.1. Security and Access Control

Security and access control can be implemented by applying different plugins to our control manager or federation. We have many to choose from this list:

And we can also enforce security policies like the ones available from these plugins:

2.2.2. Monitoring and Analytics

We can configure analytics in Kong using these plugins:

We can use these plugins to connect monitoring data to a datadog service or any other kind of service using the StatsD plugin. The idea is to provide a universal way to provide metrics to perform analysis that can be coupled with external tools.

2.2.3. Testing and Deployment

The deployment of the API gateway using Kong KONNECT is much easier in this case because once our API gateway, as we have seen before is integrated in our system, which could be something like k8s or Docker, the updates and management is done in this centralized web application.

2.2.3. Disaster Recovery and High Availability

Disaster recovery is very important. Kong KONNECT acknowledges that and that is why KONNECT never gives up on a node. Remember that above we mentioned that a node never gets removed. We only need to remove the certificate to stop the node from trying to connect. If we load balance that, then the system can remain waiting for that node.
In terms of high availability, there are multiple strategies to make sure that happens, but we also need to make sure that our services do not suffer from a DDoS attack and that usually happens when a bad actor tries to make too many requests to a service. In that case, unprepared machines can be taken down, leading up to service disruptions. In order to control traffic, there are many plugins to choose from:

One of the most used plugins of this section is the rate-limiting plugin that prevents DDoS by allowing traffic up until a point and after that, a potential bad actor may insist as much as intended but all of these requests will be denied entry until the request frequency goes down to a certain level. It still leads to some disruptions, but the service doesn’t go down leading to eventually a faster upstart of the system.

3. Conclusion

Kong KONNECT allows us to configure most of what we need for our API gateways directly and seamlessly.
Know more about Kong with some videos I have created on YouTube about these topics:

Working with Insomnia and Inso

gRPC with Kong Konnect

Kuma

Kong Konnect

4. References

ISBN Stacks — A look at a possible Spring Application implementation without annotations

João Esperancinha — Sat, 28 Dec 2024 09:13:45 +0000

1. Introduction

If you’re a big, and I mean a seriously big fan of the Spring Framework as I am, and never read something like the title of this article describes, you probably just got cold feet only by reading it. No annotations? It may sound like a bit of an insane idea. When I achieved the Spring certification, it gave me the feeling of accomplishment. There I was Spring certified. But a huge chunk of that certification path was about precisely the use of annotations. I had to learn all sorts of annotations. And there are quite a few, to say the least. The whole Spring Framework is given to us as an annotation-driven framework nowadays. We can do anything with it, from programmatically changing the way beans start, how beans are created, we can create components, services, repositories, apply the decorator, IoC, adaptor, MVC patterns, we can create triggers for specific exceptions, handlers for events, health endpoints, we can implement the Observer pattern in a seamless way, use AOP with OOP, implement batch jobs, STOMP pattern for WebSockets, conditional beans, use reactive programming, and I could go on forever. The point I’m trying to make is that for years we’ve been educated and pressured to leave an XML-driven framework as it happened up to Spring 2.5. to move on to a full and complete annotation-driven capable framework since Spring 4. Finally, in Spring 5 we get support for reactive implementations over project reactor:

Now that we’ve learned all of this, at the same time that we have Spring 5 and Spring Boot 2.5.6. moving forward, a team of the Spring Experimental Projects is developing a whole new version of Spring called Spring-Fu. Fu, means functional. They have developed two sub-versions of this: KoFu and JaFu. Spring KoFu is a Kotlin-based DSL and Spring JaFu is a Java-based DSL. The project has started on GitHub on the 31st of May 2018. There is currently some buzz around it given that the Spring-Fu team has apparently achieved better service startups and faster processing. This article is my research into this matter to find signs of these claims. All the results and implementations are available on GitHub.

2. What we are trying to find

We are going to explore the responses of three simple services. Two are reactive and one is non-reactive. One is implemented with Spring-Fu for Kotlin also known as Spring KoFu. The other two are implemented in a traditional MVC way. One is reactive and the other one is not.
The ultimate goal of reactive programming is to allow services to be more resilient and use resources in the most effective way in order to process as many possible requests as possible. The way it does this is to deliver the responsibility of processing our requests to publishers. In Spring, they are known as Mono<T>’s and Flux<T>’s. For fast processing, this causes the service to be highly available, which means that a non-reactive service will be unavailable much quicker than a reactive service. This is because the response in the former case is given back directly and so the client must wait until the request is processed, before performing a new one. For long processing requests, however, the story can be different. Since the reactive service keeps on accepting requests regardless of the previous ones being processed or not, this also means that it needs to be much more resilient than the traditional non-reactive service. As a result, a reactive service will become unresponsive much quicker, should the processing of one request take too long.
Finally, what we want to prove is, on one hand, that described theory stands. On the other hand, by comparing the two reactive services, we want to see if one is faster than the other while processing multiple fast processing requests. We also want to prove that the Spring-Fu services start faster. We may also not want to prove anything and just see signs that the Spring-Fu team is on to something.
Please keep in mind that terms like non-reactive, reactive, and KoFu are used in this article interchangeably with terms like Spring Web, Spring WebFlux, and Spring KoFu.

3. Biases

Proving and testing something like this can be very difficult and inconclusive on a local machine. A MacBook Pro or a Chrome machine or any Linux machine, even on Windows, can have its own ways of processing, interpreting, and performing. We also, when looking at new technologies, may project some of our own personal feeling to it. This is the reason why all the test results I’m presenting throughout the article have been done multiple times and I only present the ones that gave consistent results. All the other tests, including these also, are available in the reporting folder of the repo on GitHub.

4. Case

The case for this article is essentially just a list of randomly generated ISBN 13 numbers. There are two static sources of ISBN 13 codes. One is a big list of half a million ISBN and the other is a source of 50 ISBNs. The first case will simulate the case of long-lasting and heavy data requests. The latter will simulate fast processing requests. We could argue that there is actually no processing involved. The idea is not to block any part of the running application and simulate fast requests. The processing will indeed spend some short time on de server and that is what we want to explore. We will then have three services implemented as discussed before as the following diagram explains:

For these kinds of things, I think it's also important to understand this from a sequence diagram perspective.

Essentially for every service, we request ISBN from designated endpoints and then wait for a reply. We are interested in seeing in practical terms, how the services behave, how long it takes to get the requests, how many requests can they take, and how responsive and possibly resilient they are.
Now that we have had a look into the design, let’s now have a look at the following step, which is the implementation.

5. Implementation

Assuming you already know how to implement services in Spring/Spring boot, I just want to, in any case, have a quick run through all of the different implementations and the effort I made to minimize the differences on a functional level. Also, in this case, there is still a small chance of bias or inaccurate reading. The way we implement the simplest of REST responses can dictate how reliable our tests are going to be. In these implementations, the service layer and the data layer do not exist. Data is being fetched directly from memory out of two lists initialized upon startup.

5.1. Traditional `Spring Web MVC` Blocking `REST` service

It is now fairly accepted that the old Spring Web MVC is slowly being replaced by better software design patterns or an extension of it. In this implementation, however, we just want to make sure of two important aspects. One, that we have a very basic implementation. Two, that the responses are identical or similar. Out of all the dependencies we need, the one we are actually investigating is this one:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>${spring-boot.version}</version>
</dependency>

In regard to the implementation let’s look at how it’s being made:

@RestController
@RequestMapping("/api/traditional/non")
class ISBNController {
    var mapper = ObjectMapper()
    @GetMapping("hello")
    fun getMessage(): String {
        return MESSAGE
    }
    @GetMapping
    fun getJsonMessage(): ObjectNode? {
        val createObjectNode = mapper.createObjectNode()
        createObjectNode.put("message", MESSAGE)
        return createObjectNode
    }
    @GetMapping("isbns")
    fun getIsbns(): List<IsbnDto> {
        return ISBNS
    }
    @GetMapping("small/isbns")
    fun getSmallIsbns(): List<IsbnDto> {
        return SMALL_ISBNS
    }
    companion object {
        const val MESSAGE = "I will now give you 1.000.000 ISBN numbers in a Spring MVC Non-Reactive way"
    }
}

For this implementation and the following, we need to keep in mind that ISBNS is our list of half a million ISBN 13 records and that SMALL_ISBNS is our list of 50 records. Since the KoFu code is declarative, I also wanted all the other code to be as declarative as possible where reason applies.The application context (the root path of the webservice) doesn’t have to be configured in the application.properties for these tests. Not only doesn’t have to, but it can be detrimental to a valid result. This is why I’m not using it. For the rest of the implementation we just need to keep two endpoints in mind: http://localhost:8080/api/traditional/non/isbns and http://localhost:8080/api/traditional/non/small/isbns.

5.2. Traditional Spring WebFlux MVC Reactive REST service

Reactive programming and the possibilities Spring has to offer can be very quickly reduced in practical terms to the use of Flux and Mono. As explained before, these are publishers responsible for processing the data. They may have extra subscribers associated with them. The dependencies we are studying in this case are:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>${spring-boot.version}</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
    <version>${spring-boot.version}</version>
</dependency>

And the implementation of the REST controller can be done in a very similar way:

@RestController
@RequestMapping("/api/traditional")
class ISBNController {
    var mapper = ObjectMapper()
    @GetMapping("hello")
    fun getMessage(): Mono<String> {
        return Mono.just(MESSAGE)
    }
    @GetMapping
    fun getJsonMessage(): Mono<JsonNode> {
        return Mono.fromCallable {
            val createObjectNode = mapper.createObjectNode()
            createObjectNode.put("message", MESSAGE)
            createObjectNode
        }
    }
    @GetMapping("isbns")
    fun getIsbns(): Flux<IsbnDto> {
        return Flux.fromIterable(ISBNS)
    }
    @GetMapping("small/isbns")
    fun getSmallIsbns(): Flux<IsbnDto> {
        return Flux.fromIterable(SMALL_ISBNS)
    }
    companion object {
        const val MESSAGE = "I will now give you 1.000.000 ISBN numbers in a Spring MVC Reactive way"
    }
}

As we can see here, we’ll be using Flux only. In very simple terms, this is how we publish a list of elements in a reactive way. This is thus the only difference in implementation which allows us to check for differences in behavior between the first and the second implementations knowing that those differences should come only from the fact that the second is reactive and the first one is not. This service will be, as we can see from the implementation, available on http://localhost:8081/api/traditional/isbns and http://localhost:8081/api/traditional/small/isbns.

5.3. Spring KoFu implementation

This implementation is where our case is centered. Here we use a radically different approach. Instead of annotation and as we said before, we will be implementing everything in a declarative way. In my opinion, it looks quite good right off the bat. For this, we’ll need a few very important dependencies. Before adding them to our repo, we need to make sure we can reach the spring experimental libraries repo. We need to add this to our pom file:

<repositories>
    <repository>
        <id>spring-milestone</id>
        <name>Spring Milestone Repository</name>
        <url>https://repo.spring.io/milestone</url>
    </repository>
</repositories>

Then we can add the necessary dependencies:

<dependency>
    <groupId>org.springframework.fu</groupId>
    <artifactId>spring-fu-kofu</artifactId>
    <version>${spring-fu-kofu.version}</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
    <version>${spring-boot.version}</version>
</dependency>

We can see very clearly that we are also using web-flux behind Spring KoFu.
With this, we can implement the rest:

val app = reactiveWebApplication {
    beans {
        bean<ISBNSampleService>()
        bean<ISBNSampleHandler>()
    }
    webFlux {
        port = if (profiles.contains("test")) 8181 else 8080
        router {
            val handler = ref<ISBNSampleHandler>()
            GET("/api/kofu/hello", handler::hello)
            GET("/api/kofu", handler::json)
            GET("/api/kofu/isbns", handler::isbns)
            GET("/api/kofu/small/isbns", handler::smallIsbns)
        }
        codecs {
            string()
            jackson()
        }
    }
}
data class ISBNMessage(val message: String)
class ISBNSampleService {
    fun generateMessage() = "I will now give you 1.000.000 ISBN numbers in a Spring KoFu Reactive way"
}
class ISBNSampleHandler(private val isbnSampleService: ISBNSampleService) {
    fun hello(request: ServerRequest) = ServerResponse.ok().bodyValue(isbnSampleService.generateMessage())
    fun json(request: ServerRequest) = ServerResponse.ok().bodyValue(ISBNMessage(isbnSampleService.generateMessage()))
    fun isbns(request: ServerRequest) = ServerResponse.ok().bodyValue(ISBNS)
    fun smallIsbns(request: ServerRequest) = ServerResponse.ok().bodyValue(SMALL_ISBNS)
}
class ISBNStacksKoFuReactiveLauncher{
    companion object{
        @JvmStatic
        fun main(args: Array<String>) {
            app.run()
        }
    }
}

If you are familiar with NodeJS, I’m very sure that you are very surprised at the similarities between this and the same creation of a service like this in NodeJS. An example very similar to this one can be found on their GitHub repo. One of the advantages of going annotationless we can already see. A simple web service like this one only costs us a few lines of code. We can also see everything add make addon’s like the string() and jackson() codes in a completely declarative way. The implementation is quite self-explanatory, and we can see clearly that this service and the different data lists will be available on http://localhost:8080/api/kofu/isbns and http://localhost:8080/api/kofu/small/isbns.

6. Starting the services

To make it easy to perform our tests, I’ve created a docker-compose.yml file, which allows us to build and start all three services at once. I’ve also created a Makefile, with several written commands which will help us perform our performance tests. Before we start testing or doing anything with our environment we need to run this command:

make docker-clean-build-start

Once the command finishes running, we should see three running containers. We will however perform tests individually. If we perform tests while the three containers are running at the same time, they might influence each other, they may start consuming too much CPU or too much memory leading to unreliable or misleading results.
I have performed such tests locally before and couldn’t reach any conclusive result. As mentioned before, we will perform tests that I know point to some important conclusions. All reports of all my tests are placed in folder docs/reports from the root folder of the project.
We will perform tests with the locust performance tool. All locust python scripts are located in the root locust folder of the project.
To avoid deviations in our tests, we’ll also run the headless version of Locust. This means that we will see results in generated graphics from the CSV files generated.

6.1. locust-sequence-start test

In this test, we are going to perform load requests to the services, one by one, using this script located in the Makefile of the project:

locust-sequence-start:
   docker stop jofisaes_isbn_stacks_reactive
   docker stop jofisaes_isbn_stacks_mvc
   docker restart jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/kofu && locust --host=localhost --headless -u 2000 -r 1 --run-time 5m --csv kofu --exit-code-on-error 0
   docker stop jofisaes_isbn_stacks_reactive
   docker restart jofisaes_isbn_stacks_mvc
   docker stop jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/web && locust --host=localhost --headless -u 2000 -r 1 --run-time 5m --csv web --exit-code-on-error 0
   docker restart jofisaes_isbn_stacks_reactive
   docker stop jofisaes_isbn_stacks_mvc
   docker stop jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/webflux && locust --host=localhost --headless -u 2000 -r 1 --run-time 5m --csv webflux --exit-code-on-error 0

The locust root folder contains several sub-folders. The kofu, web and webflux sub-folders contain the scripts responsible to connect to large content endpoints and they represent Spring Kofu, Spring Web (Non-reactive) and Spring WebFlux respectively. From the script we see that we stop non-required containers for each single test, and we restart the target container. We give it a 5-second delay to ensure that our service has started. On average, and on a current machine, all of these implementations take less than 5 seconds to start. Locust then starts with 1 user and ramps up by a rate of 1 user per second up to 2000 users per second making requests. This ensures that we’ll reach a threshold for all services. We will, however not reach 2000 users, purely because we are also limited to 5 minutes on each test. The complete test combined takes about 15 minutes to complete, yielding the results presented in the following graphs.

We must first interpret the graphs correctly. On both graphs, we see an axis that goes from 0 to 300. These are the number of spawned users. We initially want to ramp up from 0 to 2000. However, due to machine restrictions, I found out that for my case, 300 was de best number to get to. It is not a huge number, but significant enough to measure which service starts to get disrupted first for long processing requests. As seen in the graph, the first service to start failing is KoFu, at about 60 spawned users. This is followed by the WebFlux service which starts failing at around 170 spawned users. The Web service doesn’t fail at all for this ramp-up. At this point, this could mean bad or good news. For KoFu and WebFlux the bad could mean that these architectures are just not good. But we know already, and therefore already slightly biased that WebFlux is just better at being resilient than a non-reactive service. On the good side, it could mean that they took too many requests and since they take longer to process, they got stuck in reactive limbo. If this is so, then we are actually getting a positive result. Reactive programming is not really thought out for long-lasting requests. It actually goes firmly against the handling of long-lasting requests. If we have long-lasting requests and want to comply with reactive programming then firstly we have something wrong in how we handle requests and secondly, we need to solve this problem.
For the moment the take from this test, and the other consistent ones I’ve made, is that when dealing with long-lasting requests, Spring KoFu fails first and fast, followed by the reactive service with Spring WebFlux, and finally we see that the non-reactive Spring Web service never failed in our tests. We also know that these could mean bad or good results. And this is why we now move on to our fast processing requests test.

6.2. locust-small-load-sequence-start

In local tests using my own computer, I realized that the best way to test the way the three services behave is simply to let them run for a long time. If we want to repeat such a test, then it will take about 30 minutes to complete:

locust-small-load-sequence-start:
   docker stop jofisaes_isbn_stacks_reactive
   docker stop jofisaes_isbn_stacks_mvc
   docker restart jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/small/kofu && locust --host=localhost --headless -u 2000 -r 2000 --run-time 10m --csv kofu --exit-code-on-error 0
   docker stop jofisaes_isbn_stacks_reactive
   docker restart jofisaes_isbn_stacks_mvc
   docker stop jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/small/web && locust --host=localhost --headless -u 2000 -r 2000 --run-time 10m --csv web --exit-code-on-error 0
   docker restart jofisaes_isbn_stacks_reactive
   docker stop jofisaes_isbn_stacks_mvc
   docker stop jofisaes_isbn_stacks_kofu
   sleep 5
   cd locust/small/webflux && locust --host=localhost --headless -u 2000 -r 2000 --run-time 10m --csv webflux --exit-code-on-error 0

This means that we’ll immediately begin the tests with a spawning rate of 2000 users per second. This will continue for 10 minutes on each service on separate occasions. With the resulting CSV files, I was able to build the following graphs:

In the graph above, it is not easy to distinguish between the performance of the different services. This has to do with the startup of the services and how requests are handled the first time they get to the service after a restart. This, combined with the way records are placed on the CSV files, generates a first ramp-up-like curve, which is not useful for our tests. If we cut off the startup section and visualize only the stable data afterward we see something more conclusive.

If we look at the different data, we see that in general that our Spring KoFu service was able to process much more requests than the other two. We also see that our Spring Web and Spring WebFlux services seem to duel with each other. If we look more closely, we also see that on average WebFlux does seem to process more requests than just the plain Web. Another thing we observe and that was also very surprising to me is that the Spring KoFu requests/second curve has a somewhat mirror shape to the other services. Spring Kofu seems to have, instead of low-performance peaks, it has instead high-performance peaks.
So now it seems that Spring KoFu can process more requests at a time than the other Spring services.
The Locust performance tests also provide us with an average response times table. Here we can see how fast our requests are processed using the different implementations:

Type	Name	Request Count	Median Response Time	Average Response Time	Min Response Time	Max Response Time	Requests/s
GET	/api/kofu/small/isbns	488189	1700.0	1663.5698557514636	868.2196040000001	28160.157482	812.7795532454088
GET	/api/traditional/non/small/isbns	459502	1700.0	1810.3353489623726	319.1146300000014	76340.247267	765.9678731389113
GET	/api/traditional/small/isbns	462978	1700.0	1768.4598885850269	304.2018510000162	46094.92280300003	770.7296145412935

Reading the data from the table, we can see yet again, that Spring KoFu does seem to offer better performance than the other Spring services. In terms of the number of requests, the Spring Non-Reactive Service processed 459502 requests. This was followed by Spring Reactive WebFlux which processed 462978 requests. This is already a significant difference of 3476 requests. This is made even better by Spring KoFu which was able to process 488189 requests. This is further an increase of 25211 requests! This is also reflected in the average response times. We can see that we have no errors on any of the tests, which means that our services were always available during testing. Another interesting metric that we are interested in is the longest request of all our requests. Clearly, the longest comes from the non-reactive service where a potential user would have to wait 76 seconds. This is improved by the plain reactive service with a wait time of 46 seconds and finally the Kofu service with a wait time of 28 seconds. If these numbers sound long to you it is because they are. But this is also because these tests are running on a local machine where the scale of time, failure, and delay is very different than with real servers. Finally, we also see that this all matches up with the requests per second where we also see improvements going from non-reactive to KoFu.
The take-away from this test and the previous one is that now it seems that Spring KoFu will react much faster to fast processing requests, it will be able to process more requests at the same time and pushes itself to performance peaks.

6.3. Service startup

This is our last test for the Spring KoFuframework. In this case, we are going to test how long does the service takes to startup. In this case, I’ve also created a command in the Makefile to help us in our tests:

local: no-test
   mkdir -p bin
   cp isbn-stacks-rest-kofu-mvc-reactive/target/isbn-stacks-rest-kofu-mvc-reactive-*.jar bin/isbn-stacks-rest-kofu-mvc-reactive.jar
   cp isbn-stacks-rest-traditional-mvc-non-reactive/target/isbn-stacks-rest-traditional-mvc-non-reactive-*.jar bin/isbn-stacks-rest-traditional-mvc-non-reactive.jar
   cp isbn-stacks-rest-traditional-mvc-reactive/target/isbn-stacks-rest-traditional-mvc-reactive-*.jar bin/isbn-stacks-rest-traditional-mvc-reactive.jar
   cp isbn-stacks-rest-kofu-plain/target/isbn-stacks-rest-kofu-plain-*.jar bin/isbn-stacks-rest-kofu-plain.jar

What this command does is copy all jars into a bin folder in the root.
This way we can start them by simply running java -jar , registering their startup time, and then stopping them with Ctrl-C.

Description	KoFu	WebFlux	Web	Control
	2.133	2.42	2.065	1.383
	1.436	2.026	1.895	1.566
	1.349	2.233	1.876	1.412
	1.699	2.128	1.858	1.329
	1.471	2.079	1.875	1.350
	1.492	2.34	1.87	1.431
	1.364	2.199	1.917	1.577
	1.591	1.994	1.912	1.427
	1.477	2.056	1.838	2.020
	1.426	2.013	1.94	1.354
AVERAGE	1.544	2.149	1.905	1.485

We can also see here a sign that Spring KoFu does seem to start faster. If we compare these results to WebFlux or Web we do see a remarkable improvement.

7. Conclusion

We started this journey by making 3 services and making them as simple as possible with only two endpoints. A fast endpoint to support only 50 ISBNs and a slow endpoint to support half a million ISBNs. We performed tests against them, and although we cannot make definite conclusions, we can, however, observe some behavior as a sign of confirmation of the statement of the Spring-Fu team on GitHub:
Spring Fu is an incubator for JaFu (Java DSL) and KoFu (Kotlin DSL) designed to configure Spring Boot explicitly with code in a declarative way with great discoverability thanks to auto-complete. It provides fast startup (40% faster than regular autoconfiguration on a minimal Spring MVC app), low memory consumption and is a good fit with GraalVM native thanks to its (almost) reflection-less approach.
We did see that, while coding, IntelliJ does seem to offer great auto-complete support. Though this is all true, it is likely not the most interesting aspect of Spring Fu. We did also observe that the service startup seems to be faster, although we couldn’t exactly see a 40% reduction in startup time. Although performance is not something explicit in any of the Spring Fu sources, it is in some way implicit because of its reflection-less nature. They also mention that a functional approach is much more efficient in the JVM. This at least is the theory and we do observe a remarkable difference in improvement with Spring-KoFu in our tests. We have seen this in response times and load capacity.
Making these tests was quite laborious, but it is very rewarding to see the very compelling signs of how worthwhile might be to go to a completely declarative way of working and leaving annotations behind.
As I mentioned in the introduction, I did take the Spring Certification and a lot of it was about annotations. I still feel and I always do that it is worthwhile to take certifications. Nonetheless, technology does move fast, and moving forward what we seem to achieve is an ongoing improvement in our path to enrich our knowledge in technology in general. What we know to be efficient and fantastic today may change tomorrow in the world of technology and that is ok. I’m very down with this to be very honest. My best advice to you if you feel that all of those years of learning annotations in the past are somehow being threatened by these new developments, is to just let it go and move on. What I have learned in the past is great, and I can take that and modify it to adapt it to this new way of working. If we see a Spring Functional production-ready version in the future, probably Spring-Fu, then that is all for the better and working hard to keep up with the changes gives us all a bit more magic to whatever we do and this is what we need to keep doing.
I have placed all the source code of this application on GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it. I tried to keep it small, concise and I left many small details out.
Thanks in advance for reading!

8. References

City Library — An advanced guide to Circuit Breakers in Kotlin

João Esperancinha — Fri, 27 Dec 2024 17:21:54 +0000

1. Introduction

I’ve always been very interested in circuit breakers and how do they work in Software Architecture. I’ve also worked on projects where one of the implementations of such design patterns was used. Namely, for online large distribution stores, many companies love to use Hystrix. However, Hystrix is reaching the end of the road and a new replacement is available called Resilience4j. Regardless, I’ve seen many ways in which Hystrix has been used. The possibilities are quite extensive, but in practice, in my professional life, I’ve seen that not even half of the potential of a Circuit-Breaker design pattern implementation is used. Like Resilience4J, there are many different implementations of this software design pattern available like FailSafe, GoBreaker, Circuit-Breaker-For-Scala, and many others.
It’s always good to remind ourselves that the idea of Circuit-Breaker has existed since the 19th Century and back then it was related to how a switch would break a circuit should the electrical current increase up to unacceptable levels. Fast-forward to 2017, Michael Nygardpublishes Release IT! Design and Deploy Production-Ready Software (Pragmatic Programmers) where he brings the idea of Circuit-Breakers design pattern to software development.
In this article, I’m assuming that you already have an idea of what circuit-breakers really are and what this design pattern is fundamentally about. This design pattern allows many sorts of types of configuration, and we are going to have a look at the most important ones of them as we go along. I’m also assuming that you are quite familiar with how Spring works and also that you have some idea of how the performance test tool Locust works. It is also important to have a general idea of how Reactive programming is used in Spring WebFlux. The implementation we are going to see is located on GitHub.

2. Case

For demonstration purposes, we’ll make a city library!. We are essentially going to provide one or more books to a library and just provide them to users online for reading purposes. Before we actually build our library’s software, we are right off-the-bat concerned if the library can live up to resilient expectations. As many users are going to use the library, it must be there constantly providing to users. The Library has a perfect service already working in the cloud using Reactive Spring WebFlux technologies. The only drawback is that the cloud provider goes into maintenance mode from time to time. The Library has a fallback service for this though. A very old server machine in the attic has been designed for high availability as well. Although it is an on-premise installation, it works well enough for the usual library user. It is expected that library users will experience some latency should this fallback be needed.

3. Circuit breakers

In this section, we’ll go through unit tests I’ve designed in order to analyze the specific properties of circuit breakers. We will check how the circuit-breaker status changes and understand how to see that via the Spring health endpoint. Finally, we will make a thorough analysis of the circuit-breakers created working together and make a graphic analysis of the results.

In order to continue with our example, it is important that we first take a look at some important implementation basics. It is also important to understand that in the Kotlin code, practically nothing changes in regard to the setup of our circuit breakers and so, we’ll just have a look at the implementation of circuit-breaker implementation for test 1.

@Service
open class AlmG1BookService(
    private val webClientInterface: WebClientInterface
) {
    private val logger = KotlinLogging.logger {}

    @CircuitBreaker(name = ALMR_TC_1, fallbackMethod = "getBookByIdJPA")
    open fun getBookCBById(id: Long): Mono<BookDto> =
        webClientInterface.getBookViaReactiveServiceById(id)

    open fun getBookByIdJPA(id: Long, exception: Exception): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }
    open fun getBookByIdJPA(id: Long, exception: WebClientRequestException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }

    open fun getBookByIdJPA(id: Long, exception: ReactiveAccessException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }

    open fun getBookByIdJPA(id: Long, exception: CallNotPermittedException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }

    open fun getBookByIdJPA(id: Long, exception: TimeoutException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }

    open fun getBookByIdJPA(id: Long, exception: IgnoredException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.getBookViaJpaServiceById(id)
    }

    @CircuitBreaker(name = ALMR_TC_1, fallbackMethod = "createBookByIdJPA")
    open fun createBook(bookDto: BookDto): Mono<BookDto> {
        return webClientInterface.sendBookViaReactiveService(bookDto)
    }

    open fun createBookByIdJPA(bookDto: BookDto, exception: WebClientRequestException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.sendViaJpaServiceBook(bookDto)
    }

    open fun createBookByIdJPA(bookDto: BookDto, exception: ReactiveAccessException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.sendViaJpaServiceBook(bookDto)
    }

    open fun createBookByIdJPA(bookDto: BookDto, exception: CallNotPermittedException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.sendViaJpaServiceBook(bookDto)
    }

    open fun createBookByIdJPA(bookDto: BookDto, exception: TimeoutException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.sendViaJpaServiceBook(bookDto)
    }

    open fun createBookByIdJPA(bookDto: BookDto, exception: IgnoredException): Mono<BookDto> {
        logger.info("Current Exception:", exception)
        return webClientInterface.sendViaJpaServiceBook(bookDto)
    }

    companion object {
        const val ALMR_TC_1 = "almr_testcase_1"
    }
}

Looking at this class, we see that the name of circuit-breaker for test case 1 is almr_testcase_1. The implementation is quite easy to follow. We only need to see that the fallback method is capable to receive the input parameter of the original method and that it can use that parameter in the alternative method call. For example, if we try to get a Book with Id=1 and then we get a TimeoutException, we’ll then fall into the overloaded method getBookByIdJPA, which then tries to make a call to our JPA service. In our example, we assume that the JPA service is always available. This is all possible because we have annotated our original method, the one called getBookCBById with CircuitBreaker. That way, if we call getBookViaReactiveServiceById and it succeeds, we don’t have to use the fallback method. If it does, however, then we fall into the CircuitBreaker algorithm, which is the purpose of this article.
In order to understand the behind-the-scenes of this project, it is also important that we understand what does our WebClientInterface does. In this case, it is important to check the implementation at that happens while running the containers.

@Component
@Profile("prod", "docker")
class WebClient(
    @Value("\${org.jesperancinha.management.reactive.host}")
    val reactiveHost: String,
    @Value("\${org.jesperancinha.management.mvc.host}")
    val mvcHost: String
) : WebClientInterface {


    private val webClientReactive: WebClient = create("http://$reactiveHost:8081")
    private val webClientMvc: WebClient = create("http://$mvcHost:8082")

    override fun getBookViaReactiveServiceById(id: Long): Mono<BookDto> = webClientReactive.get()
        .uri("/api/alm/reactive/books/$id").retrieve().bodyToMono()

    override fun getBookViaJpaServiceById(id: Long): Mono<BookDto> = webClientMvc.get()
        .uri("/api/alm/mvc/books/$id").retrieve().bodyToMono()

    override fun sendBookViaReactiveService(bookDto: BookDto): Mono<BookDto> = webClientReactive.post()
        .uri("/api/alm/reactive/books/create")
        .header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
        .body(Mono.just(bookDto), BookDto::class.java)
        .retrieve().bodyToMono()

    override fun sendViaJpaServiceBook(bookDto: BookDto): Mono<BookDto> = webClientMvc.post()
        .uri("/api/alm/mvc/books/create")
        .header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
        .body(Mono.just(bookDto), BookDto::class.java)
        .retrieve().bodyToMono()
}

As we have seen before, this is where we create our publishers or Mono’s. They will make calls to the reactive service or the non-reactive service depending on the interactions with the CircuitBreaker algorithm.
Now it’s time to check what happens in the tests. In this case, the unit tests are quite extensive, and so I will not show the full class in this article as I usually do. In this case, only important pieces of the code will be shown for illustration purposes. The code can however be checked on GitHub.

3.1. Gate 1 test — almr_testcase_1— General properties.

In this test, we check the registerHealthIndicator, slidingWindowSize, slidingWindowSizeType, minimumNumberOfCalls
, automaticTransitionFromOpenToHalfOpenEnabled and waitDurationInOpenState:

registerHealthIndicator: true
slidingWindowSize: 10
slidingWindowType: "COUNT_BASED"
minimumNumberOfCalls: 5
failureRateThreshold: 50
slowCallRateThreshold: 50
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 1s
recordExceptions:
  - org.springframework.web.client.HttpServerErrorException
  - java.util.concurrent.TimeoutException
  - java.io.IOException
  - org.jesperancinha.management.gate.exception.ReactiveAccessException
ignoreExceptions:
  - org.jesperancinha.management.gate.exception.IgnoredException

The first step is to register the health indicator. This is a feature that is very important to check the status of the CircuitBreaker. In order for this to work, we need to activate the right management endpoints provided by the Spring actuator:

management:
  endpoints.web.exposure.include: "*"
  endpoint.health.show-details: always
  health:
    circuitbreakers.enabled: true
    ratelimiters.enabled: true

This will make our actuator endpoint available at:

http://localhost:8080/api/almg/actuator/health

And this will give a result like this:

{
  "status": "UP",
  "components": {
    "circuitBreakers": {
      "status": "UP",
      "details": {
        "almr_testcase_3": {
          "status": "UP",
          "details": {
            "failureRate": "-1.0%",
            "failureRateThreshold": "50.0%",
            "slowCallRate": "-1.0%",
            "slowCallRateThreshold": "50.0%",
            "bufferedCalls": 3,
            "slowCalls": 0,
            "slowFailedCalls": 0,
            "failedCalls": 0,
            "notPermittedCalls": 0,
            "state": "CLOSED"
          }
        }
      },
      "diskSpace": {
        "status": "UP",
        "details": {
          "total": 62725623808,
          "free": 56340504576,
          "threshold": 10485760,
          "exists": true
        }
      },
      "ping": {
        "status": "UP"
      },
      "rateLimiters": {
        "status": "UNKNOWN"
      }
    }
  }
}

This is just a short form of the circuit-breaker health status. We normally see all statuses of all circuit-breakers. We see only almr_testcase_3 CircuitBreaker status in this case. Notice that UP is the same as CLOSED in this case.
In order to make our unit test work and this is valid for all our test cases, I created a domain model specific for the health status in order to get the status property of each CircuitBreaker. This is located on GitHub.

We can now have a look at the remaining properties. The slidingWindowSize, slidingWindowSizeType, minimumNumberOfCalls, automaticTransitionFromOpenToHalfOpenEnabled, failureRateThreshold, and waitDurationInOpenState, are all essential properties we need to configure. The documentation provided by the Resilience4J team is very extensive and documents this really well. It is nonetheless important to quickly go through these properties. Perhaps the first thing we need to look at is how does the CircuitBreaker determines when do we go to an open state. Well, it always goes to an open state once a failure has occurred. The logical question that follows is how does it know to go back to a closed state from an open state? The answer is, it doesn’t. It first needs to go to a half-open state. This happens after a timeout. This timeout is determined by our first property: waitDurationInOpenState. Once this happens, the state goes from open to half-open. There, the CircuitBreaker will redirect calls to the original service. It can here go back to a closed status or just go back to an open state. This is determined by our next property failureRateThreshold. This failure rate also determines going from close to open also on successful requests in some cases. The failure rate is a figure calculated by using other configured parameters. The minimumNumberOfCalls represents the minimum calls needed to start performing this calculation. The slidingWindowSize can be represented by either second or by a number of requests. All requests that fall within this sliding window are considered to calculate the rate of failing requests. Up until now, we know how to determine only one type of failing request. Those are the ones we do not ignore with property ignoreExceptions. Finally, we see that the property automaticTransitionFromOpenToHalfOpenEnabled is true. This only means we do not need to wait for a request to be done in order to automatically make the switch to a half-open state from an open state. It is used for corner cases.
Now it’s time to have a look at the first test. This is a simple test for properties recordExceptions and ignoreExceptions:

@Test
fun testGetBookByIdTestWhenIgnoredExceptionThenNull() {
    every { webClient.getBookViaReactiveServiceById(100L) } returns Mono.error(IgnoredException())
    every { webClient.getBookViaJpaServiceById(100L) } returns Mono.just(BookDto(0L, "Solution"))
    val bookById = almG1BookService.getBookById(100L)
    repeat(10) {
        val bookById = almG1BookService.getBookCBById(100L)
        bookById.shouldNotBeNull()
        bookById.blockOptional().ifPresent { book ->
            book.title.shouldBe("Solution")
            getCBStatus().shouldBe("UP")
        }
    }
    getCBStatus().shouldBe("UP")
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book -> book.title.shouldBe("Solution") }
}

In this case, we see that we see that the CircuitBreaker status remains open even though we threw an exception. Our IgnoredException has of course been ignored.
We can now have a look at our first complete test. Note that all the following tests will be done as a reference to this one:

every { webClient.getBookViaReactiveServiceById(100L) } returns Mono.error(ReactiveAccessException())
every { webClient.getBookViaJpaServiceById(100L) } returns Mono.just(BookDto(0L, "SolutionOpen"))
getCBStatus().shouldBe("UP")
repeat(4) {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionOpen")
    }
}

In the first instance, we create 4 requests that fail against the reactive service. We get the predicted response from the non-reactive service: SolutionOpen.

getCBStatus().shouldBe("UP")
every { webClient.getBookViaReactiveServiceById(100L) } returns Mono.just(BookDto(0L, "SolutionClosed"))
runBlocking {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}

At this point, we make a request that goes well via the reactive service. If you didn’t notice yet, we just made 4 unsuccessful requests followed by a successful request. Our minimumNumberOfCalls is set to 5 and the rate is set to 50%. We have now clearly crossed the minimum acceptable rate.

getCBStatus().shouldBe("CIRCUIT_OPEN")
repeat(3) {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionOpen")
    }
}

This is why the circuit now remains open for the remaining 3 successful calls. After this balanced ratio, we should be able to go back to a closed state.

repeat(40) {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionOpen")
    }
}

We do not get back to a closed state after 40 calls because they all happen in less than 1 second, which is what we have configured for waitDurationInOpenState.

sleep(1000)
getCBStatus().shouldBe("CIRCUIT_HALF_OPEN")
repeat(4) {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}

After waiting 1 second, we are now in a half-open state, but because we are back, the sliding window is reset. We need one more request to get back to an open state.

getCBStatus().shouldBe("CIRCUIT_HALF_OPEN")
runBlocking {
    val bookById = almG1BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}
getCBStatus().shouldBe("UP")

This test shows, how to use all of these simple properties. We are now going to dive into other less-known but equally important properties.

3.2. Gate 2 test — almr_testcase_2-automaticTransitionFromOpenToHalfOpenEnabled

By comparing this test to test one, we just want to test the effect of
the automaticTransitionFromOpenToHalfOpenEnabled property:

registerHealthIndicator: true
slidingWindowSize: 10
slidingWindowType: "COUNT_BASED"
minimumNumberOfCalls: 5
failureRateThreshold: 50
slowCallRateThreshold: 50
automaticTransitionFromOpenToHalfOpenEnabled: false
waitDurationInOpenState: 1s
recordExceptions:
  - org.springframework.web.client.HttpServerErrorException
  - java.util.concurrent.TimeoutException
  - java.io.IOException
  - org.jesperancinha.management.gate.exception.ReactiveAccessException
ignoreExceptions:
  - org.jesperancinha.management.gate.exception.IgnoredException

This test case is exactly the same case as case 1 with the exception that we do not automatically move back to half-open automatically.

repeat(40) {
    val bookById = almG2BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionOpen")
    }
}
sleep(1000)
getCBStatus().shouldBe("CIRCUIT_OPEN")
repeat(4) {
    val bookById = almG2BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}

After we send the 40 calls in less than one second, we wait one second as in the previous case. however, this time we get an open state.

getCBStatus().shouldBe("CIRCUIT_HALF_OPEN")
runBlocking {
    val bookById = almG2BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}

Following one or more requests, it finally jumps to a half-open state.

3.3. Gate 3 test — almr_testcase_3 — Slow calls

In this test, we check properties slowCallDurationThreshold and slowCallRateThreshold:

registerHealthIndicator: true
slowCallRateThreshold: 50f
slowCallDurationThreshold: 100
slidingWindowSize: 10
slidingWindowType: "COUNT_BASED"
minimumNumberOfCalls: 5
failureRateThreshold: 50
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 1s
recordExceptions:
  - org.springframework.web.client.HttpServerErrorException
  - java.util.concurrent.TimeoutException
  - java.io.IOException
  - org.jesperancinha.management.gate.exception.ReactiveAccessException
ignoreExceptions:
  - org.jesperancinha.management.gate.exception.IgnoredException

The other type of error we can count are errors based on slow calls. This is managed through slowCallRateThreshold and slowCallDurationThreshold. These are measured on percentage and milliseconds.

every { webClient.getBookViaReactiveServiceById(100L) } returns Mono.just(BookDto(0L, "SolutionSlow"))
    .delayElement(
        Duration.ofMillis(200)
    )
every { webClient.getBookViaJpaServiceById(100L) } returns Mono.just(BookDto(0L, "SolutionOpen"))
getCBStatus().shouldBe("UP")
repeat(4) {
    val bookById = almG3BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionSlow")
    }
}

Other than this, this test is no different from test case 1. The only difference is that this one is based on timeouts, and we can see that it handles the timeouts the same way as all the other accepted exceptions.

3.4. Gate 4 test — almr_testcase_4 — permittedNumberOfCallsInHalfOpenState

In comparison with the Gate test we now look at the permittedNumberOfCallsInHalfOpenState property:

registerHealthIndicator: true
slidingWindowSize: 10
slidingWindowType: "COUNT_BASED"
minimumNumberOfCalls: 5
failureRateThreshold: 50
slowCallRateThreshold: 50
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 1s
permittedNumberOfCallsInHalfOpenState: 2
recordExceptions:
  - org.springframework.web.client.HttpServerErrorException
  - java.util.concurrent.TimeoutException
  - java.io.IOException
  - org.jesperancinha.management.gate.exception.ReactiveAccessException
ignoreExceptions:
  - org.jesperancinha.management.gate.exception.IgnoredException

The property permittedNumberOfCallsInHalfOpenState, means that the half-open state will remain up to a maximum of the calls configured.

repeat(40) {
    val bookById = almG4BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionOpen")
    }
}
sleep(1000)
getCBStatus().shouldBe("CIRCUIT_HALF_OPEN")
repeat(4) {
    val bookById = almG4BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}
getCBStatus().shouldBe("UP")

In our case, we have two calls configured. Since they succeed, and we pass the 2 threshold, this means that we go directly to a closed state. This property should be used carefully because it can potentially bypass certain calculations that we may be counting on.

3.5. Gate 5 test — almr_testcase_5 — TIME_BASED

Tests the TIME_BASED value of the slidingWindowType:

registerHealthIndicator: true
slidingWindowSize: 1
slidingWindowType: "TIME_BASED"
minimumNumberOfCalls: 5
failureRateThreshold: 50
slowCallRateThreshold: 50
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 1s
recordExceptions:
  - org.springframework.web.client.HttpServerErrorException
  - java.util.concurrent.TimeoutException
  - java.io.IOException
  - org.jesperancinha.management.gate.exception.ReactiveAccessException
ignoreExceptions:
  - org.jesperancinha.management.gate.exception.IgnoredException

If the slidingWindowType is TIME_BASED then we cannot easily predict what is going to happen. We basically just say that our window slides in the orderly manner of time. Time is a variable not easy to keep up with and so the unit test for case 5 contains lots of variations in terms of expected results.

getCBStatus() shouldBeIn listOf("UP", "CIRCUIT_OPEN")
repeat(40) {
    val bookById = almG5BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title shouldBeIn listOf("SolutionClosed", "SolutionOpen")
    }
}
sleep(1000)
getCBStatus() shouldBeIn listOf("UP", "CIRCUIT_OPEN", "CIRCUIT_HALF_OPEN")
repeat(4) {
    val bookById = almG5BookService.getBookCBById(100L)
    bookById.shouldNotBeNull()
    bookById.blockOptional().ifPresent { book ->
        book.title.shouldBe("SolutionClosed")
    }
}

4. Running the example

To make things easy, I’ve created some scripts to run the whole application. It builds everything in one go and then all you have to do is start locust

Start docker-compose

shell make docker-clean-build-start

Start locust

shell make locust-start

Go to the locust main page on http://0.0.0.0:8089/

Make a choice. For me, it works better to have 10 users and a spawn rate of 10 users per second.
You can optionally create a book or several. Check the repo for more examples.

shell curl -X POST -H "Content-Type: application/json" --data '{"id": 1, "title": "Wild", "authors": [ "Chery Strayed" ], "year": 2012, "publisher": "Uitgeverij Rainbow B.V." }' http://localhost:8080/api/almg/books/g1

5. Locust Graphics

Locust is quite an easy tool to learn, especially for this example. Please look into the locust documentation for the installation notes. I’ve performed some tests and the results are available on GitHub.

Our default examples run on a waitDurationInOpenState of 1s. This doesn’t provide a great result purely because 1 second in the open state is very difficult to visualize. The time in between where we don’t get any requests is the time I’ve taken down the reactive service followed by the MVC-JPA service. To stop these services, I’ve created some scripts for that. So if you want to stop the reactive service, you can use make docker-stop-reactive. If you want to stop the MVC service you can run make docker-stop-mvc.
The past results are the reason why I’ve increased that value to 10s in the docker Spring profile.

This graph is broken into three very visible sections. The left section is where I’ve started the docker images and stopped the reactive service. As you can see, there seems to be a higher rate interval of roughly 10 seconds at the start of the test, separated by a low rate small interval. All 5 circuit breakers are still quite synchronized with each other, and they all remain in open state for 10 seconds before going back to a half-open state, where they try to reach a successful rate. That never happens and instead, they reach timeout. This contributes to the rate of responses in the following cycle. Because all of these circuit breakers have slightly different rules, their individual contributions to the overall rate of successful requests get mixed and turn quite homogenous between 11:10.15 and 11:11.07. This is where I re-activated the reactive service. This causes the CircuitBreaker to go back to a closed state. The rate of successful messages goes high because of two reasons. We are now getting responses through a reactive service and also the circuit is closed. If the circuit is closed we do not have to introduce any testing costs. At some point around 11:12.37, I stop the reactive service again. The overwhelming amount of messages sent induces a behaviour in the gate where it looks to become unresponsive for about 20 seconds. At timestamp 11:13.02, the rate goes up and down with the same pattern as in the beginning. The CircuitBreaker is again switching from open to half-open and from half-open to open. As we have seen before, these are costly operations that are needed in order to see if we can get the gate to work as efficiently as possible once the reactive service is back online again.

Type	Name	Request Count	Median Response Time	Average Response Time	Min Response Time	Max Response Time	Average Content Size	Requests/s	50%	66%	75%	80%	90%	95%	98%	99%	99.9%	99.99%	100%
GET	/api/almg/books/g1/1	8109	23	58.675751992970994	6.021648000000823	18190.172146000008	54.0	28.879575696069523	23	27	31	34	42	53	77	2200	3100	18000	18000
GET	/api/almg/books/g2/1	8168	23	68.61528426885404	6.179668000015681	18268.55734	54.0	29.089699628252045	23	28	32	35	45	58	91	2200	3100	18000	18000
GET	/api/almg/books/g3/1	8172	22	83.3162769492164	5.762567000033414	29997.531783999988	54.0	29.103945318569508	22	27	30	33	43	55	86	2200	3100	30000	30000
GET	/api/almg/books/g4/1	8028	23	41.57623488938724	6.1160610000001725	18286.493133999982	54.0	28.59110046714097	23	27	31	33	42	52	68	80	3100	18000	18000
GET	/api/almg/books/g5/1	8159	22	87.60661377055992	6.197885999995378	29991.914000999997	54.0	29.057646825037764	22	27	30	33	43	55	82	2300	18000	30000	30000
Aggregated		40636	23	68.0595606234371	5.762567000033414	29997.531783999988	54.0	144.7219679350698	23	27	31	34	43	54	77	2200	3100	30000	30000

This table is probably the most surprising of our results. As we can see, although we’ve seen enormous variations in the rate of successful requests to the gate, NONE of them failed.

And as a control test, we do see that requests do fail if the gate is running, but both reactive and MVC services are down.

6. Conclusion

What we’ve seen in this article are the multiple ways in which a circuit breaker can work in our favor using different configurations. What this example has in common with real-life scenarios is a short-circuit to other services and data sources. A circuit breaker can give you the idea that it works pretty much as a load balancer. However, the difference is quite extensive even though there are some similarities. Both of these solutions can provide an alternative path to data sources that we know provide that same information.
However, this is where the similarities end. In the case of a circuit-breaker, it really depends on requirements, but the alternative data source does not necessarily have to provide the same data as the original source. It is just there to be activated when things go wrong in the main source. The main source can for example be a cluster of services. If somehow a connection goes wrong, it is the job of the circuit-breaker to detect that and redirect all traffic to the new source. This is when we say that our circuit-breaker goes to state open.
As we have seen, we also have a detection algorithm to go back to the main source again once it is available again. That can, in turn, depend on the error frequency given slow requests or just errors on the back end. We can also define the errors that count, and the ones we can ignore. We can also configure it to simply work as a timeout. Using timeouts, we can force the circuit breaker to try and connect to the original source. This of course has a cost, but it can also make it faster to go back to a closed state from a half-open state.
The way our circuit-breaker detects that can be very simple to extremely sophisticated. We need, in that case, to use tools like Locust, Gatlin, JMeter or others, to study precisely how we want that configuration to work best in our favor.
I have placed all the source code of this application on GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it. I tried to keep it small, concise and I left many small details out.
Thanks in advance for your help, and thank you for reading!

7. References

Monitoring Secure Coroutines and WebFlux Reactive applications with Prometheus, Grafana, and InfluxDB — A webcams example

João Esperancinha — Fri, 27 Dec 2024 10:39:33 +0000

1.Introduction

For the past years, we’ve seen an increasing interest in using safe applications and making them as reactive as possible. For this, there are many technologies developed throughout the years, and today the most popular ones seem to be Spring WebFlux and Kotlin Coroutines. As I’ll show in this article, both technologies complement each other and can live in the same JVM ecosystem. As you probably have noticed already, there are many times discussions over which one of them offers the best performance, but essentially they both assume the same principle, which is to avoid anything that is blocking as much as possible. WebFlux takes advantage of the Observer Pattern using Flux and Mono to deliver collections and single objects and Coroutines use Flow and suspend functions to deliver exactly the same respectively.
In this article, we will look at one of the many ways to keep an application's behavior closely under watch and use Prometheus as a data scrapper, InfluxDB as a persistence mechanism, and Grafana to visualize our data. The project for this article is located on GitHub.
When an application gets developed, the assumption that we can move straight to production because the unit tests (TDD — Test Driven Development) and the integration tests did not detect any problem can still be very risky. Even when an experienced tester has made his best effort to pen test the application and has made all the necessary business logic checkups, we are still relying on a human factor and we still haven’t considered system or network issues. BDD (Behaviour Driven Development) many times comes into the equation as a means to prevent errors in production. There is always an extra element of reliability given that BDD checks the behavior of a running environment. A typically known framework used to achieve this is Cucumber.
In a case scenario, and only considering numbers, which gives the business a sense of how safe are we to go to production, is that we have about 90% code coverage, we have given coverage to the logic that the IDE (Integrated Development Environment) did not detect, the BDD tests are fine, and we have a superb and ultra professional DDD (Domain Driven Design) expert which assures us that all of the architecture and boundaries match all the expectations of the PO (Product Owner). We are also very proud to have a team where everyone knows all software design patterns by heart and uses them and thinks about them in their daily developments. Furthermore, everyone knows how to implement and use SOLID (Single responsibility, Open-Closed, Liskov-substitution, Interface Segregation, Dependency Injection) and ACID (Atomicity, Consistency, Isolation, Durability) principles.
Everything seems fine, we go to production, and then we realize that the application isn’t really doing what it was supposed to do. Furthermore, it seems to display all sorts of unexpected behavior. In these scenarios, the problem may well be that we did not have any concerns about the CPU, GPU, memory usage, GC (Garbage Collection), Security and the list goes on. We didn’t check the downtime of the application, we didn’t check the resilience of the application, availability, and robustness. We also didn’t figure out what to do in case of concurrency, multiple users and how much data actually needs to flow in and out of our application.
Monitoring can offer a lot of help to prevent these situations.

2. Technical Requirements

For this article, it is helpful if you have some experience and understanding of Spring and Spring Boot, albeit not essential. We will check out the complete details of the implementation, so if you don’t know some basics, then don’t worry, because I will try my best to keep it easy. What’s really important is to have the following installed on your machine:

Java 17 (any java 17 distribution will do. My best advice is to use SDK Man) to achieve this and use 17-open.
Latest gradle (At least version 7.5 in order to be compatible with Java 17)
A good IDE (IntelliJ or Eclipse for example)
Docker Desktop , but only if you are using an office-oriented machine, with a system like MAC-OS or Windows. My best advice is to just use a Linux machine with 12 cores if possible.

3. Article Goals

When talking about monitoring, we mostly make a few assumptions. It’s easy, it doesn’t take too much time to set up, and we just want to check if the system is ok. Further, there is this generalized assumption that we shouldn’t complicate it too much. Indeed, these are all valid points, up to a point. On the other hand, if we think about what actually needs to be done to make monitoring a reality in our environment, then we quickly begin to realize that there are quite a few moving parts to make monitoring a reality. Let’s think about what we need on a very high level:

We want metrics
We want to have those metrics in our system
We need to fetch metrics
We need to store metrics
We need to visualize these metrics
We need an alert system
Our system should not be impaired in any way in terms of performance

Looking at point 1, mostly, we are sure that we need metrics. Now we need to do the hard work of thinking about “which metrics do we need”? Do we need to monitor the Garbage Collection? Do we need to monitor resources? Do we need to monitor memory usage? Maybe we need all of these metrics, or maybe we already know firsthand what we want to monitor. It’s good to have an idea beforehand of what we want. For points 2 and 3, we need to make sure that we have considered using a push or a pull mechanism. In this article, we are going to look at the pull mechanism. We will see this in both Prometheus and InfluxDB. The biggest advantage of a pull mechanism is that, once our application is running, we can control at any time what we want to receive, by scrapping the data at specific and configurable points in time. A push mechanism implies that the application is constantly sending metrics through the wire even when we want to scale down our metrics fetching mechanism. In other orders, monitoring will always make a small dent in the application performance, but we can scale that down if we use a pull mechanism. What’s important is to make that decrease in performance completely negligible. Next, in point 4 we need to store our data. Fetching mechanisms come many times with ephemeral storage engines. These engines are so called because they can be reset by restart, they are only kept in memory, they have a rolling log or simply they are not well-designed to have persisted data. This is what happens with Prometheus. Although it does have provisions to persist data, it is still very basic and with some limitations in their own words:

If you run the rule backfiller multiple times with the overlapping start/end times, blocks containing the same data will be created each time the rule backfiller is run.
All rules in the recording rule files will be evaluated.
If the interval is set in the recording rule file that will take priority over the eval-interval flag in the rule backfill command.
Alerts are currently ignored if they are in the recording rule file.
Rules in the same group cannot see the results of previous rules. Meaning that rules that refer to other rules being backfilled is not supported. A workaround is to backfill multiple times and create the dependent data first (and move dependent data to the Prometheus server data dir so that it is accessible from the Prometheus API).

in https://prometheus.io/docs/prometheus/latest/storage/

For these reasons, we need another element in our system that will make sure that the longevity of our data is determined by us and not the system. We will achieve this in our example with InfluxDB. Once our data is stored we now need a clean and easy way to visualize data. That is point 5. In many systems, the visualization software also allows us to configure our requirements for point 6. Alert systems usually go hand in hand with the visualization interface we are using. This is where we will see Grafana in action. In terms of point 7, that can be a fallacy in itself. There is always a very small decrease in performance, which should always be negligible as we discussed above. We can also say that if that decrease isn’t perceptible then it’s because it does not exist. It is a fair point, and it makes complete sense, but it’s important to think about it. Misconfigured monitoring solutions can cause latency issues and make the application unresponsive or even bring a system to a shutdown.

4. Domain

Our service is a simulation of a real system. Ideally, we would surveil objects in a museum for example. We could monitor the location of boats passing by a river, check the weather in airports, the affluence of people at a concert, etc. If we have a camera, we can imagine thousands of cases of what we can do with them. In this project, we will make this very simple. We will surveil vegetables on a wooden floor! This way nothing else will matter but the goal of our project:

One service will provide two resource endpoints. One endpoint will give you the image captured at a given time to a certain camera. The other endpoint provides information about the object we are filming. For our application, we are not interested in having this information separate. Instead, we want to access this information in an aggregated way. We also do not want the front end to do all the hard aggregation work. That processing should be left to a new service that will wrap up this “raw data” service and provide the desired aggregated information. Once we have this service available, then we want to be able to visualize our data with a front-end application.
So now we know that we need at least three services. One for raw data, one for the aggregation data, and another to provide a GUI (graphic user interface) to the user. Consequentially, we also need to be able to monitor these services. This means that we’ll need at least three different metrics providers working as a sidecar along these three services.
With our 3 services to 3 metric providers, we need a scrapper to be able to visualize this data. Once we have the scrapper, we also want to be able to visualize this data and we also want to be able to store this data. So this means that we’ll need another 3 services. One scrapper, one visualizer, and a persistence service. Here is a rough sketch of how this works:

Moving Objects project

5. Implementation

Let’s start to make our application! The case we are going to look at is a system that will surveil the movement of objects.
In a previous version of this article, the cameras were provided by two rest interfaces of RapidAPI. One provided an airport list and another the public cameras around the airport. These were the Airport Finder API and the Webcams Travel API. However, because these apps are external, I could not have any control over them. I also wouldn’t be able to control any changes in the model or if the application would for one reason or the other become private, on subscription, trial, or any other form of non-public access. At some point in time, the original example of this article became effectively non-functional, non-compilable, and of course not possible to make a demo of.
This means that now, instead of surveilling locations around airports, we now have an example of surveilling moving objects on a grid of locations. I simply took a few photos of these vegetables to create an idea of movement.
In order to make things a bit more interesting, I’ve created 2 rest applications:

moving-objects-jwt-service — A secured application, JWT capable, providing two separate endpoints. One delivers the object/vegetable information and the other the camera location in a 2-dimensional grid. This application is opened at port 8081. This application has been implemented with Spring Boot and Kotlin Webflow Coroutines. It uses an r2dbc connection against a PostgreSQL database on port 5432.
moving-objects-rest-service — This one will serve the front-end application and is facing the user. We can optionally start it with OAuth2 Okta Authentication capability. It will authenticate, in order to communicate with the JWT service. It runs on port 8082. This application has been implemented with Spring Boot and WebFlux.

I have also created a GUI (Graphic User Interface). In this case, I am using Angular and for the UX (User eXperience) design implementation I am using Angular Materials.
The reason for the creation of the 2 rest services is because I find it to be one great way of showing monitoring in action. We make one service depend on the other, where the goal of the edge service is to aggregate both sources of information. Therefore, we can analyze more different data behavior cases. I have also implemented OAUTH2 for the same reasons. In other words, I wanted one service to have different behavior than the other. We will see a glimpse of that at the end of this article.

6. Application Setup

In this section, we’ll have a look at the application overview, how everything is set up, and the containerized environment.
Let’s start with the application overview:

Application Overview

In this simplified overview, we can see that, before we get any data, we are going through 3 services. We have an angular service running on NGINX, to provide the front-end, one rest server with okta authentication to provide access to an aggregation of data, and finally, a service that provides the raw data using JWT to access it securely.
Let’s have a closer look at what the containerized environment looks like. In our previous requirement enumeration, we saw that we need at least 4 things. We need three application environments, a data resource, a permanent persistence mechanism, and a visualization environment for the monitored data. In our case, these are:

JRE 17 — We’ll use this runtime in two services. As we have seen before, one runs in the Data Source domain and the other in the aggregator domain. In each of the environments, we’ll be running not only the applications but also the metrics service as a side car to the spring-boot applications. Spring boot can pack external libraries like the ones provided by Prometheus and use them in its actuator endpoints.
NGINX — With NGINX, we can deploy our front end as a static deliverable. It runs in the UX domain.
Node — Node JS enables us to run a Prometheus service to provide metrics to the Prometheus scrapper. It runs in the Metrics domain. The runtime does not have to share the same domain as the machine it's running in. With Node, we can start a service, not necessarily to get metrics about NGINX, but the machine where it runs.
Prometheus — This is the scrapper. A scrapper in this case is just a process that runs in fixed timed intervals a scrapping process to retrieve important metrics from target machines. It runs in the Metrics Domain.
InfluxDB — This is our persistence mechanism. InfluxDB also has the power to configure scrappers. They run the same way as Prometheus, but we can store the data. It includes its own metric visualization application. It runs both on the Metrics Domain and in the UX Domain
Grafana — This is our visualization tool. We can visualize all needed metrics that are scrapped by Prometheus. It runs, of course in the UX Domain.

We can make everything run in our local machine separately, but we can also speed up the development process by the use of Docker images and Docker compose. Besides speed, the other big advantage is that we can run the whole environment with one single command in a single docker-machine.

Let’s have a look at a more detailed version of our system:

Localhost configuration using Docker

As seen in the diagram, all our moving parts are containerized in one single machine on localhost. No extra configuration is needed when using Docker. It’s important to notice from the image that the only ports that are supposed to be part of the public domain are 3000, 9090, and 8080. In a local environment, we will expose all ports. Further down this document, we will see how the security version of this project works. For that one, we will also need port 8082. All other ports are inside the docker domain. Every container is aware of each other by the names we have given to them. We’ll see how to translate this system to docker-compose further on in the article.

7. Data model

Our data model is exceptionally simple. We only have two tables. One with the Object Info and the other one with the Moving Object info:

create schema if not exists mos;
drop table if exists mos.moving_object;
drop table if exists mos.info_object;
create table if not exists mos.moving_object(
    id UUID,
    code VARCHAR ( 50 ) NOT NULL,
    folder VARCHAR ( 255 ) UNIQUE NOT NULL,
    uri VARCHAR(255 ) UNIQUE NOT NULL,
    x INT,
    y INT,
    PRIMARY KEY (id)
);
create table if not exists mos.info_object(
    id UUID,
    "name" VARCHAR(50) UNIQUE NOT NULL,
    "size" INT,
    color VARCHAR ( 255 ) NOT NULL,
    code VARCHAR ( 50 ) NOT NULL,
    x INT,
    y INT,
    PRIMARY KEY (id)
);

Both tables share a code. This code is what unifies the data together. However, this data t isn’t given together to the front end. Instead, we provide this data separately. The table moving_object , can be interpreted as the location, where an object with a certain code is being filmed. The uriis the location of the camera that provides the latest snapshot being taken. The parameter folder is the actual location where the image is being saved. This latest snapshot will be delivered to the client using a streaming endpoint. Evidently, x and y are the coordinates of the object. The table info_object simply contains extra information about the object being filmed. An object, for this project, has special information regarding its name, size, color, and location. The location is here also expressed in coordinates x and y. The reason for this is to distinguish the location of the camera and the location of the object. Having said that we can also argue the coordinates of the object we are filming don’t really matter that much.
The start of the project will load the following data in the database:

truncate mos.mos.moving_object;
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'GAR', 'garlic', '/aggregator/webcams/camera/GAR',1,2);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'LAU', 'laurel', '/aggregator/webcams/camera/LAU',2,2);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'LEM', 'lemon', '/aggregator/webcams/camera/LEM',3,4);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'ONI', 'onion', '/aggregator/webcams/camera/ONI',7,2);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'PUM', 'pumpkin', '/aggregator/webcams/camera/PUM',8,1);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'RED', 'red-onion', '/aggregator/webcams/camera/RED',3,6);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'SNA', 'snail', '/aggregator/webcams/camera/SNA',1,7);
insert into mos.mos.moving_object (id,  code, folder, uri,x,y)
values (gen_random_uuid(), 'TOM', 'tomato', '/aggregator/webcams/camera/TOM',0,1);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Garlic', 2, 'white', 'GAR', 1, 2);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Laurel', 4, 'green', 'LAU', 2, 2);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Lemon', 2, 'lemon', 'LEM', 3, 4);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Onion', 4, 'gold', 'ONI', 7, 2);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Pumpkin', 10, 'orange', 'PUM', 8, 1);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Red Onion', 5, 'purple', 'RED', 3, 6);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Snail', 1, 'brown', 'SNA', 1, 7);
insert into mos.mos.info_object(id, "name", size, color, code,x,y)
values (gen_random_uuid(), 'Tomato', 2, 'red','TOM', 0, 1);

8. Code

If you already have had a sneak peek at the code on GitHub, you’ve probably already seen that the code base is quite small per module, but the conjunction of all the code may be a bit complicated to understand. This section is here so that we can see together how the code is built and what we want to achieve. Maybe the best thing to do, to start off is to have a look at the data domain in the moving-objects-jwt-service:

Dao UML for moving-objects-jwt-service

There isn’t really too much special in this UML chart. Essentially we just have two different types and we call them MovingObject and InfoObject. Cameras have got a location in an x, y grid in the same way that Objects do. We consider the MovingObject to be what the Camera is filming. We can and will use these terms interchangeably to refer to the same thing. InfoObject is just some information about the object. In this case, it’s just the size, the color of the object, and its latest location. The latest location never changes in this project. Although the JWT service is mostly implemented using Coroutine data Flows, I wanted to use WebFlux in the same project to show that both can work together in the same way. Although their reactive implementations are different, they can work together, and we can make use of the different features they both can offer. In my case, I really wanted to make use of the collectSortedList function to create a reactive Mono from a list of objects of the type MovingObject. Many people prefer to use the JpaSpecificationExecutor to get a full page in one go. Unfortunately, if we do that, then we get a non-reactive Page in return. It probably doesn’t matter much in the end because effectively a Page is by all means just one record given back to the client. However, I wanted this application service to be completely reactive regardless. And so instead of returning a Page, I get to return a Mono<Page> which keeps the usage of the application possibly to be completely reactive.
So now that we’ve looked at the Model, let’s now have look at the controller part:

REST UML for moving-objects-jwt-service

We convert the data into the DTOs in very specific ways but never join the object info with the cameras. In our case, we’ll associate a location with all cameras contained in a specific radius of it. This will give us the full list of all the cameras for a specific search, but not the object's info in relation to their sizes and color. This is for one endpoint. For the other endpoint, we can get the object's info. With this one, we’ll know the object's location, color, and size. What this rest service does not offer is an aggregation of the cameras and object info.

This service also needs to be accessed via security. For this project, we make it very simple and when we run it, we’ll get a warning that we are using one user/password combination configured in the clear in application.properties. Normally, security services have a much more complicated way of providing security. Let’s have a quick look at the code:

@Bean
fun securityFilterChain(http: ServerHttpSecurity): SecurityWebFilterChain =
    http
        .logout { logout ->
            logout
                .requiresLogout(ServerWebExchangeMatchers.pathMatchers(HttpMethod.GET, "/logoff"))
        }
        .authorizeExchange { authorize ->
            authorize
                .pathMatchers("/webjars/**")
                .permitAll()
                .pathMatchers("/info/jwt/open/**")
                .permitAll()
                .pathMatchers("/webcams/jwt/open/**")
                .permitAll()
                .pathMatchers("/v3/**")
                .permitAll()
                .pathMatchers("/actuator/**")
                .permitAll()
                .anyExchange()
                .authenticated()
        }
        .csrf().disable()
        .httpBasic(Customizer.withDefaults())
        .oauth2ResourceServer { oAuth2ResourceServerSpec -> oAuth2ResourceServerSpec.jwt() }
        .build()

What we are doing here is first allowing all matching patterns to be open publicly. These patterns refer to 3 important aspects of the application: Swagger, Actuator, and test endpoints. Swagger allows us to interact with the application via a GUI that allows us to perform requests without needing to use curl , or any other command line instructions like wget. The Actuator endpoint is vital for our project because it provides metrics including the ones Prometheus understands and uses. CSRF must be disabled in our application. The Spring documentation is very clear about why is this true for most cases:

If you are only creating a service that is used by non-browser clients, you will likely want to disable CSRF protection.

in https://docs.spring.io/spring-security/site/docs/5.0.x/reference/html/csrf.html

The authentication for our token will be asked via a specific endpoint that we’ll see later on. For now, I just want you to get the idea that with our configuration, we can access any of our endpoints either with a username/password combination or with a JWT token. The JWT token, however, will only be given via the token endpoint. We protect our application though, with the OAuthResourceServerSpec. This allows us to configure several types of security settings, but for the purpose of this article, we just create the standard JWT support.
Reactive programming also implies reactive security because security is something inherently bound to any request we make to our services. The principle though is exactly the same as in non-reactive applications. We first configure our AuthenticationManager:

@Bean
fun authenticationManager(): ReactiveAuthenticationManager {
    return UserDetailsRepositoryReactiveAuthenticationManager(userDetailsService())
}

We then need to configure encoders and decoders for the JWT token itself and also for the password we are using to get the initial JWT request for the application:

@Bean
fun jwtDecoder(): ReactiveJwtDecoder = NimbusReactiveJwtDecoder.withPublicKey(rsaPublicKey).build()

@Bean
fun jwtEncoder(): JwtEncoder =
    NimbusJwtEncoder(ImmutableJWKSet(JWKSet(RSAKey.Builder(rsaPublicKey).privateKey(rsaPrivateKey).build())))

@Bean
fun passwordEncoder(): PasswordEncoder {
    return BCryptPasswordEncoder()
}

Codecs for the JWT rest service

Once we have the codes, we can now configure our UserDetailsService:

@Bean
@SuppressWarnings("deprecated")
fun userDetailsService(): MapReactiveUserDetailsService? {
    val user = withDefaultPasswordEncoder()
        .username(username)
        .password(password)
        .roles(*roles)
        .build()
    return MapReactiveUserDetailsService(user)
}

UserDetailsService

We can already see that this UserDetailsService is something different from its Servlet counterpart (we also all this in street language MVC). The difference is that this service is implemented in a reactive way. We are using Netty here and not Tomcat as a SpringBoot embedded Service. This means that Netty, works in a reactive way, which means that we can implement our calls to the service in a reactive way. This in turn improves the capacity of our machine, given that it will be able to hold more requests at the same time, even when security is involved. In our case, our service is safeguarded with username/password admin/admin. It is always good to be reminded that at this point security can be much better implemented. Our very unsafe admin/admin combination suffices though for the purpose of this exercise.
Let’s now have a look at our service that will face the front-end. In the fully functional version of this service, we’ll use Okta to perform OAuth authentication. For the end-to-end tests, Okta will be disabled, but we will go through the whole application. Our application is called moving-objects-rest-service . The first contact with the previous application starts obviously at the data layer level:

DAO Layer of the Okta project

Looking closely at these types, we can see how directly they relate to the Dto’s created in the JWT project.

Service Layer of the Okta Project

In the service layer above, we see that we have dedicated services for each endpoint and next to it, we use an ObjectAggregatorService. This service will create the payload that we are interested in. The aggregation follows a few interesting implementations related to reactive programming. Let’s have a look at a few methods:

public Flux<MovingObjectDto> getObjectsAndCamsBySearchTermAndRadius(String term, Long radius) {
    return objectsService.getObjectsByTerm(term)
            .flatMap(objectDto -> {
                CoordinatesDto coordinates = objectDto.coordinates();
                return webCamService.getCamsByLocationAndRadius(coordinates.x(), coordinates.y(), radius)
                        .collectList().map(webCamDto -> Pair.of(objectDto, webCamDto));
            })
            .map(webCamDtoPackage -> {
                final MovingObjectDto movingObjectDto = webCamDtoPackage.getFirst();
                webCamDtoPackage.getSecond().forEach(webCam -> movingObjectDto.webCams().add(webCam));
                return movingObjectDto;
            }).distinct();

}

Creating the MovingObjectDto

In this case, we are using purely Spring WebFlux. We first search for the object information by a search term. In the second step, we search for locations related to it by radius. So, in other words, we are going to retrieve all cameras contained within a specific radius pertaining to an object we have searched by a search word. We must never forget that when using Coroutine Flows or WebFlux, we are not doing imperative programming, and we are not doing OOP (Object Oriented Programming). We are using Functional Programming, implementing the Observable Pattern, and using Reactive Principles. What this means is that the code will not execute immediately instead, it will be executed when Netty is free to do so. It is probably good at this point to remind ourselves that by only using the Spring WebFlux libraries in our dependencies, we are no longer using Tomcat by default. We are using Netty at this point. The reason I want to make a statement here is that we see very often people using Tomcat in an interchangeable manner with Spring Boot. However, SpringBoot isn’t necessarily run by Tomcat. It can be run with Undertow and Jetty, and in the case of reactive services, it can then be run with Netty. It is important to make this distinction in advanced types of service implementation.
We can now have a quick look at what the Controller layer looks like:

Controller Layer of the Okta Service

This layer is what allows us to work with our payload which has the following form:

Payload example to serve the front-end application

As we can see, we are providing details about our object, but also the details about the camera. We have thus, aggregated two endpoints into one in a reactive way that will be managed by Netty.
Finally, we can have a look at how the optional security is implemented with okta. The only thing we need to do is to first add the okta libraries and a few extra helpful libraries:

implementation("com.okta.spring:okta-spring-boot-starter:3.0.7")
implementation("com.fasterxml.jackson.module:jackson-module-kotlin:2.18.2")
implementation("me.paulschwarz:spring-dotenv:4.0.0")
implementation("org.springframework.security:spring-security-web:6.4.2")
implementation("org.jesperancinha.objects:moving-objects-security-dsl:1.0.0")

Dependencies used to provide security

The actual relevant dependency is of course the okta-spring-boot-starter library. The others are important because they provide support. They are not per se necessary. They are, however necessary for the current version of the project and its library versions. The last library, moving-objects-security-dsl, is a very small library I made in order to only allow Okta security configurations to occur when the library is present. Trying to disable okta via spring profiles, was revealed to be an odd thing to do, and it just wasn’t helpful for this project to make something like that. So the library just contains one single class:

@Configuration
class SecurityConfiguration(
    @Value("\${objects.jwt.public.key}")
    var rsaPublicKey: RSAPublicKey,

    @Value("\${objects.jwt.private.key}")
    var rsaPrivateKey: RSAPrivateKey,

    @Value("\${objects.jwt.username}")
    var username: String,

    @Value("\${objects.jwt.password}")
    var password: String,

    @Value("\${objects.jwt.roles}")
    var roles: Array<String>,
) {

    @Bean
    fun securityFilterChain(http: ServerHttpSecurity): SecurityWebFilterChain =
        http
            .logout { logout ->
                logout
                    .requiresLogout(ServerWebExchangeMatchers.pathMatchers(HttpMethod.GET, "/logoff"))
            }
            .authorizeExchange { authorize ->
                authorize
                    .pathMatchers("/webjars/**")
                    .permitAll()
                    .pathMatchers("/info/jwt/open/**")
                    .permitAll()
                    .pathMatchers("/webcams/jwt/open/**")
                    .permitAll()
                    .pathMatchers("/v3/**")
                    .permitAll()
                    .pathMatchers("/actuator/**")
                    .permitAll()
                    .anyExchange()
                    .authenticated()
            }
            .csrf().disable()
            .httpBasic(Customizer.withDefaults())
            .oauth2ResourceServer { oAuth2ResourceServerSpec -> oAuth2ResourceServerSpec.jwt() }
            .build()

    @Bean
    fun authenticationManager(): ReactiveAuthenticationManager {
        return UserDetailsRepositoryReactiveAuthenticationManager(userDetailsService())
    }

    @Bean
    fun jwtDecoder(): ReactiveJwtDecoder = NimbusReactiveJwtDecoder.withPublicKey(rsaPublicKey).build()

    @Bean
    fun jwtEncoder(): JwtEncoder =
        NimbusJwtEncoder(ImmutableJWKSet(JWKSet(RSAKey.Builder(rsaPublicKey).privateKey(rsaPrivateKey).build())))

    @Bean
    fun passwordEncoder(): PasswordEncoder {
        return BCryptPasswordEncoder()
    }

    @Bean
    @SuppressWarnings("deprecated")
    fun userDetailsService(): MapReactiveUserDetailsService? {
        val user = withDefaultPasswordEncoder()
            .username(username)
            .password(password)
            .roles(*roles)
            .build()
        return MapReactiveUserDetailsService(user)
    }
}

Secure configuration

What could have also been made is one Bean with profile prod and another bean with profile test and then just make the test bean a completely open website, with something like pathMatchers(“/**”).permitAll(). However, doing so, would still imply okta initialization any way, and a level of unpredictability that I found to be unreasonable for the purposes of this project, and since Gradle is amazing in configuring assemblies the way we want I created thus a parameter configurable build using:

if (project.hasProperty("prod")) {
    implementation("com.okta.spring:okta-spring-boot-starter:3.0.7")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin:2.18.2")
    implementation("me.paulschwarz:spring-dotenv:4.0.0")
    implementation("org.springframework.security:spring-security-web:6.4.2")
    implementation("org.jesperancinha.objects:moving-objects-security-dsl:1.0.0")
}

Configurable build with Gradle

The configuration in SpringBoot is as simple as this:

# Okta
okta.oauth2.issuer=${ISSUER}
okta.oauth2.client-id=${CLIENT_ID}
okta.oauth2.client-secret=${CLIENT_SECRET}
#okta.oauth2.redirect-uri=http://localhost:8082/aggregator/login/oauth2/code/okta
okta.oauth2.post-logout-redirect-uri=http://localhost:8082/aggregator/signout
.env.filename=.okta.env

Okta configuration

With this and if we want to run the secure version of the application we can create our build with the following command line:

gradle -Pprod clean build

Finally, module moving-objects-gui, provides a GUI to us where we can visualize our applications. We won’t go into details about it, but it is important to understand that we will be deploying it in NGINX, and we will also want to monitor that machine.

9. Monitoring Endpoints

In previous steps, we have looked into how the architecture is designed, and we have looked at our domain. We have also had a look at the code and took a dive into some interesting aspects of the implementation. We should have a very clear view of what the system is doing at the moment and how all the moving parts are working. We will now go step-by-step through all the elements of the monitoring system. In this part of the article, it’s important to bear in mind that this system can be run on a local machine. All moving parts are part of a docker-compose setup.
In this article, I wanted to make an example that would be easy to understand and modify and this is why every single component has its own custom Docker image.
Also, let’s keep in mind that the examples I’m showing happen after we have successfully run docker-compose. Further in this article, we’ll go through the details. For now, let’s keep in mind that we can run two versions of our system. For the secure version, we can run:

make dcup-full-action-secure

or if we want to run it without okta, we can then run:

make dcup-full-action

For now, let’s have a look at how the metrics can be configured in the different containers. For our example, we have 3 containers with important metric configuration needs. They are the moving-objects-jwt-service, the moving-objects-rest-service and the moving-objects-gui . For the first two services, we need to configure Prometheus for Spring Boot:

implementation("org.springframework.boot:spring-boot-starter-actuator")
implementation("io.micrometer:micrometer-core:1.14.2")
implementation("io.micrometer:micrometer-registry-prometheus:1.14.2")

Micrometer dependencies

This is the micrometer dependency that is needed to have all metrics written and exposed in the Prometheus format. In order to activate it, we also need to activate the Spring Boot’s actuator. This will still not open the endpoints though. These are actually just the libraries that will allow the endpoints to be generated.
To open our endpoints in our project we need to explicitly configure Spring Boot for that:

management.endpoints.web.exposure.include=*
management.endpoint.shutdown.enabled=true
management.endpoint.metrics.enabled=true
management.endpoint.prometheus.enabled=true
management.endpoint.httptrace.enabled=true
management.metrics.export.prometheus.enabled=true
management.trace.http.enabled=true

Enabling Metrics including Prometheus

Both rest services contain the same configuration. The actuator is needed because it provides important metrics that come out of the box in Spring Boot and can already be indirectly read by Prometheus. The micrometer properties from the io.micrometer core and theprometheus libraries provide metrics using the same format as Prometheus. This allows for a much easier data reading in the Prometheus processes from the metrics endpoints.
With these simple configuration lines, we are enabling endpoints to be created, and we are also activating other tracking mechanisms. All of these management points are part of the Spring Boot Actuator. However, a Prometheus actuator endpoint is only autoconfigured in the presence of the Spring Boot Actuator. Hence, the reason for needing the micrometer libraries and the actuator libraries simultaneously.
Prometheus has its own scrappers. They will run processes against Spring Boot applications and scrape all of those metrics periodically. We can also make our own metrics and make them available through the actuator. There are many different ways of doing this and one example is to play a bit with trace. In our metrics we can find this endpoint:

http://localhost:8080/objects/actuator/metrics/http.server.requests

Looking at the result we find:

Unchanged payload to trace requests

Here we have COUNT, TOTAL_TIME, and MAX. These are in their essence, the number of requests made to the application, the total time they took to respond, and the maximum time it took to get a response from a request. These are all important metrics. With these metrics, we can have an idea of how many people are looking into our web application. For ratings, it’s a very handy metric. We want to monitor the performance of our application. Using total time can be an option. With Prometheus, we have much better options than this, but since we have count and total time, we can actually get the average response time for a certain endpoint. As shown, we can also read the maximum time an endpoint took to respond. This can be important when noticing that if one endpoint exceeds a certain time to respond then that endpoint may have a latency issue. Therefore some sort of action must be made in terms of improving it.
What we don’t have in these metrics, is the minimum amount of time it took for an endpoint to reply. We just don’t have data for that. At no point, we have registered the time of each request. The alternative to that is to implement our own HttpTraceRepository:

@Repository
public class CustomTraceRepository implements HttpExchangeRepository {
    private final AtomicReference<Queue<HttpExchange>> lastTrace = new AtomicReference<>(new LinkedList<>());

    @Override
    public List<HttpExchange> findAll() {
        try {
            return new ArrayList<>(lastTrace.get());
        } catch (Exception e) {
            return List.of();
        }
    }

    @Override
    public void add(HttpExchange trace) {
        val httpTraces = lastTrace.get();
        if (httpTraces.size() > 10) {
            while (httpTraces.size() > 10) {
                httpTraces.poll();
            }
        }
        if ("GET".equals(trace.getRequest().getMethod())) {
            httpTraces.add(trace);
        }
    }
}

CustomTraceRepository

This creates an implementation of the HTTP tracing, This has been enabled by the HTTP tracing flag as we saw before: management.trace.http.enabled=true. We can notice already that this implementation also carries a repository. Simply speaking, I just created a FIFO (First In/ First Out) queue. In other words, I’m just keeping 10 elements in the repository at the time. This code is only responsible for the recording of GET requests. We can change this to perform other or the same operation to other requests.
The focus here is just the fact that we can change metrics the way we please and reprogram some of them to report customized data directly into Prometheus or any other monitoring fetching tool. Traces will carry the following format as an example:

Renewed Trace Metric Request

If we look at the last element of this request, we find timeTaken. This is the response time of each GET request. Our MIN could in this case be an implementation of the minimum timeTaken in this array. It’s important to notice that in this case, we are measuring milliseconds. Why is the minimum time needed? Perhaps our intention is to find the peaks in our performance instead of the peaks in memory usage or CPU. Perhaps if we know when our requests were faster, we can then also identify where we potentially had a better code or system as a whole. This endpoint is accessible in this project on endpoint http://localhost:8080/aggregator/actuator/httptrace.
We just went through the essentials for the Spring Boot back-end support for Prometheus.
Let’s have a look at how NodeJS can provide the same endpoint for Prometheus. In our example, we have a fully developed application in Angular. We could have used NGINX to deploy it directly. However, in that case, NGINX itself would have had to have been configured in order to provide the endpoints to Prometheus. However, NodeJS provides a very interesting solution by means of the libraries: prometheus-api-metrics and prom-client.
There is a server.ts example in the code, but our focus will only be at this point on two crucial lines of that code:

import "es6-shim";
import moment from "moment";
import "reflect-metadata";

import apiMetrics from "prometheus-api-metrics";

import express from "express";

class Server {

    public static bootstrap(): Server {
        return new Server();
    }

    public app: any;
    private port = 4000;

    constructor() {
        this.app = express();
        this.app.use(apiMetrics());
        // tslint:disable-next-line:no-console
        this.app.listen(this.port, () => console.log(`Express http is started ${this.port}`));
        this.app.on("error", (error: any) => {
            // tslint:disable-next-line:no-console
            console.error(moment().format(), "An error has ocurred!", error);
        });
        process.on("uncaughtException", (error: any) => {
            // tslint:disable-next-line:no-console
            console.log(moment().format(), error);
        });
    }
}

const server = Server.bootstrap();
export default server.app;

server.ts metrics supplier

The lines we are interested in, or should I just say the one line we care about is: this.app.use(apiMetrics());
The whole server.ts is an implementation of a simple NodeJS service that runs on express.
Our endpoints for Prometheus metrics can be seen in our running example on endpoints:

http://localhost:8080/objects/actuator/prometheus — JWT Service
http://localhost:8080/aggregator/actuator/prometheus — Okta Service
http://localhost:8080/metrics — NGINX Service

The metric format will be something like this:

Example metrics exposed by Prometheus enabled

11. Setting up Prometheus

To get Prometheus running, we finally need to configure all of what we have discussed into a prometheus.yml file. Our file looks like this:

global:
  scrape_interval:     15s
  evaluation_interval: 30s

scrape_configs:

  - job_name: aggregator-reactive-rest
    honor_timestamps: true
    metrics_path: /aggregator/actuator/prometheus
    static_configs:
      - targets:
          - moving-objects-rest-service:8082

  - job_name: objects-jwt-reactive-rest
    honor_timestamps: true
    metrics_path: /objects/actuator/prometheus
    static_configs:
      - targets:
          - moving-objects-jwt-service:8081

  - job_name: moving-objects-gui
    honor_timestamps: true
    metrics_path: /metrics
    static_configs:
      - targets:
          - nginx:4000

Full Prometheus configuration file

Notice that at the end, the configuration is pointing to our InfluxDB. We would be setting our scrapping interval to 15 seconds, and we would evaluate each scrape for the past 30 seconds. However, those two lines are no longer usable. InfluxDB has grown into its own scrapper framework which opens the debate of Prometheus is still a necessary thing. Here, however, we are just looking at ways to configure our monitoring application. What is important though, is that we see that we can configure our routes in a YAML fashion with property static_configs. In all three, we configure three different jobs that will scrape the data we need. Scrapping in this case means that the data will be accessible to any external visualizing tool like the Grafana framework we’ll see further on.

Prometheus Dashboard

12. Setting up `InfluxDB`

One of the greatest hurdles with working with any architecture is the persistence layer. Prometheus does not provide an easy-to-configure durable and persistent storage system. This is something that is expected from Prometheus, given that Prometheus is mostly used as a scrapper, and as we have seen before it is not easy to get it from being an ephemeral data storage system to a persistent data storage system. This is why we have such databases as InfluxDB. This is a storage system specifically designed to store metrics. Current days, it provides much more features, and it has its own data scrappers that work pretty much in the same way as Prometheus does. Let’s have a look at how we can connect to it.
InfluxDB does not need to be connected to Prometheus to store metrics anymore. In the old days, we would configure Prometheus to push the data to InfluxDB by means of Jobs. In current days we just need metric endpoints. Then InfluxDB can scrape those metrics and can even provide visualization graphics and dashboards, much in the way Grafana does. We can have it running and just use it within our system. To check that it’s actually up and running we can just run the following command:

influx ping

For this, we need to have the influx client installed.
When starting the demo locally, Cypress will run through the current InfluxDB installation, and it will create buckets and scrapers. These will be configured to store data in different buckets for different endpoints. InfluxDB comes itself with Prometheus metrics endpoints. Configuring InfluxDB from the GUI is very easy and I don’t think it is relevant to show how these buckets and scrappers get configured. However, it is important to visualize what we get after the run is successfully completed. This is what we get after configuring the buckets. When we run Cypress the first thing it creates is the buckets for our scrappers to save the data:

When the buckets are created, we can now create the scrappers:

Scrappers in InfluxDB

And finally we can create an Application Token:

Application Token in Influx DB

This application token is needed for something very interesting in InfluxDB, which made it into an even more independent platform. As we see from above, InfluxDB allows to actively scrape metrics from our applications.This is all good, but we may want to this the other way around. If we want to passively receive data in InfluxDB, we then need an external tool to do that. Prometheus has that functionality still, but, because of the security requirements, it is quite cumbersome to achieve it. InfluxDB, however, allows for external engines to connect with it quite easily including the ones provided by Prometheus. They all this concept Telegraf. A Telegraf is essentially a tool that loads these engines and starts actively collecting data from a Prometheus payload and pushes it to InfluxDB. We do not need to download or install anything other than Telegraf to do this and to configure it, we can edit the telegraf.conf file. For our project, it looks like this:

## Collect Prometheus formatted metrics
[[inputs.prometheus]]
  urls = [
  "http://localhost:8080/objects/actuator/prometheus",
  "http://localhost:8080/aggregator/actuator/prometheus",
  "http://localhost:8080/metrics"
  ]
  metric_version = 2

## Write Prometheus formatted metrics to InfluxDB
[[outputs.influxdb_v2]]
  urls = ["http://localhost:8086"]
  token = "$INFLUX_TOKEN"
  organization = "Moving Objects"
  bucket = "mos"

InfluxDB

The important thing for now is just to focus a bit on the self-explanatory parameters of this file. Essentially we need urls to let telegraf know where to scrape our metrics from, and we need to specify the destination of those metrics. For the destination we need to provide the location of InfluxDB with urls, the token we’ve generated before, the organization and the bucket. It is important to understand that the unique identifiers of our location in InfluxDB are organization and bucket. These are mandatory. At the top of these parameters we specify the type of the metrics source endpoints with inputs.prometheus and the destination type with outputs.influxdb_v2.
When running the demo, the build will start, then cypress will run and create the token at /docker-files/telegraf, and it will finally start the telegraf container in the same network of docker-compose. In other words, this will all be done automatically done for us so that we can focus on how the setup is made.
If we get the demo successfully, with script make start-demo running, we’ll then be able to create graphs like this one:

Dashboard visualization for InfluxDB

If you notice that data isn’t coming in, it could be that telegraf hasn’t started yet. We can start telegraf again by running:

make start-telegraf-container

13. Setting up Grafana

Grafana has a different way of configuring than Prometheus. Here we don’t need any code. Grafana is based purely on the contract it has with Prometheus. Before continue, let’s recap what we have so far:

2 Spring Boot applications and 1 NodeJS application
Persistent Metrics Database InfluxDB
Metrics Scrapper and management with Prometheus

We will use Prometheus with Grafana to generate graphs from Prometheus only. A detailed discussion on how graphics are built with Grafana is a topic on its own and removes the focus of this article. What’s important to understand is how can we make our Grafana configuration also persistent. We want to keep our graphic configuration persistent so that we can use them in any Grafana environment.
When we look at Grafana for the first time, we become immediately aware that we have to create a data source. With this action, we also create a Dashboard provider. Dashboard providers are kept in YAML files located in /etc/grafana/provisioning/dashboards/ in Grafana. Let’s look at our example on dashboard.yml:

apiVersion: 1

providers:
  - name: 'Prometheus'
    orgId: 1
    folder: ''
    type: file
    disableDeletion: false
    editable: true
    options:
      path: /etc/grafana/provisioning/dashboards

Grafana Dashboard configuration

With this configuration, we are telling Grafana that Prometheus is the name of its first Dashboard provider. At the same time, we are letting Grafana know that we will load all dashboards located in /etc/grafana/provisioning/dashboards. Although we have already established a data provider for our dashboards, we still need the actual endpoint for Prometheus. This is done in our datasource.yml file:

apiVersion: 1

deleteDatasources:
  - name: Prometheus
    orgId: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    orgId: 1
    url: http://prometheus:9090
    basicAuth: false
    version: 1

datasource.yml

After defining the data source and the dashboard data provider we finally need to place all our dashboards in /etc/grafana/provisioning/dashboards. Grafana provides ways to save these dashboards in a JSON format. They do not need to be referenced by Grafana. As long as they reside in that folder, we will see that Grafana will read all of them.
The Dashboard themselves are quite big files and they are available in the project in the folder /docker-files/grafana. The two JSON files there are the setup files for the NGINX machine (node-grafana.json) and for the JVM’s (jvm-grafana.json).
Grafana seems to have some compatibility issues with Firefox. I do not consider them to be critical, and they are something that I was never able to reproduce in terms of UX (User eXperience). However, they do exist and the cypress tests were able to detect a lot of them, and this is why, if we take a look at commands.ts, we’ll see a few exception rules for certain thrown errors:

Cypress.on('uncaught:exception', (err, runnable) => {
    if (err.message && err.message.trim().length > 0 && err.name && err.name.trim().length > 0) {
        if (err.message.indexOf('setting getter-only property "data"') >= 0) {
            return false;
        }
        if (err.message.indexOf('Cannot read properties of null') >= 0) {
            return false;
        }
        if (err.message.indexOf('too much recursion') >= 0) {
            return false;
        }
        if (err.message.indexOf('The operation was aborted') >= 0) {
            return false;
        }
        if (err.message.indexOf('undefined') >= 0) {
            return false;
        }
    } else {
        return false;
    }
    return true;
})

After running Grafana, we should be getting graphs like these:

Dasboards examples for Grafana

14. Docker composer orchestration

The docker-compose file located at the root of the project contains the main setup to start the application. This is how we implement our docker-compose orchestration as seen in our first diagram.
I mentioned before a few ways to start this application locally. We can try and run it all in one go. The script available to do this is the start demo:

make start-demo

If this is successful, we will start the insecure version of our application. we want to start the secure application we can then run:

make start-demo-secure

This is the look of our page at port 8080:

Moving Objects application

15. Conclusion

We have finally reached the end of this article. It is quite extended, but I hope to have been able to transmit the essential points I found to be of inspiration with this sort of Monitoring architecture. I didn’t want it to become too complicated but I also wanted to make it a fun article with a few surprises.
In this article we have seen how to set up a very basic configuration of Prometheus, Grafana and InfluxDB applied to Spring Boot and NodeJS applications.
We have seen images as examples of how the Dashboards look and feel.
Looking back to all three types of dashboards and the three different brands, we can immediately get the idea that these are essentially just different monitoring tools and they can all be based on simple Prometheus metrics endpoints. We can also see that we can make our own metrics and that those can be programmed in the code. It can be overwhelming to make a decision when using these mechanisms. For example, Grafana, seems to be a great tool to visualize data and although we have seen in this article, how to integrate it with the Prometheus service, we can also integrate it with InfluxDB. But now maybe we get a bit taken aback because InfluxDB itself has Dashboards, so why do we need Grafana anyway? Maybe there are functionalities in Grafana that you cannot find in InfluxDB. Or perhaps, in the same way, Telegraf, can lower the CPU usage of InfluxDB, and so can Grafana. In fact, Grafana can be considered just a visualization framework, Prometheus the source of all metrics, InfluxDB the persistence database, and Telegraf, the scrapper we need. If we do it this way, then we get a completely decoupled system where each part has its own responsibility. Therefore, we minimize performance issues and the influence of metrics on our application.
So, in regard to security, using JWT and Okta as authentication/authorization mechanisms, we can see that in order to be able to monitor we can make the endpoints open. But should we? We probably should not leave them open, but they frequently are open. What usually happens is that they remain open for the in-premise network, but to the outside world, they are kept private. So this means that not even someone with full access is able to access these endpoints. The techniques to do this are beyond the scope of this article, but solutions such as NGINX, Kong, and other gateways are available to provide functionality, and they usually offer integrations with Cloud providers.
I have placed all the source code of this application on GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it.
I’d love to hear your thoughts on it, so please leave your comments below.
Thanks in advance for your help and thank you for reading!
Take care, stay interested, stay logical, and stay safe!

16. References

Learning JWT security using KumuluzEE — The finances of a league of the environment

João Esperancinha — Fri, 27 Dec 2024 10:32:04 +0000

1. Introduction

Nowadays, we have more and more concerns about performance, and at the same time, we want to know how systems can communicate fast and reliably. Many times we want to send information and keep it confidential and safe as much as possible. Sensitive data sometimes must also move through the web publicly and trigger actions at the other end of the wire. Mostly, we want to generate actions that will cause data mutations. In these cases, we are not only looking at protecting our data. We want to make sure that the actions triggered by sending our data are trusted. We can protect our data in several ways. Most commonly, we send the data via a TLS (Transport Layer Security) secure connection. That will ensure that our data gets Encrypted through the wire. We use Certificates to create trusting relationships between two parties and achieve this.
In this article, I want to discuss the JWT standard and further see how can we integrate JWT into a common Enterprise application. In this case, we will have a look at KumuluzEE.
Let’s have a look at some basic concepts. JWT or JSON Web Token, or better yet, JavaScript Object Notation Web Token, is a standard defined in RFC7519. This standard has been, like all RFC (Request For Comments) standards, defined, written, and published by the IETF (Internet Engineering Task Force). It can be defined in multiple ways. Generally, we can say that JWT is a compact, safe form of transmitting claims between two parties. One way of simplifying what a claim is, is basically to describe it as a name/value pair that contains information. We need this information to guarantee a few important aspects of our internet communication. We need to make sure that the information we receive is validated and trusted in the first instance. Then we need to validate it. This is basically it.
In order to implement this standard, we can use several frameworks that can help us implement a Java enterprise application. Spring Boot is being used widely. Many times it’s also being wrapped under another name in propriety software from certain organizations like banks and other financial organizations. For our example, I decided to do something different. Instead of Spring Boot, we are going to have a look at an example with KumuluzEE. The point is to identify exactly what JWT is and what it looks like. Java Enterprise Applications are basically applications that can be deployed in an application server or just run on their own via the use of an embedded server. As an example, Spring Boot applications run on an embedded Tomcat server. In this article, our focus will be set on KumuluzEE. Just like Spring Boot it also contains an embedded server. Except that in this case it is called Jetty. This is used in combination with Weld in order to provide CDI(Context Dependency Injection). All Java EE and Jakarta EE technology standards are compatible with this framework.

2. Case Example

In order to exemplify how JWT works in its basic form, I had to think of a way to present it. Classic examples where security is a concern are banks. However, making a whole bank application to show how JWT works would be a waste of time and maybe too many concepts would be involved. Instead, what I made is a very simple banking system. Our main concern is to show how data flows through the wire and how users get access to certain areas of our application. I’m also not going to discuss TLS or how we can send encrypted information through the wire. We will keep our focus on JWT in its purest form.
Our case is a banking system used by a group defending nature and the environment. This is just a fun way to show how JWT works. The main character of this League of Nature is Lucy, who is becoming a common character in all my articles.

3. Architecture

Before we start, let’s just sketch our running application. It’s a very simple application, but it’s still a good thing to draw it:

The reason why this is so simple is that since JWT gets checked on every request and every request gets verified against the public key, then we know that as long as we send the correct token on every request we will be able to get through. JWT can be integrated with OAuth2, Okta SSO, or any other authorization mechanism. In this case, what we are doing is establishing authentication and authorization. In our application, we are going to use JWT and with it, authenticate our message using a signature. We won’t log into the application though. Instead, we will authorize users to use our application after successful authentication. At this point, it’s easy to see that JWT at its core is actually a very small part of a full application. Nonetheless, some functionality needs to be added. These are the Resources we need:

Balance system
Credit system

Let’s just say that our basic system will only register money and credit requests. Essentially it will just accumulate values. Let’s also assume that some people will be able to get credit and others won’t. Some people will be able to store money and other people will be able to get credit.

4. Choosing technologies

As mentioned in the introduction, we will use KumuluzEE as our enterprise application framework, and we will implement an ultra-basic application in a way that we can look at basic JWT terminology and concepts.
Make sure to have the correct Java version. At this stage, we will need minimal Java 17 SDK installed. We will need maven, git, a Java-compatible IDE like IntelliJ, and a shell of some sort.

5. Setup

In order to begin our application, we have a few KumuluzEE dependencies. This is mainly because KumuluzEE, just like Spring Boot needs a couple of dependencies. Let’s have a look at the POM file briefly:

<dependencies>
    <dependency>
        <groupId>com.kumuluz.ee.openapi</groupId>
        <artifactId>kumuluzee-openapi-mp</artifactId>
    </dependency>
    <dependency>
        <groupId>com.kumuluz.ee.openapi</groupId>
        <artifactId>kumuluzee-openapi-mp-ui</artifactId>
    </dependency>
    <dependency>
        <groupId>com.kumuluz.ee</groupId>
        <artifactId>kumuluzee-microProfile-3.3</artifactId>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-core</artifactId>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
    </dependency>
    <dependency>
        <groupId>org.jetbrains.kotlin</groupId>
        <artifactId>kotlin-stdlib</artifactId>
    </dependency>

    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>io.mockk</groupId>
        <artifactId>mockk-jvm</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.ninja-squad</groupId>
        <artifactId>springmockk</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>io.kotest</groupId>
        <artifactId>kotest-assertions-core-jvm</artifactId>
        <scope>test</scope>
    </dependency>
    </dependencies>

Let’s discuss a few dependencies briefly. As you read this, please follow our pom.xml file from top to bottom. This is important in order to understand the following explanation.
We need a package of dependencies to make our application work. , Fortunately, KumuluzEE, provides us with Microprofile libraries which contain basic standard bundles to start this application. This is all contained in the KumuluzEE-Microprofile library. In order to be able to configure our app with all the JWT parameters we need, we have to add a MicroProfile library to it. At the same time, we need a JSON processing library. This will be what Johnson Core does. We need of course the core of KumuluzEE to work. Jetty is the underlying server that runs the KumuluzEE framework. This is why we need it in our dependencies. Considering we need CDI, we also need a library that supports it. In order to enable our REST endpoints, we need the rest library of KumuluzEE. In order to get our API, we then need a Geronimo library. This will ensure that we have an implementation of JSR-374 available. We also need to interpret our JWT and its JSON-formatted contents.
Lombok is not really needed per se. It just makes everything beautiful and shiny! Logback is also important to have so that we can better interpret logs and understand our results.
Let’s now take a look at our resources folder.
To start off let’s first understand what we expect to find in this folder. We need to configure our application with something related to JWT, Logback and finally, we need to say something about the beans we are going to create.
Let’s look at the simplest file there. The beans.xml can be found in META-INF:

<beans xmlns="http://xmlns.jcp.org/xml/ns/javaee"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
        http://xmlns.jcp.org/xml/ns/javaee/beans_1_1.xsd"
       xmlns:weld="http://jboss.org/schema/weld/beans"
       bean-discovery-mode="all">
    <weld:scan>
        <weld:exclude name="org.jesperancinha.fintech.model.Accounts"/>
    </weld:scan>
</beans>

This is just a typical and as you may be thinking now, a bit of an old file. At this point, the idea is just to get KumuluzEE running. We do have an exclude action. This tells Weld not to consider class Accounts in its scanning for beans action. This is important because with the implementation we are using, Weld will basically consider every class with an empty constructor as a bean. We will see later on why we don’t want Accounts to be considered a bean. For the moment let’s keep in mind that we are making requests under the Request scope. This is logical because every request can have a different user.
Let’s now see how "logback" is implemented. It is also found in META-INF:

<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

This is just a very straightforward configuration for our logs.
Finally, maybe the most important file of our application. This is the config-template. At this point, it is important to note that some of the files I’ve created in this project are part of a template structure. I will explain more about that later. This template file is supposed to be turned into a config.yml file which will be read by MicroProfile. This file is located at the root of resources:

kumuluzee:
  name: your-financeje-banking
  version: 1.0.0
  jwt-auth:
    public-key: {{ publicKey }}
    issuer: {{ issuer }}
healthy: true

We will see later on what exactly all of these properties actually mean. All of them are self-explanatory. The publicKey and issuer are all parameters which will be replaced. We will explore that later on. Our bash scripts will make sure they get replaced.
We are almost ready to go coding, but first, let’s have a look at our JWT token structure.

6. Hands-on code

Let’s make our very small application. This section will explain how can we get our application to work with JWT. What we want to see is if we can specify users to access some of our REST methods and not others.
One of the ways to start looking at this code is to first have a look at our plain JWT token. Here is our admin example:

{
  "iss": "joaofilipesabinoesperancinha",
  "jti": "01MASTERFINANCE",
  "sub": "admin",
  "aud": "nature",
  "upn": "admin",
  "groups": [
    "user",
    "admin",
    "client",
    "credit"
  ],
  "user_id": 1,
  "access": "TOP",
  "name": "Admin"
}

Each one of these names in our JSON is referred to as claims. In our example, we see a few Reserved claims:

"iss" — This is the issuer of the token. We can arbitrarily choose a value for this. The value of this parameter must match the issuer variable to be replaced in the config.yml we have seen before.
"jti" — This is a unique identifier of the token. We can for example use this claim to prevent a token from being used twice or more times.
"sub" — This is the subject of the token. It can be the user or anything we like. It’s important to keep in mind that this can also be used as an identifier, key, naming, or anything we want.
"upn" — User principal name. This is used to identify the principal the user is using.
"groups" — This is an array of the groups the current user belongs to. Essentially this will determine what a request with this token can do. In our token, we then see a few Custom claims. We can use this just as well as the Reserved claims
"user_id" — We will use this to set the user id.
"access" — We will determine the access level of the user.
"name" — The name of the user.

7. Hands-on code

Let’s make a recap of what we know so far. We know we will communicate with tokens with a structure we have determined. Further, we have set up the configuration of our application, the logback configuration and finally, we set up a custom configuration for the enterprise bean lookup.
Let’s look at the package model. Here we will find 3 classes. These classes basically just represent an aggregation of accounts and the representation between client and account. This way we can start by looking at kotlin file Model.kt where Client is located at:

data class Client constructor(
    @JsonProperty
    var name: String ?= null
)

This first model class is the representation of our client. Our client for our case only has a name. This is the username represented by the "jwt" attribute name.
Further, we have Account:

data class Account(
    @JsonProperty
    val accountNumber: String?,
    @JsonProperty
    val client: Client? = null,
    @JsonProperty
    var currentValue: BigDecimal = BigDecimal.ZERO,
    @JsonProperty
    var creditValue: BigDecimal = BigDecimal.ZERO
) {
    fun addCurrentValue(value: Long) = Account(
        accountNumber, client, currentValue
            .add(BigDecimal.valueOf(value)), creditValue
    )

    fun addCreditValue(value: Long): Account = Account(
        accountNumber, client, currentValue, currentValue
            .add(BigDecimal.valueOf(value))
    )
}

In this class, we basically set up an accountNumber, a client, a currentValue and finally a creditValue. Notice that we are defaulting all values to 0. We are also using BigDecimal, purely because we are dealing with money. Money must be exact and cannot suffer system round-ups or round-downs. This means in other words and as an example that a number such as 0.0000000000000000000000000000000000000000000000000001 euros must remain that number all the time. Also, we want to add values to our account. This is where the method addCurrentValue comes to exist. For the same reasons, we will also top up our credit with the addCreditValue.
Finally, in the last piece of our data setup we come across class Accounts:

open class Accounts constructor(
    open val accountMap: MutableMap<String, Account> = mutableMapOf()
)

This is essentially just an aggregator of all our accounts. We will use its map content to mimic the behavior of a database.
Now let’s look at the controller package. This is where we create our application running with our data model. First, let’s have a look at class BankApplication:

@LoginConfig(authMethod = "MP-JWT")
@ApplicationPath("/")
@DeclareRoles("admin", "creditor", "client", "user")
class BankApplication : Application()

With this, we are saying 3 important things. With the LoginConfig annotation, we define it to use and understand JWT tokens according to MicroProfile. The ApplicationPath defines the application root. This is where the URL of the application will begin. In our example, it will be HTTP://localhost:8080. Finally, the DeclareRoles defines the roles that are going to be used and accepted by our application. Roles and Groups are interchangeable terms in this situation.
In order for injection to work efficiently, we create an annotation specific to identify the account map:

annotation class AccountsProduct

Next, we create a cache object factory AccountsFactory:

class AccountsFactory : Serializable {
    @Produces
    @AccountsProduct
    @ApplicationScoped
    fun accounts(): Accounts = Accounts(mutableMapOf())

    companion object {
        @Throws(JsonProcessingException::class)
        fun createResponse(
            currentAccount: Account,
            name: JsonString,
            accounts: Accounts,
            log: Logger,
            objectMapper: ObjectMapper,
            principal: Principal?,
            jsonWebToken: JsonWebToken?
        ): Response {
            val jsonObject = Json.createObjectBuilder()
                .add("balance", currentAccount.currentValue)
                .add("client", name)
                .build()
            accounts.accountMap[name.string] = currentAccount
            log.info("Principal: {}", objectMapper.writeValueAsString(principal))
            log.info("JSonWebToken: {}", objectMapper.writeValueAsString(jsonWebToken))
            return Response.ok(jsonObject)
                .build()
        }
    }
}

This factory is the reason why we disabled lookup specifically for Accounts. Instead of allowing the lookup process to create a bean, we create the aggregator instance ourselves. Using the Produces annotation, allows us to create the bean. Using our custom annotation, AccountsProduct, we make the use of this bean more specific. Finally, by using ApplicationScoped, we define its scope as being the Application scope. In other words, the account aggregation bean will behave as a singleton object across the application.
The "createResponse" is just a generic method to create JSON responses.
What we now need is two "Resources". This is basically the same as "Controllers" in Spring. It is a different name, but it has exactly the same usage.
Let’s look at the AccountsResource class:

@Path("accounts")
@RequestScoped
@Produces(MediaType.APPLICATION_JSON)
open class AccountResource {

    @Inject
    @AccountsProduct
    open var accounts: Accounts? = null

    @Inject
    open var principal: Principal? = null

    @Inject
    open var jsonWebToken: JsonWebToken? = null

    @Inject
    @Claim("access")
    open var access: JsonString? = null

    @Claim("iat")
    @Inject
    open var iat: JsonNumber? = null

    @Inject
    @Claim("name")
    open var name: JsonString? = null

    @Inject
    @Claim("user_id")
    open var userId: JsonNumber? = null

    @POST
    @RolesAllowed("admin", "client", "credit")
    @Throws(JsonProcessingException::class)
    open fun createAccount(): Response = createResponse(
        requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: Account(
            client = Client(name = requireNotNull(name).string),
            accountNumber = UUID.randomUUID().toString()
        )
    )

    @POST
    @RolesAllowed("admin", "user")
    @Path("user")
    @Throws(JsonProcessingException::class)
    open fun createUser(): Response {
        return createResponse(
            requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: Account(
                client = Client(name = requireNotNull(name).string),
                accountNumber = UUID.randomUUID().toString()
            )
        )
    }

    @GET
    @RolesAllowed("admin", "client")
    @Throws(JsonProcessingException::class)
    open fun getAccount(): Response? {
        return createResponse(
            requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: return Response.serverError()
                .build()
        )
    }

    @PUT
    @RolesAllowed("admin", "client")
    @Consumes(MediaType.APPLICATION_JSON)
    @Throws(
        JsonProcessingException::class
    )
    open fun cashIn(transactionBody: TransactionBody): Response? {
        val userAccount =
            requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: return Response.serverError()
                .build()

        val currentAccount = userAccount.addCurrentValue(transactionBody.saldo?: 0)
        requireNotNull(accounts).accountMap[requireNotNull(name).string] = currentAccount
        return createResponse(currentAccount)
    }

    @GET
    @Path("all")
    @Produces(MediaType.APPLICATION_JSON)
    @Throws(
        JsonProcessingException::class
    )
    open fun getAll(): Response? {
        val allAccounts = ArrayList(
            requireNotNull(accounts).accountMap
                .values
        )
        logger.info("Principal: {}", objectMapper.writeValueAsString(principal))
        logger.info("JSonWebToken: {}", objectMapper.writeValueAsString(jsonWebToken))
        return Response.ok(allAccounts)
            .build()
    }

    @GET
    @Path("summary")
    @Throws(JsonProcessingException::class)
    open fun getSummary(): Response? {
        val totalCredit = requireNotNull(accounts).accountMap
            .values
            .map(Account::currentValue)
            .stream()
            .reduce { result, u -> result.add(u) }
            .orElse(BigDecimal.ZERO)

        val jsonObject = Json.createObjectBuilder()
            .add("totalCurrent", totalCredit)
            .add("client", "Mother Nature Dream Team")
            .build()
        logger.info("Summary")
        logger.info("Principal: {}", objectMapper.writeValueAsString(principal))
        logger.info("JSonWebToken: {}", objectMapper.writeValueAsString(jsonWebToken))
        return Response.ok(jsonObject)
            .build()
    }

    @GET
    @RolesAllowed("admin", "client")
    @Path("jwt")
    open fun getJWT(): Response? {
        val jsonObject = Json.createObjectBuilder()
            .add("jwt", requireNotNull(jsonWebToken).rawToken)
            .add("userId", requireNotNull(userId).doubleValue())
            .add("access", requireNotNull(access).string)
            .add("iat", requireNotNull(iat).doubleValue())
            .build()
        return Response.ok(jsonObject)
            .build()
    }

    @Throws(JsonProcessingException::class)
    private fun createResponse(currentAccount: Account): Response =
        AccountsFactory.createResponse(
            currentAccount,
            requireNotNull(name),
            requireNotNull(accounts),
            logger,
            objectMapper,
            principal,
            jsonWebToken
        )

    companion object {
        val objectMapper: ObjectMapper = ObjectMapper()
        val logger: Logger = LoggerFactory.getLogger(AccountResource::class.java)
    }
}

Take a moment to look at this class more in detail. The Path annotation defines how to reach this resource from the root. Remember that we are using "/" as the root. In this case, "accounts" is our root access point for this resource. All of our recourses, in our case only two are running with scope RequestResource. With annotation Produces determines that all responses to all requests regardless of their type will take the form of JSON formatted messages.
To inject our aggregator we just use the combination of the Inject annotation and AccountsProduct annotation:

@Inject
@AccountsProduct
open var accounts: Accounts? = null

This matches what we defined in the factory.
Further, we are also injecting two important elements of security. A principal and the jsonWebToken:

@Inject
open var principal: Principal? = null

@Inject
open var jsonWebToken: JsonWebToken? = null

Both JsonWebToken and Principal will be the same, and we will see that in our logs.
In our resources, we can always inject claims from a request with a certain token:

@Inject
@Claim("name")
open var name: JsonString? = null

@Inject
@Claim("user_id")
open var userId: JsonNumber? = null

This is accomplished with the combination of the Inject and Claim annotations. The name placed under the Claim annotation defines which claim we want to inject. We have to be careful with the type we define our parameters with. In our example,r we only need JsonString and JsonNumber types.
First, let’s look at how we are creating accounts and users:

@POST
@RolesAllowed("admin", "client", "credit")
@Throws(JsonProcessingException::class)
open fun createAccount(): Response = createResponse(
    requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: Account(
        client = Client(name = requireNotNull(name).string),
        accountNumber = UUID.randomUUID().toString()
    )
)

@POST
@RolesAllowed("admin", "user")
@Path("user")
@Throws(JsonProcessingException::class)
open fun createUser(): Response {
    return createResponse(
        requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: Account(
            client = Client(name = requireNotNull(name).string),
            accountNumber = UUID.randomUUID().toString()
        )
    )
}

Creating accounts and users

The purpose here is to be able to separate methods and give them different permissions. In our example, they both just create an account, but it’s important to notice that only users with roles user can use the createUser method. In the same way, only users with roles of client and credit can access the method createAccount.
Let’s now look in detail at the PUT request method of this resource:

@PUT
@RolesAllowed("admin", "client")
@Consumes(MediaType.APPLICATION_JSON)
@Throws(
    JsonProcessingException::class
)
open fun cashIn(transactionBody: TransactionBody): Response? {
    val userAccount =
        requireNotNull(accounts).accountMap[requireNotNull(name).string] ?: return Response.serverError()
            .build()

    val currentAccount = userAccount.addCurrentValue(transactionBody.saldo?: 0)
    requireNotNull(accounts).accountMap[requireNotNull(name).string] = currentAccount
    return createResponse(currentAccount)
}

Cashing In

We know that annotation PUT indicates that this method is only accessible with requests of type PUT. Annotation Path then tells Jetty that the path to this method is a value. This is also known as a PathParam. Finally, we can define this method be only allowed to be used by users with roles admin or client. The input value is then passed on to our Long value variable by the usage of the PathParam.
If we don’t define any roles, then any user with the right token will be able to access these methods.
The CreditResource is implemented in the same way:

@Path("credit")
@RequestScoped
@Produces(MediaType.APPLICATION_JSON)
open class CreditResource {
    @Inject
    @AccountsProduct
    open var accounts: Accounts? = null

    @Inject
    open var principal: Principal? = null

    @Inject
    open var jsonWebToken: JsonWebToken? = null

    @Inject
    @Claim("access")
    open var access: JsonString? = null

    @Inject
    @Claim("iat")
    open var iat: JsonNumber? = null

    @Inject
    @Claim("name")
    open var name: JsonString? = null

    @Inject
    @Claim("user_id")
    open var userId: JsonNumber? = null

    @GET
    @RolesAllowed("admin", "credit")
    @Throws(JsonProcessingException::class)
    open fun getAccount(): Response = requireNotNull(accounts).let { accounts ->
        createResponse(
            accounts.accountMap[requireNotNull(name).string] ?: return Response.serverError().build()
        )
    }

    @PUT
    @RolesAllowed("admin", "credit")
    @Consumes(MediaType.APPLICATION_JSON)
    @Throws(
        JsonProcessingException::class
    )
    open fun cashIn(transactionBody: TransactionBody) = requireNotNull(accounts).let { accounts ->
        requireNotNull(name).let { name ->
            accounts.accountMap[name.string] = (accounts.accountMap[name.string] ?: return Response.serverError()
                .build()).addCreditValue(transactionBody.saldo?: 0L)
            createResponse(
                (accounts.accountMap[name.string] ?: return Response.serverError()
                    .build()).addCreditValue(transactionBody.saldo?: 0L)
            )
        }
    }

    @GET
    @Path("all")
    @Produces(MediaType.APPLICATION_JSON)
    @Throws(
        JsonProcessingException::class
    )
    open fun getAll(): Response? {
        val allAccounts = ArrayList(
            requireNotNull(accounts).accountMap
                .values
        )
        logger.info("Principal: {}", objectMapper.writeValueAsString(principal))
        logger.info("JSonWebToken: {}", objectMapper.writeValueAsString(jsonWebToken))
        return Response.ok(allAccounts)
            .build()
    }

    @GET
    @Path("summary")
    @Produces(MediaType.APPLICATION_JSON)
    @Throws(
        JsonProcessingException::class
    )
    open fun getSummary(): Response? {
        val totalCredit = requireNotNull(accounts).accountMap
            .values
            .map(Account::creditValue)
            .stream()
            .reduce { total, v -> total.add(v) }
            .orElse(BigDecimal.ZERO)
        val jsonObject = Json.createObjectBuilder()
            .add("totalCredit", totalCredit)
            .add("client", "Mother Nature Dream Team")
            .build()
        logger.info("Summary")
        logger.info("Principal: {}", objectMapper.writeValueAsString(principal))
        logger.info("JSonWebToken: {}", objectMapper.writeValueAsString(jsonWebToken))
        return Response.ok(jsonObject)
            .build()
    }

    @GET
    @RolesAllowed("admin", "client")
    @Path("jwt")
    open fun getJWT(): Response? {
        val jsonObject = Json.createObjectBuilder()
            .add("jwt", requireNotNull(jsonWebToken).rawToken)
            .add("userId", requireNotNull(userId).doubleValue())
            .add("access", requireNotNull(access).string)
            .add("iat", requireNotNull(iat).doubleValue())
            .build()
        return Response.ok(jsonObject)
            .build()
    }

    @Throws(JsonProcessingException::class)
    private fun createResponse(currentAccount: Account): Response {
        return AccountsFactory.createResponse(
            currentAccount,
            requireNotNull(name),
            requireNotNull(accounts),
            logger,
            objectMapper,
            principal,
            jsonWebToken
        )
    }

    companion object {
        val objectMapper: ObjectMapper = ObjectMapper()
        val logger: Logger = LoggerFactory.getLogger(CreditResource::class.java)
    }
}

The only difference is that instead of using roles admin and client we are now using admin and credit roles. Also, notice that accounts for users will never be created in this resource. That’s only possible via the account’s resource.
Now that we know how the code is implemented let’s first recap which methods have we made available in our REST service.

8. Application Usage

Let’s check the list of the services being used:

Type,URL,Payload,Result,Roles Allowed
POST,http://localhost:8080/accounts,n/a,Created account,admin/client/credit
POST,http://localhost:8080/accounts/user,n/a,Created user,admin/user
GET,http://localhost:8080/accounts,n/a,Matching account,admin/client
PUT,http://localhost:8080/accounts,{saldo: Long}, Current Balance,admin/client
GET,http://localhost:8080/accounts/all,n/a,All current accounts,All
GET,http://localhost:8080/accounts/summary,n/a,Sum of all balances,All
GET,http://localhost:8080/credit,n/a,Matching account,admin/client
PUT,http://localhost:8080/credit,{saldo: Long}, Current Credit,admin/client
GET,http://localhost:8080/credit/all,n/a,All credits,All
GET,http://localhost:8080/credit/summary,n/a,Sum credits,All

All Endpoints

9. Generating test environment

I have created a bash file in the root folder. This file is called "setupCertificates.sh". Let’s have a look into it to have an idea of what it does:

#!/bin/bash

mkdir -p your-finance-files

cd your-finance-files || exit

openssl genrsa -out baseKey.pem

openssl pkcs8 -topk8 -inform PEM -in baseKey.pem -out privateKey.pem -nocrypt

openssl rsa -in baseKey.pem -pubout -outform PEM -out publicKey.pem

echo -e '\033[1;32mFirst test\033[0m'
java -jar ../your-finance-jwt-generator/target/your-finance-jwt-generator.jar \
        -p ../jwt-plain-tokens/jwt-token-admin.json \
        -key ../your-finance-files/privateKey.pem >> token.jwt

CERT_PUBLIC_KEY=$(cat ../your-finance-files/publicKey.pem)
CERT_ISSUER="joaofilipesabinoesperancinha"

echo -e "\e[96mGenerated public key: \e[0m $CERT_PUBLIC_KEY"
echo -e "\e[96mIssued by: \e[0m $CERT_ISSUER"
echo -e "\e[96mYour token is: \e[0m $(cat token.jwt)"

cp ../your-financeje-banking/src/main/resources/config-template ../your-financeje-banking/src/main/resources/config_copy.yml

CERT_CLEAN0=${CERT_PUBLIC_KEY//"/"/"\/"}
CERT_CLEAN1=${CERT_CLEAN0//$'\r\n'/}
CERT_CLEAN2=${CERT_CLEAN1//$'\n'/}
CERT_CLEAN3=$(echo "$CERT_CLEAN2" | awk '{gsub("-----BEGIN PUBLIC KEY-----",""); print}')
CERT_CLEAN4=$(echo "$CERT_CLEAN3" | awk '{gsub("-----END PUBLIC KEY-----",""); print}')
CERT_CLEAN=${CERT_CLEAN4//$' '/}

echo -e "\e[96mCertificate cleanup: \e[0m ${CERT_CLEAN/$'\n'/}"

sed "s/{{ publicKey }}/$CERT_CLEAN/g" ../your-financeje-banking/src/main/resources/config_copy.yml > ../your-financeje-banking/src/main/resources/config_cert.yml
sed "s/{{ issuer }}/$CERT_ISSUER/g" ../your-financeje-banking/src/main/resources/config_cert.yml > ../your-financeje-banking/src/main/resources/config.yml

rm ../your-financeje-banking/src/main/resources/config_cert.yml
rm ../your-financeje-banking/src/main/resources/config_copy.yml

echo -e "\e[93mSecurity elements completely generated!\e[0m"

echo -e "\e[93mGenerating tokens...\e[0m"

TOKEN_FOLDER=jwt-tokens
mkdir -p ${TOKEN_FOLDER}
#
CREATE_ACCOUNT_FILE=createAccount.sh
CREATE_USER_FILE=createUser.sh
SEND_MONEY_FILE=sendMoney.sh
ASK_CREDIT_FILE=askCredit.sh
TOKEN_NAME_VALUE=tokenNameValue.csv

echo "#!/usr/bin/env bash" > ${CREATE_ACCOUNT_FILE}
chmod +x ${CREATE_ACCOUNT_FILE}
echo "#!/usr/bin/env bash" > ${CREATE_USER_FILE}
chmod +x ${CREATE_USER_FILE}
echo "#!/usr/bin/env bash" > ${SEND_MONEY_FILE}
chmod +x ${SEND_MONEY_FILE}
echo "#!/usr/bin/env bash" > ${ASK_CREDIT_FILE}
chmod +x ${ASK_CREDIT_FILE}

for item in ../jwt-plain-tokens/jwt-token*.json; do
     if [[ -f "$item" ]]; then
        filename=${item##*/}
        per_token=${filename/jwt-token-/}
        token_name=${per_token/.json/}
        cp "${item}" jwt-token.json
        java -jar ../your-finance-jwt-generator/target/your-finance-jwt-generator.jar \
                -p jwt-token.json \
                -key ../your-finance-files/privateKey.pem > token.jwt
        cp token.jwt ${TOKEN_FOLDER}/token-"${token_name}".jwt

        token=$(cat token.jwt)

        echo "# Create account: ""${token_name}" >> ${CREATE_ACCOUNT_FILE}
        echo "echo  -e \"\e[93mCreating account \e[96m${token_name}\e[0m\"" >> ${CREATE_ACCOUNT_FILE}
        echo curl -i -H"'Authorization: Bearer ""${token}""'" http://localhost:8080/accounts -X POST >> ${CREATE_ACCOUNT_FILE}
        echo "echo  -e \"\e[93m\n---\e[0m\"" >> ${CREATE_ACCOUNT_FILE}

        echo "# Create user: ""${token_name}" >> ${CREATE_USER_FILE}
        echo "echo  -e \"\e[93mCreating user \e[96m${token_name}\e[0m\"" >> ${CREATE_USER_FILE}
        echo curl -i -H"'Authorization: Bearer ""${token}""'" http://localhost:8080/accounts/user -X POST >> ${CREATE_USER_FILE}
        echo "echo  -e \"\e[93m\n---\e[0m\"" >> ${CREATE_USER_FILE}

        echo "# Send money to: "${token_name} >> ${SEND_MONEY_FILE}
        echo "echo  -e \"\e[93mSending money to \e[96m${token_name}\e[0m\"" >> ${SEND_MONEY_FILE}
        echo curl -i -H"'Content-Type: application/json'" -H"'Authorization: Bearer ""${token}""'" http://localhost:8080/accounts -X PUT -d "'{ \"saldo\": "$((1 + RANDOM % 500))"}'" >> ${SEND_MONEY_FILE}
        echo "echo  -e \"\e[93m\n---\e[0m\"" >> ${SEND_MONEY_FILE}

        echo "# Asking money credit to: "${token_name} >> ${ASK_CREDIT_FILE}
        echo "echo  -e \"\e[93mAsking credit from \e[96m${token_name}\e[0m\"" >> ${ASK_CREDIT_FILE}
        echo curl -i -H"'Content-Type: application/json'" -H"'Authorization: Bearer ""${token}""'" http://localhost:8080/credit -X PUT -d "'{ \"saldo\": "$((1 + RANDOM % 500))"}'">> ${ASK_CREDIT_FILE}
        echo "echo  -e \"\e[93m\n---\e[0m\"" >> ${ASK_CREDIT_FILE}

        echo "${token_name},${token}" >> ${TOKEN_NAME_VALUE}
      fi
done

Environment generation

Please follow the file as I explain what it does. This is important so that we understand exactly what it is doing. We first create private and public keys in a PEM format. We then use the private key with our runnable "your-finance-jwt-generator.jar" . This is our runnable jar which allows for the quick creation of tokens. The issuer cannot be changed later on. Finally, it creates a token. We will see how to read this token later on. This token contains 3 extra Header claims. These are "kid", "typ", and "alg". It follows the following format:

{
  "kid": "jwt.key",
  "typ": "JWT",
  "alg": "RS256"
}

The header of the JWT

Let’s look at these claims more closely:

"kid" — Works as a hint claim. It indicates which sort of algorithm we are using.
"typ" — It is used to declare IANA media types. There are three options JWT(JSON Web token), JWE(JSON Web Encryption), and JWA(JSON Web Algorithms). These types aren’t relevant to our experiment. We will only see that our token isn’t really well encrypted and that it’s really easy to decrypt it. We will also see that although we can decrypt tokens, we cannot that easily tamper the to perform other actions.
"alg" — This is how we define the signature type we want to use. Signing can be considered as a cryptographic operation that will ensure that the original token has not been changed and is trusted. In our case, we are using RS256 otherwise known as RSA Signature with SHA-256.

With our public key, we can finally use it to change our template. The new config.yml file should look something like this:

kumuluzee:
  name: your-financeje-banking
  version: 1.0.0
  jwt-auth:
    public-key: FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKE.FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETO.FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKEN
    issuer: joaofilipesabinoesperancinha
healthy: true

config.yml

The second step is to create four files. For every single plain token in the directory "jwt-plain-tokens", we will create four commands. The first command is to create users that can effectively do things with their accounts. These are users with profiles "admin", "client", and "credit".
Let’s run the file "createAccount.sh", in order to create them. The second command will create the rest of the users which don’t possess any rights yet. This is the file "createUser.sh". Let’s run it. Now we’ll see that all users are finally created. Let’s now look into details about transactions and look at the remaining two commands. One to "cashin" and another to ask for more credit. The first generated file is the "sendMoney.sh" bash script. Here we can find all requests to "cashin". In this file you’ll find a curl request to send random money quantities to users, per user. Let’s look at the admin case:

#!/usr/bin/env bash
# Send money to: admin
echo  -e "\e[93mSending money to \e[96madmin\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer= FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 125}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: cindy
echo  -e "\e[93mSending money to \e[96mcindy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 360}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: faustina
echo  -e "\e[93mSending money to \e[96mfaustina\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 50}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: jack
echo  -e "\e[93mSending money to \e[96mjack\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 205}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: jitska
echo  -e "\e[93mSending money to \e[96mjitska\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 332}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: judy
echo  -e "\e[93mSending money to \e[96mjudy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 295}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: lucy
echo  -e "\e[93mSending money to \e[96mlucy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 160}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: malory
echo  -e "\e[93mSending money to \e[96mmalory\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 413}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: mara
echo  -e "\e[93mSending money to \e[96mmara\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 464}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: namita
echo  -e "\e[93mSending money to \e[96mnamita\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 51}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: pietro
echo  -e "\e[93mSending money to \e[96mpietro\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 491}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: rachelle
echo  -e "\e[93mSending money to \e[96mrachelle\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 474}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: sandra
echo  -e "\e[93mSending money to \e[96msandra\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 417}'
echo  -e "\e[93m\n---\e[0m"
# Send money to: shikka
echo  -e "\e[93mSending money to \e[96mshikka\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/accounts -X PUT -d '{ "saldo": 64}'
echo  -e "\e[93m\n---\e[0m"

sendMoney.sh extract

The same users have also their credit requests assigned to them:

#!/usr/bin/env bash
# Asking money credit to: admin
echo  -e "\e[93mAsking credit from \e[96madmin\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 137}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: cindy
echo  -e "\e[93mAsking credit from \e[96mcindy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 117}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: faustina
echo  -e "\e[93mAsking credit from \e[96mfaustina\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 217}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: jack
echo  -e "\e[93mAsking credit from \e[96mjack\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 291}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: jitska
echo  -e "\e[93mAsking credit from \e[96mjitska\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 184}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: judy
echo  -e "\e[93mAsking credit from \e[96mjudy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 388}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: lucy
echo  -e "\e[93mAsking credit from \e[96mlucy\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 219}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: malory
echo  -e "\e[93mAsking credit from \e[96mmalory\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 66}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: mara
echo  -e "\e[93mAsking credit from \e[96mmara\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 441}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: namita
echo  -e "\e[93mAsking credit from \e[96mnamita\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 358}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: pietro
echo  -e "\e[93mAsking credit from \e[96mpietro\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 432}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: rachelle
echo  -e "\e[93mAsking credit from \e[96mrachelle\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 485}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: sandra
echo  -e "\e[93mAsking credit from \e[96msandra\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 500}'
echo  -e "\e[93m\n---\e[0m"
# Asking money credit to: shikka
echo  -e "\e[93mAsking credit from \e[96mshikka\e[0m"
curl -i -H'Content-Type: application/json' -H'Authorization: Bearer FAKE.FAKE.FAKE' http://localhost:8080/credit -X PUT -d '{ "saldo": 89}'
echo  -e "\e[93m\n---\e[0m"

askCredit.sh extract

All our characters are part of the League of Nature. Essentially just some group of people to be part of this banking system. In this context, they are defending the environment. It’s not really relevant for the article what this group of people does or where in the story they fit in, but for context, they participate in actions to defend the environment and slow down the effects of climate change. Some of our characters can do everything, others cannot do anything and others can only "cashin" or just "ask for credit". Also notice that I am obfuscating sensitive information. These tokens normally should not be shared or be visible in on particular URL. They are yes always available via the browser developer console but anyway is to protect some requests being made. This is a concept known as "security-per-obscurity" and although it does not technically prevent the user to become aware of the token being used, it does work as a deterrent.
In both methods, when we make a deposit or when we ask for credit, notice that for each request, we are sending a random number between 1 to 500.
We are now almost ready to start our application, but first, let’s take a dive into a bit more theory.

10. How is a `JWT` token made

Now that we have generated our tokens, let’s look into one of them. I am going to show you an obfuscated token, and we are going to use that to understand this.
Here is our token:
FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKE.FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETO.FAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKENFAKETOKEN
What’s important here to notice is that our token is divided into three parts:

Header — This is a Base64 encoded JSON configuration header, just as we discussed above.
Payload — This is a Base64 encoded JSON payload. This is where we defined our Reserved and Custom claims. We can also define here Private and Public claims. Both of them fall under Custom claims. As a quick note, we can do what we want with both of these claims. However public claims are referred to as the ones defined in the IANA JSON Web Token Registry. It is important that we name our tokens in a certain way in order to avoid collisions with the registry. Public claims can also be defined as standard optional. Private claims follow no standard and it’s up to us to define them.
Signature — This is where we can be a bit creative. The signature is a cyphered combination of the Header and the Payload. We decide the algorithm we want to use and this bit of the token will basically determine if the message we are sending is to be trusted. It is unique to that combination and our server will use the "public-key" we created to determine if we have a match. If you remember from the above we are using RS256 in our example.

Before we continue, please note that both the Header and the Payload can be decyphered in our example. We just "can’t" tamper with the payload or the header and still make it trusted. The protection against the potential effects of a malicious token can only be protected by the algorithm we choose. So choose wisely.
If you are working in an organization where top secret information is a concern, such as a bank, please DON’T do what we are about to do. This is only a way for us to check online the content of the tokens we’ve generated locally.
First, let’s go to https://jwt.io/ and fill out our JWT token. Use the token you’ve just generated:

Using https://jwt.io/ to check the contents of our token
Let’s examine what we have here. This is our administrator token. That person is "Admin" in our example. We can see that our parameters are all available. In our list we see "sub", "aud", "upn", "access", "user_id", "iss", "name", "groups" and finally the "jti". We also have some extra claims. Let’s look at them:

"auth_time" — This is when the authentication has happened. Our token as been authenticated on Sunday, 17 July 2022 16:15:47 GMT+02:00 DST
"iat" — This is when the token has been created. In our case, this happens simultaneously as the auth_time.
"exp" — This is the expiry date of the token. It expires on Sunday, 17 July 2022 16:32:27 GMT+02:00 DST. We didn’t specify any expiry date in our token. This means that JWT uses its default value of ~15 minutes.

Let’s now perform some tests.

11. Running the application

The code is ready to be used on GitHub. If we check the code out and open it with Intellij we need to be aware that we can’t run this application like a Spring Boot application. There is no "psvm" to make it run. Instead, we can just run the generated jar directly and make sure that we make an "mvn build" just before. Here is how I am using it at the moment:

[]https://github.com/jesperancinha/your-finance-je "Environment setup to run the application")

Let’s now run the "setupCertificates.sh" script again. I don’t know how much time you took to get here but it’s very likely that the 15 minutes are already gone at this point. Just in case, just run them again.
Let’s start our app!
We can start it like this:

mvn clean install
java -jar your-financeje-banking/target/your-financeje-banking.jar

Or we can just run it through our ready-to-go running configuration. Check the repo and the Makefile beforehand if you want to understand everything it does:

make dcup-full-action

This script will run 2 services. One on port 8080 and the other on port 8081. On port 8080 we will run a version of this software running our own code to generate JWT tokens. On port 8081, we will run a version using the jwtknizr generator created by Adam Bien. We will focus this article, however on the service running on port 8080. If you want, you can also run cypress with:

make cypress-open

This will open the cypress console, and you’ll be able to run the tests with the browser of your choice. However, browser options are still limited at this stage. Most of the requests will actually be command line requests provided by cypress.
For now, let’s not go into "cypress". Please go to your browser and head on to this location:

http://localhost:8080/accounts/all

We should be getting a result like this:

As we can see, "Malory", "Jack Fallout", and "Jitska" don’t have any credit or money. This is because they have only been given the user group. Notice also that Shikka has been given no credit. "Shikka", is our only client who doesn’t have the group credit.
If we look at the logs, we can see that successful operations take this format:

Sending money to admin
HTTP/1.1 200 OK
Date: Sun, 17 Jul 2022 15:01:13 GMT
X-Powered-By: KumuluzEE/4.1.0
Content-Type: application/json
Content-Length: 32
Server: Jetty(10.0.9)
{"balance":212,"client":"Admin"}

A 200 lets us know that the operation went successfully.
In the case of "Malory", "Jack Fallout", and "Jitska", both operations fail and then we will get this kind of message:

Sending money to jitska
HTTP/1.1 403 Forbidden
X-Powered-By: KumuluzEE/4.1.0
Content-Length: 0
Server: Jetty(10.0.9)

A 403 lets us know that our JWT token has been validated and it’s trusted. However, the user is forbidden to perform that operation. In other words, they have no access to the designated method.

Let’s tamper with our tokens a bit. If we change some of the tokens of the sendMoney.sh file. We should be getting this:

Sending money to admin
HTTP/1.1 401 Unauthorized
X-Powered-By: KumuluzEE/4.1.0
WWW-Authenticate: Bearer realm="MP-JWT"
Content-Length: 0
Server: Jetty(10.0.9)

This 401 means that our token was not validated. It means that the public key that the server uses to check if our token is to be trusted, has found no match. If the public key cannot evaluate and validate the signature of the JWT token, it will then reject it.

As a recap, the Header and the "Payload" are not encrypted. They are just base 64 "encoded". This means that "Decoding" allows us always to have a peek inside what the payload actually is. If we are looking to safeguard our payload from eavesdropping, we shouldn’t use the "Payload" of the token for anything else but select identification parameters. The problem lies really when someone gets their hands on the JWT token, for example, when the TLS tunnel has been compromised and someone is able to read the content of the exchanged messages. When that happens, there is still another safeguard. And this is the signature. The only one able to validate a message going in is the server that contains the public key. This public key, although public, only allows validating the incoming message by running against the signature and the "Header + Payload".

12. Conclusion

We have reached the end of our session. Thank you for following this.
We can see how JWT tokens are compact and very much less verbose than their XML counterpart, the SAML tokens. We have seen how easy it is to create and use tokens to get certain authorizations needed for certain methods and how we get there via a signed token.
I find however very important to get an idea of how JWT works. Hopefully, with this, I have given you a good introduction to how JWT tokens work.
To get a better idea of how all of this works, I advise you to play around with the implemented cypress tests. This is a great way to see how requests are being made and what we are testing and what is expected. Then you’ll also get a better idea of why some users get to perform certain operations and others don’t.
I have placed all the source code of this application on GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it.
Thank you for reading!

13. References

Kotlin Coroutines vs Java Virtual Threads — A good story, but just that…

João Esperancinha — Fri, 27 Dec 2024 10:28:52 +0000

1. Introduction

In this article, we are going to have a trial-and-error look at two different implementations of Coroutines also known as Continuations within the JVM. These are Java Virtual Threads which are a part of Project Loom and Kotlin Coroutines provided as DSLs to run on the JVM. Because of the nature of this article, it will be subject to frequent reviews. The supporting code is located on GitHub.
A bit of history
In recent years, if you work around the JVM, you must have noticed that a new player is in town in the JVM world. Enter Kotlin. Long story short, Kotlin started within JetBrains R&D Department, being named after Kotlin island in the neighborhood of St. Petersburg.
This is the short Kotlin story so far. But in order to understand where we are this year (2022) and how far we’ve come, we need to go down memory lane and understand how Java has developed and when and how other languages derived from Java have flourished. This way we can have a better picture and draw better-informed conclusions on it.

Before we continue, we have to honor the responsible people who have, according to extended documentation, started all of this JVM revolution. James Gosling is by many considered the inventor of Java and the JVM. Without him, nothing that has been invented afterward, on top of the JVM, would be possible. In the same way, Martin Odersky is pretty much the inventor of Scala. Finally, for Kotlin, we can only say that the team leader of the JetBrains team responsible for further developments is Dmitry Jemerov.
The table above is a bit of a short sketch consolidating major highlights in the history of three languages that share a common ecosystem, which is the Java Virtual Machine. Java has existed since before 1995, Scala since 2001, and Kotlin since 2010. Java is the oldest JVM language and the newest is Kotlin. Java started at least 15 years before the early beginnings of Kotlin and Scala started 9 years before Kotlin.
I couldn't find precisely where, but examining the commits for project Loom I could see that the first commit happened in 2007. What this tells us is that it is very likely that the idea of Loom started around this year. Loom is a project, that pretty much like coroutines in Kotlin, focuses on making maximum usage of system threads by fragmenting them into separate independent processes. Loom calls these processes virtual threads. In Kotlin an experimental release supporting this same idea with coroutines was released in 2018. Project Loom in Java is scheduled to be released in 2022.
With regard to Kotlin, is quite hard to pinpoint what exactly was the motivation to create a new language. The best I can find is that "new features needed to be added". In this article, I want to share with you what I have found about Kotlin Coroutines and Java Virtual Threads and then reveal a great conclusion I came up with.
I myself have never been a part of the Java Loom Team nor have I been a part of the Kotlin Coroutines team. I have done this article on the basis of the source code information, international conference videos, and papers.
But before we continue, coroutines were invented a long time ago, but, if you are not aware of it, here is a great revelation. They are indeed very old, and they are actually older than 1958. This is only the year when this term was coined by Donald Knuth and Melvin Conway. Here, people have created their own implementations of this, for example, this one by codecop.

1.1. Motivation

Software engineering has changed through the years and undoubtedly everyone strives to make everything better. We want it to be easier to create software and make our code work. To do that, we have created syntaxes and semantics that enable us to develop with an increasing level of simplicity. When Kotlin came to the scene I was almost immediately sold out to the idea of it being superior to Java on many levels. That’s what the Kotlin community mostly promotes. A few months into it, I realized a few things that defeated the reason for my excitement about it. As time went by I got more and more of this idea that Kotlin is just another language and that may be the true thing that makes it exciting is that it is different. Something new breaks up the routine and makes room for creativity. One thing I didn’t change my mind about, is that Kotlin, done the right way, can produce code that is much more beautiful than Java. But beauty is something I don’t want to discuss in this article. What this article really is about is performance. We are not going to discuss Kotlin and Java alone. We are going to discuss two implementations that make use of System Threads and a very old concept called coroutines. In Java, this is being called virtual threads in project Loom and in Kotlin this is being called… well… coroutines. As we go along in the code on both sides, we’ll make pit-stops and compare code against each other and see the differences. But first, let’s go into a bit of theory to understand exactly what are we talking about, discuss why was this not a revolution before, and why it has taken so much time to get languages to develop interfaces and semantics to be able to use system threads more efficiently.

1.2 Coroutines, what are they?

If we take the literal meaning of coroutines, purely on a semantic level, we get Co and Routines. So, a routine is just some instruction that runs. A coroutine is something that runs along. Running along, in this case, means literally suspending the original routine and allowing a completely different routine to start and then resume the original routine.
To illustrate this, I’ve gone back to 1985, and with the help of the internet, I’ve created a small program in C++ that shows some instructions about creating a table with Epoxy (don’t follow these instructions if you want to create a real Epoxy table, creating tables with epoxy require safety gear and protection, so get informed first). Why C++? Well, why not? And further, I think it is very important to start out with a neutral point. If we get these basics right, then we are on a roll! So this is the main program:

int main()
{
    printf("╔════════════════════════════════════════════════╗\n");
    printf("║ Welcome to the Epoxy Table manufacture factory ║\n");
    printf("╚════════════════════════════════════════════════╝\n");
    printf("--- Tables are made in parallel steps ---\n");
    int i;
    for (; i=processes(1, 11);){
        printf("---> ▶️ Execution (%d)starts here with id %d!\n",i,pthread_self());
        switch(i){
            case 1: printf("==>Collect Epoxy and Resin\n"); break;
            case 2: printf("==>Mix ingredients\n"); break;
            case 3: printf("==>Polish wooden plank\n"); break;
            case 4: printf("==>Water seal container\n"); break;
            case 5: printf("==>Pour epoxy mix\n"); break;
            case 6: printf("==>Wait for it to dry\n"); break;
            case 7: printf("==>Polish final edges and result\n"); break;
            case 8: printf("==>Fine polish\n"); break;
            case 9: printf("==>Paint\n"); break;
            case 10: printf("==>Slow dry surface\n");
        }
        }
    return 0;
}

So, we have a bunch of cases (10 to be exact) and for now, this piece of code doesn’t seem to show a lot. We do have something that should get your attention at the moment and that is pthread_self(). Another thing is processes(1,11), which are included in the validation check of the for-loop. Let’s dive into this method:

int processes(int startIndex, int endIndex)
{
    static long long int i;
    static int state = 0;
    switch (state)
    {
        case 0: state = 1;
                for (i = startIndex; i < endIndex; i++)
                {
                    sleep(1);
                    printf("👍 Ordering at %lld. This suspends the next run (coroutine)\n",currentTSMills());
                    return i;
                    case 1:
                        sleep(1);
                        printf("✅ Ending step. This is the callback (start of the coroutine): %lld at %lld with id %d\n",i, currentTSMills(),pthread_self());
                 }
    }
    puts("┌──────────────┐");
    puts("│ 🛑 Finishe
    d! │");
    puts("└──────────────┘");
    state = 0;
    return 0;
}

So, here, we have a strange switch-case. We are assigning 1 to the state within the case condition of 0. This causes the main thread to split, before returning. It doesn’t technically split into 2, but it does suspend on runtime, to allow the other to start. This means that when the routine hits return i, it will suspend itself and the thread will first run whatever is in the main for-loop and only then it will finish running what’s on case 1. Looking at this in C++, it may seem highly counterintuitive, but if we run the code, then we see this phenomenon taking place, and we can also see, that although the main thread has suspended and resumed different routines, they are all hanging on the same thread:

So this is essentially what a coroutine is. In this C++ example, everything is run asynchronously. There are also many ways to implement a coroutine. What project Loom and Kotlin Coroutines saw as a gold mine in the second half of the 2000s decade, was to explore this and implement coroutines in a sort of asynchronous way. Both languages have evolved, and both still have experimental features running on their respective implementations. However, Java is still in the EAB (Early Access Build) stage, although it has started its developments much earlier in the second half of this decade.

1.3 Java virtual threads

In order to discuss Java Virtual Threads, we have to get familiar with a few basic concepts: Fibers, Continuations, and of course Virtual Threads.

Fibers: To be very clear, fiber is just another way to refer to Virtual Threads. There is nothing magic about it
Virtual Threads: They have been named this way to better refer to their actual behavior. For the developer, there is no apparent difference between a Thread (Platform or System thread) and a Virtual Thread (Something run by the carrier thread that executes independently allowing more processes to run)
Carrier Thread: A term that in the first instance seems to be used by the hip and happening and looks like just another way to refer to a Platform Thread or System Thread. However, it does have a much more important role than that. A Carrier Thread is where one Virtual Thread executes. This becomes more visible when we look into the code, which we will further down below.
Continuation: Fibers and Virtual Threads are continuations. A continuation is just something that allows us to continue after yielding a result. This is the very low level of all virtual threads and how they work. We have seen before how coroutines work. This is exactly how continuations work. In fact, coroutines are just another name for continuations. In the code in the example at the beginning of this article, there would be two virtual threads. The one at the start of the execution and another when we start with the text: "Ending step".

1.3 What are Java Virtual Threads?

At this point, and from the above, I think you are getting a very clear idea of what this whole continuation and coroutines are about. The same thing right? The theory seems to be the same, but the implementation differs. At this stage let's have a look at some of the highlights of the implementation of Virtual Threads (at least in my view):

public static Thread startVirtualThread(Runnable task) {
    Objects.requireNonNull(task);
    var thread = ThreadBuilders.newVirtualThread(null, null, 0, task);
    thread.start();
    return thread;
}

in JDK 21

At this point, nothing happens. We receive a plain runnable, and we get into the method. We are executing inside the JDK19 already and this code is only JDK19 code. Once there, Loom creates a VirtualThread with our task as a parameter and starts it. When we start a virtual thread this way, we are doing so by making the first two parameters null, the third 0, and the fourth one is our task. Let’s dive into the VirtualThread first and see if we see signs of anything remotely similar to what we’ve seen and learned about what a continuation is:

VirtualThread(Executor scheduler, String name, int characteristics, Runnable task) {
    super(name, characteristics, /*bound*/ false);
    Objects.requireNonNull(task);

    // choose scheduler if not specified
    if (scheduler == null) {
        Thread parent = Thread.currentThread();
        if (parent instanceof VirtualThread vparent) {
            scheduler = vparent.scheduler;
        } else {
            scheduler = DEFAULT_SCHEDULER;
        }
    }

    this.scheduler = scheduler;
    this.cont = new VThreadContinuation(this, task);
    this.runContinuation = this::runContinuation;
}

in JDK 21

In this case what this means is that we create a virtual thread without a scheduler, without a name, and 0 characteristics. And of course, what does this all mean? Maybe here we can skip a few steps, but the thread initialization will assign an id to it and no characteristics. Since we don’t give it a name, our thread will not be identifiable with a name. Not by default at least. Before launching our thread, we get a scheduler. In this part, we are coming against some code that ensures that we get either an appropriate scheduler from the System Thread or an appropriate thread from the VirtualThread. We seem to have two types of schedulers. One for the Virtual Thread and another for the System Thread. These look actually to be reused. The new scheduler is only assigned if there is no scheduler given in the constructor, and it is assigned on the basis of the parent thread, which is the current thread. Once we have the scheduler, we can finally create a continuation (VThreadContinuation) with the current VirtualThread, and we pass the runnable task we have given. Finally, we assign the runContinuation property with the runContinuation lambda in order to be able to execute it later.
So now we have created a Virtual Thread with the scheduler of the Platform Thread, no name, one id, and 0 characteristics, and we have assigned a continuation to it and have assigned the runContinuation property with the runContinuation lambda. The scheduler we have just created is a ForkJoinPool, which is created by default with a parallelization level equivalent to the number of CPU’s provided by the machine and a maximum worker pool of 256.
From here onwards, it becomes quite complicated to describe what happens, given that this involves quite a lot of native code calls, which I do not know much about and it is irrelevant for this article. Relevant for this article, though, are the states a virtual thread goes through in its lifecycle. A virtual thread can potentially go through the following states (they are all int values):

New 0: State on the start of the thread.
Started 1: The virtual thread has started.
Runnable 2: The thread is unmounted and this state can be assigned to a thread after it has status Yielding. The thread is not running at this time.
Running 3: The thread is running and it is mounted
Parking 4: Starts disabling thread for scheduling unless the thread has a permit.
Parked 5: The thread gets Parked after a status Parking and after yielding. Parked means, in other words, waiting to be Scheduled.
Pinned 6: A thread gets pinned, when being delayed by a synchronized process, or performing some virtual thread unsupported operation as is the case of some IO operations. Other IO operations are performed in a non-blocking way. More precisely, pinning is a way to not allow a Virtual Thread to unmount if it is waiting for an object that is not available yet.
Yielding 7: The thread gets unmounted in order to yield its control of the processor and then it gets mounted again when it is allowed to do so again. In other words, it’s just returning the carrier Thread. This is also a form of context switching. Sleeping with (0) will trigger this state immediately.
Terminated 99: Final state of the Virtual Thread. It will not be used again.
Suspended 256: A Virtual Thread can be suspended after unmount.
Runnable Suspended: The thread can be runnable and suspended.
Parked Suspended: The thread can be parked and suspended.
When a virtual thread needs to sleep, it will perform a delay operation. This requires something called Yielding. By doing Yielding, we unmount the current virtual thread from its current system thread and yield its control to another virtual thread. If we are performing a blocking operation and the thread is pinning, one system thread will be blocked, but the others won't. This means that, for example. if you have 12 cores, 11 will be used to manage virtual threads, but only 1 will be blocked waiting. Blocking operations happen when using some operations that are blocking in the native code, for example using synchronized and Object.wait() cause the thread to be pinned:

@Hidden
@ChangesCurrentThread
private boolean yieldContinuation() {
    // unmount
    notifyJvmtiUnmount(/*hide*/true);
    unmount();
    try {
        return Continuation.yield(VTHREAD_SCOPE);
    } finally {
        // re-mount
        mount();
        notifyJvmtiMount(/*hide*/false);
    }
}

in JDK 21

Sleeping is one way a virtual thread will pause its execution. It has a different behavior to another virtual thread that is running on a synchronised code. For this combination we need another concept called parking in VirtualThread.java:

@Override
void park() {
    assert Thread.currentThread() == this;

    // complete immediately if parking permit available or interrupted
    if (getAndSetParkPermit(false) || interrupted)
        return;

    // park the thread
    boolean yielded = false;
    setState(PARKING);
    try {
        yielded = yieldContinuation();  // may throw
    } finally {
        assert (Thread.currentThread() == this) && (yielded == (state() == RUNNING));
        if (!yielded) {
            assert state() == PARKING;
            setState(RUNNING);
        }
    }

    // park on the carrier thread when pinned
    if (!yielded) {
        parkOnCarrierThread(false, 0);
    }
}

in JDK 21

Parking happens when we use some kind of scheduled process, for example, a queue or certain IO operations. If they cannot run and have to block on native processes as the mentioned synchronised test-case they will change state from PARKING to PINNED:

private void parkOnCarrierThread(boolean timed, long nanos) {
    assert state() == RUNNING;

    VirtualThreadPinnedEvent event;
    try {
        event = new VirtualThreadPinnedEvent();
        event.begin();
    } catch (OutOfMemoryError e) {
        event = null;
    }

    setState(PINNED);
    try {
        if (!parkPermit) {
            if (!timed) {
                U.park(false, 0);
            } else if (nanos > 0) {
                U.park(false, nanos);
            }
        }
    } finally {
        setState(RUNNING);
    }

    // consume parking permit
    setParkPermit(false);

    if (event != null) {
        try {
            event.commit();
        } catch (OutOfMemoryError e) {
            // ignore
        }
    }
}

in JDK 21

I provided an example with test-case saveWordsParking:

@Synchronized
override fun saveWordsParking(words: List<String>): String? {
    try {
        Thread.sleep(100)
    } catch (e: InterruptedException) {
        throw RuntimeException(e)
    }
    ...
}

Parked, however, is quite an odd state, and I wasn’t able to reproduce it. This has to do with this variable notifyJvmtiEvents, which apparently does something about mounting and unmounting using native methods. According to the literature, Parked is a status identifying a thread in a scheduler that is not doing anything and waiting for its turn to be Unparked and taken by the Scheduler. This should be the case with unblocking operations that the JVM can manage, i.e. native independent.

1.4 Kotlin coroutines

As we have seen before, coroutines are very similar to virtual threads. There is actually no major difference between both of them in theory. However, their implementations do differ. But before delving into them as we did before with virtual threads, let’s get familiar with some of the terms of the Kotlin world:

Suspend: Refers to the act of creating a coroutine. A method referred to as suspended, runs only in a coroutine context. This context may be switched to another during execution.
delay: A delay, is kind of like sleep, but it will just pause or suspend the running coroutine for as long as we tell it to
coroutine: Just like Virtual Threads, a coroutine runs on a platform Thread. It can also automatically switch content.

1.4.1 What are Kotlin Coroutines ?

Kotlin, as you now have probably figured out by now, is still nothing more than a simple DSL that enables some new syntax with the goal to make it easy for programmers to build their applications. What this entails is a bit of confusion when first interpreting the code and the bytecode. So, instead of clicking on something like startVirtualThread as in the case of Java with our favorite IDE, in Kotlin’s case, we need to find a way to enter the suspend code. We start by looking at an example of that like this one:

suspend fun readWordFlowBack(words: List<String>) = wordsFlow(words).toList().joinToString(" ")

Depending on your IDE, you’ll find different ways to do the following. In Intellij, there is, fortunately, a Tool that allows us to see the decompiled resulting bytecode in Java:

Once here, we can click on the button Decompile:

And we finally get this kind of code:

Pretty messy right? Well, this is the way that we currently, in 2022, get to decompile Kotlin code into Java code. It’s really not Java code per se, but it gives us a window into how things are truly translated into the JVM. If we want to skip these steps and see exactly how the code gets compiled, then you probably need to go to the command line. Just out of curiosity, if you do go to the command line and list the files in the target directory, you’ll see a lot more files than what you normally see in compiled Java classes:

Note that we have quite a few classes and some with the actual method names. Not very nice to see but Kotlin does this because Kotlin is a layer on top of Java. In other words, it’s a DSL (Domain Specific Language). This means that we will not be getting bytecode classes just like we get from Java code. In the end, you do not need Java code because the bytecode is what’s being generated under the hood at compile time. Another curious fact is that when you use Intellij by default, you don’t really see all of these files. The only thing you see is their Kotlin counterparts in an interpreted way.
Anyway, let’s go back to the decompiled code. Did you notice that we are using a Continuation? We have seen that before in Java correct? Let’s delve into it in the same way we did in Java:

@kotlin.SinceKotlin
public interface Continuation<in T> {
  public abstract val context: kotlin.coroutines.CoroutineContext

  public abstract fun resumeWith(result: kotlin.Result<T>): kotlin.Unit
}

in Kotlin 1.8

We see that a Continuation is an interface, and it has a CoroutineContext and a resumeWith function.
And this is really as far as we seem to be able to go in evaluating coroutines because the whole library is developed with Kotlin source code and that makes it reasonably difficult to see how that gets translated to Java. I guess the point I’m trying to make is that it doesn’t look like Kotlin coroutines are that much different than Java virtual threads at this point. But, on the other hand, just because the source code is written in Kotlin, it does not really mean that we can’t read it. So let’s try that.

@kotlin.PublishedApi
@kotlin.SinceKotlin
internal final expect class SafeContinuation<in T> : kotlin.coroutines.Continuation<T> {
  internal constructor(delegate: kotlin.coroutines.Continuation<T>, initialResult: kotlin.Any?) { /* compiled code */
  }

  @kotlin.PublishedApi
  internal constructor(delegate: kotlin.coroutines.Continuation<T>) { /* compiled code */
  }

  public expect open val context: kotlin.coroutines.CoroutineContext /* compiled code */

  @kotlin.PublishedApi
  internal final expect fun getOrThrow(): kotlin.Any? { /* compiled code */
  }

  public open expect fun resumeWith(result: kotlin.Result<T>): kotlin.Unit { /* compiled code */
  }
}

in KotlinX 1.10.0

SafeContinuation is an implementation of Continuation. The expect is a keyword, used in Kotlin in the same way as native is. In other words, in Kotlin this just means that the implementation is platform-dependent and of course, it’s not easy to access it as well. Further down the line in the coroutines code, it gets quite difficult to understand anything. Whereas in Java I could debug through the whole JDK, in Kotlin, it gets quite difficult, and I’m assuming that this has to do with the fact that suspend is interpreted as a keyword in Intellij and not as ordinary code. Thus, we don’t really get to debug things like Continuation that easily. But hold on! Of course, we can!. With Kotlin, just as much as with Java, we sometimes need to guess where the code is going to fall into. So we take a wild guess by opening the run method in DispatchedTask.kt:

 final override fun run() {
    assert { resumeMode != MODE_UNINITIALIZED } // should have been set before dispatching
    try {
        val delegate = delegate as DispatchedContinuation<T>
        val continuation = delegate.continuation
        withContinuationContext(continuation, delegate.countOrElement) {
            val context = continuation.context
            val state = takeState() // NOTE: Must take state in any case, even if cancelled
            val exception = getExceptionalResult(state)
            /*
             * Check whether continuation was originally resumed with an exception.
             * If so, it dominates cancellation, otherwise the original exception
             * will be silently lost.
             */
            val job = if (exception == null && resumeMode.isCancellableMode) context[Job] else null
            if (job != null && !job.isActive) {
                val cause = job.getCancellationException()
                cancelCompletedResult(state, cause)
                continuation.resumeWithStackTrace(cause)
            } else {
                if (exception != null) {
                    continuation.resumeWithException(exception)
                } else {
                    continuation.resume(getSuccessfulResult(state))
                }
            }
        }
    } catch (e: DispatchException) {
        handleCoroutineException(delegate.context, e.cause)
    } catch (e: Throwable) {
        handleFatalException(e)
    }
}

in KotlinX 1.10.0

If you run my Kotlin example, you’ll see that the code falls in here. This dispatched task is what allows our coroutine to run.
In Kotlin, we can start coroutines in several ways. We can use suspend in a function and get something to call that, we can start a coroutine context with withContext we can implement them using runBlocking, plus many other ways. In our tests example we are using something like this:

GlobalScope.launch {
  withContext(IO) {
    FileOutputStream(File(File(dumpDir, "kotlin"), "$methodName.csv"), true).use { oos ->
      (0..repeats).map {
      startProcessAsync(oos) {
        toTest()
      }
    }.awaitAll()

    }
  }
}.join()

Just remember that GlobalScope is fine to use for experiments, but we should be careful using this in production code, because the coroutine context will be open during the runtime of the whole application.

Intellij can help we figure out where coroutines are starting. In this example we are actually creating 3 coroutines:

suspend creates a coroutine with the context of the caller
GlobalScope.launch, will launch a coroutine in a global context (strongly advised against). Always recommended to use coroutineScope instead.
withContext(IO) will create a coroutine in an IO context.

The keyword suspend, creates a coroutine. We don’t see it in the example. It is associated with the parent function: suspend fun generalTest(). For that, please look for this example in the code. Then we start a new GlobalScope. The GlobalScope will start a coroutine with a global context. And of course, under it, we can start another coroutine with withContext(IO).

public fun CoroutineScope.launch(
    context: CoroutineContext = EmptyCoroutineContext,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> Unit
): Job {
    val newContext = newCoroutineContext(context)
    val coroutine = if (start.isLazy)
        LazyStandaloneCoroutine(newContext, block) else
        StandaloneCoroutine(newContext, active = true)
    coroutine.start(start, coroutine, block)
    return coroutine
}

in KotlinX 1.10.0

A deeper dive into the coroutine implementation at Tasks.kt, shows us know that a coroutine has a mode and a state.

A coroutine can have these modes:

TASK_NON_BLOCKIN 0: The task is CPU bound and will not block.
TASK_PROBABLY_BLOCKING: 1: The task will probably block. This works like a hint and just like we saw in virtual threads, this will let the scheduler know that a system thread might be needed. The states available for a Kotlin coroutine worker in CoroutineScheduler.kt are:
CPU_ACQUIRED: It acquires a CPU token and with it tries to execute a task in a non-blocking way.
BLOCKING: The task is blocking and the only mode that allows this is TASK_PROBABLY_BLOCKING.
PARKING: It parks a thread, and pretty much like we saw before, parking happens when the thread cannot be temporarily executed.
DORMANT: It stays dormant until it can execute another task. This is different than PARKING because PARKING means that the worker is already responsible for a task.
TERMINATED: This is the last state of the worker Finally, coroutines have these states in DispatchedCoroutine.kt:
RESUMED 2: Only possible to set when the coroutine is still UNDECIDED. The coroutine is proceeding with the execution
SUSPENDED 1: Only possible to set when a coroutine is still UNDECIDED. The coroutine is suspended.
UNDECIDED 0: The initial status of a coroutine (also described as _decision in the source code)

These are the familiar states when we launch a coroutine. During design time, we aren’t really concerned about how the Worker does its thing, and we are definitely not concerned with the modes. However, it can be incredibly helpful to know these basic concepts about coroutines or at least be aware that they exist.
As a recap, a coroutine can start with a suspend function, withContext, or launch. withContext and launch do not work outside a coroutine context. If you need to create such context, then you need to use something like runBlockingor a suspend function.
Similarities between Virtual Threads and coroutines
Now that we have examined the code, let’s try to make more sense of it by diving into the theory. The theory about coroutines and java virtual threads is pretty much found anywhere on the internet and the repo where I performed the tests contains many links to information about it. Perhaps what we need to know at this point in its very basics, about the two implementations is that:
1. Both are based on the original coroutine principle invented in 1958. This is indeed no new concept
2. Both are based on the idea that you can suspend one function runtime to give way to another function runtime.
3. Both implement ideas of suspend and waiting on the main thread using concepts like pinning, dormant, and parking.
4. Both are managed by the JVM and not by the system
5. Both avoid the creation of a whole new platform thread and take advantage of already running ones. They have been started in a Thread pool. ForkJoinPool for Java Virtual Threads and CoroutineScheduler for Kotlin Coroutines.
6. Although we can only have as many platform threads as our CPU cores, we can launch different processes, with a level of parallelization up to the number of cores we have, and launch as many processes we want at the same time until the limits that our machine can handle. The illusion that we perform more in parallel is created by not allowing system threads to block whenever that is possible.
7. Both do not technically sleep. At the very least they do not sleep in a blocking state. In Java, this is done seamlessly with Thread.sleep and it uses non-blocking techniques by giving the thread a PARKING status and giving it a permit. Parking means, in other words sleeping, and unparking means waking up. In Kotlin, the delay ensures that the current execution gets scheduled to execute later. But a deep dive lets us see that Parking and Unparking are also part of the implementation.
8. Both have different ways to do PINNING. In Java, Pinning is done to hold a thread tight to its carrier thread. This happens in synchronized methods. In Kotlin coroutines, the execution is PINNED to one single CPU thread. Suspend and resume operations will make sure the coroutine will run on that same thread until the end. In the same way, Kotlin has synchronized methods and of course, they also use PINNING
9. In both cases, a Thread is a thin wrapper around native threads.

2. Java Virtual Threads Test Implementation

In order to perform these test sets, I created a small framework that allows me to measure the running time of different methods with different complexities in time and space. The idea is to give enough variation to different kinds of progressions and see how that all plays out when deploying several java virtual threads at the same time. For these tests, I’m not interested in measuring the individual time it takes for one particular java virtual thread to execute. Instead, I want to measure the whole and see how it all plays out. The code for the performance measurements contains also reports code, file management code, and CSV file generation algorithms to help determine how many java virtual threads were allowed to deploy at one single point in time. Let’s have a look at the method that receives a lambda as a parameter including other arguments in order to execute, perform and measure the duration of each individual test:

 private <T> void performTest(
        String testName,
        String methodName,
        String timeComplexity,
        String spaceComplexity,
        Supplier<T> sampleTest,
        Runnable toTest,
        int repeats) {
    try (var oos = new FileOutputStream(
            new File(
                    new File(dumpDir, "java"),
                    String.format("%s.csv", methodName)),
            true)) {
        log.info("===> Starting test: {}: {} <===", testName, sampleTest.get());
        log.info("***> Processing took {} milliseconds", measureTimeMillis(() -> {
            final List<Thread> threadStream = range(0, repeats).mapToObj(i ->
                    startProcessAsync(toTest, oos)).toList();
            log.info("---> Just sent {} threads", repeats);
            threadStream
                    .forEach(thread -> {
                        try {
                            thread.join();
                        } catch (InterruptedException e) {
                            throw new RuntimeException(e);
                        }
                    });
        }, testName, methodName, timeComplexity, spaceComplexity, repeats));
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

So what I’ve created here is just a method inspired by a few things I’ve learned with Kotlin. Let’s look at them individually:

testName is just a name of a method
methodName is a parameter that lets us know what method are we testing. In Kotlin, we’ll see later that we can easily get method names via reflection without much hassle. In Java though, I still had to hardcode the method name and use it as an input parameter as a quick win-win solution.
timeComplexity is literally a String where you can put whatever you want but is meant to be used to express the big O notation for the method being tested. This is important in order to see if method complexity would play any role whatsoever in the performance
spaceComplexity is also literally a String but in this case, is used for Space complexity
sampleTest is just a supplier so that we see a fragment of an output of a single test in the logs
toTest is the actual test to be run
repeats is how many times it will run

Just for clarity, the timeComplexity and the spaceComplexity should be tested in a progressive fashion going from small input to a slowly increasing input. The progression will be available in the future on my website http://joaofilipesabinoesperancinha.nl sometime in the future. Progression tests are a bit difficult to run because of the limitations of a personal computer and so these two factors do not play a significant role in the results of this article. The individual implementation of each method should be easy to read in the project I’ve created for this article.
startProcessAsync is where the startVirtualThread method is called:

    private Thread startProcessAsync(Runnable runnable, FileOutputStream oos) {
    final Runnable threadRunnable = () -> {
    var start = LocalDateTime.now();
    runnable.run();
    var end = LocalDateTime.now();
    try {
        oos.write(
            String.format("%s,%s,%s\n", start, end, Thread.currentThread()).getBytes(StandardCharsets.UTF_8)
        );
        oos.flush();
    } catch (IOException e) {
        throw new RuntimeException (e);
    }


};
    return startVirtualThread(threadRunnable);
}

3. Diving into Coroutines

Coroutines have a slightly more extensive paradigm than Java Virtual Threads. This is because it provides us with different options on how to start them. Java Virtual Threads have this also, but Kotlin, goes a few steps beyond this, by changing its own syntax to accommodate these changes. Its complexity, however, makes it quite probably less approachable for many developers. To me, it makes it very interesting, but maybe to the average developer, it might be a step too far. In a short sentence, Kotlin coroutines allow us to start an execution asynchronously and wait for the return object, the same thing and not wait for the return object, suspend the current coroutine and execute another one instead, on a different or the same context, it has 4 different common abstractions for running context, it allows you to "sleep" under the name delay, which is scheduling of a sleeping action in the end, and it allows us to create special IO specific contexts with enabled coroutine capabilities. These are the basics of what we are going to look at in this section. For now, let’s have look at the following:

private fun runDelayExample() = runBlocking {
    println(
        "3 - This is the parent coroutine, it will not be suspended by launch ==> (${Thread.currentThread().name}) - ${
            Thread.currentThread().threadId()
        }"
    )
    launch {
        delay(2000)
        println(
            "1 - (launch) - This launches a coroutine in parallel ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
        println(
            "1 - (launch) - The coroutine should remain pinned to the original thread up until the end ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
    }
    val deferred = async {
        println(
            "4 - (async) - This coroutine is asynchronous and therefore it's thread has to be another ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
    }
    println(
        "3 - The parent coroutine will get suspended with a withContext ==> (${Thread.currentThread().name}) - ${
            Thread.currentThread().threadId()
        }"
    )
    withContext(IO) {
        println(
            "2 - (IO) This couroutine has suspended the caller coroutine and now it will be parked or scheduled to run later ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
        delay(500)
        println(
            "2 - (IO) Although the coroutine has been parked, it is now unparked and it remains on the same thread ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
    }
    withContext(Unconfined) {
        println(
            "6 - (Unconfined) This coroutine has suspended the caller coroutine and now it will be parked or scheduled to run later ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
        delay(500)
        println(
            "6 - (Unconfined) Although the coroutine has been parked, it is now unparked and it remains on the same thread ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
    }
    withContext(Default) {
        println(
            "7 - (Default) This couroutine has suspended the caller coroutine and now it will be parked or scheduled to run later ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
        delay(500)
        println(
            "7 - (Default) Although the coroutine has been parked, it is now unparked and it remains on the same thread ==> (${Thread.currentThread().name}) - ${
                Thread.currentThread().threadId()
            }"
        )
    }
    println(
        "3 - This coroutine has been suspended but now running and still on the same thread ==> (${Thread.currentThread().name}) - ${
            Thread.currentThread().threadId()
        }"
    )
    deferred.await()
    delay(2000)
}

You’ll find in many tutorials, that people use thread-like squiggles to represent the way coroutines work. I used to do that before but in my own opinion that can be a bit misleading. Or you could argue that is just an introductory representation for the initiates. However, coroutines do not work so much like Threads, although you may have that impression at some points in the code. At this point, if you read all of the above you probably already understand why am I saying this. And if you run the above code located in class CoroutinesShortExplained.kt, you’ll see that much of this code runs on thread main. So you may be asking yourself, why is it that in a single thread we can wait 2 seconds and then 2 seconds, and then the whole thing takes exactly 2 seconds to execute? Well that’s because unlike Thread.sleep (for Java), the delay operation schedules the current coroutine to execute later and parks it. This releases the main thread to continue execution. When the 2 seconds are passed, the coroutine gets unparked and it starts again. With async, we do the same as launch, but in this case, we return whatever the receiver returns. In this case, is just a Unit because it returns nothing. Finally, we encounter withContext which will have the effect of adding 500 ms to the whole waiting time of this function. The reason being is that withContext performs context switching. It suspends the calling coroutine and runs its execution, returning back to the caller at the end of it. This happens regardless of the System Thread running it. This is why when we run the whole code, we get approximately 3500 ms in runtime:

So these are the basics, but it is also important to have an idea of what the 4 different contexts do and remember that for Android there are a lot more and that we can make custom ones:

IO: This context manages coroutines during blocking operations in the same way as Java Virtual Threads do during PINNING. You can see this in execution results number 2. It is purposely made to be used during IO operations, in order to allow, when possible, IO operations to be executed in a non-blocking way.
Default: It uses at least 2 cores in order to work and by default uses a pool of threads containing as many threads as the available cores. You can see this in execution results number 7. It will use a different Thread from the available JVM pool of threads, if possible. Otherwise, it will use the first one.
Unconfined: It means that a dispatcher will not necessarily continue to execute on the same thread. You can see this in execution results number 6. Its criteria are to use the first available thread, making it quite fast. A subtle difference between this one and the Default, is that Default chooses the first different thread if possible whereas Unconfined allows the dispatcher to pick any first available one of them.
Main: This one is platform-dependent and it does not have to exist. It is sometimes referred to as an Android-specific context, but in reality, it’s just referring to the implementation of whatever the platform where you are running this defines it to be.

In project Loom, Thread.sleep, cannot necessarily be considered a blocking operation anymore. Not strictly at least. However, when running Kotlin Coroutines, the executing thread is not considered to be a virtual thread. It is instead, a Worker provided by the Kotlin coroutines core library. Worker is an implementation of the Thread interface, and so Worker is a coroutine that is also a Thread, but because it is not of the type of VirtualThread, it will not be scheduled to sleep and instead still block the whole execution:

static Thread newVirtualThread(Executor scheduler,
                               String name,
                               int characteristics,
                               Runnable task) {
    if (ContinuationSupport.isSupported()) {
        return new VirtualThread(scheduler, name, characteristics, task);
    } else {
        if (scheduler != null)
            throw new UnsupportedOperationException();
        return new BoundVirtualThread(name, characteristics, task);
    }
}

in JDK21

3. Coroutines Test Implementation

The implementation of the coroutines test function is quite similar to its java method counterpart, but it is important that we have a quick look at it:

    private suspend fun <T> performTest(
    testName: String,
    methodName: String,
    timeComplexity: String = "n/a",
    spaceComplexity: String = "n/a",
    sampleTest: suspend () -> T,
    toTest: suspend () -> T,
    repeats: Int
) {
    log.info("===> {} : {}", testName, sampleTest())
    log.info(
        "***> Processing took ${
            measureTimeMillisSave(
                testName,
                methodName,
                timeComplexity,
                spaceComplexity,
                repeats = repeats
            ) {
                withContext(IO) {
                    FileOutputStream(
                        File(
                            File(
                                dumpDir,
                                "kotlin"
                            ),
                            "$methodName.csv"
                        ),
                        true
                    ).use { oos ->
                        (0..repeats).map {
                            startProcessAsync(oos) {
                                toTest()
                            }
                        }.awaitAll()
                    }
                }
                log.info("Just sent {} threads", repeats)
            }
        } milliseconds"
    )
}

Although this bit seems to be the same, there is a small difference. Since we want to save data to a file, and we want those all to be non-blocking, then we start the whole process with a coroutine under the IO context. Once we achieve that, we can then start out the method to be tested under an async context:

    private fun <T> CoroutineScope.startProcessAsync(
    oos: FileOutputStream?,
    function: suspend () -> T
) =
    async {
        val start = LocalDateTime.now()
        function()
        val end = LocalDateTime.now()
        oos?.let {
            oos.write(
                "$start,$end,${currentThread()}\n".toByteArray(StandardCharsets.UTF_8)
            )
            oos.flush()
        }

    }

3.1. Before testing

One thing that has made this article difficult to write is to clearly explain the goal here. Am I trying to measure how Virtual Threads perform in relation to Coroutines and vice versa? Absolutely! Are Virtual Threads and Coroutines made to answer performance issues? The short answer is a big massive maybe...! The long answer is complicated. The problem Continuations are solving is the shortage of resources we have. By making the JVM handle concurrency we can now write code in a structured concurrency way, we are allowed to trigger several processes at the same time, and we can encapsulate them.
Explaining why both Java Virtual Threads and Kotlin Coroutines allow us to program in a structured concurrency kind of way would be in itself a whole new article that is really off-topic, but I think that if we just use our common sense in the short definition we can immediately see why this is so:

Structured concurrency means that lifetimes of concurrent functions are cleanly nested

We trigger them, but we don’t necessarily start to run them. Platform threads are very expensive processes that take up space, and start-up time, and they are limited to the number of cores of your machine. What this means in practice and as of result of any implementation of Continuations, is that suddenly we have so many resources that there are already talks about if concurrent and asynchronous programming is now even worth the effort. What my tests are doing is allowing me to exhaust the resources up to a point where the implementation on both sides of this discussion gets challenged. Thats’ where performance tests come in. Managing Continuations when resources are exhausted needs to be done in an intelligent way and this is why I’m stressing out these two implementations. I could find that Coroutines are much better than Virtual Threads or I could find that Virtual Threads are way better. Or maybe I will find exactly no difference which could actually be the case since we have seen that there does not seem to be any major difference between the two implementations.
There is of course a lot of code built in order to make it possible to generate such tests. If you run make clean build-run on the root of the application, you’ll see that a dump directory will be generated. Inside you’ll find two directories java and kotlin . This is where the results of our tests will go in. There are two types of files generated on each of them. There is a readable mardown file and several quite unreadable csv files. These csv files are created in pairs. One file contains the method name and the other contains the method name but ends in -ms. The first two columns of the first file, contain the start and end timestamps per virtual-thread/coroutine. The third column contains the name of the running thread that carried that process.
Finally, on the root, another markdown file is generated with a short comparison report about the different methods implemented the same way, as much as possible in Java and Kotlin. This file is called Log.md.
But we still have to look at another visualization behind the theory of both technologies. The idea is that you can execute something else while you suspend the previous execution. Virtual threads work a bit like this and this is just an oversimplified representation:

Coroutines give in practice the same kind of structure and again just another over simplified example:

The only thing that is happening in both cases regardless of how they are implemented at a low level is a switch between available threads. In a concurrent environment with just the use of just of platform Threads, making a blocking call always means waiting for the blocking call to finish before being allowed to continue. Coroutines or Continuations explore threads to the maximum by making sure that we avoid anything to block whenever that is possible. If we are waiting for a blocking call, then we’ll get back to that coroutine when we are done, but in the meantime we just let another coroutine move around in another thread or even on the same thread. This is what allows us now to implement in a structured concurrency way, which is something we still need to explicitly do in the code if we want to.
They may be different on a low level, but what I see is that on a high level, both Kotlin coroutines and Java Virtual Threads (also known in the old days as fibers) are exactly the same thing.
To make this article a bit more interesting I’ve made the data source where all of these algorithms will run against to, to be a small developing novel. The longer it gets, the harder will the two different implementations have to work. It’s all available in the GoodStory.md file located in the project repository.

4. Test results

As I mentioned before, the best way to run these tests is via the command line, but you can also run them via IntelliJ.
If you run them via Intellij you’ll need to run at least two main classes. One for Java and the other one for Kotlin. These are respectively GoodStoryJava.java and GoodStoryKotlin.kt. We’ll need to run them with these parameters:

-f docs/good.story/GoodStory.md -lf Log.md  -dump dump

And for Java specifically, we’ll have to enable JDK19 features:

--enable-preview

If you have VisualVM please have it running at the same time. I was able to grab these snapshots just before VisualVM crashed:

Java Virtual Threads capture

And I was able to capture this for the Kotlin coroutines project in the same way:

There are a few differences, but that’s just a name difference. Between the two captures, we get ForkJoinPool-1-worker-N for Java Virtual threads and DefaultDispatcher-worker-N for Kotlin coroutines. These workers are responsible to coordinate coroutines, coroutine context, context switching, and assigning a coroutine to a system thread. The Java ForkJoinPool starts setting a maximum of 256 workers. The CoroutineScheduler starts with a maximum setting of 2097150 workers.
I’ve created some CSV files to get an idea of how many Virtual Threads or Coroutines are executing at any given time. These are not accurate and the reason why is that they are assuming that these two kinds of processes run continually and never switch context in these runs, per continuation. However, we now know that this isn’t necessarily true all the time. Anyway it’s worth the effort to look into them. If we look at one of the heaviest processes we ran in these two projects. For example, let’s check what happens with the method/function: repetitionCount. This method checks how many words are repeated more than once. This means that if we find two words "dog" then that is 1 repetition. For every other "dog" found we add one more to that count. If we look at the count generation for Java we find that the number of active Virtual Threads at any given time was 12:

For Kotlin we find something off. We see that the number of active coroutines at any given time rose up to 63:

How does this happen? Well, for Java Virtual Threads, it makes perfect sense that only 12 are active at one given time. For Kotlin Coroutines it’s just strange. In this case, It’s not really clear to me what happened, but I’m guessing that this 63 number is just a misleading result because should a coroutine change context in the middle of a run, or if for whatever reason it gets suspended, then, of course, the start and end timestamps will encompass a longer delta than usual and that result will not be applicable for the initial assumption that the asynchronous processes that we have started have been run continuously without being suspended once started. We should tell have gotten 12 or less than that because that’s how many cores my machine has. Not 63! I can only wish at this point.
Finally, let’s have a look at the general results where we can compare different runs of 10000 repetitions for each implemented algorithm:

Time	Method	Time Complexity	Space Complexity	Repetitions	Java Duration	Kotlin Duration	Kotlin Loom Duration
2023-01-16T20:25:14.937724788	wait0Nanos - Wait 0 Nanos - Running - Yielding - Virtual Thread	n/a	n/a	2	10	-1	-1
2023-01-16T20:25:14.944936772	wait100Mills - Wait 100 Mills - Running - Parking - Yield - Virtual Thread	n/a	n/a	2	103	-1	-1
2023-01-16T20:25:14.944998573	saveWordsNio - Write to 1 file - Yield - Virtual Thread	n/a	n/a	2	130	-1	-1
2023-01-16T20:25:14.945033813	saveWordsNio - Write to 1 file - Pinning - Yield - Virtual Thread	n/a	n/a	2	233	-1	-1
2023-01-16T20:25:14.945061943	findAllUniqueWords - All Unique Words	n/a	n/a	10000	1674	3911	1654
2023-01-16T20:25:14.945090147	findAllUniqueWordsWithCount - All Words with count	n/a	n/a	10000	1194	1268	1068
2023-01-16T20:25:14.945118644	revertText - Reverted Text	O(n)	O(1)	10000	168	844	360
2023-01-16T20:25:14.945157538	contentSplitIterateSubtractAndSum - Double iteration of an array of words	O(n^2)	O(1)	10000	728	2987	1848
2023-01-16T20:25:14.945200581	repetitionCount - Repetition count	O(n^2)	O(n)	10000	3180	2470	1012
2023-01-16T20:25:14.945248030	createAvlTree - Create AVL Tree	O(log n)	O(n)	10000	302	1577	725
2023-01-16T20:25:14.945295143	findPrimeSecret - Secret word in Sieve of Eratosthenes	O(n * log(log n))	O(n)	10000	699	2593	470
2023-01-16T20:25:14.945338146	createSplayTree - Create Splay Tree	O(log n)	O(n)	10000	155	530	248
2023-01-16T20:25:14.945382500	quickSort - Quick sort	O(n * log n)	O(log n)	10000	1770	4075	3993
2023-01-16T20:25:14.945426448	makeTextFromWordFlow - Make text from word Flow	n/a	n/a	10000	693	614	361
2023-01-16T20:25:14.945485517	createIntersectionWordList - Intersection Text Algorithm	O(n)	O(n)	10000	85	299	181
2023-01-16T20:25:14.945525274	controlTest - N/A	n/a	n/a	10000	517	463	43
2023-01-16T20:25:14.945594012	generalTest - N/A	n/a	n/a	10000	112	46	58
2023-01-16T20:25:14.945640460	findAllUniqueWords - wait0Nanos	n/a	n/a	2	-1	16	18

Looking at the table we see that in almost all cases, the duration of throwing in ten thousand virtual threads or coroutines in methods/functions with approximately the same complexity isn’t really that different. In fact, zooming in more closely almost gives us the idea that project Loom seems better in terms of performance. Anyway, it is not enough to draw conclusions. At this point, I’ve exhausted the limits of my local machine and it has worked enough in these tests. There are indications all throughout my tests that Project Loom’s Virtual Threads do seem to perform better than Coroutines, but, as I mentioned before, it is not a definite conclusion. It is just a correlation, an idea if you will. I still wasn’t able to definitely prove that one is better than the other. What I was able to prove is that in my current local environment, there is nothing, absolutely nothing that makes me doubt any of these approaches to solve this same problem. Both of them seem good at the same level, and that slight indication that Java Virtual Threads do better is still just an indication. The other reason why this is just an indication is that I have been able to run these same tests, on other occasions, where all the coroutines implementations did better than Java Virtual Threads. It’s just the frequency that mostly seems to favor Java Virtual Threads, but this isn’t material to draw any conclusions. And maybe, not being able to draw any conclusions is in itself a conclusion already, but I let you decide that.
Conclusion
When I compare both implementations of this same idea of Continuations, I didn’t really see in practice any major difference. I find both Kotlin coroutines and Java Virtual Threads great technologies alike. When exhausting the system with coroutines and forcing all sorts of algorithms to come to action to optimize that, I didn’t see any major difference in performance.
Here is the thing. Kotlin is here to stay and Java is also here to stay. My point with the article was to lead both parties to this discussion to make a good look at what both languages have. Kotlin is an invention in 2010 and Java exists since 1995. In the same way, Scala was created, Kotlin was also created to "provide features not available before". Well, this is a tough pill for me to swallow. Do you know why? Because everything that is available in Kotlin and that we say was "needed" in Kotlin I always find it to be available in Java too! Just under a different style. This ranges from what we nowadays call idiomatic Kotlin to what we call nowadays idiomatic Java.
Since Java 8 we have lambdas which actually was the first time Java back in 2014 began to have concerns about the lack of better solutions. Lambdas do exactly the same thing as for, while and do {} while, in the same way, receivers in Kotlin do. They make everything slow as hell! You only realize this when implementing algorithms for High Availability applications or by making exercises in hacker sites concerned about the big O notation. This may be an exaggeration, but hey, I also love the elegancy that both bring and so I also use them massively, to be honest, but my point is that they are not everything. When we invest in sequences, lambdas, receivers, and map-reduce operations, we are in a way penalizing performance. Does it matter? It only matters when it matters, so my best advice is just to be an expert on them. We all truly love Lambdas and Receivers, but just don’t let them be a point of anger in your daily coder’s life, because sometimes, the good old for’s can make a real difference.
If we talk, for example, about extension functions being better than static methods in Java, that’s also not a great standpoint. When I see these discussions or when I get dragged into them, what I usually observe is that one side is extremely passionate about its language of choice, but what truly is happening, in my view, is just people defending their personal preferences. Me, I prefer to be objective and I can’t see anything objectively concerning about any of these languages. They are just different. And it’s great so!

Java is in many ways the parent of both Scala and Kotlin. I think it is kind of senseless to want or wish Kotlin to take over Java. I personally think that all languages should exist and that we should learn from all of them because the very principle that they are different but end up doing the same is exactly the same principle that keeps us active and makes us understand different perspectives on code. I don’t want Java, Kotlin, or Scala to go away. I want all of them and the other languages to evolve as well. And I want to learn from all of them. Hey, remember that I’ve started programming with tapes on a ZX-Spectrum 48K machine with rubber keys? That was during the end of the 80’s decade for me. That probably has no relevance in the world today but having that reference does allow me to understand better where we are, which problems we faced in the past, and the present, and which problems we may find in the future. The enrichment that more languages bring to the world is frequently overlooked.

I could go on forever, but what I really want to say with all of this article is plain and simple. Kotlin is a new player in town and so is the coroutines' implementation. And we all love them. But no matter what, I fail to see the engineering added value of these technologies in relation to Java Virtual Threads. I think Kotlin is just different and that adds a new flavor to the JVM. However, every single critic I became aware about Kotlin, it turns out I can see the same for Java. In the same way, for every single compliment about Kotlin, I can find exactly the same in Java. It just seems to have a different style. Of course, many things aren’t integrated into the Java SDK, but Kotlin is still just a DSL on top of the JVM . This means that if I use something like Lombok in Java I’d probably be having the same right? It’s just another DSL, just like Kotlin. Well, many of you reading this would go up in arms saying that Lombok is "a terrible idea", and then I would say "but we have record’s in Java now!" and then you’d say "Yeah but data classes do all of that together, and you can make everything immutable, and it looks so much better!". That’s all amazing and I agree with that last statement. Kotlin does look better. Or does it? Maybe I prefer using annotations, maybe I prefer using @Builder instead of data class , maybe I want to be reminded that behind on single data keyword I get a hash implementation, an equals, getters and setters, and if I use val it on all of my properties then I get an immutable object! This is where I think Kotlin is a genius language. It still makes it unclear to me as to what engineering benefit it adds to the code, and yet, by riding on our instincts and current trends it has found a golden opportunity to fill out a perceived gap that many developers and engineers face up to these days. Boilerplate, repeated code, difficult code, engineering costs, etc., etc. Plus it provides an amazing style of programming when it comes to ensuring structured concurrency. And of course our desire to do something stimulating and new. New syntax and new semantics create a whole new playground and that is just a positive thing.

Neither Kotlin nor Java are, in my view better than one another in a strict engineering sense. You may disagree of course. And I think if you come from an Android background, then you’ll have much more to say here than I could possibly say. I am very aware that Kotlin has been massively embraced by Android developers. Sounds good to me. My opinion (or lack thereof) comes from a services implementation-only perspective. Android does have a lot more to it, so I have to abstain from commenting on that one. For now, that is.
If you have to pick a new technology, my advice is, just pick the one you like best. I seriously doubt you’ll find any performance benefit from the language itself. Be in line with your team as well. If they have a passion for Kotlin then go for it. If they have a passion for Java then go for it. It’s in passion that you’ll find the most productivity. If you want to go for something efficient, and that is your only concern, then, and there is a very wide consensus on this, you may want to stay away from anything JVM related in the first place. It can be difficult to get things up and running in the JVM and this is why many are turning into Native solutions. What I also want to point out is that coroutines are sometimes discussed in the context of multithreading and providing more threads. That is just not the case. The paradigm around coroutines is essentially more related to reactive programming than with anything else. The reason why I say this is that coroutines make much more efficient use of System/Platform Threads. However, this may sound to have to do with multithreading, it is just not. This is just a way to avoid threads pausing for no good reason as they used to if you will. Whether you decide to use Kotlin coroutines or the upcoming Virtual Threads in JDK21 under Project Loom it is entirely up to you.

The idea that Java has to defend itself against Kotlin or that Kotlin possibly represents a contestant to the ruling position of Java was my initial motivation to write this article and this is because, just like the story of Lucy will one day show, sometimes we just tell very good stories to each other, but they end up meaning nothing. I will personally keep programming in whatever language I feel like when I wake up to it. At work, I stick with the plan. In my spare time though, I just choose whatever I feel like to at that time and that includes Java, Kotlin, Scala, Go, Rust, Python, Ruby, PHP, Javascript, etc.
As I mentioned in the introduction, this article will be subject to more frequent reviews given its experimental nature.
I have placed all the source code of this application in GitLab
I hope that you have enjoyed this article as much as I enjoyed writing it.
Thank you for reading!

5. Resources

Kotlin Native and GraalVM - The Story So Far

João Esperancinha — Fri, 27 Dec 2024 10:21:48 +0000

1. Introduction

There is a lot of buzz today about a technology that we tend to call Native. Native technology or Native code usually refers to code programmed in C that can be run in one particular system directly without a Virtual Machine after compiling. For example, all JVM language code (i.e. Java, Kotlin, Scala, Groovy) needs a Virtual Machine to run on. We first compile our source files to byte codes that can be read by the VM, and then we start the VM with our bytecodes. Javascript code also needs a virtual machine to run on, and many other languages also need so. However, when we compile any code natively, even C code, what it actually means is that the code is compiled in such a way that we can just call these resulting executable files from the command line without resourcing to anything else. We don’t need NodeJS to run them, we don’t need the latest JRE to run them, and we don’t need any SBT, Gradle, or Python tool to run these. We simply run the executables. Of course, a catch to this is that we can’t run these files in any system. We can only run these files in the systems that understand them, and we define this at compile time. This is why we can’t run Windows files in Linux and vice versa without anything to support them. So, for example, if we want to run any Windows executable file in MAC-OS we will need something like Wine.
If you are from a few years back, and I mean the diskette times, you are probably familiar with the concept of EXE, COM, and BAT files. Things like AUTOEXEC.BAT should come to mind. For those of you who recognize these things, I can already give you this as an example, and you probably already know this, none of this works on a MAC-OS or LINUX. These files were specific to the old MS-DOS systems and a few old Windows versions. In any case, the point here is that we now know that when we talk about Native, we are referring to a way to run our code independently. Our compiled code will work natively. It will use low-level resources and cut-through abstractions as much as possible in order to execute commands on a very low level and preferably commands that need no pre-installation.

1.1. Goals

We are going to do some experiments with Kotlin Native and other variants and forms of compiling native code. For this, we are going to need data, and that data is provided by chapter 2 of my novel on GitHub

2. Requirements

For this article, the repository that I am providing was created considering only Linux systems running containers with Linux installed. I didn’t perform tests initially for MAC-OS. I intend to provide full support to MAC-OS machines and maybe Windows further down the line. If you really need to have my repository support also Windows and MAC-OS please open issues to my repo or any of my other repositories. I will then feel compelled to help you with that and will just increase my motivation to do it. And because of this, I really would appreciate it if you could open your issues here.
What you’ll also need to consider is the capacity of your machine. I have made several tests and my machine got quite blocked during compilation times until I finally raised the file listeners number:

echo fs.inotify.max_user_watches=33554432 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
cat /proc/sys/fs/inotify/max_user_watches

The inotify watches number 3354432 is what ended up working for me, but please try more if your machine can. You can find more info about this in my all things cloud repo.
If you intend to run all the examples that I’ll be describing in the following, please bear in mind that compilation in the native world also means that the bridge from strongly typed languages like the one we are using, Kotlin, is a very slow one and so you have to expect extraordinary slow compilation times. It has nothing to do with your machine, hardware, or any operating system you may be using, and it can be quite frustrating to get compilations going. As a precaution, please do not use your IDE before performing the installation steps I indicate below.

3. Kotlin Native

With pure Kotlin native, we are only going to create some fun things with the command line. Kotlin Native provides wrappers around several command line instructions, which we can use quite easily. We will effectively use C underwater and make it run with GCC.

3.1. What is Kotlin Native and how it works?

Kotlin native is still very limited in what it can do, and it also has a very different way of running and working. Because of this, we need to first dive into what working with Kotlin native actually looks like from the basics and then build up our knowledge from there.
When we first create a Kotlin Native initial solution with the initial IntelliJ layout we may be led to think that we are going to work with a normal environment where we can just add libraries and configure Gradle to work. Instead, we are going to work with libraries that are available for Native code generation. So before we do anything we start for example a Native Multiplatform Application project:

When I did this, I realized that the resulting project is a Gradle project. For example, let’s have a look at the good-feel project first. This project is simply a console-based project which simply outputs a random positive message to the screen to give you a good feel for the rest of your day. If we look at the basic plugins that are being used, there is just one called multiplatform:

plugins {
    kotlin("multiplatform") version "1.7.22"
}

Once this plugin is in place, Intellij also configures the Kotlin Native compilation:

plugins {
    kotlin("multiplatform") version "1.7.22"
}
group = "org.jesperancinha"
version = "1.0-SNAPSHOT"
repositories {
    mavenCentral()
}
kotlin {
    val hostOs = System.getProperty("os.name")
    val isMingwX64 = hostOs.startsWith("Windows")
    val nativeTarget = when {
        hostOs == "Mac OS X" -> macosX64("native")
        hostOs == "Linux" -> linuxX64("native")
        isMingwX64 -> mingwX64("native")
        else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
    }
    nativeTarget.apply {
        binaries {
            executable {
                entryPoint = "main"
            }
        }
    }
    sourceSets {
        val nativeMain by getting
        val nativeTest by getting
    }
}

What this means is that the compiler will figure out on which machine you are running it and then compile your code to native if that’s possible. This code comprises possible compatibility with Windows, Mac OS X, and Linux. As I mentioned before, in this article we’ll only have a look at implementing Native code in a Linux ecosystem.
Let’s have a look at the actual code. It is super simple and I made it so to provide some sort of leverage to understand how native code works:

fun main() {
    println("*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*")
    println(randomMessage())
    println("*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*")
}
fun randomMessage(): String {
   return listOf(
       "Good Morning!",
       "You are looking great today!",
       "What a great day today!",
       "Good job!",
       "I really appreciate what you just did! Thank you!",
       "Thanks for bringing me coffee!",
       "You are the best pal ever!",
       "I love working with you!",
       "Rise and shine!"
   ).random()
}

So in this module, we are only printing out a few strings, which are two separators that have a good feel message in between. If we click through the sources we quickly realize that not much has changed really. This is just standard Kotlin library code. And if we compile the code in that folder with the command make b we’ll end up getting an executable file in build/bin/native/releaseExecutable/plus.kexe . We can call this file manually via this precise link or by just running the script make run . You’ll be getting something like this in the output:

*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*
Good job!
*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*

The idea of this project is simply to illustrate how easy it is to start a project in Kotlin native. We’ll also see how difficult it can get. And this is why I created project plus, which essentially is just an extension of project good-feel. When clicking through, especially the Map class and especially where the source is, we quickly realize that we are not really working with normal jar libraries. In fact, we are not working with Java libraries at all. What we are working with is .knm files. These are natively compiled libraries that IntelliJ can partially interpret and provide easy interface access to them. This all happens during design time which means that IntelliSense is working for you:

We are of course still using a standard library, but not the one we are used to using. The standard library in this case is code that runs natively. But first, let’s have a look at what are we using in the new code:

fun main() {
    println("*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*")
    println(randomMessage())
    println("*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*")
    val groupBy: Map<String, List<Int>> = allMessages().groupBy({ it }, { it.length })
    val hashMap = allMessages().groupBy({ it }, { it.length }).entries.fold(HashMap<String, List<Int>>()) { a, b ->
        a[b.key] = b.value
        a
    }
    println(groupBy::class.qualifiedName)
    println(hashMap::class.qualifiedName)
    val toTypedArray: Map<String, String> = allMessages().toTypedArray().associateBy { it }
    println(toTypedArray::class.qualifiedName)
}
fun randomMessage(): String {
    return allMessages().random()
}
private fun allMessages() = listOf(
    "Good Morning!",
    "You are looking great today!",
    "What a great day today!",
    "Good job!",
    "I really appreciate what you just did! Thank you!",
    "Thanks for bringing me coffee!",
    "You are the best pal ever!",
    "I love working with you!",
    "Rise and shine!"
)

In this case, I’m just creating some maps, then I use the entities to recreate them, and essentially I’m just creating maps. What makes super simple code interesting is that it looks and feels like Kotlin. It is in many ways Kotlin, but this code will be compiled ahead of time. We call the compiler in this case an AOT (Ahead Of Time Compiler). What this means is that the resulting code is not just an executable. When we run it, it will start with a performance peak because there will be no just-in-time compiling process. In the JVM when we make what we call an executable jar, the compilation process didn’t stop there. It’s just enough so that we can start the jar in the JVM. The complete compilation will happen while processes are being optimized during the startup of the application. This is normally what we refer to when we talk about JIT. Bear this in mind as we go further because we’ll talk about these two concepts throughout the rest of this article.
If we for example analyze one of the generated knm files, we’ll see something like the following:

// IntelliJ API Decompiler stub source generated from a class file
// Implementation of methods is not available

public fun main(): kotlin.Unit { /* compiled code */ }

public fun randomMessage(): kotlin.String { /* compiled code */ }

private fun allMessages(): kotlin.collections.List<kotlin.String> { /* compiled code */ }

We can see at the top that, instead of what we would normally see in the Kotling standard library for the SDK, we only see references like interfaces and unreadable function bodies. We also see some declared variables, but we never ever get to see any implementation. For the moment, it is not really possible to see, at least not easily, the actual implementation of such methods in the Kotlin Native standard library. The native library is of course native and we can’t see that in the same way that we cannot see the implemented functions in C using the native bindings in Java. This is also one side of what implementing with Kotlin native SDK actually means. We are now programming in the gray area which separates JVM code from Native code, and we do that of course because we want our final product not to have anything to do with the JVM anymore. Also, take notice of the first comment of the decompiler which basically states what we just discussed:

// IntelliJ API Decompiler stub source generated from a class file
// Implementation of methods is not available

This is essentially how Kotlin Native works in its purest and simplest form. We can of course combine our Kotlin code with C code, which allows us to explore more possibilities in what we can do with native code and our final assembly.

3.2. Kotlin native runner implementation

As we’ll see further, we’ll use an example where we will create a runnable that will make very simple operations. It will split a text and immediately concatenate it back without checking. This is an operation that costs time O(n) for the split and O(n) for the join which results in a total of O(n) complexity in time. In terms of space, it will also have an O(n) complexity for the split and an O(n) complexity for the concatenation. The code implementation is very simple, and it looks like this:
Native Runner without C interoperability bindings
This example was created in the same way as the good-feel example was created.

3.3. Kotlin native runner with C interop bindings runner

Before explaining how the Kotlin native runner works with C interoperability in place maybe it’s important to consider why running a code with this particular situation would be of any interest. As mentioned before, creating a code in C provides an alternative to creating native code via Kotlin code. This can provide multiple possibilities for creating new software. What we will want to test in this case is what sort of impact would have when running the same runnable with the particular difference that we use C code instead of Kotlin code. Now we can carry on to the fun part and this is the creation of the C bindings. But before we start I’d like to share a few things I quickly realized when doing this:

It is very difficult to configure Kotlin Native to compile C code into the finally compiled runnable in such a way that it works for every machine.
In this example, I found it better to have the C code compiled separately and then introduce that compiled native code into the assembly with Kotlin native.

So first we create our small library. This is the header file:

#ifndef LIB2_H_INCLUDED
#define LIB2_H_INCLUDED

char* tell_story();

int answer();

char love();

#endif

Now we need the implementation file (redcat.c):

#include "redcat.h"
#include <stdio.h>
#include <string.h>
static char c[255] = "The red cat used to roam around in the neighbourhood. For some reason this cat found in Lucy a connection and became Lucy's friend\0";
static char str[sizeof(char*)*256];
static char delim[] = " ";
static char result[sizeof(char*)*256];
static char array[sizeof(char*)*256];
char* tell_story() {
    char *namePtr;
    namePtr = c;
    return namePtr;
}
char* scramble_story() {
 char* story = tell_story();
 memcpy(str, story, sizeof(char*)*256);
 int init_size = strlen(str);
 char *ptr = strtok(str, delim);
 while (ptr != NULL)
 {
  memcpy(array, ptr, sizeof(char*)*256);
  strcat(result, array);
  strcat(result, delim);
  ptr = strtok(NULL, delim);
 }
 strcat(result, "\0");
    char *namePtr;
    namePtr = result;
 return namePtr;
}
int answer() {
  return 42;
}
char love() {
    return 'L';
}

In my case, what I wanted to have is just a static library. This is ultimately what the Kotlin Native compiler needs to access the code we developed in C. This is a binary native version of our code that we can then inject into the Kotlin code as well. In the Make file of this project, we will see this set of instructions in order to be able to compile our code into the binaries we need:

gcc -c "-I$(PWD)" src/nativeInterop/cinterop/code/redcat.c -o src/nativeInterop/cinterop/code/redcat.o
ar rcs src/nativeInterop/cinterop/code/redcat.a src/nativeInterop/cinterop/code/redcat.o
./kotlin.native/bin/cinterop -def src/nativeInterop/cinterop/redcat.def -compiler-options "-I$(PWD)/src/nativeInterop/cinterop/code" -o src/nativeInterop/cinterop/code/redcat.klib
./kotlin.native/bin/konanc -l src/nativeInterop/cinterop/code/redcat.klib src/nativeMain/kotlin/Main.kt -linker-options src/nativeInterop/cinterop/code/redcat.a -o main

With gcc we get to build our o file. The .o file is in short, a merge between the .h and the .c files. This is called the object file which the compiler can then case use to link with all needed objects in order to create a final executable or binary. The ar command creates a static library from a specified object file.
The following command we use is the cinterop command. Cinterop belongs to the Kotlin-Native Library. I provide the means to install it locally. There is a script for this called install-kotlin-native-linux but for now, that is not important. What cinterop does is precisely what’s in its name. C means C for the C language and interop stands for interoperability. The generated file is a readcat.klib file. If we type this file in the command line we’ll get something like this:

Looks familiar? Not an issue if it isn’t but remember just a few paragraphs above when we looked at the content of the Kotlin Native SDK? This is exactly the same for our own library. The Klib files are the jars in the Kotlin native world and the knm files are almost the same as Klib files but inside the SDK. So finally we have the Klib file that we can import to our project. The way to do it is by the use of .def files. These files need to be located by default in src/nativeInterop/cinterop , but not necessarily. We’ll do all of this explicitly so that we have an idea of how this works. So before we get into the details of the .def file, let’s first have a look at how the build.gradle.kts works:

kotlin {
    val hostOs = System.getProperty("os.name")
    val isMingwX64 = hostOs.startsWith("Windows")
    val nativeTarget = when {
        hostOs == "Mac OS X" -> macosX64("native")
        hostOs == "Linux" -> linuxX64("native")
        isMingwX64 -> mingwX64("native")
        else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
    }

    nativeTarget.apply {
        compilations.getByName("main") {
            cinterops {
                val redcat by creating {
                    defFile(project.file("src/nativeInterop/cinterop/redcat.def"))
                    packageName("org.jesperancinha.knative")
                    headers("src/nativeInterop/cinterop/code/redcat.h")
                    includeDirs.allHeaders("code","src/nativeInterop/cinterop/code")
                }
            }
        }
        binaries {
            executable {
                entryPoint = "main"
            }
        }
    }

    sourceSets {
        val nativeMain by getting
        val nativeTest by getting
    }

}

We’ve already covered the usage of the multiplatform plugin before. For this configuration, we are going to look a bit more in-depth the cinterops configuration. There we find that we create a variable called redcat. This variable is the configuration of our redcat libray. defFile is a method that accepts a File that contains the .def file we’ve mentioned before. packageName is the package that we want to generate via the C code. headers is the method used to configure all the headers files we are going to need and finally, we include the directories where the compiler should find all the headers. We’ve mentioned before that we make the compilation manually to avoid issues at compilation time. these last two properties headers and includeDirs aren’t necessary for our specific case.
Finally, let’s have look at our .def file!

headers = redcat.h
staticLibraries = redcat.a
libraryPaths = code src/nativeInterop/cinterop/code
package = org.jesperancinha.knative

The staticLibraries property is the one property we are interested in this file. There are plenty more properties that can be used, including the paths of the compilers, which compilers you may use, and extended amounts of configuration. For this article though and as we mentioned before, it doesn’t make sense to worry about that just yet because compiling without using the .def file is just easier. However, this could be something to debate in another article, and in the meantime here you can find more information about how to configure .def files on a more advanced level.

Finally, we can check our code:

import kotlinx.cinterop.*
import org.jesperancinha.knative.scramble_story
import org.jesperancinha.knative.tell_story

@kotlinx.cinterop.ExperimentalForeignApi
fun main(args: Array<String>) {
    val max = maxOf(1, if (args.isEmpty()) 0 else args[0].toInt())
    repeat(times = max) {
        scramble_story()
    }
    println(tell_story()?.toKString())
    println(scramble_story()?.toKString())
}

If we take a closer look, we then see our package org.jesperancinha.knative. And we can see that we get the tell_story and the scramble_story methods. This is great! We could have long discussions about naming methods, but I think that conventions discussion shouldn’t find its place in this article where we are taking a look at how Native behaves in terms of performance for the different frameworks.
Maybe it’s also important to mention what the toKString() method does. We don’t really see it in the code, but tell_story and scramble_story return both the type CPointer<ByteVar> . This is just a way to represent a c pointer to the beginning of a string in memory. If you don’t know what pointers are, it is probably a better idea to check what c pointers are first. So, via interoperability, Kotlin interprets the returned String as a ByteVar stream. Since this is Kotlin then of course we have a convenient method to do everything for us and that method is called toKString() .

3.4. Enterprise `Kotlin Native Ktor Service` implementation

One of the aspects of Kotlin Native that make it so interesting is that it allows us to create services with record starting times, and minimal resource memory usage. There could be also other benefits like performance. Here we will use the only possible alternative in the Ktor ecosystem, which is the CIO(Coroutine-based I/O) engine.
For this section, we’ll use the knowledge we’ve already built during all chapters above to create a service. If you are used to Spring, Micronaut, or any other web service framework, you’ll find this easy to understand. If not, I’ll try my best to make it clear.
Let’s recap the reasons why we want to build a Ktor Service at this stage. We want to compare Kotlin Native to GraalVM. We also want to check their similarities and differences. GraalVM is essentially used to convert jars into executables and Spring Native Cloud is used to create containers that we can run in the cloud. Spring Native, is only Native because it uses either GraalVM or a containerized version to run. The latter also uses GraalVM beneath the surface. Kotlin native has no way to create native services except by using the CIO alternative provided by Ktor, which is independent of GraalVM.
Now let’s also discuss another aspect of these services. In order to make the database work, I had to look for alternatives on the internet. There is a project called SQLDelight which provides some useful bindings that I needed for this project on GitHub. On another project by Philip Wedemann, there is code that allows connections to the database. So I used SQLDelight and some tips from postgres-native-sqldelight to create a small driver located in class Database.kt. The way the driver is implemented is quite complicated and not that useful for this article. The important part is that CIO allows for a service to be compiled natively without the need to resort to conversions. Still, it is important to understand how we get access to objects from the native code.
In this implementation, we are simply downloading the Postgres driver, extracting it, compiling it, and then we make the link using the following .def file:

headers = redcat.h /usr/include/postgresql/libpq-fe.h
staticLibraries = redcat.a postgres.a
libraryPaths = c postgresql

As we can see in the above script, we still need to access a libpq-fe.h file from the general postgresql includes. For this to work, we also need to install a few libraries locally:

sudo apt install libreadline-dev
sudo apt-get install libpq-dev
sudo apt install bison
sudo apt install flex

When that is done we still need to compile the whole driver directory we previously downloaded and that is done using the following sequence of commands:

./configure
make all

The postgres directory contains scripts that I’ve added to perform all the necessary operations automatically. At some point in my investigation of this setup, I also found out that a file explicit_bzero_chk.c is also needed to successfully compile libpq . There are lots of suggestions on how to better do this, but I found the way I did it, a perfectly fine way to perform my tests. It’s probably not how we should do for production but while we wait for a release of reliable native libraries for PostgreSQL, we don’t really have much choice other than seeking out the best way to implement this, because at the moment, there aren’t many rules on how to use SQL natively nor there are many alternatives to it.
Let’s finally have a quick look at the build.gradle.kts file:

plugins {
    application
    alias(libs.plugins.kotlin.multiplatform)
    kotlin("plugin.serialization") version "2.1.0"
}

group = "org.jesperancinha.native"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
    maven { url = uri("https://maven.pkg.jetbrains.space/public/p/ktor/eap") }
    maven { url = uri("https://repo1.maven.org/maven2/") }
}

kotlin {
    val hostOs = System.getProperty("os.name")
    val isMingwX64 = hostOs.startsWith("Windows")
    val nativeTarget = when {
        hostOs == "Mac OS X" -> macosX64("native")
        hostOs == "Linux" -> linuxX64("native")
        isMingwX64 -> mingwX64("native")
        else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
    }

    nativeTarget.apply {
        compilations.getByName("main") {
            cinterops {
                val redcat by creating {
                    defFile(project.file("src/nativeInterop/cinterop/redcat.def"))
                    packageName("org.jesperancinha.native")
                    includeDirs.allHeaders("c")
                    includeDirs.allHeaders("postgresql")
                }

            }
        }
        binaries {
            executable {
                entryPoint = "main"
            }
        }

    }
    val ktorVersion = "3.0.3"
    sourceSets {
        val nativeMain by getting {
            dependencies {
                implementation("io.ktor:ktor-server-core:$ktorVersion")
                implementation("io.ktor:ktor-server-cio:$ktorVersion")
                implementation("io.ktor:ktor-server-content-negotiation:$ktorVersion")
                implementation("io.ktor:ktor-serialization-kotlinx-json:$ktorVersion")
                implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.7.3")
                implementation("org.jetbrains.kotlinx:kotlinx-datetime:0.6.1")
                implementation("app.cash.sqldelight:runtime:2.0.2")
            }
        }
        val nativeTest by getting {
            dependencies {
                implementation(kotlin("test"))
            }
        }
    }

}

One of the things that I found to be quite limiting when implementing a native service is the fact that everything needs to be activated via a plugin in order to work. If we think about it, it does make sense, because Kotlin Native is quite a different thing to work with. All libraries that we use, need to be able to support Native and that also means that, on the Kotlin side, we only see the interfaces and not the implementation. This means that, in order to be able to use some jars, we need to have that standard support. This has limited the implementation of this CIO server a lot. This is also part of the reason why I did not import the library provided by Philip Wedemann. Native libraries are difficult to configure and this particular library is still quite dependent on a particular local configuration. The way I did it, allows us to use Libpq with the link to the compiled library optimized for our local Linux machine.
Finally, we can look at how the CIO server is implemented. It essentially follows the typical Onion pattern followed by many enterprise solutions like Spring, JEE, KumuluzEE, Micronaut, etc. It is simply an MVC implementation at its core. But let’s have a look at the endpoint implementation:

embeddedServer(CIO, port = configuration.server.port) {
    routing {
        install(ContentNegotiation) {
            json()
        }
        get {
            call.respondText("Welcome to the Cat Ktor Service!")
        }
        route("/cat") {
            route("/sayings") {
                get {
                    call.respond(listOf<CatSaying>())
                }
                get("/encoded") {
                    call.respondWithEncodedFlow(status = OK, listOf<CatSaying>())
                }
            }
            route("/saying") {
                post {
                    val catSaying = call.receive<CatSaying>()
                    call.respond(status = Created, catSaying)
                }
            }
        }
        route("/story") {
            route("/paragraph") {
                post {
                    val paragraph = call.receive<Paragraph>()
                    call.respond(status = Created, paragraph)
                }
                post("/encoded") {
                    val paragraph = call.receive<Paragraph>()
                    call.respondWithEntity(status = Created, paragraph)
                }
            }
            route("paragraphs") {
                delete {
                   call.respond(status = Accepted,"")
                }
                post("/encoded") {
                    val paragraphs = call.receive<List<Paragraph>>()
                    call.respondWithEncodedFlow(status = Created, paragraphs)
                }
                get {
                    call.respond(listOf<Paragraph>())
                }
                get("/encoded") {
                    call.respond(listOf<Paragraph>())
                }
            }
        }
    }
}.start(wait = true)

This CIO implementation is the way we are going to perform tests on all our services. Essentially, we are going to perform GET’s to the cat statements, we will POST’s to insert paragraphs into the database, we will GET them and also test algorithms to GET them in an encoded way. We will see further exactly what tests will be performed.

4. GraalVM

GraalVM is a high-performance runtime that provides support to lots of different languages like Java, Javascript, LLVM languages, etc. The particular functionality we are going to have a look at in this section is the way GraalVM creates native images and executables. The idea is that we pick up a runnable jar file and then convert it to either an executable to be used in the command line or a docker image to be used in the cloud. For GraalVM we will use one of the favorite frameworks in the market called Spring Framework. We will use its reactive form which is the way Ktor’s CIO works. However, we need to bear in mind that here in this section we are not using Spring with CIO. Instead, we’ll be using Netty which is usually the default service for reactive implementations with Spring. We also need to consider that this may have an effect on our results because simply put, CIO is not Netty.
At this point, I should give you a warning (if not given before) that the resource consumption to build stuff with GraalVM is just as high as building with Kotlin Native. Please be ready for out-of-memory issues, machine hanging, having to restart your computer, etc. It can be very difficult and frustrating at times to perform the builds.

4. 1. Runnable jar implementation

With GraalVM we can easily convert a jar into an executable native application. In the following items further in this article we’ll see how easier is to implement things with GraalVM but first let’s have a look at our gradle.build.kts file:

plugins {
    // Apply the org.jetbrains.kotlin.jvm Plugin to add support for Kotlin.
    alias(libs.plugins.kotlin.jvm)
    id("java")

    // Apply the application plugin to add support for building a CLI application in Java.
    application
}

tasks {
    val fatJar = register<Jar>("fatJar") {
        dependsOn.addAll(listOf("compileJava", "compileKotlin", "processResources"))
        archiveClassifier.set("standalone")
        duplicatesStrategy = DuplicatesStrategy.EXCLUDE
        manifest { attributes(mapOf("Main-Class" to application.mainClass)) }
        val sourcesMain = sourceSets.main.get()
        val contents = configurations.runtimeClasspath.get()
            .map { if (it.isDirectory) it else zipTree(it) } +
                sourcesMain.output
        from(contents)
    }
    build {
        dependsOn(fatJar)
    }
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

dependencies {
    // Align versions of all Kotlin components
    implementation(platform("org.jetbrains.kotlin:kotlin-bom"))

    // Use the Kotlin JDK 8 standard library.
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")

    // This dependency is used by the application.
    implementation("com.google.guava:guava:33.4.0-jre")

    // Use the Kotlin test library.
    testImplementation("org.jetbrains.kotlin:kotlin-test")

    // Use the Kotlin JUnit integration.
    testImplementation("org.jetbrains.kotlin:kotlin-test-junit")
}

application {
    // Define the main class for the application.
    mainClass.set("org.jesperancinha.knative.graalvm.AppKt")
}

You probably just realized that there is no reference to GraalVM whatsoever. This is because this is just a normal GraalVM project. We will, however, generate a native runnable by issuing the following command:

$(JAVA_HOME)/bin/native-image -jar build/libs/whiskers-runners-graalvm-standalone.jar

What this does is create a production-ready native image in the root directory.
This will create a program called whiskers-runners-graalvm-standalone which will contain exactly the same code as its Kotlin native counterparts.

class App {
    companion object {
        val story: String =
            "The red cat used to roam around in the neighbourhood. For some reason this cat found in Lucy a connection and became Lucy's friend"

        fun scrambleStory() = story.split(" ").joinToString(" ")
    }
}

fun main(args: Array<String>) {
    val max = maxOf(1, if (args.isEmpty()) 0 else args[0].toInt())
    repeat(times = max) {
        App.scrambleStory()
    }
    println(App.story)
    println(App.scrambleStory())
}

4. 2. Enterprise Spring Native Service implementation

Spring provides us with different ways to implement a native solution. One of them is a standalone solution that essentially allows us to create a compiled service in a native package that can be run in the operating system we’ve selected. For this case, our Gradle file needs to contemplate a configuration to allow this to happen. Let’s have a look at it:

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
    alias(libs.plugins.spring.boot)
    alias(libs.plugins.spring.dependency.management)
//  id("org.springframework.experimental.aot") version "0.12.1"
    alias(libs.plugins.graalvm.buildtools.native)
    alias(libs.plugins.kotlin.jvm)
    alias(libs.plugins.kotlin.spring)
}

group = "org.jesperancinha.knative"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_21

val gradleSysVersion = System.getenv("GRADLE_VERSION")

tasks.register<Wrapper>("wrapper") {
    gradleVersion =  gradleSysVersion
}

repositories {
    mavenCentral()
    maven { url = uri("https://repo.spring.io/release") }
}

extra["testcontainersVersion"] = "1.17.4"

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-data-r2dbc")
    implementation("org.springframework.boot:spring-boot-starter-webflux")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
    implementation("jakarta.validation:jakarta.validation-api:3.1.0")
    implementation("io.r2dbc:r2dbc-postgresql:0.8.13.RELEASE")
    implementation("org.springframework.boot:spring-boot-starter-validation:3.4.0")
    testImplementation("org.springframework.boot:spring-boot-starter-test")
    testImplementation("io.projectreactor:reactor-test")
    testImplementation("org.testcontainers:junit-jupiter")
    testImplementation("org.testcontainers:r2dbc")
}

dependencyManagement {
    imports {
        mavenBom("org.testcontainers:testcontainers-bom:${property("testcontainersVersion")}")
    }
}

tasks.withType<KotlinCompile> {
    kotlinOptions {
        freeCompilerArgs = listOf("-Xjsr305=strict")
        jvmTarget = "21"
    }
}

tasks.withType<Test> {
    useJUnitPlatform()
}

graalvmNative {
    binaries {
        named("main") {
            mainClass.set("org.jesperancinha.knative.WhiskersGraalvmApplicationKt")
        }
    }
}

Take a moment to find out what could possibly be related to the GraalVM in this configuration. If you didn’t find it, it is really not a problem also because it is not visible anywhere in the file. It is the org.springframework.experimental.aot plugin. It comprises already the spring-native plugin. To generate a runnable file, we just need to run the following command:

./gradlew nativeCompile

Before showing where the executable has been generated, let’s first have at the code. It is implemented according to the Onion design pattern in the way that most Spring Framework MVC environments are created:

@RestController
@RequestMapping("/story")
class StoryController(
    val storyService: StoryService
) {

    @GetMapping("/paragraphs")
    fun getAllParagrahs() = storyService.getAllParagraphs()

    @GetMapping("/paragraphs/encoded")
    fun getAllEncodedParagraphs() = storyService.getCodedParagraphs()

    @GetMapping("/paragrahs/{id}")
    suspend fun getParagraphById(
        @PathVariable
        id: Int
    ) = storyService.getParagraphById(id)

    @PostMapping("/paragraph")
    suspend fun createNewParagraph(
        @RequestBody
        paragraphDto: ParagraphDto
    ) = storyService.saveParagraph(paragraphDto)

    @PostMapping("/paragraph/encoded")
    suspend fun createEncodedParagraph(
        @RequestBody
        paragraphDto: ParagraphDto
    ) = paragraphDto.encodeParagraph()

    @PostMapping("/paragraphs/encoded")
    fun createEncodedParagraphs(
        @RequestBody
        paragraphDtos: List<ParagraphDto>
    ) = paragraphDtos.asFlow().toEncodedParagraph()

    @DeleteMapping("/paragraphs")
    suspend fun deleteAllParagraphs() = storyService.removeAll()
}

For the rest of the application, everything is done in this fashion.
Once the build has finished, a runnable will be available on this location from the root of the module:

./build/native/nativeCompile/whiskers-graalvm

4.3. Enterprise Spring Cloud Native Service implementation

A cloud-native implementation allows us to create an optimized image that runs our executable native packages inside a container created by Spring Cloud Native. How it works, which image it calls, and the volumes it creates don’t matter anymore to us and are no longer a concern. We simply can create Cloud Native images that then allow us to run our native solution inside a container. For now, let’s have a look at the Gradle build file:

plugins {
    alias(libs.plugins.spring.boot)
    alias(libs.plugins.spring.dependency.management)
//  id("org.springframework.experimental.aot") version "0.12.1"
    alias(libs.plugins.graalvm.buildtools.native)
    alias(libs.plugins.kotlin.spring)
    alias(libs.plugins.kotlin.jvm)
}

group = "org.jesperancinha.native"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_21

repositories {
    mavenLocal()
    mavenCentral()
    maven { url = uri("https://repo.spring.io/release") }
}

extra["testcontainersVersion"] = "1.17.4"

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-data-r2dbc")
    implementation("org.springframework.boot:spring-boot-starter-webflux")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
    implementation("jakarta.validation:jakarta.validation-api:3.1.0")
    implementation("io.r2dbc:r2dbc-postgresql:0.8.13.RELEASE")
    implementation("org.springframework.boot:spring-boot-starter-validation:3.4.0")
    testImplementation("org.springframework.boot:spring-boot-starter-test")
    testImplementation("io.projectreactor:reactor-test")
    testImplementation("org.testcontainers:junit-jupiter")
    testImplementation("org.testcontainers:r2dbc")
}

dependencyManagement {
    imports {
        mavenBom("org.testcontainers:testcontainers-bom:${property("testcontainersVersion")}")
    }
}

val gradleSysVersion = System.getenv("GRADLE_VERSION")

tasks.register<Wrapper>("wrapper") {
    gradleVersion =  gradleSysVersion
}

tasks.withType<KotlinCompile> {
    kotlinOptions {
        freeCompilerArgs = listOf("-Xjsr305=strict")
        jvmTarget = "21"
    }
}

tasks.getByName<BootBuildImage>("bootBuildImage") {
    builder = "paketobuildpacks/builder:tiny"
    environment = mapOf(
        "BP_NATIVE_IMAGE" to "true"
    )
    buildpacks = listOf("gcr.io/paketo-buildpacks/java-native-image:7.34.0")
}

tasks.withType<Test> {
    useJUnitPlatform()
}

In this gradle.build.kts file, it's very easy to see where the difference lies. Can you see it?

tasks.getByName<BootBuildImage>("bootBuildImage") {
    builder = "paketobuildpacks/builder:tiny"
    environment = mapOf(
        "BP_NATIVE_IMAGE" to "true"
    )
    buildpacks = listOf("gcr.io/paketo-buildpacks/java-native-image:7.34.0")
}

So using this BootBuildImage task, we can easily build an image that will be pushed to our local Docker repo and it will be called: whiskers-cloudnative:0.0.1-SNAPSHOT .
This makes it easy to run with a simple command line:

docker run whiskers-cloudnative:0.0.1-SNAPSHOT

5. Running the applications

To be able to run these applications I have created quite a lot of scripts everywhere at the root of the project and in the root of individual modules. It is very difficult to guarantee that this will work on your Linux machine, but I have tested this on two Linux machines already, and it seems to work flawlessly. So in steps, this is what we need to do.

Install the environment: run . ./graalVm.sh
Install all the necessary packages: run make install-libs
Manually install all necessary redistributables: run . ./init.sh
Make build: run make b
Run all tests: run make measure-all

All of these runs will take some time to finish. I advise you not to open IntelliJ while running the build. The build will consume resources but also creates lots of incremental files, which will make IntelliJ go completely confused. If you run into issues, and you’ll probably will, please open an issue on my repo. I’ll be very happy to assist you with that. The repo supporting this article will probably never support every single machine on this planet because of its native nature to it, but I will do my best to come as closest as possible to that goal, and with your input, I’ll reach that goal faster.

6. Performance tests

If you successfully ran all tests, you should be getting the final results in the Results.md file:

7. Conclusion

When examining Kotlin Native and how it works, we have seen in this article that reality always falls a bit short of the publicity that is being made. I truly love the idea of Kotlin Native and I think it’s great to expand and allow ourselves to get out of our comfort zone and simply put just try something new.
This is what I wanted, and it is exactly what I have achieved in writing this article and implementing the case to support it. I did see the faster startup times and I do see a future in this, namely because of how many resources we spare.
Unfortunately, not everything was very clear when it comes to performance. In the ’80s, we were used to dealing with memory manually while working directly with C and C++. Then Java came to the mainstream in the late 90s. Java’s inception went really well because most developers didn’t really want to get bothered with memory and most developers wanted a higher-level language. Kolin came to the scene around 2010 and the idea was the same. In short, Kotlin wanted to bring Java to an even higher level. And that has indeed been achieved. And the JVM has been evolving constantly to be ever more efficient and better. In my personal opinion, the idea of going native contradicts all of this because in a way we are giving credit to the way native code works and really going back to our roots when it comes to software development. This could actually be a good sign. At Uni one of my teachers kept telling me "You have to learn C, this Java thing is just a fad". Years later Kotlin Native kind of gives credit to what my teacher initially said. But unfortunately going back to C or making Kotin compile into native code also means that the people behind C and even the usual developer will also need to know how C works and that also means going back to learning how memory allocation works. This also means learning how pointers work, memory references work, string terminators, pointer of pointers, content of pointers, struct, etc. This could be the reason for the interesting phenomenon we have seen when testing performance. It seems like, for an already running process, performance actually degrades with Native Code. If the JVM is so sophisticated to the point of it being much better than anything we can come up with in a few minutes, then it is probably a better option than going native.
So the key points about this article are that there are several Pros and Cons when it comes to going native in general and that both of them apply on the same level to both Kotlin Native and GraalVM. So let’s enumerate them:

Pros:

Extremely faster startup time — But don’t get carried away. We’re talking about a few seconds' differences. For the impatient, this could be a good thing. If you are thinking about starting multiple processes multiple times then this could also be a great thing to keep on your radar.
Low memory usage — It can have a huge impact on your budget to keep using the JVM for no good reason. For multiple runs or servers or services that perform tasks that do not require high performance and don’t need to comply with demanding SLAs and High Availability requirements, this could be an ideal option.
Instant peak performance — This is a commonly not very well-interpreted property of native applications that just means that JIT doesn’t play a role in the running application because everything needed for the application has already been compiled ahead of time (AOT). This just means that once the application starts, it won’t perform any optimization tasks as it happens with the Just In Time compiler (JIT). This is the reason why multiple short runs are better performant with native and long-lasting runs in the JVM are much better performant.
No need to run the JVM — We can run the executable wherever we want without a need to worry about the JDK being available or in the correct version
Security issues — Because we are working with natively compiled code and we have that running online, most security issues related to the JVM are no longer a concern. A native compiled executable is much more difficult to change during runtime than a running jar executable. #### Cons:
Slow performance risk — As we have seen in our results, Native code, in whatever shape or form we tried it, seems to perform worse than regular code that runs in the JVM, long term.
Compilation times — Compilation times can be quite a nightmare when compiling natively. This is valid for GraalVM and Kotlin Native. It also consumes so many resources that a developer may find that their machine just got blocked, memory just ran out, and they just have to restart their machine. The point is, it is not that useful to have to wait for so long for the code to compile. This impacts development time and development costs.

If we analyze the Cons we may get convinced that Native is also a good story and that it does not make the cut yet. For me, it really does make the cut, but not for most cases. With this article and the case for it, I can already guarantee that, and as I mentioned before, Kotlin native at the moment is the best thing to do if we want to start several independent finite processes. For services, I believe we still need to wait for CIO to evolve or that Netty also supports Native eventually. For the time being, I still don’t see a clear benefit of using a service natively. Quite the opposite still.
In terms of the database usage, well, the results vary so widely per run that I could not make any conclusion. Netty and PostgresSQL Native Support for Kotlin is still pretty much being developed so my expectation wasn’t that high to reach any meaningful conclusion. However, the project is already ready to perform that test once Native support is finally available. Another thing to point out is that although CIO supports coroutines, I could not make database access that performs reactively. This means that the database access tests were made in a non-reactive way to the database which has a detrimental effect on the way we reply back to the client.
When it comes to the runnable tests, I did see some clues to indicate that using our own implementation in C may be a bit better performant than using the Kotlin-Native’s own implementation. A difference of 161 to 28 seconds in our example to be exact. In any case, the interop configuration itself doesn’t seem to affect performance at all.
Finally, I just want to mention that this article will not be subject to major updates, but only revisions. When eventually Native services get better than normal services in terms of running performance, I will then make another article. This one stays in for historic reasons.
I hope, this article, has spiced up your interest in Kotlin-Native. It is quite obvious that Kotlin Native has quite a lot of limitations still and probably forever due to its nature, but it does show its benefits with independent executions. And it does this exactly in the same way that GraalVM does it. I did this whole code in Kotlin because I did not want to include further disparities by taking in another language like Java or Scala, but here comes my point about Kotlin vs Java. GraalVM does amazing things and it is already compatible with a lot more things. Natively it can already create executables with services that already run on netty, and we don’t have any, or that many, limitations apparently on what we can implement. We don’t have to use GraalVM with Java because it is pretty much language-independent. So why Kotlin-Native? I do not know! I have no idea why Kotlin-Native specifically. In this article, I could not find a clear indication of why Kotlin-Native would be any different from using GraalVM. At the moment GraalVM does offer more possibilities, but long term, I have no idea what will happen with Kotlin-Native. They seem to be two technologies that will soon again compete with each other in some kind of Netflix-inspired nostalgia that I’m already ready to know all about and digest with a bucket of popcorn. But honestly saying like Java vs Kotlin, GraalVM vs Kotlin-Native it’s again kind of just amazing good-story and nothing more than that. Goed verhaal in Dutch anyway.
I have placed all the source code of this application on GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it.
Thank you for reading!

Forem: João Esperancinha

Implementing a Geo Location module using the Haversine formula

1. Introduction

1.1. Haversine Formula Origins

2. Theory

2.1. Latitude (ɸ)

The symbol for Latitude is ɸ.

2.2. Longitude (λ)

3. Implementation

4. Publishing a library

5. Running unit tests

6. Conclusion

7. Resources

7.1. Books & Papers

7.2. Websites

References

Making reactive applications with a Kitten Care Example

1. Introduction

1960's

2010

Reactive Manifesto

Observer Pattern

Spring Webflux

2. Environment Setup

3. Implementation

4. REST endpoints test

4.1 Returning a simple Mono

4.2 Returning the complete cat info

4.3 All cats

5. A forged identity case

6. Testing with Blockhound

7. Conclusion

8. Resources

8.1. Books and papers

8.2. Websites

How to make the best of a slow machine running on limited resources with a Windows environment as a Java Engineer

1. Environment Setup

1.1. Chocolatey

1.2. Install necessary software

2. Install Docker

3. Installing Ubuntu for windows

4. Installing Java in Ubuntu for Windows

4.1 SDK-MAN. Via SdkMan!

4.2. Ubuntu. Via Ubuntu commandline with APT

4.3. Install utilities

5. Intellij

6. Conclusion

7. Resources

Redis Cache - A String story

Publish

Listener (Subscriber)

References

Pub-sub Redis in Micronaut

References

How to Manage Your APIs Across Multi-Cloud Environments

1. Introduction

2. Configure API Gateways in Each Cloud Provider

2.1 API Gateway Federation and synchronization

2.2. API Lifecycle Management

2.2.1. Security and Access Control

2.2.2. Monitoring and Analytics

2.2.3. Testing and Deployment

2.2.3. Disaster Recovery and High Availability

3. Conclusion

4. References

ISBN Stacks — A look at a possible Spring Application implementation without annotations

1. Introduction

2. What we are trying to find

3. Biases

4. Case

5. Implementation

5.1. Traditional Spring Web MVC Blocking REST service

5.2. Traditional Spring WebFlux MVC Reactive REST service

5.3. Spring KoFu implementation

6. Starting the services

6.1. locust-sequence-start test

6.2. locust-small-load-sequence-start

6.3. Service startup

7. Conclusion

8. References

5.1. Traditional `Spring Web MVC` Blocking `REST` service

12. Setting up `InfluxDB`

10. How is a `JWT` token made

3.4. Enterprise `Kotlin Native Ktor Service` implementation