Forem: Ryosuke Hasebe

Energy-Efficient Schema-Driven Development with Spring WebMVC/WebFlux

Ryosuke Hasebe — Sun, 20 Apr 2025 08:12:22 +0000

Introduction

There is a concept known as “Schema‑Driven Development.” This is a methodology in which API documentation is first defined as an OpenAPI schema, and from that schema, server and client code are automatically generated. This approach prevents discrepancies between documentation and implementation, while accelerating the development cycle and ensuring quality.

However, writing OpenAPI schemas manually can be a poor developer experience. While tools like typespec exist to address this, they will not be covered here.

In Spring WebMVC/WebFlux, you can use springdoc-openapi to automatically generate an OpenAPI schema from Spring controller implementations. Having an OpenAPI schema enables automatic generation of HTML documentation and client code, which already provides some value.

That said, since the schema is generated from the controller implementation in this case, it may not fully qualify as schema-driven development.

Did you know that since Spring 5.1, it’s possible to define controllers using interfaces? By leveraging these interfaces, you can more easily achieve schema-driven development with less overhead.

Separating Traditional Controller Definitions into Interfaces

Here’s an example. It uses annotations like *Exchange, which might be unfamiliar to some, but the reason for using them will be explained later. Using the standard *Mapping annotations is also perfectly fine.

@HttpExchange("/api/v1/blogs")
interface BlogApi {
    @GetExchange("/")
    @Operation(
        summary = "Get blog list",
        description = "blah blah blah"
    )
    fun list(): List<Blog>

    @PostExchange("/")
    @Operation(
        summary = "Create a new blog",
        description = "blah blah blah"
    )
    fun create(@RequestBody request: BlogCreateRequest): Blog

    @GetExchange("/{id}")
    @Operation(
        summary = "Get a blog by ID",
        description = "blah blah blah"
    )
    fun get(@PathVariable id: Int): Blog

    @PutExchange("/{id}")
    @Operation(
        summary = "Update a blog by ID",
        description = "blah blah blah"
    )
    fun update(@PathVariable id: Int, @RequestBody request: BlogUpdateRequest): Blog

    @DeleteExchange("/{id}")
    @Operation(
        summary = "Delete a blog by ID",
        description = "blah blah blah"
    )
    fun delete(@PathVariable id: Int)
}

data class Blog(val id: Int, val title: String, val body: String)

data class BlogCreateRequest(val title: String, val body: String)

data class BlogUpdateRequest(val title: String, val body: String)

Next, you implement it like this.

@RestController
class BlogApiController : BlogApi {
    // Omitted...
}

Using interfaces brings the following benefits:

The interface becomes the API schema
- This can be used to generate an OpenAPI schema and HTML documentation with springdoc-openapi
Interfaces can be prepared and reviewed ahead of implementation, enabling true schema-driven development
Traditionally, Swagger annotations and implementation code are mixed within controllers, making it hard to understand the overall structure
- In the example above, there are only a few Swagger annotations, but in real-world usage, these tend to accumulate, resulting in cluttered controllers
- It also makes reviewing easier for reviewers

It’s also great that this approach doesn’t require any new tools, so it's easy to start. And if you decide to stop using it later, it's easy to revert.
(springdoc-openapi is needed, but it’s not too difficult to integrate...)

The typespec tool briefly mentioned earlier allows defining OpenAPI schemas in TypeScript-like code. Using interfaces brings a development experience similar to that. (While typespec can also be used to define JSON Schema and Protobuf, making it more powerful...)

Generating OpenAPI Schemas with springdoc-openapi

There’s no change in how this is done compared to traditional controller-based approaches, so this section will be omitted.

(There are also plenty of articles online explaining this in detail...)

Development Flow

In short, you can organize your development flow like this:

Advanced Usage: Using with Spring HTTP Interface Client

Since Spring 6, a feature called the HTTP Interface Client has been introduced.

At first, I thought it was just a knockoff of Retrofit (sorry...), but it turns out you can also use it as a controller interface. The *Exchange annotations in the earlier example come from this HTTP Interface.

In other words, if the client side is also using Spring, you can extract the controller interface and related model classes into a library, and use it to easily build the API client.

val webClient = WebClient.builder()
    .baseUrl("http://localhost:8080/")
    .build()
val proxyFactory = HttpServiceProxyFactory.builderFor(WebClientAdapter.create(webClient)).build()

val blogApi = proxyFactory.createClient(BlogApi::class.java)

This is particularly useful in environments where Spring is widely used in microservices.

However, since the interface is shared between server and client, it’s a downside that only one side cannot be Reactive(Mono/Flux).

Conclusion

This article introduced the use of controller interfaces as one way to implement schema-driven development in Spring WebMVC/WebFlux.

Traditionally, OpenAPI schemas were generated from Spring controller implementations, which did not align well with the schema-driven development philosophy. By leveraging controller interfaces available since Spring 5.1, it’s now possible to define the schema first and then base both implementations and client generation on it.

Additionally, these interfaces can be used in combination with the HTTP Interface Client introduced in Spring 6.

Some might argue that if you want true schema-driven development, you should use gRPC. However, the approach introduced here offers a development experience that gets you fairly close.

[Kotlin] Using Mappie to Reassign Data Class Properties

Ryosuke Hasebe — Sat, 15 Mar 2025 01:01:51 +0000

Motivation

When using a layered architecture, we inevitably have to repopulate data classes frequently.

I’ve grown quite accustomed to this, so it doesn’t bother me much anymore... but every once in a while, I wonder if there’s a more effortless way to handle it.

The most classic approach is to use a reflection-based library, but reflection introduces overhead, and more importantly, it’s not type-safe, which is undesirable.
(For example, if there’s a field that cannot be mapped, it might be silently ignored, causing unexpected bugs...)

In Java, there is MapStruct which uses annotation processing. However, it seems somewhat incompatible with Kotlin.

While searching for similar libraries, I found Mappie, which looked promising, so I’d like to introduce it here.

(Incidentally, there’s also konvert. It appears quite similar to MapStruct in terms of usage, but because it’s annotation-based, it might lack flexibility. Hence, this time I’ll introduce Mappie.)

Overview of Mappie

Mappie is implemented as a Kotlin Compiler Plugin, and it automatically generates object mapping code at compile time wherever possible.

First, let’s look at a simple example. By just preparing a PersonMapper that extends ObjectMappie, the code to convert from Person to PersonDto is automatically generated.

data class Person(
    val firstName: String,
    val lastName: String,
    val middleName: String?,
    val age: Int,
)

data class PersonDto(
    val firstName: String,
    val lastName: String,
    val age: Int,
)

// HERE
object PersonMapper : ObjectMappie<Person, PersonDto>()

fun main() {
    val person = Person(firstName = "Ryosuke", lastName = "Hasebe", middleName = null, age = 20)
    val personDto = PersonMapper.map(person)
    println(personDto)
}

// Result
PersonDto(firstName=Ryosuke, lastName=Hasebe, age=20)

You Can Also Explicitly Specify Mapping Rules

In the previous example, both classes had fields with the same names, so automatic mapping code could be generated.

However, consider a case where PersonDto.name cannot be mapped. This leads to a compile error, which is great because it prevents silent failures.
(Unfortunately, it didn’t show up as a compile error in IntelliJ IDEA... but perhaps improvements to the FIR implementation could address that in the future.)

data class Person(
    val firstName: String,
    val lastName: String,
    val age: Int,
)

data class PersonDto(
    val name: String,
    val age: Int,
)

object PersonMapper : ObjectMappie<Person, PersonDto>()

fun main() {
    val person = Person(firstName = "Ryosuke", lastName = "Hasebe", age = 20)
    val personDto = PersonMapper.map(person)
    println(personDto)
}

// Compile Error
Target PersonDto::name has no source defined

In such scenarios, you can resolve the issue by explicitly writing out the mapping rules. Since these rules are written in a Kotlin DSL, it seems quite flexible.

data class Person(
    val firstName: String,
    val lastName: String,
    val age: Int,
)

data class PersonDto(
    val name: String,
    val age: Int,
)

object PersonMapper : ObjectMappie<Person, PersonDto>() {
    // HERE
    override fun map(from: Person) = mapping {
        to::name fromValue "${from.firstName} ${from.lastName}"
    }
}

fun main() {
    val person = Person(firstName = "Ryosuke", lastName = "Hasebe", age = 20)
    val personDto = PersonMapper.map(person)
    println(personDto)
}

// Result
PersonDto(name=Ryosuke Hasebe, age=20)

From the documentation site, you can see that there are many other ways to define mapping rules:

https://mappie.tech/

Expressing Mapping Errors

Handling Them as Exceptions

Let’s consider a scenario in which object creation involves a validation step. Naturally, if the validation fails, an exception is thrown.

data class Person(
    val firstName: String,
    val lastName: String,
    val age: Int,
)

data class PersonDto(
    val name: String,
    val age: Int,
) {
    // HERE
    init {
        require(age >= 20) { "age must be greater than or equals to 20" }
    }
}

object PersonMapper : ObjectMappie<Person, PersonDto>() {
    override fun map(from: Person) = mapping {
        to::name fromValue "${from.firstName} ${from.lastName}"
    }
}

fun main() {
    val person = Person(firstName = "Ryosuke", lastName = "Hasebe", age = 18)
    val personDto = PersonMapper.map(person)
    println(personDto)
}

// Result
Exception in thread "main" java.lang.IllegalArgumentException: age must be greater than or equals to 20
    at demo.PersonDto.<init>(Main.kt:19)
    at demo.PersonMapper.map(Main.kt:24)
    at demo.MainKt.main(Main.kt:31)
    at demo.MainKt.main(Main.kt)

Handling Them with `kotlin.Result`

If you’d prefer to handle them with kotlin.Result rather than exceptions, you can accomplish this by simply writing an extension function, as shown below. This flexibility is really convenient.

fun <FROM, TO> ObjectMappie<FROM, TO>.mapAsResult(from: FROM): Result<TO> {
    return runCatching { map(from) }
}

fun main() {
    val person = Person(firstName = "Ryosuke", lastName = "Hasebe", age = 18)
    val personDtoResult = PersonMapper.mapAsResult(person)
    println(personDtoResult.getOrNull())
}

// Result
null

Conclusion

By leveraging Mappie, you can implement object mapping in Kotlin in a concise and type-safe manner.

In addition to the default automatic mapping, you can specify flexible mapping rules via a DSL, which supports more fine-grained control when needed. Furthermore, for error handling during mapping, you can choose to handle errors as exceptions or use kotlin.Result, offering a flexible design.

It’s still debatable whether it’s ready for production use, but it certainly seems very convenient.

Important Notes on JUnit 5.12.0+ in Gradle

Ryosuke Hasebe — Sat, 15 Mar 2025 00:42:42 +0000

TL;DR

When upgrading to JUnit 5.12.0 or later, be sure to add:

testRuntimeOnly("org.junit.platform:junit-platform-launcher")

(The version is omitted under the assumption that you're using a BOM.)

If you don't add this, the test task will fail to execute, so you'll likely notice the issue right away.

Why

If you don't explicitly declare a dependency on junit-platform-launcher, Gradle will implicitly use its bundled version.
As of Gradle 8.13, the bundled version is 1junit-platform-launcher 1.8.2` (whereas the latest version currently is 1.12.0).

JUnit 5.12 is incompatible with junit-platform-launcher 1.8.2, which results in test execution failures.

Gradle was aware of such potential issues and, starting with Gradle 8, recommended explicitly specifying junit-platform-launcher.

See the official Gradle upgrade guide: Gradle 8 Upgrade Guide

Ref

When using an S3-compatible Object Storage, be cautious when upgrading SDK for Java 2.x to version 2.30.0 or later

Ryosuke Hasebe — Sat, 08 Feb 2025 01:03:21 +0000

TL;DR

S3 has the following Data Integrity Protections mechanism:
https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html

This mechanism is now automatically enabled starting from SDK for Java 2.x 2.30.0.
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/s3-checksums.html

If the S3-compatible Object Storage you are using does not support this feature, requests to the Object Storage may result in errors.
(If you have proper tests in place, you should notice this, but it’s a good idea to be cautious when upgrading versions.)

If you encounter this issue, apply the following settings.

// sync clientの場合
S3Client
    .builder()
    // ...
    .requestChecksumCalculation(RequestChecksumCalculation.WHEN_REQUIRED)
    .responseChecksumValidation(ResponseChecksumValidation.WHEN_REQUIRED)
    // ...
    .build()

// async clientの場合
S3AsyncClient
    .builder()
    // ...
    .requestChecksumCalculation(RequestChecksumCalculation.WHEN_REQUIRED)
    .responseChecksumValidation(ResponseChecksumValidation.WHEN_REQUIRED)
    // ...
    .build()

Background

After upgrading software.amazon.awssdk:s3 to 2.30.0 or later, we started encountering the following error in the S3-compatible Object Storage used internally.

(Service: S3, Status Code: 400, Request ID: MASKED...)
software.amazon.awssdk.services.s3.model.S3Exception:  (Service: S3, Status Code: 400, Request ID: MASKED...)
    at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:155)
...

Cause

S3 has the following Data Integrity Protections mechanism:
https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html

To briefly explain the mechanism:

When uploading to S3, the SDK calculates a checksum of the content and uploads it along with the content.
- The S3 server verifies the checksum of the transmitted content.
When downloading content from S3, the checksum is also downloaded along with the content.
- The SDK verifies the checksum of the received content.

(Previously, data integrity verification could be performed by specifying the Content-MD5 header at the time of upload. This seems to be a more advanced version of that. The ability to verify data integrity during downloads is a useful addition.)

This mechanism is now automatically enabled starting from 2.30.0.
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/s3-checksums.html

However, since the S3-compatible Object Storage I was using did not support it, errors started occurring.

In my case, I quickly noticed the issue through tests, but since it was just a minor version change, I imagine many people might overlook it.

Support Status of Various S3-Compatible Object Storages

I wanted to properly investigate this but gave up!

Based on a rough check using o1 pro mode & Deep Search, MinIO supports it, but other storages do not.

Since this mechanism was introduced in February 2022, which is relatively recent (though it has been three years), support is likely still limited.
https://aws.amazon.com/jp/blogs/aws/new-additional-checksum-algorithms-for-amazon-s3/

Miscellaneous

The SDK for Java 1.x still does not support this feature, so this issue does not occur. However, since it is reaching EOL this year, those who have not yet migrated to the 2.x series should do so.
https://aws.amazon.com/jp/blogs/developer/announcing-end-of-support-for-aws-sdk-for-java-v1-x-on-december-31-2025/

About Micrometer Context Propagation

Ryosuke Hasebe — Fri, 07 Feb 2025 13:08:33 +0000

Introduction

A library called Micrometer Context Propagation has been introduced in Spring Boot and Reactor (starting from Spring Boot 3).

As the name suggests, this library facilitates Context Propagation between threads.

In this article, I will explain Micrometer Context Propagation. Since this is a niche topic and there are not many articles covering it, I hope this will be helpful to someone.

Why was Micrometer Context Propagation created?

Originally, Micrometer became popular in the Java community as a metrics library. Over time, Micrometer expanded its scope to cover tracing and logging as well, essentially extending its reach to observability.

Generally (not just in Java), tracing requires a technique called Context Propagation. Micrometer Context Propagation is designed to extract and provide this technique in an easy-to-use manner.

It is not exclusive to Micrometer but is also designed to work easily with various libraries such as Kotlin Coroutine Context and Reactor Context.

In Spring Boot, there was a library called Spring Cloud Sleuth (hereafter referred to as Sleuth), which handled tracing in Spring WebMVC/WebFlux.
https://spring.io/projects/spring-cloud-sleuth

With Spring Boot 3, Sleuth was deprecated, and its functionalities were incorporated into Micrometer and Spring Boot.

What is Context Propagation?

As the name suggests, it is a technique for correctly propagating context. At this point, you might wonder, "What is context?" In this case, you can think of it as MDC or TraceContext (a collection of values such as traceId and spanId).

Traditionally, in Java, context propagation could be achieved by simply storing values in a Thread Local. In fact, both MDC and (brave's) TraceContext are stored in Thread Local.

... This might be hard to understand just from an explanation, so let’s look at a concrete example. Let's start with MDC, which is the most familiar case.

fun main() {
    MDC.put("hoge", "hoge-value")

    someFunc1()
    someFunc2()
}

fun someFunc1() {
    println("Called someFunc1 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun someFunc2() {
    println("Called someFunc2 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

Here is the result. There should be no surprises here—the MDC setting remains intact.

Called someFunc1 func. thread=main, mdc={hoge=hoge-value}
Called someFunc2 func. thread=main, mdc={hoge=hoge-value}

Now, what about the following example? Here, someFunc2 is executed on a separate thread.

fun main() {
    MDC.put("hoge", "hoge-value")

    someFunc1()
    thread { someFunc2() }.join()
}

fun someFunc1() {
    println("Called someFunc1 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun someFunc2() {
    println("Called someFunc2 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

Here is the result. In someFunc2, the contents of MDC are null. Since MDC is based on Thread Local, it does not propagate correctly when the thread changes.

Called someFunc1 func. thread=main, mdc={hoge=hoge-value}
Called someFunc2 func. thread=Thread-0, mdc=null

To propagate it correctly, we need to use a technique like the one below (which is what Context Propagation is). Here, we define a custom function called mdcPropagatingThread.

fun main() {
    MDC.put("hoge", "hoge-value")

    someFunc1()
    mdcPropagatingThread { someFunc2() }.join()
}

fun someFunc1() {
    println("Called someFunc1 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun someFunc2() {
    println("Called someFunc2 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun mdcPropagatingThread(block: () -> Unit): Thread {
    val map = MDC.getCopyOfContextMap()
    return thread {
        MDC.setContextMap(map)
        try {
            block()
        } finally {
            MDC.clear() // In this example, it's a disposable thread, so there's no need to clear it...
        }
    }
}

Here is the result. The original MDC is successfully carried over to someFunc2.

Called someFunc1 func. thread=main, mdc={hoge=hoge-value}
Called someFunc2 func. thread=Thread-0, mdc={hoge=hoge-value}

Micrometer Context Propagation provides an abstraction of such techniques as an OSS library.

The previous example can be rewritten using Micrometer Context Propagation as follows.
At first glance, you might think that the additional MDCAccessor class makes the code longer and more cumbersome. However, since it becomes reusable in various places, it actually simplifies things overall.

val snapshotFactory = ContextSnapshotFactory.builder().build()

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())

    MDC.put("hoge", "hoge-value")

    someFunc1()
    contextPropagatingThread { someFunc2() }.join()
}

fun someFunc1() {
    println("Called someFunc1 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun someFunc2() {
    println("Called someFunc2 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

fun contextPropagatingThread(block: () -> Unit): Thread {
    val snapshot = snapshotFactory.captureAll()
    return thread {
        snapshot.setThreadLocals().use {
            block()
        }
    }
}  

class MDCAccessor : ThreadLocalAccessor<Map<String, String?>> {
    override fun key(): Any {
        return MDCAccessor::class.java
    }

    override fun getValue(): Map<String, String?>? {
        return MDC.getCopyOfContextMap()
    }

    override fun setValue(value: Map<String, String?>) {
        MDC.setContextMap(value)
    }

    override fun setValue() {
        MDC.clear()
    }
}

How to Use Micrometer Context Propagation

The introduction took too long, but now we move to the main topic—how to use Micrometer Context Propagation.

The main classes (or concepts) to remember are the following four:

ThreadLocalAccessor
ContextAccessor
ContextRegistry
ContextSnapshot

ThreadLocalAccessor

When you want to propagate Thread Local values, implement the ThreadLocalAccessor interface.

You need to override the following methods:

Object key()
- Returns a unique key.
- You can use Class<?> (e.g., SomeThreadLocalAccessor::class.java).
@Nullable V getValue()
- Returns the value to be propagated.
- Usually retrieved from Thread Local.
void setValue(V value)
- Sets the value to be propagated.
- Typically stored in Thread Local.
- V is non-null.
- https://github.com/micrometer-metrics/context-propagation/blob/f6026d7e8ac833a822bc2032ac44d462b04c0775/context-propagation/src/main/java/io/micrometer/context/DefaultContextSnapshot.java#L86
void setValue()
- Not mandatory unless overriding restore(), but required when clearMissing is enabled.
- Called when the propagated value is null.
- https://github.com/micrometer-metrics/context-propagation/blob/f6026d7e8ac833a822bc2032ac44d462b04c0775/context-propagation/src/main/java/io/micrometer/context/DefaultContextSnapshot.java#L89
void restore(V previousValue)
- Not mandatory. If not implemented, setValue(V value) is used instead.
- Called when restoring the original value.
- https://github.com/micrometer-metrics/context-propagation/blob/f6026d7e8ac833a822bc2032ac44d462b04c0775/context-propagation/src/main/java/io/micrometer/context/DefaultContextSnapshot.java#L149
- V is non-null.
void restore()
- Not mandatory. If not implemented, setValue() is used instead.
- Called when restoring a null value.
- https://github.com/micrometer-metrics/context-propagation/blob/f6026d7e8ac833a822bc2032ac44d462b04c0775/context-propagation/src/main/java/io/micrometer/context/DefaultContextSnapshot.java#L152

Here are some examples.

Example: MDC

(Reiterated example)

class MDCAccessor : ThreadLocalAccessor<Map<String, String?>> {
    override fun key(): Any {
        return MDCAccessor::class.java
    }

    override fun getValue(): Map<String, String?>? {
        return MDC.getCopyOfContextMap()
    }

    override fun setValue(value: Map<String, String?>) {
        MDC.setContextMap(value)
    }

    override fun setValue() {
        MDC.clear()
    }
}

However, since brave may update MDC, the above approach might miss some propagation. In some cases, it's better to use the following approach:

class MDCAccessor : ThreadLocalAccessor<Map<String, String?>> {
    override fun key(): Any {
        return MDCAccessor::class.java
    }

    override fun getValue(): Map<String, String?>? {
        return MDC.getCopyOfContextMap()
    }

    override fun setValue(value: Map<String, String?>) {
        // There are cases where it doesn't work well when used together with Brave's MDC decoration, so use the put method.
        // see: https://github.com/openzipkin/brave/blob/69003dfc811418f0dbc42e9e17ff880ebe1f4b02/brave/src/main/java/brave/propagation/CurrentTraceContext.java#L130
        value.forEach { (k, v) -> MDC.put(k, v) }
    }

    override fun setValue() {
        // NOOP
    }

    override fun restore(previousValue: Map<String, String?>) {
        MDC.setContextMap(previousValue)
    }

    override fun restore() {
        MDC.clear()
    }
}

With the restore method, there's some flexibility.

Example: Brave Trace Context

(Not tested, but probably works like this.)

This is just an example, but if you use micrometer-tracing, you can use the built-in ObservationThreadLocalAccessor to propagate the Trace Context.

class BraveTracingContextAccessor : ThreadLocalAccessor<TraceContext> {
    override fun key(): Any {
        return BraveTracingContextAccessor::class.java
    }

    override fun getValue(): TraceContext? {
        return Tracing.current().currentTraceContext().get()
    }

    override fun setValue(value: TraceContext) {
        Tracing.current().currentTraceContext().maybeScope(value)
    }

    override fun setValue() {
        Tracing.current().currentTraceContext().maybeScope(null)
    }
}

ContextAccessor

When you want to propagate Map-like objects such as Reactor Context, implement the ContextAccessor interface.

However, you probably won't need to implement this yourself often, so you don't need to remember it too much.

Example: Reactor Context

Reactor provides an implementation called ReactorContextAccessor.

/**
 * A {@code ContextAccessor} to enable reading values from a Reactor
 * {@link ContextView} and writing values to {@link Context}.
 * <p>
 * Please note that this public class implements the {@code libs.micrometer.contextPropagation}
 * SPI library, which is an optional dependency.
 *
 * @author Rossen Stoyanchev
 * @author Simon Baslé
 * @since 3.5.0
 */
public final class ReactorContextAccessor implements ContextAccessor<ContextView, Context> {

    @Override
    public Class<? extends ContextView> readableType() {
        return ContextView.class;
    }

    @Override
    public void readValues(ContextView source, Predicate<Object> keyPredicate, Map<Object, Object> target) {
        source.forEach((k, v) -> {
            if (keyPredicate.test(k)) {
                target.put(k, v);
            }
        });
    }

    @Override
    @Nullable
    public <T> T readValue(ContextView sourceContext, Object key) {
        return sourceContext.getOrDefault(key, null);
    }

    @Override
    public Class<? extends Context> writeableType() {
        return Context.class;
    }

    @Override
    public Context writeValues(Map<Object, Object> source, Context target) {
        return target.putAllMap(source);
    }
}

ContextRegistry

ContextRegistry is an instance that holds the previously mentioned ThreadLocalAccessor and ContextAccessor.

You can register it like this. It is expected to be registered before starting the application in the main method.

ContextRegistry.getInstance()
    .registerThreadLocalAccessor(...) // ThreadLocalAccessor
    .registerContextAccessor(...) // ContextAccesor

Although multiple instances of ContextRegistry can be created, it is recommended to share and use the instance obtained via ContextRegistry.getInstance(). At the very least, Reactor uses the instance obtained via ContextRegistry.getInstance().

Additionally, it can be automatically registered via the Service Loader.

ContextRegistry()
    .loadThreadLocalAccessors()
    .loadContextAccessors()

The instance obtained via ContextRegistry.getInstance() is pre-registered through the Service Loader.
https://github.com/micrometer-metrics/context-propagation/blob/75a243f3427d0941e09302c3fc29f5b2a0297583/context-propagation/src/main/java/io/micrometer/context/ContextRegistry.java#L40-L41

For example, the ObservationThreadLocalAccessor for Micrometer Observation is automatically registered via the Service Loader.
https://github.com/micrometer-metrics/micrometer/blob/a56b968ba5b3db9b5e4a4feac813080783a16f5f/micrometer-observation/src/main/resources/META-INF/services/io.micrometer.context.ThreadLocalAccessor

ReactorContextAccessor is also automatically registered via the Service Loader.
https://github.com/reactor/reactor-core/blob/5553aa80137482ec26acb960f4d4f42b8a44da94/reactor-core/src/main/resources/META-INF/services/io.micrometer.context.ContextAccessor

ContextSnapshot

As the name suggests, this is a snapshot of a context. By calling setThreadLocals or updateContext implemented in this class, context can be propagated to various locations (such as different threads or Reactor Context).

A ContextSnapshot can be created via ContextSnapshotFactory.

// Retrieve from the registered ThreadLocalAccessor.
val snapshot = ContextSnapshotFactory.builder().build().captureAll()

// Retrieve from the registered ThreadLocalAccessor and Reactor Context. ReactorContextAccessor must be registered.
val snapshot = ContextSnapshotFactory.builder().build().captureAll(reactorContext)

When creating a ContextSnapshotFactory, you can configure the following settings via the builder (the example above does not set anything):

contextRegistry
- By default, it uses the instance obtained from ContextRegistry.getInstance().
- It is generally fine to leave it as the default. Specify it if necessary.
captureKeyPredicate
- You can specify which ThreadLocalAccessor to capture using the value of ThreadLocalAccessor#key(). If not specified, all registered ThreadLocalAccessors will be used.
- It is generally fine to leave it as the default. Specify it if necessary.
clearMissing
- Determines whether to call ThreadLocalAccessor#setValue when the value to be propagated is missing.
- https://github.com/micrometer-metrics/context-propagation/blob/f6026d7e8ac833a822bc2032ac44d462b04c0775/context-propagation/src/main/java/io/micrometer/context/DefaultContextSnapshot.java#L89
- It is generally fine to leave it as the default. Specify it if necessary.

Creating a ContextSnapshotFactory every time is cumbersome, so it is advisable to store it as a static variable or register it as a bean.

To propagate a ContextSnapshot to another thread, you can do something like this:

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())

    MDC.put("hoge", "hoge-value")

    val snapshot = ContextSnapshotFactory.builder().build().captureAll()
    thread {
        snapshot.setThreadLocals().use {
            someFunc1()
        }
    }
}

fun someFunc1() {
    println("Called someFunc1 func. thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
}

By specifying a keyPredicate when calling setThreadLocals, you can filter which ThreadLocalAccessors will be active.
For example, if you want to activate only MDCAccessor, you can do this:

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())

    MDC.put("hoge", "hoge-value")

    val snapshot = ContextSnapshotFactory.builder().build().captureAll()
    thread {
        snapshot.setThreadLocals { key -> key == MDCAccessor::class.java }.use {
            someFunc1()
        }
    }
}

Propagating to Reactor Context

To propagate to Reactor Context using ContextSnapshot, you can do the following. ReactorContextAccessor needs to be registered.

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())
    val snapshotFactory = ContextSnapshotFactory.builder().build()

    MDC.put("hoge", "hoge-value")

    val snapshot = snapshotFactory.captureAll()
    Mono.delay(Duration.ofMillis(100))
        .flatMap { value ->
            Mono.deferContextual {
                // Propagated to the Reactor Context.
                println("thread=${Thread.currentThread().name}, mdcMap=${it.get<Map<String, String?>>(MDCAccessor::class.java)}")

                // It does not propagate to ThreadLocal.
                println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")

                // If you want to propagate it to ThreadLocal, write it like this again.
                snapshotFactory.captureFrom(it).setThreadLocals().use {
                    println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
                }

                Mono.just(value)
            }
        }
        // Propagate to the Reactor Context using the `updateContext` method.
        .contextWrite { snapshot.updateContext(it) }
        .block()
}

// Result
thread=parallel-1, mdcMap={hoge=hoge-value}
thread=parallel-1, mdc=null
thread=parallel-1, mdc={hoge=hoge-value}

However, in Reactor, a convenient method called contextCapture is available, so you can write it like this:

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())
    val snapshotFactory = ContextSnapshotFactory.builder().build()

    MDC.put("hoge", "hoge-value")

    Mono.delay(Duration.ofMillis(100))
        .flatMap { value ->
            Mono.deferContextual {
                // Propagated to the Reactor Context.
                println("thread=${Thread.currentThread().name}, mdcMap=${it.get<Map<String, String?>>(MDCAccessor::class.java)}")

                // It does not propagate to ThreadLocal.
                println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")

                // If you want to propagate it to ThreadLocal, write it like this again.
                snapshotFactory.captureFrom(it).setThreadLocals().use {
                    println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")
                }

                Mono.just(value)
            }
        }
        .contextCapture() // This
        .block()
}

... There's still quite a bit to write, making it inconvenient. Running Hooks.enableAutomaticContextPropagation() will automatically handle the following, making it significantly more convenient:

contextCapture() before block() (processing from thread local to Reactor context)
snapshotFactory.captureFrom(it).setThreadLocals() written in Mono.deferContextual (processing from Reactor context to thread local)

fun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())
    Hooks.enableAutomaticContextPropagation()

    MDC.put("hoge", "hoge-value")

    Mono.delay(Duration.ofMillis(100))
        .flatMap { value ->
            Mono.deferContextual {
                // Propagated to the Reactor Context.
                println("thread=${Thread.currentThread().name}, mdcMap=${it.get<Map<String, String?>>(MDCAccessor::class.java)}")

                // Propagated to both Reactor Context and ThreadLocal.
                println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")

                Mono.just(value)
            }
        }
        .block()
}

(Bonus) Kotlin Coroutines and Hooks.enableAutomaticContextPropagation

As mentioned earlier, using Hooks.enableAutomaticContextPropagation allows Mono.block and similar methods to automatically execute contextCapture(), but it does not work for Coroutine methods like awaitSingle.

ffun main() {
    ContextRegistry.getInstance().registerThreadLocalAccessor(MDCAccessor())
    Hooks.enableAutomaticContextPropagation()

    val job = GlobalScope.launch {
        MDC.put("hoge", "hoge-value")

        Mono.delay(Duration.ofMillis(100))
            .flatMap { value ->
                Mono.deferContextual {
                    // Not propagated to the Reactor Context.
                    println("thread=${Thread.currentThread().name}, mdcMap=${it.getOrDefault<Map<String, String?>>(MDCAccessor::class.java, null)}")

                    // Propagated to ThreadLocal.
                    println("thread=${Thread.currentThread().name}, mdc=${MDC.getCopyOfContextMap()}")

                    Mono.just(value)
                }
            }
            .awaitSingle()
    }
    runBlocking { job.join() }
}

// Result
thread=parallel-1, mdcMap=null
thread=parallel-1, mdc={hoge=hoge-value}

By defining the following class and registering it in the Service Loader, contextCapture will be automatically executed, just like with block. However, I am not entirely confident if this is the best approach yet.
(I have opened an issue about this, so if you're interested, please check it out → https://github.com/reactor/reactor-core/issues/3563)

package observation101.context.coroutines

import kotlinx.coroutines.InternalCoroutinesApi
import kotlinx.coroutines.reactive.ContextInjector
import org.reactivestreams.Publisher
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import kotlin.coroutines.CoroutineContext

@OptIn(InternalCoroutinesApi::class)
class ContextCaptureInjector : ContextInjector {
    override fun <T> injectCoroutineContext(publisher: Publisher<T>, coroutineContext: CoroutineContext): Publisher<T> {
        return when (publisher) {
            is Mono -> publisher.contextCapture()
            is Flux -> publisher.contextCapture()
            else -> publisher
        }
    }
}

// Write the following in `META-INF/services/kotlinx.coroutines.reactive.ContextInjector`.

observation101.context.coroutines.ContextCaptureInjector

Advanced Use Cases

Propagating Thread Local to Runnable/Callable

ContextSnapshot implements a method called wrap, which can be used for this purpose.

val snapshot = ContextSnapshotFactory.builder().build().captureAll()
val wrapped = snapshot.wrap {
    // ...
}

// Equivalent code to the above.
val snapshot = ContextSnapshotFactory.builder().build().captureAll()
val wrapped = Runnable {
    snapshot.setThreadLocals().use {
        // ...
    }
}

Propagating Thread Local to ExecutorService / ScheduledExecutorService

ContextExecutorService and ContextScheduledExecutorService are implemented for this purpose. You can use them.

Propagating Thread Local to Kotlin CoroutineContext

Kotlin Coroutines provide a mechanism called ThreadContextElement for Thread Local. Connecting this with Micrometer Context Propagation will make it work.

class ContextPropagationThreadLocalElement : ThreadContextElement<AutoCloseable>, AbstractCoroutineContextElement(KEY) {
    private val snapshot = factory.captureAll()

    override fun updateThreadContext(context: CoroutineContext): AutoCloseable {
        return snapshot.setThreadLocals()
    }

    override fun restoreThreadContext(context: CoroutineContext, oldState: AutoCloseable) {
        oldState.close()
    }

    companion object {
        private val KEY = object : CoroutineContext.Key<ContextPropagationThreadLocalElement> {}
        private val factory = ContextSnapshotFactory.builder().build()
    }
}

Conclusion

Since many Java ecosystems rely on Thread Local, using Reactor or Kotlin Coroutines can sometimes be challenging.

I originally wrote a similar library for work, but now that Micrometer provides an OSS solution, things have become much easier.

Additionally, as mentioned in this article, implementing in line with Micrometer Context Propagation makes it easy to integrate with Reactor and Kotlin Coroutines.

Previously, I had to write similar code separately for Reactor and Kotlin Coroutines, but Micrometer Context Propagation provides a good abstraction, allowing reuse.

Although this was quite a niche topic, I hope it is useful to someone.

How to Use Caffeine with Kotlin Coroutines - Introduction to caffeine-coroutines

Ryosuke Hasebe — Fri, 07 Feb 2025 12:37:49 +0000

Motivation

JVM has a widely used library for implementing caching called Caffeine.

https://github.com/ben-manes/caffeine

Since this library is written in Java, it cannot be used directly on Coroutines.

Caffeine provides Async support (an interface compatible with CompletableFuture), which makes it possible to use it on Coroutines with some effort. However, unless you have a deep understanding of both Caffeine and Coroutines, the learning curve can be quite steep.

In fact, there are questions on Stack Overflow regarding its usage with Coroutines, indicating that some people struggle with it.
https://stackoverflow.com/questions/71666175/what-is-the-preferred-way-how-to-add-caffeine-cache-to-kotlin-with-coroutines

To make it easy for everyone to use, I created a library called caffeine-coroutines.

https://github.com/be-hase/caffeine-coroutines

※ It has also been mentioned in the Caffeine README.

Basic Usage

First things first, let's add it to the dependencies.

implementation("dev.hsbrysk:caffeine-coroutines:{{version}}")

As for usage, there are almost no differences from the original Caffeine. The only new thing to learn is that by calling buildCoroutine, you can obtain a CoroutineCache instance that works on Coroutines.

suspend fun main() {
    val cache: CoroutineCache<String, String> = Caffeine.newBuilder()
        .maximumSize(10_000)
        .expireAfterWrite(Duration.ofMinutes(5))
        .buildCoroutine() // buildCoroutineを使う

    val value = cache.get("key") {
        delay(1000) // suspend functionが使えるようになる
        "value"
    }
    println(value)
}

Of course, the loading cache style is also supported. By passing a loader to buildCoroutine, you can obtain a CoroutineLoadingCache instance.

suspend fun main() {
    val cache: CoroutineLoadingCache<String, String> = Caffeine.newBuilder()
        .maximumSize(10_000)
        .expireAfterWrite(Duration.ofMinutes(5))
        .buildCoroutine { // buildCoroutineを使う
            delay(1000) // suspend functionが使えるようになる
            "value"
        }

    val value = cache.get("key")
    println(value)
}

Philosophy

Caffeine provides Cache and LoadingCache for blocking use cases, and AsyncCache and AsyncLoadingCache for async (CompletableFuture) use cases.

As the name suggests, this library focuses solely on providing CoroutineCache and CoroutineLoadingCache for Coroutines use cases.

The introduction of custom APIs/interfaces is kept to a minimum.

Introducing unique APIs/interfaces can confuse users, making adoption difficult and causing trouble when discontinuing usage.

Comparison with Aedile (Similar Implementations)

There is actually another Kotlin wrapper for Caffeine called Aedile.

However, when I implemented caffeine-coroutines, I encountered the following issues with Aedile:
(Many of these issues have been resolved in the recently released version 2, so using Aedile is also an option.)

Issues with Coroutine Scope Handling
- Ideally, it should execute in a scope derived from the caller's scope (otherwise, for example, the Coroutine Context will not be inherited), but for some reason, it used a separate scope.
- As a result, there were issues with inheriting things like MDC.
Issues with Coroutine Cancellation
- Although the scope handling issue mentioned above was addressed in later versions, there was an issue where an exception occurring inside the loader would cancel the calling Coroutine.
- This was fixed in version 2, but the fix seems a bit odd… (simply using the coroutineScope function to create a child scope should suffice).
Too Many Custom APIs
- To create a cache, you had to use a custom builder called caffeineBuilder.
- It’s easier to understand if you can use the official builder as is, and using the custom builder meant that some features available in the original implementation were unavailable.
- This has been resolved in version 2 by adopting a style similar to caffeine-coroutines.
- Additionally, metrics required using a custom API called CacheMetrics, which I also found problematic. This has also been deprecated in version 2.

Summary

caffeine-coroutines is a library that makes it easy to use Caffeine on Coroutines. By using the buildCoroutine extension function, you can utilize Caffeine’s features seamlessly in a Coroutines environment.

Although Aedile has improved, it previously had issues such as scope handling, cancellation behavior, and excessive custom APIs. caffeine-coroutines focuses on a simple design that leverages official APIs.

If you want to use Caffeine in a Coroutines environment, give it a try!

Visualizing Gradle Dependency Differences! Introducing "gradle-dependency-diff-action"

Ryosuke Hasebe — Wed, 29 Jan 2025 14:07:25 +0000

I’d like to introduce my GitHub Actions, gradle-dependency-diff-action. With this Action, you can easily check how Gradle dependencies change due to a Pull Request.

Motivation

Because Gradle resolves library dependencies transitively, unintentional dependency changes sometimes occur.

For example, let’s say you update a library called tink, as shown below. At first glance, it looks like just a minor update. When you get a Pull Request with such a change, you might quickly approve it, assuming there’s no issue.

 dependencies {
-    implementation("com.google.crypto.tink:tink:1.13.0")
+    implementation("com.google.crypto.tink:tink:1.14.0")
     implementation("com.google.protobuf:protobuf-java:3.25.1")
 }

However, tink actually depends on protobuf-java, and because of this update, the dependency changes as follows:
protobuf-java quietly goes from 3.25.1 to 4.27.0. In Gradle, when there’s a conflict in dependency versions, it basically adopts the latest version.

-+--- com.google.crypto.tink:tink:1.13.0
++--- com.google.crypto.tink:tink:1.14.0
 |    +--- com.google.code.findbugs:jsr305:3.0.2
 |    +--- com.google.code.gson:gson:2.10.1
 |    +--- com.google.errorprone:error_prone_annotations:2.22.0
-|    \--- com.google.protobuf:protobuf-java:3.25.1
-\--- com.google.protobuf:protobuf-java:3.25.1
+|    \--- com.google.protobuf:protobuf-java:4.27.0
+\--- com.google.protobuf:protobuf-java:3.25.1 -> 4.27.0

In this way, there can be “hidden differences” that aren’t apparent just by looking at the code diffs in the Pull Request.

I created gradle-dependency-diff-action to visualize these hidden differences.

Side note 1
- There was actually a bug involving backward incompatibility between protobuf-java 3.x and 4.x. As a result, many people encountered issues in cases like the example above.
- https://github.com/tink-crypto/tink-java/issues/31
Side note 2
- Recently, Gradle introduced something called “dependency locking,” which is somewhat like package-lock.json in npm. However, it doesn’t seem to be very popular. I tried it myself, and it felt somewhat inconvenient… (I won’t go into details here.)

What is gradle-dependency-diff-action?

This Action addresses the issue described above by doing the following for Pull Requests:

Detects any differences in the Gradle dependencies resulting from the Pull Request.
When differences are detected, you can choose how to be notified (configurable):
- Describe the differences in GitHub Checks
- Post a comment on the Pull Request
- Add a label to the Pull Request
- Upload the dependency differences to GitHub Actions Artifacts in both text and HTML format

The appearance of the PR:

The appearance of the Checks:

The HTML report generated in the actions artifact:

How to use

Apply the project-report plugin

Apply the project-report plugin to the project where you want to detect dependency differences.

plugins {
    //...
    `project-report` // HERE !
}

Write a workflow

First, try writing a workflow like the one below. It’s very easy to get started.

name: CI
on:
  pull_request:

jobs:
  dependencies-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 17
      - uses: be-hase/gradle-dependency-diff-action@v1

How does it get the dependency diff?

In short, by applying the project-report plugin, we can use the dependencyReport task. We run this task on both the base branch and the current branch, then take a diff of the outputs.

We use dependency-tree-diff to make the diff easier to understand.

Technically, we could have used the existing dependencies task instead of the dependencyReport task. However, the dependencies task doesn’t support multi-project setups, meaning we’d have to run it sequentially for each subproject, which would take a lot of time. Since the dependencyReport task supports multi-project setups, we chose to use it.

Summary

We introduced gradle-dependency-diff-action, a GitHub Action that visualizes unintended changes in Gradle dependencies.

Sometimes unexpected version updates can occur in Gradle, and those can lead to serious issues. By using this Action, you can easily check for dependency differences in each Pull Request, which will help during reviews.

Please give it a try!

Introducing Kuery Client for those who love writing SQL in Kotlin/JVM

Ryosuke Hasebe — Wed, 03 Jul 2024 07:25:50 +0000

First of all

Repository
- https://github.com/be-hase/kuery-client
Document
- https://kuery-client.hsbrysk.dev/

How can it be written?

I'd like to start with a preamble, but first, let me give you a quick overview.

suspend fun search(status: String, vip: Boolean?): List<User> = kueryClient
    .sql {
        +"""
        SELECT * FROM users
        WHERE
        status = $status
        """
        if (vip != null) {
            +"vip = $vip"
        }
    }
    .list()

You can concatenate and build SQL using + (unaryPlus)
- If you want to build SQL dynamically, please use constructs like if
- (Of course, if there is no need to build it dynamically, you can write it directly using a heredoc)
Use string interpolation to embed dynamic values
- Naturally, you might think this could lead to SQL injection, but by implementing a Kotlin compiler plugin, it is evaluated as a placeholder

Motivation

Originally, I liked MyBatis because I could write SQL by hand

In the world of JVM, there are many database client libraries, but I preferred using MyBatis.

The reasons for this preference are roughly as follows:

I want to write SQL directly
- Because it is used in a large-scale environment, I prefer to write SQL directly even if it takes a bit more effort
- Writing SQL makes it easier to perform operations like Explain
- Occasionally, there are cases where I want to specify an Index Hint, and I can respond quickly to such cases
I don't want to learn the library's unique syntax or DSL
- It would be nice if knowing SQL alone is enough

Writing SQL directly means depending on a specific database, but cases of migrating databases (e.g., from MySQL to PostgreSQL) are virtually non-existent. Even if such an opportunity arises, I would take the time to thoroughly verify it (compared to this, fixing SQL is not much effort), so I don't see this as much of a disadvantage.

MyBatis does not support R2DBC

Recently, I have had more opportunities to write applications using Spring WebFlux & Coroutines for work, and this makes me want to use R2DBC.
(Previously, I was wrapping JDBC calls with withContext(Dispatchers.IO) {...})

Unfortunately, the aforementioned MyBatis does not support R2DBC.

R2DBC libraries that allow writing SQL by hand

Existing libraries that support R2DBC often use unique syntax/DSL, which do not match my preferences.

On the other hand, these DSLs have the advantage of being type-safe. They prevent mistakes such as inserting a string into an integer column.

However, personally, I write corresponding unit tests when I write SQL, so I am indifferent to this aspect.

(Bonus) sqlc and SQLDelight

Recently, sqlc has been getting attention and seems to match my preferences very well. (I like the gRPC-like concept)

However, although it seems to support Kotlin, it unfortunately only supports JDBC.

Looking at the generated code, it seems to have only the minimum implementation compared to the Go support of sqlc.

Also, in the case of Kotlin, there is a similar tool called SQLDelight. While it claims to support R2DBC, it is difficult to use connection pools as the arguments are not ConnectionFactory, and its Transaction support is also inadequate, giving the impression that it is still in development.

Furthermore, neither supports constructing dynamic queries. (There are varying opinions on the desirability of such dynamic queries)

I want to write using string templates & string interpolation

If I'm writing SQL by hand, I also want to write using string templates & string interpolation like Doobie, which is popular in Scala.

def find(n: String): ConnectionIO[Option[Country]] =
  sql"select code, name, population from country where name = $n".query[Country].option

However, unfortunately, in Kotlin, you cannot customize the behavior of string interpolation.
(Although in Java, this has recently become possible...)

Incidentally, this topic has been discussed here:

Knowing Kotlin Compiler Plugin amidst all this

I originally focused on Kotlin/JVM, but recently I have become engrossed in KMP (Kotlin Multiplatform).
(The motivation is simple... it would be convenient if everything could be written in Kotlin...)

I noticed that libraries for KMP often provide features including Kotlin Compiler Plugin.
(Kotlin Serialization is a prime example)

I was already using the noarg plugin and allopen plugin, but I realized that even third parties are making them.

Perhaps, I thought, I could use this to change the behavior of string interpolation for specific methods...? And amidst all this, I came across the slides for the following presentation at Kotlin Fest 2024. (I couldn't attend due to a schedule conflict...)

https://speakerdeck.com/kitakkun/kotlin-fest-2024-motutokotlinwohao-kininaru-k2shi-dai-nokotlin-compiler-pluginkai-fa

As a result, I managed to create an SQL client that can be used as mentioned at the beginning.

How to Use Kuery Client

Like other libraries, just add it to your Gradle dependencies.
However, since a Kotlin Compiler Plugin is required, please also use the Gradle Plugin provided by Kuery Client.

plugins {
    id("dev.hsbrysk.kuery-client") version "{{version}}"
}

implementation("dev.hsbrysk.kuery-client:kuery-client-spring-data-r2dbc:{{version")

Features of Kuery Client

Builder and String Interpolation

In Kotlin, a style of creating a Builder Scope and constructing dynamically within it, as represented by buildString or buildList, is often adopted.

buildString {
    append("hoge")
    if (...) {
        append("bar")
    }
}

This writing style is also adopted in kotlinx.html.
Text nodes are added using +.

val body = document.body ?: error("No body")
body.append {
    div {
        p {
            +"Welcome!"
            +"Here is "
            a("https://kotlinlang.org") { +"official Kotlin site" }
        }
    }
}

Following this writing style, the style mentioned at the beginning is used. (Repost)

suspend fun search(status: String, vip: Boolean?): List<User> = kueryClient
    .sql {
        +"""
        SELECT * FROM users
        WHERE
        status = $status
        """
        if (vip != null) {
            +"vip = $vip"
        }
    }
    .list()

Based on spring-data-r2dbc or spring-data-jdbc

Currently, it is implemented based on these widely used and proven technologies. So, it can be used with both R2DBC/JDBC.
(The original motivation was to create it for R2DBC, but I decided to also support JDBC)

Depending on these for the requirements of Kuery Client may be somewhat too much, but it allows for using transaction support and type conversion as is...

(If I feel like it, I might create a module that doesn't depend on these)

Transaction

You can use Spring's Transaction support as is.

For more details, please see here.

https://kuery-client.hsbrysk.dev/transaction.html

Observation

It supports Micrometer Observation, so it can handle both Metrics and Tracing.

For more details, please see here.

https://kuery-client.hsbrysk.dev/observation.html

Type Conversion

Since it is based on spring-data, please use Spring's type conversion.
Even with custom types, it can be handled flexibly. It should be able to handle cases where only specific types need to be encrypted.

For more details, please see here.

https://kuery-client.hsbrysk.dev/type-conversion.html

Detekt Custom Rule

Writing in the following incorrect way may cause SQL Injection or similar issues.

kueryClient.sql {
    // BAD !!
    val sql = "SELECT * FROM user WHERE id = $id"
    +sql
}

Since the string interpolation is customized for specific methods of Kuery Client, such writing is not allowed.
To detect such incorrect writing, we provide a Detekt Custom Rule.

https://kuery-client.hsbrysk.dev/detekt.html

Example Code

Sample code combined with Spring Boot.

Spring WebFlux and R2DBC
- https://github.com/be-hase/kuery-client/tree/main/examples/spring-data-r2dbc
Spring WebMVC and JDBC
- https://github.com/be-hase/kuery-client/tree/main/examples/spring-data-jdbc

Conclusion

For more detailed information, please check the documentation site. (Although it's very simple at the moment...)
https://kuery-client.hsbrysk.dev/

I've already started using it for personal projects and find it quite convenient and enjoyable to use.

Looking ahead, I vaguely think it would be nice to integrate SQL-related linters since I'm writing SQL by hand.

Also, I want to implement something similar to the query type checks available in Scala's Doobie. (I haven't used it much, so I'm not very familiar yet)
https://tpolecat.github.io/doobie/docs/06-Checking.html

Although I wrote as I did in the aforementioned motivation, it is certainly better if things can be made robust.

On the other hand, DSLs have the advantage of being type-safe. They prevent mistakes such as inserting a string into an integer column.
However, personally, I write corresponding unit tests when I write SQL, so I am indifferent to this aspect.

A protoc compiler plugin that generates useful extension code for Kotlin/JVM

Ryosuke Hasebe — Wed, 29 May 2024 01:59:04 +0000

TL;DR

https://github.com/be-hase/protoc-gen-kotlin-ext

Generate *OrNull extension properties for optional field
- In protoc-gen-kotlin, this extension properties is only provided for message types. We provide it for scalar types as well.
- https://github.com/protocolbuffers/protobuf/issues/12935
Generate factory functions that resemble data class constructors
We do not generate our own code, so we can take advantage of the official ecosystem
- It only generates additional useful code.

Example

Let's assume we have the following proto file.

message Person {
  string first_name = 1;
  string last_name = 2;
  optional string middle_name = 3;
  Gender gender = 4;
  optional string nickname = 5;
  Address primary_address = 6;
}

enum Gender {
  GENDER_UNSPECIFIED = 0;
  MALE = 1;
  FEMALE = 2;
  OTHERS = 3;
}

message Address {
  string country = 1;
  string state = 2;
  string city = 3;
  string address_line_1 = 4;
  optional string address_line_2 = 5;
}

Using the generated code, you can write the following:

fun main() {
    // You can use factory functions that resemble data class constructors
    val person = Person(
        firstName = "Ryosuke",
        lastName = "Hasebe",
        middleName = null, // Optional fields become nullable
        gender = Gender.MALE,
        nickname = null, // Optional fields become nullable
        primaryAddress = Address(
            country = "JP",
            state = "Tokyo",
            city = "blah blah blah",
            addressLine1 = "blah blah blah",
            addressLine2 = null, // Optional fields become nullable
        ),
    )

    // You can use `*OrNull` extension properties for optional field
    println(person.middleNameOrNull)
    println(person.nicknameOrNull)
    println(person.primaryAddress.addressLine2OrNull)
    println(person.primaryAddressOrNull)
}

Motivation

Challenges with Pptional Field (Field Presence)

From protobuf 3.12, the long-awaited optional field (Field Presence) has been reintroduced. This allows the
representation of null (as in java/kotlin).

message Sample {
  optional string hoge = 2;
  optional int32 bar = 3;
}

However, this optional feature is somewhat tricky. When you retrieve a value with a getter, it returns the default
value (e.g., "" for strings, 0 for int32). This means that for strings, you cannot distinguish whether the value has
been explicitly set to "" or not set at all. Instead, optional fields provide a has* method to check if the value
has been set, which you should use to determine if it is null.

If you are unaware of this specification, you might mistakenly treat an "" or 0 as a valid value, potentially
causing bugs.

Sample.newBuilder().build().also {
    println("[1] hoge: ${it.hoge}")
    println("[1] hasHoge: ${it.hasHoge()}")
    println("[1] bar: ${it.bar}")
    println("[1] hasBar: ${it.hasBar()}")
}
Sample.newBuilder().setHoge("hoge").setBar(1).build().also {
    println("[2] hoge: ${it.hoge}")
    println("[2] hasHoge: ${it.hasHoge()}")
    println("[2] bar: ${it.bar}")
    println("[2] hasBar: ${it.hasBar()}")
}

[1] hoge:
[1] hasHoge: false
[1] bar: 0
[1] hasBar: false
[2] hoge: hoge
[2] hasHoge: true
[2] bar: 1
[2] hasBar: true

Even knowing this specification correctly would result in a large amount of boilerplate code, such as the following:

// 😭
val hoge = if (sample.hasHoge()) {
    sample.hoge
} else {
    null
}

To address this for Kotlin, we will auto-generate *OrNull extension properties. When implementing,
seeing *OrNull
through autocompletion should help avoid some issues. Additionally, this allows the use of the Elvis operator, resulting
in smoother code.

val BlahBlah.hogeOrNull: kotlin.String?
    get() = if (hasHoge()) hoge else null

Interestingly, for message types, using protoc-gen-kotlin already generates similar extension properties.
https://github.com/protocolbuffers/protobuf/blob/b30f3de12f946b6d610c21bc605726bf56ea889f/src/google/protobuf/compiler/java/message.cc#L1352C33-L1370

I have raised an issue requesting the addition of optional
scalar types, but it is not planned to be supported by protoc-gen-kotlin.

Challenges with Java Builder

Protobuf uses builders to set values. Since this code is written in Java, there is no non-null/nullable type checking.
When used from Kotlin, it is often misunderstood that it is okay to set null for optional fields, which results in
passing null and encountering NPEs (NullPointerExceptions).

Sample.newBuilder()
    .setHoge(null) // This code raises NPE 😭
    .setBar(1)
    .build();

When using protoc-gen-kotlin, the following DSL is generated for Kotlin. This is beneficial because it includes
non-null/nullable type checking. Great!

sample {
    hoge = null // This code results in a compile error 😀
    bar = 1
}

However, in Kotlin, it feels more natural to write using constructors with named arguments, like data classes.

Additionally, just like the builder, the DSL does not result in a compile error when a field is added later. This can
lead to cases where values are forgotten to be set. However, opinions may differ on whether this is seen as a
disadvantage or an advantage.

If it is code provided by libraries or similar, it is better not to do it as it may break
compatibility.

If it is code used within your own application, I think it is convenient. If you prioritize a more robust programming
style, it is preferable to have a compile error indicating that a value has been forgotten.

Therefore, we will auto-generate the following factory function. Note that this Sample(...) is not a constructor but a
function.
It can be written similarly to a data class, and when a field is added later, it will result in a compile error,
indicating that a value might be forgotten to be set. Since it is defined in Kotlin, nullable type checking is also
enabled.

// similar to a data class 😀
Sample(
    hoge = "hoge",
    bar = 1
)

// Of course, when it is optional, it becomes a nullable type
Sample(
    hoge = null, // This code results in a compile error 😀
    bar = 1
)

public fun Sample(hoge: String?, bar: Int?): Sample {
    val _builder = Sample.newBuilder().apply {
        hoge?.let { setHoge(it) }
        bar?.let { setBar(it) }
    }
    return _builder.build()
}

We considered adding factory functions with default arguments, but we decided not to create them at this time. If there
are default arguments, adding a field later would not result in a compile error.
If you want to use default values, you can use java builder or kotlin DSL.

How to use?

Gradle

When used with protoc-gen-kotlin:

protobuf {
    protoc {
        artifact = "com.google.protobuf:protoc:<version>"
    }
    plugins {
        id("kotlin-ext") {
            artifact = "dev.hsbrysk:protoc-gen-kotlin-ext:<version>:jdk8@jar"
        }
    }
    generateProtoTasks {
        all().forEach { task ->
            task.plugins {
                id("kotlin-ext") {
                    outputSubDir = "kotlin"
                }
            }
            task.builtins {
                id("kotlin")
            }
        }
    }
}

When used standalone:

protobuf {
    protoc {
        artifact = "com.google.protobuf:protoc:<version>"
    }
    plugins {
        id("kotlin-ext") {
            artifact = "dev.hsbrysk:protoc-gen-kotlin-ext:<version>:jdk8@jar"
        }
    }
    generateProtoTasks {
        all().forEach { task ->
            task.plugins {
                id("kotlin-ext") {
                    outputSubDir = "kotlin"
                    // This option is explained later in the document
                    option("messageOrNullGetter+")
                }
            }
        }
    }
}

Maven or `Manual protoc Usage`

Since the plugin is created using the same mechanism as protoc-gen-grpc-kotlin, please refer to this document.

Advanced

Compile Options

Perhaps you only want the *OrNull extension property and do not need the factory function. (and vice versa).

This can be achieved by setting the compile options.

// Here, we will use Gradle as an example.
protobuf {
    // ...
    generateProtoTasks {
        all().forEach { task ->
            task.plugins {
                id("kotlin-ext") {
                    // ...
                    // HERE
                    option("messageOrNullGetter+")
                }
            }
        }
    }
}

The following options are available. Appending + to the end of an option enables it, while appending - disables it.

factory
- Whether to generate a factory (default: on)
orNullGetter
- Whether to generate orNull extension functions for optional scalar fields (default: on)
messageOrNullGetter
- Whether to generate orNull extension functions for optional message fields (default: off)
- Not needed when using protobuf-kotlin.

When specifying multiple options, please separate them with commas. For
example, factory-, orNullGetter+, messageOrNullGetter+

test

Ryosuke Hasebe — Thu, 16 Nov 2017 02:50:34 +0000

test

Forem: Ryosuke Hasebe

Energy-Efficient Schema-Driven Development with Spring WebMVC/WebFlux

Introduction

Separating Traditional Controller Definitions into Interfaces

Generating OpenAPI Schemas with springdoc-openapi

Development Flow

Advanced Usage: Using with Spring HTTP Interface Client

Conclusion

[Kotlin] Using Mappie to Reassign Data Class Properties

Motivation

Overview of Mappie

You Can Also Explicitly Specify Mapping Rules

Expressing Mapping Errors

Handling Them as Exceptions

Handling Them with kotlin.Result

Conclusion

Important Notes on JUnit 5.12.0+ in Gradle

TL;DR

Why

Ref

When using an S3-compatible Object Storage, be cautious when upgrading SDK for Java 2.x to version 2.30.0 or later

TL;DR

Background

Cause

Support Status of Various S3-Compatible Object Storages

Miscellaneous

About Micrometer Context Propagation

Introduction

Why was Micrometer Context Propagation created?

What is Context Propagation?

How to Use Micrometer Context Propagation

ThreadLocalAccessor

Example: MDC

Example: Brave Trace Context

ContextAccessor

Example: Reactor Context

ContextRegistry

ContextSnapshot

Propagating to Reactor Context

(Bonus) Kotlin Coroutines and Hooks.enableAutomaticContextPropagation

Advanced Use Cases

Propagating Thread Local to Runnable/Callable

Propagating Thread Local to ExecutorService / ScheduledExecutorService

Propagating Thread Local to Kotlin CoroutineContext

Conclusion

How to Use Caffeine with Kotlin Coroutines - Introduction to caffeine-coroutines

Motivation

Basic Usage

Philosophy

Comparison with Aedile (Similar Implementations)

Summary

Visualizing Gradle Dependency Differences! Introducing "gradle-dependency-diff-action"

Motivation

What is gradle-dependency-diff-action?

How to use

Apply the project-report plugin

Write a workflow

How does it get the dependency diff?

Summary

Introducing Kuery Client for those who love writing SQL in Kotlin/JVM

First of all

How can it be written?

Motivation

Originally, I liked MyBatis because I could write SQL by hand

MyBatis does not support R2DBC

R2DBC libraries that allow writing SQL by hand

(Bonus) sqlc and SQLDelight

I want to write using string templates & string interpolation

Knowing Kotlin Compiler Plugin amidst all this

How to Use Kuery Client

Features of Kuery Client

Builder and String Interpolation

Based on spring-data-r2dbc or spring-data-jdbc

Transaction

Observation

Type Conversion

Detekt Custom Rule

Example Code

Conclusion

A protoc compiler plugin that generates useful extension code for Kotlin/JVM

Handling Them with `kotlin.Result`

Maven or `Manual protoc Usage`