Forem: David Denton

Companion Objects: Kotlin's most unassuming power feature

David Denton — Fri, 26 Feb 2021 16:15:48 +0000

In which I argue the case for Kotlin's Companion Objects being the Clark Kent of language features, in being deceptively powerful, and how we can use them in creating ways to supercharge our API design.

TL;DR

"Companion Objects are not just placeholders for constants and other static state; they allow for new strategies to help organise our code or to provide reusable factory and validation features."

Of all the features that developers enthuse about in the Kotlin world, one of the ones that you hear about least is the
humble Companion Object. On the face of it, they are merely a convenient stand-in for where you could put static state.
But to dismiss them as such is prematurely writing them off - there are several very interesting use-cases that we have
discovered for them.

Don't believe me? I'd be disappointed if you did 😉. Let's dive in with a few simple examples, all based around the Java's
trusty LocalDate class.

API extension points

A basic (and arguably the most boring) ability that Companion Objects give us is as extension points for growing
collections of similarly themed functions or values. If you define a simple concept or abstraction that will be used and
reused in your system, I'd encourage the addition of a companion object onto the class/interface definition. This
provides both a place to attach extensions which are not relevant to the core concept in all scenarios and avoids
muddying the waters... imagine how intimidating it is for a Developer (or yourself having written it!) to come across an
interface with twenty functions instead of two!

This also allows developers in other parts of your own or another code module to define new implementations of said
abstraction to have a place to collect/organise them - the IDE will pick up all of these extensions and offer them to
you through autocompletion, instead of them being strewn all over the code.

We can see this with the (somewhat contrived) example below - we define a simple Validation predicate and attach
various extension functions/implementations/properties to it. Note here that the fun interface is a Kotlin feature which converts a single function interface to a Java SAM type and allowing us to create one without the full declaration:

fun interface Validation : (LocalDate) -> Boolean {
    companion object
}

val Validation.Companion.future get() = Validation { it.isAfter(LocalDate.now()) }

fun Validation.Companion.between(start: LocalDate, end: LocalDate) =
    Validation { it.isAfter(start) && it.isBefore(end) }

// for this data...
val jan1 = LocalDate.of(2021, 1, 1)
val dec31 =  LocalDate.of(2021, 12, 31)

// we can use the Validation like so...
val isFalse = Validation.future(jan1)
val thisYear = Validation.between(jan1, dec31)
val isTrue = thisCentury(jan1)

Parsers/Factories

We also can use the Companion Objects to provide a place to ensure correct construction of micro-types from other
formats (these can also, as above, be external to the a base class as appropriate). In the example below we have a
simple value wrapper for LocalDate. But we also want to be able to parse, validate and show the value correctly from our
strange ISO compliant format (YYYY-DDD):

data class BirthDate(val value: LocalDate) {
    init {
        require(value.isBefore(LocalDate.now()))
    }

    companion object {
        private val format = ISO_ORDINAL_DATE
        fun parse(unchecked: String): BirthDate = BirthDate(LocalDate.parse(unchecked, format))
        fun show(date: BirthDate): String = format.format(date.value)
    }
}

val birth = BirthDate.parse("1976-244")
val string = BirthDate.show(birth) // prints "1976-244"

Vary your programming model!

One problem with the above approach is that traditional parse/require pattern blows up with an exception when a validation error occurs - standard object construction techniques don't give us a chance to apply more functional programming models such as Result/Either monads to our domain.

In these models, we actively try to avoid Exceptions - to help us here we can privatise the BirthDate constructor and write a function to capture the error into a Result4k type which we can then map/flatMap() over:

data class BirthDate private constructor(val value: LocalDate) {
    companion object {
        fun asResult(unchecked: LocalDate) = when {
            unchecked.isBefore(LocalDate.now()) -> Success(BirthDate(unchecked))
            else -> Failure("illegal date!")
        }
    }
}

val birth: Result<BirthDate, String> = BirthDate.asResult(LocalDate.of(1999, 12, 31))

If we want to plug in a different result monad (say Arrow's Either) or to return null on failure, it is trivial to add extension functions to cover these types. The companion object is giving us options...

Extract and reuse!

What a lot of developers don't appreciate is that the Companion has exactly the same capabilities as any other Kotlin objects - and this includes traditional inheritance from classes or interfaces.

Rewinding back to the simpler Exception-based parse example, we can extract common code to a superclass or delegate which can be reused. Thinking this one step further, we realise that any extension functions added to the superclass/interfaces will then automatically be added to our Companion Objects as well!

Here we have extracted out a common superclass DateValueFactory for all "Date wrapper" classes - and each of the functions on this class are now inherited by both OrderDate and DeliveryDate (via their Companions). As mentioned before, we have also added an extension function to all DateValueFactory instances for the Result4k construction case:

abstract class DateValueFactory<T>(private val buildFn: (LocalDate) -> T) {
    fun parse(unchecked: String) = buildFn(LocalDate.parse(unchecked))
}

class OrderDate(val value: LocalDate) {
    companion object : DateValueFactory<OrderDate>(::OrderDate)
}

class DeliveryDate(val value: LocalDate) {
    companion object : DateValueFactory<DeliveryDate>(::DeliveryDate)
}

fun <T> DateValueFactory<T>.asResult(unchecked: String) = resultFrom { parse(unchecked) }

val order = OrderDate.parse("2000-01-01")
val delivery = DeliveryDate.asResult("2099-12-31")

Summary

I've covered only a couple of use-cases above which I've come across in the last few years of using Kotlin, but this is bound to be the tip of the iceberg and there are bound to be a bunch more just waiting to be discovered. The realisation of these abilities has reinforced my belief that we should be scratching beneath the surface in Kotlin features to see what it possible.

As with a lot of inventions, it's possible that even JetBrains didn't really appreciate the depth which would be unlocked by adding such a humble feature - I'd be fascinated to learn of any of these types of uses in the Kotlin standard libraries.

PS.

For a practical example of how the these type-creation techniques are used the real world, you can take a look at the foundational Values4k library, which uses them to provide instantiation, validation, parsing and printing of immutable value types.

Enjoyed this post? You can read all of my tech writings at https://dentondav.id/writing

Smash your Adapter Monolith with the Connect pattern

David Denton — Mon, 08 Feb 2021 17:11:09 +0000

In which I describe a pattern for writing third party adapters in a modular and extensible way, hoping that it is
original enough for me to christen it. 🙃

TL;DR

"Breaking down monolithic remote API adapters into individual Actions sharing a common single-function Protocol interface allows them to not only be more decoupled at the code layer, but also simplifies both testing and extensibility."

Reader Note: whilst this post is about a pattern that is not Kotlin-specific, it does demonstrate the pattern using code written in Kotlin. This uses various language features such as Data classes, Companion Objects, Operator overloading and Extension functions. It also uses the basics of the http4k HTTP toolkit which might be worth referring to if unfamiliar with them.

The main bulk of non-operationally focussed application code in a modern Server-based HTTP microservice can be broken
down into a few broad areas:

Inbound Server-side APIs, routing and unmarshalling of incoming requests
Business logic functions
Data-access querying and mutations
Adapter code for outbound remote API communication

Structuring our inbound APIs

For 1) - the Server-side - we tend to model the application as a set of separate HTTP entrypoint classes/functions which are composed into a whole to represent the incoming HTTP API, either explicitly or via some meta-programming such as annotations. So for example, using http4k, we might create and start our server with:

fun MySecureApp(): HttpHandler =
    BearerAuth("my-very-secure-and-secret-bearer-token")
        .then(
            routes(
                echo(),
                health()
            )
        )

fun echo() = "/echo" bind POST to { req: Request -> Response(OK).body(req.bodyString()) }

fun health() = "/health" bind GET to { req: Request -> Response(OK).body("alive!") }

val server = MySecureApp().asServer(Netty(8080)).start()

In this case, the splitting up of the server-side API into separate functions allows us to maintain a decent grip on our
application as a whole and also to be able to easily test the various endpoints in the application independently of
the rest - e.g. we don't need to provide a Bearer token to access our API calls if we have access to directly test echo() and health().

Additionally, because we have modularised the code in this way, it is also reusable in other contexts - we can put common endpoint code such as health() into a shared location and reuse them across our fleet of microservices.

Structuring our outbound APIs

When it comes to 4) of the list above - adapter code for other remote APIs - we don't generally have a pattern in place to use the same structure. HTTP adapters to remote systems are usually constructed as monolithic classes with many methods, all built around a singularly configured HTTP adapter. Let's say we want to talk to the GitHub API, we would normally build an API adapter like so:

 class GitHubApi(client: HttpHandler) {
    private val http = SetBaseUriFrom(Uri.of("https://api.github.com"))
        .then(SetHeader("Accept", "application/vnd.github.v3+json"))
        .then(client)

    fun getUser(username: String): UserDetails {
        val response = http(Request(GET, "/users/$username"))
        return UserDetails(userNameFrom(response), userOrgsFrom(response))
    }

    fun getRepoLatestCommit(owner: String, repo: String) = Commit(
        authorFrom(
            http(Request(GET, "/repos/$owner/$repo/commits").query("per_page", "1"))
        )
    )
}

val gitHub: GitHubApi = GitHubApi(OkHttp())
val user: UserDetails = gitHub.getUser("octocat")

This is all quite sensible - there is a shared HTTP client which is configured to send requests to the API with the correct Accept header. Unfortunately though, as our usage of the API grows, so will the size of the GitHubApi class - it may gain many (10s or even 100s of individual) functions, all of which generally provide singular access to a single API call. We end up with a monolith object which can be thousands of lines long if left unchecked.

As there is generally no interaction between these functions - it would be desirable to structure the code in a similar way to how we structured our inbound API - in a modular, easily testable and reusable fashion. Even so, we also want to find a way to build functions which combine one or more calls to the API.

Introducing the Connect pattern

This is where the Connect pattern will help us. In essence, Connect allows the splitting of an adapter monolith into individual Actions and a shared Protocol object which centralises the communication with the API.

The pattern itself has been created around the features available in the Kotlin language - most notably the use of interfaces and extension functions. Other languages may not have these exact same facilities, but Connect should be adaptable (to greater or lesser effect). Let's split it down and take a look by reimplementing the example above.

The following explanation is based upon a simplified version of the http4k-connect library, which we're using as the canonical implementation of the pattern. As the name implies, http4k-connect is itself built upon the http4k HTTP toolkit for it's core HTTP abstractions, although there is nothing in the pattern to tie it to this library (or even to the HTTP protocol).

Action

The fundamental unit of work in the Connect pattern is the Action interface, which represents a single interaction with the remote system, generified by the type of the return object R. Each action contains the state of the data that needs to be transmitted, and also how to marshall the data within the action to and from the underlying HTTP API.

For our GitHubApi adapter, we create the superinterface and an implementation of an Action to get a user from the API along with the result type. Note that the Action and result types are modelled as Kotlin data classes - this will give us advantages which we will cover later:

interface GitHubApiAction<R> {
    fun toRequest(): Request
    fun fromResponse(response: Response): R
}

data class GetUser(val username: String) : GitHubApiAction<UserDetails> {
    override fun toRequest() = Request(GET, "/users/$username")
    override fun fromResponse(response: Response) = 
        UserDetails(userNameFrom(response), userOrgsFrom(response))
}

data class UserDetails(val name: String, val orgs: List<String>)

Adapter

The Adapter interface represents the common base protocol for interacting with the remote API - it will deal with server host location, authorisation and other headers, and perform the actual HTTP interactions. Each Adapter is modelled as a simple interface with a single generic method accepting the generic Action type.

Note here the presence of the Kotlin companion object - it is meant to be empty and is there precisely to give us a point to hook other code onto in a moment. This is to make life easier for the API user.

interface GitHubApi {
    operator fun <R : Any> invoke(action: GitHubApiAction<R>): R

    companion object
}

Our first usage of the companion object is to rewrite our previous version as an anonymous implementation of the GitHubApi and attach it to our Adapter, returned by the Http() factory function. All dependencies required by the Adapter are passed in here and closed over. Note that we explicitly pass in the HTTP client instead of constructing it inside the function - access to this is critical if we want to be able to decorate the client Adapter with call logging or other operational concerns:

fun GitHubApi.Companion.Http(client: HttpHandler) = object : GitHubApi {
    private val http = SetBaseUriFrom(Uri.of("https://api.github.com"))
        .then(SetHeader("Accept", "application/vnd.github.v3+json"))
        .then(client)

    override fun <R : Any> invoke(action: GitHubApiAction<R>) = 
        action.fromResponse(http(action.toRequest()))
}

Using the adapter

Apart from the usage of the Companion Object as a hook, construction of our Adapter looks similar to the previous version - we have not exposed any more concrete types (there is still just GitHubApi). However, calling the API does look different - because of the operator function invoke(), we now treat the Server as a simple function which takes Action instances:

val gitHub: GitHubApi = GitHubApi.Http(OkHttp())

val user: UserDetails = gitHub(GetUser("octocat"))

This change may leave a slight bad taste in the mouth as the API is no longer as IDE discoverable. Luckily, Kotlin has another trick up it's sleeve here which will help us...

Extension Functions

We can get back our old API very simply by creating another extension function for each Action that mimics the signature of the Action itself and delegates to the invoke() call in the client:

fun GitHubApi.getUser(username: String) = invoke(GetUser(username))
fun GitHubApi.getRepoLatestCommit(owner: String, repo: String): Commit = 
    invoke(GetRepoLatestCommit(owner, repo))

val user: UserDetails = gitHub.getUser("octocat")

Even better, for actions which consist more than one API call such as getLatestUser() below, we can just create more extension functions which delegate down to the individual actions. These functions can be added to GitHubApi instances at the global level, or just in the contexts or modules which make sense. The extension function effectively allow us to compose our own custom GitHubApi Adapter out of the individual Action parts that we are interested in:

fun GitHubApi.getLatestUser(owner: String, repo: String): UserDetails {
    val commit = getRepoLatestCommit(owner, repo)
    return getUser(commit.author)
}

val latestUser: UserDetails = gitHub.getLatestUser("http4k", "http4k-connect")

Testing the Connect pattern

Both the Adapter and the modularisation of the various Action classes make it very easy to write unit tests for the action code created using the Connect pattern, but it's also important to consider how the API design will affect the testing of client code.

Fortunately, the simplicity of the single arity functional Adapter interface in concert with the Actions being implemented as Kotlin data classes (which are easily comparable) make testing as a client of Connect APIs trivial at multiple levels. Consider for instance if we are intending to mock a function which has seven parameters that we don't care about - in the previous implementation we would have to mock out each of those with a value (or an any() matcher), versus a single any<Action>() covering the Connect version:

@Test
fun `get user details`() {
    val githubApi = mockk<GitHubApi>()
    val userDetails = UserDetails("bob", listOf("http4k"))
    every { githubApi(any<GetUser>()) } returns userDetails

    assertThat(githubApi.getUser("bob"), equalTo(userDetails))
}

The Action object being a single parameter of the invoke() method also gives us the ability to easily decorate our Adapter instance for testing or other purposes, for instance we can record all of the incoming calls for any purpose we want:

class RecordingGitHubApi(private val delegate: GitHubApi) : GitHubApi {
    val recorded = mutableListOf<GitHubApiAction<*>>()
    override fun <R : Any> invoke(action: GitHubApiAction<R>): R {
        recorded += action
        return delegate(action)
    }
}

val recorder = RecordingGitHubApi(GitHubApi.Http(JavaHttpClient()))
recorder.getUser("bob")

println(recorder.recorded)

Writing stub implementations of an Adapter is also very simple, and the Connect pattern also encourages the same type of decomposed structure as with the real adapter - by creating extension functions which act on our in-memory state to create their responses. Once again, this helps to keep a grip on the size of the Adapter code.

class StubGitHubApi(private val users: Map<String, UserDetails>) : GitHubApi {
    override fun <R : Any> invoke(action: GitHubApiAction<R>): R = when (action) {
        is GetUser -> getUser(action, users) as R
        is GetRepoLatestCommit -> getRepoLatestCommit(action) as R
        else -> throw UnsupportedOperationException()
    }
}

private fun getUser(action: GetUser, users: Map<String, UserDetails>) = users[action.username]
private fun getRepoLatestCommit(action: GetRepoLatestCommit) = Commit(action.owner)

Varying the programming model

Depending on the style of team, there are several different popular programming models which may be commonly found out in the wild, and this will affect the value of the R type implemented for the Action classes.

As in our example above, traditional OO-style teams using languages which embrace the throwing of Exceptions will represent R as the straight result type returned by the method, but teams that adopt a more Functional Programming approach will tend towards using a more monadic return type such as Result4k's Result, Arrow's Either, or (when it is generally available) Kotlin's built in Result type.

The good news is that due to the decoupling of the Connect abstractions, any of these models can be supported simply by writing Actions in the relevant style. Here is an alternative example for the GetUser action using the Result4k monad:

interface GitHubApiAction<R> {
    fun toRequest(): Request
    fun fromResponse(response: Response): Result<R, Exception>
}

data class GetUser(val username: String) : GitHubApiAction<UserDetails> {
    override fun toRequest() = Request(GET, "/users/$username")
    override fun fromResponse(response: Response) = when {
        response.status.successful -> 
            Success(UserDetails(userNameFrom(response), userOrgsFrom(response)))
        else -> Failure(RuntimeException("API returned: " + response.status))
    }
}

Summary

The Connect pattern combines simple abstractions to provide a model that allows us to break down the common problem of the monolithic outbound API adapter into easily digestible parts. This modularity provides a mirror image of the composability that we expect when building inbound Server-side interfaces, and this further leads to a more testable and extensible codebase. With a small example such as this there is the potential for the approach seeming like overkill, but experience tells us that it is generally much harder to retrofit a compositional design than it is to promote it from the outset.

Initially designed around HTTP, the pattern will fit any request/response protocol and can easily be adapted to different programming models including Result monads and Future types.

Finally, although not crucial to the implementation of the Connect pattern, more advanced programming languages with features such as extension functions (such as Kotlin) provide an ideal platform for working with Connect. In statically typed languages, sufficiently advanced generic capabilities are the only required language feature.

Note: The code shown in this post is available in GitHub.

Footnote on the http4k-connect implementation of the pattern

The Open Source http4k-connect Kotlin libraries provide both the basic framework for implementing Connect pattern adapters, but also a set of pre-built API adapters for communicating with popular cloud services such as AWS. Further, http4k-connect provides a set of protocol-compatible In-Memory/Runnable Fake Servers which can be used as test-doubles for the various services, and a set of Storage backends (such as In-Memory, S3 and Redis) for test-data to be housed.

The libraries are designed to be as lightweight as possible, meaning they are a perfect use-case for Serverless deployments, They use compile-time code-generation to automatically write extension functions for each of the implemented Actions using Kapt, and ships without the need for reflection in JSON message parsing by also generating message adapters for the Moshi JSON framework with the Kotshi plugin.

Enjoyed this post? You can read all of my tech writings at https://dentondav.id/writing or find out more about http4k.