Generate Kotlin client for a complex web API

Maria Krol — Thu, 09 Feb 2023 17:36:01 +0000

Automation for many routines starts with interaction via API. This case can be treated in many ways, but I want to take a look at interacting with web API using a generated Kotlin client.
I found a lot of academic examples for generating Kotlin clients (most of them are based on the PetStore), but nothing was close to real-world examples.
My goal is to build a Kotlin client for a complex API and see how it works.
Here is the source code of the Kotlin project that I will use in the post.

Domain

I chose the TeamCity API to play with. It is huge and complex, and also contains models with names that clash with built-in Java classes.

What is the goal?

A client has generated using the Kotlin language.
The build task of the project depends on the generation of the client.

Preparation

Install TeamCity

All the examples have shown in this post are performed on a TeamCity installed on my machine.
The installation process is fairly straightforward, and specific settings for the TeamCity installation process are beyond the scope of this article.

Take a look at the API

On the official TeamCity website we can find a lot of documentation about the API. In addition, we can see the automatically generated sources and find a path to the endpoint where the API schema is stored in Swagger format (/app/rest/swagger.json).
In the list of generated models, we can find types that have names that conflict with Java's built-in.

TeamCity API Type	Java Type
File	java.io.File
Type	java.lang.reflect.Type

Failed to create a client for Java. The generator imports Java's types instead of TeamCity's.
There are bugs described for the Java client in both the Swagger generator and the OpenAPI generator. Let's see how the generator behaves when building a Kotlin client.

Generate Kotlin client

Generate the client with the default settings of the generator

The source code

Create a project

Let’s start with the creation of a project. We have used the basic template of the project with Kotlin language and Gradle as a build system.

We need to add the API Schema file to the project's resources:

Choose a Generator

We have two different generators to choose from:

The main difference is described on the official page of the OpenAPI generator:

What is the difference between Swagger Codegen and OpenAPI Generator?
Swagger Codegen is driven by SmartBear while OpenAPI Generator is driven by the community. More than 40 top contributors and template creators of Swagger Codegen have joined OpenAPI Generator as the founding team members. For more details, see the Fork Q&A.
Swagger is a trademark owned by SmartBear and the use of the term "Swagger" in this project is for demo (reference) purposes only.

Both generators do not work with the TeamCity API out of the box. For the OpenAPI generator, all problems are infrastructure related: The generator produces invalid Kotlin code with some combinations of libraries, and some other combinations produce valid code, but the environment (HTTP clients or serializers) cannot be fine-tuned. The Swagger generator produces invalid Kotlin code not related to libraries or settings: for example, variable names were generated with dashes.
So let's go with the OpenAPI generator.

Generation process

We need to add the OpenAPI plugin to Gradle. (plugin page in MavenCentral)

build.gradle.kts



plugins {
   kotlin("jvm") version "1.8.0"
   id("org.openapi.generator") version "6.3.0"
}

Now we need to set up the task of the generator:

select the language;
pass path to file with API specification;
pass path where to put generated sources;
select names of packages with generated code.

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set("$buildDir/generated")
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
}

We need to run the target task in Gradle to check the result:

The process fails, we get the error Out of memory. Java heap space:

Gradle uses a default value of 512 Mb available for the Java heap. This is not enough for the OpenAPI generator in IDEA to build the client. We can change the size of the Java heap as well as other Gradle parameters in the gradle.properties file.
I use a very simple selection algorithm:
Every time I fail with the error, I increase the size of the heap to twice the size. 4Gb is not enough. You can play with the heap size between 4Gb and 8Gb to select a smaller value if needed.

gradle.properties



org.gradle.jvmargs=-Xmx8192m

Now the process is finished successfully!

Build the project

We need to add the generated files to the list of source paths in order for them to be included in the compilation:

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set(generatedSourcesPath)
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
}

kotlin.sourceSets["main"].kotlin.srcDir("$generatedSourcesPath/src/main/kotlin")

Now we need to add a dependency of the Kotlin compilation task on the generation of the client. The compilation process must be done after the generation:

build.gradle.kts



tasks.withType<KotlinCompile>().configureEach {
   dependsOn("openApiGenerate")
}

Let's run the build process and observe the errors (required dependencies are not included). Go through the errors and add the dependencies:

Unresolved reference: okhttp3 → add from mvnrepository
Unresolved reference: com.squareup.moshi → add from mvnrepository (everything needed to build a generated client is in this package)

build.gradle.kts



dependencies {
   testImplementation(kotlin("test"))

   // OpenAPI client
   implementation("com.squareup.okhttp3:okhttp:4.10.0")
   implementation("com.squareup.moshi:moshi-kotlin:1.14.0")
}

The second pass of the build is successful.

Calling the API from the client

Calling the API methods from the client is required to verify that the client works at runtime. I have chosen two endpoints to call: User and Project.
To make it easier, let's add a small wrapper that will create a basic HTTP client and set it up initially:

BaseApiClient.kt



open class BaseApiClient {
   companion object {
       const val host = ConfigurationProvider.teamCityHost
       val baseClient: OkHttpClient = getApiClient()

       private fun getApiClient(): OkHttpClient {
           return OkHttpClient.Builder()
               .build()
       }
   }
}

The TeamCity Web API is protected by authorization. The OkHttpClient builder does not know anything about authorization. OkHttpClient can be extended with interceptor, so we need to add our custom to pass authentication data.
The correct way of authorization is a separate complex topic and it is beyond the scope of this post. I have implemented the simplest, but not the most appropriate way: TeamCity provides the ability to generate authorization tokens with long expiration, I created a token and use it to authorize my requests:

This static token is passed to the interceptor without refreshing and so on:

AuthorizationInterceptor.kt



class AuthorizationInterceptor : Interceptor {
   override fun intercept(chain: Interceptor.Chain): Response {
       val newRequest = chain.request().signedRequest()
       return chain.proceed(newRequest)
   }

   private fun Request.signedRequest(): Request {
       return this.newBuilder()
           .header("Authorization", "Bearer ${ConfigurationProvider.token}")
           .build()
   }
}

Pass it to the client:

BaseApiClient.kt



private fun getApiClient(authorizationInterceptor: AuthorizationInterceptor): OkHttpClient {
   return OkHttpClient.Builder()
       .addInterceptor(authorizationInterceptor)
       .build()
}

Now we can wrap any endpoint. Let's start with the UserAPI and create methods to create, get, and delete a user:

UserApiWrapper.kt



class UserApiWrapper : BaseApiClient() {
   private val userApi: UserApi = UserApi(host, baseClient)

   fun createUser(): User {
       val newUser = User(
           username = "userName".appendRandomNumericPostfix(),
           name = "name".appendRandomNumericPostfix(),
           email = "${("fakeMail".appendRandomNumericPostfix())}@example.com",
           password = "pass".appendRandomNumericPostfix()
       )

       return userApi.addUser(body = newUser)
   }

   fun getUser(userName: String) : User {
       return userApi.getUser(userName)
   }

   fun deleteUser(user: User) {
       userApi.deleteUser(user.username!!)
   }
}

Call the API from a dummy test:

The user is successfully created. Let's continue with a TeamCity project:

ProjectsApiWrapper.kt



class ProjectsApiWrapper : BaseApiClient() {
   private val projectApi: ProjectApi = ProjectApi(host, baseClient)

   fun createProject(): Project {
       val newProject = NewProjectDescription(name = "simple_tc_project".appendRandomNumericPostfix())

       return projectApi.addProject(newProject)
   }

   fun getProject(projectName: String): Project {
       return projectApi.getProject(projectName)
   }
}

The execution fails with the error Platform class java.io.File requires explicit JsonAdapter to be registered:

This happens because the Moshi library is used for serialization, and it cannot handle collections without explicitly registered adapters. This bug has already been reported in the repo.
Since we do not create instances of Moshi that do the serialization (the generator does it under the hood), we cannot register adapters.

Fix the generated client by changing settings

We are faced with the problem described above when using a client generated with default settings. Now we need to check the settings of the generator to use something else for serialization.
The OpenAPI generator has a lot of different settings. But most interesting for us now is

library - This setting is used to control a whole set of libraries used to generate a client.
serializationLibrary - This setting controls only one serialization library.

So, we can go in two different ways:

use the setting serializationLibrary to replace the Moshi library with something else and continue to use OkHttp for HTTP client
use the setting library and replace the whole group of libraries

Use setting ‘serializationLibrary’ to replace the Moshi library

First I try to replace the serialization library. The setting ‘serializationLibrary’ has 3 different values:

The first one is not suitable for the current case, so let’s play with the two remaining.

Use `gson` as serialization library

To use the gson library, we need to add the dependency for it and change the settings of the task of the OpenAPI generator:

build.gradle.kts



implementation("com.google.code.gson:gson:2.10.1")

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set(generatedSourcesPath)
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
   configOptions.set(mapOf("serializationLibrary" to "gson"))
}

The process of building this configuration goes successfully, we can start testing and see the green result!

Use `jackson` as a serialization library

Now I try to do the same for the jackson library. Here are the dependency and the settings of the generator:

build.gradle.kts



implementation("com.fasterxml.jackson.module:jackson-module-kotlin:2.14.2")

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set(generatedSourcesPath)
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
   configOptions.set(mapOf("serializationLibrary" to "jackson"))
}

The build and test processes have also been successfully completed!

Use setting `library` to replace the whole set of libraries required for the generation process

The library setting has many possible values. Each value describes the platform, library for HTTP client, and library for serialization:

Setting	Platform	HTTP Client	JSON processor
jvm-ktor	Java Virtual Machine	Ktor	Gson, Jackson
jvm-okhttp4	Java Virtual Machine	OkHttp	Moshi
jvm-okhttp3	Java Virtual Machine	OkHttp	Moshi
jvm-retrofit2	Java Virtual Machine	Retrofit	Moshi
multiplatform	Kotlin multiplatform	Ktor	Kotlinx Serialization
jvm-volley	JVM for Android	Volley	gson
jvm-vertx	Java Virtual Machine	Vert.x Web Client	Moshi, Gson or Jackson

The setting jvm-okhttp4 which is used by default was covered by the previous paragraph. Since the platform, I use for the playground is JVM, we cannot try to use jvm-volley.
The remaining cases:

jvm-ktor
multiplatform
jvm-vertx

Using `jvm-ktor` group to generate: invalid Kotlin code

The source code

I start with jvm-ktor. The official documentation says that it uses jackson for serialization.

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set(generatedSourcesPath)
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
   configOptions.set(mapOf("library" to "jvm-ktor"))
}

Build the project. We can find errors regarding dependencies (we did not add a required set), and invalid Kotlin code: classes that describe endpoints of the API pass non-existent parameters to the superclass:

So, that case is useless for Kotlin clients now.

Using the `multiplatform` group to generate: native Kotlin variant

The source code

Now let’s try to use the multiplatform group of libraries:

build.gradle.kts



val generatedSourcesPath = "$buildDir/generated"
val apiDescriptionFile = "$rootDir/src/main/resources/teamCityRestApi-v2018.1-swagger2.0.json"
val apiRootName = "com.makrol.teamcity.api.client"

openApiGenerate {
   generatorName.set("kotlin")
   inputSpec.set(apiDescriptionFile)
   outputDir.set(generatedSourcesPath)
   apiPackage.set("$apiRootName.api")
   invokerPackage.set("$apiRootName.invoker")
   modelPackage.set("$apiRootName.model")
   configOptions.set(mapOf("library" to "multiplatform"))
}

In case we build the project, we would see only errors related to missing dependencies and not Moshi in this list. Looks good. So let's add the dependencies. (io.ktor group in mvnrepository):

build.gradle.kts



implementation("io.ktor:ktor-client-core:$ktorVersion")
implementation("io.ktor:ktor-client-content-negotiation:$ktorVersion")
implementation("io.ktor:ktor-serialization-kotlinx-json:$ktorVersion")

Now we need to update the infrastructure. We have the interceptor for OkHttp authorization. Ktor allows us to use several different engines (as well as OkHttp), but for authorization purposes it is better to use the Auth plugin.

The constructor of the endpoint class gets a builder as a parameter, let's prepare the required one using the authorization plugin:

BaseApiClient.kt



protected val setupConfig: (HttpClientConfig<*>) -> Unit = { config -> getClientConfig(config) }

private fun getClientConfig(config: HttpClientConfig<*>) {
   config
       .install(Auth) {
           bearer {
               loadTokens { BearerTokens(ConfigurationProvider.token, refreshToken = "") }
           }
       }
}

Ktor generates asynchronous code, so all functions that call the API are marked with the suspend keyword:

UserApiWrapper.kt



class UserApiWrapper : BaseApiClient() {
   private val userApi = UserApi(host, httpClientConfig = setupConfig)

   // ****

   suspend fun createUser(): User {
      val newUser = User(
          username = "userName".appendRandomNumericPostfix(),
          name = "name".appendRandomNumericPostfix(),
          email = "${("fakeMail".appendRandomNumericPostfix())}@example.com",
          password = "pass".appendRandomNumericPostfix()
      )

      return userApi.addUser(body = newUser).body()
   }

   // ****
}

We get runtime errors while trying to run tests:

Error SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". occurred because Ktor depends on SLF4J in runtime
Error Fail to prepare request body for sending. The body type is: class com.makrol.teamcity.api.client.model.User (Kotlin reflection is not available), with Content-Type: null. occurred because we did not add content negotiation and did not pass content-type header

To fix the first one we need to add dependency. (see in mvnrepository):

BaseApiClient.kt



implementation("org.slf4j:slf4j-simple:$slf4jVersion")

To fix the second one, we need to configure content negotiation through the plugin and add the content type header.
So we need to update the config builder:

BaseApiClient.kt



private fun getClientConfig(config: HttpClientConfig<*>) {
   config.install(DefaultRequest) {
       header("Content-Type", "application/json")
   }
   config
       .install(Auth) {
           bearer {
               loadTokens { BearerTokens(ConfigurationProvider.token, refreshToken = "") }
           }
       }
   config.install(ContentNegotiation) {
       json()
   }
}

We are faced with a kotlinx.serialization.SerializationException.

If we read the kotlinx documentation, we would find that we need to install a corresponding plugin (see in the mvnrepository). Let's add it:

build.gradle.kts



plugins {
   kotlin("jvm") version "1.8.0"
   kotlin("plugin.serialization") version "1.8.10"

   id("org.openapi.generator") version "6.3.0"
}

It works!

Using `jvm-vertx` group to build: authorization is not supported

The last option available is jvm-vertx. According to the documentation, the stability of the generator is "beta", but it does not support authorization yet. Since the API we are playing with is secured, the option is not available.

Conclusion

As you can see, to get a successfully generated Kotlin client, you have to spend some time and play a bit with the settings.
The OpenAPI generator has rich configuration capabilities and it helps to solve a lot of problems, but there is no silver bullet and the settings I used to get a usable Kotlin client for TeamCity API may be useless in some other cases.
By changing the generator settings, you can change a whole set of libraries used to generate a client (option library) or you can change only the serializer (option serializationLibrary).
HTTP client was successfully built using Ktor and OkHttp. Vert.x Web Client does not support authorization for now, so it cannot be used for the API explored in the post.
JSON serialization has been successfully handled by kotlinx, gson, and jackson.
Decisions about which libraries to use must be made in the context of a task and project.
Using the multiplatform group (with Ktor and Kotlinx under the hood) seems most appropriate if the entire codebase of the project is written in Kotlin. Also, Ktor has a bit more flexible customization.

Forem: Maria Krol

Generate Kotlin client for a complex web API

Domain

What is the goal?

Preparation

Install TeamCity

Take a look at the API

Generate Kotlin client

Generate the client with the default settings of the generator

Create a project

Choose a Generator

Generation process

Build the project

Calling the API from the client

Fix the generated client by changing settings

Use setting ‘serializationLibrary’ to replace the Moshi library

Use `gson` as serialization library

Use `jackson` as a serialization library

Use setting `library` to replace the whole set of libraries required for the generation process

Using `jvm-ktor` group to generate: invalid Kotlin code

Using the `multiplatform` group to generate: native Kotlin variant

Using `jvm-vertx` group to build: authorization is not supported

Conclusion

Resources

Forem: Maria Krol

Generate Kotlin client for a complex web API

Domain

What is the goal?

Preparation

Install TeamCity

Take a look at the API

Generate Kotlin client

Generate the client with the default settings of the generator

Create a project

Choose a Generator

Generation process

Build the project

Calling the API from the client

Fix the generated client by changing settings

Use setting ‘serializationLibrary’ to replace the Moshi library

Use gson as serialization library

Use jackson as a serialization library

Use setting library to replace the whole set of libraries required for the generation process

Using jvm-ktor group to generate: invalid Kotlin code

Using the multiplatform group to generate: native Kotlin variant

Using jvm-vertx group to build: authorization is not supported

Conclusion

Resources

Use `gson` as serialization library

Use `jackson` as a serialization library

Use setting `library` to replace the whole set of libraries required for the generation process

Using `jvm-ktor` group to generate: invalid Kotlin code

Using the `multiplatform` group to generate: native Kotlin variant

Using `jvm-vertx` group to build: authorization is not supported