Forem: Johnny Starr

How to publish a Micronaut project using Gradle

Johnny Starr — Sat, 17 Apr 2021 03:22:30 +0000

I have really enjoyed working with Micronaut. It's a wonderfully fast framework that makes life easier. This new framework is really easy to get up and running. Publishing on the other hand isn't well documented, so I wanted to share some tips on how I was able to get this done.

Overview

We will specifically be publishing a Micronaut app using Gradle. You will need to install the Micronaut CLI. You can grab it here: https://guides.micronaut.io/micronaut-cli/guide/index.html

Create a Micronaut App

Now that you've grabbed the CLI, create a new project. The Launcher button on the Micronaut site will help you with the CLI commands if you want specific packages. But for the sake of simplicity, we won't be installing anything special. Run the following command to build a demo app:

$ mn create-app --build=gradle --jdk=11 --lang=java --test=junit to.dev.publishing

Gradle Steps

If you're like me, you might have IntelliJ with the Gradle plugin installed. If so, this next part is a no brainer. By simply opening the project it will run Gradle in the background and generate your project. You should have a build.gradle file in the root of your project that looks something like this:

plugins {
    id("com.github.johnrengelman.shadow") version "6.1.0"
    id("io.micronaut.application") version "1.4.2"
}

version = "0.1"
group = "to.dev"

repositories {
    mavenCentral()
}

micronaut {
    runtime("netty")
    testRuntime("junit5")
    processing {
        incremental(true)
        annotations("to.dev.*")
    }
}

dependencies {
    implementation("io.micronaut:micronaut-http-client")
    implementation("io.micronaut:micronaut-runtime")
    implementation("javax.annotation:javax.annotation-api")
    implementation("io.micronaut:micronaut-validation")
    runtimeOnly("ch.qos.logback:logback-classic")
}


application {
    mainClass.set("to.dev.Application")
}
java {
    sourceCompatibility = JavaVersion.toVersion("11")
    targetCompatibility = JavaVersion.toVersion("11")
}

Running Gradle Build

Run the following command to build your artifacts:

$ ./gradlew build

> Task :compileTestJava
Note: Creating bean classes for 1 type elements

BUILD SUCCESSFUL in 11s
15 actionable tasks: 3 executed, 12 up-to-date

Your output may be slightly different depending on your dependencies and workstation.

Artifacts

To see the produced artifacts you can run:

$ ls ./build/libs/
publishing-0.1-all.jar
publishing-0.1-runner.jar
publishing-0.1.jar

Hmm... Interesting right? We have 3, so which one is should we ship to production? The short answer, is publishing-0.1-all.jar. But how do we publish it? And what are the other two JARs for? (we'll discuss this on another post)

Publishing

It took some time to understand the best publishing strategy for my current use-case. What I wanted were the following 3 things:

The primary JAR
A Zip archive of the distribution
A Tar archive of the distribution

Ok. Great... So how do we do that? Add the following lines to your build.gradle file:

publishing {
    publications {
        myDist(MavenPublication) {
            artifact shadowJar
            artifact distZip
            artifact distTar
        }
    }
}

And that's it! shadowJar produces our large publishing-0.1-all.jar. The zip/tar produce the full distribution which includes all the required JAR files for your project.

Publishing to Maven Locally "Trick"

So you're not quite ready to publish to Maven Central, Nexus3 or Artifactory. No biggie. You can use the following "trick" to publish to your local machine. This is useful if you want to inspect distribution you can delete your local ~/.m2 directory:

$ rm -rf ~/.m2

Afterwards, from your project repo run the following Gradle command:

$ ./gradlew publishMyDistPublicationToMavenLocal

Now you can inspect your "published" artifacts:

$ find ~/.m2
/Users/johnny/.m2
/Users/johnny/.m2/repository
/Users/johnny/.m2/repository/to
/Users/johnny/.m2/repository/to/dev
/Users/johnny/.m2/repository/to/dev/publishing
/Users/johnny/.m2/repository/to/dev/publishing/maven-metadata-local.xml
/Users/johnny/.m2/repository/to/dev/publishing/0.1
/Users/johnny/.m2/repository/to/dev/publishing/0.1/publishing-0.1-all.jar
/Users/johnny/.m2/repository/to/dev/publishing/0.1/publishing-0.1.pom
/Users/johnny/.m2/repository/to/dev/publishing/0.1/publishing-0.1.tar
/Users/johnny/.m2/repository/to/dev/publishing/0.1/publishing-0.1.zip

Takeaways

Micronaut is amazing and I love it. It's still quite new and documentation is hard. Getting up and running is fast, but publishing isn't super obvious at first glance.
Even without selecting it, our CLI command includes the Netty webserver. I think its pretty slick coming from a WAR/Tomcat background! Netty completely sidesteps the Servlet.
Gradle is awesome. As a UNIX expert my heart will always go out to to Make, but come on people its 2021!

I hope this helps save you some time. It took me a little bit to figure this one out.

Lessons from UNIX History: Avoiding the "second system effect"

Johnny Starr — Tue, 07 Apr 2020 18:39:34 +0000

This is the first article in my "Lessons from UNIX History" series. By day I am a DevOps Tech Lead, by night a computing archeologist. This indulgence allows me to express my findings with practical, modern day applications. Ideally, we can all benefit at present by taking a moment to reflect on the past.

This is more than history, it is our heritage as engineers.

Human Tendencies

As humans, we love upgrades and new features. In fact, where would we be without them? The software industry relies on continuous delivery of new content; which is fine generally speaking. But what happens when we get too attached to a system, product or application? This results in the "second system effect".

Second System Effect

This term is best explained by Brian Kernighan in his 2020 book "UNIX A History and a Memoir":

"... it's tempting to try to create a new system that fixes all the remaining problems with the original while adding everybody's favorite new features too. The result is often a system that's too complicated, a consequence of taking on too many different things at the same time ..."

This excerpt explains the challenges we face in today's industry quite well. But we do well to analyze the history just a bit deeper.

Multics to Unics (UNIX)

We owe a surprising amount of features and applications to the UNIX predecessor "Multics". Such as:

A shell
User authentication
Streams (output, redirection, file descriptors)
Commands like "ls" and "date"

Multics was a time-sharing operating system that was both feature rich and yet inherently flawed. So much so that General Electric (GE) withdrew funding during the late stages of development. Business aside, it was clear that the OS was too bloated (sound familiar?). This is where Ken Thompson decided to branch off to new territory. We're very happy he did, because Unics (later renamed UNIX) was born.

No doubt, he had invested a lot of time in Multics. Yet, he was emotionally mature enough to "walk away" from a dying system full of bloat. He would embrace a new way of thinking; albeit at the cost of some very favored features.

But that was 50+ years ago!

It amazes me how much history repeats itself. Really, this is just as much of a problem today. Computers have gotten smaller. Our operating systems larger. Our knowledge vaster, but deep down we have the same tendencies as our predecessors. In fact, it could be argued with continuous delivery, our necessity for favorite features have gone up, not down.

What can you do about it?

Ken Thompson was open minded about the future. He recognized the need to abandon pet-projects when they were overloaded. We need to leave behind legacy features and not be afraid to go back to the drawing board. (I realize the irony in stating that lessons from a "legacy" era help us move forward technologically).

If you are in the DevOps / systems engineering world, be open minded to what is possible today. At times, I find myself leaning on base knowledge of AWS or Docker. Due to the saturation of our industry, what was "impossible yesterday" is certainly "possible today". Don't be afraid to forge your own path. You just might invent something for the next generation!

Next Article "Avoiding Tool Sprawl"

Today, we are bombarded with hundreds of tools, packages and middleware that promises to solve all your problems. Of course, I think we can all agree, that FoSS is wonderful. But how many tools do we really need? Have you noticed how many files go into the root directory of your code repos? In the next article I will tackle this real world problem, and what we learn from the past yet again.

What DevOps Engineers can learn from Extreme Programming (XP)

Johnny Starr — Tue, 08 Oct 2019 16:34:18 +0000

Is it a stretch to say that modern day DevOps folks can learn something from a bygone movement? Absolutely not. Although XP has a bad reputation for some specific reasons, it contains some principles that are of value to us today in 2019.

YAGNI

This acronym stands for "You aren't gonna need it". This is one of the pillars of XP. It makes the assertion that you shouldn't spend time on components or features that you might need down the line. In modern development languages like GoLang, we see the application of this principle is inherit in Go's design. A practical example of this, is Get/Set methods not being idiomatic. For purest Java DEVs, this can be troubling at best. Yet, modern platforms such as Kubernetes and Docker we're written in GoLang. A cursory review of the source-code on Github reveals that the engineers hardly follow a dogmatic OOP standard.

I can't say how many times I've tried to implement code for my systems that "I might need later" only to realize that new technology has lapped us yet again. This can be frustrating, as you never seem to get to the gold-plating stage. If we are able to apply YAGNI, we can save valuable time and effort. But what happens when the time comes to make the changes? Simple, we refactor.

Continuous Refactoring

Refactoring is often neglected in the DevOps realm. This is because that "Ops" portion of your title is always going to be concerned about stability. And lets face, we're feature oriented. At the same time, many engineers fail to understand what "refactoring" truly means:

Code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior. - Wikipedia

One of the misunderstandings is that "refactoring" gets confused with "gold plating" or new features. Instead, our efforts should be on the following:

Optimization
Increased Readability
Increased Maintainability
Documentation

Test Drive Development (TDD)

So let me be clear: You cannot continuously refactor without continuous testing. If you change your code often without tests, it will lead to unmanageable code and defects. This can be challenging for engineers who "haven't gotten around to writing tests yet."

TDD ensures that you write your tests, because it requires that you write them first! TDD works like so:

Write a test for a unit of work
This test will automatically fail because this code does not yet exist
Write the code to pass the test

I have found TDD to work well with IaC projects such as Chef, Puppet, Ansible and friends.

TDD is not synonymous with XP, but it was largely embraced by it.

Peer Coding

Ok, this one is very controversial. I've heard many arguments about the practicality of a group of developers attacking a problem all at once. Like all things, you don't have to follow XP to perfection to get some benefit out of peer coding.

Working with distributed teams can be a challenge, but you shouldn't have to wait until you open a PR to have someone else take a look at your code. I have found that 1-2 hours a week can be well spent in an interactive screenshare session. Remember that diversity is great for problem solving. Having more than one mind solving an issue will unlock potential in all involved. In my opinion, the best possible scenario for team building, is team coding!

Final Thoughts

I'm sure there's still folks out there that love XP and use it daily. As a DevOps engineer, I'm fascinated with Dev practices and better understanding the platforms that others have built in the past. I strongly believe that each methodology is built on principles that have their place in the modern world. There is a lot more to XP than I've covered, and I don't claim to be an expert. I just wanted to share what has worked for myself and my team.

A sensible semantic versioning process for Docker images

Johnny Starr — Mon, 07 Oct 2019 22:22:30 +0000

I'm a big fan of Ruby. I've literally used it almost every day since 2008 when I picked up the "hummingbird" book (v1.8.7). Naturally, I am likely to have a Rakefile at the root of all my repositories. GNU Make has its place, but Rake is always the clear winner in my book.

I've noticed that CI/CD pipelines for Docker images can be challenging for newcomers, so I wanted to share a best practice for bumping versions in a codified manner. And no, I don't mean the goto VERSION text file. This method works of course, but if you want something more sophisticated read on.

To get started, you'll need Ruby installed and create the following two files:

.version.yml
Rakefile

In our .version.yml file, let's add some content

---
version: 0.0.1
promote: major
phase:         # optional

Our intention here is to cook up an initial release version 1.0.0. The value we're passing for the version doesn't really matter, as long as its below major version 1. The promote: key is going to determine the semantic bump that we plan to use. Now let's create our Rakefile:

require 'semantic'
require 'yaml'

def next_version
  # load the current version
  version_yml = YAML.safe_load(File.read('.version.yml'))
  current     = version_yml['version']
  level       = version_yml['level'].to_sym
  phase       = version_yml['phase']

  # create a forecast of what our future version will be
  version = Semantic::Version.new current
  version = version.increment!(level)
  "#{version}.#{phase}"
end

task :bump do
  sh "docker tag myproject:#{next_version}" # => myproject:1.0.0
end

So what exactly is happening here? We've defined a method in the #Kernel namespace that we can employ throughout our Rake tasks. It reads the YAML file and provides the appropriate bump to our version. The :bump task simple runs the docker tag command, but could be used in conjunction with any other release mechanisms.

Putting it to Work

In order for this to be useful to you and your team, you will need to discuss your release strategy. Ideally, you should only bump your semantic versions on merges to master. Depending on your CI/CD process, you would only run the :bump task during the release phase.

Benefits

Rake is more than it appears at first glance. It's not simply a Makefile with some Ruby features. It is Ruby; or rather a Ruby environment. By putting Rake to good use, you avoid mangled Makefiles with pseudo-bash scripting baked in.

I also view this as a great mechanism for peer reviews / pull requests. It allows the team visibility on what you're planning to release and why your chosen promotion level is appropriate (or inappropriate). Simply dropping off a number in VERSION doesn't provide the same stimulus in my experience.

The YAML File

Why YAML and not JSON or something else? Meh. I say do what you want! I chose YAML because its human readable and ubiquitous in the DevOps realm lately. You can also add additional fields at any time without breaking functionality. In the above example I left phase: blank. This is great for any modifiers such as pre-release, alpha, beta, rc1, etc.

Requirements

Unlike the yaml gem, you will need to install semantic yourself. Either use bundler or simply run the following command:

$ gem install semantic

Additionally, this gem (and of course Ruby) will have to be installed on your CI/CD agent(s). I would suggest using Bundler with a Gemfile to accomplish this. If you're concerned about keeping your agents pristine, read up on installing the gems locally to the vendor/ folder.

Final Thoughts

The DevOps frontier is still very "wild west". I'm sure someone will invent a tool that handles versioning in a more succinct way, but what would the fun be in that?