Forem: Hidde Wieringa

A dozenal clock

Hidde Wieringa — Sat, 08 Oct 2022 20:19:14 +0000

Digital clocks around the world use 24 hours each day to display the time. In the night the day starts as 00:00, 00:59, 01:00, 01:59, 02:00, up to 23:00 and 23:59. (Actually, digital time goes up to 24:00 when the day has a leap second.)

Analog clocks display the night starting at 12 (top of the clock). After the large hand has traveled all the way from 12, 1, 2, ..., 11 and back to 12, an hour has passed and the small hand points at 1. This repeats until afternoon, when the same time displays as the middle of the night and both hands point to the 12. After another 12 hours it is night again and the day has passed.

Some countries use AM and PM to denote the time. There the first hour of the day is 12PM (in the night), then 1AM, 2AM, until 12AM (end of the morning), which turns to 1PM, 2PM etc. until 12PM in the night. This notation follows the analog clock.

Questions

For a long time I have wondered why clocks work this way. I asked myself the following questions:

Why is the fist hour of the day named 12 PM, and the following hours start at 1, 2, 3, etc.?
Why does the afternoon not start at 10:00 (when the first digit in the hours changes from 0 to 1)?
Why do we have 60 minutes in an hour, but are analog clocks divided into 12 sets of 5 minutes?
If the digital time is 13:11, why can't I find these numbers on an analog clock?
Both 10:00 and 22:00 point at the same number on an analog clock. Why don't these times look similar?
Why does an analog clock have three numbers on the clock which have two digits, while the other numbers have one digit?

These questions got me thinking about time, the division of hours and minutes, and the representation of hours and minutes on a digital and analog clock.

Introduction to base-12

Instead of writing numbers in base 10 (decimal system), we will write numbers in base 12, the dozenal system.

To do that we need two new symbols to represent the decimal numbers 10 and 11.

Following the worldwide society that uses base-12 or duodecimal numbers (British, American), we shall use the ↊ and ↋ symbols. They are pronounced dek and el. (You may also find the Greek symbols χ and ξ used for dek and el, but the ↊ and ↋ symbols have been officially accepted into Unicode since 2015.)

The first twelve numbers are then: 1, 2, 3, 4, 5, 6, 7, 8, 9, ↊, ↋ and 10.

The digits ↊ and ↋ are easly and uniquely and recognizably representable on a seven-segment display (used in many digital clocks). They can be respresented as 🯲 and 🯳, but turned around 180 degrees. The 🯲 is the same upside down, and the top bar can be removed to make it unique.

There is even a LaTeX package dozenal that supports dozenal numbers. Wikipedia has a comprehensive article about the dozenal system and its history.

A new way to write the time

We will use the dozenal number system to create a better representation for clocks, both digial and analog.

For digital clocks, the following system is used:

The first digit denotes the night/morning as 0 and the afternoon/evening as 1.
The second digit denotes the hour, 0 up to ↋.
The hours and minutes are separated by a ·, although : is also accepted.
The third digit denotes the number of 5-minute groups in the hour, 0 up to ↋.
The fourth digit denotes the minute in the 5-minute group, 0 up to 4.

The second and third digits can be emphasized either by font size, or by font weight (bold font), although it is not required.

A whole day then looks as the following on a digital clock:

The first minute is 00·00.
The second minute is 00·01.
The fifth minute is 00·04.
The sixth minute is 00·10.
... and so on until the last minute of the first hour is 00:↋4.
The first minute of the of the second hour is 01:00.
The last minute before noon is 0↋:↋4.
Noon is 10:00.
The last minute of the day is 0↋:↋4.
A leap second would be represented as 20:00.

For analog clocks, the numbers from 0 to ↋ are spaced evenly around the clock. The number 0 is at the top.

For analog clocks we have the following times:

🕛 is 00·00
🕧 is 00·60
🕐 is 01·00
🕜 is 01·60
🕤 is 09·60
🕙 is 0↊·00
🕦 is 0↋·60

Note that seconds are left out of these examples, but they can be appended and represented the same way as the minutes are represented.

Think about the representation for a bit. What is often denoted as AM / PM can now be seen as the first digit. The morning and evening times 0↊·00 and 1↊·00 are represented as almost the same time. Skimming the time, it is usually only neccesary to look at the second and third digit instead of all four digits. The quarters of the hour can be identified by the digits 0, 3, 6 and 9.

In practice

The theory is nice, but I decided to put the theory into practice as well in my life, in as many places as possible.

For my digital watch (a Garmin device, with programmable clock faces) I made a Garmin watch face that displays the dozenal time. View the app on Connect IQ.

The wall in my office has a self-made dozenal analog clock. It has been laser-cut from a sheet of steel.

My laptop runs Ubuntu and its system clock is visualized in my task bar using a dozenal clock.

There are other places where I might implement dozenal clocks to replace the decimal clocks, for example:

my mobile phone
my alarm clock
system-wide date formatting, (I tried, see my question on Unix StackExchange)

Let me know if you have more ideas!

(Java)Money and Kotlin

Hidde Wieringa — Sat, 17 Apr 2021 14:55:57 +0000

The JavaMoney API is part of JSR 354 and will (hopefully) be part of the Java language in the future. The reference implementation Moneta provides good examples of implementations using the JavaMoney API.

The JavaMoney API allows working with currencies, monetary values and conversion between monetary values of different currencies through exchange rates. View the Devoxx video which explains the API in depth.

Kotlin is interoperable with compiled Java libraries. So the JavaMoney/Moneta library can be used from Kotlin code, and makes working with money in business logic easy. After all, money seems trivial but details like rounding, conversion and catalogues of locales and currencies are hard to implement correctly.

The JavaMoney API is very flexible and extendable. The classes and interfaces provide a Builder pattern to contruct instances with fluent code. For example

// Construct a monetary value of €200 using the FastMoney implementation
Monetary.getAmountFactory(FastMoney.class)
        .setCurrencyUnit("EUR")
        .setNumber(200)
        .setContext(MonetaryContextBuilder.of().set(MathContext.DECIMAL128).build())
        .create();

The builder pattern does not look like ideomatic Kotlin. In Kotlin, builder DSLs are used instead to fluently construct complex objects.

That is exactly what I implemented in Money-Kotlin.

Money-Kotlin

The Money-Kotlin library extends the JavaMoney API with Kotlin extension functions.

The example above could be rewritten in Kotlin as

// Construct a monetary value of €200 using the FastMoney implementation
200.ofCurrency<FastMoney>("EUR", monetaryContext {
    set(MathContext.DECIMAL128)
})

More details are given below. Some of the (Java) examples are taken from the Moneta User Guide.

Monetary amounts

The first example displayed above shows how to construct monetary values. Often we do not need a monetary context.

Java:

Monetary.getAmountFactory(FastMoney.class)
        .setCurrencyUnit("EUR")
        .setNumber(200.01)
        .create();

Kotlin:

val money = (200.01).ofCurrency<FastMoney>("EUR")

We can manipulate money amounts as well, just like regular numbers. This has been implemented using Kotlin operator overloading.

Java:

MonetaryAmount money = ...;

// add
money.add(money);
money.plus();

// subtract & negate
money.subtract(money);
money.negate();

// multiply
money.multiply(2.0);

// divide & remainder
money.divide(2.0);
money.remainder(2.0);

Kotlin:

// add
money + money
+money

// subtract & negate
money - money
-money

// multiply
money * 2.0
2.0 * money

// divide & remainder
money / 2.0
money % 2.0

Currencies

You can access a default or self-implemented currency (like Bitcoin) from the Monetary singleton.

Java:

CurrencyUnit currencyEUR = Monetary.getCurrency("EUR");

Kotlin:

"EUR".asCurrency()

Locales

The JavaMoney API associates a locale with a currency. For example, in The Netherlands the official currency is the Euro. Some countries have multiple currencies.

Java:

new Locale("", "NL");
Monetary.getCurrency(new Locale("", "NL"));
Monetary.getCurrencies(new Locale("", "NL"));

Kotlin:

"NL".asCountryLocale()
"NL".asCountryLocale().getCurrency()
"NL".asCountryLocale().getCurrencies()

Rounding

A monetary value can be rounded in a specific way. Construct a rounding query to get a MonetaryRounding.

Java:

Monetary.getRounding(
    RoundingQueryBuilder
        .of()
        .setScale(4)
        .set(RoundingMode.HALF_UP)
        .build()
    );

Kotlin:

Monetary.getRounding(roundingQuery {
    setScale(4)
    set(RoundingMode.HALF_UP)
})

Currency conversion

A MonetaryAmount can be converted from one currency to another currency through a CurrencyConversion. A CurrencyConversion depends on a conversion provider. There are some default providers implemented out of the box, like the European Central Bank (ECB) and International Monetary Fund (IMF). These providers provide currency exchange rates, in particular historic rates.

The order of the default providers are configurable, and you can implement your own conversion provider as well.

Java:

ExchangeRateProvider rateProvider = MonetaryConversions.getExchangeRateProvider("ECB", "IMF");

MonetaryAmount amountInEUR = ...;
MonetaryAmount amountInCHF = amountInEUR.with(rateProvider.getExchangeRate("EUR", "CHF"));

// Using the default rate providers
MonetaryAmount amountInCHF = amountInEUR.with(MonetaryConversions.getExchangeRateProvider().getExchangeRate("EUR", "CHF"));

Kotlin:

val rateProvider = MonetaryConversions.getExchangeRateProvider("ECB", "IMF");

val amountInEUR = ...
val amountInCHF = amountInEUR.convertTo("CHF", rateProvider)

// Using the default rate providers
val amountInCHF = amountInEUR.convertTo("CHF")

Formatting

Monetary amounts are formatted in the context of a locale.

Java:

MonetaryAmountFormat format = MonetaryFormats.getAmountFormat(Locale.GERMANY);
MonetaryAmount amount = ...;
format.format(amount);

Kotlin:

val amount = ...
val format = Locale.GERMANY.monetaryAmountFormat()
format.format(amount);

Getting started

You can use the Money-Kotlin library today! It has been published on Maven Central, and can be added as a dependency.

There are many more examples in the project readme.

Gradle:

implementation("nl.hiddewieringa:money-kotlin:$moneyKotlinVersion")

Maven:

<dependency>
  <groupId>nl.hiddewieringa</groupId>
  <artifactId>money-kotlin</artifactId>
  <version>${moneyKotlin.version}</version>
</dependency>

Feedback, feature requests and bug reports are welcome!

Ruby devs, how do you work?

Hidde Wieringa — Mon, 09 Mar 2020 19:27:06 +0000

This post is more like a request for discussion thread than a normal 'informational' post. First some history.

Daily work

In the company I work at, we use many different languages, but mostly based on the Java (JVM) ecosystem (Java, Groovy, Kotlin, etc.) and Ruby ecosystem (Rails, etc.). Most of the teams work with one language/ecosystem, but once in a while we collaborate on some code work in a different language than our daily work. That is fun, challenging and very good for learning new patterns and code styles.

Observations

I usually work within the JVM ecosystem (mostly Kotlin), and feel very confortable there. The standard library is great, there is a lot of learning content available for such a new language, and it is possible to build on the shoulders of a giant (Java) in terms of libraries and performance.

Whenever I collaborate and write Ruby code, I feel a little bit powerless. The first thing I do when I open a project which is not my daily code base, I scroll around the directories and files, and look around.

Then, when we start working on the problem at hand, we open the corresponding files and start editing. The first thing I do is Ctrl + click two times:

Find all the usages of the class/function we are working in (get to know the context of the work);
Find the definition of a referenced class or variable (what are we working with).

This does not work in Ruby (and many other interpreted languages), as any piece of code can call any other piece of code. Even methods (Ruby does not seem to have properties) can be added dynamically. Things like dependency injection are not really a thing (http://weblog.jamisbuck.org/2008/11/9/legos-play-doh-and-programming).

This leads me to the following set of questions I would like to ask the community.

Questions

How do you find out about usages or definitions of models in code that you did not write, know little about but have to refactor for some reason?
Apart from test coverage, how do you make sure you do not break anything when changing something else? Even with test coverage: do you unit test every property call?
How do you store knowledge in the code about some model, like which properties mean what, which are required during construction, which must exist together etcetera? Ruby is object-oriented to its core (https://www.ruby-lang.org/en/about), so encapsulation is important. I fail to understand how to encapsulate something that any other piece of code can extend or modify.

Hopefully this lets me get a better feeling on how developers can work with interpreted languages like Ruby. Thanks in advance for the discussion!

Creating a custom cycling map from open data

Hidde Wieringa — Mon, 09 Sep 2019 19:22:49 +0000

During the past month, I have created a specific style for generating cycling maps, using only publicly available data.

Goal

In July and August, I was on holyday in Poland, Slovakia and Hungary. For finding the way in those countries, I had a map (good old paper). I was cycling from camping to camping, so a good map was required. Unfortunately, Poland and Hungary had no good maps made for the purpose of cycling. That means that sometimes I could not find the way, and needed to use a mobile phone (mostly using the app Maps.Me with data of OpenStreetMap).

This problem stayed in my head. I am very enthusiastic about projects like OpenStreetMap, which collects and publishes the data for free (license: ODbL). The data quality is usually great, and the map details are far greater than for example Google Maps, Apple Maps or TomTom. Also, the data is updated very quickly if something changes, all by volunteering contributors.

The goal of this project is as follows: “Generate a printable map of any region in the world, using only publicly available data and tools, usable for cycling.” The process described in the following sections is the end result of this project. It took some weeks to learn about the technology, tooling and data. Then, by trial and error, the resulting map was improved step by step.

Stack

To reach the project goal, some tooling is needed. Fortunately, many people have gone before me that have a lot of knowledge about generating maps. I collected this knowledge from blogs, guides and websites.

For this project, I used the following stack:

Linux Subsystem for Windows. My main machine is Windows, and most of the tools do not run on Windows. Also, command-line scripting is not that easy in Windows. The command-line, apt, and the Linux ecosystem is essential for getting and running the needed tools.
Mapnik 2.x. C++ and Python bindings are available, so it is more a library than a program. There is a newer version of Mapnik (3.x), but the Python bindings do not exist or have no documentation (or both).
Python 2.7. For Mapnik 2.x the bindings are for Python 2.7. This is pre-installed in every mainstream Linux installation.
GDAL. These libraries are used for generating the contour lines and hillshade for the height data. There is a command-line interface with very good online documentation.
Postgres (with GIS extensions). A Postgres database has extensions (PostGIS) which allow loading spatial data (with locations), and generating efficient indices for querying spatial data. This is essential for processing the data quickly from the database to the map.

Getting the data

The data comes from two places. The content of the map (roads, features, land use, etc.) comes from OpenStreetMap. That data has been refined (sorted, filtered and grouped) by GeoFabrik. This means that ambiguity of some data is removed, and the data is labeled in a structured way. Also, they publish a nice document describing the structure of the data. For each country (and sometimes part of the country) a separate zip file must be downloaded.

The other required data is height data. This data is used for generating contour lines and ‘hillshading’ (if there is a steep hill, it is colored a darker shade on one side). The height data of the world is available from a number of different sources, but I get it from here. The data is measured by NASA satellites of the Shuttle Radar Topography Mission (SRTM) project.

Finally, I use the OpenStreetMap data a second time. This time, it is downloaded from another source. For the borders of countries, the GeoFabrik data is not usable (their free service doesn’t have this data, but their premium service does). The boundaries are downloaded from wambachers-osm.website instead.

Processing the data

A common way for spatial data (any data with a location attached to it) to be formatted is shapefiles. A shapefile describes vector data, a point, line or polygon, possibly with attributes. For example, a shapefile containing roads will contain lines with the shapes of the roads, and attributes with the name of the road and the speed limit. The lines can be used on the map, and the attributes can be used to style the data differently for each type of road. Each set of data has multiple files, each with a different responsibility. Most GIS applications and scripts can work with shapefiles.

For our use case shapefiles would work, but they are limited in their performance and possibilities. For example, if I want to create a map that spans multiple areas or countries, I would have to change the script to add more data sources. That is why all shapefiles are imported into a Postgres database. Shapefiles with the same kind of data are imported into the same table, and can be queried once to find all the data of that type for the map.

The height data also has to be processed. The height data does not come as a shapefile (it does not have a vector shape), but rather as a raster file. It can be seen as a large grid of ‘pixels’ having some value. In addition to the value, each pixel can have attributes, just like shapefiles. The SRTM3 data has a file per latitude and longitude degree squared.f

The height data is processed in two ways. First of all, the contour lines are generated from the height raster. The GDAL libraries are used for this. The interval of the contours can be specified (I use every 20 meters). The result is a shapefile, containing lines for each contour, with a height attribute which can be used to style the contour on the map.

gdal_contour -i 20 -snodata -32768 -a height N49E019.hgt N49E019.shp

In addition to contour lines, I also generate the hillshading. This is also done with the GDAL libraries, with the hillshade command. Note that the result is not a shape file, but another raster file. The raster file contains the ‘shade’ of the hill, a value between 0 and 1 depending on the orientation and steepness of the slope. The defaults work well: the light source comes from the north west (default for many maps), at an angle of 45 degrees.

gdaldem hillshade N49E019.hgt N49E019.shade

Creating the map style

Once the data is ready, the map can be generated by making style for each part of the data. The description that follows took many steps, and manual checking of the rendered result. This styling is in no way perfect, but at least optimized for cycling.

Each section below can be seen as a layer (actually, they are the names of the Mapnik layers). The first layer is on the bottom, and every following layer is printed ‘over’ it. Sometimes I use a transparent layer to add information to the map, while keeping the information under it visible.

Background: land use

The background of the map will be white, with an overlay of forest (light green).

Contours and hillshade

Over the background, the hillshade will be printed (with a low opacity), such that hills become visible. The contours are added to that, such that hills and mountains will become visible by rings of contour lines. Normal contour lines are every 20 meters, while thicker contour lines are every 100 meters.

Cities and military

Two special types of land use are added after the hillshading. Cities (with residential land use), and military areas are marked especially. Cities need to be visible on the map: they give structure to the road network and names. Military areas are off-limits for civilians.

Borders

Borders of countries need to be visible. They get a transparent thick background such that they are easily visible in the large-scale structure of the map.

Water

Water is important to get a good overview of the area (together with forest areas). Lakes give structure to the map, and streams and rivers also give information on hills, mountains and valleys. Rivers and streams always flow perpendicular to contour lines, and parallel to valleys.

Trains

Train tracks show connections between large cities and towns. They are visible in the landscape because they are long and straight. Stations are often listed on road signs.

Roads (and bridges)

Most of the work has been done on roads. They sound easier than they are, because there are many types and they connect with each other.

The general idea for roads is “more important is on top”. That means small paths first, then dirt roads, then general small roads, then tertiary roads (small roads with a name), then secondary roads, primary roads and trunk roads (between a primary road and motorway, the use differs per country). Finally, motorways. And to make things hard: cycleways need to be visible over every kind of road, but not too invasive.

But that is not always the case:

For all roads, the borders should always be on the bottom, and the roads themselves should be on top.
If a tertiary road contains a bridge over a motorway, it has to be displayed over the motorway (even the border!).
If there are ‘link’ segments, for example going from a secondary road to a trunk road, the crossing has to contain the link sections on the bottom, and the main roads on top.

These requirements make a beautiful list of (some repeated) styles for roads and their borders.

Names of cities, roads, and contours

One of the primary features of a map is the names of the things on the map. Cities are the most important: they give information on where roads lead. All names of cities, villages, hamlets and even suburbs of cities are printed.

But roads also have names (references). They are often listed on road signs. If there is space on the map, the names are printed along the road.

Finally, the contour line heights. They have little priority, but can give context how high the surroundings are. If there is any space left on the map, the heights of the 100-meter contour lines are also printed.

Places of interest (icons)

The last layer of the map are icons of places of interest. Especially in a city they can be good references (think of churches with a high tower). I have selected a list of places of interest that do not clutter the map too much, and give enough information to be useful. The map contains places of worship (churches, mosques, synagogues, etc.), shops, bicycle repair shops, campings and castles.

Each icon has a circle background (partly transparent) to make it more visible in the whole.

Creating the map

In Mapnik the layers above can be defined using a Python script. For each layer a data source is used, and the features of that data source are printed on the map using some style. The features can be filtered such that a style is only applied to some features. For example: the data source for roads contain all roads. It has many styles, for example one for primary roads which are a bridge.

The Python script creates all layers, and Mapnik generates an XML of the description of the map. This XML file is portable and can be used directly by anyone, without running the Python script.

Using the map description (either from the XML file of from the Python script), Mapnik can output many kinds of images. It has many options for JPG and PNG files, but it can also create PDF files. That is useful for vector features with infinite resolution (good for printing).

Scripting it all together

In addition to the Python script for creating the map, I have also made a bash script which downloads the data required for the map in an automated way. For some areas multiple files have to be downloaded and processed, and this is cumbersome to do by hand.

The bash script will download all the height data, process it into hillshade and contour lines and add it to the Postgres database. Then it will download the country borders of the selected countries and add them to the Postgres database. Finally, the map data is downloaded and also added to the database.

This makes running the scripts for a different set of countries easy: only the names of the countries, and the latitude/longitude ranges of the map are needed.

Conclusion

This is the end result of the map.

For me, this is a long time wish which has come true. I just needed a problem for which generating a custom map was the solution. I plan to use a printed version of my own map the next time I go on a cycling holiday, and test it in the real world.

The source code and scripts can be found on in the GitHub repository. Feel free to leave comments and suggestions for improvement!

hiddewie / osm-cycling-maps

Generating beatiful cycling maps using Mapnik and OpenStreetMap data

Generating a custom cycling map with Mapnik and Carto

View the blogpost describing this project at https://dev.to/hiddewie/creating-a-custom-cycling-map-3g2a.

Download the example PDF output.

Cartography and features

Cartography

Features

Getting started

There are three scripts in this repository:

scripts/download.sh

Downloads the required data into a Postgres database for the map.
scripts/generate.py

A Python script which will generate the map for a Mapnik configuration. The Mapnik configuration can be generated by running Carto against carto/project.mml.
scripts/bounds.py

A Python script which will output a list of bounding boxes that will fit the configured page size and bounding box perfectly.

See the environment variables which can be configured for the scripts below.

Copy the distributed environment file .env.dist to .env. Fill in the required credentials for the U.S. Geological Survey. You can modify the .env file to suit your needs.

The scripts are packaged as Docker images, and configured in…

View on GitHub

From QWERTY to Colemak

Hidde Wieringa — Sat, 22 Jun 2019 14:14:59 +0000

This year I made the successful switch from a QWERTY to a Colemak layout for all my keyboards. This is the story of my journey.

QWERTY and its problems

Most people in the world that communicate using Latin characters use a QWERTY layout (or some variant like AZERTY, QUERTZ, etc.) for their keyboards. This is because of historical reasons: typewriters became popular and they had the QWERTY layout. The reason was not for improving ergonomics, but rather to avoid jamming.

For computers, jamming is not an issue. Multiple layouts that have most keys different from the QWERTY layout have been invented. Examples are Dvorak, Colemak and Maltron. There are also layouts that are not widely used but shared online. Anyone can make and share a keyboard layout nowadays, for example using a tool like Microsoft Keyboard Layout Creator.

The QWERTY layout is not designed for ergonomics. For persons who write (either words or code) whole days on end, this can become an issue. Also, it is interesting to experience an alternative to the layout everyone is using.

Dvorak

A few years ago, I started learning the Dvorak layout. This layout is a popular alternative to QWERTY based layouts. There is some documentation online, and in most systems the layout is built in as a keyboard selection.

I practiced online until I could type reasonably well (but not very quickly) using the layout. After a few months I was seriously considering switching permanently. However, for my day-to-day work I noticed a large problem. The shortcuts for cut, copy and paste (and some others), were suddenly in a very different location on the keyboard. Of course, it is possible to remap these shortcuts but that was not an option. I want to be able to use a system and program as is, independent of the keyboard layout.

This was a blocker. I stopped practicing and reverted quickly to QWERTY.

Requirements

Luckily, the story did not end there. The idea to learn an alternative keyboard layout stayed in my head. I had three goals I wanted to achieve, namely:

Have around the same typing speed as I had using QWERTY (not quantified).
Type with ten fingers.

I could not type with ten fingers, but rather used around six fingers in strange combinations and patterns.
Reduce the number of errors while typing.

I made many typing errors when using QWERTY. This made for quick typing but I still lost a lot of performance because I had to ‘backspace’ to correct mistakes.

I compiled a list with requirements I wanted to have for my ideal layout. I came up with the following list:

Good ergonomic properties.

Measuring ergonomics can be done in many different ways. Layouts are often optimized for different metrics. For me, the layout must score well in usage of the home row, and alternating letters between two hands.
Some widespread use around the world.

This makes sure that there is documentation, and tools for practicing are available (online or offline). There will also be some analysis of the ergonomics available.
Software switchable.

The layout must be available by switching using software only. This is usually independent of the layout and most OSes support this.
Has to fit basic system/program shortcut keys.

I want to be able to use a system without modifying the entire system before using the layout. This is especially useful if I have to use a someone else’s system. Only a keyboard layout change is needed.
Extensible for programming.

The layout must be able to allow extensions for programming. Many layouts support this. It usually means switching the numbers and symbols around, independent of the letters.

I wanted to learn to type well (with 10 fingers), so the software requirement meant that I used the other layout when typing on a QWERTY keyboard. Looking at the keyboard while typing did not help!

One factor which was not mentioned above but which is important when choosing a layout, is the language you optimize for. For me that is English, although I am Dutch. This influences the layout a lot, because the usage of letters and combinations is different across different languages. In English the combination ‘th’ is common, while in Dutch the combination ‘ij’ is common.

Colemak, here I come!

My choice for my future keyboard layout became Colemak. As far as I could find, it is the second most used alternative layout for Latin writing, after Dvorak. There is a wiki with some history and many links and tips for learning and troubleshooting Colemak (although some resources are outdated).

Tools

Some tools are also highlighted on the wiki. There are many online and offline typing tools, and often different layouts are supported. This is important, because using a typing tool which teaches you the QWERTY layout will have different exercises than one teaching Colemak. (Think of the home row, the first lesson. QWERTY will use the ‘F’ and ‘J’ keys, but Colemak will use the ‘T’ and ‘N’ keys.)

It is important to have a tool which highlights and punishes errors while typing.

I used keybr which keeps a history of previous practicing, records typing speed and letters which are ‘hard’ (high error rate).

Installation (on Windows)

Windows does not have a native Colemak layout installed. However, the Colemak wiki contains an installer which makes installing it easy. Linux (Debian) and MacOS have the layout installed out of the box.

Once the layout is installed, switch by finding your keyboard settings and selecting the Colemak layout. Often a preview is available of the mapping.

Caps lock

The Colemak layout officially has the Caps Lock key mapped to Backspace. In Windows this can be a problem, so I did not include that mapping in my learning process.

Hardware layout

There are also keyboards which have a hardware mapping of the Colemak layout. This means that the OS will have the ‘normal’ QWERTY layout, but the keyboard will send Colemak keystrokes as if they had been typed on a QWERTY keyboard.

I did not use this option, but it can be useful when pair programming with two keyboards. One person can use QWERTY, the other can use Colemak.

Practicing

Using the tools listed above, I started practicing. I checked for compatibility with my systems (at home, at work, my mobile devices) and workflow. Most of thetime was used on evenings and in weekends to learn more and more keys of the layout. I usually took 20-40 minutes per practice session. Actually, this took more energy than I had thought. This is logical: every keystroke of every practice word takes mental effort of growing muscle memory.

All this time I was using the QWERTY layout in my day-to-day work.

Switch

After practicing for around two months, I switched my mobile phone to a Colemak layout. I knew the basic layout by then, and I only use two fingers when typing mobile, while looking at the keyboard. This worked great. My speed dropped for a few weeks, but otherwise the switch was clean. And now I was practicing extra whenever I was using my phone.

Cold turkey

My grand strategy for making the full switch of all my remaining keyboards was as follows. I had a few weeks of Christmas holiday in which I had to type very little. The perfect moment for activating a new layout and starting to type slowly and painfully. This also made sure that my work did not suffer too much.

In the Christmas holiday I also started doing that year’s Advent of Code puzzles, in a programming language I never used before (Clojure). This was a great match. Luckily, it worked and I met no surprises that made me have to switch back to QWERTY.

Day-to-day work

The second day of January I took up my regular work. This consists of programming, so a lot of typing. Of course, my colleagues knew I had made the switch to Colemak, and I encountered no problems. By forcing myself to use Colemak I quickly came to a basic typing speed which was more than enough to get my work done.

Experiences

After the full switch to the Colemak layout, I noticed some things that were not a problem but needed some attention.

Shortcuts

I use many shortcuts. However, finding them was sometimes difficult. For example, locking my computer uses ‘Super + L’, but the ‘L’ is mapped to the QWERTY ‘U’. The difference between typing and using shortcuts becomes visible: for shortcuts we often look at the keyboard to find a single key. This does not work when the keyboard keys to not match the actual letters when pressed. On the other hand, I now unconsciously remember that locking my computer requires ‘Super + U’ when I am using a Colemak layout.

And to repeat the requirement above: this was not a problem for select all, cut, copy and paste because those keys are in the same location as in the QWERTY layout.

Getting up to speed

It took around two months after the full switch to come up to speed. That is around five months since I started practicing with the Colemak layout. Notice that I don’t quantify ‘up to speed’. As discussed in my list of requirements, I wanted to become around as quick as I was before the switch, while making less mistakes and typing with ten fingers.

I see that typing is ‘quick enough’ for my needs, but also that there is a lot to be learned and practiced. A whole life of training QWERTY cannot be reverted and relearned in another layout, within half a year.

Errors

The most errors I make are usages of the ‘D’ and ‘T’ keys. They are present in many words, and typed by the same finger. That means that muscle memory sometimes swaps them around.

A see the increase of errors especially when typing while thinking hard or explaining something to someone. Hopefully this will improve over time.

Retrospective (looking back)

I am very happy with my new layout. Because of my list of requirements, I could learn the new layout and be reasonably sure that it would work out (compared to learning Dvorak). The effort required for learning the new layout is very reasonable in my opinion.

In my day-to-day work, it can be funny when other people try typing on my keyboard. They want to demonstrate something, try typing it in my console, look at the typed text, backspace it away, try again and then have a beautiful bewildered face. It is nice to see how people expect keyboards to have a certain layout.

The second thing I noticed, is that QWERTY experience disappears very quickly. Whenever I have to type QWERTY (a public computer, or my disk encryption password before my system is booted), it feels as if I first started with Colemak.

Tips

I advise others to switch their layout. The experience alone is worth it, and the benefits of Colemak (or another alternative layout) make the switch worthwhile. When you consider the switch, I found the following points helpful:

Practice first, then make cold switch.

Get to know the layout. First study the layout (it will be impossible to remember) and then start practicing. By slowly learning more and more letters you learn to type correctly. When you know the layout, make the cold turkey switch. This forces you to learn the last part of the muscle memory and become really fluent when typing.
Focus on making little errors. Speed will come later.

If you switch too early, many errors will creep in. They are very hard to unlearn (as I noticed personally when using QWERTY). When using the new layout on a day-to-day basis, speed will come.
Switch mobile quickly.

This helped me a lot when practicing the Colemak layout. Because I was already using the new layout on mobile devices, it became easier to remember the locations of new letters.

Next steps

I notice now that there are more things that I can do to improve my typing experience. The switch to Colemak made me more conscious of my keyboard usage, typing habits and shortcut usage.

For now, I use exactly the same (hardware) keyboards I used before the switch. I am looking for a different keyboard that is ergonomic and supports my Colemak typing. Some properties I want are an ortholinear layout (keys are not staggered horizontally, but rather in a grid) or vertically staggered. Also, a split keyboard would be great, so I can move and tune both parts based on the location of my hands.

Colemak specific requirements are a better programmer layout (I use the QWERTY number and symbol mappings) and mapping the Caps Lock key to the Backspace key (because who ever uses a Caps Lock key?).

I hope that this article explains some of my thinking and may inspire you to make the switch to an alternative keyboard layout!

Postscript

For this article I had to write the word ‘QWERTY’ a lot. While this is easy on a keyboard with a QWERTY layout, it is strange to type on a Colemak layout.