Forem: Thomas Hartmann

Review 2019, prepare for 2020

Thomas Hartmann — Tue, 31 Dec 2019 13:34:54 +0000

The year is coming to a close, and it's a great time to look back upon what we've achieved this past year and what we want to focus on in the coming year. Let's talk about a few key things:

What things in the last year have been the most significant to you? What are you the most proud of?
Going forward, what things do you want to focus on?
And what do you not want to learn?

My story

In trying to keep this brief and digestible, here's the short format of my 2019/2020 story. If you want to know a bit more and see the specific SMART goals that I've set for myself, check out my blog post on the topic.

Most significant events of 2019

Community: 2019 was the year I really got involved with the software community and sharing knowledge. I gave talks and workshops at local meetups, I became a co-organizer of the local Rust meetup group, and I started blogging.
Nix(OS): I switched to Linux (specifically NixOS) about a year ago, and it's been a blast. A lot of crawling the internet and irc logs, a lot of learning to use irc, a lot of time spent tinkering, but a blast. Everyone's been so friendly and welcoming and I have learned a whole lot. I've also started to rely on Nix for my dev environments at work.
Containers: Before 2019, I'd never used containers, but I was aware of them. 2019 saw me working with Docker and Kubernetes on a daily basis, and it's been fantastic. In particular, Kubernetes has become a great interest of mine, and something I'm looking to pursue further.

Goals for 2020

More community engagement: I want to further work on the community-aspects in the coming year: talk more, share more, meet more. In particular, by the end of the year I want to have:
- Given at least three (3) talks/workshops at local meetups.
- Applied to at least three (3) conferences as a speaker.
More konfiguration: A big part of 2019 was configuration, either as part of Kubernetes or as part of Nix. I'm really into both these tools, but I don't feel that I have mastered them completely just yet. To improve this situation, I want to set up a Kubernetes cluster somewhere and deploy an application developed using Nix for as much as possible.
More Emacs: Another tool that I really enjoy, but that I don't know quite well enough. I've been using Emacs for about a year and a half, and I love it. However, I've never really used raw (or custom configured) Emacs; only a version of Spacemacs. Spacemacs is great, but I feel like I need a better understanding of the underlying editor to really become one with Emacs. First step: RTFM.

`.ignore`

There's so many cool ideas and concepts around these days, you cannot possibly master them all. If you don't purposefully shut certain things out, it's easy to get overwhelmed. Here's a little list of things that I'm not planning to look into in the coming year:

Machine Learning / Artificial Intelligence
Go
IoT / Home automation

Your turn

So what about you? Got anything to share? Comments? Let's talk it out.

Happy new year, folks! Hope you achieve everything you set out to do, and if you don't: I hope that you at least learn and grow from it!

Async Rust: a gentle introduction

Thomas Hartmann — Mon, 18 Nov 2019 08:21:10 +0000

Hot off the heels of RustFest Barcelona and the stabilization of async/.await, I think it’s safe to say that one of Rust’s most anticipated language features has finally landed. And for that occasion (and because I’ve had some trouble understanding certain bits of it myself), I wanted to write a little introduction to asynchronous programming in Rust. We’ll be creating a super simple application that fetches some data from the internet using the our newfound async abilities. The stabilization of async/.await also coincides nicely with another event I’m excited about: This week saw the release of the most recent entries in the mainline Pokémon games, Pokémon Sword and Shield, and because I’m a bit too busy to pick them up just yet, I’ll make do by fetching data from the PokéAPI for now.

I’m assuming some base knowledge of Rust’s syntax and ecosystem, but I hope that this is pretty accessible even to people very new to the community.

But before diving into the coding part, let’s cover some basic concepts of asynchronous programming and how it might be a bit different in Rust than what you’d expect.

What does `async` mean?

In Rust, when we talk about async, we’re talking about running code concurrently, or having multiple overlapping (in time) computations run on a single thread. Multithreading is a related, but distinct concept. Multithreading is ideal for when you’ve got computationally intensive tasks (so-called CPU-bound tasks) that can be spread across multiple, separated cores. Concurrent programming is better suited for when the task spends a lot of time waiting, such as for a response from a server. These tasks are called IO-bound.

So asynchronous programming lets us run multiple of these IO-bound computations at the same time on a single thread. They can run at the same time because when they’re waiting for a response, they’re just idle, so we can let the computer keep working on something that isn’t waiting. When we reach a point where we need the result of an asynchronous computation, we must .await it. In Rust, values that are ‘awaitable’ are known as ‘futures’.

Rusty weirdness

async in Rust may be a bit different from what you’re used to in other languages. Having done asynchronous coding mostly in JavaScript and C#, it certainly was to me. Here’s a few key things to understand:

An `async` function does not (necessarily) start executing immediately

To start an asynchronous function, you must either .await it or launch a task using an executor (we’ll get to that in a moment). Until this happens, all you have is a Future that has not started. Let’s look at an example to make it clearer:

use async_std::task;
// ^ we need this for task spawning

async fn negate_async(n: i32) -> i32 {
    println!("Negating {}", n);
    task::sleep(std::time::Duration::from_secs(5)).await;
    println!("Finished sleeping for {}!", n);
    n * -1
}

async fn f() -> i32 {
    let neg = negate_async(1);
    // ... nothing happens yet
    let neg_task = task::spawn(negate_async(2));
    // ^ this task /is/ started
    task::sleep(std::time::Duration::from_secs(1)).await;
    // we sleep for effect.

    neg.await + neg_task.await
    // ^ this starts the first task `neg`
    // and waits for both tasks to finish
}

So in the above little code snippet, here’s what’s going on.

The first line imports async_std::task. There’s more on this below, but we need an external library to run futures as the standard library does not come with an executor.
The async function negate_async takes as input a signed integer, sleeps for 5 seconds, and returns the negated version of that integer.
The async function f is more interesting:
- The first line (let neg ...) creates a Future of the negate_async function and assigns it to the neg variable. Importantly, it does not start executing yet.
- The next line of code (let neg_task ...) uses the task::spawn function to start executing the Future returned by negate_async. Like with neg, the Future returned by negate_async is assigned to the neg_task variable.
- Next: we sleep for a second. This is so that it will be obvious from the output when a task starts running.
- Finally, we await both futures, add them together, and return them. By awaiting neg, we start executing the Future and run it to completion. Since neg_task has already been started, we just wait for it to finish.
So what’s the result of this, then?
```
Negating 2
# <- there's a 1 second pause here
Negating 1
Finished sleeping for 2!
Finished sleeping for 1!
```
As we can see, the second future, neg_task, started executing as soon as it was called—thanks to task::spawn—while neg did not start executing until it was awaited.

You need an external library to use `async/.await`

As was briefly alluded to above, you need to reach for an external library to do asynchronous programming in Rust. This took me a while to understand, as I’m used to it being part of the language experience. In Rust, however, you need a dedicated executor¹. The executor is what takes care of executing the futures, polling them and returning the results when they’re done. The standard library does not come with an executor, so we need to reach out to an external crate for this. There are a few ones to choose from, but the two most prominent ones are async-std (which we’re using here) and tokio.

A minimal async example!

Alright, let’s get practical. This is the reason that I’m writing this post. As mentioned at the start, we’ll be creating a super simple application that fetches some Pokémon data and prints it to the console. For preparation, make sure you’ve got at least version 1.39 of Rust and cargo available.

Step 1: creating the application

Let’s create a new application! Simply run this command in your preferred directory:

cargo new async-basics

Step 2: Dependencies

We’re going to be using async-std for spawning tasks, and surf to fetch data from the API. Let’s add them to the Cargo.toml file. Your whole file should look something like this:

[package]
name = "async-basics"
version = "0.1.0"
authors = ["Your Name <your.email@provider.tld>"]
edition = "2018"

[dependencies]
async-std = "1"
surf = "1"

Nice! This is going swimmingly!

Step 3: Fetch data

Okay, final step. Let’s modify the main.rs file. We’ll make it as simple as possible. Here’s what we want to use:

use async_std::task;
use surf;

// fetch data from a url and return the results as a string.
// if an error occurs, return the error.
async fn fetch(url: &str) -> Result<String, surf::Exception> {
    surf::get(url).recv_string().await
}

// execute the fetch function and print the results
async fn execute() {
    match fetch("https://pokeapi.co/api/v2/move/surf").await {
        Ok(s) => println!("Fetched results: {:#?}", s),
        Err(e) => println!("Got an error: {:?}", e),
    };
}

fn main() {
    task::block_on(execute());
    // ^ start the future and wait for it to finish
}

That’s all the code you need. In fact, it’s more than what you need, as some parts have been broken up for legibility. Let’s walk through it!

use statements: Nothing exciting here. Just importing the crates we declared in the Cargo.toml file: surf and async_std.
fetch: This is simply a thin wrapper around the surf::get function which returns either the payload as a String or an Exception if something went wrong.
execute: This function calls fetch with the endpoint for the move Surf, waits for the result to return, and then matches on the result. If everything went well: print the output. Else: print the error.
main: main simply kicks off execute and waits for it to finish. task::block_on is a synchronous counterpart to task::spawn that starts an asynchronous operation, but blocks until it has finished. Because the main function can’t itself be async (at least not at the time of writing), we can’t use .await in it, but we can block on asynchronous operations.

Step 4: Extend it!

Hey, you made it this far; congrats! That’s all I really have in store for you for this one, but if you want to play around a bit more, how about adding serde and try using surf’s recv_json<T> instead? If you’d rather keep looking at async/.await, how about performing multiple requests simultaneously? Or how about making a PokéAPI CLI? (Ooh, that sounds like fun! Hit me up if you’re doing this; I want in!)

Parting words and resources

So there you have it, dear reader. I hope you have found this useful. async/.await is finally stabilized and it feels like we’ve taken a major leap forward. I’m very much looking forward to seeing what happens in the coming months and what the community makes of this.

If you’re looking for more resources on async Rust, be sure to check out the Async Book. I also recommend the async-std book for some extra insights.

Until next time: take care!

Footnotes

See this insightful Reddit comment thread for more on this.↩

Git config management

Thomas Hartmann — Mon, 16 Sep 2019 07:00:10 +0000

We’ve been looking a lot at how to tweak git to your liking, which is usually done through your git config file. Sometimes, though, you’ll find that the configuration isn’t working as you wanted, so I thought it’d be a good idea to look at how you can inspect, unset, and list configuration values from the command line.

Before we dive in, let’s have a few words about how your git config files work. Your config files use a specific format which, according to the docs, “consists of sections and variables. A section begins with the name of the section in square brackets and continues until the next section begins.” That’s enough for what we need right now, but go read up if you’re interested.

Furthermore, git config files ‘cascade’, and git, by default, searches four locations on your system, where later entries override earlier ones:

/etc/gitconfig: The system wide config
$XDG_CONFIG_HOME/git/config: Falls back to $HOME/.config/git/config if $XDG_CONFIG_HOME is not set or empty.
~/.gitconfig: Your user-specific, ‘global’ config file
$GIT_DIR/config: Repo-specific config file

Again, see the specific section of the documentation for more information.

One last thing: All setting and unsetting of values apply only to your user configurations, and they are local—repo-specific—by default. To make them apply to your global config, pass the --global flag.

Setting values

To set a value, simply run a command on the form

# default (local)
git config <section>.<key> <value>
# global
git config --global <section>.<key> <value>

For instance, to change the value of the commentChar entry of the core section to ;:

git config core.commentChar ';'

Unsetting values

Similarly, if you want to unset a value, use the --unset flag:

git config --unset <section>.<key>

So if you have overridden the core.commentChar value, this is the command to undo that and reset it to the default value, #:

git config --unset core.commentChar

Inspecting config values

Sometimes, you want to know what a value has been set to. For this, pass the --get flag and the section and key of the value you want to look up:

git config --get <section>.<key>

So if you’re wondering what your user’s name is, for instance:

git config --get user.name

A few things to note about getting the values

Git will show you the last value it finds in the chain: So if you set your username in your global config, but set a different one in a repo and invoke this command from the repo, it would show you only the repo-specific one.
The output will be empty if you have not explicitly set the value: In other words, if it’s not in your config files, git won’t show it to you. So if you’re looking to inspect git’s default values, that’s not going to work.

Finding the config value sources

Sometimes you just want to inspect what values you’re setting in your config. You can do this by passing the --list flag:

git config --list

By default, this will just throw all your configuration at you, which isn’t all that helpful. However, it takes a pretty handy flag, --show-origin, which lists your configuration in two columns, with the first one being the file in which it was set and the second being the the key-value pairs.

If you ever find that some value isn’t taking effect the way you expect, this is a great way to check what values are getting set in what locations:

git config --list --show-origin

For instance, if you’ve changed your global user name, but it’s not taking effect in the current repo, you could pipe it into grep and look for duplicates:

git config --list --show-origin | grep "user.name="

Commit message templates

Thomas Hartmann — Mon, 09 Sep 2019 03:53:29 +0000

Here’s a statement for ya: if you’re working on a project with someone else, you should be using a commit message template. Not only will it help increase the consistency and usefulness of your messages, but also, and perhaps more importantly, it will reduce the cognitive load required to write a message and the time it takes to write it. I can only speak for myself, but I know that having to go look up information about how a commit message should be structured and what it references causes some much unneeded context switching, resulting in a less efficient workflow.

Setup

Setting up commit message templates requires only a few steps:

Create a template.

You can use any text file as a template; just put what you want to show up in the commit message buffer in there. Plain lines of text will show up as content, while lines starting with the comment character (# by default) show up as comments.
Add it to the git config as a commit template: git config commit.template path/to/file.
Done! Now try committing something.

When doing this, you’re assigning the template locally (to the current repo). If you’d rather do it globally, pass the --global option.

If you want to share the template across multiple projects in a certain subdirectory, you can do that by combining the template with git’s functionality for conditional includes. Be aware, however, that given a relative path to a file, git will look for the file relative to the including configuration’s location. While this can potentially be exploited to use different templates for different projects, I’ve not found the need for it thus far, and have stuck with a single template with an absolute path.

If you want to unset the template, use the git config --unset functionality:

git config --unset commit.template

Commit template ideas

Different teams and projects have different needs when it comes to commit messages, so there’s no ‘one size fits all’, but here’s a few ideas based on message formats in teams I’ve been a part of.

Label listing

If you label your commits with the type of work they contain to and have a defined set of labels (e.g. ‘feature’, ‘bug’, ‘chore’, ‘refactor’, etc.), it might be a good idea to list all the different options in the template, so that you can look through them whenever you’re committing some code. List them using commented lines and you don’t even have to delete them; they’ll just show up as extra info in the commit buffer.

While experienced members of the team are likely to know these by heart, it’s very helpful for a new person to have the list right there in the commit buffer so that they don’t have to go look them up.

Issue identification

If the standard says to have an issue/ticket ID listed in the commit message, you could put a little placeholder in the template that tells you to replace it with the relevant ID.

If you also work with feature branches and name them according to the feature/issue ID that they correspond to, you could even have git fill in this issue number automatically by using hooks, as explained in my previous post. I have found this to save me a surprisingly large amount of time and mental overhead.

Very specific formats

For when your message needs to match a very specific format and it’s hard to remember exactly what goes where, templates are a perfect solution.

For instance, if you’re working with a combination of the above two, where you want both the label and the ID, and you want them in a specific order, maybe even with a specific separator (seems like a hassle, but just roll with it), you could create a template that you could just fill in with the relevant parts, saving you having to look up what the format is every time.

Rebasing off a repo root

Thomas Hartmann — Mon, 02 Sep 2019 07:52:55 +0000

Ever had to change something about all the commits in a git repo before pushing it to a remote? As in from the very first commit in a repo? Maybe you’ve started a repo locally and when you’re ready to push it, you realize that you’ve used the wrong author or messed up the format of the commit messages; or maybe you just want to squash those first few commits into a single concise package.

In these situations, my first response is to do to an interactive rebase (git rebase -i). Usually when I’m rebasing, though, I’m in a project that has an upstream and where I’m rebasing off a specific commit or branch. For situations where you don’t have a commit to rebase off, but you want to rebase the entire history or at least the very first commit: What do you do?

The answer, my friend, is that you pass the --root option:

git rebase -i --root

That’ll let you pick, reword, edit, squash, fixup, exec, drop, label, reset, or merge all of your commits from the ‘dawn of time.’

Have fun!

Automate your commit messages

Thomas Hartmann — Mon, 26 Aug 2019 07:48:50 +0000

Git has a lot of inbuilt functionality that you might not use all the time, but which is oh-so-handy when you need it. One such thing is hooks. These can be used to do stuff like formatting your code before committing or running tests before pushing to the remote. Today, though, let’s look at how we can use it to automatically fill in part of your commit messsage.

How did this come up?

The team I’m on at work uses a system for commit messages and branch names where they should both start with the issue id of whatever task we’re working on. Using GitLab, they end up looking something like #2 Adds sorting functionality to UI (we can argue about commit message tenses later) and #42/new-sorting-algorithm, respectively.

If you’re like me, your mind should be poking you right about now, saying that that’s a lot of duplication; surely we don’t need all of that? Well, you could argue that with feature branches scoped to issues and no fast-forward merges, the branch name should be enough, but it doesn’t really show up in condensed commit logs etc., so it’s nice to have the issue number available for when you need it.

But not to worry; we can make the computer fill it in for us!

Picking the right hook

Git has quite a few hooks available; if you check the .git/hooks directory of a tracked project, you should find a bunch of files named <hook name>.sample. Most of the hook names are fairly descriptive, but you could always look them up if you want to know more.

In our case we’re interested in hooking into the system right before the commit message buffer appears on our screen, so we’re going to go with the prepare-commit-msg hook.

Auto-filling the message

Now, my team has a template we use for commits, which starts with #[task number]. Using this knowledge, we can do some fancy shell magic and replace it with the task from the branch name.

The whole script looks like this:

#!/bin/sh

TASK_NO=$(git rev-parse --abbrev-ref HEAD | cut -d '/' -f 1)
# BSD sed (macOS)
sed -i'' -e "s/#\[task number\]/$TASK_NO/" "$1"

# GNU sed (Linux)
# sed -i -e "s/#\[task number\]/$TASK_NO/" "$1"

Depending on whether you use BSD sed (macOS) or GNU sed (Linux), the sed command will behave slightly differently. For GNU sed, use the commented out line instead of the first one.

There is a slight difference in how you make the two versions of the programs write files in place.

The git rev-parse --abbrev-ref HEAD command gives you the name of the current branch, which we then split at the first / character using cut.

The next (and last) step is a simple sed replacement, replacing the #[task number] string with the branch’s task number.

The $1 variable at the very end of the last line is the name of the commit message file and is something the script receives automatically. We use this to tell sed what file to replace the text in.

And for the curious, these are all the variables you get access to in the pre-commit-msg hook (taken from the pre-commit-msg.sample file):


COMMIT_MSG_FILE=$1
COMMIT_SOURCE=$2
SHA1=$3

Caveats

There is a pretty significant exception to when this script does what you want it to: when you’re still on the master branch, where commit messages will start with the text master. I considered adding a special case for this, but then realized that it’s actually quite helpful in that it helps me remember when I should branch off, so I decided to keep it for now.

Another drawback is that because the commit message appears different from what git expects to find on disk (presumably), it will assume you put an actual message in and complete the commit. At least that’s what I found when using Vim. A simple workaround is deleting everything in the buffer (dae if you use the awesome text-obj-entire plugin) before exiting. Most of the time, though, I use Magit with Emacs, where canceling the commit works as expected.

Alternate approach

While I have opted for modifying the commit message before it’s shown to the user, you could potentially also have the machine add the id automatically in the pre-commit hook.

You would make the system check the message for a pattern and amend the commit with the expected issue id if it’s not present.

This solution doesn’t give you any way to verify whether it’s correct or not, though, so you’d probably want some sort of way to bypass it.

Change your git comment character

Thomas Hartmann — Mon, 19 Aug 2019 06:21:57 +0000

You prepare to commit your files, having made sure that all the right changes are in, that they belong to one logical change, and that you have the id of the task that they belong to. Your editor opens up and you see those familiar commit message instructions, but wait! You suddenly remember that you should start your commit with the id of the task prepended with a # character, but that would lead git to think that that line is a comment.

How do we solve this?

Easy! We simply tell git not to use the # character for comments, but pick a different character instead.

In short

There’s two ways to do it, both producing the same result. You can either run a command from the command line or edit your git config directly. In this case, imagine we want to use a semicolon (;).

Command line:

git config --local core.commentChar ';'

Directly modifying your git config file:

[core]
  commentChar = ";"

Motivation

So why might you want to do this? As mentioned in the introduction, maybe your team has a standard format for commit messages, where each commit should lead with the id of the task that the commit relates to.

We use GitLab at work, where if you put a number after a # in your commit message, it will create a link to the issue with that id, which also adds an entry to the history of said issue. In addition to this, if all commits are prefaced with the task id, it becomes very easy to find out what task a commit relates to when looking at the history.

What might go wrong?

This is such a simple modification that there’s not much you need to be aware of, but if you try and use a comment character consisting of more than one character, git will fail when you try and invoke a command:

error: core.commentChar should only be one character
fatal: bad config variable 'core.commentchar' in file '.git/config' at line 6

Finishing thoughts

It’s a simple thing, but it’s one of those things that improves your life just that little bit. You can even combine it with the custom config we mentioned last time to have this apply to all repos in your work directory, new and old.

Now as a final thought, what should you pick as a comment character? It’s really up to what you need for your specific use case, but I find that ; is very rarely something that you’ll want to start your sentences with, so that’s my go-to.

Modularizing your git config with conditional includes

Thomas Hartmann — Mon, 12 Aug 2019 08:27:09 +0000

Do you track your git config across machines or keep it in sync with your dotfiles? Is it full of settings that you want to share between all your different contexts? Are there certain things that you want to change depending on what the context is? Well, I’ve got news for you!

tl;dr:

If you want to use an additional config file for all subdirectories below a certain path, use the includeIf functionality that was introduced in git 2.13.0.

[includeIf "gitdir:/"]
  path = path/to/your/.config

That’s all you need to get going, but if you want to know more about how this works and some caveats you might to be aware of, keep reading.

`include` and `includeIf`

First off, I’d be remiss not to direct you to the official documentation for this feature, which can be found here. However, there’s loads more info there than you might need (and want), so let’s extract the important bits.

The include and includeIf sections work the same, except the includeIf has a condition that must be satisfied. They let you insert configuration from another file into your main one.

In my case, this meant that I could automatically change my email whenever the git directory was a subdirectory of ~/projects/work, allowing me a simple and efficient way to always use my work email for work, without changing my global git config or branching off from my main dotfiles repo.

Here’s the relevant extract from my config:

[includeIf "gitdir:~/projects/work/"]
  path = ~/projects/work/.gitconfig

I keep a separate config file in the ~/projects/work directory that overrides my email.

This would also be useful if your team has specific rules about whitespace, merge strategies, comment characters, etc. that you want to enforce.

Things that might trip you up

Syntax

Note the trailing slash after the directory path in the includeIf line. This makes it so that it will match all subdirectories of the specified directory. As explained by the docs:

If the pattern ends with /, ** will be automatically added. For example, the pattern foo/ becomes foo/** . In other words, it matches “foo” and everything inside, recursively.

This also means that if there is no trailing slash, it’ll match only that specific directory.

File insertion

Here’s another quote from the documentation:

The contents of the included file are inserted immediately, as if they had been found at the location of the include directive. If the value of the variable is a relative path, the path is considered to be relative to the configuration file in which the include directive was found.

So you’ll probably want to put the includes at the bottom of your files to make sure the included config isn’t overridden later on in the source file.

I hope that cleared some things up for you and that you found it useful; I know I did. If you want more in-depth information, see the documentation.

Emacs, VSCode, and me

Thomas Hartmann — Mon, 01 Jul 2019 07:51:32 +0000

It was about a year ago. My life was good. I had things in order (mostly). And most importantly: I had an editor I liked. VSCode and I had been going steady for about a year. With its excellent support for Python and JS—which was most of my work in those days—I rarely ever needed to venture into new territory. It felt good. I was happy.

I was happy!

Then it all came crashing down. A good friend and I decided to hang out and program one Saturday. The sky was a clear blue, the temperature high, and summer was just around the corner. We got into his office, sat down, and started setting up. That’s when it happened. Completely out of left field:

— So I found this cool thing the other day. It’s called Spacemacs.

Spacemacs?

— Yeah, it’s Emacs, but with “evil mode”. So you can get your Vim bindings!

It’s true. I had been seduced into the deep, dark depths of Vim about a year prior, and had never found my way out. Never tried, really. I had let it consume me, and there was no way I’d let it go. Not for the world. Of course I had been using the VSCode Vim plugin, but while it had given me most of the functionality I wanted, I did sometimes find myself longing for something more. So I thought I ought to at least give this Spacemacs thing a go.

With one eyebrow slightly cocked, I went ahead and downloaded Spacemacs. I had tried to run Emacs previously, but had been met with that sheer, white light that comes as the default, and had been unable to figure out how to even close the program without resorting to my mouse. Naturally, I was skeptical.

So I launch it, resist the white light as I choose ‘evil mode’ and watch the window turn dark. The logo appears. It says it’s loading. Fetching packages. Eventually, the words “Spacemacs is ready” appear at the bottom of the window. I get it up and running and do some basic programming. I don’t know what it is, but something just feels … good. I spend the rest of the day with Spacemacs, and when the time comes, I don’t want to let it go. I go home, dreaming of what I’ll do to it. VSCode is nowhere to be found.

That weekend, neither Emacs nor I left my apartment. We were consumed in the fiery throes of passion and configuration, and when the sun rose the following Monday, I knew my heart had been turned.

In truth, it would be another few weeks before I left VSCode completely and migrated to Emacs even at work, but it was coming. Oh, it was coming. To this day, I still stay faithful to Emacs for most of my endeavors, only venturing out when there is something I can’t find proper support for (and even then, it’s just a matter of time before I take matters into my own hands!).

What I love about it

The extensibility: The fact that you can make it do pretty much whatever you want to is nothing short of amazing. I have never seen an editor this powerful and versatile.
It’s the ‘self-documenting editor’: Everything in Emacs is documented. You ever wonder what a key does? C-h k <key (or combination)> has got you covered. You wonder what the value of a variable is? C-h v <variable name>. Want help with Emacs? C-h C-h. Boom.
EXWM: The Emacs X Window Manager. Just recently, I finally got it set up to the point where Emacs now manages all of my windows (yes, for all applications) for me. How many other editors can do that? Your move, ’Code.
Emacs Lisp: I think Lisp is a pretty cool guy. He lets you configure emacs down to the bone. Much like the monarch of beef patties, he let’s you ‘have it your way’, and honestly, that’s the way I like it.

What I’m not crazy about

Its single threaded nature: This is usually not much of an issue, unless you somehow end up freezing the main thread, which can happen. In that case, best restart your editor.
Lacklustre support for certain languages: While most languages I use are very well supported in Emacs, I struggle with others. In particular, Microsoft’s .NET languages have been hard to get set up properly.
Emacs Lisp: Yeah, it’s back. Mostly because I don’t really know Lisp yet (relax; it’s on my todo list). Until you’re comfortable with it, it makes changing your config just that little bit trickier.

What I miss from the VSCode days

So it’s not all roses. There are some things I miss from VSCode.

The ease of use: I can’t really that the learning curve is steeper; getting started with VSCode is just so easy. Also, installing extensions is a piece of cake.
The massive amount of plugins: Not that Emacs doesn’t have packages, but most communities I see seem to have settled on VSCode as their darling, so that’s just where you’re going to see the most activity and up-to-date plugins in a lot of languages.
The amount of documentation: I mentioned that Emacs is self-documenting. This is great. But it’s also … over 40 years old (1976?! Gee!)! So finding documentation online is tricky in comparison.
The community: Everybody loves ~~Raymond~~ VSCode. That’s great and makes it super easy to find articles on how to set it up and configure it and so on. Need help? Ask anyone. Chances are they’re using VSCode. The Emacs community is great too, but much, much smaller.

In the end, do I regret it? Not for a second. I think it’s a worthwhile tradeoff if you can take the initial learning curve and are willing to put some time into your editor. So if you feel inspired, should you do it? All I can say is that if you pay the upfront cost, Emacs will return it tenfold.

QWERTY to Dvorak

Thomas Hartmann — Mon, 24 Jun 2019 05:55:24 +0000

Ever thought about changing up your keyboard layout? Maybe something as small as going from an ISO layout to an ANSI one (hint: tall and thin to short and fat return key), or from Norwegian (or whichever your language of choice is) to American QWERTY? Maybe something more comprehensive, like learning a whole new layout, such as Dvorak, Colemak, or Workman? Well, I’ve done all of the above, and while the smaller changes can offer some pretty significant benefits, they’re pretty easy to get through. The layout-changes are more interesting. I’d like to talk briefly about how I went from QWERTY to Dvorak and what I learned along the way.

First off: you might wonder why I’d do that. I’d love to tell you that I had some grand epiphany and suddenly realized that the QWERTY life wasn’t worth living or that I discovered that Mr Dvorak himself was actually my great-great-grandfather; but nah, it was actually just an offhand comment someone made.

See, I had just recently started learning Vim and was talking about it with some fellow students when one of them jokingly suggested I do it in Dvorak instead. As if it wasn’t challenging enough already, right? I laughed it off at first, but it sparked something in me. Thirty minutes and some googling later and I had made my mind up. I didn’t know whether I was gonna make the switch, but I knew I had to check it out to see what all the fuss was about.

Before starting my degree in computer science (about two years prior to that fateful comment), I had never been much into computers and never properly learned to type, so my keyboard skills weren’t exactly award-winning: I wasn’t very fast (though not remarkably slow either), and I quite often had to look at the keyboard to find the key I was looking for.

Taking the above into consideration, I realized this was a chance to learn proper form and improve my typing skills; so after some investigation, I decided I was gonna go for it. And seeing this as a chance to pick a layout designed specifically for my purposes, I decided to go for the Programmer Dvorak layout.

Pros and cons

Now, about two years and thousands of key strokes later, I can take stock of what I have gained from switching and what issues arose because of it.

Benefits

Proper touch typing form: This might not seem like a big one, and from what I understand, it’s not necessarily the holy grail that some people make it out to be, but it has been a big boon to me. I can’t comment on it being better or more efficient than any other technique, but it sure beats my original, awkward, four-fingered approach.

Speed: My typing speed has gone up dramatically. I don’t know how many words per minute I could get out of my old QWERTY skills, but it wasn’t breaking any necks. I reckon that after about 3–4 months of Dvorak I had already caught up.

Accuracy: A nice feature of learning Dvorak on keyboards where you can’t change the physical layout is that you need to memorize the layout because—with the exception of ‘A’ and ‘M’—no letters are placed in the same spot. A side effect of this is that you can’t look down on the keyboard to find a key. In fact, that’ll probably just confuse you further.

Comfort: There’s something about the alternating hand-motion of the Dvorak layout that I really like. In fact, while learning it, I’d often come across sequences of letters that just felt good to write.

It might be more ergonomic: You often hear that the Dvorak layout is more ergonomic and better for avoiding RSI and the like. I can’t comment on the veracity of these claims, but it sounds plausible.

Drawbacks

The ramp-up time: It took a while until I got up to a respectable speed. This period where things just don’t move as fast as you want can be annoying and mentally taxing.

Extra set-up time: Dvorak is by no means the default layout for any system (as far as I’m aware), and so—if you’re using a software-defined layout—you’ll always need to go through some extra steps to set it up properly. Even more so if you go for some esoteric layout like Programmer Dvorak (though Linux comes with this available out of the box). This can of course be offset by going for a programmable keyboard (or getting a Dvorak keyboard).

Pair programming: If you’re getting help from a coworker, keep a QWERTY layout handy. It’s okay to have a little laugh when they stand there, though, perplexed at why nothing seems to work.

Losing touch with QWERTY: When I switched, I switched completely. As a result of this, every time I use a QWERTY keyboard now, I need to go looking for keys and I’m super slow. Though, for some reason, this does not apply to phone keyboards, where Dvorak feels awkward but QWERTY is perfectly fine.

Is it worth it?

Absolutely.

Seeing as I spend most of my day using a keyboard, it makes sense to invest in building skills. This might be ‘survivor bias’ talking, but I got so much out of it that I’d recommend it to anyone who’s interested. At least have a little sniff. If you’ve got the time and energy, it can be very fun and offer a real feeling of accomplishment. Plus, it’ll keep your brain active!

If that was all you wanted to know, you can stop reading now, but if you’re looking for some tips on how to learn a new layout, let’s finish it off with what I found worked for me.

How to learn

After picking my target layout (Programmer Dvorak), I had to find out how to learn. While it might be tempting to just sit down with a key map next to you (or shuffle the physical keys around if you can do that) and go for it, this is quite likely to be counterproductive for a number of different reasons; the biggest ones being that you’re more likely to slip out of proper technique and that it’s less efficient (based on my empirical studies with a sample size of one) when it comes to memorizing layout.

So what do I recommend? Glad you asked! I’ve got some tips for ya:

Use typing training software: These will typically introduce you to one new letter for each hand at a time and slowly work their way through the full layout, often focusing on common patterns. If you want tips on which one to go for, I’d recommend Typing Club; it’s free, effective, and even includes some pretty interesting bits of writing to copy as you progress, so you can learn while you learn. Yo, dawg.
Take it everywhere (work, studies, whatever): Once you’re comfortable enough with the position of the keys to not feel like you need to look up where a key is before pressing it (though thinking is okay), start using it all the time. Once you’ve got the basics down, the most important thing is to get exposure and experience with it. Depending on your situation, this can be more or less acceptable, so keep important deadlines and the like in mind, but don’t let that stop you. Keep your other layout around so you can swap back in case of emergency (or pair programming!).
Don’t give up!: Even when it seems hard and you’re tired of making the same typo for the two hundred and twenty-seventh time this hour, stick with it. Take a deep breath, have a cup of tea, and clear your mind. Go back to the keyboard and start again. Consistency is key. This goes doubly for memorizing fingerings and modified keys: Make sure you always hit the key with the correct finger. Some keys might be awkward at first, but you’ll get used to it eventually.
Accurate is the new fast: It doesn’t matter how fast you are if you hit the wrong key half the time. It’s important to focus on accuracy, especially at the start. Speed will come.

If you decide to embark on this exciting journey, be aware: it won’t be easy and the learning curve is pretty steep; but it’s not as difficult as you might think. Time and tenacity is all you need: Once you start, you simply need to stick with it.

Good luck!

Command Line Control: awk

Thomas Hartmann — Sat, 08 Jun 2019 15:19:18 +0000

Unless you spend a whole lot of time working primarily on the command line, awk is one of those commands you’ve likely come across a few times, but never really learned how to use properly. Similar to my recent adventures with xargs, though, I recently came across a little use case where I could benefit from using it, so I decided to sit down and look into it.

Even more so than with xargs, this is a really powerful tool, but let’s focus on the basics and see what we can make of it.

What is `awk`?

awk is a command that processes text using programs written in the AWK language.

To understand what it can do, understanding the language it uses is probably a good place to start. Let’s use the words of Alfred Aho—one of the creators of the AWK language—from a 2008 Computerworld interview to get a rough grasp of what it is:

AWK is a language for processing files of text. A file is treated as a sequence of records, and by default each line is a record. Each line is broken up into a sequence of fields, so we can think of the first word in a line as the first field, the second word as the second field, and so on. An AWK program is of a sequence of pattern-action statements. AWK reads the input a line at a time. A line is scanned for each pattern in the program, and for each pattern that matches, the associated action is executed.

That may be a bit dense, so let’s take it apart:

“AWK is a language for processing [..] text”

AWK is a domain-specific language (DSL) focused on text processing. The awk command expects that its first argument is a script or a string in this language.

“AWK reads the input a line at a time”

AWK is line-oriented and works through the input line by line. (It’s actually record-oriented, but the default separator is a newline character, so this is the same thing by default.)

“Each line is broken into a sequence of fields”

Each word in a line maps to a field. These fields are accessed with the $ operator, e.g. $1 for the first word, $2 for the second, and so on. $0 is the whole line. By default, fields are delimited by whitespace (which is why I’ve called them words) but this can be customized.

“An AWK program is of a sequence of pattern-action statements”

This means it’s a sequence of predicates with actions. If a predicate evaluates to true, perform the specified action. If no predicate is specified, it will always evaluate to true, and if no action is specified, it will default to printing the whole line.

Huh? Yeah, this is still a bit confusing, but maybe some examples will make it clearer:

Patterns

This is a predicate to check each line against. It usually takes the form of a regex enclosed in forward slashes. /foo/, /b[ar]?/, /^baz/, /(fizz|buzz)$/ are all examples. Most of the regex skills you have will be applicable here (character sets, character classes, alternations, etc.).

You can also match specific fields against a regex. Only want to match lines where the second field contains ‘cheese’? $2 ~ /cheese/

The pattern can also consist of functions and comparisons; so if you wish to act only on lines that aren’t empty: length > 0

If no pattern is given, every line will match.

Actions

These are commands telling awk what to execute. They are enclosed in curly braces ({}). This is where you might instruct awk to print a certain field of the string—print $3, for instance—or increment a counter if you’re counting words: word_count += NF (yeah, I’ll get to what NF means in a bit).

If no action is given, awk will print the matching line.

Basic auth `awk`

That’s a quick overview of how the language is structured. Before we start playing with it, let’s explore some of the features.

Built-in variables

awk has a number built-in variables, and while I won’t cover all of them, these are the ones that I’ve found the most useful:

NR: Gives you the line number of the current match. Could be used for adding line numbers to an input file: awk '{print NR, $0}'. Or maybe looking for lines that contain ‘ice cream’ is more your speed: awk '/ice cream/ {print NR}'.
NF: This is the number of fields in a line. Useful if you’re looking for the last field, either for finding out how many fields are in a line or for seeing if it contains a pattern: awk '$NF ~ /out/ {print NR}
FS: The field separator value. This is what awk will use to split each line into fields. By default this is whitespace. If you have a file full of comma-separated values and want to split each line on commas instead of whitespace: BEGIN {FS=","}
RS: This is the line (‘record’) equivalent of the field separator. The default is \n. Say you want to print your PATH over multiple lines: echo $PATH | awk 'BEGIN {RS=":"} {print}'

`BEGIN` and `END`

awk lets you supply commands to be executed at the start and end of a script by using BEGIN and END. While they may seem like it, they’re not really special at all. Instead, think of them as being patterns that evaluate to true before anything is evaluated and after everything is evaluated, respectively.

BEGIN could be used to set the field separator or initialize a variable.

END is useful for printing out results accrued through the life of the program, such as a word count. If we bring back our word counting example: awk '{word_count += NF} END {print word_count}'

Functions and conditionals

Like most languages, AWK has a number of built-in functions (such as the print and length functions we saw earlier) and also lets you define your own functions if you so please. This is probably overkill for most trivial operations but could come in handy in certain cases.

And AWK has conditionals too! While you can use the if else construct within actions, I’d like to highlight that you can do conditional statements based on the supplied pattern. I.e. awk '/foo/ {print "FOO"} {print "bar"}' will print ‘FOO’ for lines that match /foo/ and ‘bar’ for lines that don’t.

Applications

Now that we've got some idea how it works, what can we use it for? Let's
look at some sample applications for it:

Sorting lines in a file by length

Here's a fun little application: Let's take a file, count the number of
characters in each line, and then sort it based on the number of
characters:

awk '{print length, $0}' <file> | sort -n

And if you want to exclude empty lines, try this:

awk 'length > 0 {print length, $0}' <file> | sort -n

Friendly `PATH`

This is the same one that was listed above, and is useful if you're
looking for certain entries that can easily get lost when the path is on
one line:

# bash, zsh
echo $PATH | awk 'BEGIN {RS=":"} {print}'

# fish
echo $PATH | awk 'BEGIN {RS=" "} {print}'

Counting words

Another example that was used previously. Count the words in a file or
any input stream:

awk '{word_count += NF} END {print word_count}'

Parsing git logs

Maybe your team has agreed that all commit messages should start with
#<task_no> if it relates to a task and #-- if it doesn't. To find
all commits that relate to a specific task—say #42—we could do this:

git log --oneline | awk '/#42/ {print $1}'

Or how about finding the ratio of commits that belong to a task versus
those that don't?

git log --oneline --no-decorate | \
awk '$2 ~ /#[0-9]+/ {task += 1} {notask +=1} \
  END {printf "Tasks: %d\nNo-tasks: %d\nTask to no-task ratio: %f", \
  task, notask, task / notask }'

(Yeah, the printf function works pretty much the same as in C—so much
so that I haven't looked it up yet!)

Killing processes

This is actually what triggered me to look into awk and what
eventually led to this post being brought into existence.

There are a number of ways to kill applications from the command line,
and for a while, my default fallback has been the classic ps aux | grep <app> combo.

While this lists out all the relevant processes and their process IDs
(PIDs), you then have to manually copy the PID and put it into the
kill command to shut it down. This is annoying at best, and gets even
worse if the process has spawned children that we want to take down as
well.

How do we deal with this? Well:

ps aux | awk '/<app>/ {print $2}' | xargs kill

ps lists all processes on the system
awk then goes over every line that contains the application name,
extracts the second whitespace-delimited word—which in this case is
the PID—and prints that.
For the last stretch, we use xargs to feed the PIDs into the
kill command, thus killing all the processes.

This works just fine, though it comes with the caveat that it'll also
try and kill the awk process (because the application name is part of
the command, so it gets listed by ps), but that's only a minor
annoyance, and I'll leave fixing that as an exercise for the reader.

Now, I'm sure you can make killall do something quite similar to this,
but I've found this way to be more effective (by which I mean: closer
to what I expect).

Closing up

Phew. We've learned a lot today, and this post grew much longer than
what I had originally imagined—about four or five times longer—but it's
been quite the journey and I'm glad you were there with me, partner. I
hope you've gleaned some new insight too and that you'll find some
application for this in your day-to-day.

See you next time!

xargs and the unruly tags: A tale of two commands

Thomas Hartmann — Sat, 25 May 2019 14:01:57 +0000

I thought I was really clever when I configured my blog's CI/CD pipeline to tag commits that got deployed and push the tags back into the repo, but I’m rarely as clever as I like to think: I had forgotten to put the proper checks in place to avoid these tag pushes triggering subsequent runs of the pipeline, and things got a little … out of hand.

I’d gone to bed just after pushing an update, and when I arose to check on it, I found that the deploy tagging stage had been running over and over and over and over and … you get the point. Thankfully, it had failed after about 130 rounds, so it could have been a lot worse, but I was left with a large amount of useless and unwanted tags in the remote repo.

So how do you fix something like this? Yup, xargs to the rescue!

Where there’s a will …

At first, I didn’t really know how I’d go about it. I was hoping git would have some nice, built-in functionality for mass-deleting remote tags, but while I have found in retrospect that it does (see the postmortem), I couldn’t find it at the time.

However, because all the tags were for a specific commit, I did know that I could list all the relevant tags separated by newlines, using git tag --contains <SHA>.

So, with some helpful advice from Stack Overflow and this guy, I constructed this little command which sorted me out just fine:

git tag --contains 9216e97ce7e66090f79eba4d1abe6548d72dd638 \
| xargs -I % git push origin :refs/tags/%

Now, I’d come across xargs before, even done the ol’ copying and pasting from Stack Overflow trick, but it had always looked really complicated and no-one had ever told me why I’d need it or what it does; so I just carried on in blissful ignorance. Not this time, though. It was time to figure out what was going on.

Groking `xargs`

The way xargs was sold to me was: “execute a command for each item in a list”. It’s actually more powerful than that, but that’s a great place to start.

Let’s use the man page to find out what that -I % bit means :

-I

replace-str: “Replace occurrences in the initial-arguments with names read from standard input”

The string to use to indicate where to place arguments in the command to run. In the command above, we chose to use %, but you’re not limited to this.

Similar to printf and format strings in general, this places your arguments at your desired place in the command. In our case it both limits us to using one argument (tag) at a time, and it lets us append it to :refs/tags/ without being separated by a space.

That means that in the above snippet, xargs would, for each tag listed, run the command git push origin :refs/tags/<tag_name>, which pushes that tag with an empty reference, thereby deleting it.

If all you want is to put the argument at the end of the command, you can even do without the -I. Say you want to recursively delete all the .swp files in a directory:

find -name "*.swp" | xargs rm

Be aware, though, that without either using a -I or -n (to limit the number of arguments to use for each command), xargs will split the list you give it into sizeable chunks and apply as many arguments to the command as it can each time. That means that in this case, it’d likely end up looking something like this:

rm a.swp b.swp c.swp ...

which is usually fine and what you want, but keep this in mind for when it isn’t.

This is only scratching the surface of what xargs can do, but it’s enough to make it do some pretty heavy lifting. It might not be something to reach for very often, but for when you do need it, it’s a great tool to have in your belt.

Postmortem

Now, you might have noticed that I did a git push for each tag that I was deleting, and you might be thinking that for over a hundred tags, it must have taken quite some time. You would be right. Luckily, I was working on something else, so I could happily let it run in the background. But we can do better!

xargs has an option -P or --max-procs, which you can use to decide how many processes to run in parallel. The default is 1, but if you set it to 0, it will run as many as it can. This could have saved us quite some time, assuming git would let us run multiple push operations from the same repo at the same time. But there is an even better way:

As outlined in this Stack Overflow response, you can use a whitespace-separated list of tag names (<tags>) with git push; so we could have run git push --delete origin <tags> to achieve the same outcome as deleting them one by one.

If we rewrite the command from earlier, we can both simplify it and do it all in a single push:

git tag --contains 9216e97ce7e66090f79eba4d1abe6548d72dd638 \
| xargs git push --delete origin

… yeah, that would have been a lot more efficient 😅