Forem: David Cantrell

G̶o̶d̶ ̶i̶s̶ ̶m̶y̶ ̶C̶o̶p̶i̶l̶o̶t̶ Copilot is my new god

David Cantrell — Mon, 27 Apr 2026 17:06:51 +0000

I spent some time this weekend noodling around with Github Copilot, using the copilot-cli tool.

A while back I had created a little Electron app that searched for and displayed information about Unicode characters. I didn’t really know what I was doing, either with Electron or Javascript, and couldn’t find much in the way of correct and up-to-date beginner-level documentation on things as simple as how to wire an application together - the doco was good enough for a "Hello World" but nothing beyond that. As an aside, I have the same gripe about lots of languages and frameworks - good luck figuring out how to drive things like make and the linker from any online C documentation. Anyway, I muddled through, and you can see what I did by hand here. It works, and the same code produces both an Electron app, and a stand-alone web page, and a CLI tool, but it’s vile, especially the build process.

The updates since then are here. Copilot wrote pretty much all of that with only a few very small manual tweaks. The initial prompt that produced most of the work was something like “Make this project more idiomatically Electron and Javascript-ish”. I worked with the evil hallucination engine for a few hours, including prompting it to add tests and a Github workflow for CI. I mostly gave it its head and let it do whatever it wanted although there were a few small things where I told it to do things slightly differently. Overall, I was very impressed with the result. With my lack of knowledge of modern HTML/CSS and Javascript there’s no way I could have done this myself.

I then thought “there’s so much Javascript/Electron stuff out there for it to have learned from, let’s try something different” so I took the next little tool improvement on my to-do list and asked it to write it for me. I’d previously had a shell alias alias highlight="ack --color --passthru" - you pipe stuff in, give it a perl regex, and it highlights anything matching. I wanted to be able to control the colours, by saying something like cal | highlight --red '25|26' --green '27|28'. So I prompted copilot with something along the lines of “write a shell script that will take an optional --color_name parameter followed by a perl regex, read input from STDIN, and spit it to STDOUT with any matches highlighted in that colour. It should allow any number of such matches”. It took a few attempts, mostly it iterating towards a solution while it tested and re-tested what it had done, and required very little in the way of corrective input from me. It also wrote the documentation when I told it to. I thought I’d try giving it something that is confusing as hell even for experienced shell scripters, and asked it to make the script able to spit out its own shell tab-completion function, but it managed that too. I can now type highlight --gr<tab> and it auto-completes green.

For another little task I gave it in perl, I told it to make sure that all the changes it made to an existing code base were 100% covered by tests, including branch and condition coverage. It went away and did that. In fact it also wrote some missing tests for stuff that I had previously written. And then it wrote a test that will fail if I ever make code changes in the future that aren’t 100% covered.

At various points it said things like “I need to consult the documentation” and (after asking permission) went off and read the docs for things like Devel::Cover, and the Electron docs. It even said “I can’t figure this out from the Electron docs” (oh good, it’s not just me that thought it wasn’t very good!) and went and read the Electron source code to figure out what’s what.

I was on a bit of a roll by now, so I asked it to fix a long-standing bug in one of my perl modules, which consistently failed its tests on OpenBSD, NetBSD and OmniOS, and intermittently failed on FreeBSD, while always passing on Linux and Mac. I’ve been banging my head on this for over a year, and no-one I’ve asked for help has been able to figure it out either. It looked at the code, and at the failure reports in the OpenBSD test workflow on Github, and told me what the problem was and how to fix it. I didn’t believe it, so I said “verify that, here’s the address of an OpenBSD machine you can ssh to”. So it did. I wasn’t happy with its suggested fix - it would have fixed the test, but subtly broken interactive use of the tool by buffering STDERR output until vomiting it all up at the end - so I explained that to it and it did a massive rewrite of my code. I was rather dubious but having tested it myself and very carefully reviewed what it did - yep, it got it right. The code it produced is a little hard to read, but that’s because it’s dealing with some complicated inter-process communication and control. It wrote tests without prompting. Perhaps my only criticism is that it didn’t comment the code.

This thing is FUCKING MAGIC. I will be using it a lot for creating new stuff, and for updating small code bases. I don’t think it stands a chance in hell of being particularly useful for the software archaeology part of maintaining a large legacy application - which is what most devs spend most of the time doing at work - but I am going to at least try using it to make small updates once I have figured out what/where they should be. The quality of the code it produces is fairly good. Many of us could of course do better, but what it produces is generally readable. Copilot does require a little hand-holding and its output needs a light editing pass before it’s worthy of being put up for code review by other humans, but no more than the code I write myself does.

I strongly recommend that you give it a go.

Automatic cross-platform testing: part 7: 32 bit, again

David Cantrell — Fri, 27 Feb 2026 21:23:41 +0000

I have written about this before, but I've had to do it again. I noted previously that Github's actions for things like checking out a repository and for retrieving build artifacts are now 64-bit only. You've been able to work around that by using older versions of the actions but I didn't want to do that so I'm taking a different approach.

My new method builds on work I've done previously to pull all my testing for different Unixy platforms together into one workflow. This tests on 64-bit versions of NetBSD, FreeBSD, OpenBSD, IllumOS (that is, the modern version of Open Solaris), and Linux. And as of earlier today it also runs the tests on 32-bit Linux. As in my previous blog entry it's still really 64-bit hardware, with modern x86 ISA extensions, but it's a 32-bit OS image with 32-bit addressing, userland, compiler toolchain and so on. If you absolutely must have a pure 32-bit x86 environment then you'll need to acquire some obsolete e-waste and run it yourself. Something like this based on the Vortex86 chipset looks like a good choice.

There are only two interesting bits of the workflow. The first implements enough of Github's actions/download-artifact for my purposes. In the YAML file it's all one line because YAML and everything that uses it is hateful, I've tidied it up here:

curl -L -H "X-GitHub-Api-Version: 2022-11-28"
        -H "Authorization: Bearer ${GH_TOKEN}"
        -o dist-for-install.zip
  $(curl -L -s
         -H "Accept: application/vnd.github+json"
         -H "Authorization: Bearer ${GH_TOKEN}"
         -H "X-GitHub-Api-Version: 2022-11-28"
         https://api.github.com/repos/${GH_REPOSITORY}/actions/artifacts
         -o -
    |jq -r '
         [.artifacts.[]|select(.expired == false)]
         |max_by(.created_at).archive_download_url
    '
  )

The inner curl uses the Github API to fetch a list of all the build artifacts that have ever been created for this repository, as JSON, which it filters using jq to find the most recent one which hasn't expired, and then extract its archive_download_url field. The outer curl then fetches that.

The second bit interesting bit is specific to testing my perl code. Perl can be built on 32-bit platforms to support either 32 or 64 bit integers. On 64-bit machines perl integers are always 64-bit - and there's been a note in the docs for 20 years about how a build-time parameter for 32-bit ints may be added later! I need to test with 32 bit ints, but most OS packages, even on 32-bit machines, will provide a perl with 64-bit ints. So I need to build my own. This is dead simple if a little wordy:

curl -o perl-5.42.0-src.tar.gz https://cpan.metacpan.org/authors/id/B/BO/BOOK/perl-5.42.0.tar.gz
tar xzf perl-5.42.0-src.tar.gz
cd perl-5.42.0
sh Configure -de -Dprefix=$HOME/perl-5.42.0-installed
make -j $(nproc)
make install
$HOME/perl-5.42.0-installed/bin/perl -MCPAN -e 'install qw(App::cpanminus)'

The call to the Configure script is where the OS packaging people make it have 64-bit ints by adding the -Duse64bitint flag. Normally you would make test too but I've satisfied myself that it does indeed pass its tests using this Docker image, so I skip that step to save a lot of time.

Finally, I can extract my software and test it. It is bloody irritating that Github automatically wrap my tarball in the obsolete zip format. I blame Microsoft:

unzip dist-for-install.zip
mkdir dist-for-test
tar -C dist-for-test -xzf *.tar.gz
cd dist-for-test/*
$HOME/perl-5.42.0-installed/bin/cpanm --installdeps .
$HOME/perl-5.42.0-installed/bin/perl Makefile.PL
make test TEST_VERBOSE=1

And there you have it - how to test your code automatically, using Github actions, on a 32-bit platform.

Unicode-Search - my first Electron app!

David Cantrell — Sat, 23 Nov 2024 11:35:46 +0000

I am a Unix greybeard who mostly works in perl and shell programming, with a bit of dabbling in C and Rust, but for a long time I've wanted to do some Javascript. I just needed an interesting enough project to use it in. Well, at work I spend a lot of time toiling in the Unicode mines, fixing mojibÃ©ke errors in a somewhat elderly code base. As part of that I wrote a guide for my colleagues, which points them at a very useful reference website run by Xah Lee. That has been around for years, but as the single maintainer of several projects myself I am wary of relying too much on projects with a single maintainer. So I decided to write my own, and that this would be my interesting little Javascript project.

It seems like a sensible choice for learning a new language, as it's not doing much, just looking up data in a static structure and displaying it to the user. It's all synchronous code, there's no I/O beyond updating a web page, and the user interface can be dead simple.

Electron's quick start guide is excellent and gave me the basic boilerplate for a "Hello World" application, and then the rest was just lots (and lots, and lots) of looking things up on Stack Overflow and W3Schools. From start to finish it took about 6 hours, and my code is on Github.

I'm quite sure that those of you skilled in the ways of Javascript will find the code "idiosyncratic", but that's OK, these are baby's first steps. If you have any useful tips for improving it, and can explain them simply, then they would be most welcome. I would also welcome pull requests that make it stop looking ugly as sin. Learning how to make rounded corners and stuff in CSS wasn't in scope :-)

Update: because working on the command line is always better than using a GUI, it has sprouted a CLI tentacle. The same code does most of the work of parsing user input and looking up characters, it just has a different function for spitting the results back out to the user.

Programmable tab completion with bash

David Cantrell — Sun, 10 Nov 2024 23:27:15 +0000

We all use tab completion in the shell, but have you stopped to think about how it works? How with one command it will auto-complete only directory names, with another directories and filenames, but with another it will complete branch names for your VCS, for example? It's programmable! And so we can bend it to our will!

I recently had an annoyance. I use a tool called ts (task spooler) to run some CPU-intensive processes in the background, and to queue them so that only one such process runs at once. It is invoked like this:

$ ts command arg1 arg2 arg...

I wanted to be able to use tab completion for the command, and to then use that command's tab completion for subsequent arguments. After much swearing and cursing - the documentation in the bash manpage is not the best - and a bit of help from some nice people on Stack Overflow I came up with this:

complete -o bashdefault -o default -F __ts_bash_completions ts

__ts_bash_completions () {
    COMPREPLY=()

    if [ "$COMP_CWORD" -eq 1 ]; then
        COMPREPLY=($(compgen -c -- "${COMP_WORDS[COMP_CWORD]}"))
    else
        local command_completion_function="$(complete -p ${COMP_WORDS[1]} 2>/dev/null|sed 's/.*-F \([^ ]*\) .*/\1/')"

        if [ ! -z "$command_completion_function" ]; then
            COMP_CWORD=$(( COMP_CWORD - 1 ))
            COMP_LINE=$(echo $COMP_LINE|sed "s/^${COMP_WORDS[0]} //")
            COMP_WORDS=( "${COMP_WORDS[@]:1}" )

            $command_completion_function "${COMP_WORDS[0]}" "$2" "$3"
        fi
    fi
}

Let's go through it in detail. The first line tells the shell to use the function __ts_bash_completions when the user is typing the ts command and its subsequent arguments:

complete -F __ts_bash_completions ts

We then define that function.

__ts_bash_completions () {
    COMPREPLY=()

and the first thing we do is create an empty array COMPREPLY. bash completions populate this global variable to tell the shell what options are available. We then see how many complete words there are on the command line:

    if [ "$COMP_CWORD" -eq 1 ]; then

COMP_CWORD is another global variable that contains the number of complete words currently in the command. If that is 1 then the only word currently in the command is ts itself, we want to autocomplete the name of a command:

        COMPREPLY=($(compgen -c -- "${COMP_WORDS[COMP_CWORD]}"))

compgen (completion generator) generates a list of all the commands available in the $PATH which begin with ${COMP_WORDS[COMP_CWORD]}. This introduces yet another magic global variable, COMP_WORDS is an array (zero indexed) of all the words currently on the command line, including the one currently being typed, which may be empty. We pick the last one and pass that to compgen for it to use as a filter.

At this point we've tab completed the name of the command that ts is to run and deserve a beer.

Now we need to deal with tab completion of the arguments to the command that we want ts to run. First we need to find what function is used for completions for that command:

    else
        local command_completion_function="$(complete -p ${COMP_WORDS[1]} 2>/dev/null|sed 's/.*-F \([^ ]*\) .*/\1/')"

complete -p tells you exactly how tab completion is configured for a given command, and spits it out in the form of the command to use to configure it. The function will be the argument to the -F option, so we use a rather crude sed invocation to extract it, if it is present. The resulting string will be empty if no completion function is defined. We then check that it isn't empty and play around with the contents of the various COMP_* variables, setting them up to pretend that ts isn't involved:

        if [ ! -z "$command_completion_function" ]; then
            COMP_CWORD=$(( COMP_CWORD - 1 ))
            COMP_LINE=$(echo $COMP_LINE|sed "s/^${COMP_WORDS[0]} //")
            COMP_WORDS=( "${COMP_WORDS[@]:1}" )

We decrement COMP_CWORD because we want to ignore one of the completed words - that being ts itself. We remove ts from the start of COMP_LINE - which is a string containing the entire line of input. And we remove the first element from the COMP_WORDS array. Finally, we run the command's own completion function which will set COMPREPLY for us:

            $command_completion_function "${COMP_WORDS[0]}" "$2" "$3"

It appears that the calling convention for completion functions has changed over time. At some point in the past they were called with arguments, which only contain the command name and the last two arguments on the command line, then they were changed to instead use those global variables. In this case we can just use the arguments that were passed to our function altering only the first one, the name of the command whose arguments we are completing. Depending on how modern the completion function is for a given command, our ts completion needs to support both calling conventions.

There's one final wrinkle, which the observant amongst you may have noticed when I started going through this line by line. In my script I have -o bashdefault -o default when defining the relationship between ts and its completion function. Those are to cope with the case where the completion function leaves COMPREPLY empty. If that happens then -o bashdefault applies bash's own defaults for tab completion, ie helps you pick a filename, and if that doesn't return anything -o default applies readline's defaults. You can see this in action for git's completions. If you type git log <tab> in a git repo you get a list of branch names, and if you type git log foo<tab> you get a list of branches whose names begin with foo. But if there are no such branches you get a list of files whose names begin with foo, which is the shell's default action.

Anyway, after a great deal of wrestling with poor documentation, that's a little annoyance dealt with, and exactly the same code can also be used for other similar commands such as sudo and nohup. If you think you'll find it useful then the code is on Github and it's also in the ts mercurial repo so will no doubt be in a future release.

The typeface you didn't know you wanted and were trained to hate

David Cantrell — Tue, 05 Sep 2023 14:59:32 +0000

For the last several weeks I've been using Comic Mono in my terminal. It's a fixed width typeface based on the font that we've all been trained to despise and sneer at for almost 30 years, Comic Sans.

But you know what? Comic Mono works really well at small sizes even with my becoming-elderly failing eyes and my inability to always find my reading glasses. I can see plenty of code in my editor with lots of context. On a 40" screen at arm's length I can comfortably get 100 lines of text - so lots of context for the code I'm working on - in a terminal using 12pt text and read it easily.

Give it a go for a few days!

Number::Phone is now 64-bit only

David Cantrell — Tue, 20 Jun 2023 10:40:02 +0000

A couple of years ago I wrote that I would be dropping support for perls with 32-bit integers in Number::Phone. Well, that two year deprecation cycle is up, and last night I switched over to the new database format, the software for which requires 64-bit integers. As of the next release it will use under 10MB of disk space instead of about 100. An order of magnitude improvement, and it was fun to write as well.

Perl has supported 64-bit integers on all reasonable platforms for the last 20 years, even on those platforms which still use 32-bit pointers, so I don't expect that anyone will notice any change at all except that their hosted Docker containers will now be faster to deploy and cheaper to run.

cpulimit annoyed me so I improved it

David Cantrell — Sat, 25 Mar 2023 19:07:07 +0000

Previously | more previously

A few days ago I discovered cpulimit. It's a great tool that nicely (haha) complements nice. Where nice is normally used to reduce the amount of CPU a process uses by changing it priority, a niced process can still end up using more CPU than you want, and will of course use all that it wants if nothing with a higher priority comes along.

But sometimes you want to restrict a process to using no more than some particular fraction of CPU time regardless of priority. A good example is when you don't want those noisy PC fans to kick in and you don't care how long a job takes because whether it finishes in 15 minutes or 8 hours it's still going to finish while you're asleep.

cpulimit is perfect for this, and is simple to use. An invocation like this:

cpulimit -l 50 somecommand ...

will run somecommand but restrict it to only use 50% of a CPU. It does this by forking a watchdog process that periodically checks to see how much work somecommand is doing, and if it's used too much briefly pauses it by sending the STOP signal. After a little while it will un-pause it with the CONT signal. Of course, because the load on your machine from other processes will never quite be constant, cpulimit rarely hits the target exactly, but it gets close enough.

But I wanted to tweak it a bit. I wanted to be able to interactively "turn the volume knob" so that I could give somecommand more or less CPU whenever I fancied. The result is a pull request which unfortunately is unlikely to ever get merged, as the original author hasn't touched the project in years, but if any of you want the nifty new feature applying the patch and building your own custom cpulimit is pretty easy.

How the patch works is simple. It installs signal handlers for SIGUSR1 and SIGUSR2 which respectively increase and decrease the CPU allocation by 1%. Want to turn it up by 50%? Just write a little shell loop to send the signal 50 times:

for i in $(seq 1 50); do kill -SIGUSR1 $pid; done

Determining which process to send the signal to is a bit tricky, as there are two cpulimit processes running. There's the first one, which is just waiting in the background for somecommand to finish, then there's the watchdog that got forked off. It's the watchdog you want to send the signals to. You can tell which is the watchdog as it will generally have a higher PID and be using a little bit of CPU. If you are cpulimiting multiple processes then you can tell which watchdog is related to which process because the watchdog will have the command and its arguments on its command line. For example:

$ ps aux|grep ffmpeg|grep -v grep
david   90311 103.3  0.7 36105472 485448 s011  T     6:42pm   1:07.89 ffmpeg ...
david   90312   5.6  0.0 34221044    828 s011  S     6:42pm   0:02.82 cpulimit -l 100 ffmpeg ...
david   90310   0.0  0.0 34122740    796 s011  S     6:42pm   0:00.01 cpulimit -l 100 ffmpeg ...

You can see here that I asked cpulimit to allow ffmpeg to only use 1 CPU of the several available on this machine (ie to use 100% of a CPU - on modern machines the maximum allowed is the number of CPU cores * 100%). My shell accordingly started process 90310 which forked and execed ffmpeg with pid 90311 and forked the watchdog process as pid 90312. The watchdog is using a little bit of CPU. It is therefore to process 90312 that I should send signals.

$ for i in $(seq 1 400); do kill -SIGUSR1 90312; done

That will send the "turn it up a bit" signal 400 times, so ffmpeg is now limited to at most 500% of a CPU, and a few moments later:

$ ps aux|grep ffmpeg|grep -v grep
david   90311 497.0  0.8 36105472 507160 s011  T     6:42pm  12:42.37 ffmpeg ...
...

we can see that ffmpeg is running a lot harder, now taking just under 500% of the CPU.

TIL: diff-so-fancy; and some funky git config

David Cantrell — Wed, 25 Jan 2023 19:12:13 +0000

I just discovered diff-so-fancy, and very nice it is too. I immediately added it to my standard git config, which is semi-automatically installed on every machine I use. However, I've not (yet) installed diff-so-fancy on all the machines I use, and for those platforms for which it's not packaged I probably won't bother installing it from source.

But if I just follow the author's instructions which amount to adding this to my ~/.gitconfig:

[core]
    pager = "diff-so-fancy | less --tabs=4 -RFX"

then git diff will break:

$ git diff HEAD^
diff-so-fancy | less --tabs=4 -RFX: diff-so-fancy: command not found

but there's an easy fix! Whatever is in pager is just shell code, so this works:

[core]
    pager = "if [ ! -z \"$(which diff-so-fancy)\" ]; then diff-so-fancy | less --tabs=4 -RFX; else less; fi"

The output from git diff is piped into that little script. If diff-so-fancy is installed (ie if "$(which diff-so-fancy)" is not zero-length) then it does exactly what diff-so-fancy's author suggests. Otherwise, if diff-so-fancy isn't installed, just run less.

A Rant

David Cantrell — Fri, 20 Jan 2023 17:54:56 +0000

How the hell, given these interests:

is this post in any way relevant and deserving of being promoted to me on the front page of this site?

And no, negative weights being treated as "anti-follows" isn't enough. There are so many topics I'm not interested in that configuring them all is just too much work.

Number::Phone release candidate

David Cantrell — Thu, 12 Jan 2023 21:52:39 +0000

I've recently tackled a feature request that's been sitting in the backlog for several years, to use libphonenumber's data to validate numbers with non-geographic country codes. That's codes like +800 (international freephone), +870 (Inmarsat) and so on.

Previous versions would sort of work with these numbers, in that they could be instantiated as objects, but there was no information on validity or any other properties.

For example here it's looking at an Iridium number. First with the previous release:

Number::Phone->new("+881 672520333333333333333732")->format()

+8816 72520333333333333333732

Number::Phone->new("+881 672520732")->is_mobile()

undef (ie, dunno)

and with the release candidate:

Number::Phone->new("+881 672520333333333333333732")

undef (ie, not valid)

Number::Phone->new("+881 672520732")->format()

+881 6 725 20732

Number::Phone->new("+881 672520732")->is_mobile()

1

Much of the code is auto-generated, and this involved quite a bit of hacking around in the build system and object instantiation, so there's the potential for exciting new bugs even for numbers in boring old geographic country codes, although I didn't have to change any of the existing tests so I am hopeful that I avoided that. Here are the changes.

I'd be grateful if you could test this, and assuming that no-one finds any bugs it will be in either the March or June quarterly release. You can download it from here.

Automatic cross-platform testing: part 6: 32 bit Linux

David Cantrell — Sun, 17 Jul 2022 17:25:26 +0000

Previously on this channel

Testing on 32 bit Linux

Just about every virtual machine out there in the various CI platforms and hosting companies is a 64 bit x86 machine. But I wanted to test my code on a 32 bit machine as well. Unfortunately without having real 32 bit hardware available, or emulating it on more modern hardware, you can't quite do this, but you can at least run an OS and use libraries compiled for 32 bit hardware, 32 bit memory management, and with 32 bit pointers.

It's surprisingly easy, and lot less hackish than the solution I found for testing on OpenBSD.

Github workflows normally run directly on whatever host type you specify in the runs-on option. But you can instead tell them to run in a container, which can be anything available on Docker Hub. Jobs running in a container can have all the same steps that you would have in a non-containerized job, including using other Github workflow actions.

To test my code on 32-bit Linux I use the i386/ubuntu:latest image. I am also using the same technique to run tests for one of my projects, which is sensitive to exactly which Linux distribution you are using, on Arch Linux, using the archlinux:latest image.

Virtual machine efficiency

David Cantrell — Wed, 23 Feb 2022 21:27:22 +0000

I'm one of the CPAN testers, a bunch of people who test everything that gets uploaded to the CPAN. We test with multiple different builds of perl on multiple operating systems and different hardware platforms, and send reports of failing tests. When I'm wearing my Author Hat as opposed to my Tester Hat I very much appreciate test failure reports from platforms I don't have access to, or which are generated using perl builds with features I hadn't considered. I test on 5 different operating systems (6 if you count 32 bit x86 and 64 bit amd64 Linux separately) and 40 different builds of perl.

I mostly test in virtual machines, running on a single Intel Mac Mini. Using VMs means I don't need multiple noisy real machines of course, but it also makes them all easier to manage.

Until very recently I used Virtualbox for this, mostly because back when I started running VMs on a Mac it was the best free game in town. But it's not the most efficient and I have now switched to the much better, and still free, UTM, which is a simple GUI wrapper around the notoriously hard to configure QEMU, patched to use Apple's hypervisor.

This chart shows CPU usage over the last month. The first half is when I was using Virtualbox. Running all those test builds usage was a fairly constant 50% of what was available. The VMs could keep up with the workload, but there was little capacity for more despite all that unused CPU. No matter how many CPUs I allocated to the various VMs, it just wouldn't use all the resources. It's very clear in the chart when I started migrating VMs from Virtualbox to UTM. CPU usage starts spiking much higher - indeed, it reaches 100% and stays there when the VMs test a batch of uploads - but then drops to almost zero when that batch has finished. The work is done faster.

Virtualbox is also a memory hog. This chart shows that I couldn't really add any more Virtualbox VMs, as there was little memory left. And worse, most of what was in use (the blue section of the chart) is "wired" memory, in kernel-space. Yes, most of the memory Virtualbox was using was being used by its kernel extension. It is again clear when I started migrating VMs to UTM. Memory usage falls off a cliff, and "wired" memory usage falls even faster, with a far greater proportion of the lower total being just normal user-space memory (the red section of the chart).

Now that I've got back all that wasted memory I am, of course, beginning to fill it with more VMs, this time for development as opposed to testing. I have recently become a larval stage Rustacean and have dived headlong into reporting obscure platform-specific bugs.

But you can see from my dock that I've still got three Virtualbox VMs running. Why?

While UTM is much better at resource usage, it's not perfect. Virtualbox seems to emulate more of a system instead of passing it through to the hypervisor and so is better for running more unusual OSes. I have two VMs there running Illumos which I have yet to figure out how to boot in UTM. A problem that I think is related to UTM's greater use of the hypervisor is that you can't suspend and resume VMs that use it instead of being emulated, so I've still got a few VMs hanging around in Virtualbox which spend most of their time suspended. Finally, what stops me from using UTM at work is that you can't use it as a Vagrant provider. This is incredibly annoying, as the lack of a decent virtualization application makes the otherwise very nice M1 Macs nothing more than pretty toys. I expect that this glaring lack will be fixed within the next couple of years.