Forem: Ivan Fraixedes

Moved to the heap in Go standard pool

Ivan Fraixedes — Wed, 15 Oct 2025 17:58:25 +0000

The other day I was applying some changes to make a logic that was processing a list of "objects" sequentially to do it
concurrently¹.

The logic itself doesn't matter for the purpose of this blog, what it only matters is that that to avoid a slice
allocation of N element on each batch that was processing, the sequential logic was reusing a slice, so to reuse slices
on the current implementation, I used the standard sync.Pool.

In this post, I will use a slice of bytes as simple example, to make clear what my explanations and avoid confusion with
the all the stuff that the original implementation entails.

Use pointers

Reading the documentation, one can realize (if you slide to the example) that
pool must use pointers to avoid allocations

var bufPool = sync.Pool{
    New: func() any {
        // The Pool's New function should generally only return pointer
        // types, since a pointer can be put into the return interface
        // value without an allocation:
        return new(bytes.Buffer)
    },
}

Pointers to slices

Because the pool was storing slices, I started to mess with pointers to manipulate the slices gotten from it.

Let's see in code, what I mean

s := pool.Get().(*[]byte)
s2 := *s
s2 = s2[:0] // Reset the slice before use
s2 = append(s2, 10)

// you'll do something else here and  when you don't need this slice anymore
pool.Put(&s2)

This doesn't seem that much, but if you account that, the current logic has to keep the gotten slice during iterations
until the N elements configured in that batch is reached, you get in the call back function which is called for each
element retrieved the following logic

s = pool.Get().(*[]byte)

ranger.Range(func() {
    s2 := *s
    s2 = append(s2, 10)

    // Some more logic goes here, and at some point under a conditional you
    // capture `s2` into a goroutine and you put back the slice into the pool
    // before exiting, and this callback get a new slice from the pool an
    // assigned to `s`.
})

OK, not to complex, but then I though, instead of using a temporary variable, use the pointer directly with some utility
functions/wrappers, something like this

get := func() []byte {
    s := pool.Get().(*[]byte)
    return *s
}

put := func(s []byte) {
    pool.Put(&s)
}

Evil allocations

During the review of my changes, there was a comment to replace my utility functions/wrapper by others more ergonomic.

That made me think, all these wrappers going to save the allocations saved because of using pointers in the pool?

Myself response was: I don't know, let's write some dirty benchmarks abstracting a pool usage, not from the real
implementation, for simplicity and because I was only interested in finding out the pool behavior. Actually, the
previous snippets of codes were extracted from these benchmarks.

I started to write 3 different options, including using values in the pool instead of pointers, but when I saw that
using values was doing the same allocations than using pointers with wrappers, I wrote a few more, some of them with a
minimal changes that were expected not to provoke any allocation change because they were more about to reduce lines of
code while eliminating intermediary variables by moving the operations into the same line.

You can find it here if you're interested in
looking at it.

Running the benchmarks with go test -bench . -benchmem, I quickly saw which approach was fine because it didn't
produce any allocation.

Based on those results, it looked that it isn't a good idea to wrap anything in functions and just use directly the
pointers to make sure that there isn't unexpected allocations.

As a side note, the idea of having some wrappers was to avoid code repetition that was using intermediate variables to
dereference the pointer to the slice to add elements and reset it, until my colleague mentioned that I could simplified
with online just dereferencing when passed to append and when resetting, but also dereferencing when assigning the
result of those operations, so this ended up like

s = pool.Get().(*[]byte)
*s = (*s)[:0]
*s = append(*s, 10)

And with that simplification, there weren't needed wrappers anymore because there was nothing to encapsulate, unless
that seeing those many * bothers you.

The culprit was "moved to heap"

This could end up here, but I was curious about where those allocations where originated. I took 3 of the benchmarks and
check the output of go test -gcflags='-m' pool_test.go.

The output is quite verbose, but looking at it, I found that the important parts for this matter was the lines that
contained moved to heap.

Following are the 3 benchmarks that I put in a single file to be easy to inspect the moved to heap lines.

Benchmark with 1 allocation

package tmp

import (
    "sync"
    "testing"
)

func BenchmarkPoolGetPut(b *testing.B) {
    pool := sync.Pool{
        New: func() any {
            s := make([]byte, 0, 10000)
            return &s
        },
    }

    get := func() []byte {
        s := pool.Get().(*[]byte)
        return *s
    }

    put := func(s []byte) {
        pool.Put(&s)
    }

    s := get()
    if len(s) != 0 {
        b.Error("returned slice doesn't have the expected number of elements")
    }
    s = s[:0]
    put(s)

    for b.Loop() {
        s := get()
        s = append(s, 10)
        if len(s) != 1 {
            b.Error("returned slice doesn't have the expected number of elements")
        }
        s = s[:0]
        put(s)
    }
}

go test -gcflags='-m' bench_get_put_test.go 2>&1 | grep 'moved to heap'

./bench_get_put_test.go:11:4: moved to heap: s
./bench_get_put_test.go:9:2: moved to heap: pool
./bench_get_put_test.go:30:5: moved to heap: s
./bench_get_put_test.go:21:14: moved to heap: s

We can ignore line 9 and 11 for this one and for the rest, that's obvious that a new slice has to be allocated in the
heap when it's created.

The important ones are 21 and 30, which is where a passed slice to the put function makes the slice to be moved to the
heap.

Benchmark with 2 allocations

package tmp

import (
    "sync"
    "testing"
)

func BenchmarkDoubleAllocation(b *testing.B) {
    pool := sync.Pool{
        New: func() any {
            s := make([]byte, 0, 10000)
            return &s
        },
    }

    appendByte := func(buffer *[]byte, val byte) *[]byte {
        b := *buffer
        b = append(b, val)
        return &b
    }

    reset := func(buffer *[]byte) *[]byte {
        b := *buffer
        b = b[:0]
        return &b
    }

    s := pool.Get().(*[]byte)
    if len(*s) != 0 {
        b.Error("returned slice doesn't have the expected number of elements")
    }
    s = reset(s)
    pool.Put(s)

    for b.Loop() {
        s := pool.Get().(*[]byte)
        s = appendByte(s, 10)
        if len(*s) != 1 {
            b.Error("returned slice doesn't have the expected number of elements")
        }
        s = reset(s)
        pool.Put(s)
    }
}

go test -gcflags='-m' bench_double_allocation_test.go 2>&1 | grep 'moved to heap'

./bench_double_allocation_test.go:11:4: moved to heap: s
./bench_double_allocation_test.go:17:3: moved to heap: b
./bench_double_allocation_test.go:23:3: moved to heap: b
./bench_double_allocation_test.go:9:2: moved to heap: pool
./bench_double_allocation_test.go:32:11: moved to heap: b

The important ones are the 17 and 23. My understanding is that 32 is moved to the heap, but because it's a pointer it
doesn't allocate.

Benchmark with 0 allocations

package tmp

import (
    "sync"
    "testing"
)

func BenchmarkZero(b *testing.B) {
    pool := sync.Pool{
        New: func() any {
            s := make([]byte, 0, 10000)
            return &s
        },
    }

    appendByte := func(buffer *[]byte, val byte) *[]byte {
        b := *buffer
        b = append(b, val)
        *buffer = b
        return buffer
    }

    reset := func(buffer *[]byte) *[]byte {
        b := *buffer
        b = b[:0]
        *buffer = b
        return buffer
    }

    s := pool.Get().(*[]byte)
    if len(*s) != 0 {
        b.Error("returned slice doesn't have the expected number of elements")
    }
    s = reset(s)
    pool.Put(s)

    for b.Loop() {
        s := pool.Get().(*[]byte)
        s = appendByte(s, 10)
        if len(*s) != 1 {
            b.Error("returned slice doesn't have the expected number of elements")
        }
        s = reset(s)
        pool.Put(s)
    }
}

go test -gcflags='-m' bench_zero_allocation_test.go 2>&1 | grep 'moved to heap'

./bench_zero_allocation_test.go:11:4: moved to heap: s
./bench_zero_allocation_test.go:9:2: moved to heap: pool

There is nothing moved to the heap, hurray!

Obviously, simplifying the wrappers to online dereferencing the pointers as commented above

    appendByte := func(buffer *[]byte, val byte) *[]byte {
        *buffer = append(*buffer, val)
        return buffer
    }

    reset := func(buffer *[]byte) *[]byte {
        *buffer = (*buffer)[:0]
        return buffer
    }

produces the same result, but it defeats the purpose of them, because we can place the one line wrappers where they are
called.

Conclusion

When dealing with sync.Pool use the same pointers that you get to put them into to avoid "moved to heap" and cause
undesired allocations.

If a function creates a variable and it returns a reference to it, it's moved to the heap. Although, I could expect
this because of Rust learned lessons, I missed the point in Go.

¹ https://review.dev.storj.tools/c/storj/storj/+/18859
These are the changes that I worked and were I had to use sync.Pool if it's your interest

Arch Linux essential tweaks and applications

Ivan Fraixedes — Fri, 27 Jun 2025 16:48:44 +0000

This post is part of a series of posts about installing and configuring Arch Linux in a Slimbook Executive 14". I created them from a collection of personal notes, that I thought may be useful for others, so I published them through these series.

Base Arch Linux installation
Arch Linux network configuration
Tracking dot files
Arch Linux essential tweaks & applications

This post and the next are related to personalize my recently installed Arch Linux. Although in the previous one, I already made some choices, they are not as personalized as the ones to come.

Tweaking sudoers

I wanted my user to be able to power off, reboot, etc, using sudo but without having to type a password.

To grant my user to do so, I created a new sudoers file /etc/sudoers.d/10-common-operations-no-passwd with the following content:

ivan blacksmoke= NOPASSWD: /ur/bin/systemctl halt,/ur/bin/systemctl hibernate,/ur/bin/systemctl poweroff,/ur/bin/systemctl reboot,/ur/bin/systemctl suspend

And I ensured that only root can access to it

# chown root:root /etc/sudoers.d/10-common-operations-no-passwd
# chmod -c 0660 /etc/sudoers.d/10-common-operations-no-passwd

Sync the clock

I wanted to sync the computer clock, so I enable the systemd-timesyncd daemon with

# timedatectl set-ntp true

My default shell

I wanted to use the fish shell, so I installed and set it as a default shell for my user

# pacman -S fish
# chsh -s /usr/bin/fish

Changing my default editor

If you have read the }}">base Arch Linux installation post, I installed NeoVim, but I never set it as my default editor.

As a convention applications that requires to open an editor should read the EDITOR environment variable.

The /etc/environment file is a file that contains the environment variables thar are exported for all the sessions are created.

I created the file because it didn't exist and I added

EDITOR=/usr/bin/nvim

Comfortable using the terminal without a desktop

When you don't have a desktop environment (at least at this point, I didn't have any), tmux is one of those tools that will be your friend, why? Because between many things, the two most important features for me is that I can have tabs and multiple panes (vertical and horizontal), so I convert a single terminal session in a full fledge environment where I can have several applications running at the same time and move from to another as I do in a desktop environment.

Nowadays, there are similar terminal emulator that may be for some people a better choice, however, this isn't a debate if Tmux is the best, the worse, or the average. I use tmux because I used for long time, remotely and locally, and it works good for me, so I haven't invested my time on learning another one.

So, yes, tmux is what I wanted

# pacman -S tmux

A few essential tools

The last missing tools at this point were

man because I need to read documentation
less because I need to paginate outputs
extfat-utils because I need to mount external ExtFat disks
bat because syntax coloring to cat certain files is helpful in certain situations
vimfm because is used for certain applications to show file differences

so they were in with

# pacman -S man-db extfat-utils less bat vifm

Welcome Arch User Repository

Some good tools aren't in the official Arch linux official package repository, but they are in AUR (Arch User Repository), the package repository manage by the community.

You cannot install AUR packages with pacman, buy you can do it manually which isn't that bad, but it's a bit tedious due to the process boilerplate and you have also to repeat it manually to update each installation that you installed from AUR. Fortunately, there are tools that simplify the process and make our busy life easier.

There are many tools to install AUR packages. I chose to install Paru, however, I don't have a strong argument for it, which is I liked what I read and it's implemented in Rust.

Let's install it.

Paru

First we have to make sure that we have installed and up to date the basic tools to build Arch Linux Pacakges (a.k.a. base-devel)

# sudo pacman -S --needed base-devel

Paru is in the AUR packages, so we have to install it manually. After it, we shouldn't recur very often or even never to manually install and update AUR packages, because that's what it will do.

# cd .local/makepgk/sources
# git clone https://aur.archlinux.org/paru.git
# cd paru
# makepkg -sirc

makepkg will ask for installing Rust via the rust or rustup, I chose rust, rustup didn't work because it couldn't select which Rust version to use.

Anyway, the dependency will be deleted after the installation, that's what the -r flag does in the makepkg.

After the installation, I also deleted .cargo and .rustup directories that were created in my home directory.

Finally, I wanted to apply its general tips at the time that I installed.

To apply them and other recommended options:

I enabled the pacman's Color uncommenting it from /etc/pacman.conf.
In /etc/paru.conf, I enabled the file manager viewer to use vifm uncommenting FileManager option and making sure that it's set to vifm (i.e. FileManager = vifm). Remember to uncomment the [bin] section if it's commented because this options belongs to that section.
In /etc/paru.conf, I enabled the search results to start at the bottom and go upwards, uncommenting the option BottomUp. Remember to uncomment the [options] section if it's commented because this options belongs to that section.

And that was all for the day.

Tracking dot files

Ivan Fraixedes — Sat, 21 Jun 2025 14:41:58 +0000

Base Arch Linux installation
Arch Linux network configuration
Tracking dot files
Arch Linux essential tweaks & applications

This post isn't specifically related to Arch Linux, however, I dit it before I started to add and modify more configuration files on my home directory (a.k.a dotfiles) , I wanted to start tracking them in a Git repository.

Through the years, I've become more disciplined in how to manage my personal computers, one of those things is to track my configurations files (a.k.a dotfiles). Nowadays, when I set up a new one, I start to track them in a Git repository.

One purpose is restoring them or take some of them for other installations when they are different ones (not the same OS, or the same one with other applications, etc.), but, the other purposes are:

To see when new configuration files appear, disappear (i.e. delete) or are modified without being aware.
Track the changes with useful commit messages, so I can use them to remember how to configure or change certain files to achieve something.

NOTE I don't share this Git repository with anybody. Although it shouldn't contain any secret, I don't want to publicly share it, in case that one slides on it.

Bear in mind that the commands in this post

They are all executed from my user home directory.
They are executed on Arch Linux or a distribution based on it, although the most of them should work in other common Linux distributions.

The starting point is obvious :P

# git init -b main .

GPG

Before doing any other thing, I want to set up my GPG keys because I want all my commits to be signed with my keybase signature.

I ensure that that gnupg is installed and up to date

# sudo pacman -S --needed gnupg

I have exported my keybase private key, with the uid that reference to ifraixedes@keybase.io removed, and with uid referencing to my email account under my personal domain, and that's the one that I importing below.

Let's import the key, but before, I list them to see that I don't have any, then import it, and I list them again to see that it was imported as expected, obviously for importing you only need to execute the command in the middle

# gpg --list-keys
# gpg --import /mnt/usbstick/personal-gpg.priv.key
# gpg --list-keys

Git

I wanted to have some Git global configuration to start using it with this repository. The content of my initial .gitconfig file is

[alias]
  amend = commit --amend --no-edit
  brdel = "!f() { git push $1 :$2; }; f"
  graph = log --color --graph --pretty=format:\"%h | %ad | %an | %s%d\" --date=short
  hist = log --color --pretty=format:\"%C(yellow)%h%C(reset) %s%C(bold red)%d%C(reset) %C(green)%ad%C(reset) %C(blue)[%an]%C(reset)\" --relative-date --decorate
  lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative
  trdel = "!f() { git push $1 :refs/tags/$2; }; f"
  wdiff = diff --color-words
  wip = for-each-ref --sort='authordate:iso8601' --format=' %(color:green)%(authordate:relative)%09%(color:white)%(refname:short)' refs/heads

[commit]
  gpgSign = true

[core]
  whitepace = blank-at-eol,space-before-tab,blank-at-eol

[init]
  defaultBranch = main

[merge]
  conflictstyle = diff3

[pull]
  ff = only

[push]
  default = current
  followTags = true

[user]
  email = <redacted>
  name = Ivan Fraixedes
  signingkey = B5CB5AD5

However the only settings that matters for the signature are

[commit]
  gpgSign = true

[user]
  email = <redacted>
  name = Ivan Fraixedes
  signingkey = B5CB5AD5

I also create a .gitignore file to start ignoring what I don't want to track. I could ignore some entire directories, for example .local, however, I don't do it because I want to know when new directories appear there, despite of the extra work that brings in having to update .gitignore from time to time.

Last, but not least, I have to define the GPG_TTY environment variable, otherwise git commit fails with

error: gpg failed to sign the data:
[GNUPG:] KEY_CONSIDERED 2CFC97FB943CDD6F5DD1B8ECFB6101AFB5CB5AD5 2
[GNUPG:] BEGIN_SIGNING H8
[GNUPG:] PINENTRY_LAUNCHED 1117 curses 1.2.1 - linux - - 1000/1000 0
gpg: signing failed: Inappropriate ioctl for device
[GNUPG:] FAILURE sign 83918950
gpg: signing failed: Inappropriate ioctl for device

fatal: failed to write commit object

Why? well gpg-agent is a daemon which starts automatically to mange secret (a.k.a private) keys. You can read in its documentation it requires GPG_TTY environment variable.

Because I use fishell, I declare it into a new file under .config/fish/conf.d/env-vars.fish

set -gx GPG_TTY (tty)

and apply it to my session with source ./config/fish/conf.d/env-vars.fish.

At this point I created my first commits with the home files that I want to track and I move on from this point creating separated commits for new files, files deletions, and file modifications.

Arch Linux network configuration

Ivan Fraixedes — Wed, 30 Oct 2024 18:28:46 +0000

Base Arch Linux installation
Arch Linux network configuration
Tracking dot files
Arch Linux essential tweaks & applications

NOTE because all the operations require root permissions, I logged as root with sudo su, so all the commands in this post don't execute with sudo.

I decided to use systemd-networkd to deal with the network configuration, and I decided to fully commit to it, so I used systemd-resolved for configuration the network name resolution (a.k.a DNS). Bear in mind that my laptop only has one network interface (apart of the loopback interface).

Both services are part of the base Arch Linux installation.

I enable the systemd-networkd, so it automatically starts at boot (i.e. systemctl enable systemd-networkd.service). After executing it, you'll see a list of created symlinks in /etc/systemd/system and sub-directories to /usr/lib/systemd/system directory, not important, but I wanted to point it out.

I configured the systemd-networkd-wait-online.service which is a oneshot system service that waits for the network to be configured. I edited the file /etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service which is a symlink to /usr/lib/systemd/system/systemd-networkd-network-wait-online.service, adding the --any flag to the following line

ExecStart=/usr/lib/systemd/systemd-networkd-wait-online --any

To configure the wireless adapter, I created a new file, /etc/systemd/network/25-wireless.network, with the following content

[Match]
Name=wlan0

[Network]
DHCP=yes
IgnoreCarrierLoss=3

To obtain the name of the wireless adapter, I executed ip link.

Everything should be fine at this point, so I started the service systemd start systemd-netword.service.

Next is to configure name resolution service (TODO review if name resolution service is correct) through the systemd-resolved service.

I started enabling systemctl enable systemd-resolved.service which, as before it created some symlinks . Right after, I started it systemctl start systemd-resolved.service.

I replaced the /etc/resolv.conf by a symlink ln -sf ../run/systemd/resolve/stub-resolve.conf /etc/resolv.conf, the service had to be active, otherwise the runtime configuration (the file /run/systemd/resolve/stub-resolve.conf) didn't exist and the symlink creating would have failed.

I checked the DNS currently in use with resolvectl status and it was great, it gave me the DNS servers configured on my router.

Last, I configured the Wi-FI through the iwd, which I already installed in the base installation

First, I created the iwd network configuration file /var/lib/iwd/foo.psk, where foo is the SSID of my home Wi-FI network.

I opened the file with the editor and wrote

[Security]
Passphrase=I typed here for you to know it

Then, I started the service (systemctl start iwd.service) and confirmed that I could ping some external sites.

After this, I edited the above file to remove the passphrase because the pre-shared key was written it, so it doesn't need to be there anymore.

Because, I didn't want iwd to scan periodically for networks when it's in a disconnected state, I created the /etc/iwd/main.conf file, opened with the editor, and wrote:

[Scan]
DisablePeriodicScan=true

Last but not least, I want the service to automatically start at boot, hence I enabled the service (systemctl enable iwd.service), which created some symlinks as before.

This is all, I rebooted my computer and I could verify that it connects automatically to my home Wi-Fi network.

Base Arch Linux installation

Ivan Fraixedes — Thu, 28 Mar 2024 09:45:23 +0000

Base Arch Linux installation
Arch Linux network configuration
Tracking dot files
Arch Linux essential tweaks & applications

I installed Arch Linux in my new laptop, an Slimbook Executive 14".
I followed the official installation guide, however, I had to dig through different pages to install it how I wanted.

My requirements were:

Encrypt the full disk with dm-crypt.
Use LVM to manage disk partitions.
Use ext4 file system.

Creating disk partitions

I've mostly followed dm-crypt/Encrypting an entire system, but I summarize here my process with the options that I chose.

The laptop has 1 TB of NVMe (Non-Volatile Memory Express) and I deliberately want to encrypt the full disk.

The dm-crypt scenario that I've chosen is LVM on LUKS with encrypted boot partition over:

The cool Secure Boot and the Trusted Platform Module option because, despite, I have to annoyingly type a password every time that I boot the machine, it will be protected from boot cool boot attacks
The LVM on LUKS because I want to have everything encrypted, included the boot partition (which contains the initramfs and the kernel).

I used the ancient fdisk to create in the only disk that the laptop has, which is mapped to the device /dev/nvme0n1, all the partitions indicated in the "preparing the disk" section of LVM on LUKS with encrypted boot partition. The partitions ended up as:

Disk /dev/nvme0n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: Samsung SSD 980 PRO 1TB
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: <redacted>

Device           Start        End    Sectors   Size Type
/dev/nvme0n1p1    2048       4095       2048     1M BIOS boot
/dev/nvme0n1p2    4096    2101247    2097152     1G EFI System
/dev/nvme0n1p3 2101248 1953523711 1951422464 930.5G Linux filesystem

Let's encrypt the second partition

# cyptsetup luksFormat --type luks1 /dev/nvme0n1p3

Now let's mount it on /dev/mapper/cyrptlvm

# cryptsetup open /dev/nvme0n1p3 cryptlvm

Now we start with the LVM ceremony

First of all, I create a physical volume because I only have one hard disk

# pvcreate /dev/mapper/cryptlvm

Secondly, one logical groups because I don't need an extra degree of isolation between volumes and it's more storage efficient than having multiples groups

# vgcreate main /dev/mapper/cryptlvm

And last, but not least, three logical volumes, one for swap, one for the root partition, and one for the home directory

# lvcreate -L 64G main -n swap
# lvcreate -L 430G main -n root
# lvcreate -l 100%FREE main -n home

You may wonder why I'm booking 64 Gb for memory swap. It isn't because I'm scared of running short of RAM, this laptop has exactly 64 Gb. In the past I ran into hibernating issues because the swap partition was smaller than the size of the RAM, although nowadays, the memory image for the hibernation can be store in a file and not only in a swap partition I preferred to keep it simple and use the swap partition for hibernating because I can afford to lose 64 Gb of disk from the other two partitions and, anyway, if I need to pull up that space, I could do it resizing the LVM volumes.

Time to format the partitions, for the swap partition is a snap. For the other two, I decided to be conservative and use ext4, despite of the cool features that btrfs, I went for ensuring maturity and stability over better data integrity and error recovering and I don't need btrfs RAID, volume management, snapshots, and rollbacks because I already have LVM for them.

So the boring stuff for that is

# mkfs.ext4 /dev/main/root
# mkfs.ext4 /dev/main/home
# mkswap /dev/main/swap
# mount /dev/main/root /mnt
# mount --mkdir /dev/main/home /mnt/home
# swapon /dev/main/swap

I chose to boot in UEFI mode, so mounted the partition in /efi for compatibility with grup-install

# mkfs.fat -F32 -n EFI /dev/nvme0n1p2
# mount --mkdir /dev/nvme0n1p2 /mnt/efi

Preparation before installing

First of all, we need an internet connection. The Arch Linux installation image comes with iwd, and after I ensured that the Wi-FI network card was detected with ip link, I followed the iwctl commands in the connect to a network section.

I ensured that the system clock is accurate with timedateclt.

Installing essential packages

# pacstrap -K /mnt base linux linux-firmware

Essential configurations

I generate the fstab file from the current mounted volumes

# genfstab -U /mnt >> /mnt/etc/fstab

Then I changed the root directory to the root mounted volume, where we are installing Arch Linux.

# arch-chroot /mnt

I installed Neovim, which is the editor that I usually use, and I need one to be able to modify and create some configuration files

# pacman -S neovim

Set the timezone

# ln -sf /usr/share/zoneinfo/Europe/Madrid /etc/localtime
# hwclock --systohc

I uncommented the ca_ES.UTF-8 UTF-8, es_ES.UTF-8 UTF-8, and en_US.UTF-8 UTF-8 from /etc/locale.gen and generated the locales with

# locale-gen

I created the file /etc/locale.conf and write inside LANG=en_US.UTF-8 to set the LANG variable accordingly.

I configured the hostname writing blacksmoke in /etc/hostname, yes the laptop is called blacksmoke.

I installed the iwd network manager

# pacman -S iwd

I've decided to use a systemd-based initramfs, so I changed the mkinitcpio.conf (/etc/mkinitcpio.conf) from

HOOKS=(base udev autodetect microcode modconf kms keyboard keymap consolefont block filesystems fsck)

HOOKS=(base systemd autodetect microcode modconf kms keyboard block sd-encrypt lvm2 filesystems fsck)

Note I swapped udev by systemd, added sd-encrypt and lvm2 and removed keymap and consolefont because I'm fine with the US console keymap and the default console font.

and I installed LVM because it's obviously needed due to my partitions are on top of LVM volumes, besides that mkinitcpio needs is as you can see above that I added the lvm2 hook.

# pacman -S lvm2

Right after, I regenerated the initramfs running

# mkinitcpio -P

But I saw through the warnings printed in the console that Im missing a few firmwares. As commented in Mkinitcpio wiki page about "possible missing firmware for module XXXX, I had to install the Linux firmwares package

# pacman -S linux-firmware linux-firmware-qlogic

However, it wasn't enough, and I had to install other firmwares from AUR packages, so let's go for them.

As you may know, installing package from AUR repositories is totally different story than doing from main stream repositories, they are several tools that simplify the process by automating several manual steps, however, and I like them, but I want to decide which one to use, once I have the OS working, not at the installation time, so I installed them manually.

First, I installed the required packages to build AUR packages

# pacman -S base-devel sudo git

I authorized all the users in the wheel group creating the following file /etc/sudoers.d/00-wheel-users

%wheel ALL=(ALL:ALL) ALL

And I ensured that only root can access to it

  chmod -c 0660 /etc/sudoers.d/00-wheel-users

makepkg requires sudo, so it was the time to create my regular user and set it's password

# useradd -c "Ivan Fraixedes" -m -U -G wheel ivan
# passwd ivan

I adjusted /etc/makepkg.conf following the tips & trics, creating /etc/makepkg.conf.d/makepkg.conf with the following content

MAKEFLAGS="-j$(nproc)"

PKGDEST=~/.local/makepkg/packages
SRCDEST=~/.local/makepkg/sources
SRCPKGDEST=~/.local/makepkg/srcpackages
LOGDEST=~/.local/makepkg/logs

I logged with my user and created the directories that I configured for makepkg

# su ivan
# cd ~
# mkdir -p .local/makepkg
# cd .local/makepkg
# mkdir logs packages sources srcpackages

I installed the following firmware from AUR doing (as ivan user)

# mkdir -p tmp/aur
# cd tmp/aur
# git clone https://aur.archlinux.org/upd72020x-fw.git
# cd upd72020x-fw
# makepkg -sirc

The same package installation call mkinitcpio and I saw that there isn't a warning anymore for the xhci-pci firmware, yup!
I repeated the same process for the following AUR firmwares ast, aic94xxx, and wd719x. Cha-ching no more warning when building the initramfs with mkinitcpio.

I installed the GRUB boot loader and the efibootmgr which is required by GRUB for UEFI systems

# pacman -S grub efibootmgr

And configure it (/etc/default/grub) to allow booting from /boot on the encrypted partition and set the kernel parameters to allow initramfs to unlock the encrypted root partition using the sd-encrypt mkinitcpio's hook.
Note these lines may not be together, some may be commented, and some may have other values that you may need to keep, in any case they will ended up with these values in a different machine (e.g. the disk UUID won't be the same), hence, check the configuring GRUB section.

GRUB_ENABLE_CRYPTODISK=y
GRUB_CMDLINE_LINUX="rd.luks.name=a199a1d7-acc0-4910-babb-e0059dec293b=cryptlvm"

What disk UUID should you add in the above GRUB's configuration? I think that's quite obvious, but I didn't think that much at that moment and I set to the UUID of the wrong one and, for obvious reasons, GRUB wasn't able to decrypt the volumen and boot the OS :(. It took me several hours to find out my mistake and with a lot of rebooting to try to see if I was fixing the issues for every change that I was making in mkinitcpio.conf, grub, etc., so pay attention and don't waste marvelous hours with this stupid mistake.

It has to be the UUID of the LUKS partition where the /boot is placed, NOT the LVM one! In my case is /dev/nvme0n1p3 and to find out UUID, you can do ls -l /dev/disk/by-uuid and the link name that points to it is the UUID.

I installed GRUB to be mounted ESP for UEFI booting and generated the GRUB's configuration file

# grub-install --target=x86_64-efi --efi-directory=/efi --bootloader-id=GRUB --recheck
# grub-mkconfig -o /boot/grub/grub.cfg

OK, at this point, I rebooted the computer and yes, after booting from its disk, it asked me for the LUKS password, then GRUB menu showed up, and, after selecting boot Arch, it asked me again for LUKS password. Brilliant, everything finally worked.

Last, but not least and before finishing this blog post, I want to include here 2 things as an initial setup.

I did a couple of more things which I consider that they are part of the base installation:

Set a password for the root user, otherwise you cannot login with it. Yes, I want a password for it, I found sometimes useful in the past to be able to login as root rather than my regular user.
Add a keyfile to the LUKS partition, so it can be used to decrypted automatically by GRUB and not having to type the LUKS passphrase twice.

After booting up, I login with my regular user ivan and then, I access as root and change its password.

# sudo su
# passwd

Using the root login, I set up the keyfile for the LUKS volume following the "With a keyfile embedded in the initramfs" section of "dm-crypt/Device encryption" page.

First, I created the keyfile

# dd bs=512 count=4 if=/dev/random of=/root/cryptlvm.keyfile iflag=fullblock
# chmod 000 /root/cryptlvm.keyfile
# cryptsetup -v luksAddKey /dev/nvme0n1p3 /root/cryptlvm.keyfile

I added the keyfile to the initramfs image, that's just updating the /etc/mkinicpio.conf, adding the file path to the FILES list, in my case, FILES was empty, so it changed from

FILES=()

FILES=(/root/cryptlvm.keyfile)

Recreate the initramfs

# mkinitcpio -P

Secure the access to the embedded keyfile

# chmod 600 /boot/initramfs-linux*

Update GRUB's configuration (i.e. /etc/default/grub) to point to the keyfile. Remember, that in my case, I'm using systemd-based initramfs (i.e. sd-encrypt hook). I my case, I added rd.luks.key=a199a1d7-acc0-4910-babb-e0059dec293b=/root/cryptlvm.key to the GRUB_CMDLINE_LINUX, so it changed from

GRUB_CMDLINE_LINUX="rd.luks.name=a199a1d7-acc0-4910-babb-e0059dec293b=cryptlvm"

GRUB_CMDLINE_LINUX="rd.luks.name=a199a1d7-acc0-4910-babb-e0059dec293b=cryptlvm rd.luks.key=a199a1d7-acc0-4910-babb-e0059dec293b=/root/cryptlvm.key"

And regenerate the GRUB's configuration

# grub-mkconfig -o /boot/grub/grub.cfg

That's it for today, in future posts, I will cover what tools I'm installing and certain configurations that I tweaked to make my machine more comfortably usable.