Forem: Lucas Eduardo

How I messed up my systemd logs in NixOS and how I fixed it.

Lucas Eduardo — Mon, 07 Aug 2023 16:21:00 +0000

NixOS is an amazing distro, like, imagine configuring all your system stuff, containers, services, programs, development environments and so on using a consistent language. This is not the future but actually the present.

Even though NixOS modules can do validation of inputs and scenarios such as not allowing you to have both pipewire and pulseaudio in the same system, because it's known that it will conflict with each other, there are stuff that modules can't predict, and at the end of the day modules are made by humans and humans do make mistakes even though there is a lot of code review happening on nixpkgs.

This case is very unlikely to be treated by a module because it's more like a runtime issue.

Systemd-journald grants access to users to view their service logs using ACLs, that are basically an extension to what chmod does. Root and the systemd-logs have write access to the logs and the final user only has the privilege to read them using a special annotation. The problem is that my zfs filesystem wasn't allowing this annotation to be saved because zfs doesn't support saving these ACLs by default. For it to save you have to set the acltype attribute to posixacl using zfs set. I did know about this when I built the partition scheme for this system but I missed a small little thing: It was mounting that dataset on the wrong location: /var/journal instead of /var/log/journal.

To fix this, first I stopped the journal using systemctl stop systemd-journald so it would not try to write to that location. Second, I moved /var/log/journal to /var/log/journal.old then created an empty folder on that location so I can remount that dataset that was mounted on the wrong place to /var/log/journal. After that I copied using rsync from /var/log/journal.old to /var/log/journal, fixed the ACLs using sudo setfacl u:lucasew:r 97e3b5a7928145bdb0d45bc645b5c20e/user-1000* (you will have to change it to your own user and user id) and then tested running journalctl --user -r. Stuff then appeared nicely so it's fixed. Finally I ran systemctl start systemd-journald to start the journal service again and changed in my hardware-configuration.nix so the next time it will mount that dataset correctly.

How I did a turbo mode system in GCP for cheap

Lucas Eduardo — Wed, 06 Jul 2022 16:07:24 +0000

Cloud instances are very useful when you need compute power but don't want to buy machines and deal with the maintenance burden of maintaining physical machines, or just don't need them long enough to make it worth buying hardware. The idea is basically rent a remote machine for the time you need them and pay exactly for what you used.

This can be a pretty simple concept but can be very hard to implement. There are literally hundreds, or even thousands, of combinations and use cases that cloud providers support. Our focus on this article is the most basic use case ever: you rent a VM and pay for how much time and space it uses.

This article is focused on the Google Cloud Platform quirks, and my specific project is a NixOS VPS to use GPUs in the cloud for cheap.

The strategy

Now (6/7/2022) there are some resources that are free forever, like:

30GB of total balanced storage
A e2-micro (0.5VCPU burstable to 2VCPU) instance for the whole month
1GB of egress network in almost all continents
Free egress network for Google services

The idea is to use Terraform to migrate one instance to TURBO MODE to do the beefy stuff, and use the same system in an e2-micro for configuration, preparation, uploads, maintenance and so on.

The turbo instance can be a spot instance for extra cheap. Spot instances are cheaper because these can be interrupted at any time.

My OS of choice is NixOS because of the code based construction and, in my opinion, is more convenient to setup and replicate. I can replicate the same setup as many times as I want doing specific customizations for one specific deployment if I want.

The code

The code I used in this experiment is stored in my dotfiles repo.

The Terraform file I am using for this experiment is stored in infra/gcp.tf. The Terraform state is stored in Terraform cloud.

The NixOS machine that I run in GCP is defined in nodes/vps.

There are some CUDA specific stuff in the file nvidia.nix. Tesla K80 doesn't support the latest driver so I had to use legacy 470. Tested with some Blender renders and it's working fine with CUDA and works correctly for what I need.

How to deploy

To deploy the instance in normal mode I run the following command in the infra folder:

terraform apply

And if I want to migrate it to the turbo instance I run the following command in the infra folder:

terraform apply -var 'modo_turbo=true'

Final thoughs

It was a pretty challenging project, and I though I would spend much more money in this. I did this tiering stuff in two days. I took a lot of time to figure out how to configure CUDA then I realized that hardware.opengl.enable must be enabled to allow software to find the CUDA driver 🤡.

Future work

I need to find a way to automatically turn off the turboed machine when the job is over, automatically. I might do some wizardry with systemd units. I will not probably update this post. The code that will be changed for that project is in the VPS folder or some common module that I could use in any of my machines.

OpenCL on Blender with Intel Integrated GPUs

Lucas Eduardo — Fri, 01 Apr 2022 19:13:27 +0000

So far only AMD's OpenCL is officially supported but if you like to take some risks you can force enable OpenCL on Intel setting two environment variables:

CYCLES_OPENCL_SPLIT_KERNEL_TEST=1: That's the one referenced in any post about it
CYCLES_OPENCL_TEST=all: that variable is not so mentioned

A detail that you might forget is that you need to install the OpenCL drivers for your GPU before doing this. My 7th gen i5 still uses the legacy driver. The still maintained driver is for 8th gen and above. The driver link is broken but Wayback Machine covers this case fortunately.

Here is a full command for quick testing:

CYCLES_OPENCL_TEST=all CYCLES_OPENCL_SPLIT_KERNEL_TEST=1 blender --python-expr "import bpy; [[print(d.type, d) for d in ds] for ds in bpy.context.preferences.addons['cycles'].preferences.get_devices()]" --verbose 69 -E CYCLES -b

And that's the output that appears on my machine:

Blender 2.93.5
Read prefs: /home/lucasew/.config/blender/2.93/config/userpref.blend
[ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)
[ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)
I0401 16:07:26.994379 438579 blender_python.cpp:195] Debug flags initialized to:
CPU flags:
  AVX2       : True
  AVX        : True
  SSE4.1     : True
  SSE3       : True
  SSE2       : True
  BVH layout : EMBREE
  Split      : False
CUDA flags:
  Adaptive Compile : False
OptiX flags:
  CUDA streams : 1
OpenCL flags:
  Device type    : ALL
  Debug          : False
  Memory limit   : 0
I0401 16:07:28.091580 438579 device_cuda.cpp:56] CUEW initialization failed: Error opening the library
I0401 16:07:28.091853 438579 device_opencl.cpp:48] CLEW initialization succeeded.
I0401 16:07:28.093667 438579 opencl_util.cpp:957] Enumerating devices for platform Intel(R) OpenCL.
I0401 16:07:28.093719 438579 opencl_util.cpp:777] OpenCL driver version 1.2
I0401 16:07:28.093741 438579 opencl_util.cpp:993] Using more readable device name: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (OpenCL)
I0401 16:07:28.093761 438579 opencl_util.cpp:995] Adding new device Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (OpenCL).
CPU <bpy_struct, CyclesDeviceSettings("Intel Core i5-7200U CPU @ 2.50GHz") at 0x7f812be0b608>
OPENCL <bpy_struct, CyclesDeviceSettings("Intel Core i5-7200U CPU @ 2.50GHz (OpenCL)") at 0x7f812be0b688>
CPU <bpy_struct, CyclesDeviceSettings("Intel Core i5-7200U CPU @ 2.50GHz") at 0x7f812be0b608>

Blender quit

With this a OpenCL GPU will appear on preferences.

But beware, in my case when I started rendering the CPU only got 100% but without slowdown the computer and Blender just stop reacting to any input so working is a too strong word.

Tested with Blender 2.93. Both nixpkgs and blender-bin 2.93 definitions works basically the same way.

So videos don't have Ctrl+F...

Lucas Eduardo — Sun, 13 Sep 2020 12:56:05 +0000

Youtube-dl is an awesome download tool for videos and audios in a huge amount of sites but there are some features that people don't usually use. One of them is to download video subtitles that sometimes is available.

Youtube-dl is a CLI program, it can be a bit difficult for begginners but it will be worth learning. I promise.

People usually just use youtube-dl with a link, maybe referencing a list of links using the -a flag.

youtube-dl $link
youtube-dl -a $file

If you run youtube-dl --help there will be a lot of options and I will show some of them related to subtitles;

--list-subs: Shows the list of embedded subtitles that the video have. Usually not that useful, at least not in this case xD;
--write-subs: Downloads the subtitles that are embedded in the video;
--sub-lang: Specifies a language to the downloaded subtitles, the default is en. In brazil the language is pt;
--sub-format: Specifies a file format of the downloaded subtitles. The default is VTT and is the fallback if the specified formats aren't available to download;
--write-auto-sub: This is the flag I was talking about xD. This flag downloads the subtitle automatically generated in youtube, the video transcript. lang and format flags are hints about what subtitle will be downloaded;

After downloading the subtitle, the program will also download the video, if you just want the subtitle you can use the flag --skip-download.

Batching works too, so if you want to download a full playlist or a file with all the URLs using -a you can and it will work.

When you download a subtitle you download not a TXT file full of words but a VTT file, that if you want just the text you can delete the unwanted parts. In every text line there is a fixed lines of other data that you can delete using someting like a vim macro, sed or other text manipulation tool you like.