Forem: Talha Altınel

Packaging Go for Arch Linux Tutorial

Talha Altınel — Sun, 17 Sep 2023 08:00:00 +0000

Getting Started

In this tutorial, I will be showing how to package Go application for Arch Linux User Repository (AUR). We will be opening an AUR account and go through PKGBUILD template and follow Arch's Wiki guidelines for Go. By the end of the tutorial, you will be able to upload your own Arch package that uses Go to AUR.

The Requirements

Git
Go
Arch Linux x86_64
AUR account

Setting up AUR account and SSH key

We will fill up the username and the email in this form, as well as the most important one public ssh key. The rest are optional.

Generate and fill in the SSH public key..

$ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

$ cat ~/.ssh/aur_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDiniLTrxNbDH/R66BYUHieRT9sTqkn2678picCjF8MoTxsZume105hsDFSfg79pzfedY3iJQXMsCzk11pcnUNsGHxT/wh9s8aFrlI+n9JVMpEe7VOZqTLYyNBXtpAJUaY2Ptp4/l2p81dhpeCGTMhYNu2eDxaCaI5QvDOkyEmAZYAmuLT19OwJv8YW/1+1tg+Piaxyg/b+Dic7EeQQT10AI9drfRQG5pazREYkLGjClJP6pw/OnNcScMWR/Sd4phiz84DKnBLWXIIdbK+CDKDyFPt1FMIXkY1YSY+RAyXgJ3m1z6byCRs5BrN4RZArPcIEmVRRffkhq7tVBK0mwygTl8Hku60MqvdENLrylPcH3Ua2iqYLhftuMNfsZffUb9d5MI0BCaoQuzMfEMG0ZZZuDoZ38HZDjZbFFG0Fg+rt6IRTdRogZ0bzWacM0ig8J+HDnJnNIhXut5RC/f4W1RIXITujNp0blQRISrh9lGXcH/qz002ovcAoAd2yRkRdhh3NlP9mAZ/Rns47FwKP94ooG2/Zb2JRNJLJgdgaEWT+u1v5G4tPAoySxwZ0HBZSxSEmZC34piiPxdaHd6NAy3drt3Nt7QWVdBU9pD17lj8PsBuzXVReBkM+/0MFMLYDThunwVVhpZSHtmDTzWoGijIJnzaZrJMPcZZab/vF1WJ/yQ== talhaaltinel@hotmail.com

After you copy pasted your public ssh key into SSH public key box, change your ~/.gitconfig

[url "ssh://aur@aur.archlinux.org/"]
        insteadOf = https://aur.archlinux.org/
        insteadOf = http://aur.archlinux.org/

Finally run the final command and input the answer of the form to confirm.

Understanding PKGBUILD

If you inspect /usr/share/pacman/PKGBUILD.proto, you will see the fields you can fill.

# This is an example PKGBUILD file. Use this as a start to creating your own,
# and remove these comments. For more information, see 'man PKGBUILD'.
# NOTE: Please fill out the license field for your package! If it is unknown,
# then please put 'unknown'.

# Maintainer: Your Name <youremail@domain.com>
pkgname=NAME
pkgver=VERSION
pkgrel=1
epoch=
pkgdesc=""
arch=()
url=""
license=('GPL')
groups=()
depends=()
makedepends=()
checkdepends=()
optdepends=()
provides=()
conflicts=()
replaces=()
backup=()
options=()
install=
changelog=
source=("$pkgname-$pkgver.tar.gz"
        "$pkgname-$pkgver.patch")
noextract=()
md5sums=()
validpgpkeys=()

prepare() {
    cd "$pkgname-$pkgver"
    patch -p1 -i "$srcdir/$pkgname-$pkgver.patch"
}

build() {
    cd "$pkgname-$pkgver"
    ./configure --prefix=/usr
    make
}

check() {
    cd "$pkgname-$pkgver"
    make -k check
}

package() {
    cd "$pkgname-$pkgver"
    make DESTDIR="$pkgdir/" install
}

In summary, you don't have to fill all these fields but will be good to remember what they are.

pkgname and pkgversion are the name of the software package and the version of the software you are providing.
pkgrel and epoch are additional way of subversioning the pkgversion, most of the time, you won't use
pkgdesc is the description for your software, arch is architecture and most of the time, it is just x86_64
url and license are the url of the software repository and the actual license name, these are quite important.
groups provide us a way to install multiple software packages, think it like a gnome(very big software group)
depends is runtime dependencies
makedepends is compile-time dependencies
checkdepends is check/test dependencies
optdepends is optional runtime dependencies
provides is used as alternative replacement for another package
conflicts is used as to not install the conflicted package
replaces is used as what this built package can replace
backup lists files that should be backed up when upgrading the package
options can include things like compiler flags, compression settings, etc.
install can be used to specify custom installation scripts or commands to run during the installation process
changelog does specify the location of a changelog or relase notes file
source lists the source files required to build the package. It includes the URLs or file paths to the source code or other necessary files
noextract lets you specify files that shouldn't be extracted during the build process
md5sums, b2sums, sha512sums, sha256sums are checksums, they can also be skipped if not needed
validpgpkeys is also similar to above checksums.

there are also 4 most common standard PKGBUILD shell functions such as;

prepare() is used to make changes or apply patches to the source code before the build process begins
build() is where the actual compilation or building of the software package takes place.
check() runs tests to ensure that the software behaves as expected. It is used to verify the correctness of the package before installation
package() puts the built files into a packaged format suitable for installation. It copies files into a temporary directory structure that mirrors the final installation directory.

Create a .gitignore

Since we will be building multiple times to confirm our package, I highly recommend a good .gitignore to not accidentally push your artifacts. Below is my .gitignore file.

*
!/.gitignore
!/.SRCINFO
!/PKGBUILD

Basic PKGBUILD for Go

From the thought of my mind, I have made a very simple PKGBUILD file for k3sup which is an amazing tool for bootstrapping k3s clusters. Please check k3sup out and support if you like.

Friendly reminder for pkgname; xyzpackage means a build from stable version of the source, xyzpackage-git means a build from the latest commit of the source, xyzpackage-bin means fetching prebuilt binary without build phase.

# Maintainer: Talha Altinel <talhaaltinel@hotmail.com>

pkgname=k3sup
pkgver=0.13.0
pkgrel=1
pkgdesc='A tool to bootstrap K3s over SSH in < 60s'
arch=('x86_64')
url='https://github.com/alexellis/k3sup'
license=('MIT')
depends=('openssh')
makedepends=('git' 'go>=1.20')
source=("${pkgname}-${pkgver}.tar.gz::https://github.com/alexellis/k3sup/archive/${pkgver}.tar.gz")
sha256sums=('24939844ac6de581eb05ef6425c89c32b2d0e22800f1344c19b2164eec846c92')
_commit=('1d2e443ea56a355cc6bd0a14a8f8a2661a72f2e8')

build() {
  cd "$pkgname-$pkgver"

  CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build \
    -ldflags "-s -w -X github.com/alexellis/k3sup/cmd.Version=$pkgver -X github.com/alexellis/k3sup/cmd.GitCommit=$_commit" \
    -o k3sup \
    .

  for shell in bash fish zsh; do
    ./k3sup completion "$shell" > "$shell-completion"
  done
}

package() {
  cd "$pkgname-$pkgver"

  install -vDm755 -t "$pkgdir/usr/bin" k3sup

  mkdir -p "${pkgdir}/usr/share/bash-completion/completions/"
  mkdir -p "${pkgdir}/usr/share/zsh/site-functions/"
  mkdir -p "${pkgdir}/usr/share/fish/vendor_completions.d/"
  install -vDm644 bash-completion "$pkgdir/usr/share/bash-completion/completions/k3sup"
  install -vDm644 fish-completion "$pkgdir/usr/share/fish/vendor_completions.d/k3sup.fish"
  install -vDm644 zsh-completion "$pkgdir/usr/share/zsh/site-functions/_k3sup"

  install -vDm644 -t "$pkgdir/usr/share/licenses/$pkgname" LICENSE
}

In my build phase, I compile the source to create a k3sup binary and I also run the binary to generate shell script completions. Give kudos to this functionality which comes from spf13/cobra Go library for CLIs.

In my package phase, I move the binary, the shell script completions and the license to correct places.

That all sounds cool and sweet but we are missing couple of things, so Arch Wiki has an extensive guide about this here but long story short I need a program called namcap and when I do sudo pacman -S namcap then run namcap on PKGBUILD and produced .zst archive.

$ namcap ./PKGBUILD

$ makepkg -s && namcap k3sup-0.13.0-1-x86_64.pkg.tar.zst
k3sup W: ELF file ('usr/bin/k3sup') lacks FULL RELRO, check LDFLAGS.
k3sup W: ELF file ('usr/bin/k3sup') lacks PIE.
k3sup W: Dependency included, but may not be needed ('openssh')

the biggest surprise was all of arch guides actually only allowed specifically built type of Go binaries with CGO :( the above PKGBUILD was completely valid but if you want your package in the official arch repositories outside of AUR, you need to ensure FULL RELRO and PIE are satisfied. I won't be explaining these terms too much, it is essentially binary hardening for the extreme security.

Security Hardened PKGBUILD for Go

# Maintainer: Talha Altinel <talhaaltinel@hotmail.com>

pkgname=k3sup
pkgver=0.13.0
pkgrel=1
pkgdesc='A tool to bootstrap K3s over SSH in < 60s'
arch=('x86_64')
url='https://github.com/alexellis/k3sup'
license=('MIT')
depends=('glibc' 'openssh')
makedepends=('git' 'go>=1.20')
source=("${pkgname}-${pkgver}.tar.gz::https://github.com/alexellis/k3sup/archive/${pkgver}.tar.gz")
sha256sums=('24939844ac6de581eb05ef6425c89c32b2d0e22800f1344c19b2164eec846c92')
_commit=('1d2e443ea56a355cc6bd0a14a8f8a2661a72f2e8')

build() {
  cd "$pkgname-$pkgver"
  export CGO_CPPFLAGS="${CPPFLAGS}"
  export CGO_CFLAGS="${CFLAGS}"
  export CGO_CXXFLAGS="${CXXFLAGS}"
  export CGO_LDFLAGS="${LDFLAGS}"
  export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw"

  go build \
    -ldflags "-s -w -X github.com/alexellis/k3sup/cmd.Version=$pkgver -X github.com/alexellis/k3sup/cmd.GitCommit=$_commit" \
    -o k3sup \
    .

  for shell in bash fish zsh; do
    ./k3sup completion "$shell" > "$shell-completion"
  done
}

package() {
  cd "$pkgname-$pkgver"

  install -Dm755 -t "$pkgdir/usr/bin" k3sup

  mkdir -p "${pkgdir}/usr/share/bash-completion/completions/"
  mkdir -p "${pkgdir}/usr/share/zsh/site-functions/"
  mkdir -p "${pkgdir}/usr/share/fish/vendor_completions.d/"
  install -Dm644 bash-completion "$pkgdir/usr/share/bash-completion/completions/k3sup"
  install -Dm644 fish-completion "$pkgdir/usr/share/fish/vendor_completions.d/k3sup.fish"
  install -Dm644 zsh-completion "$pkgdir/usr/share/zsh/site-functions/_k3sup"

  install -Dm644 -t "$pkgdir/usr/share/licenses/$pkgname" LICENSE
}

Now we run namcap again to verify after we run makepkg

$ namcap ./PKGBUILD

$ makepkg -s && namcap k3sup-0.13.0-1-x86_64.pkg.tar.zst                                                ✔  03:18:13  
k3sup W: Dependency included, but may not be needed ('openssh')

now it all looks amazingly secure at a binary level if your glibc version doesn't have a security vulnerability ;)

let's push it to the AUR now. Remember to renew .SRCINFO before every push. Also pay attention to already taken package names in AUR.

$ updpkgsums && makepkg --printsrcinfo > .SRCINFO
$ git init
$ git remote add origin https://aur.archlinux.org/k3sup.git
$ git add . && git commit -m "initial release"
$ git push -u origin master

The End Result

https://aur.archlinux.org/packages/k3sup

$ git clone https://aur.archlinux.org/packages/k3sup
$ cd ./k3sup && less ./PKGBUILD
$ makepkg -si

The References

“Victory usually goes to the army who has better trained officers and men”

— Sun Tzu

Arch Installation for Beginners

Talha Altınel — Thu, 14 Sep 2023 20:35:00 +0000

Getting Started

Hello everyone, in this blog I will help you bootstrap your arch linux setup in 5-10 minutes, and teach you where you can look into when you need help. Arch Linux has been one of the most difficult distros to setup until the new convenient archinstall script. I will be using archinstall script in this guide. It is known with its non-exhaustive user friendly TUI installation phase.

First of all, ensure you install the ISO from Download Page, Arch is used within the whole world so don't be scared of picking the closest mirror to yourself. All the mirrors have SSL/TLS enabled, the contents are the same you don't need to worry about it. Also archlinux-x86_64.iso and archlinux-YYYY.MM.DD-x86_64.iso correspond to the same ISO, there is no difference.

Second of all, you need a software to burn the ISO, I used to use fedora media writer or opensuse image writer. But Balena Etcher is the cross-platform option for your own OS, after you insert the USB flash stick, just select the ISO and click burn or write.

The Requirements

USB flash stick (recommended space >= 2G)
ISO burner/ISO writer software (Balena Etcher)
The internet (either with reliable ethernet or wifi)
The keyboard

Setting up with Archinstall

After your burned the ISO then you plug in the USB flash stick and start the computer, you should see the classic Arch boot menu.

Firstly, you will need an internet connection, if you have ethernet you can just plug the ethernet cable and continue. If you have wifi, you need to type a few commands then enter the password(passphrase).

root@archiso $ iwctl
[iwd] $ help
[iwd] $ device list
[iwd] $ station wlan0 scan
[iwd] $ station wlan0 get-networks
[iwd] $ station wlan0 connect YOUR_NETWORK_NAME
Passphrase: *********
[iwd] $ station wlan0 show
[iwd] $ exit

Then we write and enter archinstall to join TUI installation phase. There are couple of options presented to us;

root@archiso $ archinstall

Language

I pick English.

Mirror

I pick the closest mirror to me as a country.

Locales

I pick my locales for my keyboard.

Disk configuration

I pick BTRFS with compression/subvolumes enabled and auto disk partitioning. Please skip the disk encryption as it complicates many things.

Bootloader

I like the standard GRUB.

Swap

Always yes.

Hostname/Root Password/User Account

Profile

I pick Desktop/Gnome/all open-source drivers. You may need to pay attention to the drivers if you have NVIDIA driver.

Audio

Pipewire for latest hardware, pulse for very old hardware.

Kernel

linux is the normal latest one. (I prefer this)
linux-hardened is full of security therefore has lots of restraints.
linux-lts is LTS (old) kernel.
linux-zen is for performance machines with higher power usage such as gaming.

Network Configuration

Pick network manager if you are using GNOME or KDE.

Timezone / Automatic Time Sync

I pick my timezone and enable automatic time sync as the clock changes in winter and summer times.

Install

Finally click on install and enjoy, you can remove the flash stick after it asks you to do so when installation finishes.

Tuning Pacman

Pacman is the fastest package manager in the unixverse, but a little bit of tuning is required to make it skyrocket out of the roof. Big warning, once it is tuned, you will never enjoy mediocre package managers.

In /etc/pacman.conf, change #color row to color, add a below row under it ILoveCandy, also change #ParallelDownloads = 5 to ParallelDownloads = 10, you can make the number higher as well if your internet is good.

We also need to set up paccache (pacman cache cleaner for weekly)

$ sudo pacman -S pacman-contrib
$ sudo systemctl enable paccache.timer

Lastly, let's edit /etc/makepkg.conf, change MAKEFLAGS="-j4" to MAKEFLAGS="-j$(nproc)"

Pacman Cheatsheet

dnf upgrade equivelant, upgrades your whole system including the kernel

$ pacman -Syu

dnf search equivelant, only searches through official repos, not AUR

$ pacman -Ss xPackage

dnf install equivelant, installs the package to the system

$ pacman -S xPackage

dnf remove equivelant, removes the package from the system

$ pacman -Rs xPackage

dnf list --all equivelant, lists all the packages

$ pacman -Q

Using AUR for Unofficial Packages

I will be showing how you can install Brave through AUR, AUR is everyone's ship to upload packages, you can make packages for anyone and upload it to AUR, so we need to be EXTREMELY careful while using AUR.

We will be downloading Brave from brave-bin, common Arch package convention is to rename packaged software accordingly;

abc (means build from stable version of the source)
abc-git (means build from main branch of the source)
abc-bin (means pre-built stable version binary)

We inspect URLs in PKGBUILD file for brave-bin to ensure we are not downloading the next crypto miner bot for our GPU.

$ mkdir AUR && cd ~/AUR
$ git clone https://aur.archlinux.org/brave-bin.git
$ cd ./brave-bin
$ less PKGBUILD
$ makepkg -si

Important note: DO NOT ever install an AUR helper, it is your job as an user to check every single AUR package before installing something, if something is not in official arch repo, use AUR wisely or even contribute to it!

Enable Microcode

Microcode is a very low-level instruction set which is stored permanently in a computer, it helps your processor's power and computing efficiency.

For intel intel-ucode, for AMD amd-ucode

$ sudo pacman -S intel-ucode

Enable Bluetooth

Sometimes bluetooth may not work on Linux in general but since you are on Arch, you just need a single package to make it work for every bluetooth driver out there. Remember this is not Linux's fault, it is the fault of hardware manufacturers due to not open sourcing bluetooth drivers.

sudo pacman -S bluez bluez-utils
systemctl start bluetooth
sudo systemctl status bluetooth
sudo systemctl enable bluetooth

● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Tue 2023-07-01 11:31:45 BST; 1 day 7h ago
       Docs: man:bluetoothd(8)
   Main PID: 633 (bluetoothd)
     Status: "Running"
      Tasks: 1 (limit: 76824)
     Memory: 3.0M
        CPU: 46.646s
     CGroup: /system.slice/bluetooth.service
             └─633 /usr/lib/bluetooth/bluetoothd

Disable GRUB Screen

The thing is when you open your computer and see GRUB screen everytime, it could be annoying unless you have multiple OSes in a single machine. So I recommend to change your /etc/default/grub. Under GRUB settings, I change GRUB_TIMEOUT_STYLE=countdown to GRUB_TIMEOUT_STYLE=hidden to get rid of waiting on GRUB screen for 5 seconds. You can still visit GRUB screen with special Fx key on your machine.

My Personal Taste of Software (Optional)

$ sudo pacman -S wget curl git neofetch sl cowsay fortune-mod lolcat nmap mandoc vlc thunderbird obsidian discord gimp converseen calibre libreoffice-still go kubectl docker terraform rsync nodejs pnpm vscode

Installing vscode on arch, requires a small metadata patching for vscode marketplace as this vscode is not closely connected to Microsoft telemetry or Microsoft binary directly. We will use AUR for this purpose, please inspect the contents of PKGBUILD and other scripts for your own safety all the time. If we don't patch our vscode marketplace, some plugins will be missing. please ALWAYS inspect the contents please.

$ cd ~/AUR
$ git clone https://aur.archlinux.org/code-marketplace.git 
$ cd ./code-marketplace
$ less PKGBUILD
$ makepkg -si

Install Zsh (Optional)

Zsh is the fully fledged shell for end-users. You can install by doing sudo pacman -S zsh then run chsh -s $(which zsh)

After you have installed zsh, we will be theming with ohmyzsh and powerlevel10k and also couple of shell plugins such as zsh completion, zsh autosuggestion, zsh highlighting

The installation of ohmyzsh is quite straight forward but please do INSPECT the shell script for the peace of the mind;

$ wget https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh
$ less install.sh
$ sh install.sh

Logout and login again and open your shell to complete prompted options.

The installation of powerlevel10k is a bit more different. You need to install manually these 4 fonts; MesloLGS NF "Regular, Bold, Italic, Bold Italic" then double click each .ttf file and click install. Here is the link to the fonts

Let's now install powerlevel10k through our ohmyzsh settings.

$ git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k

Now we can set ZSH_THEME="powerlevel10k/powerlevel10k" in ~/.zshrc file. And simply run exec zsh && p10k configure.

At least but not the last, if you check out zsh users repository we have 3 must zsh productivity plugins; completions, highlighting and autosuggestions. Let's install them one by one.

We install zsh-completions through ohmyzsh again. This will give you a custom CLI completion if a program provides its zsh completion.

$ git clone https://github.com/zsh-users/zsh-completions ${ZSH_CUSTOM:-${ZSH:-~/.oh-my-zsh}/custom}/plugins/zsh-completions

Add it to FPATH in your .zshrc by adding the following line before source "$ZSH/oh-my-zsh.sh"

fpath+=${ZSH_CUSTOM:-${ZSH:-~/.oh-my-zsh}/custom}/plugins/zsh-completions/src

Then we install zsh-syntax-highlighting and zsh-autosuggestions. This will give you correct text highlighting and autosuggestions from the previous terminal history.

$ git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
$ git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions

Now modify plugins= in your ~/.zshrc file.

plugins=( 
    # other plugins...
    zsh-syntax-highlighting
    zsh-autosuggestions
)

Additional Help

If you get stuck or encounter a very difficult problem, you can check out the forums, arch forums/wiki/AUR is what makes arch the best distro in the unixverse.

“Power to the people”

— Unknown

Why You Should Not Trust Corporate Linux Distros

Talha Altınel — Fri, 04 Aug 2023 11:47:21 +0000

The Great Drama

As you know, Red Hat(actually IBM) decided to shoot itself in the foot again a month ago by closing the RHEL source code leaving downstream community/corporate based linux distros airless and causing great distress to many businesses and many people. If you wanna get a quick summary, I suggest reading Red Hat strikes a crushing blow against RHEL downstreams

As a Fedora user, I have tried to sit back and relax and watch from a distant place to see if there were gonna be a decision to rollback on this decision but unfortunately, Red Hat press kept fueling the fire for a while, I also love to analyze fast decisions in the long term to see if it was really well-thought but I have to say, I was a bit Red Hat biased because of being a micro business owner and seeing many open source small businesses not making enough from what they provide to other people.

After giving my brain relaxed space, I noticed there are so many things wrong with Red Hat from calling people freeloaders to going legal wars against Rocky Linux who allegedly made deals with billion dollars NASA contracts. You may say, well Redhat is a business and legally they can close the source and keep selling their distro while avoiding copycats(extremely wrong to call downstreams copycats, downstream software is as important as upstream software), this was my first thought too from business perspective. But in reality, Red Hat is no longer a trustworthy company and they have gained the habit of shooting themselves in the foot when they have no longer 2 feet to stand on (remembering CentOS's gravestone and empty 10 year commercial support promises)

It was kinda sad for me because Red Hat was the most flourishing company for enterprise Linux. To extend the irony, Oracle and SUSE even went ahead and did retaliation for GPL breaches in legal cases. We do know how Oracle scrutinized open source in the old days with legal court cases against Google after they have become the JVM connoisseur.

I will also suggest watching LearnLinuxTV youtube video to build up why company owned distros like Red Hat are not as reliable as everyone thought;

My Linux Experience

I am not the best Linux user in the world but I started my journey in 2018 with Ubuntu 18.04 release at uni times. I didn't actually understand its value at that time but it was very nice to download python and other developer related things with a few commands and compile things then if I can help people's issues or questions by learning their projects.

When I started working at the end of 2019, I started hosting my developed projects in small friendly clouds like Hetzner, Digital Ocean, Linode, Vultr etc. I was actually a heavy heroku user due to having no Linux experience but I realized it is actually more interesting and cost effective to learn Linux for my own servers whether VMs or VPS machines. All of my servers used Ubuntu since 2020.

After windows 10, I kinda started to get windows alergy in terms of machine resources. It was not that bad but I was more into WSL2 debian/ubuntu shells than setting up custom shells like MSYS2 or Git Bash. WSL2 kept me there for a while in windows 10. Then windows 11 released and I was super frustrated with all the bloat software it came with, some softwares were not removable and some were but if you did delete them, they would come in next FORCED update. At the end of 2021, I have gotten rid of all windows machines and installed Fedora 34-35. I was super happy with Gnome 40 versions and I have been hooked to Fedora(simple package manager, dead easy to install) so well.

By March, 2021, I have switched all my server machines from ubuntu to debian
By Aug, 2023, I have switched my local machines from fedora to arch

Personal Decisions

Why did I switch from Ubuntu(Canonical) to something else?

I had a personal drama with Canonical, I interviewed for a snap position as a software engineer and they sent me 10 stages of interviews in 2021, luckily I have been eliminated at stage 3 by asking "how do you scan these snap packages for security? what are the regulations for someone who wants to upload random snap packages?" to an engineering manager. He replied "this is confidential corporate information, I can not answer that" like I asked for the password to the vault, Canonical looked like stressful, insincere, untrustable and unhappy people to me, there were also a few controversies from Richard Stallman calling them spyware about the sold Amazon store telemetry data in 2012, however this is no longer the case after a huge community pushback, they no longer have opted-in Amazon store telemetry. I have still a huge grudge against snaps tho. They remind me of Xbox software center of windows10, which comes back everytime with an OS update even if you remove them forcefully, it is fully dictated useless piece of software packing for convenience. And there to ruin your life with auto-updates without your consent.

Why did I switch from Fedora(Red Hat) to something else?

This was a tough choice to do as I escaped from one giant to another giant. I have trusted Fedora for 2 years but they have let me down with their stand on open source when they are heavily based on open source itself and also flatpaks I installed recently started crashing all the time to an unbearable point. I am looking at you postman flatpak. Some may say this is extremely emotional argument and I would say it is personal like my Canonical choice. I as a person have seen enough injustice and unethical behaviour until this age, so screwing up hard-working people from RockyLinux/AlmaLinux/insert X distro/OracleLinux(even them) is a big no for me. And not keeping their promises is also a huge letdown for any of Red Hat products. Also on top of that, calling their actual users freeloaders and trying to short-circuit open-source software supply chain around them?

Rational Decisions about Corporate Linux Distros

So let's visit rational decisions, There is a euphoria of consuming something that is prepared by a giant company or a brand because of giant marketing. And lots of people trust giant companies more than small companies or small communities. Why? well they think more people are more successful in creating products. This is a myth, there is even a book called "The mythical man-month" written by Frederick P. Brooks Jr. which explains this myth very well, I highly recommend checking this book out, essentially more people means more complexity to managing efficiency and productivity, people are not honey bees where you increase the count and get the best quality and large quantity of honey. This is why big companies have a higher reputation for their products than a normal unknown small company/community which does the same thing.

Unfortunately, when your business becomes the biggest and the most reputable contender, they can do anything they want and get more reluctant about what customers/clients think. Maybe exceptions like Amazon exists, but let's not get there as it is a different topic.

I do think some Linux distros of corporates are organized well and near amazing but the trust is what matters at the end of the day. I also don't believe everyone in Red Hat or Canonical is evil people, they are just a bunch of peeps trying to make a buck for their day. It is those evil managers with extreme greed who chain the wrong command sequence to the higher level and screw that reputation down to 0 trust. I am looking at you IBM people.

I still recommend beginner friendly distros like Ubuntu, Fedora to people who are dead new to Linux desktops like very rookie non-dev people. But if you are serious/experienced people, you can cut the middleman and go fully community based distro like Debian, Arch. It is also important to remember and remind the history of these corporations to new beginners so they can decide on their own fate.

At the end of the day, Linux and GNU didn't come out as easily as one giant corporation product, they were fully community based and community funded. It took everything including web servers, market share, developers, operations by storm and poured out tons of jobs and productivity gains. If you have 10000 people using a community product, a person can sky-rocket the product(the distro or the open source project) by giving just as small as $1 or filling issues or fixing bugs to gain intellectual joy or pay back for the productivity they gained from.

I also run a micro hosting business so I don't like non-trustable or non-predictable greedy companies to work with. If they dump the distro, you are on your own to migrate to a new distro with existing hosted software workloads which is not FUN at all.

Open Source Philosophy

Some companies unfortunately don't like give-take motion, it is always take take, take forever. They die and be gone then what they have achieved was a meaningless grind with high digits amount of cash. I believe in balanced open source philosophy where people give and take at the same time. The most successful innovative projects have been this way plus there is always something you can open source like an indie project or noncommercial fun project that is gonna be out there when you die one day. It could be in docs, issues form, doesn't even have to be the code.

Using a community distro means you are highly intellectual user with a goal in mind to aid the community and amplify their voice. Not every normal user may wanna go down this path as you sometimes will be checking mail lists, forums and VCS issues like instragram. But you can also just be an user as linux community disros are getting easier to use every day. I am looking at you archinstall(the best thing I have seen for arch linux desktop users) script and yes debian server installation is still dead confusing :)

Then why you should use and devote time(if you wish) to community distros?

Drama Free
No middleman, you don't depend on anyone's very dumb corporate decisions
Joyful learning cycle for the curious mind and always people there to help you out
Freedom to do anything you want without being in drama circle
Freedom to brick your operating system
Freedom to hang out on issues forums or bug forums to give an insight of what you have seen or how you have solved
You don't like take, take, take materialistic cycle; you love to produce ideas and give something back sometimes
You like to feel like attending religious communities(church, mosque, synagogue) who embraces but you hate cults and politics

“Sometimes it is the people who no one imagines anything of, who do the things that no one can imagine.”

— Alan Turing

Broke Captain's Kubernetes Cluster Guide(super simple & convenient cost)

Talha Altınel — Sun, 05 Dec 2021 03:54:23 +0000

In this guide, I will be showing how to set up a simple Kubernetes(K3S) cluster which will have 1 master node and 2 worker nodes on Hetzner Cloud. My main goal is to make newcomers' transition to Kubernetes very smooth as a person who suffered enough with complex tutorials/bills and didn't get enough chance to poke a Kubernetes cluster.

This tutorial should be applicable to any cloud provider but be warned pricing would be extremely different. If you come to learn Kubernetes, this could be your starting point to set up your own cluster and get started poking around with an actual production-ready cluster with k3s.

Quick QA

What is K3S?

It is a production-ready, stable and lightweight flavor of Kubernetes, think it is like Debian being a flavor of Linux. It is also the best choice for learning multi-master and multi-worker node architecture.

Why not teach us minikube/kind/microk8s?

They are not good enough for production workloads.

Why Hetzner cloud?

It is super cheap and simple cloud UI for me, you can do the same things in Vultr, Linode, Digital Ocean. Note: I am not sponsored by Hetzner Cloud

Do I need to install docker?

Not needed because k3s binaries ship with everything that is needed

What is the difference between a master node and a worker node?

A master node is often referred to a K3S server and a worker node is often referred to a K3S agent. For high availability(HA), the recommendation is to have at least 3 master nodes, 3 worker nodes, and 1 managed database outside of your master node instead of having an embedded SQLite database.

Setup your SSH key, network and compute instances

Before we create our compute instance(VPS), we will need SSH key setup, private network setup. Let's quickly go over it. Let's generate our ssh-key 🗝️

Let's add our ssh-key to our local machine and public ssh-key to the cloud UI. This panel may be different depending on your cloud provider. 🗝️

After that, let's quickly create our private network which will be used for our compute instances for the cluster's nodes communication between the master and the worker nodes. ☎️

We are finally ready to create our compute instances. Now I will create my master node which can also be called as k3s server. I will call my master node's name "jack-sparrow". I will pick "Debian 11" as my Linux distro choice for rock-solid server stability and being a reliable open source project.

I will also take advantage of multiple instance creation and set the instance count to 3. The other 2 instances will be my worker nodes which can also be called as k3s agents. I will call them "black-pearl" and "flying-dutchman". If you want to extend your worker nodes, you can keep going with all the ship names from Pirates of the Caribbean. For master nodes, I will be using captain names 🏴‍☠️

I have picked CX11 instance which is the cheapest option available. 6GB RAM and 60GB SSD should be sufficient enough for most of your projects. I skipped additional volume and the firewall. I added my created network and my created SSH key. Remember, this is for broke captains. 🚢

Prerequisites for K3S

SSH into the master node and the worker nodes. Update /etc/hosts files before we get our great k3s binaries which has everything including containerd runtime and CNI(container network interface). 🐋

$ ssh root@94.130.227.124

$ apt update && sudo apt upgrade

$ apt install apparmor apparmor-utils // Debian dependency for the Kernel

After you have updated every nodes' /etc/hosts file with GNU nano. Optionally you can install nmap CLI tool to make sure your network is functioning properly and other instances are connected through the web with nmap -sn 10.0.0.1/24 🕸️

Installation of K3S

In jack-sparrow instance(master node), let's install k3s server.

$ curl -sfL https://get.k3s.io | sh -

$ systemctl status k3s.service
● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-12-04 17:39:07 UTC; 3min 0s ago
       Docs: https://k3s.io
    Process: 26339 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
    Process: 26341 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 26342 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 26343 (k3s-server)
      Tasks: 18
     Memory: 502.4M
        CPU: 27.604s
     CGroup: /system.slice/k3s.service
             └─26343 /usr/local/bin/k3s server

$ kubectl get nodes -o wide
NAME           STATUS   ROLES                  AGE    VERSION        INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION   CONTAINER-RUNTIME
jack-sparrow   Ready    control-plane,master   4m4s   v1.21.7+k3s1   94.130.227.124   <none>        Debian GNU/Linux 11 (bullseye)   5.10.0-9-amd64   docker://20.10.11

Let's configure incoming/outgoing ports for jack-sparrow, our k3s server. Also obtain the token which will be used during k3s agents setup.

$ apt install ufw
$ ufw default allow outgoing
$ ufw default deny incoming
$ ufw allow 22,80,443,6443,10250/tcp
$ ufw --force enable
Firewall is active and enabled on system startup

$ cat /var/lib/rancher/k3s/server/node-token
bc3f7dee0308f09e5a3645f4b06343eea2644296cdK1d79a977d0e193a10187497f::server:9ae1e45b8b58be56a8282a84c7e3715b

Let's install k3s agents in our computing instances which are called black-pearl and flying-dutchman! Our master IP (jack-sparrow) is 10.0.0.4 in our nebula network. And our token is bc3f7dee0308f09e5a3645f4b06343eea2644296cdK1d79a977d0e193a10187497f::server:9ae1e45b8b58be56a8282a84c7e3715b

curl -sfL http://get.k3s.io | K3S_URL=https://10.0.0.4:6443 K3S_TOKEN=bc3f7dee0308f09e5a3645f4b06343eea2644296cdK1d79a977d0e193a10187497f::server:9ae1e45b8b58be56a8282a84c7e3715b sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.21.7+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO]  systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO]  systemd: Starting k3s-agent

Perfect! Let's jump into Jack Sparrow! and see what we got!

root@jack-sparrow:~# kubectl get nodes -o wide
NAME              STATUS   ROLES                  AGE   VERSION        INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION   CONTAINER-RUNTIME
jack-sparrow      Ready    control-plane,master   50m   v1.21.7+k3s1   94.130.227.124   <none>        Debian GNU/Linux 11 (bullseye)   5.10.0-9-amd64   docker://20.10.11
black-pearl       Ready    <none>                 28s   v1.21.7+k3s1   116.203.32.141   <none>        Debian GNU/Linux 11 (bullseye)   5.10.0-9-amd64   docker://20.10.11
flying-dutchman   Ready    <none>                 12s   v1.21.7+k3s1   116.203.90.71    <none>        Debian GNU/Linux 11 (bullseye)   5.10.0-9-amd64   docker://20.10.11

Extras(Install LENS)

Lens is a Kubernetes UI for managing your cluster resources. It comes bundled with Helm and kubectl for your local workstation. You can install lens binary from the github under the name lensapp/lens. We will be taking the kube config from jack sparrow and pasting it into your Lens. To do that let's find our kube config and copy paste. And change the server IP address to external IP address of our jack sparrow instead of 127.0.0.1

root@jack-sparrow:~# cat /etc/rancher/k3s/k3s.yaml

Let's create pirate-deployment.yaml, pirate-service.yaml and traefik-ingress.yaml to see generated lens metrics. And apply them in our cluster.

# pirate-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pirate-deploy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: pirate-app
  template:
    metadata:
      labels:
        app: pirate-app
    spec:
      containers:
        - name: simple-pirate
          image: mrwormhole/simple-pirate
          ports:
            - containerPort: 80

# pirate-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: pirate-svc
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: pirate-app

# traefik-ingress.yaml
apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: pirate-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service: 
            name: pirate-svc
            port: 
              number: 80

$ kubectl create -f pirate-deployment.yaml
$ kubectl create -f pirate-service.yaml
$ kubectl create -f traefik-ingress.yaml
$ kubectl get ingress
NAME             CLASS    HOSTS   ADDRESS                                       PORTS   AGE
pirate-ingress   <none>   *       116.203.32.141,116.203.90.71,94.130.227.124   80      136m

Now we can enable the lens metrics from pinned clusters and go to its settings and install all of the required things via lens metrics tab. We will need prometheus, kube state metrics and node exporter from lens metrics section.

The End

I really thank you for making it to the end! I tried to simplify things as much as possible 🙂

I want to send special thanks and give credit to Victor Shamallah and Alex Ellis I also hope this guide was helpful to the readers and the newcomers. If you have learned something new, feel free to share. If you have any feedback/suggestions/problems, spam in the comments section. Have a good day!

References

Install K3S on Ubuntu with Docker

K3S Installation Requirements

Kubernetes Ingress - Traefik

[DEV PART 2/2] Serverless Highscore Go API with Faasd and CockroachDB

Talha Altınel — Wed, 08 Sep 2021 18:17:27 +0000

The Intro

Hi everyone, this is the 2nd part of the series, we will be developing our API in this part. I will assume you have already followed the previous part and setup faasd and CockroachDB in your cloud server instance and have faas-cli in your both client computer and cloud server instance. I will also assume you have Go on your computer and a proper text editor. Let's quickly get started.

highscore-api-github-repo

Requirements:

Go knowledge
docker hub account
faas-cli
up and running faasd server
basic SQL knowledge

First, we would like to make sure your faas-cli works correctly in your server, you should already know your server IP address, your username and your password for faasd. Let's see if the server instance validates us.

faas-cli login -g http://23.88.60.124:8080 -u admin -p jackthegiant

Faasd Project Init

faas-cli template store pull golang-http
faas-cli new --lang golang-http get-highscores

The above command will create a yml file and a function handler that we will have to adjust for faasd. As an initial clean up, I will rename my get-highscores.yml to stack.yml, this file will contain our functions for faasd. It is general practice to have it as stack.yml because you will need 1 less flag during faas-cli up -f filename.yml

I will also change the provider's gateway to my server cloud instance which is http://[[SERVER_IP]]:8080.In my case, It is http://23.88.60.124:8080.

The other most important part is to give your docker hub container name to image names and turn on go modules in environment variables. Here is what it looks like after tidying up stack.yml. Make sure you login to your docker hub account and create a repository there first

version: 1.0
provider:
  name: openfaas
  gateway: http://23.88.60.124:8080
functions:
  get-highscores:
    lang: golang-http
    handler: ./get-highscores
    image: mrwormhole/get-highscores:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db

Now you have the initial configuration setup, let's deploy your generated handler to see if it is getting deployed. Your template code should look like this. The best part is now you can deploy very easily with a single command. This single up command will build your container(faas-cli build), deploy your code to the container registry(faas-cli push) then pull that container to your
cloud server(faas-cli deploy) instance.

get-highscores/handler.go

package function

import (
    "fmt"
    "net/http"

    handler "github.com/openfaas/templates-sdk/go-http"
)

// Handle a function invocation
func Handle(req handler.Request) (handler.Response, error) {
    var err error

    message := fmt.Sprintf("Body: %s", string(req.Body))

    return handler.Response{
        Body:       []byte(message),
        StatusCode: http.StatusOK,
    }, err
}

docker login
faas-cli up

You can additionally use faas-cli list to see running functions. Now I will grab sqlc to generate a repository layer for our Go function handler. To use sqlc, you will install its CLI, sqlc.json file which will point to our queries.sql and schema.sql

go get github.com/kyleconroy/sqlc/cmd/sqlc

Here is how my sqlc.json, schema.sql and queries.sql look like. If you don't know basic SQL, I strongly suggest you to visit W3C SQL docs for quick recap and have a look at sqlc docs

sqlc.json

{
  "version": "1",
  "packages": [
    {
      "path": "repository",
      "name": "repository",
      "queries": "queries.sql",
      "schema": "schema.sql"
    }
  ]
}

schema.sql

CREATE TABLE highscores (
  id BIGSERIAL PRIMARY KEY,
  username TEXT NOT NULL UNIQUE,
  score BIGINT NOT NULL
);

queries.sql

-- name: GetHighscore :one
SELECT * FROM highscores
WHERE username = $1 LIMIT 1;

-- name: ListHighscores :many
SELECT * FROM highscores
ORDER BY score;

-- name: CreateHighscore :one
INSERT INTO highscores(username, score) 
VALUES ($1, $2) RETURNING *;

-- name: UpdateHighscore :one
UPDATE highscores
SET score = $2
WHERE id = $1 RETURNING *;

-- name: DeleteHighscore :exec
DELETE FROM highscores
WHERE username = $1;

Now we can generate our repository layer since we have completed all of the database interactions. The below command will generate all of the repository code for Go from SQL.

sqlc generate

I will initialize go modules and get pq which is a pure Go postgres driver. Why do we use postgres driver for CockroachDB? CockroachDB supports PostgreSQL wire protocol. This means it is almost fully compatible with postgres drivers and ORMs.

go mod init github.com/mrwormhole/highscore-api
go get github.com/lib/pq

Let's finish up our handler for get-highscores. I will establish a database connection and check for the correct HTTP method. I will also check if there is a username query for the highscore. If yes, I will return a specific user's highscore. Otherwise, I will return all of the highscores in the database. Please make sure to import lib/pq manually.

get-highscores/handler.go

package function

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "net/url"
    "os"
    "strings"

    _ "github.com/lib/pq"
    "github.com/mrwormhole/highscore-api/repository"
    handler "github.com/openfaas/templates-sdk/go-http"
)

func Handle(req handler.Request) (handler.Response, error) {
    db, err := sql.Open("postgres", fmt.Sprintf("host=%s port=%s user=%s dbname=%s sslmode=disable",
        os.Getenv("POSTGRES_HOST"),
        os.Getenv("POSTGRES_PORT"),
        os.Getenv("POSTGRES_USER"),
        os.Getenv("POSTGRES_DB")))
    defer func() {
        err = db.Close()
        if err != nil {
            log.Printf("failed to close db: %v", err)
        }
    }()
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to connect to db: %v", err)
    }
    if req.Method != http.MethodGet {
        return handler.Response{
            StatusCode: http.StatusBadRequest,
        }, fmt.Errorf("invalid http method %s", req.Method)
    }

    values, err := url.ParseQuery(req.QueryString)
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to parse query string: %v", err)
    }

    var rawBody []byte
    queries := repository.New(db)
    username := values.Get("username")

    if strings.TrimSpace(username) != "" {
        highscore, err := queries.GetHighscore(req.Context(), username)
        if err != nil {
            if err == sql.ErrNoRows {
                return handler.Response{
                    StatusCode: http.StatusNotFound,
                }, nil
            }
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to get highscore for username %s: %v", username, err)
        }

        rawBody, err = json.Marshal(highscore)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to marshal a highscore: %v", err)
        }
    } else {
        highscores, err := queries.ListHighscores(req.Context())
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to list highscores: %v", err)
        }

        rawBody, err = json.Marshal(highscores)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to marshal highscores: %v", err)
        }
    }

    return handler.Response{
        Body:       rawBody,
        StatusCode: http.StatusOK,
    }, nil
}

Now I will create my second function and create its docker hub repo and tidy up stack.yml. I will also add a token credential so that not everyone can add highscore to my database.

faas-cli new --lang golang-http post-highscore --append stack.yml

version: 1.0
provider:
  name: openfaas
  gateway: http://23.88.60.124:8080
functions:
  get-highscores:
    lang: golang-http
    handler: ./get-highscores
    image: mrwormhole/get-highscores:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db

  post-highscore:
    lang: golang-http
    handler: ./post-highscore
    image: mrwormhole/post-highscore:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db
      BEARER_TOKEN: QeV5f7eSvJnO0dDYCc9DcH5BEwpm7P3j

I will create a package called model and middleware. My model will only contain how a request should look like and my middleware will look like a basic auth header check against our specified BEARER_TOKEN env variable.

model/highscore.go

package model

type Highscore struct {
    Username string `json:"username"`
    Score    int64  `json:"score"`
}

middleware/auth.go

package middleware

import (
    "errors"
    "os"
    "strings"

    handler "github.com/openfaas/templates-sdk/go-http"
)

func Authorization(req handler.Request) error {
    authHeader := req.Header.Get("Authorization")
    authHeaderValues := strings.Split(authHeader, " ")
    if len(authHeaderValues) != 2 || authHeaderValues[0] != "Bearer" {
        return errors.New("authorization header is in the wrong format")
    }
    if authHeaderValues[1] != os.Getenv("BEARER_TOKEN") {
        return errors.New("bearer token is not valid")
    }

    return nil
}

Finishing up the handler for post-highscore. I will establish a database connection and check for the correct HTTP method. I will check for the authorization header. If there are no users with that username, we will create a new one and return that in the body. If there is someone with that username, we will check the incoming request's highscore and compare it with the one that highscore that is persisted. If that is higher, we can go ahead and update then return that in the body. Otherwise, we return empty 200 to the request.

post-highscore/handler.go

package function

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"

    _ "github.com/lib/pq"
    "github.com/mrwormhole/highscore-api/middleware"
    "github.com/mrwormhole/highscore-api/model"
    "github.com/mrwormhole/highscore-api/repository"
    handler "github.com/openfaas/templates-sdk/go-http"
)

func Handle(req handler.Request) (handler.Response, error) {
    db, err := sql.Open("postgres", fmt.Sprintf("host=%s port=%s user=%s dbname=%s sslmode=disable",
        os.Getenv("POSTGRES_HOST"),
        os.Getenv("POSTGRES_PORT"),
        os.Getenv("POSTGRES_USER"),
        os.Getenv("POSTGRES_DB")))
    defer func() {
        err = db.Close()
        if err != nil {
            log.Printf("failed to close db: %v", err)
        }
    }()
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to connect to db: %v", err)
    }
    if req.Method != http.MethodPost {
        return handler.Response{
            StatusCode: http.StatusBadRequest,
        }, fmt.Errorf("invalid http method %s", req.Method)
    }

    err = middleware.Authorization(req)
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusBadRequest,
        }, fmt.Errorf("%v", err)
    }

    var highscore model.Highscore
    err = json.Unmarshal(req.Body, &highscore)
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to unmarshal highscore")
    }

    queries := repository.New(db)
    existingHighscore, err := queries.GetHighscore(req.Context(), highscore.Username)
    if err != nil && err != sql.ErrNoRows {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to get a highscore: %v", err)
    }

    if existingHighscore.ID == 0 {
        params := repository.CreateHighscoreParams{Username: highscore.Username, Score: highscore.Score}
        createdHighscore, err := queries.CreateHighscore(req.Context(), params)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to create a highscore: %v", err)
        }

        raw, err := json.Marshal(createdHighscore)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to marshal created highscore")
        }

        return handler.Response{
            Body:       []byte(raw),
            StatusCode: http.StatusOK,
        }, nil
    }

    if highscore.Score > existingHighscore.Score {
        params := repository.UpdateHighscoreParams{ID: existingHighscore.ID, Score: highscore.Score}
        updatedHighscore, err := queries.UpdateHighscore(req.Context(), params)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to update a highscore: %v", err)
        }

        raw, err := json.Marshal(updatedHighscore)
        if err != nil {
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to marshal updated highscore")
        }

        return handler.Response{
            Body:       []byte(raw),
            StatusCode: http.StatusOK,
        }, nil
    }

    return handler.Response{
        StatusCode: http.StatusOK,
    }, nil
}

Now I will create my third and final handler and respectively its docker hub repo. I will add a token credential to this handler as well. Because not everyone needs to delete someone else's highscore :) your final yaml structure is given below.

faas-cli new --lang golang-http delete-highscore --append stack.yml

version: 1.0
provider:
  name: openfaas
  gateway: http://23.88.60.124:8080
functions:
  get-highscores:
    lang: golang-http
    handler: ./get-highscores
    image: mrwormhole/get-highscores:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db

  post-highscore:
    lang: golang-http
    handler: ./post-highscore
    image: mrwormhole/post-highscore:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db
      BEARER_TOKEN: QeV5f7eSvJnO0dDYCc9DcH5BEwpm7P3j

  delete-highscore:
    lang: golang-http
    handler: ./delete-highscore
    image: mrwormhole/delete-highscore:latest
    build_args:
      GO111MODULE: on
    environment:
      POSTGRES_HOST: 23.88.60.124
      POSTGRES_PORT: 26257
      POSTGRES_USER: root
      POSTGRES_DB: highscore_db
      BEARER_TOKEN: Ru4BXyL7ALkey34cUJIIXBF67t1qrw37

This handler will also handle its database connection and validate the authorization header then check the username in the URL query. Afterward, we delete the highscore that matches that username.

delete-highscore/handler.go

package function

import (
    "database/sql"
    "fmt"
    "log"
    "net/http"
    "net/url"
    "os"
    "strings"

    _ "github.com/lib/pq"
    "github.com/mrwormhole/highscore-api/middleware"
    "github.com/mrwormhole/highscore-api/repository"
    handler "github.com/openfaas/templates-sdk/go-http"
)

func Handle(req handler.Request) (handler.Response, error) {
    db, err := sql.Open("postgres", fmt.Sprintf("host=%s port=%s user=%s dbname=%s sslmode=disable",
        os.Getenv("POSTGRES_HOST"),
        os.Getenv("POSTGRES_PORT"),
        os.Getenv("POSTGRES_USER"),
        os.Getenv("POSTGRES_DB")))
    defer func() {
        err = db.Close()
        if err != nil {
            log.Printf("failed to close db: %v", err)
        }
    }()
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to connect to db: %v", err)
    }
    if req.Method != http.MethodDelete {
        return handler.Response{
            StatusCode: http.StatusBadRequest,
        }, fmt.Errorf("invalid http method %s", req.Method)
    }

    err = middleware.Authorization(req)
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusBadRequest,
        }, fmt.Errorf("%v", err)
    }

    values, err := url.ParseQuery(req.QueryString)
    if err != nil {
        return handler.Response{
            StatusCode: http.StatusInternalServerError,
        }, fmt.Errorf("failed to parse query string: %v", err)
    }

    queries := repository.New(db)
    username := values.Get("username")

    if strings.TrimSpace(username) != "" {
        err = queries.DeleteHighscore(req.Context(), username)
        if err != nil {
            if err == sql.ErrNoRows {
                return handler.Response{
                    StatusCode: http.StatusNotFound,
                }, nil
            }
            return handler.Response{
                StatusCode: http.StatusInternalServerError,
            }, fmt.Errorf("failed to delete a highscore for username %s: %v", username, err)
        }
    }

    return handler.Response{
        StatusCode: http.StatusOK,
    }, nil
}

Now we can do faas-cli up and see the deployed functions. You can also check out the dashboard to get the endpoint names.

If you are getting internal server error 500, that means you are returning an error to the function handler and you can easily debug your server. For example, I am returning an error for invalid HTTP methods. I can easily see logs with this command

journalctl -t openfaas-fn:get-highscores -r --lines 20

The end

These are the endpoints we have created. Overall, I enjoyed how we can have a serverless developer experience without the need for any giant cloud service that is impossible to move around. Faasd is still a young but promising project for developers who don't want to deal with k8s infra complexity. Hope you enjoyed and learned something new. If you have any questions/issues, feel free to let me know. Take care!

[INFRA PART 1/2] Serverless Highscore Go API with Faasd and CockroachDB

Talha Altınel — Wed, 08 Sep 2021 18:17:21 +0000

The Series Intro

Hi everyone, in this series we will be creating serverless highscore REST API in Go and utilize the most advanced and bleeding-edge open-source technologies such as Faasd(OpenFaaS engine) and CockroachDB(our cluster database). Keep in mind that we will actually need a server to do serverless computing :) (Plot Twist)

In this 1st part, we will be setting up the infrastructure side for Hetzner Cloud with Terraform then in the 2nd part we will develop/deploy our functions with help of faas-cli. Faasd is developed by OpenFaaS to make self-hosted serverless functions much easier to develop/deploy without any vendor lock-in giant cloud company or K8s requirement. We will also be using CockroachDB as a single node database for our cloud server instance. There are some requirements but keep in mind that Terraform and Hetzner Cloud are not mandatory requirements.

The Aim

The aim is to give everyone self-hosted basic serverless REST API understanding, the DevOps cycle of it and how to interact with the self-hosted distributed/resilient open-source database CockroachDB from serverless REST API.

The High-level Diagram

Requirements:

Non-windows based Terminal(Must have, could be WSL, Linux Terminal or Mac Terminal, just not Windows Powershell)
Faas-cli in your client computer (Must have, we will be using this for pushing our code to our cloud server instance)
CockroachDB (Must have, we will be using this as a database in our cloud server instance)
Docker in your client computer (Must have, we will be using this for docker container registry)
Terraform(optional)
Hetzner Cloud account and API access token(optional)
SSH key and key name created in Hetzner Cloud(optional)

Github Repository for setting up faasd server on Hetzner Cloud in the blink of an eye:
hetzner-terraform-faasd-repository

Why Hetzner Cloud?

At least 1GB is required to be safe. So the pricing for CX11 and 2 gigs of RAM is really nice. You are free to use any cloud you like but I suggest small cloud providers instead of giant cloud providers due to the simplicity. Other alternatives could be Vultr, Linode, DigitalOcean...

Why Faasd and CockroachDB?

Faasd is open-source serverless container technology which uses an actual physical server and manages your functions in AWS lambda fashion with so little cost and being extremely lightweight. If you need global scale geo-distributed serverless container technology, you can also port your serverless functions to OpenFaaS(uses k8s under the hood) at any time. Faasd gets rid of any vendor-lock in serverless X cloud technology and allows you to focus only on your code. You also get a great dashboard.

CockroachDB is a global scale geo-distributed open-source database. I will be using CockroachDB due to its simplicity and resilience to setup as a single node in our cloud server instance. If you need the managed production database, you can always switch to the free tier of cockroach cloud which gives you ultimate powers of geo-distributed feature and auto-TLS. Again, you also get a perfect dashboard.

Note: In your faasd server, running a local database with docker can confuse containerd runtime. It is a must to install only faasd server via faasd github repository's "./hack/install.sh" which will only install what is necessary such as containerd, faasd and faas-cli.

For demo purposes and purity, we won't be adding TLS to our server and database but I will leave links at the end. You should never run this in production without TLS certificates and you will at least need 3 nodes for your production CockroachDB cluster.

Intro

First of all, you need faas-cli in your client local machine. You should get the binary and set it to your path. If you are on linux or mac machine, moving the binary into "/usr/local/bin" will work. For windows machines, you need to set environment variables in Control Panel>System and Security>System>Advanced System Settings(single-binary-faas-cli)

For setting up faasd in your cloud server instance, easiest way to install faas-cli and faasd is to run commands below. Then visit the dashboard and login with your credentials on http://[[SERVER_IP]]:8080/ui/ Skip the manual commands below if you have Terraform.

git clone https://github.com/openfaas/faasd --depth=1
./faasd/hack/install.sh
sudo cat /var/lib/faasd/secrets/basic-auth-user; echo
sudo cat /var/lib/faasd/secrets/basic-auth-password; echo

Having Terraform and Hetzner Cloud is not a hard requirement. But if you are into Terraform and you have a Hetzner Cloud account, that's great, you can run Terraform to provision your infrastructure faster and get a verbose output from your CLI. Just make sure to set your Hetzner api token and Hetzner ssh key name in vars.tf file.

git clone https://github.com/MrWormHole/hetzner-terraform-faasd
cd hetzner-terraform-faasd
terraform init
terraform plan
terraform apply --auto-approve
terraform output --json

After you visit the URL and enter your username and password, you should be seeing the pretty OpenFaaS dashboard.

If you are stuck with a problem, make sure faas-cli installed properly in your client and server by checking faas-cli version. You should also check if faasd.service is running on systemd.

systemctl status faasd.service

After you SSH into your cloud instance, you can setup your single node CockroachDB. And check out the dashboard and the databases at http://[[SERVER_IP]]:7070.

curl https://binaries.cockroachdb.com/cockroach-v21.1.7.linux-amd64.tgz | tar -xz; sudo cp -i cockroach-v21.1.7.linux-amd64/cockroach /usr/local/bin/

cockroach start-single-node \
--insecure \
--listen-addr=0.0.0.0:26257 \
--http-addr=0.0.0.0:7070 \
--background \
--accept-sql-without-tls

Before we close cockroachDB, let's create our schema and sample records.

cockroach sql --insecure --host=localhost:26257
CREATE DATABASE highscore_db;
USE highscore_db;
SHOW TABLES;
CREATE TABLE highscores (
  id BIGSERIAL PRIMARY KEY,
  username TEXT NOT NULL UNIQUE,
  score BIGINT NOT NULL
);
INSERT INTO highscores(username, score) VALUES ('SCORPION', 100);
INSERT INTO highscores(username, score) VALUES ('SUBZERO', 100);
INSERT INTO highscores(username, score) VALUES ('KITANA', 300);
INSERT INTO highscores(username, score) VALUES ('MILEENA', 400);
SELECT * FROM highscores;

To quit, we can use this

cockroach quit --insecure --host=localhost:26257

This sums up this infrastructure part. Let's move to the actual development

"it is actually so easy after you did it once" -The ancient old devops guy who died during the renewal of a manual TLS certificate

The end

How to secure faasd server with caddy. Link -> tls-caddy-faasd-server
How to secure CockroachDB as a whole cluster. Link -> tls-cockroachdb

The references

Faasd in-depth look -> faasd-book
CockroachDB in-depth look -> cockroachdb-university

Localstack with Terraform and Docker for running AWS locally

Talha Altınel — Sat, 03 Jul 2021 22:00:01 +0000

Hello everyone, in this post I will be demonstrating how you can run localstack with Terraform and Docker and give you a proof of concept go application so you can tweak it according to your logic and follow anything you want to do such as integration/system tests for AWS services in your own CI/CD or localhost.

Github Repository for PoC(proof of concept):
hotdog-PoC-repository

Requirements:

Docker
docker-compose
Terraform
Go
aws CLI
A bit of lambda, dynamodb and kinesis knowledge

Intro

Localstack is a testing/mocking framework for developing Cloud applications locally. Where in theory, you can stick any AWS service and emulate them in localhost without ever needing the real AWS account.
Localstack’s primary goal to make integration/system testing less painful for developers.

What was built?

I built an imaginary hotdog food chain! (Note: No dogs were harmed in this process). Essentially PoC logic was I had 1 dogs dynamodb table which consist a dog model with 3 attributes ID, name, isAlive and isEaten. Then I had 3 lambdas dogCatcher, dogProcessor and hotDogDespatcher. dog catcher's responsibility is to get alive dogs via external API requests(I generated data for simplicity) with unique IDs and different names. Dog processor's responsibility is to kill the dogs and persist the data that was sent from dog catcher. Hot dog despatcher's responsibility is to give processed dogs(hot dogs) to people and observe which ones were eaten via external API requests(I assumed hot dogs get eaten if their name has case-insensitive "e" or "a" letter)

Aside from lambdas, I had 3 kinesis streams and 3 kinesis triggers in order to make lambdas talk to each other. The named kinesis streams is as follows; caughtDogs, hotDogs, eatenHotDogs.

Starting Localstack docker container with docker-compose

version: '3.8'

services:
    localstack:
        container_name: "localstack_main"
        image: localstack/localstack:latest
        environment: 
            - SERVICES=dynamodb,lambda,kinesis
            - LAMBDA_EXECUTOR=docker_reuse
            - DOCKER_HOST=unix:///var/run/docker.sock
            - DEFAULT_REGION=ap-southeast-2
            - DEBUG=1
            - DATA_DIR=/tmp/localstack/data
            - PORT_WEB_UI=8080
            - LAMBDA_DOCKER_NETWORK=localstack-tutorial
            - KINESIS_PROVIDER=kinesalite
        ports:
            - "53:53"
            - "53:53/udp"
            - "443:443"
            - "4566:4566"
            - "4571:4571"
            - "8080:8080"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - localstack_data:/tmp/localstack/data
        networks:
            default:

volumes:
    localstack_data:
networks:
    default:
        external:
            name: localstack-tutorial

docker-compose up -d --build

Bootstrapping our infra with Terraform

provider "aws" {
  region                      = "ap-southeast-2"
  access_key                  = "fake"
  secret_key                  = "fake"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    dynamodb = "http://localhost:4566"
    lambda   = "http://localhost:4566"
    kinesis  = "http://localhost:4566"
  }
}

// DYNAMODB TABLES
resource "aws_dynamodb_table" "dogs" {
  name           = "dogs"
  read_capacity  = "20"
  write_capacity = "20"
  hash_key       = "ID"

  attribute {
    name = "ID"
    type = "S"
  }
}

// KINESIS STREAMS
resource "aws_kinesis_stream" "caught_dogs_stream" {
  name = "caughtDogs"
  shard_count = 1
  retention_period = 30

  shard_level_metrics = [
    "IncomingBytes",
    "OutgoingBytes",
  ]
}

resource "aws_kinesis_stream" "hot_dogs_stream" {
  name = "hotDogs"
  shard_count = 1
  retention_period = 30

  shard_level_metrics = [
    "IncomingBytes",
    "OutgoingBytes",
  ]
}

resource "aws_kinesis_stream" "eaten_hot_dogs_stream" {
  name="eatenHotDogs"
  shard_count = 1
  retention_period = 30

  shard_level_metrics = [
    "IncomingBytes",
    "OutgoingBytes",
  ]
}

// LAMBDA FUNCTIONS
resource "aws_lambda_function" "dog_catcher_lambda" {
  function_name = "dogCatcher"
  filename      = "dogCatcher.zip"
  handler       = "main"
  role          = "fake_role"
  runtime       = "go1.x"
  timeout       = 5
  memory_size   = 128
}

resource "aws_lambda_function" "dog_processor_lambda" {
  function_name = "dogProcessor"
  filename      = "dogProcessor.zip"
  handler       = "main"
  role          = "fake_role"
  runtime       = "go1.x"
  timeout       = 5
  memory_size   = 128
}

resource "aws_lambda_function" "hot_dog_despatcher_lambda" {
  function_name = "hotDogDespatcher"
  filename      = "hotDogDespatcher.zip"
  handler       = "main"
  role          = "fake_role"
  runtime       = "go1.x"
  timeout       = 5
  memory_size   = 128
}

// LAMBDA TRIGGERS
resource "aws_lambda_event_source_mapping" "dog_processor_trigger" {
  event_source_arn              = aws_kinesis_stream.caught_dogs_stream.arn
  function_name                 = "dogProcessor"
  batch_size                    = 1
  starting_position             = "LATEST"
  enabled                       = true
  maximum_record_age_in_seconds = 604800
}

resource "aws_lambda_event_source_mapping" "dog_processor_trigger_2" {
  event_source_arn              = aws_kinesis_stream.eaten_hot_dogs_stream.arn
  function_name                 = "dogProcessor"
  batch_size                    = 1
  starting_position             = "LATEST"
  enabled                       = true
  maximum_record_age_in_seconds = 604800
}

resource "aws_lambda_event_source_mapping" "hot_dog_despatcher_trigger" {
  event_source_arn = aws_kinesis_stream.hot_dogs_stream.arn
  function_name = "hotDogDespatcher"
  batch_size = 1
  starting_position = "LATEST"
  enabled = true
  maximum_record_age_in_seconds = 604800
}

./zip-it.sh
terraform init
terraform plan
terraform apply --auto-approve

Checking with aws CLI if everything is setup correctly

To see if everything was working correctly, I invoke dogCatcher and check out the dynamodb table;

aws lambda invoke --function-name dogCatcher --endpoint-url=http://localhost:4566 --payload '{"quantity": 2}' output.txt

aws dynamodb scan --endpoint-url http://localhost:4566 --table-name dogs

Result

I had pretty much great experience with Localstack. I think even though Localstack is quite new, it seems like it can be used for learning AWS SDKs as a developer without actually using live AWS services and getting billed for it. This can also speed up developer's integration tests(along with CI/CD) and debugging processes if configured properly because there are many services Localstack provides and I have only configured and used 3 of them here. This also saves lots of costs for any companies.

Also don't forget to check out Localstack's slack channel, they are really helpful for any issues you run into and follow me on Twitter for further questions!
localstack-slack
my-twitter