Forem: Sergei Gannochenko

Converting materialized paths into a tree with generics: a Golang kata

Sergei Gannochenko — Tue, 16 May 2023 21:15:28 +0000

I have finally switched over to go1.19, so it's time to start making use of generics. So here comes my next Golang kata.

Many languages generics use the <P> notation. The creators of Go decided to make things differently and use an alternative [P] notation instead. In my perspective,
it made code a bit harder to comprehend, because such a notation blends in with slice declarations.

I am going to solve a task of converting materialized path into a nested in-memory tree structure with numeric payload on leafs.

Structures

A tree node has children of the recursive type, a name, and also a generalized payload.

package tree

type PayloadType interface {
    int32 | int64
}

type Node[P PayloadType] struct {
    Name    string      `json:"name"`
    Nodes   *[]*Node[P] `json:"nodes"`
    Payload P           `json:"payload"`
}

So here the P generic has a restriction, and can only be of type int32 or int64.

Also, the Nodes field is a pointer to an array, which is, in its turn, contains pointers as well.

The implementation

The logic itself is kept in a separate structure with additional methods for inserting a path and printing the tree out into JSON.

package tree

import (
    "encoding/json"
    "strings"

    "tree/internal/domain/tree"
)

type Tree[P tree.PayloadType] struct {
    Root *tree.Node[P]
    Refs map[string]*tree.Node[P]
}

func New[P tree.PayloadType]() *Tree[P] {
    items := make([]*tree.Node[P], 0)
    rootNode := &tree.Node[P]{
        Nodes: &items,
    }
    refs := make(map[string]*tree.Node[P])
    refs[""] = rootNode

    return &Tree[P]{
        Root: rootNode,
        Refs: refs,
    }
}

func (t *Tree[P]) AddNode(path []string, payload P) {
    if len(path) > 0 {
        pathParts := make([]string, 0)

        for index, pathElement := range path {
            pathParts = append(pathParts, pathElement)
            parentPath := path[:len(pathParts)-1]
            pathKey := strings.Join(pathParts, ".")
            parentPathKey := strings.Join(parentPath, ".")

            if _, ok := t.Refs[pathKey]; !ok {
                items := make([]*tree.Node[P], 0)
                newNode := tree.Node[P]{
                    Name:  pathElement,
                    Nodes: &items,
                }

                if index == len(path)-1 {
                    // it's a leaf, adding payload there
                    newNode.Payload = payload
                }

                toNode := t.Refs[parentPathKey]
                toNodeNodesDeref := *toNode.Nodes
                toNodeNodesDeref = append(toNodeNodesDeref, &newNode)
                toNode.Nodes = &toNodeNodesDeref

                t.Refs[pathKey] = &newNode
            }
        }
    }
}

func (t *Tree[P]) ToJSON() string {
    jsonData, _ := json.MarshalIndent(t.Root, "", "  ")
    return string(jsonData)
}

So here I iterate over the path segment and add more and more tree branches. To prevent searching on every insert, I keep an
index of tree nodes, referenced by a materialized sub-path as a key.

The AddNode() function has time complexity of O(N), where N is the depth of the path provided.
The memory complexity is O(N+logM), because of that map of size M to store the keys.

This implementation assumes, that the materialized path list can be unordered, which gives the algorithm more flexibility.

Also, please note that when it comes to storing the nodes, I operate only with pointers everywhere. This is to make sure that I avoid data cloning.
I spent half an hour trying to figure why my tree was only 2 levels deep, and all that was because the append() function actually makes a copy of an object in case if a non-pointer value is passed. This mistake is easy to make if a person comes from the Javascript background, like myself.

Running the thing

The main file is fairly simple and contains some demo data to work with.

package main

import (
    "fmt"
    "strings"

    "tree/internal/lib/tree"
)

func main() {
    materialisedTree := map[string]int32{
        "shop.foo.bar":    1,
        "shop.foo.baz":    3,
        "package.foo.bar": 7,
        "package.foo.m":   10,
    }

    treeInst := tree.New[int32]()

    for path, weight := range materialisedTree {
        treeInst.AddNode(strings.Split(path, "."), weight)
    }

    fmt.Println(treeInst.ToJSON())
}

So, as we can see, generics is an amazing tool, that can make Go code really flexible. The code an be then shipped as a standalone library and re-used in the other projects.

By the way, there is a good library, and it's all about generics. Definitely worth looking into, as it may save a lot of lines of code.

Cover image by AI at Midjourney

Battle-tested technical interview plan for Lead Engineers

Sergei Gannochenko — Mon, 27 Jun 2022 18:59:06 +0000

A few days ago I have conducted my 16th technical interview. After a while, I have defined a set of rules and formalised some good practices.
Now I feel confident enough to share it.

Every time I keep an interview, I mostly don't improvise. I have a script I normally stick to, occasionally deviating from it if I see fit.

☝️ Note: this article is a work in progress. As I keep practicing, I might add a few things or remove something I find no longer relevant.

The preparations

Of course, certain homework must be done before jumping into the call.

Here is a list of typical things I do:

Get a CV of the candidate, read it carefully.
Try to figure how to pronounce the candidate's name correctly. If a company hires worldwide, it can be a real challenge sometimes.
Scan the CV for unusual things and prepare questions. For example:
- If the candidate's previous position was a lead (IC4), and now they are going for a senior (IC3), then it wouldn't be odd to ask about the motivation.
- Same counts for situations when they worked for 10 years in fullstack, but applied for the front-end position.
Prepare a blank page in Notion (or any other tool) with the following:
1. A pre-created link to the online technical assignment, if any.
2. As list of custom questions, if there is any.

The Script

A typical interview lasts roughly 60 minutes. Hence, for each step there is a time limit.

Greeting - 1 min
Brief introduction - 1 min
Tech talks and CV discussion - 28 min
Tech assignment - 25 min
Q&A from the candidate and wrap up - 5 min

Let me dive deeper into each step details.

Greeting

The Greeting is a formal part. First I say some standard things like "How are you? Welcome to the interview!".

Then, I say "All right, here is the plan ...", and then I name the steps mentioned above. After that I seamlessly move on to the introduction.

Brief introduction

This part is also formal. At this point, the candidate is not really interested in getting to know me, since their main concern is how to pass the interview.
This is why, in order to save time, I bring this part down to absolute minimum and only tell the following:

My name.
My occupation, what I do in the company.
When I joined.
What is the current project I am working on.

As soon as I'm done with this part, I ask if the candidate wants to take the stage. And that is when the real deal begins.

Tech talks and CV discussion

At this step I usually expect the candidate to talk about either their latest position, or something they are really proud of.

⚠️ The ideal situation would be this part turning into a pleasant conversation, rather than a questionnaire.

What could be even better is when I could actually learn something new from the candidate: a new library to explore, or a decent article to read.

Nevertheless, there is a set of questions I ask almost every time:

Can you give me an example of where you have a system built completely end-to-end?
How is your team structured and how are tasks assigned? What is your role in all that?
Can you give me an example of mentoring? Have you led any teams?
Are you involved in setting direction and goals for your team?

I may also have CV-specific questions, or I can ask to elaborate on something that caught my attention while the candidate was talking.

There is also a tech-specific part, where I do quick recap of the candidate's tech skills, as per how it is specified in their CV.
If we talk about React and web applications in general, I may examine their experience with performance issues and how those can be mitigated.
We may also talk about favourite libraries for state management, testing, styling, etc.

The most important part here is to ask questions on purpose, to screen for different aspects, traits and gotchas of the candidate.
For instance, when I ask about native DOM or vanilla JS, I try to figure whether the candidate is used to work only with certain frameworks, or there is something more to it.

Meanwhile I actively write the notes down. I ask to give me a few seconds to finish writing, in case if I am too slow.

Tech assignment

Typically, the initial tech screening is more technology-focused.
Due to the fact that there is a time limit of 25 minutes, I usually offer something more or less close to real life.
I choose an exercise where only the basic skills are checked.
Brain teasing with an algorithm-based question is a topic for the other kind of interviews.

For example, if I test knowledge of React, I may ask to build a component, check how the candidate structures it, how they work with hooks, event handling, basic CSS.

⚠️ This is also the part when I expect the candidate to really drive the interview.

I ask to be verbose, so it helps me to understand where the train of thoughts is going.
This also gives me insights on how good collaborator the candidate really is.
If the candidate nevertheless stays silent, it may indicate they are in panic or simply not listening to what I say.
In both cases it is not a good sign.

I general, I expect the candidate to follow this framework:

Read the task carefully.
Ask additional questions and clarify the conditions. A small recap of the task is also appreciated.
Briefly explain the solution they are going for.
Code the solution, run it, make sure it really works.
Tell me how the solution can be improved.

Some people say that finishing all the tasks is not crucial, since the most important thing is the way the candidate drives the technical part.
Well, due to the fact that I am asking for the absolute basics here, I really expect everything to be completed within the time frame.
If the candidate is unable to nail it down and fails to cope with stress, then I declare it as red flag.

Other possible red/yellow flags:

The candidate is not listening to me when I ask them to stop and re-evaluate the chosen approach.
The answer is really unstructured. For example, they jump between different parts of the code.
The candidate constantly gets lost between == and === operators.
They don't care much about variable naming or indentation.
The candidate stops typing, then a short pause happens, and then suddenly the candidate speeds up again. It may indicate them googling at the background.

If the candidate is failing, but still looking really motivated, I sometimes can go overtime.

Occasionally I show professional courtesy and help them to get the solution to a certain point.
By doing so I let them learn something from their failure.

Q&A from the candidate and wrap up

At this point the candidate is either devastated with the failure, or encouraged with the way the interview is going so far.

Here I expect some questions to be asked by the candidate. It does not matter whether those are just placeholder questions like
"Tell me about the company's culture", or anything more sophisticated, but there is got to be something.

Should the candidate ask no questions, it automatically counts as a red flag. It either indicates them being in 10 hiring processes at the same moment, or they simply don't care much, or they were lazy to do their part of the job.

At the very end I tell the candidate the standard phrase: "Thank you for joining me, it was pleasure. I will make a summary of my notes and pass it over to the recruiter. They will contact you shortly." After that I wish a great day and disconnect.

The Report

After the interview is done, the results must be reported. I have a pre-defined report template for that too:

The final verdict: yes or no.
Does the candidate have what it takes to functionally do the job? Here we specify if the candidate has good spoken English, a mindset of an engineer, required technical skills.
Will the candidate be successful in our environment? Write down all strong sides of the candidate, and how they are aligned with the company values.
Are there any red/yellow flags? Here goes a list of concerns about this candidate.
Technical assignment. I provide a link to the tech assignment, if any, and write the observations down.

All of it is not just about the numbers though. Sometimes the candidate does not posses the required skills for this particular role, but could be offered a similar position one level below.

Final advises

I DO NOT hire "maybe-s". My experience tells me it is way cheaper for everyone to stop the process earlier if something doesn't feel right, rather than to hire a wrong engineer and then try to fix them, or ask to leave. There ought to be positive vibes between me and the person.
I try to prevent getting biased by the candidate's appearance, tone of their voice or something else like that. The person could be in pain at the moment, or his favourite guinea pig might have just died. You'll never know.

Well, I hope this article was helpful and gave you, my dear reader, some food for thoughts!

Cover image by Stitch at Flickr

The meme is made using a frame from The Morning Show

Baremetal Kubernetes with kubeadm in 10 minutes? Easy!

Sergei Gannochenko — Wed, 25 Nov 2020 23:20:47 +0000

A while ago, being on sick leave, I challenged myself with setting up a baremetal K8s cluster. I chose Kubeadm back then, and so now I am sharing the results.

Yeh, maybe you already have the following question popped up in your mind:

Well, you may want to consider a bare-metal cluster, because

👉 you are setting up an internal infrastructure for your company, and using cloud providers is not an option,

👉 you wish to receive close-to-production experience with you own pet K8s,

👉 you are well-aware of Minikube, MicroK8s and others, but always wanted to have a real multi-node cluster out there,

👉 you don't want to pay for a pre-configured AKS/GKE/EKS/whatever solution just yet,

👉 you want to set everything up from scratch, at least once, out of curiosity,

👉 you have free time :)

Why making another article while there is plenty of them already? Well because most of the material is obviously not for beginners.
Sometimes the material feels fragmented, ripped ouf of the context of something bigger or simply not covering the entire process.

What you should understand just before we continue.

☝️ It still won't be a production-ready environment for mature projects! 🚨🚨🚨

☝️ However, with extra hours dedicated, the cluster can become production-ready.

☝️ But even so, it will be good enough for hobby projects, proof of concepts, learning and exploration.

☝️ You will not receive any GUI control panel, only raw kubectl.

Resonates with your needs? Then let's do it!

Nodes

Kubernetes is a cluster, it means that typically more than one machine is involved.
I got for myself two instances of VDS (VPS), with 2 Cores and 4 GB of RAM each (those are the minimum requirements), Centos7 on-board. It was a cheap russian hosting with a data center in Moscow and unlimited bandwidth.
It works for me just fine, but you can pick AWS's EC2, DigitalOcean or Vultr, or any other provider.

You may as well get two virtual machines with VirtualBox and Vagrant, but it will not give you the same experience.

X.X.X.X - IP address of Node A
Y.Y.Y.Y - IP address of Node B

Just as a part of pre-flight preparations, I run the following commands to make the whole process a bit easier.

On my machine:

printf "\nX.X.X.X k8s-master\nY.Y.Y.Y k8s-node01\n" | sudo tee -a /etc/hosts;
scp ~/.ssh/id_rsa.pub root@X.X.X.X:~
scp ~/.ssh/id_rsa.pub root@Y.Y.Y.Y:~
~~~{% endraw %}

On each node under {% raw %}`root`{% endraw %}:{% raw %}

~~~bash
yum update

# create a non-root user
useradd admin

# enable sudo
usermod -aG wheel admin
passwd admin
passwd -l admin

# enable key-based sign-in
cd /home/admin/
mkdir ./.ssh
chmod 711 ./.ssh
cat ~/id_rsa.pub >> ./.ssh/authorized_keys
chmod 600 ./.ssh/authorized_keys
chown -R admin:admin ./.ssh
~~~{% endraw %}

You may also want to disable password-based SSH authentication, disable root login and switch SSH to a different port to ward off brute-forcers.

Now I can sign-in the nodes by typing {% raw %}`ssh admin@k8s-node01`{% endraw %} without password and able to run {% raw %}`sudo`{% endraw %}.

## Common setup

Kubernetes logically consists of two parts: {% raw %}`Control plane`{% endraw %} and {% raw %}`Compute`{% endraw %}.

* {% raw %}`Control plane`{% endraw %} operates the cluster, and it is hosted on so-called {% raw %}`master node(s)`{% endraw %}.
* {% raw %}`Compute`{% endraw %} runs containerized applications and is hosted on {% raw %}`worker nodes`{% endraw %}.

Therefore, I choose {% raw %}`Node A`{% endraw %} to be the {% raw %}`master node`{% endraw %}, whereas {% raw %}`Node B`{% endraw %} - {% raw %}`worker node`{% endraw %}.

The following set of commands I need to run <ins>on each node</ins>:

➡️ Allow nodes to address each other by name:{% raw %}

~~~bash
printf "\nX.X.X.X master\nY.Y.Y.Y node01\n" | sudo tee -a /etc/hosts;
~~~{% endraw %}

Make sure nodes can actually talk to each other:{% raw %}

~~~bash
ping master
ping node01
~~~{% endraw %}

➡️ K8s does not play nicely with SELinux, switching it off:{% raw %}

~~~bash
sudo setenforce 0
sudo sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
~~~{% endraw %}

➡️ K8s uses {% raw %}`br_netfilter`{% endraw %} kernel module for it's internal networking. So I check if module even exists, load it, and enable on-boot load:{% raw %} 

~~~bash
sudo modprobe br_netfilter
sudo echo "br_netfilter" | sudo tee -a /etc/modules-load.d/br_netfilter.conf
~~~

➡️ Tell the `bridge module` to use `iptables`:

~~~bash
echo '1' | sudo tee -a /proc/sys/net/bridge/bridge-nf-call-iptables
printf "\nnet.bridge.bridge-nf-call-iptables = 1\nnet.ipv4.ip_forward=1\n" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
~~~

➡️ Disable firewalld 👻 😱

Some out-of-the-box firewall rules are set in a way it paralyzes cluster networking. In production it totally makes sense to find what causes that and fix the issue. Since I am building a non-production environment, I don't bother investing time. So I just disable {% raw %}`firewalld`{% endraw %} and by doing so I wipe out all firewall rules.{% raw %}

~~~bash
sudo systemctl stop firewalld
sudo systemctl disable firewalld
sudo systemctl mask --now firewalld

# making sure there is not a single rule left, and the policy is "ACCEPT" on every chain
sudo iptables -L -v -n
~~~{% endraw %}

➡️ K8s requires the {% raw %}`swap`{% endraw %} partition to be disabled for good:{% raw %}

~~~bash
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab;
~~~{% endraw %}

➡️ Install and configure {% raw %}`Docker`{% endraw %}{% raw %}

~~~bash
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install -y docker-ce

# changing docker cgroupdriver to "systemd" 
sudo mkdir /etc/docker
sudo tee /etc/docker/daemon.json > /dev/null <<'EOF'
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
~~~{% endraw %}

➡️ Install {% raw %}`Kubernetes`{% endraw %} and {% raw %}`kubeadm`{% endraw %}{% raw %}

~~~bash
sudo tee /etc/yum.repos.d/kubernetes.repo > /dev/null <<'EOF'
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

sudo yum install -y kubelet kubeadm kubectl
~~~{% endraw %}

➡️ Launch {% raw %}`Docker`{% endraw %} daemon{% raw %}

~~~bash
sudo systemctl start docker
sudo systemctl enable docker
~~~{% endraw %}

➡️ Launch {% raw %}`Kubelet`{% endraw %} daemon

Kubeled daemon in charge of doing all K8s-related stuff on this particular node.{% raw %}

~~~bash
sudo systemctl start kubelet
sudo systemctl enable kubelet
~~~{% endraw %}

Allright, hopefully no errors there, and we can proceed to more neat stuff.

## Master-specific setup

The commands below should be executed <ins>on the master node only</ins>.

➡️ Let {% raw %}`kubeadm`{% endraw %} do all the heavy lifting:{% raw %}
~~~bash
sudo kubeadm init --apiserver-advertise-address=X.X.X.X --pod-network-cidr=10.244.0.0/16 --apiserver-cert-extra-sans=localhost,127.0.0.1
~~~{% endraw %}

If everything went well, I just need to do a few more extra steps.

➡️ Create a configuration file for the current user (admin){% raw %}

~~~bash
mkdir -p ${HOME}/.kube
sudo cp -i /etc/kubernetes/admin.conf ${HOME}/.kube/config
sudo chown $(id -u):$(id -g) ${HOME}/.kube/config
~~~{% endraw %}

➡️ Deploy a pod network

So we have containers. A {% raw %}`container`{% endraw %} is a containerized application written in any language (Node, Java, Go, ...) and running inside of the cluster.
A {% raw %}`pod`{% endraw %} is a group of {% raw %}`containers`{% endraw %}, which share common resources.
{% raw %}`Pods`{% endraw %} are exposed to the outer world via {% raw %}`services`{% endraw %}.

Yeh, sounds complicated. But this is what makes K8s quite abstracted and therefore unbiased and flexible.

So the {% raw %}`pod network`{% endraw %} then is an implementation of K8s networking model: it enables pods talking to each other, to services, and via services - to the outer world.

There are numerous implementations out there, but I will stick to the kinda default one, which is called {% raw %}`flannel`{% endraw %}. It is basic and proven to be working.{% raw %}

~~~bash
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
~~~{% endraw %}

## Joining the nodes

Hooray! Technically nothing stops us from joining the nodes together now.

On the master node I create a new token for joining the cluster:{% raw %}

~~~bash
sudo kubeadm token create
~~~{% endraw %}

This will get me the token of format {% raw %}`xxxxxx.xxxxxxxxxxxxxxxx`{% endraw %}.

## Worker-specific setup

The commands below should be executed <ins>on the worker node(s) only</ins>.

➡️ Add a worker node to the cluster

Member that token I made? I need that one to execute the {% raw %}`join`{% endraw %} command <ins>on each worker node</ins>:{% raw %}

~~~bash
sudo kubeadm join X.X.X.X:6443 --token xxxxxx.xxxxxxxxxxxxxxxx --discovery-token-unsafe-skip-ca-verification
~~~{% endraw %}

From this moment on, there will be no need to do something else on the {% raw %}`worker`{% endraw %} node, so I may end that ssh session.

## Checking the status of the cluster

I can now go back to the {% raw %}`master`{% endraw %} node and check how the cluster is doing:{% raw %}

~~~bash
kubectl get nodes
~~~{% endraw %}

If I see something like this...{% raw %}

~~~text
NAME     STATUS   ROLES    AGE    VERSION
node01   Ready    <none>   4h7m   v1.19.3
master   Ready    master   45h    v1.19.3
~~~{% endraw %}

... that means: "Congratulations 🎉🎉🎉 You and I just got our own K8s clusters up and running!" 

Most of the tutorials usually end here, or they maybe show how to expose an Apache container via a {% raw %}`NodePort`{% endraw %}-type service (make a container accessible on the worker node directly via a random port greater than 1023). This obviously is not commonly-used IRL, as it defeats one of the key advantages of K8s. So instead let's go a bit further and setup `ingress`. 

➡️ Ingress

Basically, ingress tells the cluster *&laquo;Hey, for this given domain, path and protocol route all requests to the service A. For other domain, path and protocol - to the service B (and so on)&raquo;*. So ingress works as a router for the incoming requests.

Sounds quite powerful, doesn't it? I am gonna use {% raw %}`ingress-nginx`{% endraw %}, the default kubernetes ingress, as it is pretty simple and easily configurable:{% raw %}

~~~bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.40.2/deploy/static/provider/baremetal/deploy.yaml
~~~{% endraw %}

Let's check how ingress is doing:

~~~bash
kubectl get pods -n ingress-nginx
~~~

In a few minutes, the status should look somewhat like this:

~~~text
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-7dhcj        0/1     Completed   0          22m
ingress-nginx-admission-patch-kcsvt         0/1     Completed   0          22m
ingress-nginx-controller-785557f9c9-7ldcl   1/1     Running     0          22m
~~~

➡️ Load balancer

What ingress does not know, obviously, is how exactly the cluster is connected to the outer world. This task is solved by a `load balancer` that typically works outside of the cluster. Every cloud provider implements it's own one, and their load balancers vary from one to another. Since I don't have any cloud provider here, I need a substitution.

So there is a project called `Metallb`, which I am going to leverage for this matter. Please find which version is [the latest](https://metallb.universe.tf/). Also, sometimes they do backward-incompatible installation instruction updates, so if something does not work, feel free to [check their docs out](https://metallb.universe.tf/installation/)!

~~~bash
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.4/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.4/manifests/metallb.yaml
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
~~~

I also need to provide a config file, otherwise the balancer stays dormant.

~~~bash
tee /tmp/metallb-conf.yml > /dev/null <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
    namespace: metallb-system
    name: config
data:
    config: |
        address-pools:
        - name: default
          protocol: layer2
          addresses:
          - Y.Y.Y.Y
EOF
kubectl apply -f /tmp/metallb-conf.yml;
rm /tmp/metallb-conf.yml;
~~~

To see how the balancer is doing:

~~~bash
kubectl get pods -n metallb-system
~~~

It should look like this:

~~~text
NAME                         READY   STATUS    RESTARTS   AGE
controller-8687cdc65-h6d5m   1/1     Running   0          14s
speaker-b4p95                1/1     Running   0          14s
speaker-xhm72                1/1     Running   0          14s
~~~

***

Okay, so this is pretty it. I have set the cluster up, and it is working. In the next article (which is coming soon) I will deploy an infrastructure using Terraform, with actual domains and SSL support! Stay tuned.

## Troubleshooting

### 👉🐛 When joining a new node, the process hangs at the "[preflight] Running pre-flight checks" stage forever.

Endless process or timeout error sometimes lie in the I/O plane. Please make sure that nodes can ping each other by IP and the `master` node has only kubelet-produced rules in `iptables`.

### 👉🐛 Pods are stuck at "ContainerCreating" step

This problem may have many causes. In general, to see the events of a pod and where it stuck, a `describe pods` command is used:

~~~bash
kubectl describe pods -n ingress-nginx
~~~

If something goes totally pear-shaped, you may consider running the following piece of commands <ins>on each node</ins> to revert the cluster to pre `kubectl init (join)` state:

~~~bash
sudo kubeadm reset -f
sudo rm -rf /etc/cni/net.d

sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT
sudo iptables -t nat -F
sudo iptables -t mangle -F
sudo iptables -F
sudo iptables -X

sudo systemctl restart docker
sudo systemctl restart kubelet

rm -rf ${HOME}/.kube
~~~

## Useful links

* [kubeadm](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) - K8s's official bootstrapping tool
* [kops](https://kops.sigs.k8s.io/) - K8s installer on AWS/GCE/OpenStack/Digital Ocean/Spot Ocean
* [kubespray](https://kubespray.io/) - another installer
* [k8s cheatsheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/)
* [list of pod networks in the alphabetical order](https://kubernetes.io/docs/concepts/cluster-administration/networking/)
* [list of ingress controllers](https://medium.com/flant-com/comparing-ingress-controllers-for-kubernetes-9b397483b46b)

---

Cover image by Toni Vuohelainen at [Flickr](https://www.flickr.com/photos/tonivuohelainen/)

You probably don't need extra tools to connect Google Forms to your Slack channel

Sergei Gannochenko — Fri, 09 Oct 2020 10:13:14 +0000

One day came, and I have faced the challenge of how to post a new message on Slack when someone submits a Google Form. Well, not a challenge really.
In the beginning, I thought to just plug in Zapier or any other tool like that.

But then I asked myself "I am a developer, am I not? Could it be done by using glue, duct tape and a hammer, like in old times?" And in fact, it could be.

Step 1: Create a form

So yeah, here goes my form.

I know, I know, it is better to have a fully-featured bug tracker here, but I wanted to keep it simple for now.
So the idea would be: whenever someone submits the form, I want to receive a message in Slack right away.

Step 2: A little bit of engineering

In the form editor there is a link called Script editor:

If I press it, a simple IDE will appear.

Meet Google Script!

GS is sort of like VBA (if you know what it is 🤪), but with ECMAScript syntax. I got the impression that GS was initially built to automate Google Sheets, but for other types of documents it also can do a trick or two.

Replace the content of the tab with the following listing:

function onFormSumbit(event) {
    Logger.log("authMode=%s, source.getId()=%s", event.authMode, event.source.getId());

    var items = event.response.getItemResponses();

    for (var i in items) {
        const item = items[i];

        Logger.log("%s = %s", item.getItem().getTitle(), item.getResponse());
    }
}
~~~{% endraw %}

and press the {% raw %}`Save`{% endraw %} button.

For now, this function is just a function. I can try to run the script by pressing the {% raw %}`Run`{% endraw %} button, but I will only get an error message saying that {% raw %}`event`{% endraw %} is undefined.
I need to bind the function to the form submission event.

To do that, I go to {% raw %}`Edit`{% endraw %} ➡️ {% raw %}`Current project's triggers`{% endraw %} menu item. In the list of triggers I hit {% raw %}`+ Add trigger`{% endraw %} button located at the bottom.

![Trigger list](https://gannochenko.dev/static/30864db1fb72acf32f0549d8a6cd271e/e3189/trigger_list.png)

A form will pop up then. I check if the function name is right, and also change {% raw %}`Select event type`{% endraw %} to {% raw %}`On form submit`{% endraw %}.

![Add new trigger](https://gannochenko.dev/static/1e24d071ea10198ea4002f366612ac17/e3189/add_new_trigger.png)

After I hit {% raw %}`Save`{% endraw %}, I will be asked to grant permissions. Usual deal. In the list of triggers I can now see my trigger that was never executed before.

I go back to my form and fill it up, then submit:

![Form filled](https://gannochenko.dev/static/d53603ddc712343b1fc00869780008f5/e3189/form_filled.png)

In the list of triggers I can see that it was just executed a few moments ago:

![Trigger executed](https://gannochenko.dev/static/7138337d1d816a668333e109a867f8cd/e3189/trigger_executed.png)

If I proceed to {% raw %}`My executions`{% endraw %} section, I can get the output of the script:

![My executions](https://gannochenko.dev/static/9c5c0aca3b03f4991ec37633d6f0dd54/e3189/my_executions.png)

Wow, it works! What a twist!

## Step 3: Wire up Slack

Before I go further, I need to set {% raw %}`Slack`{% endraw %} up a bit. So, {% raw %}`Slack`{% endraw %} allows posting a message to a channel, by providing a secret URL ({% raw %}`Webhook`{% endraw %}). To create such {% raw %}`webhook`{% endraw %} I need to create a {% raw %}`Slack`{% endraw %} application first.
So I go to [Slack API panel](https://api.slack.com/apps/) and hit {% raw %}`Create an App`. Here I need to specify the app name and the workspace.

When the app is ready, I go to {% raw %}`Add features and functionality`{% endraw %} and then click on {% raw %}`Incoming Webhooks`{% endraw %}.

![Icoming Webhooks](https://gannochenko.dev/static/7999e5aa312341b1c2f4146c07cea9c7/e3189/incoming_webhook.png)

If {% raw %}`Incoming Webhooks`{% endraw %} were not activated before, I activate them now. Then I scroll down to the list of hooks and hit the {% raw %}`Add new Webhook to Workspace`{% endraw %} button.
I need to specify the channel I am going to post the messages on. When all of this is done, I can see my hook in the list, and copy it's URL from there.

The URL should have a format of `https://hooks.slack.com/services/XXXXXXXX/YYYYYYY/ZZZZZZZZZZZZZZZZ`.

## Step 4: The code

Allright!! Time to write some cool stuff here.

I go back to my `GS` code editor and change the function like this:

~~~js
function onFormSumbit(event) {
    var items = event.response.getItemResponses();

    var summary = '';
    var severity = '';

    for (var i in items) {
        var item = items[i];
        var title = item.getItem().getTitle();
        var value = item.getResponse();

        if (title === 'Tell us what happened') {
            summary = value;
        } else if (title === 'How severe the problem is?') {
            severity = value;
        }
    }

    if (summary) {
        postMessage(summary, severity);
    }
}

function postMessage(summary, severity) {
    var message = {
        "channel" : "react-calendar",
        "text" : "Brace for impact! Another ticket is there: "+severity,
        "attachments": [{
            "text": ":fire: "+summary,
            "footer": "<https://docs.google.com/spreadsheets/d/XXXXXXXX/edit|See all issues>",
            "mrkdwn_in": ["text"]
        }]
    }

    var options = {
        "method" : "post",
        "contentType" : "application/json",
        "payload" : JSON.stringify(message)
    };

    return UrlFetchApp.fetch("https://hooks.slack.com/services/XXXXXXXX/YYYYYYY/ZZZZZZZZZZZZZZZZ", options);
}
~~~

⚠️⚠️⚠️ Please replace my stub URL with your own one! There is also a link to a spreadsheet with all the submissions, and its URL is stubbed as well.

Well, it is not a very brilliant code though, and it most definitely can be improved. Later. For now, I save the code in the editor and submit the form once again.

Aaaaaand nothing works. In the output of the execution, there is an error `Exception: You do not have permission to call UrlFetchApp.fetch.`
Well, this happens because I created a trigger when there was no `UrlFetchApp.fetch()` call in the code. Since it needs elevated permission, my trigger does not work anymore.

I need:

1. to revoke [permissions](https://security.google.com/settings/security/permissions) of the script in my Google account,
2. to kill the trigger and re-create it again (Google may get a bit paranoid and warn me that I should not trust the code I wrote a couple of minutes ago. And heck, Google is wise in that one).

As soon as this is done, I try to re-submit the form one more time.

![Worked](https://gannochenko.dev/static/2fb7ac9ee265c68b8a1568b184c53ad4/e3189/worked.png)

Oh. My. Glob. It worked! Now I am not gonna miss another bug report ever!

---

## Conclusion

Sometimes instead of rushing into a third-party overkilling solution, it may be preferable to utilize your engineering skills. Of course, this is always a trade-off, but if you have time, then why not dare?

---

Cover image by Cesar Carlevarino Aragon at [Unsplash](https://unsplash.com/@carlevarino)

How I migrated from Redux and Saga to MobX and became a happier person

Sergei Gannochenko — Thu, 16 Jul 2020 19:26:50 +0000

For quite a while I was using Redux together with Redux Saga and was satisfied. Well, sort of, because I was constantly asking myself "Should the state management always be that complex?"
Redux was a great library for it's time, when the concept of a centralized state was not well-coined yet. Nevertheless, one day came, and I have decided it was time to move.

Requirements

In my applications I had:

one special reducer that keeps data of the currently authenticated user, feature flags and some other different stuff,
one reducer per page, combined all together with combineReducers,
one saga per page,
a way to observe and affect the state outside of the rendering context,
a way to keep routing history and state in sync.

So will see how I'll tackle all of this with MobX. But first, a short intro to MobX is required.

The State

According to MobX, a state is the heart of an application.

As a substitution for reducers I have decided to make several sub-states. So, there is a "root" state called State:

import { observable, computed, configure } from 'mobx';
import { Nullable } from '../../type';

// pre-configure MobX to have better experience
// see https://mobx.js.org/refguide/api.html
configure({ enforceActions: 'observed', computedRequiresReaction: true });

export class State {
    @observable public loading = false;
    @observable public error: Nullable<Error> = null;

    @computed get ready(): boolean {
        return this.loading || !this.error;
    }
}

The MobX state is not just a plain object. The state object has a prototype, which automatically enables the ultimate fun with all that OOP stuff: getters / setters, methods, incapsulation and so on. I am not a huge fan of OOP though, but occasionally it may be helpful.

You may have already noticed a couple of decorators back there.

@observable decorator tells MobX to observe this particular property for changes,
@computed decorator tells that there is a getter returning a value that is dependent on some observed properties and does not have side effects. This enables memoization.

Consuming the state

First thing first, I need to inject my state into components. React context and hooks are to my rescue here!

import React, { useContext } from 'react';

import { State } from './state';
import { Nullable } from '../../type';

type NullableState = Nullable<State>;

export type StatePropsType = {
    state: State;
};

export const StateContext = React.createContext<NullableState>(null);

export const StateProvider = StateContext.Provider;

export const useGlobalState = () => useContext<NullableState>(StateContext);

Then I modify my Providers component like this:

import React, { FunctionComponent } from 'react';

import { State } from '../../state/state';
import { StateProvider } from '../../state/context';

const state = new State();

export const Providers: FunctionComponent = ({ children }) => (
    <StateProvider value={state}>
        {children}
    </StateProvider>
);

And finally, everywhere I want this state consumed I just use useGlobalState hook:

import React, { useEffect, FunctionComponent } from 'react';

import { ApplicationProps } from './type';
import { useGlobalState } from '../state/context';

export const Application: FunctionComponent<ApplicationProps> = () => {
    const state = useGlobalState()!;

    return (
        <div>
            MobX is cool!
            <Loader state={state} />
        </div>
    );
};

Reacting to state changes

Having the state consumed does not automatically mean the component will react on it's changes. However, just like Redux has a connector module react-redux, there is a module called mobx-react.

import { observer } from 'mobx-react';

const Loader = observer(({ state }: StatePropsType) => <span>{state.ready ? '' : 'Loading...'}</span>);

A few important things here:

observer is a wrapper function that makes our component "sensitive" to state changes. This is an analog of connect() from 'react-redux'.
There is also an @observer decorator, but it works only with class components (which I use less and less frequently as time goes by).
I shall keep my observers as tiny as possible, to avoid re-rendering of different neighboring chunks of UI that do not care about the state at all.

Changing the state

There are different ways to get the state changed. Personally, I prefer doing it Redux-way: via actions. But actions in MobX is nothing like in Redux: I don't need to dispatch anything anywhere hopefully :) An action here is just a method of the State class.

Let's modify the State class by adding an action or two!

import { observable, computed, configure, action } from 'mobx';
import { Nullable } from '../../type';

// pre-configure MobX to have better experience
// see https://mobx.js.org/refguide/api.html
configure({ enforceActions: 'observed', computedRequiresReaction: true });

export class State {
    @observable public loading = false;
    @observable public error: Nullable<Error> = null;

    @computed get ready(): boolean {
        return this.loading || !this.error;
    }

    @action.bound
    public start() {
        this.error = null;
        this.loading = true;
    }

    @action.bound
    public stop(error?: Error) {
        if (error) {
            this.error = Error;
        }
        this.loading = false;
    }
}

Here we have an @action.bound decorator (bound means that I can safely use this inside).

Then nothing stops me from calling these two beautiful actions to change the state:

import React, { useEffect, FunctionComponent } from 'react';

import { ApplicationProps } from './type';
import { useGlobalState } from '../state/context';

export const Application: FunctionComponent<ApplicationProps> = () => {
    const state = useGlobalState()!;

    return (
        <div>
            MobX is cool!<br />
            <button onClick={() => state.start()}>Start</button>{'  '}<button onClick={() => state.stop()}>Start</button><br />
            <Loader state={state} />
        </div>
    );
};

Asynchronous actions

While @action.bound works nicely for sync actions, for the async ones things turn a bit more complicated.

It might seem so at the beginning, that I just need to add async / await to an action, and that will be enough. Well, yeh, technically it solves the case, but what if I need for something more reliable?
The problem with async / await is that I don't really have any control over the flow, and if an action takes longer than anticipated initially to execute, I might get into race condition troubles there.

So consider the following situation:

A user opens a page.
The data starts to load, the user is looking at the loading indicator meanwhile.
The user leaves the page.
It takes some time to load the data due to poor network connection.
The data arrives, the state gets updated even if the displayed page is not the same the data was obtained for.
???

Is there a way to tell our async action, that we are leaving the page and no longer interested in the data?

Yes.

This is where generators come out. The coolest thing about a generator yielding promises is that it forms an async process which can be interrupted in the middle, right between two yield operators. From the developers perspective, a generator looks like just a normal function.

So this is how I have implemented this.

Sub-states

I have decided to make a sub-state for each page of my application, just like I used to have separate reducers in Redux before. That is my home page for instance:

import React, { FunctionComponent, useEffect } from 'react';

import { Layout } from '../../components';
import { useGlobalState } from '../../state/context';

export const HomePage: FunctionComponent = () => {
    const state = useGlobalState()!;
    const { homePage } = state;

    useEffect(() => {
        homePage.onLoad();

        return () => homePage.onUnload();
    }, [homePage]);

    return (
        <Layout>
            Honey, I am home!
        </Layout>
    );
};

When I enter or leave the page, the onLoad() or onUnload() is called respectively.

And now the sub-state itself:

import { action, observable, flow, isFlowCancellationError } from 'mobx';
import { CancellablePromise } from 'mobx/lib/api/flow';
import { Nullable, ObjectLiteral } from '../../../type';

export class HomePageState {
    @observable ready = false;
    @observable loading = false;
    @observable error: Nullable<Error> = null;

    private queryLoad: Nullable<CancellablePromise<unknown>> = null;

    onLoad() {
        if (this.queryLoad) {
            // something is already being loaded for this page. Non-relevant, abort
            this.queryLoad.cancel();
        }

        // start new query
        this.queryLoad = this.startLoading();
        this.queryLoad.catch((error) => {
            if (!isFlowCancellationError(error)) {
                // the query was not cancelled, some other error popped up
                console.error(error);
            }
        });
    }

    onUnload() {
        if (this.queryLoad) {
            // we are leaving, stop whatever you are doing now
            this.queryLoad.cancel();
            this.queryLoad = null;
        }
        this.reset();
    }

    @action.bound
    reset(): void {
        this.ready = false;
        this.loading = false;
        this.error = null;
    }

    // that is our async process
    startLoading = flow(function* startLoading() {
        // @ts-ignore who cares?:)
        const self = this as HomePageState;

        self.loading = true;
        self.error = null;
        self.ready = false;

        // load something really heavy
        yield new Promise((resolve) => {
            setTimeout(resolve, 5000);
        });

        yield self.finishLoading();
    });

    @action.bound
    finishLoading(error?: Error): void {
        this.loading = false;

        if (error) {
            this.error = error;
        }

        this.ready = true;
    }
}

Okay, so flow is the cake here. Since it is a generator, it can be cancelled between yields. So cool.

Flow here acts as a direct analog of Redux Saga.

I inject a sub-state into the main State like this:

import { observable, computed, configure, action } from 'mobx';
import { Nullable } from '../../type';
import { HomePageState } from '../../pages/HomePage';

// pre-configure MobX to have better experience
// see https://mobx.js.org/refguide/api.html
configure({ enforceActions: 'observed', computedRequiresReaction: true });

export class State {
    @observable public loading = false;
    @observable public error: Nullable<Error> = null;

    public homePage = new HomePageState();

    @computed get ready(): boolean {
        return this.loading || !this.error;
    }

    @action.bound
    public start() {
        this.error = null;
        this.loading = true;
    }

    @action.bound
    public stop(error?: Error) {
        if (error) {
            this.error = Error;
        }
        this.loading = false;
    }
}

Conclusion

That is basically it! For me the codebase shrank significantly. All the cryptic reducers and the "plain object" state concept is now gone. This all brings me to the following outcome:

Pros of MobX

MobX is (almost) completely decoupled from the rendering cycle. For me it is a plus, since personally I don't like the do-everything-in-the-component approach. IMO, React should only know how to render the UI, and everything else should be delegated to other parties. Unfortunately, this can also manifest itself as a drawback (see below).
State, Reducer and Action entities are now united into one single entity called State. It just feels more natural. I can also treat the entire thing as a blackbox when needed.
All the complexity introduced by Redux is no longer on the picture (Phewww...)

Cons of MobX

Decorators work only with class components. If you need to turn a function component into an observer, you need to use alternative observer() syntax, which does not please the eye at all.
MobX is (almost) completely decoupled from React. It means that one day MobX may become ideologically incompatible with potentially upcoming React features.
The state changes are not discrete, and this fact makes the system more difficult to debug.
Therefore, No time travel debugging and undo ability

I have prepared a proof of concept repo to show the thing in action :)

Alternatives

If MobX does not suit you well, there are many-many alternatives out there. Just check them out.

Apollo link state
Overmind
Recoil by Facebook
Context API
Unstated-next
... and many others :)

I hope it was helpful. So, what do you think of MobX? Text me a message to discuss!

Header photo by Kin Li on Unsplash

How to make a code generator in 5 minutes (or less)

Sergei Gannochenko — Tue, 25 Feb 2020 07:16:58 +0000

Liquid syntax error: Unknown tag 'endraw'