Forem: Harsha Vaidya

Build a Raspberry Pi K8s Server For Your Side Projects

Harsha Vaidya — Wed, 10 Mar 2021 04:47:29 +0000

Have you ever thought of shipping that side project of yours but felt like it's not worth the money to spend on cloud costs?

This was my prime motivation to build this project: A hosting solution for all my side projects. So let me share the solution I've come up with.

The goals for this project are:

Accessible to anyone on the local network or wifi in my case.
Low maintenance.
Easy to deploy or make changes.

To achieve these you'll be needing:

A Raspberry Pi ( I've used the new 4B model ) with at least 4GB of memory and 4 cores in total.
At least 20 GB of free storage on your Pi.
Your local network credentials if you are using wifi.
A USB keyboard, Monitor with HDMI slot, a microHDMI cable.

Alright, once those are in place, let's start.

Install the Raspberry Pi Imager
Install Ubuntu 20.04.02 LTS (64-bit version) on your microSD card.
Edit the network config ( only if you are using wifi )
Place your SD card in the slot in the Pi. Connect the keyboard and Monitor to the Pi and connect the Pi to power.
Once it boots up it will ask for login and password. Use ubuntu for both and it will prompt you to change the password. Set a password then.
Reboot the server. sudo reboot
Install net-tools package. sudo apt-get install net-tools
Get the IP address of the server on your network. For this we can do ifconfig and find it out. If you have used wifi for this setup, look under the wlan0 header.
Setup SSH. On your computer run the command ssh-copy-id ubuntu@<ip_of_your_server> . It will prompt for the password. Supply it and you are done. Additionally update your ssh config file with a custom hostname for the pi for ease of use.
Now you can remotely access the server without needing a monitor or a keyboard connected to your Pi.

Now the setup is complete. To deploy your apps there are many ways. In this post I will be using MicroK8s

SSH into your Pi.
Enable cgroups on your Pi. Append cgroup_enable=memory cgroup_memory=1 to the end of /boot/firmware/cmdline.txt in the Pi.
Reboot. sudo reboot
Install microk8s. sudo snap install microk8s --classic
Add your current user to the microk8s group and gain access to the .kube caching directory:

sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube

Start the cluster. microk8s start
Check status. microk8s status
Enable DNS, Ingress for exposing your apps on your local network. microk8s enable dns ingress
Get the kubectl config for the cluster. microk8s config
Copy the kubectl config you got in the above step to your computer to enable access to the control plane from your computer. This eliminates the need to SSH into the server when you want to access the cluster.

Now our single-node cluster is setup and ready to be used. You can now package your projects as containers and deploy them to this setup.

Alternatively, if you don't know K8s or don't want to use it, you can still deploy your projects as simple docker containers or even as plain-old services and this setup would still work.

So, did we meet our goals?

We made something which we can access from our local network. ✅
The server would be fine if you keep in mind the resource limits it has and always connect it power and network. ✅
Since, I've used k8s deploying and making changes is a breeze. I also use Github actions for my projects which automate the building of docker images. So, effectively only one command (IDE takes care of git push 😉) I have to run to apply new changes to the deployed project. kubectl rollout restart deployment/<deployment_name> ✅

Happy hacking! Hope you learnt something new today 😄.

How to build a thread-safe Queue in Go?

Harsha Vaidya — Wed, 17 Feb 2021 16:30:21 +0000

A thread-safe queue is a queue which can be used in a multi-threaded environment without any loss in data. Before we build one, let us discuss more on why we need a thread-safe queue. Feel free to skip the explanation and head to the bottom of this blog to find the full implementation for the queue if you know the prerequisites 😄.

Why does one need a thread-safe queue?

Consider this go program with a queue implementation. This is the main function of that program. It creates a queue with size 1 and inserts an element into it. Then we spawn two go routines which remove the inserted element.

func main() {
    q := CreateQueue(1)
    q.Insert(0)

    // Start two go routines
    for i := 0; i < 2; i++ {
        // This go routine will try to call the Remove function of the queue.
        go func(q *Queue) {
            if removed, e := q.Remove(); e == nil {
                fmt.Printf("Removed %d\n", removed)
            } else {
                fmt.Printf("got error %v\n", e)
            }

        }(q)
    }
    time.Sleep(100 * time.Second)
}

Also look at the Remove function. I've added the time.Sleep function to show what happens when things take too long (1ns=1x10^-9s might not be too long for us but can't say the same about computers 😉) in a multi-threaded environment.

func (q *Queue) Remove() (int, error) {
    if len(q.q) > 0 {
        time.Sleep(1 * time.Nanosecond)
        item := q.q[0]
        q.q = q.q[1:]
        return item, nil
    }
    return 0, errors.New("Queue is empty")
}

In a single-threaded environment, no problem would occur and everything would work as expected i.e two Remove calls which would remove the element for the first call and return an error for the second call.

But for multi-threaded environments, something very interesting happens. If you were to run that program in the playground you would get a panic from Go. Something like this:

panic: runtime error: index out of range [0] with length 0

goroutine 6 [running]:
main.(*Queue).Remove(0xc00000c0a0, 0x0, 0x0, 0x0)
    /tmp/sandbox828372841/prog.go:33 +0xf9
main.main.func1(0xc00000c0a0)
    /tmp/sandbox828372841/prog.go:56 +0x32
created by main.main
    /tmp/sandbox828372841/prog.go:55 +0xcd

Program exited: status 2.

So what happened there?

The two go routines start executing one after the other in parallel. The first go routine calls Remove and then it checks the length in Remove (which is 1) and then enters the if block and sleeps for 1ns. Now the second go routine also calls Remove and checks the length and gets 1 as length and enters the if block and sleeps for 1ns. Why does it enter? It's because the first go routine is still sleeping, so it hasn't removed the element from the queue yet so the length was going to be 1 for sure. Now while second go routine is sleeping, the first one wakes up and removes the element. When the second one wakes up it will try to fetch the first element in the queue but, it doesn't exist! And that explains the panic we got i.e runtime error: index out of range [0] with length 0.

What did we learn from the above experiment?

A small delay in the form of time.Sleep(1 * time.Nanosecond) i.e sleeping for 1 ns is enough to cause problems. This tells how sensitive threads are to the code we write 🤯, meaning, if I were to add some lines to that function which take >= 1ns to run then we would be seeing this issue. Now I think it's clear why we need to handle operations differently in a multi-threaded environment.

So what's the solution?

TLDR; The solution is to use a mutex.

Intuitively, we need a mechanism that makes sure only one thread can Remove from the queue at any given instance. Particularly the if block since, that caused the problem in the first place. Here the if block is said to be the critical section of the Remove function. So, we would somehow "lock" the critical section of the Remove function when a thread approaches it and then "unlock" it when it's done with the operations it has to do. To do this "lock" and "unlock" we use a mutex in Go.

The mutex will make sure only one thread accesses the critical section at any given instance. So, when you Remove, before the if block, we need to Lock the following operations and when we are done we can Unlock. This is the updated snippet and below is the updated Remove which is now thread-safe.

func (q *Queue) Remove() (int, error) {
    q.mu.Lock()
    defer q.mu.Unlock()
    if len(q.q) > 0 {
        time.Sleep(1 * time.Nanosecond)
        item := q.q[0]
        q.q = q.q[1:]
        return item, nil
    }
    return 0, errors.New("Queue is empty")
}

I've included the mutex in the Queue struct so that every instance of the queue has it's own mutex.
If you were to run the program now, you would see everything works fine as expected. No more panics ✨!

How did it work?

When we acquired the lock using q.mu.Lock() we made sure that only one go routine can access the if block. The defer q.mu.Unlock() just makes sure we unlock it after the function returns. Try not calling Unlock and ponder what happens then 😉.

Give me the whole code 😡

Alright I've heard you 😅. Here it is:

import (
    "errors"
    "time"
    "sync"
)

type Queue struct {
    mu sync.Mutex
    capacity int
    q        []int
}

// FifoQueue 
type FifoQueue interface {
    Insert()
    Remove()
}

// Insert inserts the item into the queue
func (q *Queue) Insert(item int) error {
    q.mu.Lock()
         defer q.mu.Unlock()
    if len(q.q) < int(q.capacity) {
        q.q = append(q.q, item)
        return nil
    }
    return errors.New("Queue is full")
}

// Remove removes the oldest element from the queue
func (q *Queue) Remove() (int, error) {
    q.mu.Lock()
         defer q.mu.Unlock()
    if len(q.q) > 0 {
        item := q.q[0]
        q.q = q.q[1:]
        return item, nil
    }
    return 0, errors.New("Queue is empty")
}

// CreateQueue creates an empty queue with desired capacity
func CreateQueue(capacity int) *Queue {
    return &Queue{
        capacity: capacity,
        q:        make([]int, 0, capacity),
    }
}

Happy coding! Hope you learnt something new today 😄.

Save precious build time with custom Docker registry

Harsha Vaidya — Thu, 21 May 2020 03:34:38 +0000

At our company we have a relatively new product in development. On the DevOps side, we have a pretty simple workflow for the testing stage. It involves:

Git repository triggers Jenkins build.
Latest pull is taken from the repository
Frontend is built using npm
Backend Docker image is built (which contains frontend files)
The image is pushed to Dockerhub
Using Ansible, we run a script in the target server.

On the target server (on premises):

Pull the latest image.
Restart all related containers using Docker Compose.

This is a pretty simple CI/CD system. Nothing too complex. Our codebase is consistently growing and each build took around 20 minutes. We have around 5 builds per day on average with 6 devs working on the project. So, we realised it's high time we optimised the build system a bit to handle more builds per day and reduce time for building priority images.

So, we looked into our Jenkins' build log and found out that pushing and pulling from Dockerhub took the most time (around 10 minutes). Since our test servers are on premises, we decided that having a custom registry (available as a Docker image) is the best solution to save time. So I looked into the Registry documentation (which is very well written 😀) and deployed it on one of the test servers. Then, I tweaked the build script a little bit to accommodate for the changes and Voila! The build time reduced to 12 minutes flat!!

One of by-products of this effort was reduced image size. I noticed that I was adding node modules and git folders (newbie mistake 😛) into the image which were bloating the image resulting in unnecessary pushes and pulls of the layers of the image. The image size reduced to 5GB from 8GB. This decreased the build time to 10 minutes flat.

Some lessons learned:

Sometimes cloud is too expensive in terms of time (network latency 😞 ).
Always inspect your image sizes and keep them as low as possible.
In certain situations, managing your own services instead of cloud is not a bad solution.
Monitor builds periodically for rapidly changing codebases for redundancies which can be potentially delaying your builds.