Forem: Vandebron Technology

How to Spin Up A Kubernetes Cluster On Your Macbook

Vandebron Technology — Tue, 25 Apr 2023 19:12:35 +0000

In Vandebron we have been using container clusters to host our services since the foundation of our Big Data team. Recently our cluster of choice has declared End-Of-Life development stage, so we decided to take a step forward and get a ticket for the Kubernetes boat.

A change in the OS that is used to run your services and applications can look quite challenging and not everyone is on the same experience level. To make everyone comfortable it is a good choice to give everyone the possibility to play with the new tools and learn what can be done and how: you need a sandbox.

Our developers are provided with a Macbook and at the moment of writing there some options you can go for when deciding how to set up your playground:

Docker CE Kubernetes: This is the easiest solution since there is a handy button to run your containers into a Kubernetes environment.
Vagrant and Virtualbox: This solution is the one that can give you more control and you can easily create a cluster the size you want, but you need to be handy with VMs, the hypervisor of choice, and Vagrant. It's the old school way to do it but, while it's a chunk your platform engineers can bite, it can be a steep and frustrating process for people that are not used to handle VMs.
Multipass + some bash magic glue: Since Canonical created this tool for macOS, creating an Ubuntu VM became a breeze and you can have a single, easily manageable VM with its networking up and running in less than a minute, without having to handle disks, distros, and stuff. On top of it, the command line interface is straight forward and it has just the basic commands we will need, so wrapping the entire process into a bash script is a piece of cake.

I have found this super cool in-depth article from Jason Yee (kudos to you bruh) that guided me through the installation of my first single node Kubernetes cluster.

The process is not that long but it involves a lot of copy/pasting and, once learned the basics, I didn't want to go under the same process more times, plus it could be interesting for me as a Platform Engineer, but it may be boring and pointless for developers who just want to have a sandbox replica of what they are working on in the remote environment. My automator (aka do-it-once-never-do-it-again) spirit kicked in and I decided to wrap every step in a small command-line tool with only 3 options:

install
cleanup
help

What is happening under the hood

What the script does is substantially automating all the steps needed to:

Create a new VM using Multipass (tool released by Canonical)
Fetch the VM IP address and adding it to your local /etc/hosts file
Install k3s (a lightweight distribution of Kubernetes) on top of the VM
Install the Kubernetes command-line tools on your laptop
Install Helm (the Kubernetes package manager) on your laptop
Install cert-manager (certificate manager) package on top of your k3s cluster
Install Rancher (a Kubernetes control plane) package on top of your k3s cluster

If you are looking for a more in-depth breakdown of the single steps you can download and inspect the script (one of the many advantages of OpenSource projects) or checkout and read the original article: it explains line by line what the specific commands are doing.

1. Multipass VM

Multipass is a tool from Canonical (the company developing and maintaining the Ubuntu Linux distribution) that leverages Hyperkit (macOS feature to handle virtualization) to create and handle a Virtual Machine directly on your Mac.

2. Edit /etc/hosts

Once we have our VM up and running we need to make it available with an easy url that is also gonna be used to generate the SSL certificate, in our case we picked up rancher.localdev. It is important to have a name setup in the beginning since this one will need to match with the certificate so we can use it programmatically.

3. Install K3S

This step is pretty straightforward: just fetch a script that is publicly available on the k3s official website and feed it to your bash. K3s is a lightweight version of Kubernetes with all the needed dependencies and executable packaged in a convenient installation script. Because of its light nature, it is often used in embedded devices that have a limited amount of resources to offer.

4 & 5. Kubernetes and Helm cli

Kubernetes cli (kubectl) is used to talk and interact with your Kubernetes cluster. It can be used to manage multiple clusters according to the content of your KUBECONFIG environment variable. The variable itself contains just a path to where your cluster configuration is stored, so you can switch from a cluster to another by simply pointing to another file that contains the configuration of another cluster.

Helm instead is the "package manager" of Kubernetes: you can use it to add repositories to specific charts which are the blueprint that contains a way to install a specific tool on your cluster. Both of these tools have to be installed and run from your local laptop, either in the case you are managing a local VM or in the case you are interacting with a remote cluster.

6 & 7. cert-manager and Rancher

Rancher is the control plane for our cluster: it provides a GUI and an overview of our single node cluster. It offers other goodies like management of multiple clusters, deployed on different locations like AWS Azure and GCP or even on your own hardware, plus certificate deployment and some other handy functionalities.

cert-manager is installed via Helm chart and it is the tool used by Rancher to generate and deploy a certificate across the entire cluster.

How to use it

All the steps will involve the use of a Terminal window

Installation

The first thing you need to do is download this script and save it in a folder on your Mac (let's assume ~/bronernetes) by executing

    ```
{% endraw %}
bash
mkdir ~/bronernetes
    cd ~/bronernetes
    curl https://gist.githubusercontent.com/nikotrone/50b1a5f8d137411879eb2467e689bfbe/raw/090b4b4323d96ac28d96bbb346e2e657073722e6/bronernetes > bronernetes
    export PATH=$PATH:$(pwd)
{% raw %}




Now we have the toolset and you can confirm it works by simply running `bronernetes help`.

#### Spin up Kubernetes

The next step is to run the installation process with the command `bronernetes install`

#### Clean up

When you are done or you just want to hard reset your environment you can just type `bronernetes cleanup` and it will take care of cleaning up the VM you just used, leaving you with a pristine machine, as nothing ever happened :)

### Conclusion

Having a sandbox is very useful to play around with the concepts of a new setup or service and it packs up a huge amount of positive sides. No matter what is the language or the nature of the system you are trying to replicate, it can be challenging and involve a long list of instructions or manual operations and, sometimes, even dedicated hardware. Although with some bash glue, it is possible to automate most of those processes and the investment cost can be enormously beneficial for yourself (less work the next time you do it) and for the other people working with you (they can use the tool, comment and suggest improvements). Most of all, in the case of infrastructure, it helps raise the knowledge of "what's going on here" and documents for the ones interested in taking a trip down the rabbit hole.

***

This post was originally published on [vandebron.tech](https://vandebron.tech/blog/spin-up-kubernetes-on-macbook). Reposted automatically with [Reposted.io](https://reposted.io?utm_source=postFooter).

Cypress.io Component Design Technique for React Applications

Vandebron Technology — Tue, 25 Apr 2023 19:11:32 +0000

Cypress is a game-changer in the automation testing world, the way that Cypress was built and its architecture allows us as testers to cover more scenarios.

Cypress is not Selenium; in fact, it is different. And the way to build and design a framework should be different as well.

The most famous design technique in Selenium is the Page Object Model, and many testers use the same design technique with Cypress. Even that Cypress on their official website recommended us not to go with that approach.

Page Object Model

The main benefit of using the page object model Is to make the automation framework maintenance-friendly. We can define a specific page's selectors in a separate file and then use these selectors in our test cases.

class SignInPage {
  visit() {
    cy.visit("/signin");
  }
  getEmailError() {
    return cy.get(`[data-testid=SignInEmailError]`);
  }
  getPasswordError() {
    return cy.get(`[data-testid=SignInPasswordError]`);
  }
  fillEmail(value) {
    const field = cy.get(`[data-testid=SignInEmailField]`);
    field.clear();
    field.type(value);
    return this;
  }
  fillPassword(value) {
    const field = cy.get(`[data-testid=SignInPasswordField]`);
    field.clear();
    field.type(value);
    return this;
  }
  submit() {
    const button = cy.get(`[data-testid=SignInSubmitButton]`);
    button.click();
  }
}
export default SignInPage;

The main two downsides using the typical page object model with cypress are:

Page objects introduce an additional state into the tests, separate from the application’s internal state. This makes understanding the tests and failures harder.
Page objects make tests slow because they force the tests to always go through the application user interface.

Component-Based Architecture

On the other hand, a React application is component-based, where a specific page will be built from a collection of components. And components in React can be used on different pages too. So if we want to use the Page Object Model, we may define the same locator twice on different pages.

So having these two facts, At Vandebron, we came up with a new way to design our Cypress Automation framework by creating a separate JavaScript file for every component in our application, inside a folder called components within our Cypress project as below:

// Locators
export const getEmailError = () => cy.get(`[data-testid=SignInEmailError]`);
export const getPasswordError = () =>
  cy.get(`[data-testid=SignInPasswordError]`);
export const emailField = () => cy.get(`[data-testid=SignInEmailField]`);
export const passwordField = () => cy.get(`[data-testid=SignInPasswordField]`);
export const submitButton = () => cy.get(`[data-testid=SignInSubmitButton]`);

// Actions
export const visit = () => cy.visit("/signin");
export const performLogin = (email, password) => {
  emailField().clear().type(email);
  passwordField().clear().type(password);
  submitButton().click();
};

Having it built this way, we eliminated all the previous problems mentioned earlier; we are not adding any classes, and we are defining objects within our test cases. And the most important part is that we are following the way that Cypress recommends it.

And after defining the component locators and actions, we can import them inside our test case and use them as below:

import LoginComponent from "../components/loginComponent";
import Menu from "../components/Menu";

describe("Test Login Page", () => {
  it("should show an error message if the password in wrong", () => {
    LoginComponent.visit();
    LoginComponent.performLogin("email@gmail.com", "wrongPassword");
    LoginComponent.getPasswordError().should("be.visible");
  });
  it("should show the logout button if the user logged in succesfully", () => {
    LoginComponent.visit();
    LoginComponent.performLogin("email@gmail.com", "correctPassword");
    Menu.LogoutButton().should("be.visible");
  });
});

And as you can see, our test cases are readable for anyone! And if any locator changes in any of the components, we can easily fix it in one location and from the same file. And lastly, if a component will be used in different places, we can use the same code.

In the next article, I will talk about how we use Cypress in our manual testing during the sprint and how it saves us tons of time and effort.

This post was originally published on vandebron.tech. Reposted automatically with Reposted.io.

Migrating from DCOS to Kubernetes, dealing with the l4lb loadbalancer

Vandebron Technology — Tue, 25 Apr 2023 18:55:39 +0000

In October 2020 D2IQ announced that they are moving onwards with their Kubernetes offering. Vandebron has been a D2IQ customer for their DCOS offering, we were just in the middle of a migration of our first workloads to DCOS Enterprise. We have evaluated the D2IQ K8s offering and decided to go for another Kubernetes product. We had a few migrations over the years, we migrated from Azure to AWS, we migrated workloads from normal instances to spot instances and all these migrations were done with nearly any downtime. We plan to reduce the downtime to a couple of minutes this migration and this is a real challenge. The first challenge that we will discuss today: We want to pair our Kubernetes clusters to the DCOS/Mesos clusters, while we move a workload it should be able to connect to its dependencies in the DCOS cluster. We use DCOS for our NoSQL databases like Cassandra, internal data that we want to keep internal. Pairing DCOS and Kubernetes clusters enable us to reduce downtime, enabling us to switch back if we run into issues and move faster because it reduces complexity.

L4LB

The internal layer 4 load balancer DCOS provides is used in the majority of our workloads. When our data scientists schedule a spark driver, they connect to the spark dispatcher through the Layer 4 load balancer. Most of the DCOS frameworks use this Layer 4 load balancer as an internal service discovery tool, with Vandebron we use this layer 4 load balancer to communicate between services. In a default DCOS set up this load balancer responds on domain names like: spark-dispatcher.marathon.l4lb.thisdcos.directory:7077

When we ping the spark dispatcher we get the following:

PING spark-dispatcher.marathon.l4lb.thisdcos.directory (11.155.161.35) 56(84) bytes of data.
64 bytes from 11.155.161.35 (11.155.161.35): icmp_seq=1 ttl=64 time=0.024 ms

After some investigation we found out that this IP range is not actually on a network interface, it is a Linux kernel functionality called IPVS. With IPVS you can do layer 4 load balancing, you provide the target location and the location you want to respond on.

When we search for the IP from the spark dispatcher with ipvsadm, we get 3 results:

sudo ipvsadm -L -n |grep --color '11.155.161.35\|$'
TCP  11.155.161.35:80 wlc
  -> 10.2.7.146:16827             Masq    1      0          0
TCP  11.155.161.35:4040 wlc
  -> 10.2.7.146:16826             Masq    1      0          0
TCP  11.155.161.35:7077 wlc
  -> 10.2.7.146:16825             Masq    1      0          0

As you can see the IP 11.155.161.35 points towards 10.2.7.146, even the ports are configured and forwarded. We can add our route with ipvsadm, to understand IPVS a bit better. For example:

sudo ipvsadm -A -t 1.2.3.4:80 -s wlc # we add the target server and assign the scheduler
sudo ipvsadm -a -r 10.2.7.146:16825 -t 1.2.3.4:80 -m # we configure the real server and target server and configure Masquerading
curl 1.2.3.4:80
{
  "action" : "ErrorResponse",
  "message" : "Missing protocol version. Please submit requests through http://[host]:[port]/v1/submissions/...",
  "serverSparkVersion" : "2.3.4"
}

This results in that the spark dispatcher now also is available on 1.2.3.4:80. As mentioned before we wanted to connect our DCOS and Kubernetes clusters, getting hundreds of entries from ipvsadm and manually adding them one by one didn’t sound appealing to us. Especially if you consider that sometimes services fail and run on a different port or different host after recovery, maintaining this by hand would be a nightmare. We therefore decided to build a tool to sync IPVS entries from DCOS to Kubernetes.

Stack

Within Vandebron we have our tech stack, we strongly believe it is good to eat your own dog food. When possible and when our use cases are similar we use the same tools as our Developers use. The parts of the stack we will be using are:

AWS ELB in front of Traefik 1.7
DCOS
Kubernetes

Within our platform team, we use Golang as our scripting language. Golang gives us the ability to build binary files with all the required libraries in the binary, we don’t have to install any packages, we do not even need to install Golang on the machine the application will be running on.

In our DCOS cluster we use Traefik 1.7, this version of Traefik only forwards HTTP requests. We decided to use Traefik to expose a JSON endpoint so we can gather the IPVS information from this location.

ipvs-server

Within our DCOS cluster we will expose the IPVS information through a JSON endpoint. We have built a tool for this to expose this information in multiple ways. In the next section, we are going to discuss some of the concepts and choices we made, we won’t deep dive into Go specifics. We have provided the entire code for this project in the examples directory of our GitHub repo: https://github.com/Vandebron/tech-blog

First, let’s discuss the library we use: https://github.com/nanobox-io/golang-lvs. This library in its essence translates to ipvsadm commands, it helped save us time to implement this ourselves. There are some gotcha’s, such as newlines are not filtered out from the output. We solved this by cleaning up some of the data.

In the childChan function we create a go channel that is responsible for polling ipvsadm every 10 seconds and stores the result in a couple of variables we use in our HTTP endpoints. IPVS is a Linux kernel functionality and should be highly performant, we do not want to trigger kernel panics when the server gets overloaded with requests. We expect that every 10 seconds gives us accurate enough results, we can always lower this interval to ensure faster results. We also added in this function the string manipulation to ensure all the newlines were gone in the JSON output. The newline gave issues when we tried to add the IPVS scheduler entries.

func childChan(c chan bool) {
   fmt.Println("Starting time based IPVS Admin poll")

   pollInterval := 10
   timerCh := time.Tick(time.Duration(pollInterval) * time.Second)
   // Time based loop to generate Global variable
   for range timerCh {
       select {
       // when shutdown is received we break
       case <-c:
           fmt.Println("Received shutdown, stopping timer")
           break
       default:
           var err error
           listIpvs.Save()
           ipvsString = fmt.Sprintln(listIpvs.Services)

           res := &responseObject{
               Services: listIpvs.Services,
           }

           ipvsJSONbyte, err := json.Marshal(res)
           if err != nil {
               logToErr.Printf("ERROR: -- Marshal JSON -- %v\n", err)
           }

           ipvsString = string(ipvsJSONbyte)
           ipvsJSON = strings.Replace(ipvsString, `\n`, ``, -1)
           if debug != false {
               logToOut.Println("DEBUG: -- ipvsJSON --", ipvsJSON)
           }
       }
   }
}

Next is the index handler, we set our headers correctly and print the result as we would receive through ipvsadm. The index is mainly for our platform engineers to debug and verify the output. Thanks to this overview we found much faster that there was a newline hidden in the scheduler output.

func index() http.Handler {
   // Generating the Index
   return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

       // Only available when debug is on
       if debug != false {
           logToOut.Println("DEBUG: -- index --", ipvsString)
       }

       if r.URL.Path != "/" {
           http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
           return
       }
       w.Header().Set("Content-Type", "text/plain; charset=utf-8")
       // Site security testers expect this header to be set
       w.Header().Set("X-Content-Type-Options", "nosniff")
       w.WriteHeader(http.StatusOK)
       fmt.Fprintln(w, ipvsString)
   })
}

The JSON endpoint is what we use in the client communicate with the server.

func jsonz() http.Handler {
   // Generating the Index
   return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

       // Only available when debug is on
       if debug != false {
           logToOut.Println("DEBUG: -- jsonz --", ipvsJSON)
       }

       if r.URL.Path != "/json" {
           http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
           return
       }
       w.Header().Set("Content-Type", "application/json; charset=utf-8")
       // Site security testers expect this header to be set
       w.Header().Set("X-Content-Type-Options", "nosniff")
       w.WriteHeader(http.StatusOK)
       fmt.Fprintln(w, ipvsJSON)
   })
}

We ask our Developers often to implement a basic health endpoint, in DCOS we use this to see if a service needs to be restarted. In our application we enable set the statusOK in the index or in the JSON endpoint.

func healthz() http.Handler {
   // Generating the healthz endpoint
   return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
       if atomic.LoadInt32(&healthy) == 1 {
           w.WriteHeader(http.StatusNoContent)
           return
       }
       w.WriteHeader(http.StatusServiceUnavailable)
   })
}

In our logging and tracing functions we want to register the clients that are connecting, this gives us information where calls are coming from. It helps us debugging if we see weird behaviour.

func tracing(nextRequestID func() string) func(http.Handler) http.Handler {
   // Tracing the http requests so its easier to check if server is reached
   return func(next http.Handler) http.Handler {
       return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
           requestID := r.Header.Get("X-Request-Id")
           if requestID == "" {
               requestID = nextRequestID()
           }
           ctx := context.WithValue(r.Context(), requestIDKey, requestID)
           w.Header().Set("X-Request-Id", requestID)
           next.ServeHTTP(w, r.WithContext(ctx))
       })
   }
}

func logging(logToOut *log.Logger) func(http.Handler) http.Handler {
   // Creating logging entry tracing the http requests
   return func(next http.Handler) http.Handler {
       return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
           defer func() {
               requestID, ok := r.Context().Value(requestIDKey).(string)
               if !ok {
                   requestID = "unknown"
               }
               logToOut.Println(requestID, r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent())
           }()
           next.ServeHTTP(w, r)
       })
   }
}

IPVS needs to be executed with root privileges, to ensure this is correct we get the userid and print it when starting the server.

// getProcessOwner function is to see who is running the process. It needs to be a sudo / root user
func getProcessOwner() string {
   stdout, err := exec.Command("ps", "-o", "user=", "-p", strconv.Itoa(os.Getpid())).Output()
   if err != nil {
       logToErr.Printf("ERROR: -- getProcessOwner -- %v\n", err)
       os.Exit(1)
   }
   return string(stdout)
}

We added the init function to ensure we print results the moment the server starts up, if we would not do this it would take 10 seconds for the go channel to activate

func init() {
   // Placing the Save and val in the init, else we will need to wait for channel to perform its first run
   listIpvs.Save()
   ipvsString = fmt.Sprintln(listIpvs.Services)
}

In the main function, we set the configurable flags, such as debugging to show error messages. It proved useful during the creation of this tool to keep track and print output. If we would print the output at every call to our logs, our Elastic cluster would get thousands of logs that add little to no value.

We configure the listen port in the flags, we can use the portIndex from DCOS to assign a random port on the host to listen on. We also provided to print the version we are running. In our versioning, we use a constant to list the application semver version, we also provide the git-commit hash. When we begin the server we print the version information, the port we listen on and the user running the process. We then start the server process with the go channel, in setting up the go channel we ensure that when the server stops we try to gracefully stop the server within a 30-second timeframe. Since our ipvsadm timer is 10 seconds it should be able to cleanly shutdown within that period.

Docker build

In the repository, we have included a Dockerfile and a script to build the Dockerfile. In this Dockerfile, we pass the git commit hash to the go install. This way we always get the Git Hash from our GitHub repo and we can use this information in our version output.

DCOS service.json

In the repository, we have provided the service.json file, since it is opinionated on using Traefik you might need to change it. But in this service.json you see how we set up Traefik, the health check, and port index. Since the Mesos UCR container has fewer abstractions and has fewer limited capabilities. We can run the IPVS server inside a UCR container and get all the output as if we were running this directly as root on the host machine.

ipvs-client

The IPVS client is the component we use in the Kubernetes environment. The client connects to the server and gets the IPVS entries from the IPVS server inside our DCOS cluster. It then adds these IPVS entries to each node in the Kubernetes cluster. You, therefore, need to run each client per Kubernetes node.

You can find the code from the IPVS client in our repository.

func httpGet(remoteURL string) []byte {
   if debug != false {
       _, err := url.ParseRequestURI(remoteURL)
       if err != nil {
           panic(err)
       }
   }

   req, err := http.NewRequest(http.MethodGet, remoteURL, nil)
   if err != nil {
       logToErr.Fatalf("ERROR: -- new HTTP request -- %v", err)
   }

   ipvsClient := http.Client{
       Timeout: time.Second * 2, // Timeout after 2 seconds
   }
   req.Header.Set("User-Agent", "go-ipvs-get \tversion: "+version+"\t Git Commit: "+gitCommit)
   res, err := ipvsClient.Do(req)
   if err != nil {
       logToErr.Fatalf("ERROR: -- ipvsClient -- %v\n", err)
   }

   if res.Body != nil {
       defer res.Body.Close()
   }

   body, readErr := ioutil.ReadAll(res.Body)
   if readErr != nil {
       logToErr.Fatalf("ERROR: -- body -- %v\n", readErr)
   }

   return body
}

In the httpGet function we can debug the URL and check if it is valid. Again we set the correct headers and retrieve the JSON body.

func unmarshal(body []byte) []lvs.Service {

   res := &responseObject{
       Services: listIpvs.Services,
   }

   jsonErr := json.Unmarshal(body, &res)
   if jsonErr != nil {
       logToErr.Fatalf("ERROR: -- Unmarshal -- %v \n", jsonErr)
   }

   if debug != false {
       logToOut.Fatalf("DEBUG: -- res -- %v \n", res.Services)
   }

   r := res.Services

   return r
}

In the unmarshal function we unmarshal the JSON and turn it in a slice of lvs.Service.

func addServers(remoteAddr string) {
   body := httpGet(remoteAddr)
   jsonData := unmarshal(body)

   for i, v := range jsonData {
       if debug != false {
           logToOut.Printf("DEBUG: -- range jsonDATA --\n")
           logToOut.Printf("ipvsCount=%v, value=%v", i, v)
       }

       err := lvs.DefaultIpvs.AddService(v)
       if err != nil {
           logToErr.Printf("ERROR: -- AddService -- %v", err)
       }

       i++
       ipvsServerCount = float64(i)
   }
}

In the addServers function we add the servers to IPVS.

func clientChan(c chan bool) {
   logToOut.Println("Starting time based IPVS Admin add")

   pollInterval := 10
   timerCh := time.Tick(time.Duration(pollInterval) * time.Second)
   // Time based loop to generate Global variable
   for range timerCh {
       select {
       // when shutdown is received we break
       case <-c:
           logToOut.Println("Received shutdown, stopping timer")
           break
       default:

           logToOut.Println("Clearing & Adding servers...")
           // Before we add Servers we need to clear the existing list
           lvs.Clear()
           addServers(remoteAddr)
           if debug != false {
               logToOut.Printf("IPVS servers added:\t%v", ipvsServerCount)
           }
       }
   }
}

Like we did in the IPVS server we create a go channel to poll every 10 seconds the server endpoint. We perform this to get at a set interval the IPVS entries.

Since we run the IPVS client as a binary directly on the Kubernetes hosts we build the binary with a few parameters we pass to the go build command. The binary we build with this command we host on an internal s3 bucket, we can download this binary with systemd unit files.

GOOS=linux
GOARCH=amd64
GIT_COMMIT=$(git rev-list -1 HEAD)

export GOOS
export GOARCH
export GIT_COMMIT

env GOOS=${GOOS} GOARCH=${GOARCH} go build -v -ldflags "-X main.gitCommit=${GIT_COMMIT}" .

When we run the IPVS client we can verify if the IPVS routes are added by running the ipvsadm -L -n command.

Unit files

Since IPVS is part of the Linux kernel it is hard to deploy this in a docker container, the capabilities are more restricted in Kubernetes. We decided to deploy the IPVS client on each host machine through a systemd unit file, the main reason was that we ran into restrictions that slowed us down and this is not a permanent solution. By adding the IPVS client on the machines alone does not make it possible for containers to use the IPVS routes. We needed to add NET_ADMIN capabilities to all containers using the l4lb loadbalancer locations and configure hostNetworking: true in the Kubernetes pods.

We provided a deployment.yml file that runs a Ubuntu docker container with ipvsadm only installed extra. When the pods are deployed in this deployment you can use kubectl exec to get into the pod and run the ipvsadm -L -n command.

Vacancy at Vandebron

We are looking for a platform engineer in Vandebron. As you can understand this is not a typical scenario we daily run across, but it is part of the workloads that we will support when working on our platform. Within Vandebron we try to use the best technology available, when it is not available we build it. Due to this as platform engineers, we have many interesting challenges and offer engineers to support further than only a strict domain. We support all components of our entire platform, regardless if it is a Linux kernel issue like this, involves setting up and maintaining a NoSQL cluster, or helping the business with something like requesting a certificate.

If you are interested in learning more about this position, take a look at our Vacancy and get in contact with us. https://werkenbij.vandebron.nl/

This post was originally published on vandebron.tech. Reposted automatically with Reposted.io.

Looking back at the Vandebron GreenTech Hackathon 2021

Vandebron Technology — Tue, 25 Apr 2023 15:19:12 +0000

At the end of March, we organized a public hackathon to create solutions to tackle climate challenges. After having done internal hackathons, we thought it was time to share our technologies with other innovative people and companies. Together with a group of partners, and enthusiastic participants, spend three full days of (remote) hacking with great results.

Why organize a public hackathon?

Climate change is one of the many pressing challenges our society is currently facing. At Vandebron, we want to continue finding ways to tackle this immense challenge. That’s why we decided to organize a 3-day GreenTech hackathon that ran from March 31st to April 2nd, 2021. We've been organizing internal hackathons for the past four years, to foster innovation within our company and allow our developers to work on something exciting without any constraints. If you want to read more about why we organize internal hackathons, you can find an article by our CTO here.

By organizing a public hackathon, we hoped to attract a bigger audience, possibly even outside our country, The Netherlands, and attract partners to work together with. We succeeded in both, and together with Hack the Planet and Top Dutch Solar Racing, we wanted to find technological solutions to problems in wildlife conservation and renewable energy. For these three days, all participants got the opportunity to work on challenges from our partners, access their technology and knowledge, and got the chance to win unique prizes. Also, we organized a free event with speakers Florian Dirkse (The Ocean Cleanup), Thijs Suijten (Hack the Planet) and Heleen Klinkert (Nieuw Groen).

Looking back

The event started on March 31st, when all hackathon challenges were presented and the participants could select which challenge they wanted to work on. People from all over The Netherlands (and even beyond) signed up for the hackathon, ranging from students from the University of Amsterdam to young professionals looking for a job. The first challenge the participants could subscribe to was from Vandebron itself, where teams got the opportunity to use a selection of our Electronic Vehicle (EV) data. With this data, they could for example make a forecast on the amount of charging sessions we could expect on a typical day. Second, our partner Hack the Planet presented their challenge that was aimed at thinking of innovative solutions for their project Hack the Poacher. With Hack the Poacher, they install smart camera traps in African wildlife reservations to detect poachers. The teams could use their camera traps and data to create more solutions to map the poachers or use the camera traps for other needs. Finally, the students from Top Dutch Solar Racing presented a challenge to simulate the race they were supposed to join at the end of the year in Australia. Using their weather and traffic data, the teams could simulate the race and predict how much time they would need to complete the race. After selecting a challenge, all teams started the hackathon and participated in sessions to learn more about the challenges to get started.

All teams continued working on the hackathon challenge on the second day, after a nice warming-up quiz about climate change in the morning. For most teams this second day was when their project started to take shape, and they got a better idea about what they would be presenting on the final day. This second day was also an opportunity for non-technical people to get to know Vandebron and their partners better as we organized inspirational sessions with talks from different speakers in the afternoon. One of the co-founders from The Ocean Cleanup, Florian Dirkse, inspired us with his story behind making a difference in the world. After which, one of our hackathon partners Thijs Suijten, from Hack the Planet, demonstrated how technology can be used for the good. Our third, and final, speaker Heleen Klinkert (Nieuw Groen), showed how we can compensate for our CO2 emissions by storing them in the soil.

On the final day of the hackathon, all teams had to finalize their projects and create a presentation for the closing ceremony. During this ceremony, all participants and partners looked back at the past three days and shared what they had been working on during the hackathon. For every challenge, one team could win and take home several prizes, sponsored by Marie-Stella-Maris, EV Experience, and Klimaatroute. The first presentations were for the Vandebron challenge about EV forecasts. This challenge was won by not one but two teams as the jury and audience were so impressed by their solutions. Both teams created not only the forecast based on the sample data provided, but also created interactive dashboards. On the challenge for Hack the Planet, the team that won came up with a unique solution to use the camera traps to detect wild animals on the streets. For countries like India, this is a huge problem, as wild animals get stuck in traffic or walk through rural areas. The final winner of the hackathon was a group of students that simulated the Top Dutch Solar Racing trip through Australia and forecasted they could complete the race within 7 days.

Thanks everyone

I'd like to thank all the participants, prize/challenge partners, and speakers for their efforts during these days. The GreenTech Hackathon 2021 was a huge success thanks to everyone that has been involved. Keep following the vandebron.tech to be updated on future hackathons and events.

This post was originally published on vandebron.tech. Reposted automatically with Reposted.io.