Forem: appfleet

appfleet, a new edge compute platform goes live

Dmitriy A. — Mon, 01 Mar 2021 10:39:10 +0000

First of all what is appfleet? appfleet is an edge compute platform that allows people to deploy their web applications globally. Instead of running your code in a single centralized location you can now run it everywhere, at the same time.

In simpler terms appfleet is a next-gen CDN, instead of being limited to only serving static content closer to your users you can now do the same thing for your whole codebase. Run the whole thing where just your cache used to be.

This results in drastic performance improvement, lower latency, better uptime and an enormous amount of new use-cases and possibilities.

In part it's because we are not limiting our users to HTTP services. You have complete freedom to run any kind of service over any protocol you want, on any port.

Do you want to build your own global nameservers? Deploy a container running a DNS server over UDP port 53. It takes only a few clicks. A globally distributed database? Sure thing. Or something simple like image optimization on the edge? No problem. How about a monstrous container running a web service, redis, ssh, DNS, MySQL and an admin service all on different ports, all at the same time? Who are we to stop you, go ahead!

We launched our closed beta quite a while ago. And since then we have worked with many developers and business owners to improve our platform and support the many exciting use-cases that people kept coming up with.

And while the system was technically ready to accept real customers months ago, we decided it was best to stay in beta and ensure the stability of the platform.

Since then we have polished our user-experience, redesigned many parts of our UI multiple times and made sure our backend is ready for whatever people throw at it.

Do you know what is the first thing people run when given a container and asked to test it? They run a fork bomb. Of course we were ready for that and while the specific instance became unavailable it had no impact on our system or other clients. All thanks to the multiple layers of isolation we have built. And nowadays the instance won't even go down, so fork-bomb away.

Each container runs in a virtualized box of its own, with its own resources, filesystem and security in place. Security and stability were top priorities for us and everything we built had that in mind.

Whenever we were working on a new feature we kept asking ourselves "What if?". "What if this system goes down?", "What if the user does something unexpected?"

appfleet is based on multiple services and modules interacting with each other. We made sure that even if something breaks, like our whole API goes down, or our DB, or anything else really, the already running applications would not feel a thing. Everything is built to run in standalone mode and if the worst happens to wait until things get better.

We also drew from our experience building and maintaining jsDelivr, a free CDN for open source projects that currently serves 100 billion requests every month and more than 3 Petabytes of traffic! It is used by millions of websites all over the world and all of them trust us to ensure it never goes down. We integrated multiple levels of failover with multiple checks on every step to ensure the system can automatically handle different kinds of issues and fix itself.

The appfleet platform was built to be as simple as possible and make edge compute accessible to everyone, from open source projects, to solo developers and even big enterprises that need something that works without relying on an army of DevOps engineers. This is one of the reasons we decided to build on top of containers, this allows our users to easily migrate to or from appfleet and to even run legacy applications.

Today March 1st 2021 is the beginning of a long an exciting journey to make the web faster and accessible to all!

Register now and get $10 of free credits to use as you see fit. No need to enter your credit card until you are ready for production workloads.

And if you are a non-profit or an open source project let us know to get sponsored with free services. We even offer free design services

Join us and make sure you send us your feedback and ideas! Even better email the founder directly at d@appfleet.com

Demystifying Open-Source Orchestration of Unikernels With Unik

Dmitriy A. — Thu, 24 Sep 2020 17:10:23 +0000

Abstract

As the cloud-native ecosystem continues to evolve, many alternative solutions are popping-up, that challenges the status quo of application deployment methodologies. One of these solutions that is quickly gaining traction is Unikernels, which are executable images that can run natively on a hypervisor without the need for a separate operating system.

For cloud-native platforms to integrate unikernels into their ecosystem, they are required to provide unikernels with the same services they provide for containers. In this post, we are going to introduce UniK, an open-source orchestration system for unikernels.

UniK (pronounced you-neek) is a tool for simplifying compilation and orchestration of unikernels. Similar to the way Docker builds and orchestrates containers, UniK automates compilation of popular languages (C/C++, Golang, Java, Node.js. Python) into unikernels. UniK deploys unikernels as virtual machines on various cloud platforms.

UniK Design

The UniK Daemon consists of 3 major components.

The API server
Compilers
Providers

The API Server handles requests from the CLI or any HTTP Client, then determines which is the appropriate provider and/or compiler to service the request.

When the API Server receives a build request (POST /images/:image_name/create), it calls the specified compiler to build the raw image, and then passes the raw image to the specified provider, which processes the raw image with the Stage() method, turning it into an infrastructure-specific bootable image (e.g. an Amazon AMI on AWS)

Let's go ahead and try spinning up a unikernel ourselves.

Deploying a GO HTTP daemon with UniK

In this tutorial, we are going to be:

Install, configure, and launch UniK

Prerequisites

Ensure that each of the following are installed

Docker installed and running with at least 6GB available space for building images
jq
make
Virtualbox

Install UniK

$ git clone https://github.com/solo-io/unik.git
$ cd unik
$ make binary

Note: make will take quite a few minutes the first time it runs. The UniK Makefile is pulling all of the Docker images that bundle UniK's dependencies.

Then, place the unik executable in your $PATH to make running UniK commands easier:

$ mv _build/unik /usr/local/bin/

Configure a Host-only network on VirtualBox

1.Open Virtualbox.
2.Open Preferences > Network > Host-only Networks.
3.Click the green Add button on the right side of the UI.
4.Record the name of the new Host-only adapter (e.g. "vboxnet0"). You will need this in your UniK configuration.
5.Ensure that the Virtualbox DHCP Server is Enabled for this Host-only Network
6.With the Host-only Network selected, Click the edit button (screwdriver image)
7.In the Adapter tab, note the IPv4 address and netmask of the adapter.
8.In the DHCP Server tab, check the Enable Server box.
9.Set Server Address an IP on the same subnet as the Adapter IP. For example, if the adapter IP is 192.168.100.1, make set the DHCP server IP as 192.168.100.X, where X is a number between 2-254.
10.Set Server Mask to the netmask you just noted.
11.Set Upper/Lower Address Bound to a range of IPs on the same subnet. We recommend using the range X-254 where X is one higher than the IP you used for the DHCP server itself. E.g., if your DHCP server is 192.168.100.2, you can set the lower and upper bounds to 192.168.100.3 and 192.168.100.254, respectively.

Configure UniK daemon

UniK configuration files are stored in $HOME/.unik. Create this directory, if it is not present:

$mkdir $HOME/.unik

Using a text editor, create and save the following to $HOME/.unik/daemon-config.yaml:

providers:
  virtualbox:
    - name: my-vbox
      adapter_type: host_only
      adapter_name: NEW_HOST_ONLY_ADAPTER

Replacing NEW_HOST_ONLY_ADAPTER with the name of the network adapter you created.

Launch UniK and automatically deploy the Virtualbox Instance Listener

Open a new terminal window/tab. This terminal will be where we leave the UniK daemon running.
cd to the _builddirectory created by make
run unik daemon --debug (the --debug flag is optional, if you want to see more verbose output)
UniK will compile and deploy its own 30 MB unikernel. This unikernel is the Unik Instance Listener. The Instance Listener uses udp broadcast to detect (the IP address) and bootstrap instances running on Virtualbox.
After this is finished, UniK is running and ready to accept commands.
Open a new terminal window and type unik target --host localhost to set the CLI target to the your local machine.

Write a Go HTTP server

Open a new terminal window, but leave the window with the daemon running. This window will be used for running UniK CLI commands.

1.Create httpd.go file

Create a file httpd.go using a text editor. Copy and paste the following code in that file:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "my first unikernel!")
}

2.Run the code

Try running this code with go run httpd.go. Visit http://localhost:8080/ to see that the server is running.

3.Create a dummy Godeps file

We need to create a dummy Godeps file. This is necessary to tell the Go compiler how Go projects and their dependencies are structured. Fortunately, with this example, our project has no dependencies, and we can just fill out a simple Godeps file without installing godep.

Note: For Go projects with imported dependencies, and nested packages, you will need to install Godeps and run GO15VENDOREXPERIMENT=1 godep save ./... in your project. See Compiling Go Apps with UniK for more information.

To create the dummy Godeps file, create a folder named Godeps in the same directory as httpd.go. Inside, create a file named Godeps.json and paste the following inside:

{
    "ImportPath": "my_httpd",
    "GoVersion": "go1.6",
    "GodepVersion": "v63",
    "Packages": [
        "./.."
    ],
    "Deps": [
        {
            "ImportPath": "github.com/solo-io/unik/docs/examples",
            "Rev": "f8cc0dd435de36377eac060c93481cc9f3ae9688"
        }
    ]
}

For the purposes of this example, what matters here is my_httpd. It instructs the Go compiler that the project should be installed from $GOPATH/src/my_httpd.

4.Great! Now we're ready to compile this code to a unikernel.

Compile an image to run on VirtualBox

1.Compile sources

Run the following command from the directory where your httpd.go is located:

unik build --name myImage --path ./ --base rump --language go --provider virtualbox

This command will instruct UniK to compile the sources found in the working directory (./) using the rump-go-virtualbox compiler.

2.Watch output

You can watch the output of the build command in the terminal window running the daemon.

3.Locate disk image

When build finishes, the resulting disk image will reside at $HOME/.unik/virtualbox/images/myImage/boot.vmdk

4.Run instance of disk image

Run an instance of this image with:

unik run --instanceName myInstance --imageName myImage

5.Check IP

When the instance finishes launching, let's check its IP and see if it is running our application. Run unik instances. The instance IP Address should be listed.

6.View the instance!

Direct your browser to http://instance-ip:8080 and see that your instance is running!

Finishing up

To clean up your image and the instance you created

unik rmi --force --image myImage

And we're done. We hope you will benefit from this post and tutorial and stay tuned for future posts!

Building Images Faster and Better With Multi-Stage Builds

Dmitriy A. — Thu, 24 Sep 2020 17:09:55 +0000

There is no doubt about the fact that Docker makes it very easy to deploy multiple applications on a single box. Be it different versions of the same tool, different applications with different version dependencies - Docker has you covered. But then nothing comes free. This flexibility comes with some problems - like high disk usage and large images. With Docker, you have to be careful about writing your Dockerfile efficiently in order to reduce the image size and also improve the build times.

Docker provides a set of standard practices to follow in order to keep your image size small - also covers multi-stage builds in brief.

Multi-stage builds are specifically useful for use cases where we build an artifact, binary or executable. Usually, there are lots of dependencies required for building the binary - for example - GCC, Maven, build-essentials, etc., but once you have the executable, you don’t need those dependencies to run the executable. Multi-stage builds use this to skim the image size. They let you build the executable in a separate environment and then build the final image only with the executable and minimal dependencies required to run the executable.

For example, here’s a simple application written in Go. All it does is to print “Hello World!!” as output. Let’s start without using multi-stage builds.

Dockerfile
FROM golang
ADD . /app
WORKDIR /app
RUN go build # This will create a binary file named app
ENTRYPOINT /app/app

Build and run the image

docker build -t goapp .
~/g/helloworld ❯❯❯ docker run -it --rm goapp
Hello World!!

Now let us check the image size

~/g/helloworld ❯❯❯ docker images | grep goapp
goapp                                          latest              b4221e45dfa0        18 seconds ago      805MB

New Dockerfile

# Build executable stage
FROM golang
ADD . /app
WORKDIR /app
RUN go build
ENTRYPOINT /app/app
# Build final image
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /app/app .
CMD ["./app"]

Re-build and run the image

docker build -t goapp .
~/g/helloworld ❯❯❯ docker run -it --rm goapp
Hello World!!

Let us check the image again

~/g/helloworld ❯❯❯ docker images | grep goapp
goapp                                          latest              100f92d756da        8 seconds ago       8.15MB
~/g/helloworld ❯❯❯

We can see a massive reduction in the image size -> From 805 MB ** we are down to **8.15MB. This is mostly because the Golang image has lots of dependencies which our final executable doesn’t even require for running.

What’s happening here?

We are building the image in two stages. First, we are using a Golang base image, copying our code inside it and building our executable file App. Now in the next stage, we are using a new Alpine base image and copying the binary which we built earlier to our new stage. Important point to note here is that the image built at each stage is entirely independent.

Stage 0

# Build executable stage
FROM golang
ADD . /app
WORKDIR /app
RUN go build
ENTRYPOINT /app/app

* Stage 1

# Build final image
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /app/app .
CMD ["./app”]

Note the line COPY —from=0 /app/app -> this lets you access data from inside the image built in previous image.

How multi-stage builds work?

If you look at the process carefully multi-stage builds are not much different from actual Docker builds. The only major difference is that you build multiple independent images (1 per stage) and you get the capability to copy artifacts/files from one image to another easily. The feature which multi-stage build is providing right now was earlier archived from through scripts. People used to create the build image - copy artifact from it manually - and then copy it to a new image with no additional dependencies. In the above example, we build one image on Stage 0 and then in Stage 1 we build another image, to which we copy files from the older image - nothing complicated.

NOTE: We are copying /app from one image to another - not one container to another.

This can speed up deployments and save cost in multiple ways

You build efficient lightweight images - hence you ship lesser data during deployment which saves both cost and time.
You can stop a multi-stage build at any stage - so you can use it to avoid the builder pattern with multi-stage builds. Have a single Dockerfile for both dev, staging and deployment.

The above was just a small example, multi-stage builds can be used to improve Docker images of applications written in other languages as well. Also, multi-stage builds can help to avoid writing multiple Dockerfiles (builder pattern) - instead a single Dockerfile with multiple stages can be adapted to streamline the development process. If you haven't explored it already - go ahead and do it.

Tutorial: Kubernetes-Native Backup and Recovery With Stash

Dmitriy A. — Fri, 18 Sep 2020 09:29:42 +0000

Intro

Having a proper backup recovery plan is vital to any organization's IT operation. However, when you begin to distribute workloads across data centers and regions, that process begins to become more and more complex. Container orchestration platforms such as Kubernetes have begun to ease this burden and enabled the management of distributed workloads in areas that were previously very challenging.

In this post, we are going to introduce you to a Kubernetes-native tool for taking backups of your disks, helping with the crucial recovery plan. Stash is a Restic Operator that accelerates the task of backing up and recovering your Kubernetes infrastructure. You can read more about the Operator Framework via this blog post.

How does Stash work?

Using Stash, you can backup Kubernetes volumes mounted in following types of workloads:

Deployment
DaemonSet
ReplicaSet
ReplicationController
StatefulSet

At the heart of Stash is a Kubernetes controller which uses Custom Resource Definition (CRD) to specify targets and behaviors of the backup and restore process in a Kubernetes native way. A simplified architecture of Stash is shown below:

Installing Stash

Using Helm 3

Stash can be installed via Helm using the chart from AppsCode Charts Repository. To install the chart with the release name stash-operator:

$ helm repo add appscode https://charts.appscode.com/stable/
$ helm repo update
$ helm search repo appscode/stash --version v0.9.0-rc.6
NAME            CHART          VERSION      APP VERSION DESCRIPTION
appscode/stash  v0.9.0-rc.6    v0.9.0-rc.6  Stash by AppsCode - Backup your Kubernetes Volumes

$ helm install stash-operator appscode/stash \
  --version v0.9.0-rc.6 \
  --namespace kube-system

Using YAML

If you prefer to not use Helm, you can generate YAMLs from Stash chart and deploy using kubectl:

$ helm repo add appscode https://charts.appscode.com/stable/
$ helm repo update
$ helm search repo appscode/stash --version v0.9.0-rc.6
NAME            CHART VERSION APP VERSION DESCRIPTION
appscode/stash  v0.9.0-rc.6    v0.9.0-rc.6  Stash by AppsCode - Backup your Kubernetes Volumes

$ helm template stash-operator appscode/stash \
  --version v0.9.0-rc.6 \
  --namespace kube-system \
  --no-hooks | kubectl apply -f -

Installing on GKE Cluster

If you are installing Stash on a GKE cluster, you will need cluster admin permissions to install Stash operator. Run the following command to grant admin permission to the cluster.

$ kubectl create clusterrolebinding "cluster-admin-$(whoami)" \
  --clusterrole=cluster-admin \
  --user="$(gcloud config get-value core/account)"

In addition, if your GKE cluster is a private cluster, you will need to either add an additional firewall rule that allows master nodes access port 8443/tcp on worker nodes, or change the existing rule that allows access to ports 443/tcp and 10250/tcp to also allow access to port 8443/tcp. The procedure to add or modify firewall rules is described in the official GKE documentation for private clusters mentioned above.

Verify installation

To check if Stash operator pods have started, run the following command:

$ kubectl get pods --all-namespaces -l app=stash --watch

NAMESPACE     NAME                              READY     STATUS    RESTARTS   AGE
kube-system   stash-operator-859d6bdb56-m9br5   2/2       Running   2          5s

Once the operator pods are running, you can cancel the above command by typing Ctrl+C.

Now, to confirm CRD groups have been registered by the operator, run the following command:

$ kubectl get crd -l app=stash

NAME                                 AGE
recoveries.stash.appscode.com        5s
repositories.stash.appscode.com      5s
restics.stash.appscode.com           5s

With this, you are ready to take your first backup using Stash.

Configuring Auto Backup for Database

To keep everything isolated, we are going to use a separate namespace called demo throughout this tutorial.

$ kubectl create ns demo
namespace/demo created

Prepare Backup Blueprint

We are going to use GCS Backend to store the backed up data. You can use any supported backend you prefer. You just have to configure Storage Secret and spec.backend section of BackupBlueprint to match with your backend. Visit here to learn which backends are supported by Stash and how to configure them.

For GCS backend, if the bucket does not exist, Stash needs Storage Object Admin role permissions to create the bucket. For more details, please check the following guide.

Create Storage Secret:

At first, let’s create a Storage Secret for the GCS backend,

$ echo -n 'changeit' > RESTIC_PASSWORD
$ echo -n '<your-project-id>' > GOOGLE_PROJECT_ID
$ mv downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY
$ kubectl create secret generic -n demo gcs-secret \
    --from-file=./RESTIC_PASSWORD \
    --from-file=./GOOGLE_PROJECT_ID \
    --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY
secret/gcs-secret created

Create BackupBlueprint:

Next, we have to create a BackupBlueprint CRD with a blueprint for Repository and BackupConfiguration object.

Below is the YAML of the BackupBlueprint object that we are going to create:

apiVersion: stash.appscode.com/v1beta1
kind: BackupBlueprint
metadata:
  name: postgres-backup-blueprint
spec:
  # ============== Blueprint for Repository ==========================
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/${TARGET_NAMESPACE}/${TARGET_APP_RESOURCE}/${TARGET_NAME}
    storageSecretName: gcs-secret
  # ============== Blueprint for BackupConfiguration =================
  task:
    name: postgres-backup-${TARGET_APP_VERSION}
  schedule: "*/5 * * * *"
  retentionPolicy:
    name: 'keep-last-5'
    keepLast: 5
    prune: true

Note that we have used few variables (format: ${<variable name>}) in the spec.backend.gcs.prefix field. Stash will substitute these variables with values from the respective target. To learn which variables you can use in the prefix field, please visit here.

Let’s create the BackupBlueprint that we have shown above.

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/backupblueprint.yaml
backupblueprint.stash.appscode.com/postgres-backup-blueprint created

With this, automatic backup is configured for PostgreSQL database. We just have to add an annotation to the AppBinding of the targeted database.

Required Annotation for Auto-Backup Database:

You have to add the following annotation to the AppBinding CRD of the targeted database to enable backup for it:

stash.appscode.com/backup-blueprint: <BackupBlueprint name>

This annotation specifies the name of the BackupBlueprint object where a blueprint for Repository and BackupConfiguration has been defined.

Prepare Databases

Next, we are going to deploy two sample PostgreSQL databases of two different versions using KubeDB. We are going to backup these two databases using auto-backup.

Deploy First PostgreSQL Sample:

Below is the YAML of the first Postgres CRD:

apiVersion: kubedb.com/v1alpha1
kind: Postgres
metadata:
  name: sample-postgres-1
  namespace: demo
spec:
  version: "11.2"
  storageType: Durable
  storage:
    storageClassName: "standard"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  terminationPolicy: Delete

Let’s create the Postgres we have shown above:

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/sample-postgres-1.yaml
postgres.kubedb.com/sample-postgres-1 created

KubeDB will deploy a PostgreSQL database according to the above specification and it will create the necessary secrets and services to access the database. It will also create an AppBinding CRD that holds the necessary information to connect with the database.

Verify that an AppBinding has been created for this PostgreSQL sample:

$ kubectl get appbinding -n demo
NAME                AGE
sample-postgres-1   47s

If you view the YAML of this AppBinding, you will see it holds service and secret information. Stash uses this information to connect with the database.

$ kubectl get appbinding -n demo sample-postgres-1 -o yaml

apiVersion: appcatalog.appscode.com/v1alpha1
kind: AppBinding
metadata:
  name: sample-postgres-1
  namespace: demo
  ...
spec:
  clientConfig:
    service:
      name: sample-postgres-1
      path: /
      port: 5432
      query: sslmode=disable
      scheme: postgresql
  secret:
    name: sample-postgres-1-auth
  secretTransforms:
  - renameKey:
      from: POSTGRES_USER
      to: username
  - renameKey:
      from: POSTGRES_PASSWORD
      to: password
  type: kubedb.com/postgres
  version: "11.2"

Deploy Second PostgreSQL Sample:

Below is the YAML of the second Postgres object:

apiVersion: kubedb.com/v1alpha1
kind: Postgres
metadata:
  name: sample-postgres-2
  namespace: demo
spec:
  version: "10.6-v2"
  storageType: Durable
  storage:
    storageClassName: "standard"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  terminationPolicy: Delete

Let’s create the Postgres we have shown above.

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/sample-postgres-2.yaml
postgres.kubedb.com/sample-postgres-2 created

Verify that an AppBinding has been created for this PostgreSQL database:

$ kubectl get appbinding -n demo
NAME                AGE
sample-postgres-1   2m49s
sample-postgres-2   10s

Here, we can see AppBinding sample-postgres-2 has been created for our second PostgreSQL sample.

Backup

Next, we are going to add auto-backup specific annotation to the AppBinding of our desired database. Stash watches for AppBinding CRD. Once it finds an AppBinding with auto-backup annotation, it will create a Repository and a BackupConfiguration CRD according to respective BackupBlueprint. Then, rest of the backup process will proceed as normal database backup as described here.

Backup First PostgreSQL Sample

Let’s backup our first PostgreSQL sample using auto-backup.

Add Annotations:

At first, add the auto-backup specific annotation to the AppBinding sample-postgres-1:

$ kubectl annotate appbinding sample-postgres-1 -n demo --overwrite \
  stash.appscode.com/backup-blueprint=postgres-backup-blueprint

Verify that the annotation has been added successfully:

$ kubectl get appbinding -n demo sample-postgres-1 -o yaml

apiVersion: appcatalog.appscode.com/v1alpha1
kind: AppBinding
metadata:
  annotations:
    stash.appscode.com/backup-blueprint: postgres-backup-blueprint
  name: sample-postgres-1
  namespace: demo
  ...
spec:
  clientConfig:
    service:
      name: sample-postgres-1
      path: /
      port: 5432
      query: sslmode=disable
      scheme: postgresql
  secret:
    name: sample-postgres-1-auth
  secretTransforms:
  - renameKey:
      from: POSTGRES_USER
      to: username
  - renameKey:
      from: POSTGRES_PASSWORD
      to: password
  type: kubedb.com/postgres
  version: "11.2"

Following this, Stash will create a Repository and a BackupConfiguration CRD according to the blueprint.

Verify Repository:

Verify that the Repository has been created successfully by the following command:

$ kubectl get repository -n demo
NAME                         INTEGRITY   SIZE   SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1                                                                2m23s

If we view the YAML of this Repository, we are going to see that the variables ${TARGET_NAMESPACE}, ${TARGET_APP_RESOURCE} and ${TARGET_NAME} has been replaced by demo, postgres and sample-postgres-1 respectively.

$ kubectl get repository -n demo postgres-sample-postgres-1 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: Repository
metadata:
  creationTimestamp: "2019-08-01T13:54:48Z"
  finalizers:
  - stash
  generation: 1
  name: postgres-sample-postgres-1
  namespace: demo
  resourceVersion: "50171"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/repositories/postgres-sample-postgres-1
  uid: ed49dde4-b463-11e9-a6a0-080027aded7e
spec:
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/demo/postgres/sample-postgres-1
    storageSecretName: gcs-secret

Verify BackupConfiguration:

Verify that the *BackupConfiguration CRD has been created by the following command:

$ kubectl get backupconfiguration -n demo
NAME                         TASK                   SCHEDULE      PAUSED   AGE
postgres-sample-postgres-1   postgres-backup-11.2   */5 * * * *            3m39s

Notice the TASK field. It denotes that this backup will be performed using postgres-backup-11.2 task. We had specified postgres-backup-${TARGET_APP_VERSION} as task name in the BackupBlueprint. Here, the variable ${TARGET_APP_VERSION} has been substituted by the database version.

Let’s check the YAML of this BackupConfiguration.

$ kubectl get backupconfiguration -n demo postgres-sample-postgres-1 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: BackupConfiguration
metadata:
  creationTimestamp: "2019-08-01T13:54:48Z"
  finalizers:
  - stash.appscode.com
  generation: 1
  name: postgres-sample-postgres-1
  namespace: demo
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: false
    kind: AppBinding
    name: sample-postgres-1
    uid: a799156e-b463-11e9-a6a0-080027aded7e
  resourceVersion: "50170"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/backupconfigurations/postgres-sample-postgres-1
  uid: ed4bd257-b463-11e9-a6a0-080027aded7e
spec:
  repository:
    name: postgres-sample-postgres-1
  retentionPolicy:
    keepLast: 5
    name: keep-last-5
    prune: true
  runtimeSettings: {}
  schedule: '*/5 * * * *'
  target:
    ref:
      apiVersion: v1
      kind: AppBinding
      name: sample-postgres-1
  task:
    name: postgres-backup-11.2
  tempDir: {}

Notice that the spec.target.ref is pointing to the AppBinding sample-postgres-1 that we have just annotated with auto-backup annotation.

Wait for BackupSession:

Now, wait for the next backup schedule. Run the following command to watch BackupSession CRD:

$ watch -n 1 kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-1

Every 1.0s: kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-1  workstation: Thu Aug  1 20:35:43 2019

NAME                                    INVOKER-TYPE          INVOKER-NAME                 PHASE       AGE
postgres-sample-postgres-1-1564670101   BackupConfiguration   postgres-sample-postgres-1   Succeeded   42s

Note: Backup CronJob creates BackupSession CRD with the following label stash.appscode.com/backup-configuration=<BackupConfiguration crd name>. We can use this label to watch only the BackupSession of our desired BackupConfiguration.

Verify Backup:

When backup session is completed, Stash will update the respective Repository to reflect the latest state of backed up data.

Run the following command to check if a snapshot has been sent to the backend:

$ kubectl get repository -n demo postgres-sample-postgres-1
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1   true        1.324 KiB   1                73s                      6m7s

If we navigate to stash-backup/demo/postgres/sample-postgres-1 directory of our GCS bucket, we are going to see that the snapshot has been stored there.

Backup Second Sample PostgreSQL

Now, lets backup our second PostgreSQL sample using the same BackupBlueprint we have used to backup the first PostgreSQL sample.

Add Annotations:

Add the auto backup specific annotation to AppBinding sample-postgres-2.

$ kubectl annotate appbinding sample-postgres-2 -n demo --overwrite \
  stash.appscode.com/backup-blueprint=postgres-backup-blueprint

Verify Repository:

Verify that the Repository has been created successfully by the following command:

$ kubectl get repository -n demo
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1   true        1.324 KiB   1                2m3s                     6m57s
postgres-sample-postgres-2                                                                     15s

Here, repository postgres-sample-postgres-2 has been created for the second PostgreSQL sample.

If we view the YAML of this Repository, we will see that the variables ${TARGET_NAMESPACE}, ${TARGET_APP_RESOURCE} and ${TARGET_NAME} have been replaced by demo, postgres and sample-postgres-2 respectively.

$ kubectl get repository -n demo postgres-sample-postgres-2 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: Repository
metadata:
  creationTimestamp: "2019-08-01T14:37:22Z"
  finalizers:
  - stash
  generation: 1
  name: postgres-sample-postgres-2
  namespace: demo
  resourceVersion: "56103"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/repositories/postgres-sample-postgres-2
  uid: df58523c-b469-11e9-a6a0-080027aded7e
spec:
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/demo/postgres/sample-postgres-2
    storageSecretName: gcs-secret

Verify BackupConfiguration:

Verify that the BackupConfiguration CRD has been created by the following command:

$ kubectl get backupconfiguration -n demo
NAME                         TASK                   SCHEDULE      PAUSED   AGE
postgres-sample-postgres-1   postgres-backup-11.2   */5 * * * *            7m52s
postgres-sample-postgres-2   postgres-backup-10.6   */5 * * * *            70s

Again, notice the TASK field. This time, ${TARGET_APP_VERSION} has been replaced with 10.6 which is the database version of our second sample.

Wait for BackupSession:

Now, wait for the next backup schedule. Run the following command to watch BackupSession CRD:

$ watch -n 1 kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-2
Every 1.0s: kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-2  workstation: Thu Aug  1 20:55:40 2019

NAME                                    INVOKER-TYPE          INVOKER-NAME                 PHASE       AGE
postgres-sample-postgres-2-1564671303   BackupConfiguration   postgres-sample-postgres-2   Succeeded   37s

Verify Backup:

Run the following command to check if a snapshot has been sent to the backend:

$ kubectl get repository -n demo postgres-sample-postgres-2
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-2   true        1.324 KiB   1                52s                      19m

If we navigate to stash-backup/demo/postgres/sample-postgres-2 directory of our GCS bucket, we are going to see that the snapshot has been stored there.

Cleanup

To cleanup the Kubernetes resources created by this tutorial, run:

kubectl delete -n demo pg/sample-postgres-1
kubectl delete -n demo pg/sample-postgres-2

kubectl delete -n demo repository/postgres-sample-postgres-1
kubectl delete -n demo repository/postgres-sample-postgres-2

kubectl delete -n demo backupblueprint/postgres-backup-blueprint

Final thoughts

You've now gotten a deep dive into setting up a Kubernetes-native disaster recovery and backup solution with Stash. You can find a lot of really helpful information on their documentation site here. More content can be found at appfleet

Enabling multicloud K8s communication with Skupper

Dmitriy A. — Tue, 15 Sep 2020 20:25:24 +0000

Intro

There are many challenges that engineering teams face when attempting to incorporate a multi-cloud approach into their infrastructure goals. Kubernetes does a good job of addressing some of these issues, but managing the communication of clusters that span multiple cloud providers in multiple regions can become a daunting task for teams. Often this requires complex VPNs and special firewall rules to multi-cloud cluster communication.

In this post, I will be introducing you to Skupper, an open source project for enabling secure communication across Kubernetes cluster. Skupper allows your application to span multiple cloud providers, data centers, and regions. Let's see it in action!

Getting Started

This tutorial will demonstrate how to distribute the Istio Bookinfo Application microservices across multiple public and private clusters. The services require no coding changes to work in the distributed application environment. With Skupper, the application behaves as if all the services are running in the same cluster.

In this tutorial, you will deploy the productpage and ratings services on a remote, public cluster in namespace aws-eu-west and the details and reviews services in a local, on-premises cluster in namespace laptop.

Overview

Figure 1 - Bookinfo service deployment

The image above shows how the services will be deployed.

Each cluster runs two of the application services.
An ingress route to the productpage service provides internet user access to the application.

If all services were installed on the public cluster, then the application would work as originally designed. However, since two of the services are on the laptop cluster, the application fails. productpage can not send requests to details or to reviews.

This demo will show how Skupper can solve the connectivity problem presented by this arrangement of service deployments.

Figure 2 - Bookinfo service deployment with Skupper

Skupper is a distributed system with installations running in one or more clusters or namespaces. Connected Skupper installations share information about what services each installation exposes. Each Skupper installation learns which services are exposed on every other installation. Skupper then runs proxy service endpoints in each namespace to properly route requests to or from every exposed service.

In the public namespace, the details and reviews proxies intercept requests for their services and forward them to the Skupper network.
In the private namespace, the details and reviews proxies receive requests from the Skupper network and send them to the related service.
In the private namespace, the ratings proxy intercepts requests for its service and forwards them to the Skupper network.
In the public namespace, the ratings proxy receives requests from the Skupper network and sends them to the related service.

Prerequisites

To run this tutorial you will need:

The kubectl command-line tool, version 1.15 or later (installation guide)
The skupper command-line tool, the latest version (installation guide)
Two Kubernetes namespaces, from any providers you choose, on any clusters you choose
The yaml files from https://github.com/skupperproject/skupper-examples-bookinfo.git
Two logged-in console terminals, one for each cluster or namespace

Step 1: Deploy the Bookinfo application

This step creates a service and a deployment for each of the four Bookinfo microservices.

Namespace aws-eu-west:

$ kubectl apply -f public-cloud.yaml
service/productpage created
deployment.extensions/productpage-v1 created
service/ratings created
deployment.extensions/ratings-v1 created

Namespace laptop:

$ kubectl apply -f private-cloud.yaml 
service/details created
deployment.extensions/details-v1 created
service/reviews created
deployment.extensions/reviews-v3 created

Step 2: Expose the public productpage service

Namespace aws-eu-west:

kubectl expose deployment/productpage-v1 --port 9080 --type LoadBalancer

The Bookinfo application is accessed from the public internet through this ingress port to the productpage service.

Step 3: Observe that the application does not work

The web address for the Bookinfo application can be discovered from namespace aws-eu-west:

$ echo $(kubectl get service/productpage -o jsonpath='http://{.status.loadBalancer.ingress[0].hostname}:9080')

Open the address in a web browser. Productpage responds but the page will show errors as services in namespace laptop are not reachable.

We can fix that now.

Step 4: Set up Skupper

This step initializes the Skupper environment on each cluster.
Namespace laptop:

skupper init

Namespace aws-eu-west:

skupper init

Now the Skupper infrastructure is running. Use skupper status in each console terminal to see that Skupper is available.

$ skupper status
Namespace '<ns>' is ready.  It is connected to 0 other namespaces.

As you move through the steps that follow, you can use skupper status at any time to check your progress.

Step 5: Connect your Skupper installations

Now you need to connect your namespaces with a Skupper connection.

This is a two step process.
The skupper connection-token <file> command directs Skupper to generate a secret token file with certificates that grant permission to other Skupper instances to connect to this Skupper's network.

Note: Protect this file as you would do for any file that holds login credentials.

The skupper connect <file> command directs Skupper to connect to another Skupper's network. This step completes the Skupper connection.

Note that in this arrangement the Skupper instances join to form peer networks. Typically the Skupper opening the network port will be on the public cluster. A cluster running on laptop may not even have an address that is reachable from the internet. After the connection is made, the Skupper network members are peers and it does not matter which Skupper opened the network port and which connected to it.

The console terminals in this demo are run by the same user on the same host. This makes the token file in the ${HOME} directory available to both terminals. If your terminals are on different machines then you may need to use scp or a similar tool to transfer the token file to the system hosting the laptop terminal.

Generate a Skupper network connection token

Namespace aws-eu-west:

skupper connection-token ${HOME}/PVT-to-PUB-connection-token.yaml

Open a Skupper connection

Namespace laptop:

skupper connect ${HOME}/PVT-to-PUB-connection-token.yaml

Check the connection

Namespace aws-eu-west:

$ skupper status
Skupper enabled for "aws-eu-west". It is connected to 1 other sites.

Namespace laptop:

$ skupper status
Skupper enabled for "laptop". It is connected to 1 other sites.

Step 6: Virtualize the services you want shared

You now have a Skupper network capable of multi-cluster communication but no services are associated with it. This step uses the kubectl annotate command to notify Skupper that a service is to be included in the Skupper network.

Skupper uses the annotation as the indication that a service must be virtualized. The service that receives the annotation is the physical target for network requests and the proxies that Skupper deploys in other namespaces are the virtual targets for network requests. The Skupper infrastructure then routes requests between the virtual services and the target service.

Namespace aws-eu-west:

$ kubectl annotate service ratings skupper.io/proxy=http
service/ratings annotated

Namespace laptop:

$ kubectl annotate service details skupper.io/proxy=http
service/details annotated

$ kubectl annotate service reviews skupper.io/proxy=http
service/reviews annotated

Skupper is now making the annotated services available to every namespace in the Skupper network. The Bookinfo application will work as the productpage service on the public cluster has access to the details and reviews services on the private cluster and as the reviews service on the private cluster has access to the ratings service on the public cluster.

Step 7: Observe that the application works

The web address for the Bookinfo app can be discovered from namespace aws-eu-west:

$ echo $(kubectl get service/productpage -o jsonpath='http://{.status.loadBalancer.ingress[0].hostname}:9080')

Open the address in a web browser. The application should now work with no errors.

Clean up

Skupper and the Bookinfo services may be removed from the clusters.

Namespace aws-eu-west:

skupper delete
kubectl delete -f public-cloud.yaml

Namespace laptop:

skupper delete
kubectl delete -f private-cloud.yaml

Final Thoughts

Enabling a multi-cloud approach has a lot of benefits and is getting easier, thanks to tools like Skupper. If you have time, try some of Skupper's other examples on its Github Repo. I hope you learned something from this post. Stay tuned for more!

Cloud-native benchmarking with Kubestone

Dmitriy A. — Mon, 14 Sep 2020 18:40:45 +0000

Intro

Organizations are increasingly looking to containers and distributed applications to provide the agility and scalability needed to satisfy their clients. While doing so, modern enterprises also need the ability to benchmark their application and be aware of certain metrics in relation to their infrastructure.

In this post, I am introducing you to a cloud-native bench-marking tool known as Kubestone. This tool is meant to assist your development teams with getting performance metrics from your Kubernetes clusters.

How does Kubestone work?

At it's core, Kubestone is implemented as a Kubernetes Operator in Go language with the help of Kubebuilder. You can find more info on the Operator Framework via this blog post.

Kubestone leverages Open Source benchmarks to measure Core Kubernetes and Application performance. As benchmarks are executed in Kubernetes, they must be containerized to work on the cluster. A certified set of benchmark containers is provided via xridge's DockerHub space. Here is a list of currently supported benchmarks:

Type	Benchmark Name	Status
Core/CPU	sysbench	Supported
Core/Disk	fio	Supported
Core/Disk	ioping	Supported
Core/Memory	sysbench	Supported
Core/Network	iperf3	Supported
Core/Network	qperf	Supported
HTTP Load Tester	drill	Supported
Application/Etcd	etcd	Planned
Application/K8S	kubeperf	Planned
Application/PostgreSQL	pgbench	Supported
Application/Spark	sparkbench	Planned

Let's try installing Kubestone and running a benchmark ourselves and see how it works.

Installing Kubestone

Requirements

Kubernetes v1.13 (or newer)
Kustomize v3.1.0
Cluster admin privileges

Deploy Kubestone to kubestone-system namespace with the following command:

$ kustomize build github.com/xridge/kubestone/config/default | kubectl create -f -

Once deployed, Kubestone will listen for Custom Resources created with the kubestone.xridge.io group.

Benchmarking

Benchmarks can be executed via Kubestone by creating Custom Resources in your cluster.

Namespace

It is recommended to create a dedicated namespace for benchmarking.

$ kubectl create namespace kubestone

After the namespace is created, you can use it to post a benchmark request to the cluster.

The resulting benchmark executions will reside in this namespace.

Custom Resource rendering

We will be using kustomize to render the Custom Resource from the github repository.

Kustomize takes a base yaml, and patches with an overlay file to render the final yaml file, which describes the benchmark.

$ kustomize build github.com/xridge/kubestone/config/samples/fio/overlays/pvc

The Custom Resource (rendered yaml) looks as follows:

apiVersion: perf.kubestone.xridge.io/v1alpha1
kind: Fio
metadata:
  name: fio-sample
spec:
  cmdLineArgs: --name=randwrite --iodepth=1 --rw=randwrite --bs=4m --size=256M
  image:
    name: xridge/fio:3.13
  volume:
    persistentVolumeClaimSpec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
    volumeSource:
      persistentVolumeClaim:
        claimName: GENERATED

When we create this resource in Kubernetes, the operator interprets it and creates the associated benchmark. The fields of the Custom Resource controls how the benchmark will be executed:

metadata.name: Identifies the Custom Resource. Later, this can be used to query or delete the benchmark in the cluster.
cmdLineArgs: Arguments passed to the benchmark. In this case we are providing the arguments to Fio (a filesystem benchmark). It instructs the benchmark to execute a random write test with 4Mb of block size with an overall transfer size of 256 MB.
image.name: Describes the Docker Image of the benchmark. In case of Fio, we are using xridge's fio Docker Image, which is built from this repository.
volume.persistentVolumeClaimSpec: Given that Fio is a disk benchmark, we can set a PersistentVolumeClaim for the benchmark to be executed. The above setup instructs Kubernetes to take 1GB of space from the default StorageClass and use it for the benchmark.

Running the benchmark

Now, as we understand the definition of the benchmark, we can try to execute it.

Note: Make sure you installed the kubestone operator and have it running before executing this step.

$ kustomize build github.com/xridge/kubestone/config/samples/fio/overlays/pvc | kubectl create --namespace kubestone -f -

Since we pipe the output of the kustomize build command into kubectl create, it will create the object in our Kubernetes cluster.

The resulting object can be queried using the object's type (fio) and it's name (fio-sample):

$ kubectl describe --namespace kubestone fio fio-sample
Name:         fio-sample
Namespace:    kubestone
Labels:       <none>
Annotations:  <none>
API Version:  perf.kubestone.xridge.io/v1alpha1
Kind:         Fio
Metadata:
  Creation Timestamp:  2019-09-14T11:31:02Z
  Generation:          1
  Resource Version:    31488293
  Self Link:           /apis/perf.kubestone.xridge.io/v1alpha1/namespaces/kubestone/fios/fio-sample
  UID:                 21cdbe92-d6e3-11e9-ba70-4439c4920abc
Spec:
  Cmd Line Args:  --name=randwrite --iodepth=1 --rw=randwrite --bs=4m --size=256M
  Image:
    Name:  xridge/fio:3.13
  Volume:
    Persistent Volume Claim Spec:
      Access Modes:
        ReadWriteOnce
      Resources:
        Requests:
          Storage:  1Gi
    Volume Source:
      Persistent Volume Claim:
        Claim Name:  GENERATED
Status:
  Completed:  true
  Running:    false
Events:
  Type    Reason           Age   From       Message
  ----    ------           ----  ----       -------
  Normal  Created  11s   kubestone  Created /api/v1/namespaces/kubestone/configmaps/fio-sample
  Normal  Created  11s   kubestone  Created /api/v1/namespaces/kubestone/persistentvolumeclaims/fio-sample
  Normal  Created  11s   kubestone  Created /apis/batch/v1/namespaces/kubestone/jobs/fio-sample

As the Events section shows, Kubestone has created a ConfigMap, a PersistentVolumeClaim and a Job for the provided Custom Resource. The Status field tells us that the benchmark has completed.

Inspecting the benchmark

The created objects related to the benchmark can be listed using kubectl command:

$ kubectl get pods,jobs,configmaps,pvc --namespace kubestone
NAME                   READY   STATUS      RESTARTS   AGE
pod/fio-sample-bqqmm   0/1     Completed   0          54s

NAME                   COMPLETIONS   DURATION   AGE
job.batch/fio-sample   1/1           15s        54s

NAME                   DATA   AGE
configmap/fio-sample   0      54s

NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
persistentvolumeclaim/fio-sample   Bound    pvc-b3898236-c698-11e9-8071-4439c4920abc   1Gi        RWO            rook-ceph-block   54s

As shown above, Fio controller has created a PersistentVolumeClaim and a ConfigMap which is used by the Fio Job during benchmark execution. The Fio Job has an associated Pod which contains our test execution. The results of the run can be shown with the kubectl logs command:

$ kubectl logs --namespace kubestone fio-sample-bqqmm
randwrite: (g=0): rw=randwrite, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.13
Starting 1 process
randwrite: Laying out IO file (1 file / 256MiB)

randwrite: (groupid=0, jobs=1): err= 0: pid=47: Sat Aug 24 17:58:10 2019
  write: IOPS=470, BW=1882MiB/s (1974MB/s)(256MiB/136msec); 0 zone resets
    clat (usec): min=1887, max=2595, avg=2042.76, stdev=136.56
     lat (usec): min=1953, max=2688, avg=2107.35, stdev=142.94
    clat percentiles (usec):
     |  1.00th=[ 1893],  5.00th=[ 1926], 10.00th=[ 1926], 20.00th=[ 1958],
     | 30.00th=[ 1991], 40.00th=[ 2008], 50.00th=[ 2024], 60.00th=[ 2040],
     | 70.00th=[ 2057], 80.00th=[ 2073], 90.00th=[ 2114], 95.00th=[ 2409],
     | 99.00th=[ 2606], 99.50th=[ 2606], 99.90th=[ 2606], 99.95th=[ 2606],
     | 99.99th=[ 2606]
  lat (msec)   : 2=34.38%, 4=65.62%
  cpu          : usr=2.22%, sys=97.78%, ctx=1, majf=0, minf=9
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,64,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=1882MiB/s (1974MB/s), 1882MiB/s-1882MiB/s (1974MB/s-1974MB/s), io=256MiB (268MB), run=136-136msec

Disk stats (read/write):
  rbd7: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

Listing benchmarks

We have learned that Kubestone uses Custom Resources to define benchmarks. We can list the installed custom resources using the kubectl get crds command:

$ kubectl get crds | grep kubestone
drills.perf.kubestone.xridge.io         2019-09-08T05:51:26Z
fios.perf.kubestone.xridge.io           2019-09-08T05:51:26Z
iopings.perf.kubestone.xridge.io        2019-09-08T05:51:26Z
iperf3s.perf.kubestone.xridge.io        2019-09-08T05:51:26Z
pgbenches.perf.kubestone.xridge.io      2019-09-08T05:51:26Z
sysbenches.perf.kubestone.xridge.io     2019-09-08T05:51:26Z

Using the CRD names above, we can list the executed benchmarks in the system.

Kubernetes provides a convenience feature regarding CRDs: one can use the shortened name of the CRD, which is the singular part of the fully qualified CRD name. In our case, fios.perf.kubestone.xridge.io can be shortened to fio. Hence, we can list the executed fio benchmark using the following command:

$ kubectl get --namespace kubestone fios.perf.kubestone.xridge.io
NAME         RUNNING   COMPLETED
fio-sample   false     true

Cleaning up

After a successful benchmark run the resulting objects are stored in the Kubernetes cluster. Given that Kubernetes can hold a limited number of pods in the system, it is advised that the user cleans up the benchmark runs time to time. This can be achieved by deleting the Custom Resource, which initiated the benchmark:

$ kubectl delete --namespace kubestone fio fio-sample

Since the Custom Resource has ownership on the created resources, the underlying pods, jobs, configmaps, pvcs, etc. are also removed by this operation.

Next steps

Now you are familiar with the key concepts of Kubestone, it is time to explore and benchmark. You can play around with Fio Benchmark via it's cmdLineArgs, Persistent Volume and Scheduling related settings. You can find more information about that in Fio's benchmark page. Hopefully you gained some valuable knowledge from this post!

Using Helm with Kubernetes

Dmitriy A. — Mon, 14 Sep 2020 18:39:50 +0000

Kubernetes is a powerful orchestration system, however, it can be really hard to configure its deployment process. Specific apps can help you manage multiple independent resources like pods, services, deployments, and replica sets. Yet, each must be described in the YAML manifest file.

It’s not a problem for a single trivial app, but during production, it’s best to simplify this process: search, use, and share already implemented configurations, deploy these configurations, create configuration templates, and deploy them without effort. In other words, we need an extended version of a package manager like APT for Ubuntu or PIP for Python to work with the Kubernetes cluster. Luckily, we have Helm as a package manager.

What is Helm?

Helm is an open-source package manager for Kubernetes that allows developers and operators to package, configure, and deploy applications and services onto Kubernetes clusters easily. It was inspired by Homebrew for macOS and now is a part of the Cloud Native Computing Foundation.

In this article, we will explore Helm 3.x which is the newest version at the time of writing this article.

Searches on Helm Hub for PostgreSQL from dozens of different repositories

Helm can install software and dependencies, upgrade software, configure software deployments, fetch packages from repositories, alongside managing repositories.

Some key features of Helm include:

Role-based access controls (RBAC)
Golang templates which allows you to work with configuration as text
Lua scripts to process configuration as an object
Deployment versions control system

Templates allow you to configure your deployments by changing few variable values without changing the template directly. Helm packages are called charts, and they consist of a few YAML configuration files and templates that are rendered into Kubernetes manifest files.

The basic package (chart) structure:

chart.yaml - a YAML file containing information about the chart
LICENSE (optional) - a plain text file containing the license for the chart
README.md (optional) - a human-readable README file
values.yaml - the default configuration values for this chart
values.schema.json (optional) - a JSON Schema for imposing a structure on the values.yaml file
charts/ - defines chart dependencies (recommended to use the dependencies section in chart.yaml)
crds/ - Custom Resource Definitions
templates/ - directory of templates that when combined with values, will generate valid Kubernetes manifest files

Templates give you a wide range of capabilities. You can use variables from context, apply different functions (such as ‘quote’, sha256sum), use cycles and conditional cases, and import other files (also other templates or partials).

What are Helm’s abilities?

As you operate Helm though a Command Line Interface (CLI), the helm search command allows you to search for a package by keywords from the repositories.
You can inspect chart.yaml, values.yaml, and README.md for a certain package. along with creating your own chart with the helm create <chart-name> command. This command will generate a folder with a specified name in which you can find the mentioned structure.
Helm can install both folder or .tgz archives. To create a .tgz from your package folder, use the helm package <path to folder> command. This will create a <package_name> package in your working directory, using the name and version from the metadata defined in the chart.yaml file.
Helm has built-in support for installing packages from an HTTP server. Helm reads a repository index hosted on the server, which describes what chart packages are available and where they are located. This is how the default stable repository works.
You can also create a repository from your machine with helm serve. This eventually lets you create your own corporate repository or contribute to the official stable one.
You can also call the helm dependencies update <package name> command which verifies that the required charts, as expressed in chart.yaml, are present in charts/ and are in an acceptable version. It will additionally pull down the latest charts that satisfy the dependencies, and clean up the old dependencies.
Apart from Chart and Repository another significant concept you should know is Release which is an instance of a chart running in a Kubernetes cluster. One chart can often be installed many times into the same cluster. And each time it is installed, a new Release is created. So, you can have multiple PostgreSQL in the same cluster, in which each Release will have its own release name. You can think of this like 'multiple Docker containers from one image'.

How does it work?

Source: developer.ibm.com

Helm client is used for installing, updating and creating charts, as well as compiling and sending them to a Kubernetes API in an acceptable form. The previous version had a client-server architecture, using a program run on a cluster with Kubernetes, called Tiller. This software was responsible for deployment’s lifetime. But this approach led to some security issues which is one of the reasons why all functions are now handled by the client.

Installing Helm 3 is noticeably easier than the previous version since only the client needs to be installed. It is available for Windows, macOS, and Linux. You can install the program from binary releases, Homebrew, or through a configured installation script.

Let’s try an example

1.Let's start with installing Helm.

bash master $ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100  6794  100  6794    0     0  25961      0 --:--:-- --:--:-- --:--:-- 25931Error: could not find tillerHelm v3.1.2 is available. Changing from version .Downloading https://get.helm.sh/helm-v3.1.2-linux-amd64.tar.gzPreparing to install helm into /usr/local/binhelm installed into /usr/local/bin/helm

2.Check if everything is installed properly.

master $ helm version --short
V3.1.2+gd878d4d

3.By default, Helm doesn’t have a connection to any of the repositories. Let’s add connection to the most common stable one. (You can check all the available repositories with helm repo list).

master $ helm repo add stable 

https://kubernetes-charts.storage.googleapis.com/
"stable" has been added to your repositories

4.After adding the repository, we should let Helm get updated. The current local state of Helm is kept in your environment in the home location.

master $ helm repo update

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈

The Helm command defaults to discovering the host already set in ~/.kube/config. There is a way to change or override the host, but that's beyond the scope of this scenario.

master $ helm env

HELM_BIN="helm"
HELM_DEBUG="false"
HELM_KUBECONTEXT=""
HELM_NAMESPACE="default"
HELM_PLUGINS="/root/.local/share/helm/plugins"
HELM_REGISTRY_CONFIG="/root/.config/helm/registry.json"
HELM_REPOSITORY_CACHE="/root/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/root/.config/helm/repositories.yaml"

5.Let's search for a WordPress in the Helm Hub

master $ helm search hub wordpress

URL                                                     CHART VERSION   APP VERSION     DESCRIPTION https://hub.helm.sh/charts/presslabs/wordpress-...      v0.8.4          v0.8.4          Presslabs WordPress Operator Helm Chart
https://hub.helm.sh/charts/presslabs/wordpress-...      v0.8.3          v0.8.3          A Helm chart for deploying a WordPress site on ...
https://hub.helm.sh/charts/bitnami/wordpress            9.0.3           5.3.2           Web publishing platform for building blogs and ...

And also search in our repositories (we have only stable for now).

master $ helm search repo wordpress

NAME                    CHART VERSION   APP VERSION     DESCRIPTION
stable/wordpress        9.0.2           5.3.2           DEPRECATED Web publishing platform for building...

6.As mentioned earlier, you can inspect a Chart. For example, let’s take info from chart.yaml for the Wordpress chart. You can also check helm show readme stable/wordpress and helm show values stable/wordpress.

master $ helm show chart stable/wordpress

apiVersion: v1
appVersion: 5.3.2
dependencies:
- condition: mariadb.enabled
  name: mariadb
  repository: https://kubernetes-charts.storage.googleapis.com/
  tags:
  - wordpress-database
  version: 7.x.xdeprecated: truedescription: DEPRECATED Web publishing platform for building blogs and websites.
home: http://www.wordpress.com/
icon: https://bitnami.com/assets/stacks/wordpress/img/wordpress-stack-220x234.png
keywords:- wordpress- cms
- blog
- http- web- application
- php
name: wordpress
sources:
- https://github.com/bitnami/bitnami-docker-wordpress
version: 9.0.2

7.Let’s create a namespace for WordPress and install a test chart.

master $ kubectl create namespace wordpress

namespace/wordpress created

master $ helm install test-wordpress stable/wordpress --namespace wordpress

The output of this command appears messy just because it’s so big.

You can also set variables, such as:

helm install test-wordpress \
  --set wordpressUsername=admin \
  --set wordpressPassword=password \
  --set mariadb.mariadbRootPassword=secretpassword \
    stable/wordpress

8.For now, let’s ensure that everything is deployed correctly:

As you can see, everything has been deployed properly.

Conclusion

Helm is a popular open-source package manager that offers users a more flexible way to manage Kubernetes cluster. You can either create your own, or use public packages from your own or external repositories. Each package is quite flexible and, in most cases, all you need is define the right constants from which the template will be compiled to suit your needs. To create your own chart, you can use the power of Go templates and/or Lua scripts. Each update will create a history unit to which you can rollback anytime you want. With Helm, you have all the power of Kubernetes. And, in the end, Helm allows you to work with role-based access, so you can manage your cluster in a team.

This brings us to the end of this brief article explaining the basics and features of Helm. We hope you enjoyed it and were able to make use of it.

Autoscaling an Amazon Elastic Kubernetes Service cluster

Dmitriy A. — Mon, 14 Sep 2020 18:39:11 +0000

In this article we are going to consider the two most common methods for Autoscaling in EKS cluster:

Horizontal Pod Autoscaler (HPA)
Cluster Autoscaler (CA)

The Horizontal Pod Autoscaler or HPA is a Kubernetes component that automatically scales your service based on metrics such as CPU utilization or others, as defined through the Kubernetes metric server. The HPA scales the pods in either a deployment or replica set, and is implemented as a Kubernetes API resource and a controller. The Controller Manager queries the resource utilization against the metrics specified in each horizontal pod autoscaler definition. It obtains the metrics from either the resource metrics API for per pod metrics or the custom metrics API for any other metrics.

To see this in action, we are going to configure HPA and then apply some load to our system to see it in action.

To start with, let us start with installing Helm as a package manager for Kubernetes.

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > helm.sh
 chmod +x helm.sh
 ./helm.sh

Now, we are going to set up the server base portion of Helm called Tiller. This requires a service account:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

The above defines a Tiller service account to which we have assigned the cluster admin role. Now let's go ahead and apply the configuration:

kubectl apply -f tiller.yml

Run helm init using the Tiller service account we have just created:

helm init --service-account tiller

With this we have installed Tiller onto the cluster, which gives access to manage those resources within it.

With Helm installed, we can now deploy the metric server. Metric servers are cluster wide aggregators of resource usage data where metrics are collected by kubelet on each worker node, and are used to dictate the scaling behavior of deployments.

So let's go ahead and install that now:

helm install stable/metrics-server --name metrics-server --version 2.0.4 --namespace metrics

Once all checks have passed, we are ready to scale the application.

For the purpose of this article, we will deploy a special build of Apache and PHP designed to generate CPU utilization:

kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80

Now, let us autoscale our deployment:

kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10

The above specifies that the HPA will increase or decrease the number of replicas to maintain an average CPU utilization across all pods by 50%. Since each pod requests 200 millicores (as specified in the previous command), the average CPU utilization of 100 millicores is maintained.

Let's check the status:

kubectl get hpa

Review Targets column, if it says unknown/50% then it means that the current CPU consumption is 0%, as we are not currently sending any request to the server. This will take a couple of minutes to show the correct value, so let us grab a cup of coffee and come back when we have got some data here.

Rerun the last command and confirm that Targets column is now 0%/50%. Now, let's generate some load in order to trigger scaling by running the following :

kubectl run -i --tty load-generator --image=busybox /bin/sh

Inside this container, we are going to send an infinite number of requests to our service. If we flip back over to the other terminal, we can watch the autoscaler in action:

kubectl get hpa -w

We can watch the HPA scaler pod up from 1 to our configured maximum of 10, until the average CPU utilization is below our target of 50%. It will take about 10 minutes to run and you could see we are now having 10 replicas. If we flip back to the other terminal to terminate the load test, and flip back to the scaler terminal, we can see the HPA reduce the replica count back to the minimum.

Cluster Autoscaler

The Cluster Autoscaler is the default Kubernetes component that can scale either pods or nodes in a cluster. It automatically increases the size of an autoscaling group, so that pods can continue to get placed successfully. It also tries to remove unused worker nodes from the autoscaling group (the ones with no pods running).

The following AWS CLI command will create an Auto scaling group with minimum of one and maximum count of ten:

eksctl create nodegroup --cluster <CLUSTER_NAME> --node-zones <REGION_CODE> --name <REGION_CODE> --asg-access --nodes-min 1 --nodes 5 --nodes-max 10 --managed

Now, we need to apply an inline IAM policy to our worker nodes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeTags",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "ec2:DescribeLaunchTemplateVersions"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

This basically allows the EC2 worker nodes posting the cluster auto scaler the ability to manipulate auto scaling. Copy it and add to your EC2 IAM role.

Next, download the following file:

wget https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

And update the following line with your cluster name:

            - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<YOUR CLUSTER NAME>

Finally, we can deploy our Autoscaler:

kubectl apply -f cluster-autoscaler-autodiscover.yaml

Of course we should wait for the pods to finish creating. Once done, we can scale our cluster out. We will consider a simple nginx application with the following yaml file:

apiVersion: extensions/v1beta2
kind: Deployment
metadata:
  name: nginx-scale
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        resources: 
          limits:
            cpu: 500m
            memory: 512Mi
          requests:
            cpu: 500m
            memory: 512Mi

Let's go ahead and deploy the application:

kubectl apply -f nginx.yaml

And check the deployment:

kubectl get deployment/nginx-scale

Now, let's scale a replica up to 10:

kubectl scale --replicas=10 deployment/nginx-scale

We can see our some pods in the pending state, which is the trigger that the cluster auto scaler uses to scale out our fleet of EC2 instances.

kubectl get pods -o wide --watch

Conclusion

In this article, we considered both types of EKS cluster autoscaling. We learnt how the Cluster Autoscaler initiates scale-in and scale-out operations each time it detects under-utilized instances or pending pods. Horizontal Pod Autoscaler and Cluster Autoscaler are essential features of Kubernetes when it comes to scaling a microservice application. Hope you found this article useful but there is more to come. Till then, happy scaling!

Optimize Ghost Blog Performance Including Rewriting Image Domains to a CDN

Dmitriy A. — Thu, 10 Sep 2020 18:23:33 +0000

The Ghost blogging platform offers a lean and minimalist experience. And that's why we love it. But unfortunately sometimes, it can be too lean for our requirements.

Web performance has become more important and relevant than ever, especially since Google started including it as a parameter in its SEO rankings. We make sure to optimize our websites as much as possible, offering the best possible user experience. This article will walk you through the steps you can take to optimize a Ghost Blog's performance while keeping it lean and resourceful.

When we started working on the appfleet blog we began with a few simple things:

Ghost responsive images

The featured image in a blog have lots of parameters, which is a good thing. For example, you can set multiple sizes in package.json and have Ghost automatically resize them for a responsive experience for users on mobile devices or smaller screens.

"config": {
        "posts_per_page": 10,
        "image_sizes": {
            "xxs": {
                "width": 30
            },
            "xs": {
                "width": 100
            },
            "s": {
                "width": 300
            },
            "m": {
                "width": 600
            },
            "l": {
                "width": 900
            },
            "xl": {
                "width": 1200
            }
                 }
}

And then, all you have to do is update the theme's code

<img class="feature-image"
    srcset="{{img_url feature_image size="s"}} 300w,
            {{img_url feature_image size="m"}} 600w,
            {{img_url feature_image size="l"}} 900w,
            {{img_url feature_image size="xl"}} 1200w"
    sizes="800px"
    src="{{img_url feature_image size="l"}}"
    alt="{{title}}"
/>

Common HTML tags for performance

Next we take a few simple steps to optimize Asset Download Time. That includes adding preconnect and preload headers in default.hbs:

<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin="anonymous">
<link rel="preconnect" href="https://cdn.jsdelivr.net/" crossorigin="anonymous">
<link rel="preconnect" href="https://widget.appfleet.com/" crossorigin="anonymous">

<link rel="preload" as="style" href="https://fonts.googleapis.com/css?family=Red+Hat+Display:400,500,700&display=swap" />
<link rel="preload" as="style" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@5.13.0/css/all.min.css" />

As we load many files from jsDelivr to improve our performance, we instruct the browser to establish a connection with the domain as soon as possible. Same goes for Google Fonts and the sidebar widget that was custom coded.

Most often than not, users coming from Google or some other source to a specific blog post will navigate to the homepage to check what else we have written. For the same reason, on blog posts we also added prefetch and prerender tags for the main blog page.

That way the browser will asynchronously download and cache it, making the next most probable action of the user almost instant:

<link rel="prefetch" href="https://appfleet.com/blog">
<link rel="prerender" href="https://appfleet.com/blog">

Now these optimizations definitely helped but we still had a big problem. Our posts often have many screenshots and images in them, eventually impacting the page load time.

To solve this problem we took two steps. Lazy load the images and use a CDN. The issue is that Ghost doesn't allow to modify or filter the contents of the post. All you can do is output the HTML.

The easiest solution to this is to use a dynamic content CDN like Cloudflare. A CDN will proxy the whole site, won't cache the HTML, but cache all static content like images. They also have an option to lazy load all images by injecting their own Javascript.

But we didn't want to use Cloudflare in this case. And didn't feel like injecting third-party JS to lazy load the images either. So what did we do?

Nginx to the rescue!

Our blog is hosted on a DigitalOcean droplet created using its marketplace apps. It's basically an Ubuntu VM that comes pre-installed with Node.js, NPM, Nginx and Ghost.

Note that even if you don't use DigitalOcean, you are still recommended to use Nginx in-front of the Node.js app of Ghost.

This eventually makes the solution pretty simple. We use Nginx to rewrite the HTML, along with enabling a CDN and lazy-loading images at the same time, without any extra JS.

For CDN, you may also use the free CDN offered by Google to all AMP projects. Not many people are aware that you can use it as a regular CDN without actually implementing AMP.

All you have to do is use this URL in front of your images:

https://appfleet-com.cdn.ampproject.org/i/s/appfleet.com/

Replace the domains with your own and change your <img> tags, and you are done. All images are now served through Google's CDN.

The best part is that the images are not only served but optimized as well. Additionally, it will even serve a WebP version of the image when possible, further improving the performance of your site.

As for lazy loading, you may use the native functionality of modern browsers that looks like this <img loading="lazy". By adding loading="lazy" to all images, you instruct the browsers to automatically lazy load them once they become visible by the user.

And now the code itself to achieve this:

server {
    listen 80;

    server_name NAME;

    location ^~ /blog/ {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host       "appfleet.com";
        proxy_set_header        X-Forwarded-Proto https;
        proxy_pass http://127.0.0.1:2368;
        proxy_redirect off;

        #disable compression 
        proxy_set_header Accept-Encoding "";
        #rewrite the html
        sub_filter_once off;
        sub_filter_types text/html;
        sub_filter '<img src="https://appfleet.com' '<img loading="lazy" src="https://appfleet-com.cdn.ampproject.org/i/s/appfleet.com';
    }

}

First we disable compression between node.js and nginx. Otherwise nginx can't modify the HTML if it comes in binary form.

Next we use the sub_filter parameter to rewrite the HTML. Ghost is using absolute paths in images, so we add the beginning as well. And in 1 line enabled both the CDN and lazyloading.

Reload the config and you are good to go. Check our blog to see this in real time.

Local Kubernetes testing with KIND

Dmitriy A. — Thu, 10 Sep 2020 18:22:22 +0000

Intro

If you've spent days (or even weeks?) trying to spin up a Kubernetes cluster for learning purposes or to test your application, then your worries are over. Spawned from a Kubernetes Special Interest Group, KIND is a tool that provisions a Kubernetes cluster running IN Docker.

From the docs:

kind is a tool for running local Kubernetes clusters using Docker container "nodes".
kind is primarily designed for testing Kubernetes 1.11+, initially targeting the conformance tests.

Installing KIND

As it is built using go, you will need to make sure you have the latest version of golang installed on your machine.

According to the k8s docs, golang -v 1.11.5 is preferred. To install kind, run these commands (it takes a while):

go get -u sigs.k8s.io/kind
kind create cluster

Then confirm kind cluster is available:

kind get clusters

Setting up kubectl

Also, install the latest kubernetes-cli using Homebrew or Chocolatey.The latest Docker has Kubernetes feature but it may come with older kubectl . Check its version by running this command:

kubectl version

Make sure it shows GitVersion: "v1.14.1" or above.If you find you are running kubectl from Docker, try brew link or reorder path environment variable.

Once kubectl and kind are ready, open bash console and run these commands:

export KUBECONFIG=”$(kind get kubeconfig-path)”
kubectl cluster-info

If kind is properly set up, some information will be shown.Now you are ready to proceed. Yay!

Deploying first application

What should we deploy on the cluster? We are going to attempt deploying Cassandra since the docs have a pretty decent walk-through on it.

First of all, download cassandra-service.yaml and cassandra-statefulset.yaml for later. Then create kustomization.yaml by running two cat commands.Once those yaml files are prepared, layout them as following:

k8s-wp/
  kustomization.yaml
  mysql-deployment.yaml
  wordpress-deployment.yaml

Then apply them to your cluster:

cd k8s-wp
kubectl apply -k ./

Validating (optional)

Get the Cassandra Service.

kubectl get svc cassandra

The response is:

NAME        TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
cassandra   ClusterIP   None         <none>        9042/TCP   45s

Note that Service creation might have failed if anything else is returned. Read Debug Services for common issues.

Finishing up

That's really all you need to know to get started with KIND, I hope this makes your life a little easier and lets you play with Kubernetes a little bit more :)

Understanding Amazon Elastic Container Service for Kubernetes (EKS)

Dmitriy A. — Fri, 04 Sep 2020 18:24:14 +0000

Amazon Elastic Container Service for Kubernetes or EKS provides a Managed Kubernetes Service. Amazon does the undifferentiated heavy lifting, such as provisioning the cluster, performing upgrades and patching. Although it is compatible with existing plugins and tooling, EKS is not a proprietary AWS fork of Kubernetes in any way. This means you can easily migrate any standard Kubernetes application to EKS without any changes to your code base. You'll connect to your EKS cluster with kubectl in the same way you would have done in a self-hosted Kubernetes.

At this stage, EKS is very loosely integrated with other AWS services. This is definitely expected to change over time though, as EKS adoption increases. That, said, Kubernetes is much more popular than either Elastic Beanstalk or ECS.

Managed Control Plane

EKS provides a Managed Control Plane, which includes Kubernetes master nodes, API server and the etcd persistence layer. As part of the highly-available control plane, you get 3 masters, 3 etcd and 3 worker nodes, where AWS provisions automatic backup snapshotting of etcd nodes alongside automated scaling.

With EKS, AWS is responsible for maintaining master nodes for you by provisioning these nodes in multiple high-availability zones to maintain redundancy. So, as your workload increases, AWS will add master nodes for you. If you were running your own Kubernetes cluster, you'd have to scale it up whenever you added a worker node.

VPC Networking

EKS runs a network topology that integrates tightly with a Virtual Private Network. EKS uses a Container Network Interface plugin that integrates the standard Kubernetes overlay network with VPC networking. This plugin allows you to treat your EKS deployment as just another part of your existing AWS infrastructure. Things like network access, control list, routing tables and subnets are all available in the Kubernetes applications running in EKS.

Each pod gets an IP address on an Elastic Network Interface, where these addresses belong to the block of the subnet where the worker node is deployed. In the diagram above, you can see the IP addresses assigned to the Virtual Ethernet Adapter on each pod. These pod IP addresses are fully rideable within the VPC, and they comply with all the policies and access controls at the network level. So, things like security groups and ACL remain in effect. On each EC2 instance or worker node, Kubernetes runs a daemon set that hosts the CNI plugin. This plugin is a thin layer that communicates with the network local control point. This network local control plane maintains a pool of available IP addresses. So, when the kubelet on a node schedules a pod, it asks the CNI plugin to allocate an IP address. At this point, the CNI plugin allocates an IP, grabs secondary IP address and associates it with the pod. It then hands that configuration back to the kubelet.

EKS-Optimized AMI

This thing is based on Amazon Linux too. It comes pre-configured to work with EKS out-of-the-box. It has all the required services pre-installed including Docker, the Kubelet and AWS IAM Authenticator. When you are provisioning your EKS worker nodes with the AWS supplied cloud formation template, it launches your worker nodes with some EC2 user data script which bootstraps the nodes with configuration allowing them to join your EKS cluster automatically.

Conclusion

Amazon EKS on AWS provides a great opportunity to create a self-hosted managed Kubernetes cluster. It is also compatible with open-source Kubernetes, and can be safely migrated to any other Kubernetes instance at any time. Worth to mention, that for users who use solutions for centralized management of Kubernetes clusters, it makes sense to go with EKS instead of any other option such as ECS, since EKS exposes the same API as an open-source Kubernetes.

Understanding and building Kubernetes Custom Resource Definitions (CRDs)

Dmitriy A. — Fri, 04 Sep 2020 18:22:27 +0000

Intro

So, let's say you had a service or application that was built on an orchestration platform such as Kubernetes. In doing so, you must also address an overflowing array of architectural issues, including security, multi-tenancy, API gateways, CLI, configuration management, and logging.

Wouldn't you like to save some manpower and development time and focus on creating something unique to your problem?

Well, it just so happens that your solution lies in what's called a Custom Resource Definition, or CRD. The CRD enables engineers to plug in your own Object and application as if they were a native Kubernetes component. This is extremely powerful in creating tool and services built on Kubernetes

By doing this, you can build out the custom resources for your application as well as use Kubernetes RBAC to provide security and authentication to your application. These custom resources will be stored in the integrated etcd repository with replication and proper lifecycle management. They will also leverage all the built-in cluster management features which come with Kubernetes.

Why do I need a CRD?

The easiest way to answer this is that you might not! It really depends on your specific project and needs. The Kubernetes Docs answer this question like so:

Use a ConfigMap if any of the following apply:

There is an existing, well-documented config file format, such as a mysql.cnf or pom.xml.
You want to put the entire config file into one key of a ConfigMap.
The main use of the config file is for a program running in a Pod on your cluster to consume the file to configure itself.
Consumers of the file prefer to consume file via a Pod or an environment variable in a Pod, rather than the Kubernetes API.
You want to perform rolling updates via Deployment, etc., when the file is updated.

Note: Use a secret for sensitive data, which is similar to a ConfigMap but more secure.

Use a Custom Resource Definition (CRD or Aggregated API) if most of the following apply:

You want to use Kubernetes client libraries and CLIs to create and update a new resource.
You want top-level support from kubectl (for example: kubectl get my-object object-name).
You want to build new automation that watches for updates on the new object, and then CRUD other objects, or vice versa.
You want to write automation that handles updates to the object.
You want to use Kubernetes API conventions like .spec, .status, and .metadata.
You want the object to be an abstraction over a collection of controlled resources or a summation of other resources.

Everyone has different goals in mind and context surrounding those goals, however if your project needs a flexible way to extend Kubernetes and is trying to stick heavily to the "Kubernetes-native" way of doing things then CRDs are right up your alley. You might be asking now, what are some typical CRDs? I'm glad you asked!

Githook example CRD

This CRD is called GitHook. It defines git webhook events and a build pipeline. GitHook controller will subscribe webhook events to git repo and when the events happen, it will run a build pipeline as defined in the CRD.

GitHook CRD controller’s job is:

Make sure that webhook is registered to git repo with correct information.
Make sure that there is a service running and wait for the webhook events. It uses Knative service to receive that webhook since it is easy to implement and can scale to 0 when it is not in use.

With a little work and a good team, you could string some CRDs like this together and have a startup like CircleCI or Gitlab!

Final thoughts

So in closing, CRDs are really amazing extensions of the Kubernetes API and allow a lot of flexibility in the creation of K8s native applications. Hope this helps you wrap your head around CRDs and their uses a bit more, thanks for reading!