Forem: Surendu Suresh

Setting up kubernetes cluster in AWS - Free of cost

Surendu Suresh — Tue, 30 May 2023 04:42:25 +0000

In my previous article, I was talking about Kubernetes fundamentals and a small hands on exercise of kubernetes using Docker Desktop. So I thought of showing how to set up a basic kubernetes cluster in AWS EC2, using only free tier components. We will be using kubeadm to setup the cluster.

Prerequisites

AWS Free Tier Account Access. Use this link to set it up.
Basic knowledge of AWS
Basic knowledge of Linux

Standing up two EC2 instances

We will be using two ubuntu t2.micro instances for this purpose. Create the instance as below.

As a first step, create a security group with below inbound rules. I am enabling all ICMP and TCP traffic between the servers in the same security groups. Also, enable SSH from all sources, if you want to use SSH client to connect to the servers.

After creating the security group, click on launch instance in instance screen. I am calling first instance as kubemaster and second instance as node01.
Select t2.micro instance type and proceed without a key pair. Kubernetes recommends at least 2 CPUs and 2 GB RAM for the nodes. However, as this is just for setting up basic cluster, i am going with t2.micro instances. Also, you can choose to create key pairs if you want to connect instances using ssh clients. I will be using EC2 Instance Connect to establish connection with servers.

Click EDIT on the network settings and select the same subnet for both instances. I am selecting the first subnet from the list. This is done to ensure that the network will be open between these two instances. Select the security group we created in the first step.

Click Advanced Network Configuration and set two consecutive primary ip addresses for kubemaster and node01. Please make sure the IP address is from the subnet range. I am selecting 172.31.32.10 and 172.31.32.11 for kubemaster and node01 respectively.

Leave other fields as it is and launch the instance. Do the same steps for node01 instance.

Make sure both instance are in running state and then select one instance at a time and click on the connect button on top of the page to connect using EC2 instance connect. Or you can choose to connect using the ssh client of your choice using the ssh key.

Setting up Kubernetes cluster

Setting up instance hostname and hosts file

As a first step, we will setup the hostname for these two ec2 instances and set up /etc/hosts file to resolve both these machines.

On kubemaster instance:

root@ip-172-31-32-10:~# hostnamectl set-hostname kubemaster
root@ip-172-31-32-10:~# vi /etc/hosts

Enter below and save in hosts file. Use your specific IP addresses.

127.0.0.1 localhost kubemaster
172.31.32.10 kubemaster
172.31.32.11 node01

On node01 instance:

root@ip-172-31-32-11:~# hostnamectl set-hostname node01
root@ip-172-31-32-11:~# vi /etc/hosts

Enter below and save in hosts file. Use your specific IP addresses.

127.0.0.1 localhost node01
172.31.32.11 node01
172.31.32.10 kubemaster

Installing Container Runtime

We will be using Docker Engine for our purpose. Run below commands in both kubemaster and node01 instances.

Forwarding IPv4 and letting iptables see bridged traffic

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

Installing Docker Engine

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce

Installing cri-dockerd. This might take few minutes to complete.

git clone https://github.com/Mirantis/cri-dockerd.git

# Run these commands as root
###Install GO###
wget https://storage.googleapis.com/golang/getgo/installer_linux
chmod +x ./installer_linux
./installer_linux
source ~/.bash_profile

cd cri-dockerd
mkdir bin
go build -o bin/cri-dockerd
mkdir -p /usr/local/bin
install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
cp -a packaging/systemd/* /etc/systemd/system
sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
systemctl daemon-reload
systemctl enable cri-docker.service
systemctl enable --now cri-docker.socket

Installing kubeadm, kubelet and kubectl

These commands needs to be run on both kubemaster and node01.

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Initializing the controlplane using kubeadm

These commands needs to be run only on kubemaster

kubeadm init --pod-network-cidr=192.16.0.0/16 --apiserver-advertise-address=172.31.32.10 --cri-socket=unix:///var/run/cri-dockerd.sock --ignore-preflight-errors=NumCPU,Mem

Here I am suppressing the errors due to the low CPU and Memory of t2.micro instance as our aim is just to bring up the bare minimum cluster. Once the above command is run, controlplane will be initialized and you will get an output similar to the below.

From the output, please keep a note of the kubeadm join command somewhere for setting up node01 later.

Run the below commands from the kubeadm init output to setup kubectl configuration.

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Now you can see that the controlplane and components are running.

root@ip-172-31-32-10:~# kubectl get nodes
NAME         STATUS     ROLES           AGE     VERSION
kubemaster   NotReady   control-plane   4m46s   v1.27.2
root@ip-172-31-32-10:~# kubectl get pods -A
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-5d78c9869d-67skv             0/1     Pending   0          4m34s
kube-system   coredns-5d78c9869d-xsck7             0/1     Pending   0          4m34s
kube-system   etcd-kubemaster                      1/1     Running   0          4m48s
kube-system   kube-apiserver-kubemaster            1/1     Running   0          4m51s
kube-system   kube-controller-manager-kubemaster   1/1     Running   0          4m51s
kube-system   kube-proxy-7qfxs                     1/1     Running   0          4m34s
kube-system   kube-scheduler-kubemaster            1/1     Running   0          4m46s
root@ip-172-31-32-10:~

Installing pod network using weave-net

Run the below command using kubectl

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

We need to update the pod network cidr we provided during the kubeadm init to the Weave-net configuration. Run the below command and edit the environment variable as shown below. We need to include a environment variable named IPALLOC_RANGE and value 192.16.0.0/16. Please make sure you are editing the container named 'weave'.

kubectl edit daemonset weave-net -n kube-system

spec:
      containers:
      - command:
        - /home/weave/launch.sh
        env:
        - name: IPALLOC_RANGE
          value: 192.16.0.0/16
        - name: INIT_CONTAINER
          value: "true"
        - name: HOSTNAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: weaveworks/weave-kube:latest
        imagePullPolicy: Always
        name: weave

Joining the node01 instance to the cluster.

In node01 terminal, use the kubeadm join command we had copied before to join the node01 to the cluster. At the end of the command add the --cri-socket=unix:///var/run/cri-dockerd.sock, to select docker as the container runtime.

root@ip-172-31-32-11:~# kubeadm join 172.31.32.10:6443 --token pew42g.8wcpqxe3c2avc3ky         --discovery-token-ca-cert-hash sha256:4fb3037e7e98599763dae8f2aa62deac27139c1af96cb8b8e482590ebaaeb45c --cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[kubelet-check] Initial timeout of 40s passed.

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

We have successfully added two node cluster in AWS EC2.

root@ip-172-31-32-10:~# kubectl get nodes
NAME         STATUS   ROLES           AGE     VERSION
kubemaster   Ready    control-plane   62m     v1.27.2
node01       Ready    <none>          2m56s   v1.27.2

Please note that since we used the lowest capacity EC2 instances for the cluster, we should expect low performance on this Kubernetes cluster. But this would give an experience on how to setup the cluster using kubeadm.

Kubernetes - Getting Started

Surendu Suresh — Tue, 23 May 2023 09:40:05 +0000

When you think about DevOps, Kubernetes is one of the most important tool you need to learn. It is easy to grasp and valuable tool to have under your belt. This article will give you some basic understanding of Kubernetes and a bit of hands-on experience.

Prerequisites

Basic knowledge of Linux operating system.
Basic knowledge of container technologies like Docker.
Basic knowledge of docker registry like DockerHub.

What is Kubernetes

Kubernetes is an open-source container orchestration tool used for deploying, managing and monitoring the containerized applications. It was developed by Google and later donated to Cloud Native Computing Foundation (CNCF) which is a sub-foundation of Linux Foundation.

Kubernetes supports different container runtimes, including containerd, Docker Engine, CRI-O, Mirantis Container Runtime etc.

What are Containers, Pods and Nodes

Container is a bundling of software that packages up code and all its dependencies.
Pod is a group of one or more containers, with shared storage, network resources and a specification for how to run the containers.
Node is a physical or virtual machine which runs the workload Kubernetes places in it.

You can think of Kubernetes as a ship carrying lot of containers on it. In reality, Kubernetes cluster, is a group of physical or virtual machines, which can run and manage multiple containers in the form of Pods.

So, let's look at the architecture of Kubernetes and it's building blocks.

Control Plane Components

kube-apiserver

The API Server acts as a front end for the Kubernetes. It manages all the interactions between all other components. You can run multiple instances of kube-apiserver by deploying more instances.

etcd

etcd is a key value store used by Kubernetes to store all data related to the Kubernetes cluster. You can backup the cluster by taking backup of the etcd data.

kube-scheduler

kube-scheduler is responsible for watching for unassigned pods and scheduling it in one of the available nodes. The node is selected based on the resource requirements of the pod and the resource availability in the nodes among other things.

kube-controller-manager

This component runs all the controller processes. There are many controller processes, Node controller, Job controller, Replication Controller, etc.

cloud-controller-manager

This is an optional component which helps in embedding cloud specific control logic. It lets you link the cluster to your cloud provider.

DNS

Cluster DNS is a DNS server which stores a DNS record for Kubernetes services. Each new services and pods created in Kubernetes has an entry in the DNS.

Node Components

Node components runs on every node and responsible for managing the pods.

kubelet

kubelet is responsible for running the containers in the pod. It is responsible for restarting any crashed pods and making sure the desired amount of pods are running as per the specification.

kube-proxy

kube-proxy maintains networks rules on nodes. It enables communication between pods and from outside your cluster.

Container runtime

Container runtime is responsible for running the container inside the pod. Kubernetes supports various container runtimes which follows the Kubernetes Container Runtime Interface (CRI).

Playing with Kubernetes

Enough with the theory! Let's get started running a Kubernetes cluster on your laptop. Easiest way to get your hands on Kubernetes is to install Docker Desktop application.

Docker Desktop Installation For Windows

Above link will guide you through the steps to install Docker Desktop for Windows. Similar guides are available for Mac and Linux OS as well. For Windows, we need to enable WSL 2 (Windows Subsystem for Linux).

Once Docker Desktop is installed, go to Settings -> Kubernetes -> Enable Kubernetes as below. Wait for the Kubernetes to be up.

kubectl

kubectl is the Command Line Interface (cli) tool used for managing all the operations of Kubernetes cluster. It can be used for monitoring the nodes, pods, services etc. It can create, modify and delete pods, deployments, services etc.

Open Ubuntu for Windows or Terminal to run the below commands once Kubernetes is up.

For example:

To list all the nodes in the cluster

ubuntu: ~ > kubectl get nodes
NAME             STATUS   ROLES           AGE   VERSION
docker-desktop   Ready    control-plane   27d   v1.25.4

Above shows, there is only one node in the cluster named docker-desktop. It is in ready status and version is 1.25.4.

To list all the pods in all namespaces

ubuntu: ~ > kubectl get pods --all-namespaces
NAMESPACE     NAME                                     READY   STATUS    RESTARTS         AGE
kube-system   coredns-565d847f94-f7k2x                 1/1     Running   4 (6d1h ago)     27d
kube-system   coredns-565d847f94-s78sp                 1/1     Running   4 (6d1h ago)     27d
kube-system   etcd-docker-desktop                      1/1     Running   4 (6d1h ago)     27d
kube-system   kube-apiserver-docker-desktop            1/1     Running   4 (6d1h ago)     27d
kube-system   kube-controller-manager-docker-desktop   1/1     Running   4 (6d1h ago)     27d
kube-system   kube-proxy-9ll9n                         1/1     Running   4 (6d1h ago)     27d
kube-system   kube-scheduler-docker-desktop            1/1     Running   4 (6d1h ago)     27d
kube-system   storage-provisioner                      1/1     Running   68 (6d1h ago)    27d
kube-system   vpnkit-controller                        1/1     Running   1040 (12m ago)   27d

Hope you are seeing few familiar names above. All the controlplane components are in kube-system namespace. Namespace is a logical separator for the objects we create in Kubernetes.

Run an nginx server at port 80

ubuntu: ~ > kubectl run nginx --image=nginx --port=80
pod/nginx created
ubuntu: ~ > kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          10s

kubectl run command creates a new pod with the docker image provided by --image argument.

To access the nginx container in the pod from outside.

ubuntu: ~ > kubectl port-forward nginx 30081:80
Forwarding from 127.0.0.1:30081 -> 80
Forwarding from [::1]:30081 -> 80
Handling connection for 30081
Handling connection for 30081

This is for local testing only. After the above command is run, if you access http://localhost:30081, you will get the nginx default page as below.

Delete the nginx pod

ubuntu: ~ > kubectl delete pod nginx
pod "nginx" deleted
ubuntu: ~ >

Creating a deployment using nginx image

ubuntu: ~ > kubectl create deployment nginx --image=nginx --replicas=2
deployment.apps/nginx created
ubuntu: ~ > kubectl get all
NAME                        READY   STATUS    RESTARTS   AGE
pod/nginx-76d6c9b8c-vz677   1/1     Running   0          8s
pod/nginx-76d6c9b8c-zw999   1/1     Running   0          8s

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   27d

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx   2/2     2            2           8s

NAME                              DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-76d6c9b8c   2         2         2       8s
ubuntu: ~ >

Here we are creating a new deployment with the nginx image. Number of replicas are mentioned using --replicas argument. You can see there are 2 pods created by this deployment. Replicaset is another object created by the deployment to control the replication process.

Scaling up deployment

ubuntu: ~ > kubectl scale deployment nginx --replicas=3
deployment.apps/nginx scaled
ubuntu: ~ > kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-76d6c9b8c-56llc   1/1     Running   0          8s
nginx-76d6c9b8c-vz677   1/1     Running   0          3m11s
nginx-76d6c9b8c-zw999   1/1     Running   0          3m11s

This command created an additional pod for nginx

Scaling down deployment

ubuntu: ~ > kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled
ubuntu: ~ > k get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-76d6c9b8c-zw999   1/1     Running   0          4m28s

This command deleted two replicas of nginx and kept only one.

This is just a brief introduction to the world of Kubernetes. You can explore further on your own using the Docker Desktop setup or you can use one of the cloud providers, which is paid service.

Happy learning!!