Forem: Harish

From Code to Cloud: A Hands-On DevOps Journey with Kubernetes Deployment

Harish — Sat, 20 Jan 2024 08:02:42 +0000

Welcome to a hands-on journey through the realms of DevOps! In this blog, we'll dive into the practical world of code deployment, exploring the seamless integration of development and operations that powers modern software delivery. Join me as we unravel the intricacies of a real-world DevOps assignment, focusing on Kubernetes deployment. From transforming code into containerized brilliance to orchestrating it in the cloud, this blog is your guide to mastering the practical aspects of DevOps. Ready to embark on a journey that bridges code, clouds, and containers? Let's dive in!

DevOps Workshop 2

You should try the workshop on your own first.

Workshop Requirements

Source Code:
Version Control:
Continuous Integration (CI):
Containerization:
Continuous Deployment (CD):
Monitoring (Optional):

Follow these steps, explaining your choices and demonstrating the flow. This assignment will showcase your skills in setting up a CI/CD pipeline and deploying applications to Kubernetes. If you have any questions along the way, feel free to ask!

First, build a simple Web server

You can choose the language for this task, I am using the Go language.

Create a main.go file.

touch main.go

go mod init webserver

package mainimport ("fmt""net/http")func main() { fs := http.FileServer(http.Dir("./static")) http.Handle("/", fs) port := 3000 fmt.Printf("Server is listening on 3000") err := http.ListenAndServe(fmt.Sprintf(":%d", port), nil) if err != nil { fmt.Printf("Error: ", err) }}

This server is just serving a index.html and main.js file.

Create a static directory for html and js files.

mkdir statictouch index.html main.js

Run the website

go run main.go

Go to 127.0.0.1:3000

Containerize your application using Docker

We use docker to containerize our application, let's write the Dockerfile.

We are going to use the Multistage Docker build or Distroless Images.

Must read about Multi-Stage Docker Build of GO web server

# Build stageFROM golang:latest as buildWORKDIR /appCOPY go.mod ./RUN go mod download# Copy the necessary files of applicationCOPY main.go .COPY static ./static# build the binary of app named "main"RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .# 2nd stage, run the binary of applicationFROM scratch# Copy "main" binary and static folder into current dirCOPY --from=build /app/main .COPY --from=build /app/static ./staticEXPOSE 3000# execute the binaryCMD ["./main"]

Build the Docker Image

docker build -t memesoftheday-image .

Build & run the container and check whether the application is running or not.

docker container run -d -p 4000:3000 --name memes-container memesoftheday-image

Version Control

Use Git to control the version of source code and push it to GitHub.

Continuous Integration (CI)

If you are a beginner to CI, so understand the problem first

The Image of the application is present on your machine and let's say we are deploying our application on AWS or Azure -

We don't like manual things, we are Engineers, lets Automate it using CI pipeline.

What's the solution?

We build a Ci pipeline which uses our Dockerfile from the repository to build the image and push it to a registry.

so, you don't have to worry about building images each time you make changes.
and it uses CI server computing to build the image.

There are so many CI tools like Jenkins, Github Actions and many more. I think GitHub Actions is the right choice for our requirements.

Understand the workflow of the CI pipeline

Check out the Github repository.
Build the docker image using Dockerfile.
Log in to Docker Hub.
Push the image to Docker Hub.

Read about using Github Actions

Manage Secrets
Github Actions Officials(https://docs.github.com/en/actions/publishing-packages/publishing-docker-images)
Create a .github/workflows directory.

mkdir -p .github/workflows && cd .github/workflows

It triggers the build-on code push in the main branch.

name: Create and build docker image to Docker Hubon: push: branches: ["main"]env: REGISTRY: docker.io IMAGE_NAME: "memesoftheday-image"jobs: push_to_registry: name: Push Image to Docker Hub runs-on: ubuntu-latest steps: - name: checkout the repository uses: actions/checkout@v4 - name: Login to Docker Hub uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a with: username: ${{secrets.DOCKERHUB_USERNAME}} password: ${{secrets.DOCKERHUB_PASSWORD}} - name: Extract metadata (tags, labels) for Docker id: meta uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7 with: images: ${{secrets.DOCKERHUB_USERNAME}}/memesoftheday-image - name: Build & push the image uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671 with: context: . file: ./Dockerfile push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }}

Continuous Deployment (CD)

We want to reflect the changes we made in our code into the applications which we deploy on Kubernetes, which comes into the picture of the CD.

We are going to use the ArgoCD , so install the ArgoCD as an operator on our k8s cluster.

Create Kubernetes Cluster We use Minikuebe as local k8s cluster Install Minikube
Start Minikube cluster

minikube start --driver=docker

Install Kubectl
Install ArgoCD operator
Create a new Argo CD cluster with the default configuration using this manifest file.

mkdir k8stouch argocd.yml

apiVersion: argoproj.io/v1alpha1kind: ArgoCDmetadata: name: example-argocd labels: example: basicspec: {}

ArgoCD is created as a service, you can verify using

kubectl get svc

Now if you see that it is created as ClusterIP , which can access only within the cluster, so, to access ArgoCD in our browser we have to change its type to NodePort.

kubectl edit svc example-argocd-server

change the type from Cluster IP to NodePort

Check the service again, and check its port - 31080

kubectl get svc

example-argocd-server NodePort 10.105.171.247 <none> 80:31080/TCP,443:32101/TCP 21h

So, to access the argocd on the browser using node port, we need the IP of the cluster

minikube ip

Visit the URL - http://minikubeip:port

In my case, it is 192.168.49.2:31080

Username is admin.
Password is stored as a secret.

kubectl get secrets

you'll get example-argocd-cluster, password is stored in this secret.

kubectl edit secrets example-argocd-cluster

Copy the password.

Minikube uses a simple encryption algorithm base64.

echo dkxpeEEzeWVnYk53NEZrU1VtdW4xUlBhOHBoTWNyN0k= | base64 -d

Create Deployment and service for your application.

Create Deployment.yml in k8s directory.

# This file describes a Kubernetes Deployment, which manages a set of pods.apiVersion: apps/v1# Specifies the Kubernetes API version used for this resource.kind: Deployment# Indicates that this is a Deployment resource, used to manage application deployments.metadata: name: memesoftheday-deployment # Assigns a name to the Deployment, in this case, "memesoftheday-deployment." labels: app: memesoftheday # Labels are used to organize and select resources. This label is named "app" and set to "memesoftheday."spec: replicas: 3 # Specifies that the desired number of replicas (pods) is 3. selector: matchLabels: app: memesoftheday # Defines how the Deployment identifies which pods to manage based on labels. template: metadata: labels: app: memesoftheday # Labels assigned to the pods created by this Deployment. spec: containers: - name: memesostheday-container # Specifies the name of the container within the pod. image: harisheoran/memesoftheday-image:main # Specifies the Docker image to be used for the container, pulled from "harisheoran/memesoftheday-image" with the "main" tag. ports: - containerPort: 3000 # Specifies that the container within the pod will listen on port 3000. imagePullPolicy: If

Create service.yml in the same directory.

# This file describes a Kubernetes Service, which exposes pods to the network.apiVersion: v1# Specifies the Kubernetes API version used for this resource.kind: Service# Indicates that this is a Service resource, used to expose pods.metadata: name: memesoftheday-service # Assigns a name to the Service, in this case, "memesoftheday-service."spec: type: NodePort # Specifies that the Service should be of type NodePort, making it accessible externally on each node. selector: app: memesoftheday # Specifies the labels used to select pods that this service will route traffic to. In this case, pods with the label "app: memesoftheday." ports: - port: 80 # Specifies that the Service should be accessible on port 80 externally. targetPort: 3000 # Specifies that incoming traffic on port 80 should be forwarded to the pods on port 3000. nodePort: 30007 # Specifies a static port on each node (accessible externally) where the Service will be available, set to 30007.

and create a new app and fill in the details accordingly to your repo in the below format.

Please wait for it, it will create the pods of our application defined in our deployment.

We successfully deployed the application pods.

You can check the pods using command.

kubectl get pods -o wide

Check the port of pods.

kubectl get svc

visit your minikube ip with node port, in my case it is - http://192.168.49.2:30007/

Monitoring

To monitor our k8s cluster, we use Prometheus and Grafana.

Install Prometheus using Helm charts

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

helm repo update

helm install prometheus prometheus-community/prometheus

Now, it will create Prometheus pod and service but it is available as Cluster IP, to access the Prometheus, we need to change its service to NodePort.

kubectl expose service prometheus-server --type=NodePort --target-port=9090 --name=prometheus-server-ext

Check it port using the command.

kubectl get svc

Visit this port in browser with minikube IP.

To view these logs in graph form, we use Grafana.

Install Grafana using Helm Chart.

helm repo add grafana https://grafana.github.io/helm-charts

helm repo update

helm install grafana grafana/grafana

Expose Grafana service as NodePort to access in browser.

kubectl expose service grafana --type=NodePort --target-port=3000 --name=grafana-ext

Check the Grafana Nodeport using command and visit the port with minikube ip.

kubectl get svc

Click on it to create your first Data source.
Add a Data Source to Prometheus.
Enter the Prometheus IP address.
Go back to Home and click on Create DashBoard > Import DashBoard > Enter 3662 ID to import the template of Dashboard.
Our Grafana Dashboard is ready.

That's it, our project is done, if you have any questions about this project or if stuck at any point, feel free to ping me.

Source Code

GitHub Repository - https://github.com/harisheoran/WebServer-Docker-CI-CD-k8s
Connect with me on

Git Basics

Harish — Sun, 25 Sep 2022 07:55:29 +0000

Git is an version controller tool developed by legend Linus Torvalds, which helps Developer tracking changes in any set of files, sometimes its a life saver, believe me

Learn Git fundamentals in this short blog OR use it as a
cheatsheet for future references.

First initialize the present local directory

git init

Write some code

Stage the code to commit

Stage all files

git add .

Stage a specfic file

git add file.html

Unstage a file

$ git restore — staged name.txt

Now commit the staged code

with a commit message for future references

git commit -m "First Git commit"

We can check commited code status by

```
git status
```

Check history of your commits

```
git log
```

Upload files to your Github Account

Now connect your Github Account and upload your files.

First, we need to create a Github repository

Copy the HTTPS link of your repository

Now add remote access to your GitHub repository

git remote add origin https://github.com/sheoranharis/sheoranharis.git

**origin** is name of the link to your repo

Create a branch by which you will push your code to the repo

git checkout -b mybranch

We need remote access to our Github Account

First, generate a SSH Key -
Github Docs

then add this SSH key to your Github account -
Github Docs

OR

If you are using VS code, just login into VScode by your Github account
and you are good to go.

Push your code to Github using Git

git push origin mybranch

We are pushing our code by mybranch to origin

Check your Github account to see changes

See you in next blog.....Lets Connect

Linux System Administrator

Harish — Wed, 25 May 2022 16:40:30 +0000

Welcome to the part 2 of Journey to become Linux Wizard Part 1

User Management

User & Groups

User ID [UID] use to identify user
Group ID [GID] use to identify groups
Other users like System Daemon, Root user/Superuser, etc.

Root User

User who has complete access to the system.
Its UID is 0.
use sudo command to execute the command as the root user.
sudo su command to become a root user.
command you use as a root user, their history will not be saved so use them carefully.

Who are these other users in system other than me and root?

sudo cat /etc/passwd to see a list of all users,you'll find a lot of users but lets focus on first one to understand all this gibberish,

'root' is the username.
'x' is the password for the user, which is encrypted .and stored in /etc/shadow
'0' is the User ID.
'0' is Group ID.
User info which is the root user in our case.
user's home directory.
/bin/bash is the User's Shell.

`/etc/shadow`

type the following command to see the password details of users

'root' is the username.
'!' is an encrypted password.
'19056' is the date of the last password change since 1st Jan. 1970.
'0' is the Minimum password age.
'99999' is the Maximum password age.
'7' is the warning period to change the password.
':' is the password expiry period.
':' is the Account expiration Date.
':' is Reserved Field.

`/etc/group`

type the command

'root' is the group name.
'x' is the group password.
'0' is the group ID.
Last empty field is the list of users in this group.

How to create & delete a user?

sudo useradd nameofuser

sudo userdel nameofuser

How to change password and hostname?

sudo passwd userName

sudo hostname newUser

Permissions

Types

Read (r)
Write (w)
Execute (e)type the following command ls -l

start with the gibberish on the left sidedrwxrwxr-x

Understand this by dividing it into four separate parts.

Tells about type
- d is a directory
- '-' is a file
- l is a link
rwx are user permissions.
rwx are group permissions.
r-x are others' permission.

so, what is this 'rwx' and 'r-x' means ?

r is readability permission
w is writability permission
x is executability permission
- empty

so in the above photo, the user and group have all permissions to read, write and execute the directory but other users only have permission to read and execute

Modifying Permissions

let say i have a file named myfile.txt with permission -rw-rw-r--

change permission using chmod command

$ chmod o+w myfile.txt

o is for other users

w is to give it writing permission

+ is to add permission

- is to remove permission

Modifying Multiple Permissions

$ chmod ugo-rwx myfile.txt

ugo is for user, group & others
- is to remove permission
rwx is read, write and execution permission
we removed all three permissions for three kind of users

Numerical representation of permissions

read (r) 4
write(w) 2
execute(x) 1

to grant all permission by using numerical representation

$ chmod +777 myfile.txt

+ to add permission
7 to add all permission to user
7 to add all permission to group
7 to add all permission to other users

Numerical presentation

0 No permission
1 execution permission
2 write permission
3 write and execution permission
4 read permission
5 read and execution permission
6 read and write permission
7 read, write and execution permission

Change user ownership of file

$ sudo chown pintu myfile.txt

so ownership will transfer to the username pintu

Change group of file

$ sudo chgrp

Set User ID (SUID)

lets type a command in your terminal

$ ls -l /usr/bin/passwd

How am i able to access this data without being root user?

you can see its permission, lets break down it to understand it better

rwsr-xr-x

we have an s in user permission,s is SUID which give us ownership and execution permission.

S will only give only ownership

Add SUID to user

$ sudo chmod u+s myfile.txt

add permission to user

$ sudo chmod 4755 myfile.txt

4 is for SUID

Add Group ID (GUID)

$ sudo chmod 2755 myfile.txt

2 for GUID

Sticky Bit Permission

It grants all permission but only delete access is only for user

type the command ls -ld/tmp

you will see

drwxrwxrwt

t is Sticky Bit

Modify Sticky Bit

$ sudo chmod +t directory

$ sudo chmod 1755 directory

1 for sticky bit permission

_See you in the next blog of Linux Process _

First step toward becoming a Linux Wizard.

Harish — Sun, 15 May 2022 04:36:12 +0000

To become a Linux Wizard you must conquer Command Line Interface,

Part 1

Watch Video

Start with, what is a CLI?

It is a user interface, which doesn't have pretty graphics.
We'll use the commands to mess around.
Let's say if you want to open a folder on your PC, we'll be using commands for that, don't freak out, its easy than it seems just bear with me,
By the way, CLI uses BASH as a programming language.

CLI

here, haris is PC admin name
cosmos is name of my PC
~ is representing Home directory
$ means we're in the system shell, and ready to type commands

Understand Directories and their Path

All folders are like Tree Hierarchy, starting from Root Directory,

Basic Commands

pwd present working directory
ls check the content of the directory
cd change directory
cd.. change directory to previous directory
cd../..change directory to previous two times
cd~ change directory to Home Directory
flags are arguments/options
ls -a to list all the hidden files
ls -l show detailed versions of files
ls -la show detailed version of all hidden files
clear clear the terminal (ctrl+l)
. current directory
.. parent Directory
~home Directory
- previous Directory
mkdir make a directory
touch Create a new file
file give info about file
gedit edit file
cat view content of a file, concatenate files
historydisplays history of commands that were executed
uparrow display previous commands
ctrl + r reverse search
ctrl + cexit
Autocomplete - type first letter of the directory then press tab
cp copy files
mv move files
rm remove files
mkdir -p make a directory in the subdirectory
rm -r remove recursively
rm -rv remove recursively and display removed directory
find ~ -name find a file in home directory

or find it only with file type or only with file name

find ~ -type d -name find folder
man find the manual of any command
whatis tells about command

Now you are ready to become a Linux Wizard,
See you in Part 2

Computer Netwoking - OSI Model

Harish — Sun, 20 Feb 2022 16:00:07 +0000

OSI - Open Systems Interconnection Model

So, basically it tells us about how one computer communicate to another computer via Internet.
Overview Of OSI Model
For Deep dive into OSI Model
OSI Model contains 7 layers,

1. Application Layer

Layer where user interact with it.
Consist of application like Web Browsers, Chat Applications,

A server is a computer or system that provides resources, data, services, or programs to other computers.
so, the application has two sides: Client side & server side and they communicate through each other,
Ping measures the round trip time for messages send from the originating host to destination computer & are echoed back,

ping google.com

Web protocols;

TCP/IP:
HTTP - Hyper text transfer protocol
DHCP - Dynamic Host Control Protocol
FTP - File transfer Protocol
SMTP - Simple mail transfer Protocol
POP3 & IMAC - used to receive Email
SSH - Seceure Shell
VNC - Virtual Networking Computing
Basically it provide services for network applications with help of protocols.

2. Presentation Layer

Receive data from application layer,

-So, the data we receive is in either alphabet or digits,
first we convert that data into binary ,

then presentation layer compress the data

-and then it encrypt that data by Secure Sockets Layer (SSL).

3. Session Layer

Helps is setting up & managing connection enabling sending & receiving data followed by sessions,
Perform Authentication
Perform Authorization
Manage Sessions

4. Transport Layer

Layer 4 is responsible for end-to-end communication between the two devices.

Consist of -
1.Segmentation
2.Flow Control
3.Error Control

-Segmentation this includes taking data from the session layer and breaking it up into chunks called segments before sending it to layer,

Sequence Number - Each block of data is identified with a sequence number. Helps to reassemble segments in correct order to make correct messages/ user input.

Port Number -A port is identified for each transport protocol and address combination by a 16-bit unsigned number, known as the port number.

-Flow Control Control amount of Data being transmitted

-Error Control Error control is basically process in data link layer of detecting or identifying and re-transmitting data frames that might be lost or corrupted during transmission

-Protocol of Transport Layer

UDP (User Datagram prortocol) - Data may or may not be delivered/ changed/ ordered,

02.TCP (Transmission Control Protocol)- 100% data transfer.

05. Network Layer

Work for transmission of data from one computer to another computer which is connected to another network.
01. Logical Addressing -

06. Data Link Layer -

01. Framing a frame is a simple container for a single network packet

02.Control how data is placed and received from media

07. Physical Layer

-Physical layer is the lowest layer of the OSI reference model. It is responsible for sending bits from one computer to another. This layer is not concerned with the meaning of the bits and deals with the setup of physical connection to the network and with transmission and reception of signals.