Forem: Poonam Pawar

AWS Project using SHELL SCRIPTING for DevOps

Poonam Pawar — Wed, 31 May 2023 10:00:31 +0000

Introduction

In the real-world DevOps scenario, The AWS Resource Tracker script is widely used to provide an overview of the AWS resources being utilised within an environment.

It aims to help organisations monitor and manage their AWS resources effectively. The script utilises the AWS Command Line Interface (CLI) to fetch information about different AWS services, such as S3 buckets, EC2 instances, Lambda functions, and IAM users.

What does this project do?

By running the AWS Resource Tracker script, users can quickly obtain a list of S3 buckets, EC2 instances, Lambda functions, and IAM users associated with their AWS account.

This information can be valuable for various purposes, including auditing, inventory management, resource optimisation, and security assessment.

Build the project

Create EC2 Instance

Step 1. Go to your AWS account and log in, then search for EC2 Instances in the search bar. Or you can click on the services button that is on the top left corner of your dashboard, from there also you can search for the same.

Then click on the Launch Instance button.

Step 2. Give it a name of your choice, ubuntu as a machine image, select your key-pair or create one if not have any. Leave the instance type t2.micro ie free tier as the same and click again on the Launch Instance button.

Now you can see your instance up and running like this.

Step 3. Now click on the instance id which will give you detailed information about the running instance, there copy the public IP address.

Run the Instance

Step 4. Now open up your terminal and run the below command.

ssh -i /Users/poonampawar/Downloads/my-key-pair.pem ubuntu@ip_add

We have to go inside the directory where you have downloaded your key-pair and in my case, it is in /Downloads folder.

Check yours and give a path accordingly. And don't forget to replace ip_add with your copied one. This will take you to the virtual machine which we have created in AWS.

Create the Script

Step 5. Now create a shell script file named aws_resource_tracker.sh and copy and paste the below script.



#########################
# Author: Your Name
# Date: 28/05/23
# Version: v1
#
# This script will report the AWS resource usage
########################

set -x

# AWS S3
# AWS EC2
# AWS Lambda
# AWS IAM Users

# list s3 buckets
echo "Print list of s3 buckets"
aws s3 ls

# list EC2 Instances
echo "Print list of ec2 instances"
aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId'

# list lambda
echo "Print list of lambda functions"
aws lambda list-functions

# list IAM Users
echo "Print list of iam users"
aws iam list-users

It is a good convention to always provide your details so that it will make it easier for other developers to contribute if they want to ie, metadata.
The command set -x is given here in order to debug the script, this will print the command which we are running and then prints the output.
The commands aws s3 ls, aws lambda list-functions and aws iam list-users print the information of the given statement.

The command

aws ec2 describe instances | jq '.Reservations[].Instances[].InstanceId' will give you all the Instance IDs present in your aws in JSON format.

Test the script

Step 6. Run the below command to see the output.

./aws_resource_tracker.sh

The output will look like this.

Apply CronJob

To use cron jobs with your script, you can schedule it to run at specific intervals using the cron syntax. Here's an example of how you can modify your script to use cron jobs:

Open a terminal and run the following command to edit the
crontab file:
crontab -e
If prompted, choose your preferred text editor.

Add a new line to the crontab file to schedule your script. For example, to run the script every day at 9 AM, you can add the following line: 0 9 * * * /path/to/your/script.sh

Modify the path /path/to/your/script.sh to the actual
path where your script is located.

Save the crontab file and exit the text editor.
The above cron expression (0 9 * * *) represents the schedule: minute (0), hour (9), any day of the month, any month, and any day of the week. You can customize the schedule based on your requirements using the cron syntax.
By adding this line to the crontab file, your script will be executed automatically according to the defined schedule. The output of the script will be sent to your email address by default, or you can redirect the output to a file if needed.
Make sure that the script is executable (chmod +x /path/to/your/script.sh) and that the necessary environment variables and AWS CLI configurations are set up correctly for the script to run successfully within the cron environment.

Use Cases

Monitoring and auditing: By running this script periodically using cron jobs, you can monitor and audit your AWS resources. It provides insights into the status and details of different resources, such as S3 buckets, EC2 instances, Lambda functions, and IAM users.
Resource inventory: The script helps in maintaining an up-to-date inventory of your AWS resources. It lists the S3 buckets, EC2 instances, Lambda functions, and IAM users, allowing you to have a clear understanding of what resources exist in your environment.
Troubleshooting: In case of any issues or incidents, this script can be used to quickly gather information about the relevant AWS resources. For example, if there is an issue with an EC2 instance, you can run the script to get the instance ID and other details for further investigation.
Automation and reporting: The script can be integrated into an automated pipeline or workflow to generate regular reports about AWS resource usage. This information can be valuable for tracking costs, resource utilization, and compliance requirements.
Scalability and efficiency: In larger environments with numerous AWS resources, manually retrieving information about each resource can be time-consuming and error-prone. By using this script, you can automate the process and retrieve resource details in a consistent and efficient manner.

Overall, this script simplifies the process of gathering information about AWS resources, enhances visibility into your infrastructure, and supports effective management and monitoring of your DevOps environment.

Github link: https://github.com/Poonam1607/shell-scripting-projects

Resource: For better understanding and visual learning you can check out this tutorial - https://youtu.be/gx5E47R9fGk

This project is purely based on my learnings. It may occur error while performing in your setup. If you find any issue with it, feel free to reach out to me.

Thank you🖤!

Kubernetes Cluster Maintenance

Poonam Pawar — Tue, 30 May 2023 10:16:13 +0000

Introduction✍️

Till now we have done a lot of things!!🥹

Recap👇

Kubernetes Installation & Configurations

Kubernetes Networking, Workloads, Services

Kubernetes Storage & Security

kudos to you!👏🫡 This is literally a lot...!😮‍💨

By doing all these workloads your cluster is tiered now🤕. You have to give the energy back and make it faster. It is very important to make your cluster healthy😄 and fine.

So it's high time to understand the Kubernetes cluster maintenance stuff now.

In our today's learning, we will be covering cluster upgradation, backing up and restoring the data and scaling our Kubernetes cluster. So, let's get started!🚀

Cluster Upgradation♿

Let's say you are running your application in the Kubernetes cluster having master node and worker nodes. Pods and replicas are up and running.
Now you want to upgrade your nodes. As everyone wants to keep updated themselves and so do the nodes.
When a node is in upgradation mode, it generally goes down and is no longer in use. So you cannot keep your master and worker node together in the upgrade state.
So firstly, you will upgrade the master node. While it's upgrading, your worker node cannot deploy new pods or do any modifications. The Pods that are running already, only they will be available for the users to access.
Though the users are not going to be impacted as they still have the application up and running.
When the master node is done with the upgradation and again up and running, now we can upgrade our worker nodes. But in this also, we cannot make worker nodes goes down altogether as this may impact the users who are using the applications.
If we have three Worker nodes in our cluster, they will go one after the other in the upgrade state. When node01 goes down, the pods and replicas running in that node will shift to the other working nodes for a while ie in node02 and node03.
Then node02 will go down after node01 is upgraded and available again for the users. The pod distribution of node02 will go to node01 and node03.
And the same procedure will follow up to upgrade node03. This is how we upgrade our cluster in Kubernetes.
There is another way to upgrade the cluster, you can deploy a worker node with the updated version into your cluster and shift the workloads of the older one to the new ones then delete the older node. This is how you can achieve the upgradation.

Now let's do this practically. First, the master node:

kubeadm which is a tool for managing clusters has an upgrade command that helps in upgrading the clusters. Run:

kubeadm upgrade plan

Run the above command to see the detailed information on the upgrade plan and gives you the information of the upgrade plan if your system is needed.

Then run the drain command to make it un-schedulable:

Now install all the packages to use kubelet as it is a must in running controlplane node:

Now, Run the below command to upgrade the version:

apt-get upgrade -y kubeadm=1.12.0-00

Now to upgrade the cluster, run:

kubeadm upgrade apply v1.12.0

It will pull the necessary images and upgrade the cluster components.

Now run the below command to see the changes

systemctl restart kubelet

Now it's time to upgrade the worker node one at a time. Follow these commands:

# First move all the workloads of node-1 to the others

$ kubectl drain node-1
# this terminate all the pods from a node & reschedule them on others

$ apt-get upgrade -y kubeadm=1.12.0-00
$ apt-get upgrade -y kubelet=1.12.0-00

$ kubeadm upgrade node config --kubelet-version v1.12.0
# upgrade the node config for the new kubelet version

$ systemctl restart kubelet

# as we marked the node un-schedulable above, wee need to make schedule again
$ kubectl uncordon node-1

kubectl command will not work on the worker node, it will only work on the master node that's why after applying all the commands come back on the controlplane and make the node available again for scheduling.

Backup & Restore Methods🛗

We have till now deployed many numbers of applications in the Kubernetes cluster using pods, deployments and services.
So there are many files that are important to be backed up. Like the ETCD cluster where all the information about clusters is stored.
Persistent volumes storage is where we store the pod's data as we learned above.
You can store all these files in a source code repository like GitHub which is a good practice. In this even if you lost your whole cluster you still can deploy it again if you're using GitHub.
A better approach to back up your file is to query the kube-api server using the kubectl or by accessing the API server directly and saving all resource configurations for all objects created on the cluster as a copy.
You can also choose to back up the ETCD server itself instead of the files.
Like the screenshots on your phone, you take snapshots here of the database by using the etdctl utilities snapshot save command.

ETCDCTL_API=3 etcdctl \ snapshot save <name>

Now, if you want to restore the snapshot. First, you have to stop the server as the restore process requires the restart ETCD cluster and kube-api server

service kube-apiserver stop

Then run the restore command:

ETCDCTL_API=3 etcdctl \ snapshot restore <name> \ --data-dir <path>

Now restart the services which we stopped earlier

$systemctl daemon-reload
$service etcd restart
$service kube-apiserver start

Scaling Clusters📶

We have done the scaling of pods in the Kubernetes cluster very well. Now what if you want to scale your cluster? Let's see how it can be done.

According to the capacity worker nodes adjust themselves by adding or removing from the cluster.

Kubernetes provides several tools and methods for scaling a cluster, like:

Manual Scaling🫥
Horizontal Pod Autoscaler (HPA)▶️
Cluster Autoscaler↗️
Vertical Pod Autoscaler (VPA)⏫

Let's have a look at all one by one

Manual Scaling - As the name suggests, we have to scale it manually using the kubectl command. Or if you're using any cloud provider, increase or decrease the number of worker nodes manually.

Horizontal Pod Autoscaler (HPA) - It automatically scales the number of replicas of a deployment or a replica set based on the observed CPU utilisation or other custom metrics.

When defining the definition file, you must ensure the usage of memory and cpu

To use utilisation-based resource scaling, like this:

 type: Resource
 resource:
   name: cpu
   target:
     type: Utilization
     averageUtilization: 60

This is also known as "scaling out". It involves adding more replicas of a pod to a deployment or replica set to handle the increased load.

Cluster Autoscaler - Based on the pending pods and the available resources in the cluster, it automatically scales the number of worker nodes in a cluster.

Read more about this scaler in detail here!

Vertical Pod Autoscaler (VPA) - It automatically adjusts the resource requests and limits of the containers in a pod based on the observed resource usage.

It is also known as "scaling up," which involves increasing the CPU, memory, or other resources allocated to a single pod.

Thank you!🖤

Simply Deploying Kubernetes Workloads

Poonam Pawar — Mon, 29 May 2023 06:00:29 +0000

Introduction✍️

Before moving forward directly to our main topics let's recall the sub-topics which is crucial for the next ones. So that you can grasp the context in a much more clear way.

Every other Relationship👀

First, let's talk about every relationship which we have to keep in mind forever from now onward.

POD -> CONTAINER -> NODE -> CLUSTER

Are you thinking of the same as what I am thinking?

Dabbe pe dabba, uske upper phir se ek dabba...🤪

No?? Ok ok! Forgive me:)

(translation: don't mind it, please. Thank you!)

So basically you got the idea of how pods are encapsulated into the containers which are placed inside the node and the group of nodes becomes a cluster.

And we have already covered their workings in the previous articles.

Pre-requisites👉

So a must-have pre-requisite topic is ReplicaSets.

Now be serious guys and let's start the conversation on replicaset aka rs

ReplicaSets🗂️

Continuing the imagination from the previous blog of your application deployment...

You have your application running into a pod. Now suddenly your application traffic grows and you didn't prepare your app for this huge number and due to this it crashes.

Or imagine another scenario where you have to update your app version from 1.0 to 2.0 ie v1 to v2 and in doing so your app stops running and the users fail to access the application.

In case of application failure, you need another instance of your application that saves you from crashes and running at the same time. So that the users cannot lose access to the application.

This is where the replication controller (now as replicaset the upgradation version) comes in as a savior. ReplicaSets always takes care of running multiple instances of a single pod running in the k8s cluster.

It helps us to automatically bring up the new pod when the existing ones fail to run.

You can set it to either one or hundreds, it's totally up to your choice.

It also helps us to balance the load in our k8s clusters. It maintains the load balance when the demand increases by creating instances of the pod in other clusters too.

So, it helps us to scale our application when the demand increases.

A simple Replica Yaml file:

---
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: replicaset-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      tier: nginx
  template:
    metadata:
      labels:
        tier: nginx
    spec:
      containers:
      - name: nginx
        image: nginx

Then run:

kubectl create -f /root/replicaset-demo.yaml

To scale up or down your replicas, you can use kubectl scale command.

run:

kubectl scale rs replicaset-demo --replicas=5

To check the replicasets use get command:

kubectl get rs

This will show you all the previous and newly created replicas by you in the system.

This is how you can create replicas of your pods.

Deployments🏗️

This comes to the highest in the hierarchy level of deploying our applications to production.

When you have many instances running into the k8s cluster and want to update the version of your application then the rolling updates do their job one after the other instead of making it down at the same time and then up altogether for obvious reasons.

These rolling updates and rollback performance is done by the k8s deployment.

As we know that every pod deploys single instances of our application and each container is encapsulated in the pod and then such pods are deployed using replica sets.

Then comes Deployment with all the capabilities of doing upgradation of the whole set of environment production.

A simple deploymentyaml file deployment-definition-httpd.yaml:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd-frontend
spec:
  replicas: 3
  selector:
    matchLabels:
      name: httpd-frontend
  template:
    metadata:
      labels:
        name: httpd-frontend
    spec:
      containers:
      - name: httpd-frontend
        image: httpd:2.4-alpine

Then run

kubectl create -f deployment-definition-httpd.yaml

To check the deployments you have created use get command, deployment aka deploy :

kubectl get deploy

You can also check for the specific one by giving its name:

kubectl get deploy httpd-frontend

To get more detailed information regarding the deployment, run the describe command:

kubectl describe deploy httpd-frontend

To check all the workloads you have created till now, use get all commands:

kubectl get all

This is how you can deploy your pods.

StatefulSets🏗️📑

Some topics will repeat themselves but for the sake of connecting them to the next one, it is necessary to do so. So bear with me.
The use of Deployment is to deploy all the pods together to make sure every pod is up and running, all these things we have seen above.
But what if you have many servers in the form of pods that you want it to run in order? For this, deployment will not help you because there is no order specified in it for running pods in the k8s cluster.
For that, you need StatefulSet. It ensures to run the pods in the sequential order which you want to run. First, the pod is deployed and it must be in a running state then only the second one will be deployed.
Stateful set is similar to deployment sets, they create pods based on the templates.
They can scale up and scale down as per the requirements. And can perform rolling updates and rollbacks.
Like, you want to deploy the master-node first then the worker-node-1 up completely then starts running and after that worker-node-2 will be up and run itself into the k8s cluster. StatefulSet can help you to achieve this.
As in the deployment sets, when a pod goes down and a new pod comes up, it is with a different pod name and different IPs come in.
But in StatefulSets when a pod goes down and a new pod comes up, it will be the same name that you have specifically defined for that pod.
So the StateulSets maintain an identity for each of its pods that helps to maintain the order of the deployment of your pods.
If you want to use StatefulSets just for the sake of identity purposes for the pods and not for the sequential deploying order then you can manually remove the commands of the order maintenance, just you have to make some changes in the YAML file.
Though It is not necessary for you to use this StatefulSet. As it totally depends on your need for the application. If you have servers that require an order to run or you need a stable naming convention for your pods. Then this is the right choice to use.
You can create a StatefulSet yaml file just like the deployment file with some changes like the main one is kind as StatefulSet. Take a look:

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  labels:
        app: msql
spec:
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql
  replicas: 3
  selector:
      matchLabels:
          app: mysql
  serviceName: mysql-h

than run:

kubectl create -f statefulset-definition.yml

To scale up or down use the scale command with the numbers you wanted to scale:

kubectl scale statefulset mysql --replicas=5

This is how you can work with StatefulSets.

DaemonSets🤖

Till now, we have deployed the replicasets as per the demand so the required number of pods is always up and running.

Daemon sets are like replica sets. It helps you to deploy multiple instances of the pod.

So what's the difference?

It runs one copy of your pod on each node of your cluster.

Whenever you add a new node, it makes sure that a replica of the pod is automatically added to that node. And removed automatically when the node is removed.

The main difference is, the daemon sets make sure that one copy of the pod is always present in all the nodes in the k8s cluster. On the other hand replica set runs a specified number of replicas of the pod which you defined in the YAML file.

It is also used to deploy the monitoring agent in the form of a pod.

Creating a daemon file is much similar to the replica file which we have already created above. You just have to make some changes and you are done.

The main difference is of course the kind ie ReplicaSet to DaemonSet

Take a look fluentd.yaml is the name of the file with the content below:

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: elasticsearch
  name: elasticsearch
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - image: registry.k8s.io/fluentd-elasticsearch:1.20
        name: fluentd-elasticsearch

Then run the following command:

kubectl apply -f fluentd.yaml

To get the daemonstate with all the namespace which you have created, run:

kubectl get daemonsets --all-namespaces

This is how you can create DemonSets

JOBS🧑‍💻

As we all know, Kubernetes ensures that our application is always up and running no matter what.

Let's say we perform work in our application and when it successfully completed it shows the successful message and then again comes in a running state because that's the nature of k8s.

Jobs in k8s does the work in your application and when it becomes successful it shows the completed message and then stops the work as it is no longer in use.

But why do we need JOBS?

Let's assume you are using your camera and clicked some pictures. So this enabling of camera work is going on in your application, k8s is running your app in the pod and after use, you disabled your camera. But k8s again starts running it as it always looks up for the application up and running. And you are unaware of this pod running.

Then isn't it a dangerous task?

That's why we need JOBS so that in a specific work when the task is completed it does not go into a running state again.

So a task you may not want to run continuously, then using a Job would be appropriate. Once the task is complete, the Job can be terminated, and the pod will not start again unless a new Job is created.

It is done so because a spec member ie restartPolicy is set as always by default.

So in the creation of the JOBS yaml file, we set it to never

Run the command to create a job definition file.

kubectl create job throw-dice-job --image=kodekloud/throw-dice --dry-run=client -o yaml > throw-dice-job.yaml

Use the following YAML file to create the job:

apiVersion: batch/v1
kind: Job
metadata:
  name: throw-dice-job
spec:
  backoffLimit: 15 # This is so the job does not quit before it succeeds.
  template:
    spec:
      containers:
      - name: throw-dice
        image: kodekloud/throw-dice
      restartPolicy: Never

In the above file, we have created a JOB to throw the dice until it gets six and the chances offered is 15 times to play.

kubectl apply -f throw-dice-job.yaml

This is how you can create your own job file.

CronJob🥸

A CronJob is a JOB that will perform the task on a given time period.

You can schedule your job to do a task at a specific time you want.

It supports complex scheduling, with the ability to specify the minute, hour, day of the month, month, and day of the week.

It can be used to create one-off Jobs or Jobs that run multiple times.

It can be used to run parallel processing tasks by specifying the number of pods to be created.

Example,

Updating the phone at midnight to avoid interruptions while using the phone.🤳
Email scheduling, you schedule an email using cronjob to perform this task. periodically.🧑‍💻

Let us now schedule a job to run at 21:30 hours every day.

Create a CronJob for this.

Use the following YAML file:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: throw-dice-cron-job
spec:
  schedule: "30 21 * * *"
  jobTemplate:
    spec:
      completions: 3
      parallelism: 3
      backoffLimit: 25 # This is so the job does not quit before it succeeds.
      template:
        spec:
          containers:
          - name: throw-dice
            image: kodekloud/throw-dice
          restartPolicy: Never

This is how you can use the CronJob file.

Every example I have given above is just for the sake of explanation. Don't take that seriously.

Now we have deployed every workload of the Kubernetes cluster. It takes practice to make hands-free in creating replicas, deployments, jobs etc. So do the practice.

Thank you!🖤

Kubernetes Networking

Poonam Pawar — Sat, 27 May 2023 09:18:32 +0000

Introduction✍️

Networking in k8s is such a crucial topic to understand and a must for working in k8s. Here we will discuss networking, particularly in the work use of k8s directly. So before you come to this topic you must have knowledge of basic computer networking.

Switching
Routing
Gateways
Bridges

PODS📦

Before jumping right into the Kubernetes networking stuff. First, let's recall the Pods concept because we will be using Pods in every sentence further.
Whatever we are doing, the main goal is to deploy our application in the form of containers on a worker node in the cluster which must be up and running.
As Kubernetes does not deploy the containers directly on the nodes.
The containers are encapsulated into an object known as Pods
The Pod is a single instance of an application.
A Pod is the smallest object that you can create in Kubernetes.
A simple yaml pod file:


      apiVersion: v1
      kind: Pod
      metadata:
        name: demo-pod
      spec:
        containers:
        - name: demo-container
          image: nginx
          ports:
          - containerPort: 80

Then run the kubectl apply command once you have created the YAML file to deploy it in the Kubernetes cluster with the necessary file name you have given.

kubectl apply -f demo-pod.yaml

This is how you can start creating PODs.

Network Policies📋

In Kubernetes, every routing network traffic instruction is set by the network policies.
A set of tools that defines the communication between the pods in a cluster.
It is a powerful tool used for the security of network traffic in k8s clusters.
Allowance of traffics from one specific pod to another one.
Restricting traffic to a specific set of ports and protocols.
Implementation is done by Network APIs.
It can be applied to namespaces or individual pods.
A simple network policy yaml file:

  apiVersion: networking.k8s.io/v1
  kind: NetworkPolicy
  metadata:
    name: demo-network-policy
  spec:
    podSelector:
      matchLabels:
        app: my-app
    policyTypes:
    - Ingress
    ingress:
    - from:
      - podSelector:
          matchLabels:
            app: allowed-app
      ports:
      - protocol: TCP
        port: 80

For the deployment of the Network Policy YAML file, use the kubectl apply command:

kubectl apply -f demo-network-policy.yaml

This is how you can define your own network policies.

Services🪖

Let's say, you deployed a pod having a web application running on it and you want to access your application outside the pod. Then how to access the application as an external user?

The k8s cluster setup in your local machine has an IP address similar to your system IP but the pod has a different IP address which is inside the node.

As the pod and your system share different addresses, there is no way to access the application directly into the system.

So we need something in-between to fill the gap and gives us access to web application directly from our systems.

Getting started with Kubernetes Services - Spectro Cloud

This is where Kubernetes Services comes into the picture.

k8s services activate the communication between several components inside-out the application.
It helps us to connect the application together with other applications and users.
It is an object just like PODs, Replicas and Deployments.
It listens to a port on the node and forwards the requests to the port where the application is running inside the pod.

A simple service yaml file:

apiVersion: v1
kind: Service
metadata:
  name: demo-service
spec:
  selector:
    app.kubernetes.io/name: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

Services Types📑:

NodePort Service
ClusterIP Service
LoadBalancer Service

1) NodePort Service - It makes an internal POD accessible to the Port on the Node.

the port number inside the pod is the target port
the port number in the service area is the 2nd Port
and the port on the node is NodePort where we're going to access the web server externally.
the valid range of NodePort (by default): 3000 to 32767
labels & selectors are necessary to describe in the spec section in case of multiple pods in a node or multiple nodes in a cluster.

2) ClusterIP Service - It creates a virtual IP inside the cluster to enable communication between different services.

a service created of a tier of your application like backend or frontend which helps to group all the pods of a tier and provide a single interface for other pods to access this service.

3) LoadBalancer Service - It exposes the service on a publicly accessible IP address in a supported cloud provider.

A demo NodePort yaml file:

apiVersion: v1
kind: Service
metadata:
  name: my-demo-service
spec:
  type: NodePort
  selector:
    run: nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30007

now run,

kubectl create -f myservice.yaml

This is how you can create your service of NodePort type. By changing the spec type to ClusterIP or LoadBalancer, whatever you wish to work with just change it with the name and port according to it.

CNI (Computer Network Interface)🌐

CNI is a set of standards that defines how to configure networking challenges in a container runtime environment like Docker and Kubernetes.
It is a simple plugin-based architecture.
It defines how the plugin should be developed.
Plugins are responsible for configuring the network interface.
CNI comes with a set of supported plugins already. Like bridge, VLAN, IPVLAN, MACVLAN.
Docker does not implement CNI. It has its own set of standards known as CNM ie, Container Network Model.
We cannot run a docker container to specify the network plugin to use CNI.
But you can do it manually by creating a docker container without any network configuration

docker run --network=none nginx
then invoke the bridge plugin by yourself.

CNI in k8s🕸️

Kubernetes is responsible for creating container network namespaces
Attaching those namespaces to the right network
You can see the network plugins set to CNI by running the below command

ps -aux | grep kubelet

The CNI plugin is configured in the kubelet service on each node in the cluster
The CNI bin directory has all the supported CNI plugins as executables.

ls /opt/cni/bin

k8s supports various CNI plugins like Calico, Weave Net, Flannel, DHCP and many more.
You can identify which plugin is currently used with the help of this command

ls /etc/cni/net.d

kubelet finds out which plugin will be used.

DNS in k8s📡

Kubernetes deploys a built-in DNS server by default when you set up a cluster.
Let's say we have three nodes k8s cluster having pods and services deployed in it having node name and IP address assigned to each one.
All pods and services can connect to each other using their IPs and to make the web app available to external users we have services defined on them.
Every new entry of a pod goes into the DNS server record by

cat >> /etc/resolv.conf

The k8s DNS keeps the record of the created services and maps the service name to the IP address. So anyone can reach by the service name itself.
DNS implemented by Kubernetes was known as kube-dns and later versions CoreDNS.
The CoreDNS server is deployed as a POD in the kube-system namespace in the k8s cluster.
To look for DNS Pods, run the command

kubectl get pods -n kube-system

Ingress🔵

Let's say you have an application deployed in the k8s cluster with a database pod having required services for external accessing with the NodePort in it. The replicas are going up and down as per the demand of the application.
You configured the DNS server so that anyone can access the web app by typing their name instead of IP address every time
like, http://my-app.com:port instead of http://<node-ip>:port and if you do not want others to type port number also then you served a proxy-server layer between your cluster and DNS server. so that anyone can access it by just
http://my-app.com
Now, you want some new features to be added to your application. You developed the code and deployed the webapp again for the new features of pods and services into the cluster. Again setting up the proxy server in-between to access the webapp in just a single domain.
Maintaining all these outside the cluster is a tedious task to do when your application scales. Every time a new feature adds on, you have to do all the layerings again and again.
This is where Ingress comes in. Ingress helps users to access your application with a single URL.
Ingress is just another k8s definition file inside the cluster
You have to expose it one time to make it accessible outside the cluster either with NodePort or the cloud provider like GCP which uses the LoadBalancer service.

Ingress Controller🔹

In the case of using ingress in a k8s cluster, an ingress controller must be deployed and configured.
Mostly used ingress controllers are Nginx, Traefik, and Istio.
Deploy the application on the k8s cluster and configure them to route traffic to other services.
The configuration involves defining URL Routes, Configuring SSL certificates etc. This set of rules to configure ingress is called an Ingress Resource.
Once the controller is running, resources can be created next and configured to route traffic to different services in the cluster.
It is created using definition files like the previous ones we created for Pods, Services and Deployments.
Ingress Controller is not built-in in them by default. You have to create it manually.
A simple ingress yaml file for the sake of explanation:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: minimal-ingress
        annotations
          nginx.ingress.kubernetes.io/rewrite-target: /
      spec:
        ingressClassName: nginx-example
        rules:
        - http:
            paths:
            - path: /testpath
              pathType: Prefix
              backend:
                service:
                  name: test
                  port:
                    number: 80

Create a single ingress called 'simple' that directs requests to foo.com/bar to svc # svc1:8080 with a tls secret "my-cert"

      kubectl create ingress simple --rule="foo.com/bar=svc1:8080,tls=my-cert"

This is how you can start working with Ingress in Kubernetes.

Thankyou!🖤

Kubernetes PODS & SERVICES Discovery

Poonam Pawar — Thu, 25 May 2023 13:38:26 +0000

Introduction✍️

In this particular blog, we are going to talk about how all these workloads can be exposed outside to the world with the help of these Services and DNS. So that external users can also access your application.

How to expose Kubernetes workloads to the outside world using Services?🌏

Before heading to these topics you must have the knowledge of all the workloads.

Pre-requisites👇

PODS📦
DNS🧑‍💻
Services⚙️

Services are available in three main types:

ClusterIP
NodePort
LoadBalancer

ClusterIP is the default service created by k8s. With the help of this type, you can access your application just within your cluster. It will provide you with the benefit of load balancing and discovery, which we will learn about in the below section in detail.

If you want to check all your created services, use get command:

Also, you can write with the alias:

kubectl aka k

services aka svc

Trust me, this saves a lot of time.🥺

And also, I have covered the file creation already in detail.

NodePort can provide you to access your application within the cluster as well as to those who have access to the worker nodes you have set up during the creation.

Creating a service of NodePort type, the /root/service-definition-1.yaml file as follows:

---
apiVersion: v1
kind: Service
metadata:
  name: webapp-service
  namespace: default
spec:
  ports:
  - nodePort: 30080
    port: 8080
    targetPort: 8080
  selector:
    name: simple-webapp
  type: NodePort

Run the following command to create a webapp-service service as follows: -

kubectl apply -f /root/service-definition-1.yaml

Create the service-definition-1.yaml -> Add the service template using vi/vim -> Edit the details in the file -> Use command apply to create it

I am not going into details about the types of services because I have covered everything in my previous blog learnings.🫣

We will only be focusing on:

Exposing Kubernetes workloads to the outside world👶

LoadBalancer is the type of service that will expose your application to outside the world.

This will only work on cloud providers. There will be an elastic load balancer that will be created by the cloud providers which is a public IP address through which you can access your application.

Cloud Control Manager which is inside your master node will generate public IP adders using any cloud provider eg AWS and returns to the service so that anyone can access your application using that IP address.

To create a service obviously, we need all the pods and deployment-definition file.

So here it is:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - name: webapp
        image: my/webapp:latest
        ports:
        - containerPort: 80

Now, time to create a service-definition-2 yaml file with LoadBalance type:

apiVersion: v1
kind: Service
metadata:
  name: webapp-service
spec:
  selector:
    app: webapp
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer

First run,

kubectl apply -f /root/service-definition-2.yaml

Now the service is created. You can access this through the internet with the correct IP address and port number provided by this service.

Check the IP address and Ports by get command:

kubectl get svc <name>

How to discover Services and Pods within a Kubernetes cluster using DNS and other mechanisms?🧐

As we know how service works and its purpose But did you think how it tackled the problems? How does it send requests to the users? How does it provide everything requested by the users?

Let's see the solution:

Service act as a load balancer by using a component known as kube-proxy.

So instead of accessing the IP addresses, it helps the users to give a specific service name to the users to access the application.

kube-proxy will forward the requests coming from several users.

So without Services, your application will not work even if you have everything like pods, deployments, etc ready. When a pod goes down then you're unable to provide your applications to the users in case of no service available.

But the question remains the same. How service is handling the IP addresses, as every time a pod goes down, it comes up with a different address. So how this is going on?

If three of the pods are deployed in the cluster and each one has its IP address and whenever a pod goes down and comes up always with the new IPs then how service is managed all the time with the new IPs and managing all the user's requests?

This is handled by Service Discovery.

Service Discovery🕵️

This comes up with a new process called Labels & Selectors

Instead of working with IP addresses and keeping track of it. Service is using the concept of labels & selectors.

Labels📋

It is just a key-value pair.
You can name it by your conventions.
It is used to organize and select the objects in the cluster
You can label Pods, Deployments, and Services.
Example, key=app and value=myapp

metadata:
  labels:
    app: myapp

Selectors✅

It must be defined same name as the labels defined.
It is used to select objects based on their labels.
It can be used for a single or a groped objects that matches specific labels.
Example, create a Service that selects all Pods with the label app=myapp:

yamlCopy codespec:
  selector:
    app: myapp

We will give labels in our pod definition yaml file so that whenever a pod goes down and comes up with new IPs, it will always have the same label in it as we defined in the template. A new pod is always created with the given template.

The replica sets will deploy the pods using the labels.
Services will keep track of all the deployments by the labels.
Labels are just a name given to a pod nothing else.

So this is the service discovery mechanism that uses labels & selectors.

So you created a deployment definition yaml file and on that the metadata section you will define a label which can be any name provided by you for your deployed application.

Pod Discovery Using DNS🌐

The CoreDNS server is deployed as a POD in kube-system namespace in the k8s cluster.

It creates a service to make it available to other components within a cluster. And the service is named kube-dns by default.

kubectl get service -n kube-system

It uses a file called Corefile which is located at /etc/coredns

cat /etc/coredns/Corefile

In this file, you have all the configurations of plugins.

One of the plugins that make CoreDNS work with k8s is the Kubernetes plugin.

A top-level domain name is set as a cluster.local

The DNS configuration on PODS is done by k8s automatically when the pod is created.

config file of the kubelet will give you the IP of the DNS server and the domain:

cat /var/lib/kubelet/config.yml

You can access your service by just

name-service or

name-service.default or

name-service.default.svc or

name-service.default.svc.cluster.local

<service-name>.<namespace>.svc.cluster.local

Example, service-name=mywebapp and namespace=default which is created automatically as default. Then,

mywebapp.default.svc.cluster.local

If you try to manually lookup for the service using nslookup or the host command

host <name-service>

Pod Discovery Using Environment Variables🔗

The container which has the information of Services and PODS that are running inside the k8s cluster has an environment variable automatically set by Kubernetes.

These variables are used to discover the IP address, port numbers and hostname for the services and running pods.

Example,

the SERVICE_HOST and SERVICE_PORT environment variables are automatically set in containers running inside a Service's Pod.

the HOSTNAME environment variable is set to the hostname of the Pod and the MY_POD_IP environment variable is set to the Pod's IP address.

Let's create a simple service yaml file to use for further process with type ClusterIP:

apiVersion: v1
kind: Service
metadata:
  name: myapp-service
  labels:
    app: myapp
spec:
  type: ClusterIP
  selector:
    app: myapp
  ports:
  - name: http
    port: 80
    targetPort: 8080

Now, let's create a pod YAML file that runs image=nginx by setting two env variables MYAPP_SERVICE_HOST and MYAPP_POD_IP

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: nginx
    env:
    - name: MYAPP_SERVICE_HOST
      value: "myapp-service.default.svc.cluster.local"
    - name: MYAPP_POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP

MYAPP_SERVICE_HOST is set to myapp.default.svc.cluster.local and

MYAPP_POD_IP is set using a fieldRef that retrieves the Pod's IP address from its status.

Together, these files define a web application that can be accessed using the virtual IP address assigned to the Service.

When a client sends a request to the Service's IP address on port 80, the request will be forwarded to one of the Pods with the app: myapp label and the response will be sent back to the client.

This is how you can discover pods using environment variables.

Thank you!🖤

Kubernetes: Storage & Security

Poonam Pawar — Thu, 18 May 2023 10:48:47 +0000

Introduction✍️

It's time to share Secrets!🤫

Obviously not mine.😜

In this blog let's talk about the storage, storage classes, security policies, network policies, security layers, and everything that comes under the store room and the security room of the Kubernetes. These topics play a very crucial role in the making of k8s clusters. Let's see how it does!

Storage🛢️

Volumes🎚️

In Kubernetes, the pods are not permanent. Once a pod is created, it will be destroyed when the requirement finishes and one comes up in place of it. Just like docker containers, the data processed inside the pod also get deleted when the pod destroys. This will occur data loss which is a big problem.
To resolve this problem, volumes come up
We attach a volume to the pod. So now, whenever a pod processed some data inside it, it gives also gets stored in the volume. If now pod is going to be deleted we have still our data alive.
The data generated by the pod is now stored in the volume!

Let's create a simple single node k8s cluster volume-specific file:

apiVersion: v1
kind: Pod
metadata:
  name: randon-number-generator
spec:
  container:
  - image: alpine
    name: alpine
    command: ["/bin/sh","-c"]
    args: ["shuf -i 0-100 -n 1 >> /opt/number.out;"]
    volumeMounts:
    - mountPath: /opt
      name: data-volume
  volumes:
  - name: data-volume
    hostPath:
      path: /data
      type: Directory

In the above yaml file, we are generating a random number in a given range so that we can save that number in our volumes. volumeMounts are used to get the same data within the pod. hostPath is simply giving the path directory where the data will be stored.

This is how even after pod deletion we can still have our data in /data dir.

Note: This is only for single-node k8s clusters.

When you will work on a large-scale project, there will not be a single node cluster. Hundreds of Pods running at a time and giving the volume path as /data to all the pod's data is not recommended in the multi-node cluster for obvious reasons.
This is because the PODs would use the /data directory on all the nodes.
So we need proper storage solutions. Kubernetes supports different standard storage solutions like NFS, glusterFS and many more.

You will get a basic idea of how to define them in the further topics.

Persistent Volumes📻

Whenever we create volumes, every time we need to configure them in a pod-definition file. So every configuration information is required to configure storage within the file.
Now imagine we are running hundreds of pods and each time when a user wants to deploy the pods, they would have to configure storage every time for each pod in their environment.
So doing this every time is not a best practice for us.
Here, Persistent Volumes come up. It is a cluster-wide room of storage volumes configured by an administrator to make use of the users deploying applications on the cluster.
Now we can use the storage using persisting volume claims

Let's create a definition yaml file named pv-definition.yaml for this:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  accessModes:
    - ReadWriteOnce
  capacity: 
    storage: 1Gi
  awsElasticBlockStore:
    volumeID: <volume-id>
    fsType: ext4

There can be different kinds of accessMode like,

ReadOnlyMany
ReadWriteOnce
ReadWriteMany

awsElasicBlockStore is one of the supported storage solutions we talked about above for the multi-node cluster. This will provide specific volume-id and type instead of just /data dir path to differentiate better.

Now run,

kubectl create -f pv-definition.yaml

To check the created persistent volumes, you know what command to use now:

kubectl get persistentvolume

Persistent Volume Claims📑

After creating the persistentvolume, it's the time to create a persistent volume claim to make the storage available to a node.
Persistent volumes and Persistent volume claims are two separate objects in k8s.
An administrator creates a persistent volume and a user creates a persistent volume claim to use the storage.
Kubernetes binds the persistent volume of the claims as per the requests and required properties set on the volumes.
Kubernetes checks the sufficient capacity while binding volumes to the claims.
It also uses labels & selectors to bind on a specific persistent in case of multiple possible matches to the right volumes.

Let's create a claim definition file named pvc-definition.yaml now,

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myclaim
spec:
  accessModes:
    - ReadWriteOnce
  resources: 
    requests:
      storage: 500Mi

When this file is created, Kubernetes looks for the volume file which was created above.

kubectl create -f pvc-definition.yaml

The accessModes match, then it checks for the storage capacity and here to store 500Mi in 1Gi is the perfect match for them since there is no other available.

So the claim is bound to the volume.

To check the claims file use the get command:

kubectl get persistentvolumeclaim

Storage Classes🗑️

We have created the persistent volumes and persistent volume claims but before creating this volume you must have created a disk on google cloud.
You have to manually provision the disk whenever your application needs storage on gc. And then manually create a persistent volume file using the same name defined during the creation of disk specifications.
This whole process is called Static Provisioning Volumes.

To automate this process fully, we have Storage Class.

You just have to define a provisioner like Google Storage and then everything will be seen by the provisioner. It automatically provisions the storage and attaches that to the pod when a claim is made.
This is called Dynamic Provisioning Volumes.
Now you don't have to create persistent volume manually.
There are many storage provisioners such as AWSEBS, AzrueFile, AzureDisk, CephFS and many more

Take a look at a storage class definition file as sc-definition.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: google-storage
provisioner: kubernetes.io/gce-pd

After defining this file, you have to give the storage class name in pvc file as the same as you have defined in the sv file as google-storage in this case.

You can create different kinds of classes in storage using different types of disks like

Silver🔘 Storage Class with the standard disk,
Gold🟠 Storage Class with SSD drives and
Platinum⚪ Storage Class with SSD drives and replication.

StatefulSets🏗️

It creates pods based on the templates.
It can scale up and scale down as per the requirements. And can perform rolling updates and rollbacks.
Like, you want to deploy the master-node first then the worker-node-1 up completely then starts running and after that worker-node-2 will be up and run itself into the k8s cluster. StatefulSet can help you to achieve this.
In this, when a pod goes down and a new pod comes up, it will be the same name that you have specifically defined for that pod.
It maintains an identity for each of its pods that helps to maintain the order of the deployment of your pods.
If you want to use StatefulSets just for the sake of identity purposes for the pods and not for the sequential deploying order then you can manually remove the commands of the order maintenance, just you have to make some changes in the YAML file.

Note: If you have already learned this topic in my previous blog then feel free to skip this topic😇. Otherwise, continue learning!✊

Though you don't need to use this StatefulSet. As it totally depends on your need for the application. If you have servers that require an order to run or you need a stable naming convention for your pods. Then this is the right choice to use.

You can create a StatefulSet yaml file just like the deployment file with some changes like the main one is kind as StatefulSet. Take a look:

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  labels:
        app: msql
spec:
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql
  replicas: 3
  selector:
      matchLabels:
          app: mysql
  serviceName: mysql-h

than run:

kubectl create -f statefulset-definition.yml

To scale up or down use the scale command with the numbers you wanted to scale:

kubectl scale statefulset mysql --replicas=5

This is how you can work with StatefulSets.

Security🕵️

RBAC (Role-Based Access Control)🛂

-As the name suggests itself the working, it is used to define roles to users or a group and bind them who granted those permissions.

Simply, granting the permissions for who will do what.

Let's quickly see how to create a role-definition file:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "list", "create", "delete"]
- apiGroups: [""] # "" indicates the core API group
  resources: ["ConfigMap"]
  verbs: ["create"]

In the above file, we are creating the role for a developer who has resources as pods in which they can modify the pods and can allow to configure them.

Now run create command to create the role:

kubectl create -f developer-role.yaml

Now, link the user to that role. For this create another object file called RoleBinding:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: devuser-developer-binding
subjects:
- kind: User # "" indicates the core API group
  name: dev-user
  apiGroups: rbac.authorization.k8s.io
roleRef: 
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

Now run create command to bind this:

kubectl ceate -f devuser-developer-binding.yaml

To view the created roles, run the get command:

kubectl get roles

To see the bindings you have created run:

kubectl get rolebindings

To view the role in detail run describe command:

kubectl describe role developer

To check your accessibility for a particular object, run the:

kubectl auth can-i create deployments

By this, you can check access that you want to know like delete nodes

Pod Security Policies (PSPs)📦

It is a security policy defined for the users or a group to access the pods in the Kubernetes cluster.
Yes! You guessed it right. It is one of the RBAC to grant permissions and bind them with whom granted with.
It defines a set of security policies that are applied to the pods based on their labels and annotations.
When a pod is created, Kubernetes checks for the specific labels and annotations which we have defined above in the file to proceed it from creating to running state.

Note: Removed feature
PodSecurityPolicy was deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25.

Instead of using PodSecurityPolicy, you can enforce similar restrictions on Pods using either or both:

Pod Security Admission
a 3rd party admission plugin, that you deploy and configure yourself

These new Pod Security Standards define three different policies to broadly cover the security spectrum.

These policies are cumulative and range from highly-permissive to highly-restrictive.

Learn more here!

Secrets🧑‍💻

Finally, it's time to decrypt the secrets in Kubernetes.

Never tell your secrets to anyone. Ok!

But what is meant by secrets in k8s?

Let's say, you have a simple web application that is connected to a database that displays a successful message on the screen while getting connected.
First, you have coded your user name and passwords into the source code. Then you understand that it's not a good idea to provide credentials like this.
Then you create an object file called ConfigMap and put those values inside the yaml file. ConfigMap stores data in a plain text format. And that's again not a good idea.

So here, Secrets come in. It is used to store sensitive information which you don't want to share it with anybody.

It is similar to configMaps but it stores the information in encoded format instead of plain text.

So first encode your data. To do so, run:

$ echo -n 'mysql' | base64
bXlzcWw=

vice-versa for decoding the text:

$ echo -n 'bXlzcWw=' | base64 --decode
mysql

To check secrets, run:

kubectl get secrets

To get detailed information on secrets, run:

kubectl describe secrets

Now, create a secret-definition.yaml file:

apiVersion: v1
kind: Secret
metadat:
  name: myapp-secret
data:
  DB_Host: bXlzcWw=
  DB_User: cm9vdA==
  DB_Password: cGFzd3Jk

Now configure this secret with a pod-definition.yaml file:

apiVersion: v1
kind: Pod
metadata:
  name: simple-webapp-color
  labels:
    name: simple-webapp-color
spec:
  containers:
  - name: simple-webapp-color
    image: simple-webapp-color
    ports:
      - containerPort: 8080
    envFrom:
      - secretRef: 
          name: myapp-secret

Now run:

kubectl create -f pod-definition.yaml

This is how you can create your own secret file and configure it in the pod-definition file by adding the envFrom section.

And list as many variables as you want as per your need. The name should be matched with the one you have created in the secret-definition file. In this case, it is myapp-secret

Network Policies📋

In Kubernetes, every routing network traffic instruction is set by the network policies.
A set of tools that defines the communication between the pods in a cluster.
It is a powerful tool used for the security of network traffic in k8s clusters.
Allowance of traffics from one specific pod to another one.
Restricting traffic to a specific set of ports and protocols.
Implementation is done by Network APIs.
It can be applied to namespaces or individual pods.

A simple network policy yaml file:


    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: demo-network-policy
    spec:
      podSelector:
        matchLabels:
          app: my-app
      policyTypes:
      - Ingress
      ingress:
      - from:
        - podSelector:
            matchLabels:
              app: allowed-app
        ports:
        - protocol: TCP
          port: 80

For the deployment of the Network Policy YAML file, use the kubectl apply command:

kubectl apply -f demo-network-policy.yaml

This is how you can define your own network policies.

TLS (Transport Layer Security)🌐

Every communication over the internet must be secure. Otherwise, there is a high risk of hackers hacking the data which we are transferring through it.
Likewise in Kubernetes clusters, there is a set of master nodes and worker nodes who communicates thoroughly.
The communications between all the pods, nodes and API servers must be secured and encrypted.
Administrators trying to communicate with the master node via kubelet or through APIs directly, everything must be fully secure.
The communication between the servers and the clients must be secure and encrypted.
To fulfill all these requirements we need security certificates. We all know about TLS certificates and how it works.
It works with an asymmetric key format where every end has its own key-value pair lock-key to decrypt the data.

Now let's understand how TLS can be defined in Kubernetes.

In Kubernetes, it has two sides, one is the server side and another is the client side. Both sides must have security certificates signed by CA (Certificate Authority) to verify their identity. Let's look at both of them.

Server Certificates of the Servers🔏

KUBE-API Server - This helps the k8s to expose the HTTP service that other components and external users use to manage the cluster. So it is an important component that must be secured enough.
It has a certificate and a key-pair named apiserver.cert
and apiserver.key
ETCD Server - It stores all the information about the clusters so generate a certificate and key-value pair for this also. The naming conventions are
etcdserver.cert and etcdserver.key
KUBELET Server - It belongs in the worker node which also exposes the HTTP API endpoints to interact with others. Their cert and key-value are

kubelet.cert and kubelet.key

Client Certificate of the Clients🔐

The Administrator - The clients who access the services are the admins. It also requires a certificate and a key-value pair for access. Naming them as
admin.cert and admin.key
The Schedulers - It is another client who communicates with the kube-api server to schedule the objects as per the requirements. So it also needs verification to talk to the server.
Naming them as scheduler.cert and scheduler.key
KUBE-CONTROLLER MANAGER - It also communicates with the kube-api server and for the authentication it requires the same security checks.
controller-manager.cert and controller-manage.key
KUBE-PROXY - Another client-side component has the
certificates and key
kube-proxy.cert and kube-proxy.key

Now it's time to generate a certificate for our cluster. There are many tools to do so like EASYRSA, OpenSSL, CFSSL and many more. We will be using OpenSSL in this.

Create a private key using the openssl command:

$ openssl genrsa -out apiserver.key 2048

It will generate a key called apiserver.key

Now use the request command to generate a certificate signing request for the previous one

$ openssl req -new -key apiserver.key -out apiserver.csr -subj "/CN=kube-apiserver"
  $ openssl x509 -req -in apiserver.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out apiserver.crt -days 365

Now, create a secret to store the private key and certificates

$ kubectl create secret tls apiserver-certs --key=apiserver.key --cert=apiserver.crt -n kube-system

Modify the Kubernetes API server configuration to use the TLS certificates:

Now edit the file of api server config to use tls cert:

  $ vi /etc/kubernetes/manifests/kube-apiserver.yaml

  spec:
    containers:
    - name: kube-apiserver
      volumeMounts:
      - mountPath: /etc/kubernetes/pki/apiserver
        name: apiserver-certs
        readOnly: true
    volumes:
    - name: apiserver-certs
      secret:
        secretName: apiserver-certs

Restart the k8s api server to apply the new config

$ systemctl restart kubelet

This is how you can generate a certificate and configure TLS for the Kubernetes API server.

Thank you!🖤

Kubernetes: Architecture, Components, Installation & Configuration

Poonam Pawar — Mon, 15 May 2023 08:33:53 +0000

Introduction🧑‍🦯

There are many ways to learn Kubernetes aka k8s. And one of the best ways to learn is by the official documentation itself. But for very beginners, it gets quite difficult to understand all the terms and technology directly through it.

So, in this blog, I will break it down into as many pieces as I can. I will share how I learn in the simplest form.

Kubernetes🕸️ -

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.

Let me break down this definition.

Container⛴️+ Orchestration🛟

Let's imagine you have your application ready inside the Docker container🐬 to run. Now the next thing that should come to your mind is the further process of deployment, scaling and updating your application.

The complete process of automatically deploying and managing is known as Container Orchestration.

K8s provides you with an orchestration platform through which you can perform these tasks smoothly.

Architecture 👾

Now that you got a basic understanding of what k8s do, let's jump into the architecture of it.

Everything you're seeing in the below architecture is needed to set up your k8s cluster. And before going into the k8s cluster let us first understand the other main components of the architecture.

This includes the two major nodes ie Master node and the Worker node

Before figuring out who is the master and who is the ~~slave~~ worker node, first, understand all about NODES.

Nodes📦 -

A node is a worker machine where we're going to install our k8s and the containers with your application ready will be launched by k8s.

To scale up and down as per the demand there has to be more than one node.

So here Cluster comes!

Cluster🗂️ -

A cluster is a set of nodes grouped together which will help your application is always running state if other nodes fail to perform.

This means if you have a cluster with multiple nodes running your application inside a container. And if one node goes down then the other one is up and running to save your application from crashing.

Master Node📑 -

Let's imagine you have a factory where all the machines are placed together in a room and working simultaneously as per the demand and requirements.

Now if you noticed there is always a control room where everything is managed and controlled inside that room. This is the Master Node in our case scenario.

This node keeps an eye about:

Keeps watches on the nodes in the cluster.
Responsible for the actual orchestration of containers on the worker node.
Information about the member cluster node is stored in it.
Responsible for managing the cluster.
Node monitoring configuration.
Managing workload balance.
Controls the scheduling of containers onto worker nodes.
Provides an API endpoint that can be used to interact with the cluster.

Worker Node🧾 -

The factory room where all the machines are placed to work ie, worker node in our case scenario.

Let's see what the worker nodes do:

Responsible for running the containers where your application resides.
Receives commands from the master node to run which and when node specifically.
Reports back to the master node of their entire performance structure.

Components🧰

Now let's deep dive into all the components you have seen on the architecture diagram quickly.

kube-apiserver🔗:

To interact with the k8s cluster, this API server helps to get done all the talks of it. It acts as the front end of the k8s. The command-line interface, management devices, and everything covers under this server.
etcd🖇️:

All data is stored to manage the cluster in the key-value pair format. Basically, it is a backing storage of your k8s cluster. It implements the locks within the cluster to avoid conflicts between multiple nodes and masters.
kube-scheduler🧷:

The distribution of workload and containers is done by the scheduler. It also assigns the container to the correct and required node. When anything goes down it notices and responds to the master and brings up the new container into the node as per the requirements.

All these above components come under the master node.

Container runtime📎:

It is the software that is responsible to run the containers eg, Docker.
kubelet🔎:

It is the agent that runs on each node in the cluster.
kube-proxy📈:

This handles network traffic between your containers.

All the above three are the worker node category component.

Installation⚙️ & Configuration🌐

To run the Kubernetes cluster in your local machine environment you need pre-requisite software to install.

Let's start with step-by-step guidance:

Update your Ubuntu package list:

sudo apt-get update

Pre-requisite for installing k8s: Docker

     sudo apt install docker.io -y
     sudo systemctl start docker
     sudo systemctl enable docker

Install all the necessary packages for k8s:

    sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive- 
    keyring.gpg 
    https://packages.cloud.google.com/apt/doc/apt- 
    key.gpg

Now, add the Kubernetes signing key:

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

Update the system and install k8s:

  sudo apt update -y
  sudo apt install kubeadm=1.20.0-00 kubectl=1.20.0-00 kubelet=1.20.0-00 -y

Initialize the cluster (Master):

  sudo su
  kubeadm init

Run this command on the Master Node:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Generate the Token for the configuration of Worker Node:

kubeadm token create --print-join-command

Paste the generated token output in the Worker Node:

  sudo su
  ---Paste the Join command on worker node with `--v=5`

Now execute this command in the Master Node :

kubectl get nodes

Installation differ with the differnet operating system. If you are using other OS please checkout the steps here!

Thankyou!🖤

Cloud Journey

Poonam Pawar — Sat, 13 May 2023 14:05:38 +0000

Apnaa dhyaan ekatrit kar lijiye, ab ye vimaan(blog) apko aasmano ki unchaiyon me le jane vala hai.......

(translation)

Please fasten your seatbelt, now this airplane (blog) is going to take you to the heights of the sky...

First flight?✈️I mean new to cloud computing?😶‍🌫️

No worries!😉I got you:)

After reading this blog you will definitely gain some cloud computing fundamentals.

What is Cloud Computing?

Cloud Computing is the delivery of on-demand computing services, including servers, storage, databases, networking, software and analytics, over the Internet, on a pay-as-you-go basis.

Essentially, cloud computing allows users to access these computing resources remotely, without having to build or maintain their own infrastructure.

This is what AI says about cc. (alias: cloud computing ~ cc)

Let's understand cc more simply,

CC is like renting a house🏡 instead of buying one.

Just like how you don't need to worry about the maintenance of the house when you rent it, in cc you don't need to worry about the maintenance of the physical server or the infrastructure.

Instead, you can rent a virtual🏕️ space on the internet where you can store your data and run your applications. This virtual space is called the Cloud.☁️

In the DevOps world, cc is important because it allows the team🤖 to quickly and easily deploy and manage applications without worrying about the underlying infrastructure.

With cc, developers can easily provision the required resources and test their applications, and operation teams can easily monitor🕵️ and manage the applications in production.

Why cloud-computing is important in DevOps?

CC is a key enabler of the continuous delivery and deployment of applications.
Cloud platforms offer a variety of services, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS), which can be leveraged to build, test, and deploy applications.
The cloud provides a flexible and scalable infrastructure that can be rapidly provisioned and de-provisioned to meet the demands of modern application development and deployment.

What are some common use cases?

Application hosting: Cloud platforms offer a range of services that can be used to host web applications, mobile applications, and APIs. These services can be scaled up or down based on demand, allowing companies to handle traffic spikes without overprovisioning their infrastructure.
Continuous integration and delivery: Cloud platforms provide tools and services that enable teams to build, test, and deploy applications in a continuous and automated fashion.
DevOps toolchain: Cloud platforms offer a range of tools and services that can be used to manage the entire DevOps toolchain, from version control to monitoring and alerting.
Disaster Recovery & Backup: Cloud platforms can be used to store backups and replicate data to provide redundancy in case of a disaster.

How can you choose a cloud platform?

To start deploying applications on the cloud, you first need to choose a cloud provider.

Some popular options are:

Amazon Web Services (AWS),

Microsoft Azure, and

Google Cloud Platform (GCP)

Each provider offers their own set of services, so it's important to do some research to find the one that best fits your needs.

Some following factors you can keep in your mind while choosing the one:-

💸Cost: Different cloud platforms have different pricing models, and it is important to choose a platform that fits your budget.
📡Services: Different cloud platforms offer different services, and it is important to choose a platform that offers the services you need.
⚙️Integration: It is important to choose a cloud platform that integrates well with your existing tools and infrastructure.
📈Scalability: It is important to choose a cloud platform that can scale up or down based on demand.

You can start it for free as well for practicing purposes.

All AWS, Microsoft Azure, GCP, Heroku, Digital Ocean etc offers a free trial that allows you to access some of their popular services for a limited time period.

Once you've chosen a cloud platform, you can deploy your applications to it by creating virtual machines or using containerization technologies like Docker.

With cloud computing, you can easily scale your applications up or down as needed, and you only pay for the resources you use.

How to use a cloud platform?

Creating a virtual machine or container: This involves creating a virtual machine or container that will host your application.
Configuring the virtual machine or container: This involves configuring the virtual machine or container to run your application, including installing dependencies and configuring environment variables.
Deploying the application: This involves copying the application files to the virtual machine or container and starting the application.

Projects & Resources

The best way to learn something is by reading their official documentation page.

Cloud service provider documentation: Cloud service providers like AWS, Microsoft Azure, and Google Cloud Platform offer extensive documentation on their platforms, including tutorials and guides.
Free Trial and Free Tier | Google Cloud
Start building on Google Cloud with $300 in free credits and free usage of 20+ products like Compute Engine and Cloud Storage, up to monthly limits.

Google Cloud Free trail

These can be great resources for learning how to use their services and getting certified in cloud computing.

AWS Tutorial by Edureka

Start applying what you have learned by doing it practically. Try these projects for a kickstart

Simple DevOps Project

This is just a demo project so that you can get a glimpse of how things actually work. You can deploy your own web application into the cloud.

What next?

Now,

💡 You know the fundamentals of cloud computing,

🏁 You know where and how to start,

🧰 You have the resources,

🗯️ You have the projects,

🏃 Go! Start your cloud journey now.🚀

Read the documentation, do hands-on practice, create your free account, and deploy applications.

Just go and play around the clouds.

Thank You! for giving your valuable time.🖤

Linux: A Super Hero

Poonam Pawar — Thu, 04 May 2023 15:00:09 +0000

Introduction

Linux is a powerful and versatile open-source operating system that is a Unix-like os that can be used in a variety of applications, from small-scale personal use to large-scale enterprise use.

It offers many benefits including, stability, security and flexibility, making it a popular choice for many users and organisations.

Shell

The Linux shell is a program that allows text-based interaction between the user and the operating system.

This interaction is carried out by typing commands into the interface and receiving the response in the same way.

Types of shell

Bourne Shell (sh)
Korn Shell (ksh)
C Shell (csh or tcsh)
Z Shell (zsh)
Bourne again Shell (bash)

These shells may differ in their core concepts but all of them have one common thing to communicate between the users and os.

When we log into the shell, the first thing to show up to you is your home directory.

Home Directory - /home is a system-created directory that contains the home directories for almost all users in the Linux system.

It allows user to store their personal data in the form of files and folders.

Each user in the system gets their own unique home directory with all access.

Representation- It is represented by the "tilde ~" symbol in the command line.

Home Directory = ~ (tilde)
[~]$

Commands and Arguments

To interact with the Linux system using the shell, a user has to give Commands. Like-

echo: to print a line of text on the screen use the echo command.

[~]$ echo
[~]$

when you used echo cmd, you did not tell what you want to print and as a result, it prints nothing.

That's where an Argument comes into the picture.

An Argument acts as an input to the command.

[~]$ echo Hello
Hello
[~]$

It's not always necessary to give arguments. There can be a lot of commands which can run without any arguments. eg:

[~]$ uptime
12:53  up 2 days, 22:05, 3 users, load averages: 2.89 2.61 2.79

This uptime is used to print information about how long the system has been running since the last reboot.

command <arguments>
echo = command
Hello = argument

To check your current shell use-

[~]$ echo $SHELL
/bin/bash

Fun Task:

Let's do a quick task with some basic and most important Linux commands so that you can learn by doing.

Task - Create a directory structure with 3 directories under the home directory which is already created /home/dev. These directories' name represents the categories of animal herbivorous, carnivorous and omnivorous which comes under home dir like /home/dev/herbivorous under each category there are some animals and under this some foods. Each item is a directory.

Let me explain this to you more easily with a diagram-

We will be in our home directory by default

[~]$ pwd
/home/dev

pwd command prints the present working directory which is home in our case and dev is our username.

[~]$ mkdir herbivorous

mkdir command is used to make a new directory on the given path

[~]$ mkdir carnivorous omnivorous

we can create as many directories as we want by writing them in one go as the above done.

now all three directories have been created in our home path.

[~]$ ls
herbivorous carnivorous omnivorous

ls command is used to show the list of files and folders on that particular directory.

Now we have to make directories of cows and giraffes in the herbivorous directory.

To do this, we have to change our dir from home and go inside the herbivorous dir.

[~]$ cd herbivorous
[~/herbivorous]$

the cd command is used to change the directory.

[~/herbivorous]$ pwd
/home/dev/herbivorous

[~/herbivorous]$ mkdir cow giraffe

[~/herbivorous]$ mkdir cow/grasses

here we made a new directory of grasses inside cow dir without going inside it.

[~/herbivorous]$ mkdir -p cow/grasses

here we're creating directories cow and under that grasses together by using the <-p> option.

Let's check out some more of the basic commands

mv This mv command is used to move dir from src A to dest B

[~]$ mv /home/dev/source_dir /home/dev/destination_dir

cp This command is used to copy file/dir from one src to another

$ ls
a.txt  b.txt  new

#Initially new is empty
$ ls new
#empty

$ cp a.txt b.txt new
#copying file a and b a file named new

$ ls new
a.txt  b.txt

rm This command is used to remove file/dir

$ ls
a.txt b.txt new
$ rm new
$ ls
a.txt b.txt
# file named new is being removed now

cat This command can be used in several ways. Some of them are:

$ cat a.txt
This will show the content of file a.txt

$ cat a.txt > b.txt
This will copy the content of file a.txt to b.txt file

$ cat > newfile
This will create a new file named newfile

touch This command is used to create a new file

[~]$ touch /home/dev/omnivorous/human/food.txt

Some Tips:

The alternative of cd cmd is pushd cmd. This cmd remembers the current working directory before changing to the directory you want.

let's say you're in home dir and want to go to /etc

[~] pushd /etc
/etc ~

now you can dir as many times as you want cd /var cd /tmp

and to go back to the original dir use popd

[/tmp] popd
[~]

To change the prompt - If you want to change the way how your prompt look to something else like your server name or your name itself

[~]: echo $PS1
[~]$
# current prompt lool like

[~]$ PS1="ubuntu-server:"
ubuntu-server:
# now it changes to the server name

ubuntu-server: PS1="[\d \t \u@\h:\w ] $ "
[Mon Mar 06 13:30:54 dev@macair:~ ] $
# now displaying with current date and time with user name too

date, time and username before '@' and /h /w give the hostname and the present working directory of the user, followed by '$' indicating a regular user.

Kernel

The major component of the operating system and an interface between the system's hardware and processes.

It can be thought of as a bridge between the hardware and the software components of a computer system.

[~]$ uname
Linux

Let's say you are running a program that needs access to the computer's memory. when the request is made, the kernel is responsible for granting the program access to the memory.

[~]$ uname -r
4.15.0.72-generic

This shows up in the kernel versions.

The kernel looks up the four major tasks-

Memory Management - Keeps track of how much memory is used.
Process Management - Which process can use the CPU?
Device Drivers - An interpreter between the hardware and the processes.
System Calls & Security - Receive requests for services from the processes.

Some Core Concepts

File Compression:

To inspect the size of the file use du command

du stands for Disk Usage and -sh is a humanly readable size format.

Archive File:

Grouping multiple files into a single directory or a single file, you can use tar command

-c flag specifies that the tar should create a new archive.

-v flag specifies that the tar will display the name of the file added to the archive.

-f flag specifies the name of the archive file that tar should create.

-cvf together, it will create a new archive with the specified name and add the specified files to it while displaying their names as they are added.

-xf flag is used to extract the contents from the tar file.

-czvf flag is used to compress the tar file to reduce its size.

the tar archive "mydoc.tar" was created, and its content was compressed using gzip to create a compressed tar archive named "mydoc"

  $tar -czvf mydoc.tar.gz mydoc/
  mydoc/
  mydoc/file1.txt
  mydoc/file2.txt
  mydoc/file3.txt

-c creates a new archive file.

-z compress the archive using gzip.

-v displays the verbose output during the creation of the archive.

-f specify the name of the archive file.

source:

KodeKloud

geeksforgeeks

And that's all for you to start your Linux ride. I have done my certifications in Linux Basics Course from KodeKloud.

Certificate Verification – KodeKloud

kodekloud.com

I am not promoting anything, just sharing my learning resources if you want to check them out.

You can have the training and get certified by Linux Foundation itself.

Linux Training & Certifications

Thanks for the read!🖤🐧

DevOps

Poonam Pawar — Mon, 01 May 2023 06:55:39 +0000

Those who are new to DevOps, find it quite difficult to know where to start their journey, I find it too. So, In this blog, I'll be sharing all my learnings (pre-requisite) so that you can start yours too and get your basics done.

What is DevOps?

DevOps combines development and operations to increase the efficiency, speed and security of software development, to automate all the processes and delivery compared to the traditional process.

The main key🗝 practices involved in DevOps are:

CI/Cd, IaC, Monitoring and Logging

Let me break down this for you🫵 guys!

Development + Operations = DevOps

Development, "Dev" is the process of generating code that requires any software to work which involves practices like designing, writing and testing code.

The operation, "Ops" is the process of deploying, maintaining and monitoring the software application.

Both teams work closely to ensure that the code is properly integrated with the infrastructure and other components of the system. In DevOps, they work together as a single team to break down the traditional silos between these two groups so that development and operation can be more effective and efficient.

More simply, you can think of it as making a toy house building🏢.

In the making🏗 process, it's a teamwork game to play. You need to work together with your friends to make a strong building with different tools and materials like toy bricks, glue, paint🎨 etc.

DevOps is the same, as working together in a team to build a software program using different tools and techniques like coding, testing and automation.

Learning Path:

Now, the following is the step-by-step learning path to begin your journey:-

Step 1: Getting started with GIT & GITHUB

The first and most important tool you have to learn to get started with DevOps is GIT & GITHUB.

Learn the fundamentals of version control and collaboration with Git and Github.

Git: An open-source distributed version control system that allows developers and operations teams to collaborate and keep records and save the changes made on a project. It's like saving the history of your project so that you can make changes as as many times you want without losing your previous one.

Github: A web-based platform that provides hosting for Git repositories and offers additional features for managing software development projects.

I'll not be going into the deep. I'll share the resources at the end instead.
Git & Github

Step 2: Understanding Linux and Shell Scripting

Explore the power of the Linux operating system and learn how to automate tasks with shell scripting.

Linux is a free and open-source operating system based on Unix os.

Shell Scripting is a way of writing programs that can be run on a Linux or Unix command-line interface, called a shell.

Step 3: Programming Fundamentals with Golang and Python

The next step is to learn a programming language, don't worry, you don't have to master anyone. Just for getting started you must know the basics of these.

Learn these two popular programming languages for DevOps and how they can be used to automate infrastructure and application deployments.

# This program prints Hello, world!
package main
import "fmt"

func main() {
  fmt.Println("Hello World!")
}

Step 4: Building and Deploying applications with Docker
An illustration of a docker container

Learn how to package, deploy and manage applications with a docker container.

Docker is an essential tool in DevOps. It allows developers to package and deploy their applications using containerisation technology.

Docker Deployment of your application:

Step 5: Automating workflows with Jenkins

Master the basics of CI and learn how to automate builds, tests, and deploy with Jenkins.

Creating new item on Jenkins Dashboard to set up Job

Jenkins is an open-source automation server that supports continuous integration(CI) and continuous delivery(CD) workflows which enables teams to automatically build, test and deploy their code.

Step 6: Orchestrating with Kubernetes

Dive into the world of orchestration and learn how to deploy, scale and manage containerised applications across the cluster of servers.

Kubernetes was originally developed by Google and is now maintained by the Cloud Native Computing Foundation (CNCF). It is an open-source platform for managing and orchestrating containerised applications.

Click here to: Learn Kubernetes Basics

Step 7: Infrastructure as Code with Ansible

Explore the power of configuration management and learn how to automate infrastructure with Ansible.

Ansible is a configuration management tool that allows you to define infrastructure as code and automate tasks such as configuration updates, software installation and system updates. It is a "push-based" configuration model.

sudo apt update sudo apt install software-properties-common sudo add-apt-repository --yes --update ppa:ansible/ansible sudo apt install ansible

Step 8: Provisioning cloud resources with Terraform

Discover how to define, provision and manage infrastructure with Terraform, a popular tool for infrastructure as code.

Terraform is a tool for the provisioning and management of cloud resources that allows you to create, update and destroy cloud resources such as virtual machines, databases and storage services.

Terraform

Step 9: Automating configuration with Chef and Puppet

Learn how to automate server configurations with two popular configuration tools, Chef and Puppet.

Both have similar functionality used for automating the deployment and configuration of infrastructure. Chef is a Ruby-based system automation-friendly. The automation tool used to configure, manage, deploy and orchestrate applications. These are "pull-based" configuration models.

Step 10: Integrating Security into DevOps with DevSecOps

Understand the principles of DevSecOps and how to integrate security into your DevOps workflow.

An approach to software development that integrates building security controls into every stage of the software development lifecycle from development and testing to deployment and operations.

That's all to get your basics done.

resource: (https://kubernetes.io/)

This is not the exact path you have to follow, you can jumble. There are a lot of tools out there to learn for DevOps, once you start your journey you get to know about many more. I have just given you one way of learning. Start your journey and find your own way.

Learning Project Ideas:

Some beginner-level project ideas that you should do while learning~

System Monitor - Develop a shell script that monitors system usage such as CPU, memory, and disk usage. It can send an alert to the user when certain thresholds are reached.
Multi-container application with Docker Compose - Create a multi-container application using docker-compose, which allows you to define and run a multi-docker container as a single service.
Load-balancing with Kubernetes - Configure load balancing for a Kubernetes cluster by creating a service that distributes traffic across multiple pods.
Ansible playbook for security hardening - Develop an ansible playbook that automates security hardening tasks, such as disabling unnecessary services and setting up firewalls.
Jenkins pipeline - Create a Jenkins pipeline that automates the entire software delivery process including building, testing and deploying applications.

Learning Resources:
There are a few free amazing resources that you must check out and get started with:

Kunal Kushwaha: This is an amazing free DevOps boot camp by Kunal Kushwaha on his youtube channel to get started with.

TechWorld with Nana: To learn basic concepts of various tools, you can check this youtube channel.

That DevOps Guy: This youtube channel is completely based on DevOps learnings by MarcelDempers.

Pradumna Saraf: This is one of the best resources by Pradumna Saraf you can find on his GitHub to get notes, play labs or anything else to get started.

Sometimes it becomes quite difficult to set up systems for running DevOps tools getting head clear with the doubts. So a paid course can also be an option in this scenario.

Kodekloud: This is one of the best DevOps courses out there by Mumshad Mannambeth. They have hands-on lab practices after lecture videos and also have a playground where you can do whatever you have learned through the lectures. And that's what I'm doing too.

TechWorld with Nana: Another amazing paid course by Nana you can do.

EddieHub

Community of inclusive Open Source people - Collaboration 1st, Code 2nd! Join our GitHub Org 👇 - EddieHub

github.com

Join this amazing GitHub community to get started with your open-source journey while learning DevOps to grow faster. They welcome newcomers open handedly.

Learn -> Contribute -> Collaborate -> Grow

That's All!! GO🏄‍♀️ and start your DevOps journey today!!

Forem: Poonam Pawar

AWS Project using SHELL SCRIPTING for DevOps

Introduction

What does this project do?

Build the project

Create EC2 Instance

Create the Script

Test the script

Apply CronJob

Use Cases

Kubernetes Cluster Maintenance

Introduction✍️

Cluster Upgradation♿

Backup & Restore Methods🛗

Scaling Clusters📶

Simply Deploying Kubernetes Workloads

Introduction✍️

Every other Relationship👀

Pre-requisites👉

ReplicaSets🗂️

Deployments🏗️

StatefulSets🏗️📑

DaemonSets🤖

JOBS🧑‍💻

CronJob🥸

Kubernetes Networking

Introduction✍️

PODS📦

Network Policies📋

Services🪖

Services Types📑:

CNI (Computer Network Interface)🌐

CNI in k8s🕸️

DNS in k8s📡

Ingress🔵

Ingress Controller🔹

Kubernetes PODS & SERVICES Discovery

Introduction✍️

How to expose Kubernetes workloads to the outside world using Services?🌏

Pre-requisites👇

Exposing Kubernetes workloads to the outside world👶

How to discover Services and Pods within a Kubernetes cluster using DNS and other mechanisms?🧐

Service Discovery🕵️

Labels📋

Selectors✅

Pod Discovery Using DNS🌐

Pod Discovery Using Environment Variables🔗

Kubernetes: Storage & Security

Introduction✍️

Storage🛢️

Volumes🎚️

Persistent Volumes📻

Persistent Volume Claims📑

Storage Classes🗑️

StatefulSets🏗️

Security🕵️

RBAC (Role-Based Access Control)🛂

Pod Security Policies (PSPs)📦

Secrets🧑‍💻

Network Policies📋

TLS (Transport Layer Security)🌐

Server Certificates of the Servers🔏

Client Certificate of the Clients🔐

Kubernetes: Architecture, Components, Installation & Configuration

Introduction🧑‍🦯

Kubernetes🕸️ -

Architecture 👾

Nodes📦 -

Cluster🗂️ -

Master Node📑 -

Worker Node🧾 -

Components🧰

kube-apiserver🔗:

etcd🖇️:

kube-scheduler🧷:

Container runtime📎:

kubelet🔎:

kube-proxy📈:

Installation⚙️ & Configuration🌐

Cloud Journey