Forem: Nikhil Raj A

Building a Netflix Clone with DevSecOps: A Complete DevSecOps Project.

Nikhil Raj A — Sun, 13 Jul 2025 04:27:14 +0000

Introduction

In today’s rapidly evolving cloud landscape, DevSecOps is no longer optional — it is essential for delivering secure, scalable, and high-quality applications. This project demonstrates how to deploy a Netflix Clone on AWS using Jenkins CI/CD pipelines, Docker, SonarQube, Trivy, Prometheus, and Grafana while adhering to DevSecOps best practices.

Prerequisites

AWS Account.
Basic Git, Docker,Kubernetes and Linux knowledge.
Jenkins and CI/CD familiarity.
TMDB API Key for the Netflix clone.

Phase 1 : Initial Setup and Deployment

Step 1: Launch EC2 (Ubuntu 22.04):

Provision an EC2 instance on AWS with Ubuntu 22.04, t2.large and 25 GB storage.
Connect to the instance using SSH.

Step 2: Clone the Code:

Update all the packages and then clone the code.
Clone your application’s code repository onto the EC2 instance:

git clone https://github.com/NikhilRaj-2003/devsecops-netflix-clone.git

Step 3: Install Docker and Run the App Using a Container:

Set up Docker on the EC2 instance:


sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker $USER  # Replace with your system's username, e.g., 'ubuntu'
newgrp docker
sudo chmod 777 /var/run/docker.sock

Build and run your application using Docker containers:

docker build -t netflix .
docker run -d --name netflix -p 8081:80 netflix:latest
#to delete
docker stop <containerid>
docker rmi -f netflix

It will show an error cause you need API key

Step 4: Get the API Key:

Open a web browser and navigate to TMDB (The Movie Database) website.
Click on “Login” and create an account.
Once logged in, go to your profile and select “Settings.”
Click on “API” from the left-side panel.
Create a new API key by clicking “Create” and accepting the terms and conditions.
Provide the required basic details and click “Submit.”
You will receive your TMDB API key.

Now recreate the Docker image with your api key:

docker build --build-arg TMDB_V3_API_KEY=<your-api-key> -t netflix .

Phase 2: Intializing Security for checking code quality and image quality .

Install SonarQube and Trivy:

Install SonarQube and Trivy on the EC2 instance to scan for vulnerabilities.

sonarqube :

docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

To access:

publicIP:9000 (by default username & password is admin)

Trivy :

sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy

to scan image using trivy

trivy image <imageid>

Phase 3: CI/CD Setup.

1. Install Jenkins for Automation:

Install Jenkins on the EC2 instance to automate deployment: Install Java

sudo apt update 
sudo apt install fontconfig openjdk-17-jre 
java -version openjdk version "17.0.8" 2023–07–18
 OpenJDK Runtime Environment (build 17.0.8+7-Debian-1deb12u1) 
OpenJDK 64-Bit Server VM (build 17.0.8+7-Debian-1deb12u1, mixed mode, sharing) 
#jenkins sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \ 
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key 
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \ 
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \ /etc/apt/sources.list.d/jenkins.list > /dev/null 
sudo apt-get update sudo apt-get install jenkins 
sudo systemctl start jenkins 
sudo systemctl enable jenkins

Access Jenkins in a web browser using the public IP of your EC2 instance.
publicIp:8080

2. Install Necessary Plugins in Jenkins:

Goto Manage Jenkins →Plugins → Available Plugins →

Install below plugins :

1 Eclipse Temurin Installer (Install without restart)

2 SonarQube Scanner (Install without restart)

3 NodeJs Plugin (Install Without restart)

4 Email Extension Plugin

Configure Java and Nodejs in Global Tool Configuration

Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save

SonarQube

Create the token
Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this
After adding sonar token
Click on Apply and Save

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

Create a Jenkins webhook

Configure CI/CD Pipeline in Jenkins:

Create a CI/CD pipeline in Jenkins to automate your application deployment.

pipeline {
    agent any
    tools {
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME = tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace') {
            steps {
                cleanWs()
            }
        }
        stage('Checkout from Git') {
            steps {
                git branch: 'main', url: https://github.com/NikhilRaj-2003/devsecops-netflix-clone.git'
            }
        }
        stage("Sonarqube Analysis") {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
                    -Dsonar.projectKey=Netflix'''
                }
            }
        }
        stage("quality gate") {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
    }
}

Certainly, here are the instructions without step numbers:

Install Dependency-Check and Docker Tools in Jenkins

Install Dependency-Check Plugin:

Go to “Dashboard” in your Jenkins web interface.
Navigate to “Manage Jenkins” → “Manage Plugins.”
Click on the “Available” tab and search for “OWASP Dependency-Check.”
Check the checkbox for “OWASP Dependency-Check” and click on the “Install without restart” button.

Configure Dependency-Check Tool:

After installing the Dependency-Check plugin, you need to configure the tool.
Go to “Dashboard” → “Manage Jenkins” → “Global Tool Configuration.”
Find the section for “OWASP Dependency-Check.”
Add the tool’s name, e.g., “DP-Check.”
Save your settings.

Install Docker Tools and Docker Plugins:

Go to “Dashboard” in your Jenkins web interface.
Navigate to “Manage Jenkins” → “Manage Plugins.”
Click on the “Available” tab and search for “Docker.”
Check the following Docker-related plugins:
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step

Click on the “Install without restart” button to install these plugins.

Add DockerHub Credentials:

To securely handle DockerHub credentials in your Jenkins pipeline, follow these steps:
Go to “Dashboard” → “Manage Jenkins” → “Manage Credentials.”
Click on “System” and then “Global credentials (unrestricted).”
Click on “Add Credentials” on the left side.
Choose “Secret text” as the kind of credentials.
Enter your DockerHub credentials (Username and Password) and give the credentials an ID (e.g., “docker”).
Click “OK” to save your DockerHub credentials.

Now, you have installed the Dependency-Check plugin, configured the tool, and added Docker-related plugins along with your DockerHub credentials in Jenkins. You can now proceed with configuring your Jenkins pipeline to include these tools and credentials in your CI/CD process.


pipeline{
    agent any
    tools{
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage('Checkout from Git'){
            steps{
                git branch: 'main', url: 'https://github.com/N4si/DevSecOps-Project.git'
            }
        }
        stage("Sonarqube Analysis "){
            steps{
                withSonarQubeEnv('sonar-server') {
                    sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
                    -Dsonar.projectKey=Netflix '''
                }
            }
        }
        stage("quality gate"){
           steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token' 
                }
            } 
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
        stage('OWASP FS SCAN') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }
        stage("Docker Build & Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){   
                       sh "docker build --build-arg TMDB_V3_API_KEY=<your-api-key>  -t netflix ."
                       sh "docker tag netflix nikhilaj2003/netflix:latest "
                       sh "docker push nikhilaj2003/netflix:latest "
                    }
                }
            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image nikhilaj2003/netflix:latest > trivyimage.txt" 
            }
        }
        stage('Deploy to container'){
            steps{
                sh 'docker run -d -p 8081:80 nikhilaj2003/netflix:latest'
            }
        }
    }
}

If you get docker login failed error , then run this commands on your EC2 instance. Then build again .

sudo su
sudo usermod -aG docker jenkins
sudo systemctl restart jenkins

Phase 4: Monitoring

Install Prometheus and Grafana:
Set up Prometheus and Grafana to monitor your application.

Installing Prometheus:

First, create a dedicated Linux user for Prometheus and download Prometheus:

sudo useradd - system - no-create-home - shell /bin/false prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.47.1/prometheus-2.47.1.linux-amd64.tar.gz

Extract Prometheus files, move them, and create directories:

tar -xvf prometheus-2.47.1.linux-amd64.tar.gz
cd prometheus-2.47.1.linux-amd64/
sudo mkdir -p /data /etc/prometheus
sudo mv prometheus promtool /usr/local/bin/
sudo mv consoles/ console_libraries/ /etc/prometheus/
sudo mv prometheus.yml /etc/prometheus/prometheus.yml

Set ownership for directories:

sudo chown -R prometheus:prometheus /etc/prometheus/ /data/

Create a systemd unit configuration file for Prometheus:

sudo nano /etc/systemd/system/prometheus.service

Add the following content to the prometheus.service file:

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=prometheus
Group=prometheus
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/data \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.enable-lifecycle
[Install]
WantedBy=multi-user.target

Here’s a brief explanation of the key parts in this prometheus.service file:

User and Group specify the Linux user and group under which Prometheus will run.
ExecStart is where you specify the Prometheus binary path, the location of the configuration file (prometheus.yml), the storage directory, and other settings.
web.listen-address configures Prometheus to listen on all network interfaces on port 9090.
web.enable-lifecycle allows for management of Prometheus through API calls.

Enable and start Prometheus:

sudo systemctl enable prometheus sudo systemctl start prometheus

Verify Prometheus’s status:

sudo systemctl status prometheus

You can access Prometheus in a web browser using your server’s IP and port 9090:
http://<your-server-ip>:9090

Installing Node Exporter:

Create a system user for Node Exporter and download Node Exporter:

sudo useradd --system --no-create-home --shell /bin/false node_exporter
wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz

Extract Node Exporter files, move the binary, and clean up:

tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz
sudo mv node_exporter-1.6.1.linux-amd64/node_exporter /usr/local/bin/
rm -rf node_exporter*

Create a systemd unit configuration file for Node Exporter:

sudo nano /etc/systemd/system/node_exporter.service

Add the following content to the node_exporter.service file:

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/node_exporter --collector.logind
[Install]
WantedBy=multi-user.target

Replace --collector.logind with any additional flags as needed.
Enable and start Node Exporter:

sudo systemctl enable node_exporter sudo systemctl start node_exporter

Verify the Node Exporter’s status:

sudo systemctl status node_exporter

You can access Node Exporter metrics in Prometheus.

Configure Prometheus Plugin Integration:

Integrate Jenkins with Prometheus to monitor the CI/CD pipeline.
Prometheus Configuration:
To configure Prometheus to scrape metrics from Node Exporter and Jenkins, you need to modify the prometheus.yml file. Here is an example prometheus.yml configuration for your setup:

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['localhost:9100']
  - job_name: 'jenkins'
    metrics_path: '/prometheus'
    static_configs:
      - targets: ['<your-jenkins-ip>:<your-jenkins-port>']

Make sure to replace <your-jenkins-ip> and <your-jenkins-port> with the appropriate values for your Jenkins setup.
Check the validity of the configuration file:

promtool check config /etc/prometheus/prometheus.yml

Reload the Prometheus configuration without restarting:

curl -X POST http://localhost:9090/-/reload

You can access Prometheus targets at:

http://<your-prometheus-ip>:9090/targets

Grafana

Install Grafana on Ubuntu 22.04 and Set it up to Work with Prometheus

Step 1: Install Dependencies:

First, ensure that all necessary dependencies are installed:

sudo apt-get update
sudo apt-get install -y apt-transport-https software-properties-common

Step 2: Add the GPG Key:

Add the GPG key for Grafana:

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

Step 3: Add Grafana Repository:

Add the repository for Grafana stable releases:

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

Step 4: Update and Install Grafana:

Update the package list and install Grafana:

sudo apt-get update
sudo apt-get -y install grafana

Step 5: Enable and Start Grafana Service:

To automatically start Grafana after a reboot, enable the service:

sudo systemctl enable grafana-server

Then, start Grafana:

sudo systemctl start grafana-server

Step 6: Check Grafana Status:

Verify the status of the Grafana service to ensure it’s running correctly:

sudo systemctl status grafana-server

Step 7: Access Grafana Web Interface:

Open a web browser and navigate to Grafana using your server’s IP address. The default port for Grafana is 3000. For example:

http://<your-server-ip>:3000

You’ll be prompted to log in to Grafana. The default username is “admin,” and the default password is also “admin.”

Step 8: Change the Default Password:

When you log in for the first time, Grafana will prompt you to change the default password for security reasons. Follow the prompts to set a new password.

Step 9: Add Prometheus Data Source:

To visualize metrics, you need to add a data source. Follow these steps:

Click on the gear icon (⚙️) in the left sidebar to open the “Configuration” menu.
Select “Data Sources.”
Click on the “Add data source” button.
Choose “Prometheus” as the data source type.
In the “HTTP” section:
Set the “URL” to http://localhost:9090 (assuming Prometheus is running on the same server).
Click the “Save & Test” button to ensure the data source is working.

Step 10: Import a Dashboard:

To make it easier to view metrics, you can import a pre-configured dashboard. Follow these steps:

Click on the “+” (plus) icon in the left sidebar to open the “Create” menu.
Select “Dashboard.”
Click on the “Import” dashboard option.
Enter the dashboard code you want to import (e.g., code 1860).
Click the “Load” button.
Select the data source you added (Prometheus) from the dropdown.
Click on the “Import” button.

You should now have a Grafana dashboard set up to visualize metrics from Prometheus.

Grafana is a powerful tool for creating visualizations and dashboards, and you can further customize it to suit your specific monitoring needs.

That’s it! You’ve successfully installed and set up Grafana to work with Prometheus for monitoring and visualization.

Configure Prometheus Plugin Integration:

Integrate Jenkins with Prometheus to monitor the CI/CD pipeline.

Phase 5: Notification

Implement Notification Services:

Set up email notifications in Jenkins or other notification mechanisms.

Phase 6: Kubernetes

Create Kubernetes Cluster with Nodegroups

In this phase, you’ll set up a Kubernetes cluster with node groups. This will provide a scalable environment to deploy and manage your applications.

Monitor Kubernetes with Prometheus

Prometheus is a powerful monitoring and alerting toolkit, and you’ll use it to monitor your Kubernetes cluster. Additionally, you’ll install the node exporter using Helm to collect metrics from your cluster nodes.

Install Node Exporter using Helm

To begin monitoring your Kubernetes cluster, you’ll install the Prometheus Node Exporter. This component allows you to collect system-level metrics from your cluster nodes. Here are the steps to install the Node Exporter using Helm:

Add the Prometheus Community Helm repository:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

2 .Create a Kubernetes namespace for the Node Exporter:

kubectl create namespace prometheus-node-exporter

Install the Node Exporter using Helm:

helm install prometheus-node-exporter prometheus-community/prometheus-node-exporter - namespace

Add a Job to Scrape Metrics on nodeip:9001/metrics in prometheus.yml:

Update your Prometheus configuration (prometheus.yml) to add a new job for scraping metrics from nodeip:9001/metrics. You can do this by adding the following configuration to your prometheus.yml file:

- job_name: 'Netflix'
    metrics_path: '/metrics'
    static_configs:
      - targets: ['node1Ip:9100']

Replace ‘your-job-name’ with a descriptive name for your job. The static_configs section specifies the targets to scrape metrics from, and in this case, it’s set to nodeip:9001.

Don’t forget to reload or restart Prometheus to apply these changes to your configuration.

To deploy an application with ArgoCD, you can follow these steps, which I’ll outline in Markdown format:

Deploy Application with ArgoCD

Install ArgoCD:

You can install ArgoCD on your Kubernetes cluster by following the instructions provided in the EKS Workshop documentation.

Set Your GitHub Repository as a Source:

After installing ArgoCD, you need to set up your GitHub repository as a source for your application deployment. This typically involves configuring the connection to your repository and defining the source for your ArgoCD application. The specific steps will depend on your setup and requirements.

Create an ArgoCD Application:

name: Set the name for your application.
destination: Define the destination where your application should be deployed.
project: Specify the project the application belongs to.
source: Set the source of your application, including the GitHub repository URL, revision, and the path to the application within the repository.
syncPolicy: Configure the sync policy, including automatic syncing, pruning, and self-healing.

Access your Application :

To Access the app make sure port 30007 is open in your security group and then open a new tab paste your NodeIP:30007, your app should be running.

Phase 7: Cleanup

Cleanup AWS EC2 Instances:

Terminate AWS EC2 instances that are no longer needed.

Conclusion

This project demonstrates a complete DevSecOps pipeline by securely building, testing, and deploying a Netflix Clone on AWS using Jenkins, Docker, SonarQube, and Prometheus-Grafana monitoring. It showcases practical cloud automation and security best practices while strengthening your skills in CI/CD and cloud-native deployments.

How to Host a Static Website on AWS S3 Using Terraform: Step-by-Step Guide for DevOps Beginners.

Nikhil Raj A — Tue, 08 Jul 2025 14:43:08 +0000

Introduction

As a DevOps beginner, deploying your first static website using Terraform on AWS S3 is one of the best hands-on projects to understand Infrastructure as Code (IaC), AWS services, and real-world DevOps workflows.

In this guide, you will learn how to automate S3 bucket creation, configure it for static website hosting, and upload your website files using Terraform.

What is Terraform?

Terraform is an open-source Infrastructure as Code (IaC) tool created by HashiCorp that allows you to define, provision, and manage cloud infrastructure using code.

Terraform setup : https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli

Prerequisites

An AWS account.
Terraform installed on your system( Either on VS code or on EC2 instance).
AWS CLI configured (aws configure).
Basic HTML files (index.html, style.css).

Why this project?

Get hands-on with Terraform basics.
Understand S3 static website hosting.
Automate infrastructure, avoiding manual AWS console clicks.

Step 1: Create Your Terraform Project

Create where all the details regarding the project is present .

resource "aws_s3_bucket" "bucket" {
  bucket = var.bucket_name
}
resource "aws_s3_bucket_ownership_controls" "bucket" {
  bucket = aws_s3_bucket.bucket.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}
resource "aws_s3_bucket_public_access_block" "bucket" {
  bucket = aws_s3_bucket.bucket.id
  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}
resource "aws_s3_bucket_acl" "bucket" {
  depends_on = [    aws_s3_bucket_ownership_controls.bucket,
    aws_s3_bucket_public_access_block.bucket
  ]
  bucket = aws_s3_bucket.bucket.id
  acl    = "public-read"
}
resource "aws_s3_bucket_policy" "public_read" {
  bucket = aws_s3_bucket.bucket.id
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [      {
        Effect = "Allow",
        Principal = "*",
        Action = "s3:GetObject",
        Resource = "${aws_s3_bucket.bucket.arn}/*"
      }
    ]
  })
}
resource "aws_s3_object" "index" {
  bucket       = aws_s3_bucket.bucket.id
  key          = "index.html"
  source       = "index.html"
  content_type = "text/html"
}
resource "aws_s3_object" "style" {
  bucket       = aws_s3_bucket.bucket.id
  key          = "style.css"
  source       = "style.css"
  content_type = "text/css"
}
resource "aws_s3_object" "error" {
  bucket       = aws_s3_bucket.bucket.id
  key          = "error.html"
  source       = "error.html"
  content_type = "text/html"
}
resource "aws_s3_bucket_website_configuration" "website" {
  bucket = aws_s3_bucket.bucket.id
  index_document {
    suffix = "index.html"
  }
  error_document {
    key = "error.html"
  }
  depends_on = [aws_s3_bucket_acl.bucket]
}

create , where all the variables are present .

variable "bucket_name" {
  default = "myterraformbucket0019"
}

create , which is used to mention which cloud Provider we are using for this project .

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.10.0"
    }
  }
}
provider "aws" {
  region = "ap-south-1"
}

Create

output "websiteendpoint" {
    value = "aws_s3_bucket.bucket.website_endpoint"
}

Step 2: Add Website Files

Create index.html and style.css for the making of the website .

index.html : https://github.com/NikhilRaj-2003/terraform-s3-static-website-hosting/blob/main/design/index.html
style.css : https://github.com/NikhilRaj-2003/terraform-s3-static-website-hosting/blob/main/design/style.css

Step 3: Initialize and Deploy

terraform init : initializes your Terraform working directory.

It downloads the required provider plugins (like AWS, Azure) needed to work with your cloud resources.
It sets up the backend configuration (where your state file will be stored, like local or S3).
It prepares the .terraform folder inside your project.

terraform plan: Used to preview changes Terraform will make before applying .Reads your Terraform configuration files (.tf files).

Checks the current state of your infrastructure.
Generates a detailed execution plan showing actions needed to match your desired configuration.
Indicates which resources will be created, updated, or destroyed.
Helps you review and verify expected changes safely.
Reduces the chances of accidental modifications to your infrastructure.

terraform apply: Used to execute actions required to reach the desired state of your infrastructure.

Reads your Terraform configuration files (.tf files).
Runs after reviewing the plan shown by terraform plan .
Automatically creates, updates, or deletes resources on your cloud provider.
Prompts for confirmation before proceeding (unless using -auto-approve).

After giving terraform apply command it creates S3-Bucket in AWS Account , without opening AWS .

Step 4: Access Your Website

Go to your AWS S3 bucket.
Go to Properties → Static Website Hosting.
Copy the bucket website endpoint URL
Open in your browser and see your Terraform-hosted static website live.

What You Learned

1.Hosting a static website on AWS S3 using Terraform.
2.Automating bucket creation, ACL, and public read policies using IaC.
3.Practical DevOps workflow for portfolio-building.

Conclusion

Through this project, you have learned how to host a static website on AWS S3 using Terraform, enabling you to understand and apply the principles of Infrastructure as Code (IaC). You automated the creation of an S3 bucket, configured it for static website hosting, and managed public access using Terraform, eliminating manual steps in the AWS console.

Secure and Simple: Enabling Passwordless SSH Login on Linux Servers

Nikhil Raj A — Mon, 23 Jun 2025 17:43:58 +0000

Introduction

When managing multiple Linux servers, frequent SSH logins can become repetitive and time-consuming — especially when automating tasks, running scripts remotely, or setting up services like Ansible. A passwordless SSH connection offers a secure and convenient way to streamline server communication without compromising on security.

In this guide, we’ll walk through how to set up passwordless SSH authentication between two Linux servers using SSH key-based authentication. This method eliminates the need to manually enter a password every time you connect from one server to another. Instead, it uses a public-private key pair to authenticate access securely.

Whether you’re a system administrator, DevOps engineer, or just someone managing a few Linux boxes, this setup will make your workflow more efficient and secure.

What is SSH?

SSH (Secure Shell) is a cryptographic network protocol used to securely connect to remote systems over an unsecured network. It enables users to access and control remote machines, typically via a command-line interface, while encrypting all communications to protect against eavesdropping, tampering, and impersonation.

Key Features:

Encrypted communication
Remote command execution
Tunnel creation for port forwarding

Prerequisites

Two Linux servers (Server A and Server B).
Administrative access to both servers.
AWS Account.

Step 1 : Create 2 Linux Servers from EC2

1 . Provide a name for the instance.

2 . Give number of instances as 2, because we are gonna need 2 servers to interact with and without 2 servers the passwordless connection won’t be possible.

Select Amazon Linux 2023 AMI or Ubuntu AMI based on your convenience.
Select the required key pair.
Click on Launch Instance , after setting up the instance.

Step 2 : Generate SSH Key Pair on Server-A

Connect the server1 using SSH or Physical accesss.
Provide a name for Server-A as Server1 using the command

    sudo hostnamectl set-hostname server1

Generate an SSH key pair by running the following command on Server1, then Click enter 3 times for confirming the generation of the key — pair.

After generating the key pair on server1 , there would be 2 key-pairs known as Public-key (id_rsa.pub) and Private_key (id_rsa).
To view the content inside of both the keys , then enter the following command :

Step 3: Copy the Public key into the Server B

Change the hostname of the Server B to Server2 using the above shown command .
Copy the Public key content and paste it into the Server2. The Public key must be pasted into a authorized_keys which allows passwordless SSH from the machine which holds the corresponding Private-key. The command is :

echo "public key of server1" >> authorized_keys

The correct example of the above command is given below :

After pasting the Public-key of server1 into server2 , then go to server1 for accessing server2 from server1.

Step 4 : Test the Passwordless Connection

Attempt to access Server2 from Server1 :

In order to access server 2 from server1 you need to enter or run a command on server1(Server A):

So the actual command of accessing the server2 from server1 and the command must be run on server1 itself or else the connection would not happen. Run (ssh ec2-user@13.203.67.192) this on server1, 13.203.67.192 is the public IP of Server 2 .

Troubleshooting Tips

If the passwordless connection doesn’t work, here are some troubleshooting steps:

Check Permissions: Ensure the .ssh directory on both Server A and Server B has the correct permissions. It should be owned by the user and have restricted permissions:

chmod 700 ~/.ssh 
chmod 600 ~/.ssh/authorized_keys

Key File Names: Verify that you are using the default key names (id_rsa and id_rsa.pub) or the names you specified during key generation.
SSH Agent: Make sure you have added the private key to the SSH agent on Server A using ssh-add:

ssh-add ~/.ssh/id_rsa

Conclusion

Setting up a passwordless SSH connection between two Linux servers is a simple yet powerful way to streamline secure access, automate tasks, and improve system administration efficiency. By generating an SSH key pair and copying the public key to the remote server, you eliminate the need to enter a password each time you connect — without compromising on security.

Python vs Bash Scripting: Differences, Advantages & When to Use Each

Nikhil Raj A — Sat, 21 Jun 2025 08:16:16 +0000

“ Should I write this script in Python or Bash? “
That one question has haunted developers and DevOps engineers alike. On the surface, both get the job done — but under the hood, they’re built for different worlds. In this blog, we’ll break down the real-world differences between Bash and Python scripting, their pros and most importantly — when you should use each.

What Is Scripting?

Scripting, at its core, is about giving your computer a to-do list — a set of instructions it can follow automatically, step by step. Think of it like writing a recipe: instead of telling a person how to cook a dish, you’re telling the computer how to carry out a task.

Why Scripting Matters ?

Let’s be honest — nobody enjoys doing the same repetitive tasks every day. Whether it’s moving files, cleaning up logs, or setting up your dev environment for the 10th time this week, it gets old fast. That’s where scripting comes in — and it’s a total game changer.

Scripting is like giving your computer a checklist and saying, “You handle this. I’ve got better things to do.” Once you write a script, it takes over the boring stuff — no complaints, no forgetfulness, just results. Scripting helps you reduce errors, save time, and focus on the stuff that actually matters. Trust me the moment you start automating even the smallest tasks, you’ll wonder how you ever lived without it.

Some of the most popular scripting languages include:

Bash for system tasks on Unix/Linux.
Python for more complex automation and cross-platform scripting.
JavaScript for client-side browser scripting.
PowerShell for automation in Windows environments.

Bash Scripting — The Command-Line Ninja

Bash (Bourne Again Shell) is the default shell on most Linux distributions and macOS. It’s designed to interact directly with the operating system. Think of Bash as a glue that connects other CLI tools together.

What Makes Bash So Special?

Bash isn’t just that black box you type commands into — it’s much more than that. It’s like your backstage pass to the entire operating system. With Bash, you’re not just running commands, you’re connecting them, chaining them, and automating them like a pro.

Think of Bash as your personal assistant for the command line. You can write a small script to do boring, repetitive things — like moving files, cleaning up folders, or checking system health — and Bash will handle it all for you, without breaking a sweat.

The real magic? Bash lets you glue together tons of other tools — like ls, grep, awk, sed, find, and curl. On their own, these tools are powerful. But with Bash, you can make them work together like a well-rehearsed orchestra. One command filters, another searches, another renames — and Bash makes it all flow smoothly in just a few lines of script.

Advantages of Bash Scripting

Perfect for interacting with the OS, managing files, users, permissions, services, etc.
Executes quickly with minimal overhead — ideal for short scripts or quick fixes.
Easily connects tools like grep, awk, sed, find, etc. in one-liners or scripts.
No need for setup — just open the terminal and start scripting.
Bash is often the go-to choice for scheduled tasks and sysadmin routines.

Few Common Bash Commands :

ls — it is a command that is used to List Directory , Files and also Folders in long format (with all the permissions , Date and size)

ls -al

cd — it is a command which is used to change a directory or also move from one directory to an another .

cd /var/log

Use cd .. to go up one level, or cd alone to return to your home directory.

touch — this is used to create a New Empty File . For example shown below it creates a text file called report.txt.

touch report.txt

mkdir — its the most commonly used command when your required to make a Directory because without making a directory you can’t survive. Now in the below example it creates a (dir) called projects.

mkdir projects

rm — used to remove Files or Directories recursively and forcefully . But be carefull because there’s no undo.

rm -rf old_file.txt

cp — this is used to Copy Files or Folders into your desired location or directory. Use -r for copying directories: cp -r source/ dest/

cp file.txt backup.txt

mv — Moves _data.csv_ into the _archive_ folder. You can also use it to rename: mv oldname.txt newname.txt .

mv data.csv archive/data.csv

echo — commanly used to display the content or Print something in the Terminal. For an example “Hello, World!” would be printed onto the screen.

echo "Hello, World!"

cat — this command is used to view File Contents with opening the file itself .

cat notes.txt

grep — command mainly used for searching or matching. Used for Text in files, file names present inside a directory.

grep "error" logfile.txt

Python Scripting — The Swiss Army Knife of Automation

Python wasn’t built solely for scripting, but it’s one of the best tools out there when it comes to getting things done efficiently. It’s like that reliable friend who somehow knows how to fix your Wi-Fi, automate your spreadsheet, and build a website — all before lunch. The beauty of Python lies in its readability and simplicity. You don’t need to write 20 lines of code to do something basic. Want to rename 500 files? Scrape data from a website? Monitor a folder for changes? Python makes all of that feel incredibly straightforward.

And thanks to its massive library ecosystem — from os and shutil for file handling, to requests for working with APIs, to pandas for data wrangling — you rarely start from scratch. It’s versatile enough to automate daily tasks, yet powerful enough to build entire applications. Whether you’re a beginner writing your first script or a pro building robust automation pipelines, Python is the kind of language that scales with you — and always has your back.

What Makes Python So Special?

Python is special because it’s simple, powerful, and insanely versatile. The code reads like plain English, so it’s easy to learn and easy to remember. Whether you’re automating tasks, building websites, crunching data, or diving into AI — Python can handle it all. Plus, with thousands of libraries, there’s a tool for pretty much anything you want to do. It’s the kind of language that grows with you, no matter where you start.

Advantages of Python Scripting

Clean syntax that feels like English — great for beginners and large teams.
From file handling to web scraping to machine learning — there’s a library for almost everything.
Perfect for logic-heavy tasks, data manipulation, API integration, and beyond.
Python scripts run smoothly on Windows, macOS, and Linux.
You can start with a simple script and grow it into a full-blown application.

Few Common Python Commands :

print() — command used to displays text or variables on the screen.

print("Hello, world!")

input() — commonly used to take input from the user_._

name = input("What's your name? ")

len() — Returns the length of a string, list, or other data types.

len("Python")

type() — Tells you the data type (e.g., int, str, list).

type(42)

range() —Generates a sequence of numbers, often used in loops_._

for i in range(5):
    print(i)

if, elif, else — command widely used for decision-making in your script. It’s outcome solely depends on the conditions.

if age < 18:
    print("Minor")
else:
    print("Adult")

def — Used to define functions (reusable blocks of code).

def greet():
    print("Hello!")

import — Lets you use built-in or used to extract the external modules

import math
print(math.sqrt(25))

list.append() — Adds an item to the end of a list_._

fruits = ["apple", "banana"]
fruits.append("orange")

open() — Opens a file for reading or writing

file = open("data.txt", "r")

Python vs Bash — Side-by-Side Example

Task: List all **.log** files and count how many lines contain the word “error”

Bash Script:

#!/bin/bash
for file in *.log; do
  echo "$file: $(grep -i error "$file" | wc -l) error(s)"
done

Python Script:

#!/usr/bin/env python3
import glob
for file in glob.glob("*.log"):
    with open(file, "r") as f:
        count = sum(1 for line in f if "error" in line.lower())
    print(f"{file}: {count} error(s)")

Observation:

Bash is concise, efficient, and perfect for file processing.
Python is clearer, easier to maintain, and handles edge cases more gracefully.

When to Use Bash vs Python: The Right Tool for the Right Task

Let’s be real — when you’re diving into scripting, it’s not about which language is better. It’s about which one makes your life easier for the task you’re tackling.

If you’re working closely with the Linux terminal, Bash is often your best friend. It’s great for those quick-and-dirty tasks like moving files around, starting or stopping services, scheduling cron jobs, or stringing together commands with pipes. Bash is fast, lightweight, and made for interacting with the shell. It really shines in DevOps work, like managing EC2 instances, running shell scripts during deployments, or automating things through AWS CLI.

Now, if your task involves more logic or data crunching, Python is the way to go. Need to parse a massive log file? Read and write JSON or CSV? Call APIs? Handle errors gracefully and keep your script maintainable? Python does all that and more. It’s clean, powerful, and has a huge set of libraries that make complex tasks feel simple. It’s also great if your script might evolve into something bigger over time — like a command-line tool, automation framework, or even a web service.

Sometimes, though, the smartest move is to use both. For example, you might use a Bash script to keep an eye on your system, and then let Python jump in when there’s real work to do — like processing data or sending out a notification. It’s a powerful combo: Bash handles the grunt work, Python brings the brains.

So here’s the bottom line:

Use Bash when you’re doing quick shell-level stuff.
Use Python when your logic gets heavier or your task gets smarter.

Conclusion

Choosing between Bash and Python isn’t about picking a winner — it’s about using the right tool for the job. Bash is unbeatable for quick, low-level system tasks and chaining CLI commands like a pro. Python steps in when your scripts need structure, logic, or cross-platform flexibility. In reality, the best automation setups often use both, playing to each of their strengths. So instead of asking “Which one should I learn?”, ask “When should I use which?” Master both, and you won’t just write scripts — you’ll build smart, elegant solutions that actually make your life easier.

Automatically Create Timestamped S3 Buckets with AWS Lambda and Send Gmail Alerts via SNS

Nikhil Raj A — Thu, 19 Jun 2025 15:27:53 +0000

Learn how to automatically create timestamped S3 buckets using AWS Lambda and get instant email alerts via Amazon SNS to your Gmail. Step-by-step guide for automation on AWS.

Introduction

Amazon S3 (Simple Storage Service) is a powerful cloud storage solution provided by AWS that allows users to store and retrieve any amount of data at any time. In many real-world scenarios, organizations require automation to manage data more effectively. One such use case is the automated creation of new S3 buckets with timestamps whenever a new object is uploaded to an existing bucket. This approach not only helps in organizing data chronologically but also enables better tracking and archiving of uploads.

To further enhance this automation, integrating email notifications ensures that users or administrators are immediately alerted when an upload occurs. This is accomplished by leveraging AWS services such as S3 Event Notifications, Lambda Functions, SNS (Simple Notification Service), and CloudWatch. The Lambda function responds to upload events in the source bucket, dynamically creates a new bucket with a timestamped name, and triggers an SNS topic to send an email alert.

What is Simple Storage Service (S3)?

Amazon Simple Storage Service (Amazon S3) is a scalable, high-speed, web-based cloud storage service designed to store and retrieve any amount of data at any time. Launched by Amazon Web Services (AWS), S3 is widely used for a variety of purposes, including backup and restore, content distribution, disaster recovery, and data archiving.

What is Simple Notification Service(SNS)?

Amazon Simple Notification Service (SNS) is a fully managed messaging service provided by AWS that allows you to send notifications to a large number of recipients or systems. It provides a highly scalable, reliable, and cost-effective solution for sending messages, alerts, and notifications to end-users or other applications. SNS enables the creation and management of topics, to which subscribers can subscribe to receive messages.

What is Lambda ?

AWS Lambda is a fully managed serverless computing service provided by Amazon Web Services (AWS) that lets you run your code in response to events without provisioning or managing servers. With Lambda, you can write code to process events such as file uploads, database changes, or HTTP requests, and AWS automatically manages the infrastructure required to run that code.

Prerequisites

To get started with this project , you need the following:

AWS Account
Gmail Account

Step 1 : Go to S3 and create a S3 Bucket

Select General Purpose as Bucket-type.
Provide a name for the S3 bucket.

Untick the Block all public access, and also enable bucket-versioning .
Enable Bucket-key and Click create bucket . Then a bucket will be created under the name you have provided .

Step 2 : Provide the bucket policy of the S3 bucket

Once the S3 bucket is created , Go to Permissions to provide the S3 bucket policy .

Under Permissions you will find bucket-policy , click on Edit to provide the bucket policy. The policy must be written using JSON.

{
  "Version": "2012-10-17",
  "Statement": [    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::bucket-genie/*"
    }
  ]
}

Then after the policy is given inside the bucket policy , click on save changes to update the changes in the S3 bucket.

Step 3 : Go to Lambda and create a Lambda function

Click on “create a function” , Select an option called “author from Scratch”.
Provide a name for your function.

Select python 3.13 as lambda runtime , and x86_64 as the Architecture.

After selecting the runtime and architecture , click on create function a lambda function will be created.
Once the function is created you can see a image like this with the function name on the top.

Under code section provide the Python code which is the main key for the project .

import boto3
from datetime import datetime
import urllib.parse
s3 = boto3.client('s3', region_name='ap-south-1')
sns = boto3.client('sns', region_name='ap-south-1')
# Fixed bucket name with timestamp - generated only once
BUCKET_NAME = 'bucket-genie-backup-20250509'
REGION = 'ap-south-1'
SNS_TOPIC_ARN = 'arn:aws:sns:ap-south-1:530424100396:S3'
def bucket_exists(bucket_name):
    try:
        s3.head_bucket(Bucket=bucket_name)
        return True
    except s3.exceptions.ClientError:
        return False
def create_bucket(bucket_name):
    s3.create_bucket(
        Bucket=bucket_name,
        CreateBucketConfiguration={'LocationConstraint': REGION}
    )
    print(f" Created bucket: {bucket_name}")
def copy_to_bucket(source_bucket, object_key, destination_bucket):
    copy_source = {'Bucket': source_bucket, 'Key': object_key}
    s3.copy_object(
        Bucket=destination_bucket,
        CopySource=copy_source,
        Key=object_key
    )
    print(f" Copied {object_key} from {source_bucket} to {destination_bucket}")
def send_sns_notification(bucket_name, created_time):
    message = f"""Hello user,
This is to inform you that a new S3 bucket has been created for backup purposes.
🔹 Bucket Name: {bucket_name}
🔹 Region: {REGION}
🔹 Created At (UTC): {created_time}
All future uploads will be backed up to this bucket.
Thank You,
AWS Lambda Backup System
"""
    sns.publish(
        TopicArn=SNS_TOPIC_ARN,
        Subject=' S3 Backup Bucket Created',
        Message=message
    )
    print("📧 Email notification sent via SNS.")
def lambda_handler(event, context):
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    object_key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])
    try:
        is_new = False
        if not bucket_exists(BUCKET_NAME):
            create_bucket(BUCKET_NAME)
            is_new = True
        copy_to_bucket(source_bucket, object_key, BUCKET_NAME)
        if is_new:
            created_time = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')
            send_sns_notification(BUCKET_NAME, created_time)
        return {
            'statusCode': 200,
            'body': f"Backup complete. Object '{object_key}' copied to '{BUCKET_NAME}'."
        }
    except Exception as e:
        print(f" Error: {str(e)}")
        raise e

After adding the code you need to test and deploy the code to make sure the code is running perfectly without any errors .

Click on Add trigger , so that whenever an object is been uploaded into the S3 bucket then it automatically triggers the lambda function.

Select the Source as S3 , under bucket section select the bucket that you have created . Then click on Add to add the trigger into lambda function.

Step 4 : Go to IAM Roles and Attach the policies.

Under lambda function you can see permissions , click on the role name shown below the role name which will redirect to IAM roles and policies .

You can see lambda-genie-role-4ttdny90 is the role name , click on Add Permissions to attach the policies.

Search for AmazonS3FullAccess and once you find the policy , select that policy and click on Add permission.

Step 5 : Go to SNS and create a SNS topic for Notification

click on next step to create a topic , select the topic type as standard .
provide the name and Display name for the topic so that when you get notified , it shows by the display name.

Then click on create topic to create topic . And once the topic is been create you need to create subscription because the SNS itself doesn’t know the destination point, so we give subscription as Email , SMS , HTTP/HTTPS endpoint or SQS .
while creating subscription under protocol, select the type of platform you need to the message to be delivered . In my case i have selected the email , so the message will be delivered to my Gmail.

Click on create subscription after giving all the details . After creating the subscription a mail would be delivered to your Gmail by which you have registered for the subscription . You need to confirm the subscription by click on it .

After the subscription is been confirmed , you’ll need to click on publish message for publishing the desired message to the user.

Provide the subject if needed , then go to message body and type the desired the message that you want to publish . Then click on publish Message. Remember that publish message is used for publishing the message manually.

step 6 : Upload some files into your S3 bucket.

Go to your s3 bucket and upload some files or objects into it .

After uploading of the files and objects , click on upload . After clicking on upload you should see a new s3 bucket created with timestamp.

Let me Breakdown the newly created S3-bucket name.

Now go to Gmail , where you will be notified that a new s3 bucket as been created . And this is the final result .

Conclusion

When a file is uploaded to the bucket-genie, a Lambda function creates a new timestamped bucket, copies the file, and sends an email notification via SNS. It demonstrates automated event handling, object replication, and alerting in a scalable cloud setup. This approach is useful for real-time monitoring, backups, and alert-driven applications.

Automated Deployment of Java Applications to Apache Tomcat using Jenkins.

Nikhil Raj A — Mon, 16 Jun 2025 18:47:48 +0000

Introduction

This project aims to automate the deployment of Java applications to an Apache Tomcat server using Jenkins. Instead of manually building and copying .war files, we use a CI/CD pipeline where Jenkins pulls code from a Git repository, builds it using Maven, and deploys it to Tomcat automatically. This streamlines the development process, reduces human error, and enables faster, more reliable application delivery, following modern DevOps practices.

Why This Project?

Manual deployment of Java applications is time-consuming and error-prone. This project solves that by introducing automation using Jenkins, which ensures that every time code is updated, it is built, tested, and deployed automatically to a Tomcat server. It saves developer time, reduces bugs, and supports faster and more reliable software delivery — all of which are essential in modern DevOps and agile environments.

Prerequsities :

Tomcat Server
Jenkins Server
A java Project

References

Step — 1 : Install Git into your Jenkins server

install git and check the git version .

# Become a root 
sudo su -
# Install git.
yum install git -y
# Check Version of git.
git --version

Open Jenkins in web-browser, click on manage jenkins then select Global Tool Configuration.

Then under Global tool configuration , add git in Git Installations

In git installations , click Add git then select git.

Enter the Git name and path , later click apply and save

Now we have successfully integrated Git with Jenkins . Then click on Global Tool Configuration , under that go to Maven installtions . After giving the details click on apply and save .

Now Maven is also successfully integrated with jenkins .

Steps — 2 : Install a Plugin called ‘Deploy to container ‘

Go to manage jenkins > plugins > Available plugins > Deploy to container .

Click on the restart jenkins after installations , after the installations the jenkins will restart and you need to enter the username and password .

step — 3 : Create a Global Credentials

Go to Credentials > Select system > Select Global Credentials then click on Add Credentials .

After clicking on add credentials , Enter the username and password

Once your done giviing the details , then you can see the credentails you have created .

Step — 4: Create a Jenkins Job

Head to Dashboard > New Item , then provide a name for the jenkins job and select freestyle project as project type .

Enter the github repo and also change the branch name from master to main , because there is no master branch .

Under Build steps > invoke top level maven targets , then provide maven command ‘ clean package’ . And there is no need to mention mvn as we give in the terminal , it automatically takes as mvn and then runs the command .

In Post-build Actions click Add Post-build Actions and Select Deploy war/ear to container.

Enter the details as following then click on apply and save.

Now Click on Build Now

The Build is Successfull .

Output of the project

Why to use Github Webhook ?

Using a GitHub webhook with Jenkins allows you to automatically trigger a build and deploy your Java web application whenever someone pushes code to your GitHub repository.

Step — 5 : Automate Build and Deploy using Github webhook

Go to Github Account > Settings > Webhook , then create a webhook.

Payload URL is public ip if jenkins server.

http://<public ip address>:8080/github-webhook/

Set content type as application/JSON .
Select Event Trigger as just the push event .e

Then go to jenkins dashboard > tomcat-jenkins > configure > Triggers, there you need to enable the Github hook trigger for GITscm polling.

Whenever there is a commit / changes made in the java web — application , then the webhook will automatically create build . Later click on Commit .

After the Changes made when you click on commit , a build will be created in jenkins dashboard automatically where it builds and deploys the web application with (.war ) file.

The changes made in github repository will be automatically changed in the web — application without any manual intervention . Webhook trigger whenever there is a commit in that repository .

Conclusion

This mini project successfully demonstrates a CI/CD pipeline where a Java web application is automatically built and deployed to Apache Tomcat using Jenkins. By integrating GitHub Webhooks, the pipeline is triggered instantly upon every code push, ensuring that the latest changes are continuously tested and deployed without manual intervention.

Automate EC2 Start/Stop with AWS Lambda and CloudWatch — Step-by-Step Guide with Alerts

Nikhil Raj A — Sat, 14 Jun 2025 17:46:37 +0000

Reference

Introduction

In AWS (Amazon Web Services), Elastic Compute Cloud (EC2) instances are widely used for running applications, websites, and services. However, one common issue many users face is forgetting to shut down unused instances — leading to unnecessary billing.

This project solves that problem by automatically shutting down idle EC2 instances if they remain inactive for more than 5 minutes and notifying the user instantly via email or SMS.
By integrating AWS CloudWatch, Lambda, and SNS, we can build an intelligent, self-managing solution that minimizes costs and ensures no resource goes wasted.

Whether you’re a solo developer, startup, or enterprise team, this mini project is a simple but powerful way to automate your cloud operations and improve cost efficiency.

What is Lambda ??

AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. You simply upload your code, set up a trigger (like an alarm or event), and AWS automatically runs the code for you when needed.

What is EC2 in AWS ?

Amazon EC2 (Elastic Compute Cloud) is a web service that provides resizable computing capacity in the cloud. Think of EC2 as a virtual computer where you can run applications, host websites, or perform any tasks you would normally do on a physical server.

What is SNS in AWS ?

Amazon SNS (Simple Notification Service) is a fully managed messaging service that enables you to send notifications to users or other systems. It can send messages via email, SMS, or push notifications.

Prerequisites

AWS Account
Gmail Account

Step — 1 : Create a IAM Role and Policy

Go to AWS Management console .
Create a IAM policy and provide EC2 , SNS and also Logs Permissions . Then click on “create policy”

{
 "Version": "2012-10-17",
 "Statement": [  {
   "Effect": "Allow",
   "Action": [    "logs:CreateLogGroup",
    "logs:CreateLogStream",
    "logs:PutLogEvents"
   ],
   "Resource": "arn:aws:logs:*:*:*"
  },
  {
   "Effect": "Allow",
   "Action": [    "ec2:Start*",
    "ec2:Stop*"
   ],
   "Resource": "*"
  }
 ]
}

Create a IAM Role by clicking on the “create role” and attach that policy to that newly created role. Select Trusted entity type as “AWS Service” and also Select use case as “Lambda”

Select and attach the policy that you have created, then click on “create role” . A role would be created with the desired permissions required .

Step — 2 : Create 2 Lambda Functions for starting and stopping instance

Stopec2 Function

Go to AWS Services → Lambda and click on ‘Create function’.
Choose an Author from scratch
Under Basic information, enter the following information: For Function name, enter a name that identifies it as the function that’s used to stop your EC2 instances. For example, “stopec2”.
For Runtime, choose Python 3.9
Under Permissions, expand Change default execution role
Under the Execution role, choose to Use an existing role
Under Existing role, choose the IAM role that you created
Choose Create function
After creating the Lambda Function, go to → Under Code, Code source, copy and paste the below Python code.

import boto3
from datetime import datetime
# Configuration
region = 'ap-south-1'
instances = ['i-036bb14d2a7a28941']
sns_topic_arn = 'arn:aws:sns:ap-south-1:530424100396:stopec2'  # Replace with your actual SNS topic ARN
# Create clients
ec2 = boto3.client('ec2', region_name=region)
sns = boto3.client('sns', region_name=region)
def lambda_handler(event, context):
    # Stop the instance
    ec2.stop_instances(InstanceIds=instances)
    print('Stopped your instance: ' + str(instances))
    # Prepare and send SNS notification
    timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
    message = f"EC2 instance {instances[0]} was stopped at {timestamp}."
    sns.publish(
        TopicArn=sns_topic_arn,
        Subject="EC2 Instance Stopped",
        Message=message
    )
    return {
        'statusCode': 200,
        'body': message
    }

Replace instance — id with your instance id and provide the region which your instance is been running .

Aftering altering the code , then click on “Deploy” for deploying the code.

Startec2 Function

Repeat the steps from 1–9 , but in this function you need to change the code . Because this code used for starting the EC2 instance .

import boto3
from datetime import datetime
# Configuration
region = 'ap-south-1'
instances = ['i-036bb14d2a7a28941']
sns_topic_arn = 'arn:aws:sns:ap-south-1:530424100396:startec2'  # replace with your actual SNS topic ARN
# Clients
ec2 = boto3.client('ec2', region_name=region)
sns = boto3.client('sns', region_name=region)
def lambda_handler(event, context):
    ec2.start_instances(InstanceIds=instances)
    print('Started your instance: ' + str(instances))

    # Compose notification
    timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
    message = f"✅ EC2 instance {instances[0]} started at {timestamp}."

    # Send notification via SNS
    sns.publish(
        TopicArn=sns_topic_arn,
        Subject='EC2 Instance Started',
        Message=message
    )
    return {
        'statusCode': 200,
        'body': message
    }

Click on “Deploy” for deploying the code . And you may also click on “test” for testing the code manually for checking the code performance .

Step — 3 : Create 2 SNS Topics for start and stop instance notifications

Click on “Next Step” and provide the details like name and type of the SNS . Then click on “create topic”

Create another topic with “standard” as its type and also Provide the name as “stopec2rule” . Click on “create topic”

After creation of the topics , create subscription for both the topics . Where you need to provide the type of the notifications to be received like “Email , SMS or Email Json” etc . Provide your mail — id if you wanna be notified via mail .
Later a Mail will be sent to your EMail for confirming the subscription, so that the notifications will be forwarded to the email which you have confirmed .

Step — 4 : Create 2 Rules for Start and stop of EC2 instance under EventBridge Scheduler

startec2Rule

Go to the Lambda Function(startec2) and click on “add Trigger” .
Select “EventBridge(cloudwatch Events)” as the trigger and provide the details .
create a new rule and provide a name(startec2rule) for the rule .
Select “schedule expression” as rule type.
Provide the schedule expression in the “cron” format . I have given a cron expression where the EC2 instance must start running at 8:25pm of 23 may 2025 . You can also the cron expression based on your comfort.
click on “Add” for adding a trigger .

stopec2Rule

Repeat the steps from 1–4 , in the 5th step you need to add a different cron expression for stopping the ec2 instance .
Provide the schedule expression in the “cron” format . I have given a cron expression where the EC2 instance must start running at 8:40 pm of 23 may 2025 . You can also the cron expression based on your comfort.

Results

Starting of the EC2 instance and getting notified about the starting of the instance.

Stopping of the EC2 instance and getting notified about the Stopping of the instance.

Conclusion

By scheduling start and stop actions through cron expressions, the instance operates only when needed, optimizing cost and efficiency. SNS notifications ensure real-time alerts on each action. The project is simple, cost-effective, and a great example of using AWS services to automate cloud operations.

How to Crack Password-Protected ZIP Files Using John the Ripper on Kali Linux

Nikhil Raj A — Fri, 13 Jun 2025 08:36:54 +0000

Learn how to crack password-protected ZIP files using John the Ripper on Kali Linux in this step-by-step cybersecurity project.

Introduction

John the Ripper is a powerful and widely used open-source password cracking tool designed to test password strength and perform security audits. In this blog, we’ll walk through a practical, hands-on cybersecurity project where we use John the Ripper in Kali Linux to crack a ZIP file password. This exercise is ideal for cybersecurity students and beginners looking to understand password hashing and cracking fundamentals in a controlled, ethical environment.

What is John the Ripper?

John the Ripper (JTR) is an advanced password recovery tool used in penetration testing and digital forensics. It supports various hash types and file formats, including ZIP, RAR, Linux shadow files, and more. It works by attempting dictionary or brute-force attacks on hashed passwords to recover the original plaintext passwords.

Why We Used a ZIP File

We used a ZIP file because it’s a widely supported and beginner-friendly archive format that allows password protection. It integrates smoothly with John the Ripper through the zip2john utility, making it easy to extract password hashes. Compared to other formats like RAR or PDF, ZIP files are quicker to set up and crack, making them ideal for educational and demonstration purposes.

Project Setup

For this project, we created a password-protected ZIP file. We used Kali Linux as our ethical hacking environment and accessed it via Remote Desktop Protocol (RDP).

Steps Overview:

Create a ZIP file with a password.
Start Kali Linux with sudo service xrdp start.
Use ip add to obtain the IP address of Kali.
Connect via RDP and login.
Transfer the ZIP file to Kali Linux Desktop.
Use John the Ripper to extract and crack the password.

Step-by-Step Implementation

Create a Password-Protected ZIP File : Choose an existing file and archive it into a ZIP format.

Secure the file with a password to make sure no one can access the file for sensitive information present inside the file. (e.g., 121314).

2. Start Kali Linux Environment :

Launch Kali Linux and run the command to start the XRDP service. This allows us to start the Kali Linux Service. After entering the command, the system prompts us to enter the system password for authentication .

sudo service xrdp start

3. Finding the IP Address of Kali Linux :

To connect to the Kali Linux machine from another desktop or device — especially when retrieving files like passwords — we’ll need its IP address. This address acts as a unique identifier on the network.

To find it, open the terminal in Kali Linux and run the following command:

ip add

From the output, locate the IP address assigned to your system. In our case, the IP address was 172.26.123.22 . This IP will be used later when establishing a remote desktop session or transferring files to and from Kali Linux.

Connecting to Kali Linux via Remote Desktop :

Now that we have the IP address of our Kali Linux machine, it’s time to connect to it remotely from another device or laptop.

On your Windows system, open the Remote Desktop Connection app (you can simply search for it in the Start menu). Once it launches, you’ll see a field where you need to enter the IP address — in our case, it’s 172.26.123.22. After typing it in, click Connect.

A login screen will appear asking for your Kali Linux credentials. Just enter your username and password, and you’ll be logged into the Kali desktop environment — all from your remote device!

Logging into Kali Linux :

Once the remote connection is established, you’ll be redirected to the Kali Linux login screen. Here, simply enter the username and password you set up earlier during the Kali installation.

After logging in successfully, you’ll have full access to the Kali Linux desktop environment — ready to explore its powerful tools and features, all from your remote device.

6. Transfer the ZIP File :
Once you’re logged into Kali Linux through the remote desktop, the next step is to transfer the file you previously created on your main desktop. This file needs to be copied and pasted into the Kali Linux desktop environment.

7. Preparing the File for John the Ripper :

After successfully logging into Kali Linux, the next step is to transfer the file you created earlier on your main desktop to the Kali Linux desktop. This makes the file easily accessible for the John the Ripper tool, simplifying the cracking process.

Once the file is pasted onto the Kali desktop, open the Terminal to proceed. To navigate to the desktop where the file is located, use the following command:

cd Desktop

This command changes the current working directory to the desktop, allowing you to interact with the file directly from the terminal.

8. Extracting the Hash from the ZIP File :

With the Directory now pointed to the desktop, we can begin using John the Ripper. To extract the hash from the ZIP file, use the following command:

sudo zip2john cybersecurity.zip

In this command, zip2john is the tool that processes the ZIP file, and cybersecurity.zip is the name of the file you want to crack. Make sure you replace the filename if your ZIP file has a different name.

After running the command, the system will prompt you to enter your sudo (admin) password for authentication. Once authenticated, the tool will output the encrypted hash of the ZIP file — this is the data that John the Ripper will attempt to crack.

9. Saving the Hash to a Text File :

The encrypted data output by the zip2john command is in hash format, which John the Ripper can analyze and crack efficiently. To make the process smoother, we need to save this hash into a text file.

You can do this by running the following command in the terminal:

sudo zip2john cybersecurity.zip > hash.txt

This command redirects the hashed output into a file named hash.txt. By doing this, we allow John the Ripper to focus directly on the hash file, making the password-cracking process more streamlined and effective.

10. Cracking the Password Using John the Ripper :

Now that the hash has been successfully saved into a text file (hash.txt), it’s time to use John the Ripper to crack the password.

Run the following command in the terminal:

john hash.txt

This tells John the Ripper to begin analyzing the hash and attempt to recover the original password.

After a few moments, the tool will display the cracked password. In our case, it revealed:

You’ll see the password appear alongside the filename on the terminal screen. And just like that — the password-protected ZIP file has been cracked successfully!

Results

John the Ripper successfully cracked the ZIP file password. The output displayed the plaintext password next to the filename, verifying the tool’s capability to efficiently perform dictionary-based cracking.

Conclusion

This project demonstrated how ethical hackers and cybersecurity students can use John the Ripper to test the strength of password-protected files. It reinforces the importance of using strong, complex passwords and the need for continuous security awareness.

Learn how to design and deploy a 3‑tier web architecture on AWS — step‑by‑step guide.

Nikhil Raj A — Thu, 12 Jun 2025 14:27:37 +0000

Introduction

A 3-tier architecture divides a web application into three layers: Web (presentation), Application (logic), and Database (data). This design improves scalability, security, and maintainability.

Using AWS services like EC2, Elastic Load Balancer, RDS, and VPC, we can build a cloud-based 3-tier architecture where each layer is logically separated and independently scalable, ensuring better performance and reliability for modern web applications.

What is a VPC?

VPC stands for Virtual Private Cloud.
It is a logically isolated section of the AWS Cloud where you can launch AWS resources (like EC2, RDS, etc.) in a virtual network that you define.

Why we need a VPC???

We need a VPC (Virtual Private Cloud) to create a secure, isolated network environment within AWS where we can launch and manage resources like EC2, RDS, and load balancers. It gives us full control over IP addressing, subnets, routing, and access rules, allowing us to build architectures with public and private zones, enforce security, and connect securely to the internet or on-premises networks.

The architecture consists of:

Web Application Tier (Presentation Layer): This tier handles the user interface and user experience, providing a seamless interaction platform for customers. Thinking of an online clothing store, this is where customers browse through clothing items, view details of each product, add items to their cart, and proceed to checkout.
Application Tier (Business Logic): This tier processes the core functionalities of the e-commerce platform, including order processing, user authentication, and product catalog management. In the context of an online clothing store, this tier manages the logic for adding items to the cart, processing payments, managing user accounts, and updating inventory levels.

3. Data Tier (Database): This tier securely stores all the critical data, such as customer information, transaction records, and product details. For an online clothing store, this includes storing details of each clothing item, customer profiles, order histories, and payment information.

Prerequisites

AWS Account
EC2
VPC
Subnets
Internet Gateway and NAT Gateway.
Elastic Load Balancer (ELB)
RDS (MySQL/PostgreSQL)
Security Groups and Network ACLs

Check out Medium Profile : https://medium.com/@nikhilsiri2003

Clone the github Repo for accessing the application code into your local

https://github.com/NikhilRaj-2003/3-tier-architecture-using-aws.git

Step 1 : Create a IAM role for EC2 instance

Under IAM Dashboard > Accesss Management > Roles , under that click on create role .
Select AWS Service as the trusted entity type , and use case as EC2.
Under permission policies , search the policy like AmazonSSMManagedInstance and AmazonS3readonly.Then Click on Next.
Provide a name for the Role , then Click on create role.

Step 2 : Building the Architecture

Create a VPC with 2 Public subnets for the Web-Tier and 4 Private subnets , under those 4 private subnets 2 are for the Application-Tier and rest of the 2 Private subnets are for the Database .
Click on Your VPC > Create VPC .

Select VPC only if your just creating a VPC . You can also choose VPC and more if you want to create vpc , subnets , route tables , internet-gateway and NAT-gateway in just one-shot .
Provide a name for the VPC , and also the IPv4 CIDR (10.0.0.0/16) . Then click on create VPC.

After Creation of VPC , create public and private subnet

Under subnet section , click on create subnet to create the subnets .
Select the VPC you have created first , then start to build the subnets .
Create 2 public subnet with 2 availability zones and also provide the IPv4 subnet CIDR block.

Like this shown above you need to create 5 more subnets , then the layers will be shown in the resource map of the VPC for better understanding

Now create Internet gateway after subnet creation .

Go to VPC > Internet gateways would be present on the left side . Then click on create internet gateway
Provide a name for the internet-gateway , then click on create internet gateway .
Later select that internet gateway > Actions > attach to VPC , now your internet gateway has been attached to the VPC .

Then create NAT gateway after Internet gateway

Under VPC > Nat gateways > click on create on NAT gateway .

Provide a name for the NAT-gateway , Select Either of the 2 public subnet and allocate elastic IP . Remember that Elastic IP address is allocated only to public subnet . Then click on create NAT gateway .

After Nat-gateway , then create Route tables

Add a route that directs traffic from the VPC to the internet gateway. In other words, for all traffic destined for IPs outside the VPC CDIR range, we add an entry that directs it to the internet gateway as a target.
Under VPC dashboard > Route tables > Provide a name for the route table > VPC: select your VPC > Edit routes > Add route > Target: Internet Gateway > Save changes.
In Subnet associations of the route table > Edit explicit subnet associations > Select the two Public Subnets > Save associations .We will create 2 more route tables, one for each of the App layer private subnet in each availability zone. These route tables will route app layer traffic destined for outside the VPC to the NAT gateway in the respective availability zone. Let’s add the appropriate routes for this scenario.
Under VPC > Route tables > Create route table > Name the table: Private_Route_AZ1 > Select your VPC > Create > Routes > Edit routes > Destination: Everywhere > Target: Nat Gateway > NAT gtw for AZ 1 ( repeat the steps for NAT gtw AZ 2)
Edit routes for each of the 2 Route tables: Edit subnet association > Select the Private Subnet for AZ 1 (repeat for AZ 2)

Step 3 : Create 5 Security groups for app , web and database tier

**Security groups : **it is used to tighten the rules around which traffic will be allowed to our Elastic Load Balancers and EC2 instances. This being said, will need 5 security groups: one for the External LB, one for the Internal LB, and one for each of the 3 tiers: Web, App and DB tier.

Web-tier security group

Go to security group under EC2 . Then click on create security group .
Provide a name for the security group , then select the VPC that you have created .
Then provide the in-bound rules for allowing the incoming traffic with desired port number .

Web-ALB security group

Provide a security name for the security group .
Then give a description for the security group (optional).
Select the VPC which you have created .
Later assign the bound rules which proper port-number like HTTP and HTPPS. Then click on create security group.

Application-tier security group

Firstly remember that the application is gonna run on port number-4000 , so we need to allow incoming traffic with port number 4000 using custom TCP .
Name the security group for the application-tier . Then description is given (optional).
Later select the VPC which is created by you .
Assign the in-bound rules with custom TCP , port range 4000 . CLick on create security group.

Application-Internal-alb security group

This is a internal application load balancer , and it also runs o port number 80 .
Provide a name for the security group with description as optional .
Then select the VPC which is been created .
Assgin the in-bound the rules with HTTP and open for port 80 .

Database-tier security group

Before creation of the security group remember that the mysql runs on port number 3306 . So we need allow port 3306 in the in-bound rules .
Create a security group for database -tier . Provide a name for the security group .
Then give the description for the security group(optional).
Select the VPC which was created in the beginning .
Under in-bound rules select type as MYSQL/Aurora and then Port number as 3306 . Later click on create security group.

Step 4 : Create a RDS database and DB subnet group

Creating DB subnet group

Provide a name for the DB subnet group , so that we can identify them easily
Choose the VPC that was created in the beginning .
Choose the availability zones , select 2 zones ( ap-south-1a & ap-south-1b)
Then under add subnets , select the private DB subnets created earlier .
Click on Create

Creating DB instances

Select standard create for database creation method .
Go for MYSQL for the engine type under engine options .
Then select MYSQL 8.0.35 as the Engine Version

Under templates select the Free-tier template , so that you wont be charged .
Provide the Username or use default username . Click on self-managed for the credentials management . Later provide a custom password and confirm the password .
Select DB instances class as d4.t4g.micro , gp2(General Purpose SSD*)* for the storage and the allocated storage would be 20 .

Then select the VPC which was created , later select the DB subnet group . Click on No for public access .
Choose the existing security group as DB-sg which we had created while creating the security groups . Also select either of the availability zones (ap-south-1a or ap-south-1b). Then click on create Database .

Step 5 : Launch an EC2 instance (Application-tier)

Provide a name { app-ec2} for the EC2 instance .
Select Amazon Linux 2 as the AMI(application machine image)
Instance type as t2.micro . Select the key-pair .
Under Network settings > VPC select the newly created VPC . We are creating application tier instance so assign it with private-app-subnet (1 or 2 ).
Provide the existing security group which we had created , that is app-sg which runs on port number 4000. Select the IAM role which was created earlier . Then click on Launch instance.
After launching the instance connect the instance using session manager . You cant connect through SSH because its running on private subnet and there will be no public-IP.

Configuration of Application-tier EC2 instance

app-tier setup : https://github.com/NikhilRaj-2003/3-tier-architecture-using-aws/blob/main/Implementation/Application-tier.md

Create a Load Balancer with Target groups for Application-tier

Target Group

Choose target type as instances
Provide a name for the target group , and your app-ec2 is running on port 4000 so you need to open the port 4000 in the target group.
Then Click on next . Select the instances which you need to add into the target group and also click on include as pending below . click on Create target group

Load Balancer

Choose Application Load balancer as the load balancer types
Provide a name for the load balancer , and select scheme as internal .
Select the VPC which we created . And the select the private-app-subnets with 2 availiability zones . Choose the app-int-lb security which we created .
Under listeners and routing > Listener select the target group created for forwarding the traffic .
Then click on create load balancer.

Step 6 : Launch an EC2 instance ( Web-tier)

Provide a name {Web-ec2} for the EC2 instance .
Select Amazon Linux 2 as the AMI(application machine image)
Instance type as t2.micro . Select the key-pair .
Under Network settings > VPC select the newly created VPC . We are creating application tier instance so assign it with public-subnet (1 or 2)
Also Enable Auto-assign public-IP for assigning the IP-address .
Provide the existing security group which we had created , that is web-sg . Select the IAM role which was created earlier . Then click on Launch instance.

Configuration of Web-Tier EC2 instance

web-tier setup : https://github.com/NikhilRaj-2003/3-tier-architecture-using-aws/blob/main/Implementation/Web-tier.md

Output

Now you can provide the data into it and all the data will be stored into the database that we had created .

Create Load Balancer with Target group for Web-tier

Target Group

Choose target type as instances
Provide a name for the target group , and your web-ec2 is running on port 80 so you need to open the port 80 in the target group.
Then Click on next . Select the instances which you need to add into the target group and also click on include as pending below . click on Create target group

Load Balancer

Choose Application Load balancer as the load balancer types
Provide a name for the load balancer , and select scheme as Internet-facing.
Select the VPC which we created . And the select the public-web-subnets with 2 availiability zones . Choose web-alb-sg security group which we created .
Under listeners and routing > Listener select the target group created for forwarding the traffic .
Then click on create load balancer. After creation of the load balancer you can access the website using load balancer DNS name .

Note :

If you dont want to use the Load Balancer DNS name then create a custom DNS name with A Record . Later you can access the website using your custom domain.
And the 2 EC2 instance which you have created , take their application machine image and create their image and launch template by providing the name , select the AMI which you created , with created VPC , subnets and exisitng security group . click on launch template .
Then create autoscaling group for both application-tier and Web-tier. Select the launch template created for app-tier and web-tier , Assign the VPC that is created , and their desired subnets , with load balancer, Then click on Create autoscaling group .Which will create multiple ec2 instances for both app-tier and web tier . It will autoatically be attached to load balancer in the target groups.

Conclusion

A 3-tier architecture separates an application into three logical layers: web (presentation), app (business logic), and database (data storage). This structure enhances scalability, security, and maintenance by isolating each layer. It allows each tier to be managed, updated, and scaled independently. In cloud environments like AWS, it’s a best-practice model for building robust and modular applications.

AWS 3-Tier-Architecture EC2 VPC Designing