Forem: Vikas Arora

Say Goodbye to Orphaned Snapshots: Automate Cleanup with Serverless, Terraform, and AWS EventBridge!

Vikas Arora — Tue, 29 Oct 2024 13:49:34 +0000

Over time, AWS accounts can accumulate resources that are no longer necessary but continue to incur costs. One common example is orphaned EBS snapshots left behind after volumes are deleted. Managing these snapshots manually can be tedious and costly.

This guide shows how to automate the cleanup of orphaned EBS snapshots using Python (Boto3) in an AWS Lambda function, which is then triggered using AWS EventBridge on a schedule or event.

By the end, you’ll have a complete serverless solution to keep your AWS environment clean and cost-effective.

Prerequisites

Installing AWS CLI and Terraform

First, let’s ensure the essential tools are installed.

AWS CLI
The AWS CLI allows command-line access to AWS services. Install it according to your operating system:

macOS: brew install awscli
Windows: AWS CLI Installer
Linux: Use the package manager (e.g., sudo apt install awscli for Ubuntu).
Verify installation:

aws --version

Terraform
Terraform is a popular Infrastructure as Code (IaC) tool for defining and managing AWS resources.

macOS: brew install terraform
Windows: Terraform Installer
Linux: Download the binary and move it to /usr/local/bin.

Verify installation:

terraform -version

Configuring AWS Access

Configure your AWS CLI with access keys to allow Terraform and Lambda to authenticate with AWS services.

Get Access Keys from your AWS account (AWS IAM Console).
Configure AWS CLI:

aws configure

Follow the prompts to enter your Access Key, Secret Access Key, default region (e.g., us-east-1), and output format (e.g., json).

Next, since we are going to build the entire stack with Terraform, please fork the repository located here, which contains the full code for the project.

Clone it to your local machine and open it in a code editor.

I have used Visual Studio Code, and it appears as follows:

Delete the following two files from the project, as these will be recreated when you run the terraform from your code editor:

orphan-snapshot-delete.zip
.terraform.lock.hcl

Next, lets configure the S3 backend:

Create an S3 Bucket for Terraform State

1. Go to the S3 Console:

2. Create a New Bucket:

Click Create bucket.
Give the bucket a unique name, such as my-terraform-state-bucket.
Choose an AWS Region that matches your infrastructure region for latency reasons.

3. Configure Bucket Settings:

Keep Block Public Access settings enabled to restrict access to the bucket.
Versioning: Enable versioning to maintain a history of changes to the state file. This is useful for disaster recovery or rollbacks.
Leave other settings as default.

4. Create the Bucket:

Click Create bucket to finalize the setup.

Create a DynamoDB Table for State Locking (Optional but Recommended)

Using a DynamoDB table for state locking ensures that only one Terraform process can modify the state at a time, preventing conflicts.

1. Go to the DynamoDB Console:

In your AWS Console, go to DynamoDB.

2. Create a New Table:

Click Create table.
Name your table, e.g., terraform-state-locking.
Partition Key: Set the partition key to LockID and use the String data type.

3. Configure Settings:

Leave default settings (such as read and write capacity) unless you have specific requirements.
Create the table by clicking Create table.

Configure IAM Permissions for Terraform

Terraform needs specific permissions to interact with S3 and DynamoDB (if using locking).

This step is necessary only if you are operating under the least privileged access. If you already have administrator access, you can skip this step.

1. Create or Use an IAM User:

If you don’t have an IAM user for Terraform (You can use your own IAM user and attach these policies to it), create one in the IAM Console.
Attach policies that grant permissions to access S3 and DynamoDB.

2. Attach S3 and DynamoDB Policies:

Use an inline policy or add the following permissions:

Access to the S3 bucket.
Access to the DynamoDB table (if using locking).

Example IAM Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::my-terraform-state-bucket/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:GetItem",
                "dynamodb:DeleteItem",
                "dynamodb:DescribeTable"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/terraform-state-locking"
        }
    ]
}

After completing all the prerequisites, let's examine the Python and Terraform code that will perform the actual magic.

Step 1: Python Code for Orphaned Snapshot Cleanup

In the code editor, open the orphan-snapshot-delete.py file.

The complete function code is as follows:

import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    ec2_cli = boto3.client("ec2")
    response = ec2_cli.describe_snapshots(OwnerIds=["self"], DryRun=False)
    snapshot_id = []
    for each_snapshot in response["Snapshots"]:
        try:
            volume_stat = ec2_cli.describe_volume_status(
                VolumeIds=[each_snapshot["VolumeId"]], DryRun=False
            )
        except ec2_cli.exceptions.ClientError as e:
            if e.response["Error"]["Code"] == "InvalidVolume.NotFound":
                snapshot_id.append(each_snapshot["SnapshotId"])
            else:
                raise e

    if snapshot_id:
        for each_snap in snapshot_id:
            try:
                ec2_cli.delete_snapshot(SnapshotId=each_snap)
                logger.info(f"Deleted SnapshotId {each_snap}")
            except ec2_cli.exceptions.ClientError as e:
                return {
                    "statusCode": 500,
                    "body": f"Error deleting snapshot {each_snap}: {e}",
                }

    return {"statusCode": 200}

This Lambda function uses Boto3, AWS’s Python SDK, to list all EBS snapshots, check their associated volume status, and delete snapshots where the volume is no longer available. Here’s the complete function code:

Step 2: Terraform Configuration for Serverless Infrastructure

Using Terraform, we’ll create a Lambda function, IAM role, and policy to deploy this script to AWS. Additionally, we’ll set up an EventBridge rule to trigger Lambda on a regular schedule.

Terraform Setup and Provider Configuration
This section configures Terraform, including setting up remote state management in S3.

Open the terraform file name main.tf in code editor and start reviewing the code as shown in the following sections.

Terraform Setup and Provider Configuration

This section configures Terraform, including setting up remote state management in S3.

Note:

Change the required_version value as per the terraform -version output.
Update the bucket, key, and dynamodb_table values for the S3 backend to match what you have created in the previous steps.

terraform {
  required_version = ">=1.5.6"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.72.0"
    }
  }
  backend "s3" {
    bucket         = "terraform-state-files-0110"
    key            = "delete-orphan-snapshots/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "tf_state_file_locking"
  }
}

provider "aws" {
  region = var.aws_region
}

IAM Role and Policy for Lambda
This IAM configuration sets up permissions for Lambda to access EC2 and CloudWatch, enabling snapshot deletion and logging.

resource "aws_iam_role" "lambda_role" {
  name               = "terraform_orphan_snapshots_delete_role"
  assume_role_policy = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": "sts:AssumeRole",
          "Principal": { "Service": "lambda.amazonaws.com" },
          "Effect": "Allow"
        }
      ]
    }
EOF
}

resource "aws_iam_policy" "iam_policy_for_lambda" {
  name   = "terraform_orphan_snapshots_delete_policy"
  policy = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
              "logs:CreateLogGroup",
              "logs:CreateLogStream",
              "logs:PutLogEvents"
          ],
          "Resource": "arn:aws:logs:*:*:*"
        },
        {
          "Effect": "Allow",
          "Action": [
              "ec2:DescribeVolumeStatus",
              "ec2:DescribeSnapshots",
              "ec2:DeleteSnapshot"
          ],
          "Resource": "*"
        }
      ]
    }
EOF
}

resource "aws_iam_role_policy_attachment" "attach_iam_policy_to_iam_role" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = aws_iam_policy.iam_policy_for_lambda.arn
}

Packaging and Deploying the Lambda Function
Here, we package the Python code and deploy it as a Lambda function.

data "archive_file" "lambda_zip" {
  type        = "zip"
  source_file = "${path.module}/python/orphan-snapshots-delete.py"
  output_path = "${path.module}/python/orphan-snapshots-delete.zip"
}

resource "aws_lambda_function" "lambda_function" {
  filename      = data.archive_file.lambda_zip.output_path
  function_name = "orphan-snapshots-delete"
  role          = aws_iam_role.lambda_role.arn
  handler       = "orphan-snapshots-delete.lambda_handler"
  runtime       = "python3.12"
  timeout       = 30
}

EventBridge Rule for Lambda Invocation
AWS EventBridge allows you to create scheduled or event-based triggers for Lambda functions. Here, we’ll configure EventBridge to invoke our Lambda function on a schedule, like every 24 hours.

You can learn more about EventBridge and scheduled events in AWS documentation here.

resource "aws_cloudwatch_event_rule" "schedule_rule" {
  name        = "orphan-snapshots-schedule-rule"
  description = "Trigger Lambda every day to delete orphaned snapshots"
  schedule_expression = "rate(24 hours)"
}

resource "aws_cloudwatch_event_target" "target" {
  rule      = aws_cloudwatch_event_rule.schedule_rule.name
  arn       = aws_lambda_function.lambda_function.arn
}

resource "aws_lambda_permission" "allow_eventbridge" {
  statement_id  = "AllowExecutionFromEventBridge"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_function.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.schedule_rule.arn
}

Step 3: Applying the Terraform Configuration

After defining the infrastructure, initialize and apply the Terraform configuration:

terraform init
terraform apply

Step 4: Testing and Monitoring the Lambda Function

To verify that the solution works:

Manually Trigger the Event (optional): For initial testing, trigger the Lambda function manually from the AWS Lambda console.
Monitor CloudWatch Logs: The Lambda function writes logs to CloudWatch, where you can review entries to verify snapshot deletions.
Adjust the Schedule as Needed: Modify the schedule_expression to set a custom frequency for snapshot cleanup.

Enhancements

The following enhancements could be implemented in this project:

Instead of scheduling an Eventbridge rule, the deletion of EBS volumes could be detected by Eventbridge, which would then trigger the Lambda function to delete the corresponding snapshot.
Paging could be incorporated into the Python function to manage situations where the number of snapshots is substantial.

Wrapping Up
By combining Python (Boto3), Lambda, AWS EventBridge, and Terraform, we’ve created a fully automated, serverless solution to clean up orphaned EBS snapshots. This setup not only reduces cloud costs but also promotes a tidy, efficient AWS environment. With scheduled invocations, you can rest assured that orphaned resources are consistently removed.

Try this solution in your own AWS account and experience the benefits of automation in cloud resource management!

Please feel free to share your thoughts on this article in the comments section. Thank you for reading.

Deploy Docker Example Voting App on AWS EKS using Github actions and expose it over the internet using Nginx Ingress Controller

Vikas Arora — Mon, 14 Aug 2023 11:33:01 +0000

Working on an end-to-end project was my way of reinforcing the knowledge I gained from the Kubernetes hands-on course that I took last month. At the same time, I wanted to share what I learned.

I wanted to use a microservices-based application for this project to unleash the full potential of Kubernetes. Luckily, I stumbled upon Docker’s example-voting-app when I was going over docker-compose concepts earlier.

The original Docker’s Example Voting App repository helps you to deploy and run the application on your local machine using docker-compose or kubernetes.

I have modified some manifest files and created Github actions (CI/CD) workflows to deploy it on AWS EKS cluster and expose it over the internet.

Application architecture

The architecture shows each component of the application and a brief description of what they do

voting-app: Front-end of the application, written in python flask framework, which allows the users to cast their votes.
redis: An in-memory data structure store, used as a temporary database to store the votes casted by the users.
worker: Service written in .NET, that retrieves the votes data from redis and stores it into PostgreSQL DB service.
PostgreSQL DB: PostgreSQL DB used as persistent storage database.
result-app: Service written in node js, displays the voting results to the user.

Pre-requisites

An IAM user with Administrator access and Access keys to be used with GitHub Actions and Terraform. You can follow the link in the step-by-step approach below.
AWS CLI V2, steps here.
Dynamodb table in AWS to store the terraform state locking, steps here. Make Partition Key as ‘LockID’.
AWS ECR (Amazon Elastic Container) Registry with name voting-app (case sensitive) using steps defined here.
S3 bucket to store Terraform state files. Steps given here.
Terraform V1.0 or higher, install to your local terminal. Steps provided here.
Kubectl, install to your local terminal using the steps given here.
A valid domain name (I used cloudempowered.co for this demo) that you can get from any domain registrar (for example, godaddy, namecheap, etc.) or from Amazon route 53.
Project has been executed in the AWS North Virginia (us-east-1) region. If you want to use a different region, you will need to find and change the region value in the project repositories.

Technical Architectural overview

Build and deployment of the example-voting-app on Kubernetes is done using the following DevOps tools/AWS Services:

Terraform: It deploys the AWS EKS cluster and the necessary AWS resources.
GitHub: It is the code repository for the application.
GitHub actions: It is the CI/CD tool that build and deploy the application on the EKS cluster.
Nginx Ingress Controller: It creates a network load balancer in AWS to enable external access to the application.
AWS EKS: It is the Kubernetes service in AWS.
Amazon VPC: It hosts the AWS EKS Cluster and all the required networking components for the project.
AWS ECR: It is a private repository (like docker hub private registry) that hosts the application docker images.
AWS Route 53: DNS service provided by AWS.
AWS Certification Manager: Certification authority that provides the SSL certificate to the validated domain name.

This is an intermediate level of implementation, and i assume some familiarity with AWS, Kubernetes, Github and CI/CD. I have provided links for more details where possible.

You might incur high cost if you keep the infrastructure components deployed on AWS for longer than needed. Destroy them through Terraform when your work is over.

Step-by-step approach to implement:

First fork these two GitHub repositories:

devopsulting/k8s-cluster-creation (github.com): This
repository contains Terraform code for creating the AWS
infrastructure needed to deploy the example-voting-app into
an AWS EKS cluster.

devopsulting/k8s-example-voting-app (github.com):
This repository contains the application source code and the
Kubernetes manifests for the example-voting-app.

To begin the setup, clone these two GitHub repositories to your local machine.
Setup your EKS infrastructure using the terraform repo k8s-cluster-creation and process defined by Ankit Jodhani here. I used his Terraform repository with necessary modifications. This article also covers how to create an IAM user, set up AWS CLI, and installing Terraform itself. Please do not clone any of the repositories mentioned in this article.
I have added some extra Terraform code to create Route 53 and SSL Certificate resources. These resources will allow to access the vote and result user interfaces over the internet using your own domain name. To do this, you need to change the value of the HOSTED_ZONE variable in the infra-creation/terraform.tfvars file to match your domain name.
Create your AWS infrastructure, including EKS cluster by running the Terraform.
As soon as this infrastructure is created, copy the NS (Nameserver) values from your domain’s hosted zone in Route 53, and update them in the Nameserver settings for your domain name with respective domain registrar/provider (goadaddy, namecheap etc.)

Route 53

Update NS settings for the domain name with your domain registrar/provider

Pictures above shows the NS values that I used for my domain name. You should not use the same values for your domain name. Route 53 will generate specific NS entries for your hosted zone.

Also, above hosted zone setup assumes that you registered your domain name with a different domain registrar than Route 53. If you registered your domain name with Route 53, then you don’t need to do this setup. Route 53 will automatically create a hosted zone for your domain name and update the name servers for you. In that case, you can remove the Terraform code for Route 53 setup.

Significance of this step is that your domain name will be validated by the AWS Certificate Manager and a valid SSL certificate will be issued for your domain name. The verification process may take 15–30 minutes (for domain names that are not registered with Route 53). During this time, terraform setup will wait for the verification to finish. Therefore, you should do this step as soon as your hosted zone is created in Route 53.

Before moving ahead with application deployment using k8s-example-voting-app repository, we need to understand the brief about Github actions, it’s setup and pivotal role it played in creating the application docker images, storing them in the AWS ECR and finally pushing them to the EKS cluster using manifests.

Open the k8s-example-voting-app repository, and follow along.

Github Actions (GA)

We are using GA as a CI/CD tool to perform build, test, and deployment actions. This is how we deploy the application on the EKS cluster. You can learn more about GA in this article here.

GA has some advantages over other CI/CD tools. It is integrated with Github, so you don’t need to install or manage a separate server or plugins. It also has a feature that allows you to connect to AWS using OIDC (OpenID Connect, a Web-identity based role). This means you don’t have to add AWS credentials to Github, which makes it more secure. However, i did not use this feature in this project, because it adds some complexity. But I plan to implement it later as an improvement to this project.
GA workflows are created within a Github repository, by:

Creating a .github/workflows directory.
In the .github/workflows directory, create the required yaml files, which define the necessary action.
This project uses two yaml files, called build.yaml and deploy.yaml, for the CI/CD process. You will see these files in the .github/workflows folder when you fork the k8s-example-voting-app repository.

Add AWS Access Key and Secret Access Key in the Github repository settings (your forked k8s-example-voting-app repository), for the Github secrets, which will be used by GA to work with AWS.

build.yaml

The build.yaml workflow is responsible for building the docker images from the three application modules: vote, worker, and result. It then stores them in the ECR private registry (Dockerhub registry can also be used alternatively, though ECR is seamless being an AWS service) . These docker images will then be deployed to the Kubernetes (EKS) cluster from the ECR registry.

The postgresql and redis images will be downloaded directly from the Docker registry and used in Kubernetes.

Here is the code for build.yaml

name: Build Application Images and store in ECR
on:
  push:
    branches:
      - "main"
    paths:
      - .github/workflows/build.yaml
      - "result/**"
      - "vote/**"
      - "worker/**"
env:
  AWS_REGION : "us-east-1"
  ENV: "prod"
permissions:
  id-token: write
  contents: read
jobs:
  build:
    name: create application images from source code and store them in the ECR
    runs-on: ubuntu-latest
    steps:
      - name: Update runner's docker Version, as worker module requires it
        run: |
          docker --version
          sudo apt update
          sudo apt install apt-transport-https ca-certificates curl software-properties-common -y
          curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
          echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
          sudo apt update
          apt-cache policy docker-ce
          sudo apt install docker-ce -y
          docker --version

      - name: Checkout code from GitHub to runner
        uses: actions/checkout@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_AP_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_AP_SECRET_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Build, tag, and push the vote docker image to Amazon ECR
        id: build-vote-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: voting-app
          IMAGE_TAG: vote-${{ env.ENV }}-latest
        # Build docker images for vote module and push it to ECR so that it can be deployed to EKS
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG --build-arg aws_region=${{ env.AWS_REGION }} --build-arg copy_or_mount="copy" -f vote/Dockerfile ./vote
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"


      - name: Build, tag, and push the result docker image to Amazon ECR
        id: build-result-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: voting-app
          IMAGE_TAG: result-${{ env.ENV }}-latest
        # Build docker images for result module and push it to ECR so that it can be deployed to EKS.
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG --build-arg aws_region=${{ env.AWS_REGION }} --build-arg in_aws="yes" --build-arg copy_or_mount="copy" -f result/Dockerfile ./result
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"


      - name: Build, tag, and push the worker docker image to Amazon ECR
        id: build-worker-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: voting-app
          IMAGE_TAG: worker-${{ env.ENV }}-latest
        # Build docker images for worker module and push it to ECR so that it can be deployed to EKS
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG --build-arg aws_region=${{ env.AWS_REGION }} --build-arg copy_or_mount="copy" -f worker/Dockerfile ./worker
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

The build.yaml workflow will run when you make a pull request from your working branch to the main branch. This means that you want to merge the changes you made in your working branch to the main branch. The workflow will build the docker images from your updated code and store them in the ECR registry.

deploy.yaml

The deploy.yaml workflow uses the manifests in the k8s-specifications folder of the k8s-example-voting-app repository to deploy the application on the EKS cluster. This will create the necessary components for the application, such as:

Nginx Ingress Controller: This will set up a network load balancer in AWS that will allow us to access the voting and result user interfaces from the internet.
Application Ingress: This will create a mapping between the network load balancer URL and the domain names of the voting and result user interfaces. This way, we can use our own domain names to access the application.
Postgresql db service and deployment
Redis service and deployment
Vote service and deployment
Result service and deployment
Worker deployment

Here is the code for the deploy.yaml

name: Deploy images from ECR to k8s
on:
  workflow_dispatch
env:
  AWS_REGION: "us-east-1"
  ENV: "prod"
permissions:
  id-token: write
  contents: read
jobs:
  deployment:
    name: Deploy application to EKS cluster
    runs-on: ubuntu-latest
    steps: 
    - name: Checkout code from GitHub to runner
      uses: actions/checkout@v2
      with:
        token: ${{ secrets.GITHUB_TOKEN }}

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        aws-access-key-id: ${{ secrets.AWS_AP_ACCESS_KEY }}
        aws-secret-access-key: ${{ secrets.AWS_AP_SECRET_KEY }}
        aws-region: ${{ env.AWS_REGION }}

    - name: Install kubectl
      uses: azure/setup-kubectl@v3
        # with:
        #   version: '1.27' # default is latest stable
    - name: Update kube config
      run: |
        aws eks update-kubeconfig --name k8s-voting-app --region ${{ env.AWS_REGION }}  

    - name: Deploy application images to EKS cluster using manifest
      run: |
        kubectl version --short
        kubectl apply -f k8s-specifications/db-secret.yaml
        kubectl apply -f k8s-specifications/db-deployment.yaml
        kubectl apply -f k8s-specifications/db-service.yaml
        kubectl apply -f k8s-specifications/redis-deployment.yaml
        kubectl apply -f k8s-specifications/redis-service.yaml
        kubectl apply -f k8s-specifications/worker-deployment.yaml
        kubectl apply -f k8s-specifications/ingress-controller.yaml
        sleep 15
        kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
        kubectl apply -f k8s-specifications/vote-deployment.yaml
        kubectl apply -f k8s-specifications/vote-service.yaml        
        kubectl apply -f k8s-specifications/result-deployment.yaml 
        kubectl apply -f k8s-specifications/result-service.yaml 
        kubectl apply -f k8s-specifications/app-ingress.yaml

Very important point to note about deploy.yaml is that it’s triggered manually, from the Actions tab.

Lets continue with the remaining steps to build and deploy the application to the EKS.

Create a local working branch for the k8s-example-voting-app cloned repository and switch to it. You can name the branch whatever you like. I used the name feature for my branch.
Open the k8s-specifications/ ingress-controller.yaml file and find the line 348 where it says service.beta.kubernetes.io/aws-load-balancer-ssl-cert:. Replace it’s value with the SSL certificate ARN that you got from the AWS Certificate Manager service for your domain name. This certificate was created by the Terraform code that you ran earlier.
Open k8s-specifications/app-ingress.yaml, and update both the host entries on line 10 and 20 respectively, with your domain name (vote.yourdomainname/result.yourdomainname)
You need to change the image field in the vote-deployment.yaml, result-deployment.yaml and worker-deployment.yaml files to match your AWS account number instead of 123456789012.

Now publish this branch to the remote, followed by the pull request to the main branch.

Note: While creating the pull request, choose your own github main branch as the base and not the source fork repository(devopsulting/k8s-example-voting-app) for the pull request.

Here’s how these actions takes place:

Once Pull request is merged, GitHub actions is triggered and can be observed from the Actions tab

Note: If you have no real code changes to make, you can still trigger the build workflow by editing a comment in any of the files in the vote, result or worker modules, or in the build.yaml file. Then, push the changes to your remote working branch and create a pull request to the main branch.

This action will build the docker images from the application modules, namely vote, worker and result respectively and store these images in the ECR private registry named voting-app.

Next step is to deploy these images, along with ingress functionality on the EKS cluster, using manifests.
Make sure that the build workflow has finished successfully before you run the deploy workflow. This will ensure that the application images are stored in the ECR and ready to be deployed.
On the Actions tab, in the left menu, you will find link Deploy images from ECR to k8s. From there you need to expand Run workflow dropdown and then click on the Run workflow button to run it, as shown below.

After the deploy workflow is done, the application will be set up on the EKS cluster. Only thing left to do is to check the ingress mapping to the network load balancer and create two A records in the route 53 hosting zone. This will allow you to access the vote and result interfaces using your domain name.
Run the following commands on your local terminal (where you configured your AWS-CLI and kubectl), approx. after 5 mins of completion of deploy workflow.

aws eks update-kubeconfig --name k8s-voting-app --region us-east-1
kubectl describe ingress basic-routing

It should look something like this:

Note that Address field is updated with the Network Load Balancer url.

After you have verified that the network load balancer is correctly mapped to the ingress host URLs, you need to create two A records in the Route 53 hosted zone as shown below:

For each A record, you need to select the network load balancer as the alias target. It should be done like below:

Now, you have to wait for about 15 minutes for the DNS settings to propagate. After that, you can try to access the vote and result user interfaces using your domain name. These should look like below:

Test your application by voting from different browsers and check how CATS and DOGS percentage changes in the result screen.
Once you have tested the application, it’s time to delete the entire infrastructure to avoid incurring unnecessary cost.
Start by deleting the three entries in the Route 53 hosted zone. These are the two A records and the CNAME record.

Delete the Network Load Balancer from the EC2 Load Balancer section.
Next, you can run the command terraform destroy to destroy the entire infrastructure that you created with Terraform.
After that, you also need to delete the ECR repository, S3 bucket, and Dynamodb table that you created manually.
Finally, you should check all the AWS services that you used and make sure that they are destroyed and no longer available. This will prevent any unwanted charges.
You also need to restore your domain registrar NS settings to the previous ones. This will disconnect your domain name from AWS and make it available for other purposes.

I hope you enjoyed the article and learned something new. Please tell me how it went for you, or ask me for help if you faced any difficulties. I would love to hear your feedback.

kubernetes #k8s #github #githubactions #cicd #aws #eks #ecr #python #nodejs #redis #postgresql #devops #containerization #terraform