Forem: Amit Kushwaha

->> Day-27 Automating AWS Infrastructure Using Terraform & Github Actions

Amit Kushwaha — Tue, 03 Mar 2026 08:12:53 +0000

In modern cloud environments, manually provisioning infrastructure is inefficient, error-prone, and not scalable.

To solve this, I built a fully automated AWS infrastructure using Terraform integrated with GitHub Actions for CI/CD.

This project provisions a production-style architecture including:

Custom VPC
Application Load Balancer
Auto Scaling Group
EC2 instances
Remote backend using S3
Multi-environment configuration (dev, test, prod)

All infrastructure is defined as code and deployed automatically via GitHub.

No manual console clicks. Just version-controlled automation.

Architecture

Deployment Flow

Developer pushes Terraform code to GitHub
GitHub Actions workflow triggers
Terraform executes:
- terraform init
- terraform validate
- terraform plan
- Manual approval required
- terraform apply

AWS infrastructure is provisioned automatically

Tech Stack

Terraform (Infrastructure as Code)
GitHub Actions (CI/CD automation)
AWS (VPC, EC2, ASG, ALB, S3)

- Remote backend with S3 for state management

Project Structure

.
├── terraform/
│   ├── main.tf
│   ├── vpc.tf
│   ├── security_groups.tf
│   ├── alb.tf
│   ├── asg.tf
│   ├── s3.tf
│   ├── backend.tf
│   ├── dev.tfvars
│   ├── test.tfvars
│   └── prod.tfvars
│
├── .github/workflows/
│   ├── terraform.yaml
│   └── terraform-destroy.yaml
│
├── scripts/
│   └── user_data.sh
│
└── README.md

Multi Environment Deployment

One of the key design decisions was environment separation.

This project supports:

dev
test
prod

Each environment has its own `.tfvars` file, allowing controlled configuration changes without modifying core infrastructure code.

Remote State Management

Terraform state is stored in:

S3 (remote backend)

This ensures:

Centralized state storage
Team collaboration support
State consistency

This avoids local state conflicts and improves production readiness.

Github Actions Workflow

Two workflows were implemented:

1. Deployment Workflow

Triggers on push and performs:
Checkout repository
Configure AWS credentials via GitHub Secrets
Setup Terraform
Initialize backend
Validate configuration
Plan and Apply infrastructure

2. Destroy Workflow

Allows controlled teardown of infrastructure using:

terraform destroy

This helps prevent unnecessary AWS costs.

Implemented Features

Infrastructure as Code using Terraform
CI/CD pipeline integration
Auto Scaling architecture
Application Load Balancer routing
EC2 bootstrapping via user_data script
Multi-environment deployment

- Remote backend configuration

Conclusion

This project demonstrates how Terraform and GitHub Actions can be combined to build a fully automated, scalable AWS infrastructure.

By eliminating manual provisioning and adopting Infrastructure as Code, we achieve:

Consistency
Scalability
Faster deployments

- Reduced human error

Resources

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Dev.to / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

->> Day-26 Provisioning an AWS S3 Bucket using HCP Terraform

Amit Kushwaha — Mon, 02 Mar 2026 12:26:02 +0000

In this blog, I implemented a cloud-based Terraform workflow using HCP Terraform integrated with Github to provision an AWS S3 in a prodcution style setup.

>> Project Objective:

The goal was to:

Define AWS infrastructure using Terraform
Store and version control the code in Github
Execute Terraform runs a remotely using HCP Terraform
Implement a VCS- driven automated workflow
Manage state securely in the cloud
Isolate environments using Projects and Workspaces

>> Architecture Overview:

The deployment workflow follows this structure:

Developer -> Github -> HCP -> Terraform -> AWS -> S3 Bucket

Execution Flow

Write Terraform configuration for S3 bucket.
Push the code to GitHub.
HCP Terraform detects the change.
Automatically runs terraform init and terraform plan.
Review the plan in the UI.
Confirm and apply the changes.
AWS provisions the S3 bucket.

Step-by-Step Guide to deploy an aws s3 bucket using HCP Terraform

Prerequisites

Before starting, make sure you have:

AWS Account
GitHub Account
HCP Terraform Account
Basic knowledge of Terraform syntax

Step 1: Create a GitHub Repository

Log in to GitHub.
Create a new repository (e.g., terraform-s3-demo).
Clone it locally:

git clone https://github.com/your-username/terraform-s3-demo.git
cd terraform-s3-demo

Step:2 Write Terraform Configuration

Create the following files:

main.tf

provider "aws" {
  region = var.region
}

resource "aws_s3_bucket" "mybucket" {
  bucket = var.bucket_name

  tags = {
    Name        = var.bucket_name
    Environment = var.environment
  }
}

variables.tf

variable "region" {}
variable "bucket_name" {}
variable "environment" {}

Step3: Push Code to Github

git add .
git commit -m "Initial S3 bucket Terraform configuration"
git push origin main

Your Terraform code is now version-controlled.

Step 4: Set Up HCP Terraform

Log in to HCP Terraform
Create a new Organization
Inside the organization, create a Project

Projects help logically group infrastructure.

Step 5: Create a VCS-Driven Workspace

Click Create Workspace
Select Version Control Workflow
Connect your GitHub account
Choose the repository (terraform-s3-demo)
Set working directory (if needed)
Create workspace

Now your repo is linked to HCP Terraform.

Step:6 Configure Varibales in Workspace

Inside the Workspace -> varibales section:

Add Environment Variables

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

mark them as sensitive.

Add Terraform variables
Example:

region = ap-south-1
bucket_name = amit-terraform-demo-bucket
environment = dev

Do NOT hardcode credentials in code.

Step 7: Trigger the First Run

Now go back to GitHub and make a small change (or re-push code).

HCP Terraform will automatically:

Clone the repository
Run terraform init
Run terraform plan
Show execution plan in UI

Step 8: Review and Apply

Review the plan output.
Click Confirm & Apply.
Wait for execution to complete.

If successful, your S3 bucket will be created in AWS.

Step 9: Verify in AWS Console

Log in to AWS.
Navigate to S3.
Confirm the bucket is created.

Congratulations - infrastructure deployed using cloud-based Terraform workflow.

>> Secure Credential Management:

AWS credentials were added as sensitive environment variables inside the HCP Terraform workspace.

This ensures:

No secrets in source code
Secure execution
Production-aligned security practice

Resource:
Github Repo: Github Repo
Hashicorp: Hashicorp

Conslusion

This project showcases how to provision AWS infrastructure using a cloud-native Terraform workflow powered by HCP Terraform and GitHub.

By combining Infrastructure as Code with automated VCS-driven execution, the deployment process becomes:

Repeatable
Secure
Collaborative

- Production-ready

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

->> Day-25 Terraform Import In AWS

Amit Kushwaha — Sat, 28 Feb 2026 12:38:42 +0000

managing Existing AWS Infrastructure Using Terraform Import!

When working in real-world cloud environments, infrastructure is not always created using infrastructure as Code from day one.

Sometimes resources already exist - created manually through the AWS Console.

So the question becomes:

How do we bring those existing resources under Terraform management safely?

That’s exactly what this project demonstrates.

The Problem
You already have:

A VPC
A Security Group
Possibly EC2 instances

But none of them are managed through Terraform.

Managing infrastructure manually:

X Is not version controlled
X Is not reproducible
X Is error-prone

We need a structured way to manage it using Iac - without recreating everything.

The Solution: `terraform import`

Terraform provides a command that allows you to map existing cloud resources into Terraform state:

terraform import <resource_type.resource_name> <resource_id>

This command does not create infrastructure.
It simply tells Terraform:

| "This resource already exists. start managing it."

Architecture Overview

The workflow looks like this:

Write Terraform configuration files (.tf)
Configure AWS provider
Reference existing VPC using a data source
Define the Security Group in Terraform
Use terraform import to attach the real AWS resource to Terraform state
Validate using terraform plan

Once imported, Terraform can now track and manage that resource.

Project Structure

terraform/
├── main.tf           # Provider configuration
├── variables.tf      # Region and VPC input
├── vpc.tf            # Fetch existing VPC using data source
├── security_group.tf # Define Security Group to import

Import Workflow

Initialize Terraform

terraform init

Import Existing Security Group

terraform import aws_security_group.app_sg sg-xxxxxxxx

Terraform now maps the real AWS Security Group to the resource block.

Terraform validate

terraform plan

If everything matches, you’ll see No changes.

That means Terraform and AWS are in sync.

Conclusion

Adopting Infrastructure as Code doesn’t mean you need to rebuild everything from scratch.

With terraform import, you can gradually transition manual cloud infrastructure into a version-controlled, structured Terraform workflow.

This is a practical and realistic DevOps approach - especially in environments where infrastructure already exists.

Resources:

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

->> Day-24 Highly Available and Scalable Architecture Using Terraform

Amit Kushwaha — Fri, 27 Feb 2026 20:10:41 +0000

Deploying a Scalable Dockerized Django Application on AWS using Terraform

Modern applications are not deployed on a single EC2 instance anymore.

In this project, I built a production-style architecture on AWS using Terraform, where:

A Dockerized Django application runs on EC2 instances
Instances are deployed in private subnets
Application Load Balancer handles incoming traffic
Auto Scaling Group maintains availability
NAT Gateways provide outbound internet access
docker images are pulled from Docker Hub

Everything was provisioned using Infrastructure as Code.

Project Goal

Deploy a containerized Django app securely
Follow real production architecture patterns
Keep EC2 instance private
Enable horizontal scaling
Automate the full infrastructure using Terraform

Architecture Diagram

Internet → ALB (Public) → EC2 Instances (Private) → NAT Gateways → Internet
                                ↓
                          Django Docker App

Networking Design

VPC

Custom VPC created using Terraform
CIDR-based subnet planning

Public Subnets (AZ1 & AZ2)

Host Application Load Balancer
Host NAT Gateways
Connected to Internet Gateway

Private Subnets (AZ1 & AZ2)

Host EC2 instances
No direct internet access
Outbound access via NAT Gateway

Why this matters:

Private subnets ensure your application servers are not publicly exposed.

Application Load Balancer (ALB)

The ALB:

Listens on HTTP (80) / HTTPS (if configured)
Forwards traffic to target group
Performs health checks
Distributes traffic across instances in multiple AZs

Auto Scaling Group (ASG)

Configured with:

Minimum: 1 instance
Desired: 2 instances
Maximum: 5 instances

The ASG:

Maintains desired capacity
Replaces unhealthy instances
Works across multiple availability zones

If one AZ fails, traffic shifts automatically.

EC2 + Docker Deployment

EC2 instances are launched using a Launch Template.

Inside the user data script:

Docker is installed
Django image is pulled from Docker Hub
Container is started with port mapping

docker run -d -p 80:8000 your-django-image

Port mapping:

Django runs internally on 8000
Exposed externally via port 80
ALB forwards traffic to port 80

NAT Gateway Usage

Since EC2 instances are in private subnets:

They cannot directly access the internet.

NAT Gateway allows:

Pulling Docker images from Docker Hub
Installing updates
Accessing external services

Deployment Steps

terraform init
terraform plan
terraform apply

Github Repo

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

->> Day-23 Setup End-to-End Observability in AWS Using Terraform

Amit Kushwaha — Mon, 23 Feb 2026 12:25:11 +0000

A production-ready AWS Lambda function for automated image processing with enterprise-grade CloudWatch monitoring, implemented using modular Terraform.

Overview:

This project demonstrates AWS serverless best practices by combining:

Lambda-based image processing (resize, compress, format conversion)
S3 event-driven architecture (automatic triggering)
Comprehensive CloudWatch monitoring (metrics, alarms, dashboards)
SNS alerting (email/SMS notifications)
Modular Terraform (reusable, maintainable infrastructure)

What It Does-

Upload an image to S3 upload bucket
Lambda function automatically triggers
Processes image (creates 5 variants: compressed, low-quality, WebP, PNG, thumbnail)
Saves processed images to destination bucket
Monitors everything with CloudWatch metrics and alarms
Sends alerts via SNS when issues occur

Architecture

Key Features:-

Image Processing

Multiple format supports (JPEG, PNG, WebP, BMP, TIFF)
Automatic format conversion
Quality-based compression (85%, 60%)
Thumbnail generation (300x300)
Large image resizing (max 4096px)
Automatic color space conversion

Monitoring & Observability

12 CloudWatch Alarms:

Error rate monitoring
Duration/timeout warnings
Throttle detection
Memory usage tracking
Concurrent execution limits
Log-based error patterns

Custom Metrics:

Image processing time
Image sizes processed
Success/failure rates
Business-level insights

Comprehensive Dashboard:

Real-time metrics visualization
AWS metrics + custom metrics
Log insights integration
Performance trends

Log-Based Alerts:

Timeout detection
Memory errors
S3 permission issues
Image processing failures
Critical application errors

Infrastructure

Modular Terraform (6 reusable modules)
Security best practices (IAM least privilege, S3 encryption)
Scalable architecture (auto-scaling Lambda)
Cost-optimized (pay per use)
Environment-agnostic (dev/staging/prod)

->> Day-22 2-Tier Architecture Setup on AWS Using Terraform

Amit Kushwaha — Sun, 08 Feb 2026 13:30:11 +0000

Introduction:

In the world of DevOps, building secure, scalable, and automated infrastructure is a superpower. Today, we are going to provision a classic Two-Tier Architecture on AWS completely from scratch using Terraform. We won't just launch an EC2 instance, we are going to build a production-ready environment with custom VPC networking, private subnets for the database, secret management, and automated application bootstrapping.

->> We are Building:

Web Tier: A Flask python application running on an ubuntu EC2 instance in a public subnet.
Database Tier: A managed MySQL RDS instance hidden securely in private subnets.
Security: Minimal privilege security Groups and AWS Secrets Manager for credential handling.
Automation: Zero manual configuration inside the server, everything is scripted via Terraform user_data.

The Architecture

Before writing code, let's visualize what we are building.

VPC: Our own isolated network (10.0.0.0/16).
Public Subnet: For the Web Server (Internet accessible).
Private Subnets: Two of them across different Availability Zones for the RDS database (No direct Internet access).
Internet Gateway: To allow the web server to talk to the world.

Step 1: The Networking Foundation(VPC)

We need a VPC, one public subnet for the web server, and two private subnets for the RDS instance (AWS RDS requires at least two AZs for high availability).

Resource: aws_vpc, aws_subnets, aws_internet_gateway, aws_route_table.

# VPC Module
module "vpc" {
  source = "./modules/vpc"

  project_name         = var.project_name
  environment          = var.environment
  aws_region           = var.aws_region
  vpc_cidr             = var.vpc_cidr
  public_subnet_cidr   = var.public_subnet_cidr
  private_subnet_cidrs = var.private_subnet_cidrs
}

## Step 2: Security & Firewall Rules

Security Groups act as our virtual firewalls. We follow the Principle of Least Privilege.

Web SG: Allows HTTP (80) from anywhere (0.0.0.0/0) and SSH (22) for management.
Database SG: This is where the magic happens. We do not open port 3306 to the world. We only allow traffic from the Web Security Group.

# Security Groups Module
module "security_groups" {
  source = "./modules/security_groups"

  project_name = var.project_name
  environment  = var.environment
  vpc_id       = module.vpc.vpc_id
}

## Step 3: Managing Secrets

Never, ever hardcode database passwords in your main.tf files. We use the Terraform Random Provider to generate a password and store it immediately in AWS Secrets Manager.

# Secrets Module
module "secrets" {
  source = "./modules/secrets"

  project_name = var.project_name
  environment  = var.environment
  db_username  = var.db_username
}

Step 4: The Database (RDS MySQL)

We provision an AWS RDS instance running MySQL. We place it in the private subnets using a db_subnet_group so it's not accessible from the public internet.

The password? It's pulled dynamically from our Secrets module!

resource "aws_db_instance" "main" {
  identifier             = "${var.project_name}-db"
  allocated_storage      = var.allocated_storage
  storage_type           = "gp2"
  engine                 = "mysql"
  engine_version         = var.engine_version
  instance_class         = var.instance_class
  db_name                = var.db_name
  username               = var.db_username
  password               = var.db_password
  parameter_group_name   = "default.mysql8.0"
  skip_final_snapshot    = true
  vpc_security_group_ids = [var.db_security_group_id]
  db_subnet_group_name   = aws_db_subnet_group.main.name
  publicly_accessible    = false

  tags = {
    Name        = "${var.project_name}-rds"
    Environment = var.environment
  }
}

Step 5: The Application Server (EC2 + User Data)

This is the coolest part. We don't want to SSH in and manually install Python, Flask, and Git. We use Terraform's user_data to script the entire setup process.

When the instance launches, it:

Updates Ubuntu packages.
Installs Python3 and Flask.
Creates a simple Flask App connecting to our MySQL DB.
Injects the Database Host and Credentials (passed from Terraform variables) directly into the Python code.
Starts the web server as a systemd service.

# modules/ec2/templates/user_data.sh
#!/bin/bash
pip install flask mysql-connector-python

# Terraform Template Injection happening here:
DB_CONFIG = {
    "host": "${db_host}",
    "user": "${db_username}",
    "password": "${db_password}",
    "database": "${db_name}"
}
# ... application logic ...

Deployment Time!

With our modular structure in place (vpc, ec2, rds, secrets, security_groups), deploying the entire stack is as simple as:

Initialize: terraform init
Plan: terraform plan
Apply: terraform apply -auto-approve

Wait about 5-8 minutes (RDS takes a while to spin up), and Terraform will output your application URL.

Verification

Open the application_url in your browser. You should see the nice blue and white "Terraform RDS Demo" dashboard.

Try typing a message and clicking Save.
If it appears in "Recent Messages", congratulations! Your EC2 instance successfully talked to your RDS database in the private subnet!

Conclusion

In this project, we moved beyond simple resource creation and built a fully integrated environment.

Key Takeaways:

Modular Design: Reusable code makes infrastructure manageable.
Security First: Private subnets and strict Security Groups are essential.
Automation: user_data saves hours of manual configuration.
Secret Management: Never store secrets in plain text.

GitHub Repository

Github Repository

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

Thanks for reading! Happy Terraforming!

->> Day-21 AWS Policy and Governance Setup Using Terraform

Amit Kushwaha — Sat, 07 Feb 2026 20:05:39 +0000

Introduction

In this blog, I share my experience implementing AWS Policy and Governance using Terraform as part of my #30DaysOfAWSTerraform journey. The goal was to build a secure-by-default foundation that enforces policies and continuously monitors compliance.

This project combines IAM guardrails, AWS Config, and a secure S3 bucket for configuration history. It helped me learn how prevention (IAM policies) and detection (Config rules) work together in real-world cloud governance.

Project Objective

Implement IAM policies for security guardrails
Enable AWS Config for continuous monitoring
Store configuration history securely in S3
Enforce tagging standards
Track compliance and violations
Automate governance using Terraform

Architecture

IAM Policies to prevent risky actions (MFA delete, TLS-only S3 access, required tags).
AWS Config to record configuration changes and evaluate compliance rules.
S3 Bucket to store AWS Config snapshots securely with encryption and versioning.

The IAM policies enforce guardrails upfront, AWS Config continuously checks resource compliance, and S3 stores audit data.

Implementation Steps:

Step 1: IAM Policy Setup
I created policies for:

MFA Delete Policy to block S3 object deletion without MFA
S3 Encryption in Transit to enforce HTTPS/TLS
Required Tags Policy to ensure resources include Environment and Owner

These policies matter because they stop risky actions before they happen.

Step 2: AWS Config Setup
I configured:

Config Recorder to track resource changes
Delivery Channel to store snapshots in S3
Recorder Status to start compliance tracking

Step 3: Adding Config Rules
I added AWS managed rules to validate governance:

S3 Public Write Prohibited - Prevents public write access to S3 buckets
S3 Encryption Enabled - Ensures server-side encryption on S3 buckets
S3 Public Read Prohibited - Blocks public read access to S3 buckets
EBS Volumes Encrypted - Verifies all EBS volumes are encrypted
Required Tags - Checks for Environment and Owner tags
IAM Password Policy - Enforces strong password requirements
Root MFA Enabled - Ensures root account has MFA configured

Non-compliant means the resource violates a rule (for example, missing tags or encryption)

Step 5: Terraform Automation

Terraform let me define everything as code: IAM policies, the Config recorder, rules, and the S3 bucket. This made the setup repeatable and version controlled.

Benefits:

Faster deployments
Consistent governance
Easy auditing and updates

Step 6: Testing & Validation

I ran terraform plan and terraform apply, then verified compliance using AWS Config. The dashboard showed compliant and non-compliant resources clearly.

Monitoring Dashboard

AWS Config provides a central view of compliance status across rules and resources. It helps quickly identify violations and track fixes over time.

Cost Considerations

AWS Config is a paid service, so I kept the scope small and cleaned up resources when done using terraform destroy. This helps control costs while still learning the full workflow.

Conclusion

This project showed how governance can be automated with Terraform by combining IAM guardrails, AWS Config compliance checks, and secure S3 storage. It reinforced the value of policy‑as‑code, continuous monitoring, and defense‑in‑depth in real AWS environments. Most importantly, it mirrors how cloud teams enforce security at scale—making it a practical and recruiter‑relevant demonstration of cloud governance skills.

Reference:

Resources:

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

Happy Terraforming and Deploying!!

->> Day-20 Terraform Custom Modules for EKS - From Zero to Production

Amit Kushwaha — Tue, 03 Feb 2026 00:01:13 +0000

Kubernetes (K8s) has become the de facto standard for orchestrating containerized applications. It provides powerful primitives for deploying, scaling, and managing containerized workloads, making it a top choice for modern DevOps teams and cloud-native development.

In this blog series, we’ll explore how to set up a production-ready Kubernetes environment on AWS using Amazon Elastic Kubernetes Service (EKS) and Terraform, starting with the foundational infrastructure.

Why EKS?

Amazon EKS is a fully managed Kubernetes service that makes it easy to run Kubernetes on AWS without needing to install or operate your own control plane or nodes. EKS handles high availability, scalability, and patching of the Kubernetes control plane, so you can focus on running your applications instead of managing infrastructure.

-> Benefits of using EKS:

Managed control plane: No need to run your own etcd or master nodes.
Native AWS integration: IAM, VPC, CloudWatch, EC2, ECR and more.
Secure by default: Runs in a dedicated, isolated VPC.
Scalable and production-ready.

-> In our setup:

The VPC module creates a network with public and private subnets.
The IAM module creates cluster roles, node roles, and OIDC provider for Kubernetes-AWS integration.
The ECR module creates a container registry to store and manage Docker images.
The EKS module provisions the EKS control plane and worker nodes in private subnets.
The Secrets Manager module stores optional database, API, and application configuration secrets.

Architecture Overview

Here's how the setup works at high level:

VPC is created with 3 Availability Zones for high availability.
Each AZ contains both a public and a private subnet.
EKS worker nodes (EC2 instances) are launched in private subnets for better security.
A NAT Gateway is provisioned in a public subnet to allow worker nodes in private subnets to pull images and updates from the internet (e.g., from ECR, Docker Hub).
EKS control plane (managed by AWS) communicates with the worker nodes securely within the VPC.
The Internet Gateway in the public subnet provides external users access to the Kubernetes LoadBalancer service for the demo website.
IAM roles and OIDC provider enable pod-level permissions through IRSA (IAM Roles for Service Accounts).
KMS encryption secures the etcd database at rest on the EKS control plane.

This setup ensures that your nodes are not directly exposed to the internet while still having outbound internet access via the NAT gateway.

The Five Custom Terraform Modules

Step 1: Create the VPC

The foundation of our infrastructure. Creates networking with high availability across multiple AZs.

# Custom VPC Module
module "vpc" {
  source = "./modules/vpc"

  name_prefix     = var.cluster_name
  vpc_cidr        = var.vpc_cidr
  azs             = slice(data.aws_availability_zones.available.names, 0, 3)
  private_subnets = var.private_subnets
  public_subnets  = var.public_subnets

  enable_nat_gateway = true
  single_nat_gateway = true

  # Required tags for EKS
  public_subnet_tags = {
    "kubernetes.io/role/elb"                    = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb"           = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  }

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Cluster"
  }
}

what it creates:

VPC with CIDR 10.0.0.0/16
3 public subnets (10.0.1-3.0/24) for NAT Gateway and Internet Gateway
3 private subnets (10.0.11-13.0/24) for EKS nodes
Single NAT Gateway for cost optimization
Internet Gateway for public internet access
20 total resources

Step 2: IAM Module

Handles all identity and across management. Enables secure communication between Kubernetes and AWS services.

# Custom IAM Module
module "iam" {
  source = "./modules/iam"

  cluster_name = var.cluster_name

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Cluster"
  }
}

what it creates

EKS cluster IAM role with necessary permissions
EC2 node IAM role for worker nodes
OIDC Provider for Kubernetes-AWS integration
IRSA (IAM Roles for Service Accounts) configuration
Inline policies for EKS and node permissions
7 total resources

Step 3: Create the EKS Cluster

Provisions the Kubernetes cluster with managed control plane and worker nodes.

We use the terraform-aws-eks module to spin up the cluster. This will provision:

A managed EKS control plane
A node group with autoscaling enabled
Nodes inside private subnets with internet access via NAT Gateway

# Custom EKS Module
module "eks" {
  source = "./modules/eks"

  cluster_name       = var.cluster_name
  kubernetes_version = var.kubernetes_version
  vpc_id             = module.vpc.vpc_id
  subnet_ids         = module.vpc.private_subnets

  cluster_role_arn = module.iam.cluster_role_arn
  node_role_arn    = module.iam.node_group_role_arn

  endpoint_public_access  = true
  endpoint_private_access = true
  public_access_cidrs     = ["0.0.0.0/0"]

  enable_irsa = true

  # Node groups configuration
  node_groups = {
    general = {
      instance_types = ["t3.medium"]
      desired_size   = 2
      min_size       = 2
      max_size       = 4
      capacity_type  = "ON_DEMAND"
      disk_size      = 20

      labels = {
        role = "general"
      }

      tags = {
        NodeGroup = "general"
      }
    }

    spot = {
      instance_types = ["t3.medium", "t3a.medium"]
      desired_size   = 1
      min_size       = 1
      max_size       = 3
      capacity_type  = "SPOT"
      disk_size      = 20

      labels = {
        role = "spot"
      }

      taints = [{
        key    = "spot"
        value  = "true"
        effect = "NO_SCHEDULE"
      }]

      tags = {
        NodeGroup = "spot"
      }
    }
  }

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Cluster"
  }

  depends_on = [module.iam]
}

what it creates:

EKS cluster control plane (Kubernetes 1.31)
4 worker node groups: 2 on-demand (general), 1 on-demand (general), 1 spot (cost-optimized)
Cluster security groups and node security groups
CloudWatch logging configuration
Add-ons (CoreDNS, kube-proxy, VPC CNI, EBS CSI)
KMS encryption for etcd
17 total resources

Step 4: ECR Module

Container registry for strong and managing Docker images.

module "ecr" {
  source = "./modules/ecr"

  repository_name = "demo-website"

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Cluster"
  }
}

what it creates:

Elastic Container Registry repository
Image scanning on push
Lifecycle policies for image retention
1 total resource

Step 5: Secrets Manager Module

Securely stores sensitive data like database credentials and API keys (Optional).

module "secrets_manager" {
  source = "./modules/secrets-manager"

  name_prefix = var.cluster_name

  # Enable secrets as needed
  create_db_secret         = var.enable_db_secret
  create_api_secret        = var.enable_api_secret
  create_app_config_secret = var.enable_app_config_secret

  # Database credentials (if enabled)
  db_username = var.db_username
  db_password = var.db_password
  db_engine   = var.db_engine
  db_host     = var.db_host
  db_port     = var.db_port
  db_name     = var.db_name

  # API keys (if enabled)
  api_key    = var.api_key
  api_secret = var.api_secret

  # App config (if enabled)
  app_config = var.app_config

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Cluster"
  }
}

What it Creates:

Optional database secrets
Optional API secrets
Optional application configuration secrets
0-3 total resources (optional)

How Modules Work Together

In our setup:

The VPC module creates a network with public and private subnets
The EKS module provisions the EKS control plane and worker nodes in private subnets
The IAM module creates cluster roles, node roles, and OIDC provider for Kubernetes-AWS integration
The ECR module creates a container registry to store and manage Docker images
The Secrets Manager module stores optional database, API, and application configuration secrets.

Deploying the Infrastructure

Step 1: Initialize Terraform

cd terraform
terraform init

This downloads the required Terraform providers and initializes the working directory.

Step 2: Review the Plan

terraform plan

This shows all 45 resources that will be created.

Step 3: Apply Configuration

terraform apply

Apply complete! Resources: 45 added, 0 changed, 0 destroyed.


![ ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2e4tc8uo4l56lb4ahmgm.png)


![ ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9dx9ngv84i012smoxw9j.png)

Outputs:
cluster_endpoint = "https://EA6F63CF5CF44B594EA9533013CF21C4.gr7.us-east-1.eks.amazonaws.com"
cluster_name = "eks-custom-modules-cluster"
ecr_repository_url = "123456789.dkr.ecr.us-east-1.amazonaws.com/demo-website"

Step 4: Configure Kubectl

terraform output -raw configure_kubectl

This outputs the aws eks update-kubeconfig command. Run it to connect kubectl to your cluster.

Step 5: Deploy Demo Application
Build and push a Docker Image to ECR:

cd ../demo-website

# Build Docker image
docker build -t demo-website:latest .

# Get ECR login command
cd ../terraform
$(terraform output -raw ecr_login_command)

# Tag and push to ECR
docker tag demo-website:latest <ECR_URL>:latest
docker push <ECR_URL>:latest

Deploy to Kubernetes:

cd ../demo-website
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

# Get LoadBalancer URL
kubectl get svc demo-website -o wide

Check deployment status:

$ kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
demo-website-5d9c8d7f6-2m4kl    1/1     Running   0          30s
demo-website-5d9c8d7f6-7p9q2    1/1     Running   0          30s

$ kubectl get svc
NAME           TYPE           CLUSTER-IP    EXTERNAL-IP                                        PORT(S)        AGE
demo-website   LoadBalancer   172.20.0.1    a1234567890.elb.us-east-1.amazonaws.com           80:31234/TCP   45s

Access the demo website at the LoadBalancer URL!

Cleanup

Once you're done experimenting, clean up resources to avoid charges:

# Delete Kubernetes resources
kubectl delete svc demo-website
kubectl delete deployment demo-website

# Destroy infrastructure
cd terraform
terraform destroy -auto-approve

Conclusion

We've successfully set up a production-grade Kubernetes cluster on AWS using custom Terraform modules. By building our own modules, we achieved.

References:

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

Happy Terraforming and Deploying!!

->> Day-19 Terraform Provisioners in Action: A Hands-On Demo (local-exec, remote-exec & file)

Amit Kushwaha — Tue, 27 Jan 2026 11:09:12 +0000

When learning Terraform, most people stop at creating infrastructure.

But what if you want to run scripts, install packages, or trigger actions after a resource is created?

That's where Terraform Provisioners come in.

In this blog, I will walk through:

What provisioner really are
When (and when not) to use them
A hands-on AWS EC2 demo using: - local-exec - remote-exec - file + remote-exec

What are Terraform Provisioners?

Provisioners let Terraform execute scripts or commands during resource creation or destruction.

They’re useful for tasks that Terraform can’t model declaratively, such as:

Installing software
Running bootstrap scripts
Registering resources in external systems
Sending notifications
Copying files to servers

⚠️ Terraform officially recommends using provisioners as a last resort.
Prefer user_data, cloud-init, Packer, or configuration management tools for serious production setups.

Types of Provisioners

1. local-exec

Runs commands on your local machine.

Use case:

Trigger webhooks
Call APIs
Write to local inventory files
Send Slack notifications

   provisioner "local-exec" {
   command = "echo 'Local-exec: created instance ${self.id} with IP ${self.public_ip}'"
   }

2. remote-exec

Runs commands on the remote resource via SSH (Linux) or WinRM (Windows).

Use Cases:

Install packages (nginx, docker, node, etc.)
Configure OS settings
Start services
Quick bootstrap scripts

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "echo 'Hello from remote-exec' | sudo tee /tmp/remote_exec.txt",
    ]
  }

3. file

Copies files from your machine to the remote resource.

Use Cases:

Copy setup scripts
Upload config files
Transfer certificates
Deploy small binaries

  provisioner "file" {
    source      = "${path.module}/scripts/welcome.sh"
    destination = "/tmp/welcome.sh"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo chmod +x /tmp/welcome.sh",
      "sudo /tmp/welcome.sh"
    ]
  }

Architecture

Laptop / CI
     │
     ▼
Terraform
     │
     ▼
AWS EC2 (Ubuntu)
     │
     ├── local-exec   → runs locally
     ├── remote-exec  → runs via SSH on EC2
     └── file         → copies scripts to EC2

-> Prerequisites

Before running the demo:

AWS credentials configured
Terraform v1.0+ installed
AWS CLI installed
SSH client installed
An EC2 key pair created

aws ec2 create-key-pair --key-name terraform-demo-key \
  --query 'KeyMaterial' --output text > terraform-demo-key.pem

chmod 400 terraform-demo-key.pem

Main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

# EC2 instance used for provisioner demos.
# Each provisioner block is included below but wrapped in block comments (/* ... */).
# For the demo, uncomment one provisioner block at a time, then `terraform apply`.

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical (Ubuntu official) - Current owner ID

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

resource "aws_security_group" "ssh" {
  name        = "tf-prov-demo-ssh"
  description = "Allow SSH inbound"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "demo" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  tags = {
    Name = "terraform-provisioner-demo"
  }

  connection {
    type        = "ssh"
    user        = var.ssh_user
    private_key = file(var.private_key_path)
    host        = self.public_ip
    timeout     = "5m"
  }

  /*
  ------------------------------------------------------------------
  Provisioner 1: local-exec
  - Runs on the machine where you run Terraform (your laptop/CI agent).
  - Useful for local tasks, logging, calling local scripts, etc.
  - To demo: uncomment this block, then run `terraform apply`.
  ------------------------------------------------------------------
  */

  # provisioner "local-exec" {
  #   command = "echo 'Local-exec: created instance ${self.id} with IP ${self.public_ip}'"
  # }


  /*
  ------------------------------------------------------------------
  Provisioner 2: remote-exec
  - Runs commands on the remote instance over SSH.
  - Requires SSH access (security group + key pair + reachable IP).
  - To demo: uncomment this block, ensure `var.private_key_path` is correct, then run `terraform apply`.
  ------------------------------------------------------------------
  */

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "echo 'Hello from remote-exec' | sudo tee /tmp/remote_exec.txt",
    ]
  }


  /*
  ------------------------------------------------------------------
  Provisioner 3: file + remote-exec
  - Copies a script (scripts/welcome.sh) to the instance, then executes it.
  - Good pattern for more complex bootstrapping when script files are preferred.
  - To demo: uncomment both the file provisioner and the remote-exec block below.
  ------------------------------------------------------------------
  */

  # provisioner "file" {
  #   source      = "${path.module}/scripts/welcome.sh"
  #   destination = "/tmp/welcome.sh"
  # }

  # provisioner "remote-exec" {
  #   inline = [
  #     "sudo chmod +x /tmp/welcome.sh",
  #     "sudo /tmp/welcome.sh"
  #   ]
  # }

}

Demo 1: local-exec

provisioner "local-exec" {
  command = "echo 'Created ${self.id} with IP ${self.public_ip}'"
}

What happens:

EC2 is created
Terraform prints a message on your computer
Nothing changes inside the EC2

Demo 2: remote-exec

provisioner "remote-exec" {
  inline = [
    "sudo apt-get update",
    "echo 'Hello from remote-exec' | sudo tee /tmp/remote_exec.txt"
  ]
}

What happens:

Terraform waits for SSH
Connects to EC2
Runs the commands remotely
Creates /tmp/remote_exec.txt

Verify:

ssh -i terraform-demo-key.pem ubuntu@<PUBLIC_IP>

cat /tmp/remote_exec.txt

Expected output:

Hello from remote-exec

Demo 3: file + remote-exec

provisioner "file" {
  source      = "scripts/welcome.sh"
  destination = "/tmp/welcome.sh"
}

provisioner "remote-exec" {
  inline = [
    "sudo chmod +x /tmp/welcome.sh",
    "sudo /tmp/welcome.sh"
  ]
}

welcome.sh

#!/bin/bash
echo "Welcome to the Provisioner Demo" | sudo tee /tmp/welcome_msg.txt
uname -a | sudo tee -a /tmp/welcome_msg.txt

What happens:

Script is copied to EC2
Script is executed
Output file is created at /tmp/welcome_msg.txt

Resources

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

Cloud Cost Optimization Using Boto3: Automating EC2 Management with AWS Lambda

Amit Kushwaha — Thu, 22 Jan 2026 08:24:35 +0000

->> Introduction

Why do companies migrate from on-premises servers to the cloud?

Simple reasons:

High Maintenance overhead
Expensive infrastructure
Poor scalability
Operational inefficiency

Cloud Platforms like AWS, Azure, and GCP promise:

Pay-as-you-go pricing
Elastic scalability
Managed services
Faster innovation

Sounds perfect right?
But here's the reality check:

Migrating to the cloud does not automatically mean your costs will go down.

Cloud cost optimization is a shared responsibility.

Cloud providers give you powerful tools.
It's your job to use them responsibly.

As a DevOps Engineer, one of your core responsibilities is:

Managing resources efficiently and cleaning up unused or stale infrastructure

And That's exactly what this project is about.

The Real Problem?

Let's take a very common scenario.

You spin up an Ec2 instance to host an application.
Its run in Dev/Test environment.
The workday ends...
But the instance keeps running overnight.
And the next night.
And the next week.

Money is burning silently.

Now multiply this by:

20 developers
Multiple AWS regions
Multiple projects
Multiple Environments

Manually stopping instances?
Not scalable.
Not reliable.
Not realistic.

Solution Overview

I built a serverless AWS cost optimization system that:

Automatically detects non-critical EC2 instances
Checks business hours
Analyzes CPU usage
Stops underutilized machines
Logs every action
Sends email alerts
Runs on a schedule

All powered by:

AWS Lambds
Boto3 (Python SDK)
CloudWatch Metrics
DynamoDB
SNS
EventBridge

Why Serverless (Why Lambda?)

Lambda is perfect for this use case because:
No server management
Pay only for execution time
Event-driven automation
Auto scales
Secure with IAM

Instead of running a VM 24/7 just to check EC2 usage...

We let AWS run code only when needed.

That's peak cloud efficiency.

Architecture

Here's how the system works:

EventBridge (Schedule)
        ↓
AWS Lambda (Cost Optimizer)
        ↓
EC2 Instances  ←→  CloudWatch Metrics
        ↓
DynamoDB Logs
        ↓
SNS Email Alerts

Flow Explanation:

EventBridge triggers Lambda on a Schedule
Lambda scans EC2 instances using tags
Lambda checks:
- Business hours
- CPU utilization
If eligible -> stops EC2
Logs action into DynamoDB
Sends email alert using SNS

Tag-Based Safety Layer

One of the smartest design choices was tag-based filtering.

Only EC2 instances with these tags processed:

Tag Key ** **Required Value
AutoStop Yes
Environment Dev or Test
Critical No

This ensures:

X Production systems are never touched.
X Critical workloads are protected.
Only disposable environments are optimized.

Time-Based Optimization

Business hours rules:

If current time >= 8 PM OR < 8 AM
→ instance is eligible for stopping

Time is calculated using a configurable time zone.

so, the system works globally.

CPU-Based Optimization

Even during business hours, some machines are just... idle.

Lambda fetches CPU metrics from CloudWatch:

Average CPU < CPU_THRESHOLD
→ instance is eligible for stopping

Lambda fetches waste from zombie EC2 instances doing absolutely nothing.

DRY_RUN Mode
One of the most important features:

DRY_RUN = true

In this mode:

Lambda does NOT stop EC2
Logs and alerts still run
You can safely test logic

When ready:

DRY_RUN = false

Now Lambda actually stops EC2.
This prevents accidental disasters

Core Lambda Code

import boto3
import datetime
import os
import pytz

ec2 = boto3.client('ec2')
cloudwatch = boto3.client('cloudwatch')
sns = boto3.client('sns')
ddb = boto3.resource('dynamodb')

TABLE_NAME = os.environ['TABLE_NAME']
SNS_TOPIC_ARN = os.environ['SNS_TOPIC_ARN']
DRY_RUN = os.environ['DRY_RUN'].lower() == "true"
CPU_THRESHOLD = int(os.environ['CPU_THRESHOLD'])
TIMEZONE = os.environ['TIMEZONE']

table = ddb.Table(TABLE_NAME)

def get_cpu_utilization(instance_id):
    response = cloudwatch.get_metric_statistics(
        Namespace='AWS/EC2',
        MetricName='CPUUtilization',
        Dimensions=[{'Name': 'InstanceId', 'Value': instance_id}],
        StartTime=datetime.datetime.utcnow() - datetime.timedelta(minutes=30),
        EndTime=datetime.datetime.utcnow(),
        Period=300,
        Statistics=['Average']
    )

    if not response['Datapoints']:
        return None

    latest = sorted(response['Datapoints'], key=lambda x: x['Timestamp'])[-1]
    return latest['Average']

def lambda_handler(event, context):
    tz = pytz.timezone(TIMEZONE)
    now = datetime.datetime.now(tz)
    hour = now.hour

    instances = ec2.describe_instances(Filters=[
        {'Name': 'tag:AutoStop', 'Values': ['Yes']},
        {'Name': 'tag:Environment', 'Values': ['Dev', 'Test']},
        {'Name': 'tag:Critical', 'Values': ['No']},
        {'Name': 'instance-state-name', 'Values': ['running']}
    ])

    for res in instances['Reservations']:
        for inst in res['Instances']:
            instance_id = inst['InstanceId']
            cpu = get_cpu_utilization(instance_id)

            reason = None
            if hour >= 20 or hour < 8:
                reason = "After hours"
            elif cpu is not None and cpu < CPU_THRESHOLD:
                reason = f"Low CPU: {cpu}%"

            if reason:
                if not DRY_RUN:
                    ec2.stop_instances(InstanceIds=[instance_id])

                table.put_item(Item={
                    "InstanceId": instance_id,
                    "Timestamp": str(now),
                    "Reason": reason
                })

                sns.publish(
                    TopicArn=SNS_TOPIC_ARN,
                    Subject="EC2 Optimization Alert",
                    Message=f"Stopped {instance_id} due to {reason}"
                )

    return {"statusCode": 200, "body": "Optimization complete"}

DynamoDB Logging

Each stop action is logged:

{
  "InstanceId": "i-0abc12345",
  "Timestamp": "2026-01-22 20:15:04",
  "Reason": "Low CPU: 4.2%"
}

This gives:

Audit history
Compliance Data
Debugging Visibility
Cost optimization reports

SNS Notifications

Every action triggers an email alert:

Subject:

EC2 Optimization Alert

Message:

Stopped i-0abc12345 due to After hours

This ensures:

Transparency
Human awareness
Manual override if needed

Automation with EventBridge

EventBridge schedules Lambda:

Rule Purpose

EC2AutoStopRule Runs every hour or at 8PM
EC2AutoStartRule Starts instances at 8 AM

Now the system is fully hands-free.

Challenges I Faced

Runtime mismatch
Python 3.14 Lambda + Python 3.10 layer -> crash

Fix: matched runtime versions
IAM permission hell
Missing permissions broke EC2 stop, DynamoDB logs, SNS alerts

Fix: attached least-privilege IAM policies
Environment variable bugs
Wrong variable names caused KeyErrors

Fix: standardized env vars (TABLE_NAME, SNS_TOPIC_ARN)
Lambda working but EC2 not stopping
Turns out DRY_RUN was still true

Fix: flipped it to false
No CPU metrics available
CloudWatch had no datapoints
Fix: added safe fallback logic

Conclusion

Cloud cost optimization is not just about cutting bills - it’s about building responsible, automated, and scalable systems.

In this project, we proved that with the right mix of AWS serverless services and Python automation, it’s possible to create a production-grade cost optimization system that:

Automatically stops non-essential EC2 instances
Protects critical workloads using tags
Makes smart decisions using time and CPU metrics
Logs every action for audit and visibility
Sends real-time alerts for transparency
Runs fully hands-free using EventBridge
Includes a DRY_RUN safety mode to prevent accidents

This approach eliminates manual intervention, reduces human error, and ensures that cloud resources are used only when they are truly needed.

By combining Lambda, Boto3, CloudWatch, DynamoDB, SNS, and EventBridge, we created a lightweight yet powerful solution that can be easily extended to:

Auto-start instances in the morning
Optimize EBS volumes and snapshots
Integrate Slack or Teams notifications
Track cost trends using AWS Cost Explorer
Manage multi-account AWS environments

Resources:

->> Day-18 Image Processing Serverless Project using AWS Lambda (with terraform)

Amit Kushwaha — Tue, 13 Jan 2026 22:12:33 +0000

In this tutorial, I'll show you how to build a production-ready serverless image processing pipeline that automatically creates multiple image variants when you upload a photo to S3.

What we'll build:

Automatic image processing triggered by S3 uploads
5 different image variants (compressed, low-quality, WebP, PNG, thumbnail)
Email notifications via SNS
Complete Infrastructure as Code using Terraform
Cross-platform Lambda Layer build with Docker

Tech Stack:

AWS Lambda (Python 3.12)
AWS S3 (storage)
AWS SNS (notifications)
Terraform (infrastructure)
Docker (Lambda layer build)
Pillow (image processing)

Architecture Overview

The flow is simple:

User uploads an image to the Source S3 Bucket
S3 event triggers the Lambda Function
Lambda (with Pillow layer) processes the image into 5 variants
Processed images are saved to the Destination S3 Bucket
SNS sends an email notification with processing details
CloudWatch logs everything for monitoring

Why This Architecture?

Serverless Benefits
No Server Management

No EC2 instances to maintain
No patching or updates
Automatic scaling

Cost-Effective

Pay only for execution time
~$0.14/month for 1,000 images
Free tier covers most small projects

Event-Driven

Automatic processing on upload
No polling or cron jobs needed
Real-time processing

Prerequisites

Before we start, make sure you have:

AWS Account with CLI configured

aws configure

Terraform (v1.0+)

terraform --version

Docker desktop(running)

docker info

Basic knowledge of:

AWS services (S3, Lambda, SNS)
Terraform Basics
Python

Project Structure

Day-18/
├── Assets/
│   ├── architecture-diagram.jpg
│   └── ... (screenshots)
├── lambda/
│   ├── lambda_function.py
│   └── requirements.txt
├── scripts/
│   ├── build_layer_docker.sh
│   ├── deploy.sh
│   └── destroy.sh
├── terraform/
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   ├── provider.tf
│   └── terraform.tfvars.example
└── Readme.md

Step1: Lambda Function
Let's start with the core - the Lambda function that processes images.

The Image Processor

import json
import boto3
import os
from PIL import Image
from io import BytesIO
import uuid

s3_client = boto3.client('s3')
sns_client = boto3.client('sns')

def lambda_handler(event, context):
    """Process uploaded images into multiple variants"""

    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']

        # Download image
        response = s3_client.get_object(Bucket=bucket, Key=key)
        image_data = response['Body'].read()

        # Process image
        processed_images = process_image(image_data, key)

        # Upload variants
        processed_bucket = os.environ['PROCESSED_BUCKET']
        for img in processed_images:
            s3_client.put_object(
                Bucket=processed_bucket,
                Key=img['key'],
                Body=img['data'],
                ContentType=img['content_type']
            )

        # Send notification
        send_notification(key, processed_images, processed_bucket)

    return {'statusCode': 200}

Creating Image Variants

def process_image(image_data, original_key):
    """Create 5 variants of the image"""
    processed_images = []
    image = Image.open(BytesIO(image_data))

    # Convert RGBA to RGB for JPEG compatibility
    if image.mode in ('RGBA', 'LA', 'P'):
        background = Image.new('RGB', image.size, (255, 255, 255))
        if image.mode == 'P':
            image = image.convert('RGBA')
        background.paste(image, mask=image.split()[-1])
        image = background

    # Auto-resize large images
    if image.size[0] > 4096 or image.size[1] > 4096:
        ratio = min(4096 / image.size[0], 4096 / image.size[1])
        new_size = (int(image.size[0] * ratio), int(image.size[1] * ratio))
        image = image.resize(new_size, Image.Resampling.LANCZOS)

    base_name = os.path.splitext(original_key)[0]
    unique_id = str(uuid.uuid4())[:8]

    # Create variants
    variants = [
        {'format': 'JPEG', 'quality': 85, 'suffix': 'compressed'},
        {'format': 'JPEG', 'quality': 60, 'suffix': 'low'},
        {'format': 'WEBP', 'quality': 85, 'suffix': 'webp'},
        {'format': 'PNG', 'quality': None, 'suffix': 'png'}
    ]

    for variant in variants:
        output = BytesIO()
        if variant['quality']:
            image.save(output, format=variant['format'], 
                      quality=variant['quality'], optimize=True)
        else:
            image.save(output, format=variant['format'], optimize=True)

        output.seek(0)
        extension = variant['format'].lower()
        if extension == 'jpeg':
            extension = 'jpg'

        processed_images.append({
            'key': f"{base_name}_{variant['suffix']}_{unique_id}.{extension}",
            'data': output.getvalue(),
            'content_type': f"image/{variant['format'].lower()}"
        })

    # Create thumbnail
    thumbnail = image.copy()
    thumbnail.thumbnail((300, 300), Image.Resampling.LANCZOS)
    thumb_output = BytesIO()
    thumbnail.save(thumb_output, format='JPEG', quality=80, optimize=True)
    thumb_output.seek(0)

    processed_images.append({
        'key': f"{base_name}_thumbnail_{unique_id}.jpg",
        'data': thumb_output.getvalue(),
        'content_type': 'image/jpeg'
    })

    return processed_images

Step 2: Building the Lambda Layer

The Docker Challenge
Problem: AWS Lambda runs on Linux, but you might be developing on Windows or Mac. The Pillow library has C dependencies that must be compiled for the target OS.

Solution: Use Docker to create a Linux environment and build the layer there.

The Build Script

#!/bin/bash
set -e

echo "🚀 Building Lambda Layer with Pillow using Docker..."

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_DIR="$( cd "$SCRIPT_DIR/.." && pwd )"
TERRAFORM_DIR="$PROJECT_DIR/terraform"

# Check Docker is running
if ! docker info &> /dev/null 2>&1; then
    echo "❌ Docker is not running. Please start Docker first."
    exit 1
fi

# Get Windows-compatible path
if command -v cygpath &> /dev/null; then
    DOCKER_MOUNT_PATH=$(cygpath -w "$TERRAFORM_DIR")
elif [[ -n "$WINDIR" ]]; then
    DOCKER_MOUNT_PATH=$(cd "$TERRAFORM_DIR" && pwd -W 2>/dev/null || pwd)
else
    DOCKER_MOUNT_PATH="$TERRAFORM_DIR"
fi

# Build layer in Linux container
docker run --rm \
  --platform linux/amd64 \
  -v "$DOCKER_MOUNT_PATH":/output \
  python:3.12-slim \
  bash -c "
    pip install --quiet Pillow==10.4.0 -t /tmp/python/lib/python3.12/site-packages/ && \
    cd /tmp && \
    apt-get update -qq && apt-get install -y -qq zip > /dev/null 2>&1 && \
    zip -q -r pillow_layer.zip python/ && \
    cp pillow_layer.zip /output/ && \
    echo '✅ Layer built successfully!'
  "

echo "📍 Location: $TERRAFORM_DIR/pillow_layer.zip"

Step 3: Terraform Infrastructure

Main Infrastructure (main.tf)

# S3 Upload Bucket
resource "aws_s3_bucket" "upload_bucket" {
  bucket        = local.upload_bucket_name
  force_destroy = true  # Allows easy cleanup
}

resource "aws_s3_bucket_versioning" "upload_bucket" {
  bucket = aws_s3_bucket.upload_bucket.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "upload_bucket" {
  bucket = aws_s3_bucket.upload_bucket.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# S3 Processed Bucket
resource "aws_s3_bucket" "processed_bucket" {
  bucket        = local.processed_bucket_name
  force_destroy = true
}

# Lambda Function
resource "aws_lambda_function" "image_processor" {
  filename         = data.archive_file.lambda_zip.output_path
  function_name    = local.lambda_function_name
  role             = aws_iam_role.lambda_role.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.12"
  timeout          = var.lambda_timeout
  memory_size      = var.lambda_memory_size
  layers           = [aws_lambda_layer_version.pillow_layer.arn]

  environment {
    variables = {
      PROCESSED_BUCKET = aws_s3_bucket.processed_bucket.id
      SNS_TOPIC_ARN    = var.notification_email != "" ? aws_sns_topic.processing_notifications[0].arn : ""
    }
  }
}

# Lambda Layer
resource "aws_lambda_layer_version" "pillow_layer" {
  filename            = "${path.module}/pillow_layer.zip"
  layer_name          = "${var.project_name}-pillow-layer"
  compatible_runtimes = ["python3.12"]
  description         = "Pillow library for image processing"
}

# S3 Event Trigger
resource "aws_s3_bucket_notification" "upload_bucket_notification" {
  bucket = aws_s3_bucket.upload_bucket.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.image_processor.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.allow_s3]
}

# SNS Topic
resource "aws_sns_topic" "processing_notifications" {
  count        = var.notification_email != "" ? 1 : 0
  name         = "${var.project_name}-${var.environment}-notifications"
  display_name = "Image Processing Notifications"
}

resource "aws_sns_topic_subscription" "email_subscription" {
  count     = var.notification_email != "" ? 1 : 0
  topic_arn = aws_sns_topic.processing_notifications[0].arn
  protocol  = "email"
  endpoint  = var.notification_email
}

IAM Permissions

resource "aws_iam_role" "lambda_role" {
  name = "${local.lambda_function_name}-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy" "lambda_policy" {
  name = "${local.lambda_function_name}-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ]
        Resource = "arn:aws:logs:${var.aws_region}:*:*"
      },
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:GetObjectVersion"
        ]
        Resource = "${aws_s3_bucket.upload_bucket.arn}/*"
      },
      {
        Effect = "Allow"
        Action = [
          "s3:PutObject",
          "s3:PutObjectAcl"
        ]
        Resource = "${aws_s3_bucket.processed_bucket.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = var.notification_email != "" ? aws_sns_topic.processing_notifications[0].arn : "*"
      }
    ]
  })
}

Step 4: Deployment

Configuration
Create terraform.tfvars:

aws_region         = "us-east-1"
environment        = "dev"
project_name       = "serverless-image-processor"
lambda_timeout     = 60
lambda_memory_size = 1024
notification_email = "your-email@example.com"

Deploy with Scripts

# 1. Build Lambda Layer
cd scripts
./build_layer_docker.sh

# 2. Deploy Infrastructure
./deploy.sh

Manual Deployment

# 1. Build layer
cd scripts
./build_layer_docker.sh

# 2. Initialize Terraform
cd ../terraform
terraform init

# 3. Plan
terraform plan

# 4. Apply
terraform apply

Testing the Pipeline

1. Confirm SNS Subscription
Check your email for the AWS SNS confirmation and click "Confirm subscription".

2. Upload a Test Image

# Get bucket name
terraform output upload_bucket_name

# Upload image
aws s3 cp test-image.jpg s3://YOUR-UPLOAD-BUCKET/

3. Check Processed Images

# List processed variants
aws s3 ls s3://YOUR-PROCESSED-BUCKET/ --recursive

Expected output:

test-image_compressed_a1b2c3d4.jpg
test-image_low_a1b2c3d4.jpg
test-image_webp_a1b2c3d4.webp
test-image_png_a1b2c3d4.png
test-image_thumbnail_a1b2c3d4.jpg

Lessons:

1. Docker is Essential for Lambda Layers
Initially, I tried installing Pillow directly on Windows. The layer worked locally but failed on Lambda with:

Unable to import module 'lambda_function': No module named '_imaging'

Solution: Always use Docker to build layers for Lambda, regardless of your development OS.

2. Force Destroy is Your Friend (in Dev)
Without force_destroy = true on S3 buckets, terraform destroy fails if buckets contain objects.

resource "aws_s3_bucket" "upload_bucket" {
  bucket        = local.upload_bucket_name
  force_destroy = true  # Enables easy cleanup
}

Warning: Never use this in production!

3. Image Format Conversion is Tricky
JPEG doesn't support transparency. Converting RGBA images directly to JPEG results in black backgrounds.

Solution: Create a white background and paste the image with alpha channel as mask:

if image.mode in ('RGBA', 'LA', 'P'):
    background = Image.new('RGB', image.size, (255, 255, 255))
    background.paste(image, mask=image.split()[-1])
    image = background

4. SNS Requires Email Confirmation
SNS subscriptions aren't active until the user confirms via email. Make sure to mention this in documentation!

5. Unique Filenames Prevent Conflicts
Using UUIDs in filenames prevents overwriting when processing multiple images with the same name:

unique_id = str(uuid.uuid4())[:8]
output_key = f"{base_name}_{variant['suffix']}_{unique_id}.{extension}"

Conclusion

We've built a production-ready serverless image processing pipeline that:

Automatically processes images on upload
Creates 5 optimized variants
Sends email notifications
Costs less than $0.15/month for 1,000 images
Scales automatically
Requires zero server management

Resources:

Github Repository
AWS Lambda
Pillow Docs
Terraform AWS Provider

Reference:

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Found this helpful? Drop a ❤️ and follow for more AWS and Terraform tutorials!

Questions? Drop them in the comments below! 👇

->> Day-17 AWS Terraform Blue-Green Deployment Using Elastic Beanstalk

Amit Kushwaha — Sun, 11 Jan 2026 08:16:36 +0000

Introduction:

In a traditional approach to application deployment, you typically fix a failed deployment by redeploying an earlier, stable version of the application. Redeployment in traditional data centers is typically done on the same set of resources due to the cost and effort of provisioning additional resources. Although this approach works, it has many shortcomings. Rollback isn’t easy because it’s implemented by redeployment of an earlier version from scratch. This process takes time, making the application potentially unavailable for long periods. Even in situations where the application is only impaired, a rollback is required, which overwrites the faulty version. As a result, you have no opportunity to debug the faulty application in place.

Applying the principles of agility, scalability, utility consumption, as well as the automation capabilities of Amazon Web Services can shift the paradigm of application deployment. This enables a better deployment technique called blue/green deployment.

Blue/Green Deployment Methodology:

Blue/green deployments provide releases with near zero-downtime and rollback capabilities. The fundamental idea behind blue/green deployment is to shift traffic between two identical environments that are running different versions of your application. The blue environment represents the current application version serving production traffic. In parallel, the green environment is staged running a different version of your application. After the green environment is ready and tested, production traffic is redirected from blue to green. If any problems are identified, you can roll back by reverting traffic back to the blue environment.

Architecture diagram

Benefits of Blue/Green Deployment:

Zero Downtime: Seamless traffic transition ensures uninterrupted service.
Safe Rollbacks: Easy reversion to the Blue environment if issues arise.
Efficient Testing: Allows comprehensive testing of the new environment without affecting live users.
Scalability: Works well for both monolithic and microservices architectures.

AWS Elastic Beanstalk:

AWS Elastic Beanstalk performs an in-place update when you update your application versions, your application might become unavailable to users for a short period of time. To avoid this, perform a blue/green deployment. To do this, deploy the new version to a separate environment, and then swap the CNAMEs of the two environments to redirect traffic to the new version instantly.

To validate this approach, I tested and deployed Blue-Green Deployment with javascript application, This hands-on implementation ensured real-world feasibility and reliability.

Project Highlights:

Tech Stack: Terraform, Elastic Beanstalk, EC2, S3.
Goal: Achieve zero downtime while updating my application.
Key Challenges: Ensuring that visitors never experience an outage during deployments.
Solution: Implementing Blue-Green Deployment using Elastic Beanstalk and EC2.

Project Structure
My project is structured in the following way:

Day-17/
├── Readme.md
├── Assets/
└── terraform/
    ├── .terraform/
    ├── .terraform.lock.hcl
    ├── app-v1/
    │   ├── app.js
    │   ├── package.json
    │   └── app-v1.zip
    ├── app-v2/
    │   ├── app.js
    │   ├── package.json
    │   └── app-v2.zip
    ├── blue-environments.tf
    ├── green-environments.tf
    ├── main.tf
    ├── outputs.tf
    ├── package-apps.ps1
    ├── package-apps.sh
    ├── swap-environments.ps1
    ├── swap-environments.sh
    ├── terraform.tfstate
    ├── terraform.tfstate.backup
    ├── terraform.tfvars.example
    └── variables.tf

Checkout my Github Repository

Step-by-step Implementation:

Step 1: Package Your Applications

Before deploying, you need to package your application code into ZIP files. The repository includes packaging scripts for different operating systems.

For Linux/Mac:

chmod +x package-apps.sh
./package-apps.sh

For Windows:

.\package-apps.ps1

These scripts create app-v1.zip and app-v2.zip files that Terraform will upload to S3.

Step 2: Core Infrastructure Setup (main.tf)
The main.tf file establishes the foundational infrastructure:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

# IAM Role for Elastic Beanstalk EC2 instances
resource "aws_iam_role" "eb_ec2_role" {
  name = "${var.app_name}-eb-ec2-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })

  tags = var.tags
}

# Attach the AWS managed policy for Web Tier
resource "aws_iam_role_policy_attachment" "eb_web_tier" {
  role       = aws_iam_role.eb_ec2_role.name
  policy_arn = "arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier"
}

# Attach the AWS managed policy for Worker Tier
resource "aws_iam_role_policy_attachment" "eb_worker_tier" {
  role       = aws_iam_role.eb_ec2_role.name
  policy_arn = "arn:aws:iam::aws:policy/AWSElasticBeanstalkWorkerTier"
}

# Attach the AWS managed policy for Multicontainer Docker
resource "aws_iam_role_policy_attachment" "eb_multicontainer_docker" {
  role       = aws_iam_role.eb_ec2_role.name
  policy_arn = "arn:aws:iam::aws:policy/AWSElasticBeanstalkMulticontainerDocker"
}

# Instance Profile
resource "aws_iam_instance_profile" "eb_ec2_profile" {
  name = "${var.app_name}-eb-ec2-profile"
  role = aws_iam_role.eb_ec2_role.name

  tags = var.tags
}

# IAM Role for Elastic Beanstalk Service
resource "aws_iam_role" "eb_service_role" {
  name = "${var.app_name}-eb-service-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "elasticbeanstalk.amazonaws.com"
        }
      }
    ]
  })

  tags = var.tags
}

# Attach Enhanced Health Reporting policy
resource "aws_iam_role_policy_attachment" "eb_service_health" {
  role       = aws_iam_role.eb_service_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSElasticBeanstalkEnhancedHealth"
}

# Attach Managed Updates policy
resource "aws_iam_role_policy_attachment" "eb_service_managed_updates" {
  role       = aws_iam_role.eb_service_role.name
  policy_arn = "arn:aws:iam::aws:policy/AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy"
}

# Elastic Beanstalk Application
resource "aws_elastic_beanstalk_application" "app" {
  name        = var.app_name
  description = "Blue-Green Deployment Demo Application"

  tags = var.tags
}

# S3 Bucket for application versions
resource "aws_s3_bucket" "app_versions" {
  bucket = "${var.app_name}-versions-${data.aws_caller_identity.current.account_id}"

  tags = var.tags
}

# Block public access to S3 bucket
resource "aws_s3_bucket_public_access_block" "app_versions" {
  bucket = aws_s3_bucket.app_versions.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# Data source for current AWS account
data "aws_caller_identity" "current" {}

Step 3: Blue Environment Configuration

The blue-environment.tf file defines the production environment running version 1.0:

# Application Version 1.0 (Blue Environment - Production)
resource "aws_s3_object" "app_v1" {
  bucket = aws_s3_bucket.app_versions.id
  key    = "app-v1.zip"
  source = "${path.module}/app-v1/app-v1.zip"
  etag   = filemd5("${path.module}/app-v1/app-v1.zip")

  tags = var.tags
}

resource "aws_elastic_beanstalk_application_version" "v1" {
  name        = "${var.app_name}-v1"
  application = aws_elastic_beanstalk_application.app.name
  description = "Application Version 1.0 - Initial Release"
  bucket      = aws_s3_bucket.app_versions.id
  key         = aws_s3_object.app_v1.id

  tags = var.tags
}

# Blue Environment (Production)
resource "aws_elastic_beanstalk_environment" "blue" {
  name                = "${var.app_name}-blue"
  application         = aws_elastic_beanstalk_application.app.name
  solution_stack_name = var.solution_stack_name
  tier                = "WebServer"
  version_label       = aws_elastic_beanstalk_application_version.v1.name

  # IAM Settings
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "IamInstanceProfile"
    value     = aws_iam_instance_profile.eb_ec2_profile.name
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "ServiceRole"
    value     = aws_iam_role.eb_service_role.name
  }

  # Instance Settings
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "InstanceType"
    value     = var.instance_type
  }

  # Environment Type (Load Balanced)
  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "EnvironmentType"
    value     = "LoadBalanced"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "LoadBalancerType"
    value     = "application"
  }

  # Auto Scaling Settings
  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MinSize"
    value     = "1"
  }

  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MaxSize"
    value     = "2"
  }

  # Health Reporting
  setting {
    namespace = "aws:elasticbeanstalk:healthreporting:system"
    name      = "SystemType"
    value     = "enhanced"
  }

  # Platform Settings
  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "HealthCheckPath"
    value     = "/"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "Port"
    value     = "8080"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "Protocol"
    value     = "HTTP"
  }

  # Environment Variables
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "ENVIRONMENT"
    value     = "blue"
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "VERSION"
    value     = "1.0"
  }

  # Deployment Policy
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "DeploymentPolicy"
    value     = "Rolling"
  }

  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "BatchSizeType"
    value     = "Percentage"
  }

  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "BatchSize"
    value     = "50"
  }

  # Managed Updates
  setting {
    namespace = "aws:elasticbeanstalk:managedactions"
    name      = "ManagedActionsEnabled"
    value     = "false"
  }

  tags = merge(
    var.tags,
    {
      Environment = "blue"
      Role        = "production"
    }
  )
}

Step 4: Green Environment Configuration

The green-environment.tf file creates an identical staging environment with version 2.0:

# Application Version 2.0 (Green Environment - Staging)
resource "aws_s3_object" "app_v2" {
  bucket = aws_s3_bucket.app_versions.id
  key    = "app-v2.zip"
  source = "${path.module}/app-v2/app-v2.zip"
  etag   = filemd5("${path.module}/app-v2/app-v2.zip")

  tags = var.tags
}

resource "aws_elastic_beanstalk_application_version" "v2" {
  name        = "${var.app_name}-v2"
  application = aws_elastic_beanstalk_application.app.name
  description = "Application Version 2.0 - New Feature Release"
  bucket      = aws_s3_bucket.app_versions.id
  key         = aws_s3_object.app_v2.id

  tags = var.tags
}

# Green Environment (Staging/Pre-production)
resource "aws_elastic_beanstalk_environment" "green" {
  name                = "${var.app_name}-green"
  application         = aws_elastic_beanstalk_application.app.name
  solution_stack_name = var.solution_stack_name
  tier                = "WebServer"
  version_label       = aws_elastic_beanstalk_application_version.v2.name

  # IAM Settings
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "IamInstanceProfile"
    value     = aws_iam_instance_profile.eb_ec2_profile.name
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "ServiceRole"
    value     = aws_iam_role.eb_service_role.name
  }

  # Instance Settings
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "InstanceType"
    value     = var.instance_type
  }

  # Environment Type (Load Balanced)
  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "EnvironmentType"
    value     = "LoadBalanced"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "LoadBalancerType"
    value     = "application"
  }

  # Auto Scaling Settings
  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MinSize"
    value     = "1"
  }

  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MaxSize"
    value     = "2"
  }

  # Health Reporting
  setting {
    namespace = "aws:elasticbeanstalk:healthreporting:system"
    name      = "SystemType"
    value     = "enhanced"
  }

  # Platform Settings
  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "HealthCheckPath"
    value     = "/"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "Port"
    value     = "8080"
  }

  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "Protocol"
    value     = "HTTP"
  }

  # Environment Variables
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "ENVIRONMENT"
    value     = "green"
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "VERSION"
    value     = "2.0"
  }

  # Deployment Policy
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "DeploymentPolicy"
    value     = "Rolling"
  }

  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "BatchSizeType"
    value     = "Percentage"
  }

  setting {
    namespace = "aws:elasticbeanstalk:command"
    name      = "BatchSize"
    value     = "50"
  }

  # Managed Updates
  setting {
    namespace = "aws:elasticbeanstalk:managedactions"
    name      = "ManagedActionsEnabled"
    value     = "false"
  }

  tags = merge(
    var.tags,
    {
      Environment = "green"
      Role        = "staging"
    }
  )
}

Step 5: Deploy the Infrastructure

Execute the Terraform workflow:

# Initialize Terraform
terraform init

# Review the planned changes
terraform plan

# Apply the configuration
terraform apply

This creates both environments and uploads your application versions to S3. The output will display the URLs for both environments.

Step 6: Verify Deployment

After deployment, access both environments using the provided URLs:

Blue Environment: http://blue-environment.xxx.elasticbeanstalk.com

Displays: "This is version 2.0"

Status: Current production environment

Green Environment: http://green-environment.xxx.elasticbeanstalk.com

Displays: "This is version 2.0 with new features"

Status: Staging environment

Step 7: Perform the Blue-Green Swap

Method: AWS Console

Navigate to Elastic Beanstalk → Environments
Select the blue-environment
Click Actions → Swap environment URLs
Choose green-environment as the target
Click Swap

After swapping, verify that:

The blue URL now serves version 2.0
The green URL now serves version 1.0

Cleanup

After completing your testing, destroy all resources to avoid charges:

terraform destroy

Conclusion

Blue-green deployments with Terraform and AWS Elastic Beanstalk provide a robust framework for releasing applications with confidence. This approach eliminates downtime, enables instant rollbacks, and maintains high availability during deployments.

Reference:

>> Connect With Me

If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:

💼 LinkedIn: Amit Kushwaha

🐙 GitHub: Amit Kushwaha

📝 Hashnode / Amit Kushwaha

🐦 Twitter/X: Amit Kushwaha

Forem: Amit Kushwaha

->> Day-27 Automating AWS Infrastructure Using Terraform & Github Actions

Architecture

AWS infrastructure is provisioned automatically

Tech Stack

- Remote backend with S3 for state management

Project Structure

Multi Environment Deployment

Each environment has its own .tfvars file, allowing controlled configuration changes without modifying core infrastructure code.

Remote State Management

This avoids local state conflicts and improves production readiness.

Github Actions Workflow

This helps prevent unnecessary AWS costs.

Implemented Features

- Remote backend configuration

Conclusion

- Reduced human error

Resources

>> Connect With Me

Questions? Drop them in the comments below! 👇

->> Day-26 Provisioning an AWS S3 Bucket using HCP Terraform

>> Project Objective:

>> Architecture Overview:

Step-by-Step Guide to deploy an aws s3 bucket using HCP Terraform

Prerequisites

Step 1: Create a GitHub Repository

Step:2 Write Terraform Configuration

main.tf

variables.tf

Step3: Push Code to Github

Step 4: Set Up HCP Terraform

Step 5: Create a VCS-Driven Workspace

Step:6 Configure Varibales in Workspace

mark them as sensitive.

Do NOT hardcode credentials in code.

Step 7: Trigger the First Run

Step 8: Review and Apply

Step 9: Verify in AWS Console

Congratulations - infrastructure deployed using cloud-based Terraform workflow.

>> Secure Credential Management:

Conslusion

- Production-ready

>> Connect With Me

Questions? Drop them in the comments below! 👇

->> Day-25 Terraform Import In AWS

managing Existing AWS Infrastructure Using Terraform Import!

The Solution: terraform import

Architecture Overview

Project Structure

Import Workflow

Terraform validate

Conclusion

Resources:

>> Connect With Me

Questions? Drop them in the comments below! 👇

->> Day-24 Highly Available and Scalable Architecture Using Terraform

Deploying a Scalable Dockerized Django Application on AWS using Terraform

Project Goal

Architecture Diagram

Networking Design

VPC

Public Subnets (AZ1 & AZ2)

Private Subnets (AZ1 & AZ2)

Why this matters:

Application Load Balancer (ALB)

The ALB:

Auto Scaling Group (ASG)

Configured with:

The ASG:

EC2 + Docker Deployment

NAT Gateway Usage

NAT Gateway allows:

Deployment Steps

>> Connect With Me

Questions? Drop them in the comments below! 👇

->> Day-23 Setup End-to-End Observability in AWS Using Terraform

Overview:

What It Does-

Architecture

Key Features:-

Each environment has its own `.tfvars` file, allowing controlled configuration changes without modifying core infrastructure code.

The Solution: `terraform import`

Fix: standardized env vars (`TABLE_NAME`, `SNS_TOPIC_ARN`)