Forem: Anil KUMAR

AWS Lambda: Deactivate Inactive IAM Keys

Anil KUMAR — Wed, 28 Jan 2026 11:03:40 +0000

This marks the first blog of AWS Lambda Series. I will be doing some automations on AWS using Lambda and will be posting them here with a blog. In this blog, we will use Lambda and Event driven functions to deactivate/disable the keys which are older than 3 months to keep our AWS account safe and secure.

Imagine you are working in a big team and you have multiple people working across various environments. You will be having a lot of security credentials lying with of no use. So for those use cases, you can consider this automation where it will remove the access keys and secret_access_keys of the users in lower environments which are older than a month.

Consider this automation in 2 phases:

PHASE-1 Notification & Test Setup :

Create a SNS Topic and add your email id to the subscription topic of that SNS. So In this case whenever a key has been disbaled you will be informed via email.

Once the SNS is created and your email has been added to the subscription, then go to the email and confirm the subscription.

Now Create a Dummy IAM user for practice purpose and create security access and secret access keys.

PHASE-2 Lambda & Event Configuration:

Create a IAM Execution Role for Lambda so that it should have all the permissions for the lambda to execute.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:ListUsers",
                "iam:ListAccessKeys",
                "iam:UpdateAccessKey",
                "sns:Publish",
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

You could see that the above role have access to list users, keys and publish it to sns and create logs for the same.

Develop the Lambda Function logic with all the code and pre-requisites.
Give the runtime as Python 3.12 and also extend the timeout to 1 minute as the default timeout is 3 seconds which will be not sufficient.
Copy the code from this repo and save it in the Lambda.
Now create the EventBridge function to trigger Lambda in relevant events. While creating, select a target of AWS Service such as Lambda and create a role for that. Select the rate of 1 minute so that it will execute for every one minute.

Phase 3: Automated Remediation

Now for testing function, go to the lambda function and click on deploy and test, you can see that the security keys will be disabled, you can check the same in the Monitoring tab of the Lambda Function.

Now we will add the above trigger to the Lambda function we have created so that everything will be automated instead of us going to the Lambda function to deploy.

Go to Lambda -> Select the function -> Click on Add Trigger and select the created Eventbridge Function.

I have created the sample security keys for a user.
Now you can see from the above 2 pics that the keys have been disabled automatically after one minute.

Conclusion:

Exposed IAM access keys are one of the most common causes of AWS account compromise. Manual monitoring does not scale, and delayed response often leads to serious incidents.

The above automation solves the above problems and helps us to:

Reacts automatically
Notifies the right people
Removes the risk without human intervention

Thanks to Sai Kiran Pinapathruni for the Youtube videos and if you have any doubts refer to the youtube video below:

Day 22: Two -Tier Architecture Setup On AWS

Anil KUMAR — Mon, 22 Dec 2025 07:53:23 +0000

Today marks the Day 22 of 30 Days of AWS Terraform Challenge by Piyush Sachdeva. Today we will discuss about the designing and implementation of a secure, modular two-tier AWS architecture using Terraform. Think of this like a secure building with a public lobby and a high-security vault.

Architecture Flow:

The design follows a classic two-tier application model, implemented with security and scalability in mind:

┌─────────────────────────────────────────────────────────────┐
│                         VPC (10.0.0.0/16)                   │
│  ┌─────────────────────┐   ┌─────────────────────────────┐  │
│  │   Public Subnet     │   │     Private Subnets         │  │
│  │   (10.0.1.0/24)     │   │  (10.0.2.0/24, 10.0.3.0/24) │  │
│  │                     │   │                             │  │
│  │  ┌───────────────┐  │   │    ┌──────────────────┐     │  │
│  │  │  EC2 (Flask)  │──│───│───►│   RDS MySQL      │     │  │
│  │  │  Web Server   │  │   │    │   Database       │     │  │
│  │  └───────────────┘  │   │    └──────────────────┘     │  │
│  └─────────────────────┘   └─────────────────────────────┘  │
│           │                                                  │
│           ▼                                                  │
│  ┌─────────────────┐                                        │
│  │ Internet Gateway│                                        │
│  └─────────────────┘                                        │
└─────────────────────────────────────────────────────────────┘
           │
           ▼
      Internet (Users)

Web Tier (Public)

EC2 instance running a Flask application
Deployed in a public subnet
Internet access via Internet Gateway
Listens on port 80

Data Tier (Private)

RDS MySQL instance
Deployed in private subnets across multiple AZs
No direct inbound internet access
Outbound connectivity via NAT Gateway (for patching, backups, etc.)

Secrets Management

AWS Secrets Manager stores database credentials
Terraform generates a strong random password
Secrets stored as JSON (username, password, engine, host)

Network & Security

Custom VPC with public and private subnets
NAT Gateway for private subnet outbound traffic
Tight security group rules enforcing least privilege

The Public Lobby (The Web Tier)

This is where the "front door" of the application lives. It hosts the website that users interact with.

What it is: A computer (EC2) running a simple website (Flask).

Security: It is open to the public so people can visit the site, but it is heavily guarded to only allow web traffic.

The Private Vault (The Data Tier)
This is where all the important information is stored.

What it is: A database (RDS) that holds all the user records.

Security: This area has no internet access. It is buried deep inside the network. The only way to get in is through the "Public Lobby." If a hacker tries to find the database from the outside, it simply doesn’t exist to them.

Secrets Manager:

No Written Passwords: I told the system to create a random, 16-character password automatically.

Hidden Keys: The password is kept in a digital safe. When the website starts up, it asks the safe for the key, uses it to talk to the database, and then throws the "memory" of it away. It’s never saved in a file where people can see it.

Modules:

To keep the code clean, reusable, and production-ready, the project is broken into custom Terraform modules. Each module owns a single responsibility.

secret
Generates a secure database password and stores credentials in AWS Secrets Manager

vpc
Provisions the VPC, public & private subnets, Internet Gateway, route tables, and NAT Gateway

security_group
Creates separate security groups for the web and database tiers

rds
Deploys a private RDS MySQL instance using credentials from Secrets Manager

ec2
Provisions the web server and deploys the Flask app via user data

The root module wires everything together using outputs and input variables.

Implementation:

Secure Secrets Handling:

resource "random_password" "db_password" {
  length           = 16
  special          = true
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

resource "random_id" "suffix" {
  byte_length = 4
}

resource "aws_secretsmanager_secret" "db_password" {
  name        = "${var.project_name}-${var.environment}-db-password-${random_id.suffix.hex}"
  description = "Database password for ${var.project_name}"

  tags = {
    Name        = "${var.project_name}-db-password"
    Environment = var.environment
  }
}

resource "aws_secretsmanager_secret_version" "db_password" {
  secret_id = aws_secretsmanager_secret.db_password.id
  secret_string = jsonencode({
    username = var.db_username
    password = random_password.db_password.result
    engine   = "mysql"
    host     = "" # Will be populated by application or looked up
  })
}

Instead of hardcoding credentials:

Terraform generates a 16-character random password resource.
Credentials are stored securely in AWS Secrets Manager
Secret payload is structured JSON

Only required outputs are passed to dependent modules, making communication easier between modules.

VPC, Subnets, and NAT Gateway:

We have used a custom module of vpc to create all this networking components.

Public subnets host the EC2 instance
Private subnets host the RDS instance.
Internet gateway for instance in Public subnet to access the Internet.

Security Groups:

We used this Security Groups module to make this project Least Privilege by Design.

Web Security Group

Allow HTTP (80) from anywhere
SSH restricted to a specific IP only

Database Security Group

Allow MySQL (3306) only from the web security group
No public exposure

Security groups reference each other instead of CIDR blocks — a cleaner and safer approach.

RDS Module:

We uses this module to create a RDS database.

Deployed across multiple private subnets
Not publicly accessible
Ingress limited strictly to web tier
Designed for availability and isolation

EC2 Module:

The EC2 instance uses a user data template to:

Install system dependencies
Deploy a Flask application
Inject database connection details

The app exposes:

/ – homepage
/health – database connectivity check
/db/info – database metadata

It performs basic insert and read operations to validate end-to-end connectivity.

Implementation:

terraform init
terraform plan
terraform apply

RDS takes time to come up, so wait till RDS is up and running while executing terraform apply.

Validation Steps

After deployment:

Verified Flask app via EC2 public DNS
Tested /health and /db/info endpoints
Confirmed RDS had no public access
Checked Secrets Manager values

Conclusion:

This concludes the Mini project of 2 tier architecture using RDS, EC2 and Networking components.

Day 21: AWS IAM Policy and Governance Setup Using Terraform

Anil KUMAR — Fri, 19 Dec 2025 12:29:01 +0000

Today marks the Day 21 of 30 days of AWS Terraform challeneg Initiative by Piyush Sachdeva. Today we will learn about the AWS IAM Policy and Governance Setup using Terraform.

What is Policy and Governance?

Policy enforces rules to prevent non-compliant actions in AWS:

Users without MFA cannot delete resources
S3 uploads must use HTTPS (encrypted in transit)
Resources like S3 buckets/EC2 must have required tags (e.g., environment=dev)

IAM policies block bad actions before they occur.

If any one tries to delete a S3 bucket with IAM policy not having required permissions for it, then it will block that action preventing deletion of that bucket.

Governance tracks compliance via AWS Config:

Monitors resources after creation.
Logs compliant/non-compliant status.
Stores audit logs in secure S3 bucket.

On the other hand, Governance will not prevent that action but will store everything in a config file inside S3 bucket or any origin source provided.

Ex: If we tried creating a S3 Bucket or an EC2 instance without tags, then IAM policy will prevent that action whereas Governance will not block that action but logs that action as non-complaint status in config.

┌─────────────────────────────────────────────────────┐
│                                                     │
│   1. PREVENTIVE (IAM Policies)                     │
│      → Blocks bad actions BEFORE they happen       │
│      → Example: "Cannot delete S3 without MFA"     │
│                                                     │
│   2. DETECTIVE (AWS Config)                        │
│      → Finds violations AFTER they happen          │
│      → Example: "This bucket is not encrypted"     │
│                                                     │
└─────────────────────────────────────────────────────┘

Project Architecture Overview:

Policy (Prevent) → AWS Config (Detect) → S3 Audit Logs (Store)

Key Components Created:

Encrypted, versioned S3 bucket for audit logs (public access blocked).
3 IAM policies for enforcement.
6 AWS Config rules for compliance monitoring.
IAM roles/users for service access

         ┌──────────────────┐
         │   IAM POLICIES   │  ◄── PREVENT bad actions
         │  • MFA Delete    │
         │  • Encryption    │
         │  • Required Tags │
         └────────┬─────────┘
                  │
                  ▼
         ┌──────────────────┐
         │   AWS CONFIG     │  ◄── DETECT violations
         │   6 Rules        │
         │  (Compliance)    │
         └────────┬─────────┘
                  │
                  ▼
         ┌──────────────────┐
         │    S3 BUCKET     │  ◄── STORE logs
         │  🔒 Encrypted    │
         │  🔒 Versioned    │
         └──────────────────┘

Project Objectives:

Policy Creation: Implement IAM policies to enforce security best practices
Governance Setup: Configure AWS Config for continuous compliance monitoring
Resource Tagging: Demonstrate tagging strategies for resource management
S3 Security: Apply encryption, versioning, and access controls
Compliance Monitoring: Track configuration changes and detect violations

Terraform Implementation Breakdown:

day21/
├── provider.tf       # AWS provider configuration
├── variables.tf      # Input variables
├── main.tf          # S3 bucket and shared resources
├── iam.tf           # IAM policies and roles
├── config.tf        # AWS Config recorder and rules
├── outputs.tf       # Output values
└── README.md        # This file

provider.tf — AWS Provider
What it does: Tells Terraform to use AWS.

variables.tf — Inputs
What it does: Makes the code reusable

main.tf:

In this file, we will be creating S3 Bucket for Audit Logs.

# S3 Bucket to store AWS Config history
resource "aws_s3_bucket" "config_bucket" {
  bucket        = "${var.project_name}-config-bucket-${random_string.suffix.result}"
  force_destroy = true

  tags = {
    Name        = "${var.project_name}-config-bucket"
    Environment = "governance"
    Purpose     = "aws-config-storage"
    ManagedBy   = "terraform"
  }
}

resource "random_string" "suffix" {
  length  = 6
  special = false
  upper   = false
}

# Enable versioning on Config bucket
resource "aws_s3_bucket_versioning" "config_bucket_versioning" {
  bucket = aws_s3_bucket.config_bucket.id
  versioning_configuration {
    status = "Enabled"
  }
}

# Enable encryption on Config bucket
resource "aws_s3_bucket_server_side_encryption_configuration" "config_bucket_encryption" {
  bucket = aws_s3_bucket.config_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# Block public access to Config bucket
resource "aws_s3_bucket_public_access_block" "config_bucket_public_access" {
  bucket = aws_s3_bucket.config_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# S3 Bucket Policy for Config
resource "aws_s3_bucket_policy" "config_bucket_policy" {
  bucket = aws_s3_bucket.config_bucket.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "AWSConfigBucketPermissionsCheck"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
        Action   = "s3:GetBucketAcl"
        Resource = aws_s3_bucket.config_bucket.arn
      },
      {
        Sid    = "AWSConfigBucketExistenceCheck"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
        Action   = "s3:ListBucket"
        Resource = aws_s3_bucket.config_bucket.arn
      },
      {
        Sid    = "AWSConfigBucketPutObject"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
        Action   = "s3:PutObject"
        Resource = "${aws_s3_bucket.config_bucket.arn}/*"
        Condition = {
          StringEquals = {
            "s3:x-amz-acl" = "bucket-owner-full-control"
          }
        }
      },
      {
        Sid       = "DenyInsecureTransport"
        Effect    = "Deny"
        Principal = "*"
        Action    = "s3:*"
        Resource = [
          aws_s3_bucket.config_bucket.arn,
          "${aws_s3_bucket.config_bucket.arn}/*"
        ]
        Condition = {
          Bool = {
            "aws:SecureTransport" = "false"
          }
        }
      }
    ]
  })

  depends_on = [aws_s3_bucket_public_access_block.config_bucket_public_access]
}

For the above S3 bucket creation, we have used random_string of suffix for unique name of S3 bucket and then we have enabled Versioning, Encryption (Server-Side-encryption) and blocked public access for that bucket.

Also, we have added a S3 Bucket Policy for Config so that config should be able to write logs to this bucket without which config will not be able to write logs to this bucket.

iam.tf:

In this file, we will be creating IAM Policy Examples following which the failure of the implementation should result in the errors.

# ------------------------------------------------------------------------------
# 1. IAM Policy Examples
# ------------------------------------------------------------------------------

# Create a custom IAM policy that enforces MFA for deleting S3 objects
resource "aws_iam_policy" "mfa_delete_policy" {
  name        = "${var.project_name}-mfa-delete-policy"
  description = "Policy that requires MFA to delete S3 objects"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "DenyDeleteWithoutMFA"
        Effect   = "Deny"
        Action   = "s3:DeleteObject"
        Resource = "*"
        Condition = {
          BoolIfExists = {
            "aws:MultiFactorAuthPresent" = "false"
          }
        }
      }
    ]
  })
}

# IAM Policy: Enforce encryption in transit for S3
resource "aws_iam_policy" "enforce_s3_encryption_transit" {
  name        = "${var.project_name}-s3-encryption-transit"
  description = "Deny S3 actions without encryption in transit"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "DenyUnencryptedObjectUploads"
        Effect   = "Deny"
        Action   = "s3:PutObject"
        Resource = "*"
        Condition = {
          Bool = {
            "aws:SecureTransport" = "false"
          }
        }
      }
    ]
  })
}

# IAM Policy: Require tagging for resource creation
resource "aws_iam_policy" "require_tags_policy" {
  name        = "${var.project_name}-require-tags"
  description = "Require specific tags when creating resources"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "RequireTagsOnEC2"
        Effect = "Deny"
        Action = [
          "ec2:RunInstances"
        ]
        Resource = "arn:aws:ec2:*:*:instance/*"
        Condition = {
          StringNotLike = {
            "aws:RequestTag/Environment" = ["dev", "staging", "prod"]
          }
        }
      },
      {
        Sid    = "RequireOwnerTag"
        Effect = "Deny"
        Action = [
          "ec2:RunInstances"
        ]
        Resource = "arn:aws:ec2:*:*:instance/*"
        Condition = {
          "Null" = {
            "aws:RequestTag/Owner" = "true"
          }
        }
      }
    ]
  })
}

# IAM User for demonstration
resource "aws_iam_user" "demo_user" {
  name = "${var.project_name}-demo-user"
  path = "/governance/"

  tags = {
    Environment = "demo"
    Purpose     = "governance-training"
  }
}

# Attach MFA delete policy to demo user
resource "aws_iam_user_policy_attachment" "demo_user_mfa" {
  user       = aws_iam_user.demo_user.name
  policy_arn = aws_iam_policy.mfa_delete_policy.arn
}

# ------------------------------------------------------------------------------
# 2. IAM Role for AWS Config Service
# ------------------------------------------------------------------------------

# IAM Role for AWS Config Service
resource "aws_iam_role" "config_role" {
  name = "${var.project_name}-config-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
      }
    ]
  })
}

# Attach managed policy to Config Role
resource "aws_iam_role_policy_attachment" "config_policy_attach" {
  role       = aws_iam_role.config_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWS_ConfigRole"
}

# Additional policy for Config to write to S3
resource "aws_iam_role_policy" "config_s3_policy" {
  name = "${var.project_name}-config-s3-policy"
  role = aws_iam_role.config_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetBucketVersioning",
          "s3:PutObject",
          "s3:GetObject"
        ]
        Resource = [
          aws_s3_bucket.config_bucket.arn,
          "${aws_s3_bucket.config_bucket.arn}/*"
        ]
      }
    ]
  })
}

In the first block, we will create a custom IAM policy that enforces MFA for deleting S3 objects, If MFA is absent, it will deny deletion of that resource.

In the second block, we will enforce encryption in transit for S3, so whenever you are trying to upload an object to S3 over http without secure, it will deny that request.

In the third block, It requires tagging for resource creation. When users tried to create any resource without tagging, it will fail. We have created this in such a way that if anyone tries to create a EC2 instance, it should have mandatory tags of Environment=["dev", "staging", "prod"] and have Owner tag too.

In the fourth block, It shows the creation of IAM User and it attaches the above MFA delete policy to demo user we have created.

In the fifth block, It creates an IAM Role for AWS Config Service and attaching a policy for that after which we create an additional policy too for writing config logs to S3 bucket.

config.tf:

The above block creates Config recorder + 6 compliance rules.

# ------------------------------------------------------------------------------
# AWS Config Recorder and Delivery Channel
# ------------------------------------------------------------------------------

# AWS Config Recorder
resource "aws_config_configuration_recorder" "main" {
  name     = "${var.project_name}-recorder"
  role_arn = aws_iam_role.config_role.arn

  recording_group {
    all_supported                 = true
    include_global_resource_types = true
  }
}

This records every configuration change in your AWS account.

# AWS Config Delivery Channel
resource "aws_config_delivery_channel" "main" {
  name           = "${var.project_name}-delivery-channel"
  s3_bucket_name = aws_s3_bucket.config_bucket.bucket
  depends_on     = [aws_config_configuration_recorder.main]
}

# Start the Config Recorder
resource "aws_config_configuration_recorder_status" "main" {
  name       = aws_config_configuration_recorder.main.name
  is_enabled = true
  depends_on = [aws_config_delivery_channel.main]
}

The above block creates a Config Delivery channel which records all the config related information directly to the S3 bucket and starting the config recorder.

# Config Rule: Ensure S3 buckets do not allow public write
resource "aws_config_config_rule" "s3_public_write_prohibited" {
  name = "s3-bucket-public-write-prohibited"

  source {
    owner             = "AWS"
    source_identifier = "S3_BUCKET_PUBLIC_WRITE_PROHIBITED"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

# Config Rule: Ensure S3 buckets have encryption enabled
resource "aws_config_config_rule" "s3_encryption" {
  name = "s3-bucket-server-side-encryption-enabled"

  source {
    owner             = "AWS"
    source_identifier = "S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

# Config Rule: Ensure S3 buckets block public access
resource "aws_config_config_rule" "s3_public_read_prohibited" {
  name = "s3-bucket-public-read-prohibited"

  source {
    owner             = "AWS"
    source_identifier = "S3_BUCKET_PUBLIC_READ_PROHIBITED"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

# Config Rule: Ensure EBS volumes are encrypted
resource "aws_config_config_rule" "ebs_encryption" {
  name = "encrypted-volumes"

  source {
    owner             = "AWS"
    source_identifier = "ENCRYPTED_VOLUMES"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

# Config Rule: Ensure EC2 instances have required tags
resource "aws_config_config_rule" "required_tags" {
  name = "required-tags"

  source {
    owner             = "AWS"
    source_identifier = "REQUIRED_TAGS"
  }

  input_parameters = jsonencode({
    tag1Key = "Environment"
    tag2Key = "Owner"
  })

  scope {
    compliance_resource_types = [
      "AWS::EC2::Instance",
      "AWS::S3::Bucket"
    ]
  }

  depends_on = [aws_config_configuration_recorder.main]
}

# Config Rule: Ensure root account has MFA enabled
resource "aws_config_config_rule" "root_mfa_enabled" {
  name = "root-account-mfa-enabled"

  source {
    owner             = "AWS"
    source_identifier = "ROOT_ACCOUNT_MFA_ENABLED"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

The above block contains all the config rules which we have created of the config to detect any complaint issues.

It contains mainly 6 complaince blocks:

Rule	Source Identifier	Purpose
S3 Public Write Prohibited	S3_BUCKET_PUBLIC_WRITE_PROHIBITED	Block public writes
S3 Encryption Enabled	S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED	Enforce SSE s
S3 Public Read Prohibited	S3_BUCKET_PUBLIC_READ_PROHIBITED	Block public reads
EBS Volumes Encrypted	VOLUME_EBS_ENCRYPTED	Encrypted volumes
Required Tags	Custom parameters	Tag validation
Root MFA Enabled	ROOT_ACCOUNT_MFA_ENABLED	Root account security

output.tf

It shows important information after deployment

output "config_rules" {
  value = [list of all rule names]
}
output "config_recorder_status" {
  value = true  # Recorder is running
}

Verification & Testing:

Now trigger the creation of all the resources using terraform commands.

terraform init
terraform plan
terraform apply

After terraform apply, we can see 23 resources created getting created.

IAM policies
S3 bucket with security settings
Config recorder
6 Config rules

✅ S3 Bucket Properties

✅ Versioning enabled - ✅ AES256 encryption - ✅ Public access blocked - ✅ AWS logs folder created

✅ AWS Config Dashboard:

We could see the resources with the status as complaint or non-complaint.

Compliant: 4 rules ✓
Non-Compliant: 1 resource (missing tags) ✗

Test Compliance Detection:

Created untagged S3 bucket → Detected as non-compliant after 2 minutes.

Check Config -> Wait 2–3 minutes, then:

Go to AWS Config → Rules
Click “s3-bucket-server-side-encryption-enabled”
See new bucket as NON-COMPLIANT (red)
Explain: Config detected the violation automatically!

Key Learnings

Policy vs Config Rules:

IAM Policy: Prevents actions before execution
Config Rules: Detects issues after creation

Terraform Best Practices:

Use random_string for unique names
Explicit dependencies between resources
Reference Terraform Registry for resource syntax

Security Tip:

Always use the Principle of Least Privilege. While we used Resource: "*" for this lab, in production, always restrict policies to specific ARNs!

Conclusion:

See the below video for more understanding about AWS IAM Policy and Governance Setup using Terraform.

Day 20: Terraform Custom Modules for EKS

Anil KUMAR — Wed, 17 Dec 2025 17:45:00 +0000

Welcome to the Day 20 of 30 days of AWS Terraform challenge initiative by Piyush Sachdeva. In this blog, we will understand about modules, what exactly are modules, when we will use them and what are the use cases of modules in Terraform real life use cases.

Modules:

Modules are nothing but a reusable piece of code in which you can encapsulate all the complexity of Terraform and can reuse them in all the possible situations.

In other words, Modules are self-contained packages of configuration code that you create yourself to group related resources together. Think of them as custom functions for your infrastructure: you define the logic once, and then call it multiple times with different parameters. In other programming languages, we have functions whereas in terraform, modules will replace them.

Use Cases of Modules

Abstraction - You want to hide 50 lines of complex networking code behind a simple 5-line block.

Reduced Complexity — A custom module lets you expose only few variables your team actually cares about.

Consistency — ensuring every EKS cluster in your company is built exactly the same way across Dev, Staging, and Prod.

Types of Modules:

Modules are further divided into 3 types based on their use cases and functionality.

Public Modules:

Public Modules are the modules which have been built and maintained by Cloud providers like AWS, Azure, Hashicorp, GCP. These modules will be developed by those cloud providers and will be available for anyone to use without any restrictions.

These are community-driven modules hosted on the Terraform Registry. They are publicly accessible and free for anyone to use.

Maintained by: Individual contributors, the open-source community, or smaller organizations.

Key Feature: Great for common tasks (e.g., setting up a basic S3 bucket or a generic VPC).

Risk: Quality and maintenance can vary; always check the download count and "stars" before using.

Partner Modules:

Partner modules are the modules which are maintained by both Hashicorp and the Partner, Here Hashicorp will also be maintaining that Modules along with the partner.

These are a subset of public modules but carry a "Verified" or "Partner" badge on the Terraform Registry.

Maintained by: Major technology companies (like AWS, Azure, Google Cloud, or HashiCorp itself) in partnership with HashiCorp.

Key Feature: These undergo a rigorous verification process to ensure they follow best practices and are actively maintained.

Use Case: Ideal for mission-critical infrastructure where stability and official support are required.

Custom Modules:

Custom modules are the modules which will be developed by the community or the Individuals.
We can customize these modules based on the requirements we have and can utilize them based on our test cases.

These are modules created internally by you or your organization to meet specific business needs or security standards.

Maintained by: Your internal DevOps or Platform teams.

Key Feature: They often wrap public/partner modules to "hard-code" organizational standards (e.g., always enabling encryption or specific tagging).

Source: Usually stored in a Private Module Registry (via HCP Terraform/Enterprise) or directly in a private Git repository.

Modules working flow Architecture:

In today blog, we will see how this Terraform configuration demonstrates custom module creation for EKS cluster deployment.

Generally for the creation of EKS Cluster, we need mainly VPC for networking, IAM for roles of worker nodes, eks for creating master and worker nodes and also ec2 for creating required instances.

Structure:

modules/
├── vpc/              # Custom VPC module
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── iam/              # Custom IAM roles module
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── eks/              # Custom EKS cluster module
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   └── templates/
│       └── userdata.sh
└── secrets-manager/  # Custom Secrets Manager module
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

Modules Overview:

1. VPC Module (modules/vpc/):

Creates networking infrastructure:

VPC with custom CIDR
Public subnets (3 AZs) with Internet Gateway
Private subnets (3 AZs) with NAT Gateway
Route tables and associations
EKS-required subnet tags

2. IAM Module (modules/iam/)

Creates IAM resources:

EKS cluster IAM role with policies
Node group IAM role with policies
OIDC provider for IRSA (IAM Roles for Service Accounts)

3. EKS Module (modules/eks/)

Creates EKS cluster resources:

EKS control plane with KMS encryption
CloudWatch log group
Security groups (cluster + nodes)
EKS addons (CoreDNS, kube-proxy, VPC CNI)
Managed node groups with launch templates
Customizable node group configurations

4. Secrets Manager Module (modules/secrets-manager/)

Creates secrets management resources:

KMS key for secrets encryption
Database credentials secret (optional)
API keys secret (optional)
Application config secret (optional)
IAM policy for reading secrets

The setup includes:

VPC: Custom VPC with public and private subnets across 3 availability zones

EKS Cluster: Managed Kubernetes cluster with latest version
Node Groups: General purpose node group (on-demand instances), Spot instance node group for cost optimization
Add-ons: CoreDNS, kube-proxy, VPC CNI, and EBS CSI driver
IRSA: IAM Roles for Service Accounts enabled for fine-grained permissions

Think of this structure as Parent and Child Modules.

Inside each module be it Parent module or child module of VPC, IAM or EKS, we will be having separate terraform files like main.tf, variables.tf, output.tf, providers.tf amd so on..

Variables.tf and outputs.tf plays a key role as they help in how the root and child module communicates with each other.

Also main.tf of root and child modules will also communicate with each other mainly with Call by Parameters.

Lets deep dive into the VPC module now:

Below is the main.tf code for root Module:

# Custom VPC Module
module "vpc" {
  source = "./modules/vpc"

  name_prefix     = var.cluster_name
  vpc_cidr        = var.vpc_cidr
  azs             = slice(data.aws_availability_zones.available.names, 0, 3)
  private_subnets = var.private_subnets
  public_subnets  = var.public_subnets

  enable_nat_gateway = true
  single_nat_gateway = true

  # Required tags for EKS
  public_subnet_tags = {
    "kubernetes.io/role/elb"                    = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb"           = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  }

  tags = {
    Environment = var.environment
    Terraform   = "true"
    Project     = "EKS-Day20"
  }
}

Instead of writing out the entire EKS resource, your Root configuration (where you run terraform apply) looks like this:

Here you can see we have not written the configuration for VPC instead pointed out the source block to the custom module we have written for VPC.

source = "./modules/vpc"

Lets take an example of a argument and see how the parameters are passed between Root and child modules.

  azs = slice(data.aws_availability_zones.available.names, 0, 3)

In the above block, you can see that we have taken the availability zones from a Data source instead of hardcoding and storing that in a variable named azs.

Now to reference the value of azs in the VPC module, we need to pass the variable azs inside the variables.tf of VPC module and then use it in the main.tf of VPC module, so that it understands what is the data type and the value it is holding from the main module.

variable "azs" {
  description = "List of availability zones"
  type        = list(string)
}

# VPC
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true
}

# Public Subnets
resource "aws_subnet" "public" {
  count                   = length(var.public_subnets)
  vpc_id                  = aws_vpc.main.id
  cidr_block              = var.public_subnets[count.index]
  availability_zone       = var.azs[count.index]
  map_public_ip_on_launch = true
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = length(var.private_subnets)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.private_subnets[count.index]
  availability_zone = var.azs[count.index]
}

So whatever the values we are passing in the variables.tf of root module must be passed to the variables.tf of child module too.

These custom modules doesn't talk with each other like VPC module doesn't communicate directly with EKS module or vice-versa instead they will communicate with main module and main module will communicate with all the child modules.

Also if we want to pass any value from child module to root module, we can output that vale in child module and then reference that value in main.tf of root module.

Also we can add dependency in modules:

# Custom EKS Module
module "eks" {
  source = "./modules/eks"

  cluster_name       = var.cluster_name
  kubernetes_version = var.kubernetes_version
  vpc_id             = module.vpc.vpc_id
  subnet_ids         = module.vpc.private_subnets

  cluster_role_arn = module.iam.cluster_role_arn
  node_role_arn    = module.iam.node_group_role_arn

  endpoint_public_access  = true
  endpoint_private_access = true
  public_access_cidrs     = ["0.0.0.0/0"]

  enable_irsa = true

In the above block, we can see we have mentioned about vpc_id and subnet_id in EKS module, so EKS module will only be created after the complete creation of VPC module.

You can also see the vpc_id and subnet_ids are referencing to the VPC module where they have been declared in the output.tf of VPC module.

output "vpc_id" {
  description = "The ID of the VPC"
  value       = aws_vpc.main.id
}

output "vpc_cidr_block" {
  description = "The CIDR block of the VPC"
  value       = aws_vpc.main.cidr_block
}

output "public_subnets" {
  description = "List of IDs of public subnets"
  value       = aws_subnet.public[*].id
}

output "private_subnets" {
  description = "List of IDs of private subnets"
  value       = aws_subnet.private[*].id
}

So the values in output.tf of VPC module was referenced in the root module main.tf using variables.tf and output.tf.

Hope you understand the flow.

Execution:

tf init
tf plan
tf apply

By executing the above commands you could see the infrastructure getting created, first VPC and IAM will be created parallely followed by EKS.

Conclusion:

Custom Terraform modules transform infrastructure from scripts into systems. For EKS in particular, modular design is the difference between a demo cluster and a production-grade platform.

This concludes Day 20 of custom modules for EKS Cluster. See you in the next blog.

Day 19: Terraform Provisioners

Anil KUMAR — Mon, 15 Dec 2025 17:37:06 +0000

Today marks the Day 19 of 30 days of AWS Terraform challenge initiative by Piyush Sachdeva. Today we will deep dive into the Terraform Provisoners concept, what exactly is a provisoners. What are the different types of provisioners and how provisioners will be helpful in writing clean and effective code along with critical world test cases.

Provisoner:

Think of a Provisoner like something that performs a task like executing a script, running a command or doing some operation.

Provisioners are Terraform's way to execute scripts or commands during resource creation or destruction. They enable you to perform actions that go beyond Terraform's declarative resource management.

Key Concepts:

Bootstrapping: Performing initial setup like installing software, configuring services, or preparing an instance for service.

File Transfer: Copying files or directories between the machine running Terraform and the newly created remote resource.

Post-Deployment Cleanup/Operations: Executing final commands or scripts after a resource is created or before it is destroyed.

Provisioners run during resource lifecycle events (creation or destruction)
They are a "last resort" - Terraform recommends using native cloud-init, user_data, or configuration management tools when possible
They execute only once during resource creation (not on updates)
Failure handling: By default, if a provisioner fails, the resource is marked as "tainted" and will be recreated on next apply

Types of Provisioners:

There are 3 types of provisioners available based on their use cases such as local-exec, remote-exec and file provisioner.

Local-exec:

Local-exec provisioners are used to run basically on the machine where the Terraform host machine is present. It doesn't require any connection like SSH or RDP for execution.

Use cases:

Trigger webhooks or API calls
Update local inventory files
Run local scripts for orchestration
Send notifications (Slack, email)
Register resources in external systems

Syntax:

provisioner "local-exec" {
  command = "echo ${self.public_ip} >> inventory.txt"
}

In the above block, we are just executing a simple echo command on the Terraform host machine.

Tip: Always remember that provisoners must be inside of the Terraform resource blocks.

Remote-exec:

We will use this remote-exec provisioner when we would like to perform some tasks on remote machines by using SSH connectivity.
Like you have created a EC2 Instance resource using terraform and want to install nginx or apache server on that instance on the go in the same code, then you can use this remote exec provisoner which will wait till the creation of EC2 instance and then executes the installation of nginx.

The connection we will be using for this is SSH for Linux and WinRM for Windows.

Use cases:

Install packages (nginx, docker, etc.)
Run initialization commands
Configure system settings
Start services or daemons
Quick bootstrap tasks

Syntax:

provisioner "remote-exec" {
  inline = [
    "sudo apt-get update",
    "sudo apt-get install -y nginx",
    "sudo systemctl start nginx"
  ]
}

We will be using an incline block while executing remote-exec provisoners.

file Provisioner:

We will use this file provisoner when we want to copy a file from one machine to another machine. For example in your local you have a file and you want to copy that file to the remote instance you have created with your Terraform code.

In this case, we can use the file provisioner and yes we will need the SSH conectivity for this provisioner just like remote-exec and also if you want to connect to an instance, then you need a key-pair too.

Use cases:

Copy configuration files
Deploy scripts for execution
Transfer SSL certificates
Upload application binaries

Syntax:

provisioner "file" {
  source      = "scripts/setup.sh"
  destination = "/tmp/setup.sh"
}

provisioner "remote-exec" {
  inline = [
    "chmod +x /tmp/setup.sh",
    "/tmp/setup.sh"
  ]
}

In the above block, you could see that we have used file provisioner to copy file from source to destination and then used remote-exec provisioner to setup required permissions for that file and execute that.

Connection Block:

For remote-exec and file provisioners, you need a connection block:

connection {
  type        = "ssh"              # or "winrm" for Windows
  user        = "ubuntu"           # SSH user
  private_key = file("~/.ssh/id_rsa")  # SSH private key
  host        = self.public_ip     # Target host
  timeout     = "5m"               # Connection timeout
}

Best Practices:

DO:

Use provisioners as a last resort
Prefer cloud-init, user_data, or AMI baking (Packer)
Keep provisioner scripts idempotent
Handle errors gracefully with on_failure parameter
Use connection timeouts to avoid hanging
Test thoroughly in non-production environments

DON'T:

Use provisioners when native Terraform resources exist
Rely on provisioners for critical configuration
Forget that provisioners only run on creation
Store sensitive data in provisioner commands
Use complex logic - move to proper config management tools

Code Execution:

main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical (Ubuntu official)

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

resource "aws_security_group" "ssh" {
  name        = "tf-prov-demo-ssh"
  description = "Allow SSH inbound"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "demo" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  tags = {
    Name = "terraform-provisioner-demo"
  }

}

In the above code, we have created a basic terraform code for creation of an Ec2 instance using AMI data source and with a security group allowing ports 22 and outbound traffic as all.

Local-exec:

resource "aws_instance" "demo" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  tags = {
    Name = "terraform-provisioner-demo"
  }
  provisioner "local-exec" {
    command = "echo 'Local-exec: created instance ${self.id} with IP ${self.public_ip}'"
 }
}

We have initiated a local-exec command which will execute a simple echo command with instance_id and instance_public_ip of the created EC2 instance.

remote-exec:

resource "aws_instance" "demo" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  tags = {
    Name = "terraform-provisioner-demo"
  }
  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "echo 'Hello from remote-exec' | sudo tee /tmp/remote_exec.txt",
    ]
  }
}

We have added a remote-exec provisoner of updating the packages in the remote machine of aws_instance.demo server and then echoing a simple line to the file "/tmp/remote_exec.txt".

After the resource is created, you can ssh into that instance using the key-pair and can check whether the file has been created in that location or not.

file provisoner:

resource "aws_instance" "demo" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  tags = {
    Name = "terraform-provisioner-demo"
  }
  provisioner "file" {
    source      = "${path.module}/scripts/welcome.sh"
    destination = "/tmp/welcome.sh"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo chmod +x /tmp/welcome.sh",
      "sudo /tmp/welcome.sh"
    ]
  }
}

Here we are using both remote-exec and file provisioners. The above script copies a script (scripts/welcome.sh) to the instance, then executes it. Good pattern for more complex bootstrapping when script files are preferred.

After execution of that script, you can ssh into that instance and check whether everything is working fine as per mentioned in the provisoners.

Also while doing hands-on exercises for the provisioners, make sure dont always follow terraform destroy followed by terraform apply which will be unnecessary and a waste of time.

Taint:

Instead use taint, when you mark a resource as taint, then Terraform will re-create that instance on the next terraform apply.

so just taint the aws_instance.demo and then do the terraform apply so that instance will be recreated with local-exec, remote-exec and file provisoners

terraform taint aws_instance.demo

Conclusion:

This marks the Day 19 of 30 days of AWS Terraform challenge. We have done a deep dive into the Terraform provisoners, what exactly are provisioners, different types of provisioners and what are the use cases for each of them and when we will use a specific provisioner.

Day 18: Image Processing Serverless Project using AWS Lambda

Anil KUMAR — Sun, 14 Dec 2025 17:41:59 +0000

Today marks the Day 18 of 30 Days of Terraform challenge by Piyush Sachdeva. In this Blog, we will deep dive into a project of Images Processing Serverless Project using AWS Lambda entirely using Terraform. We’ll walk through an end-to-end image processing project i.e. from uploading a file to S3, to automatically processing it using a Lambda function, all orchestrated through Terraform.

Before diving deep into the project, lets first understand what exactly is AWS Lambda and why it is used and what is the significance of that.

AWS Lambda:

At its core, AWS Lambda is a serverless function. What is a serverless, you would imagine that for any service to be up and running we would need a service. yeah that's true.

When you want to host any app or server, you need to set up a service and deploy that app on that server like EC2 Instance.

But What do you mean by Serverless, There are no servers at all, which isn’t really true. There are servers involved, but the difference is we don’t manage them.

With Lambda, we don’t provision servers at all. Instead of thinking about machines, we think about functions. We simply write our application code, package it, and upload it as a Lambda function.

AWS takes care of everything else.

The servers still exist, but AWS manages them for us. We don’t worry about operating systems, scaling, or uptime. Our responsibility ends with the code.

And here’s the key difference:
a Lambda function does not run all the time and runs only when something triggers it.

An event could be:

A file being uploaded to an S3 bucket
A scheduled time (for example, every Monday at 7 AM)
A real-time system event

Project Architecture:

┌─────────────────┐
│  Upload Image   │  You upload image via AWS CLI or SDK
│   to S3 Bucket  │
└────────┬────────┘
         │ s3:ObjectCreated:* event
         ↓
┌─────────────────┐
│ Lambda Function │  Automatically triggered
│ Image Processor │  - Compresses JPEG (quality 85)
└────────┬────────┘  - Low quality JPEG (quality 60)
         │            - WebP format
         │            - PNG format
         │            - Thumbnail (200x200)
         ↓
┌─────────────────┐
│ Processed S3    │  5 variants saved automatically
│    Bucket       │
└─────────────────┘

We’ll have two S3 buckets:

One bucket where we upload the original image

Another bucket where the processed images will be stored

The first bucket is our source bucket. Whenever we upload an image to this bucket, that upload creates an S3 event.

And remember what we discussed earlier, events are exactly what serverless functions like Lambda are waiting for.

So as soon as an image is uploaded, that S3 event will trigger our Lambda function.

This Lambda function is where all the image processing logic lives. It will take the original image and automatically generate:

A JPEG image with 85% quality

Another JPEG image with 60% quality

A WebP version

A PNG version

And a thumbnail image resized to 200 by 200

All of this happens without us clicking any extra buttons or running any manual commands.

Components:

Upload S3 Bucket: Source bucket for original images
Processed S3 Bucket: Destination bucket for processed variants
Lambda Function: Image processor with Pillow library
Lambda Layer: Pillow 10.4.0 for image manipulation
S3 Event Trigger: Automatically invokes Lambda on upload

Terraform Code:

Now we will go through the code for the project execution.

The first step is to clone the repository and move into the Day 18 directory.

The repository lives here

Once we clone it, navigate into the day-18 folder and then into the terraform directory. This is where all the Terraform files for today’s project live.

1. Making unique resource names:

resource "random_id" "suffix" {
  byte_length = 4
}

locals {
  bucket_prefix         = "${var.project_name}-${var.environment}"
  upload_bucket_name    = "${local.bucket_prefix}-upload-${random_id.suffix.hex}"
  processed_bucket_name = "${local.bucket_prefix}-processed-${random_id.suffix.hex}"
  lambda_function_name  = "${var.project_name}-${var.environment}-processor"
}

As we know the names of the S3 bucket should be unique, we need to name the S3 buckets carefully making sure those names do not exist earlier. For this we use a resource of random_id in Terraform which generates random characters and we append them before and after our bucket name.

We build a common bucket prefix using the project name and environment
We create two bucket names i.e. one for uploads and one for processed images
We append the random suffix so the names stay unique
We also define a clear name for our Lambda function

2. Creating the Source S3 Bucket

This project begins with an S3 bucket that acts as the source bucket. We will be uploading image which will start the entire image processing workflow.

# S3 Bucket for uploading original images (SOURCE)
resource "aws_s3_bucket" "upload_bucket" {
  bucket = local.upload_bucket_name
}

We’re simply creating an S3 bucket and giving it the name we already prepared using locals.

3. Enabling Versioning:

resource "aws_s3_bucket_versioning" "upload_bucket" {
  bucket = aws_s3_bucket.upload_bucket.id

  versioning_configuration {
    status = "Enabled"
  }
}

Versioning helps us keep track of changes. If the same file name is uploaded again, S3 doesn’t overwrite the old object, it stores a new version instead. Eventhough it is not needed in this project, we will keep it as it is best industry standard.

4. Enabling Server-Side Encryption:

Next, we enable server-side encryption.

resource "aws_s3_bucket_server_side_encryption_configuration" "upload_bucket" {
  bucket = aws_s3_bucket.upload_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

Here we are enabling the server_side_encryption on the bucket so the files in the bucket will be encrypted. This ensures that any image uploaded to the bucket is encrypted at rest using AES-256. We don’t need to manage encryption keys manually, AWS takes care of that for us.

5. Making Bucket Private:

We are making the source bucket private so that no one else will be accessing this source bucket.

resource "aws_s3_bucket_public_access_block" "upload_bucket" {
  bucket = aws_s3_bucket.upload_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

By blocking public ACLs and policies, we make sure the bucket isn’t accidentally exposed. In real production systems, public access is usually handled through controlled layers in front of S3, not directly on the bucket itself.

6. Creating Destination S3 Bucket:

Now we will be doing the same above steps for Destination Bucket too.

resource "aws_s3_bucket" "processed_bucket" {
  bucket = local.processed_bucket_name
}

resource "aws_s3_bucket_versioning" "processed_bucket" {
  bucket = aws_s3_bucket.processed_bucket.id

  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "processed_bucket" {
  bucket = aws_s3_bucket.processed_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_public_access_block" "processed_bucket" {
  bucket = aws_s3_bucket.processed_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

We have done creating the bucket, enabling versioning on that, enabling server-side encryption and making that bucket access private.

IAM Roles and Policies:

This is an important section and we need to be very careful about what access does a Lambda role needs for this project.

Instead of hard-coding permissions or credentials, AWS uses roles to define what a service is allowed to do. In our case, we want the Lambda function to:

Write logs to cloudwatch so we can see what’s happening
Read images from the source bucket
Write processed images to the destination bucket

1. Creating the IAM Role for Lambda:

We start by creating an IAM role that Lambda can assume.

resource "aws_iam_role" "lambda_role" {
  name = "${local.lambda_function_name}-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

This role doesn’t give any permissions yet. It simply says:
This role can be assumed by AWS Lambda.
AWS even provides a policy generator to help create these documents, which makes life easier when you’re starting out.

2. Defining the Permissions with an IAM Policy:

Next, we create a policy that tells AWS exactly what this Lambda function is allowed to do.

resource "aws_iam_role_policy" "lambda_policy" {
  name = "${local.lambda_function_name}-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ]
        Resource = "arn:aws:logs:${var.aws_region}:*:*"
      },
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:GetObjectVersion"
        ]
        Resource = "${aws_s3_bucket.upload_bucket.arn}/*"
      },
      {
        Effect = "Allow"
        Action = [
          "s3:PutObject",
          "s3:PutObjectAcl"
        ]
        Resource = "${aws_s3_bucket.processed_bucket.arn}/*"
      }
    ]
  })
}

There are 3 blocks in the above IAM JSON Policy:

The first block allows the Lambda function to create log groups, log streams, and write logs. Without this, we’d have no visibility into what the function is doing, especially if something goes wrong.

The second block allows Lambda to read objects from the source bucket. This is how it gets access to the uploaded image.

The third block allows Lambda to write objects to the destination bucket. This is where all the processed images will be stored.

We can also give S3 full access but it is not recommended for Best practices.

With this IAM role and policy in place, our Lambda function will be able to:

Read images from S3 bucket
Process them using Pillow Libraries
Store the results to Destination Bucket
Write logs to Cloudwatch to inspect

3. LAMBDA LAYER (Pillow):

A Lambda layer is a way to package external libraries and dependencies separately from our function code. Instead of bundling everything inside the function zip, we place shared or heavy dependencies into a layer and then attach that layer to the Lambda function.

This keeps the function code clean and makes dependencies easier to manage.

resource "aws_lambda_layer_version" "pillow_layer" {
  filename            = "${path.module}/pillow_layer.zip"
  layer_name          = "${var.project_name}-pillow-layer"
  compatible_runtimes = ["python3.12"]
  description         = "Pillow library for image processing"
}

Here’s what’s happening:

filename points to a zip file that contains the Pillow library
layer_name gives the layer a clear, readable name
compatible_runtimes ensures this layer works with Python 3.12

How do we create the pillow_layer.zip file in the first place?

Because AWS Lambda runs on Linux, the dependencies inside the layer must also be built for a Linux environment. This is important, especially if you’re working on macOS or Windows.

To solve this, we use Docker.

4. LAMBDA FUNCTION (Image Processor):

Our Lambda function is written in Python and lives inside the repository. To package it correctly, we use a Terraform data source.

# Data source for Lambda function zip
data "archive_file" "lambda_zip" {
  type        = "zip"
  source_file = "${path.module}/../lambda/lambda_function.py"
  output_path = "${path.module}/lambda_function.zip"
}

This data source takes the Python file, compresses it into a zip archive, and makes it ready for deployment.

Even though we’re working with a local file here, Terraform treats this as data it needs to reference during deployment which is exactly what data sources are designed for.

5. Defining the Lambda Function:

Now we define the Lambda function itself.

resource "aws_lambda_function" "image_processor" {
  filename         = data.archive_file.lambda_zip.output_path
  function_name    = local.lambda_function_name
  role             = aws_iam_role.lambda_role.arn
  handler          = "lambda_function.lambda_handler"
  source_code_hash = data.archive_file.lambda_zip.output_base64sha256
  runtime          = "python3.12"
  timeout          = 60
  memory_size      = 1024

  layers = [aws_lambda_layer_version.pillow_layer.arn]

  environment {
    variables = {
      PROCESSED_BUCKET = aws_s3_bucket.processed_bucket.id
      LOG_LEVEL        = "INFO"
    }
  }
}

In the above block:

filename points to the zip file created earlier
function_name gives the Lambda function a clear identity
role attaches the IAM role we created, allowing the function to access S3 and logs
handler tells Lambda where execution begins in the Python file
runtime specifies Python 3.12
timeout is set to 60 seconds, which is more than enough for image processing
memory_size is set to 1024 MB to give the function enough resources

6. CloudWatch Logs:

We will create a cloudwatch log group to make sure logs are retained in a predictable way.

resource "aws_cloudwatch_log_group" "lambda_processor" {
  name              = "/aws/lambda/${local.lambda_function_name}"
  retention_in_days = 7
}

7. S3 EVENT TRIGGER:

Now, we will give S3 permission to invoke our Lambda function.

# Lambda permission to be invoked by S3
resource "aws_lambda_permission" "allow_s3" {
  statement_id  = "AllowExecutionFromS3"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.image_processor.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.upload_bucket.arn
}

# S3 bucket notification to trigger Lambda
resource "aws_s3_bucket_notification" "upload_bucket_notification" {
  bucket = aws_s3_bucket.upload_bucket.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.image_processor.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.allow_s3]
}

Without the above permission, S3 events would never be able to trigger the function, even if everything else was configured correctly.

Whenever an object is created, in any way, invoke this Lambda function. As long as an object is created in the bucket, the event fires.

Deployment:

Now everything is set, now all that is left is to deploy them. We will be deploying this entire project using a shell script named deploy.sh

When we run this script, here’s what it does.

First, it performs a few basic checks. It makes sure:

AWS CLI is installed
Terraform is installed

If either of these is missing, the script stops and tells us exactly what’s wrong. This saves time and avoids confusion later.

Next, the script builds the Lambda layer.

This is an important step. Remember, the Pillow library needs to be compiled in a Linux environment to work correctly with AWS Lambda. Instead of doing this manually, the script calls another helper script that uses Docker to:

Spin up a Linux-based Python environment
Install Pillow in the correct directory structure
Package everything into a pillow_layer.zip file

Once that’s done, the script moves into the Terraform directory and runs the familiar commands:

terraform init
terraform plan
terraform apply

Terraform then takes over and creates every AWS resource we discussed:

Both S3 buckets
IAM roles and policies
Lambda layer
Lambda function
CloudWatch log group
S3 event trigger

When the deployment finishes, the script prints out something very useful information in the terraform output console.

Testing / Verification:

Setup is all done, Now all we need to do is to just upload an image to the upload bucket.

It can be any JPG or JPEG file. We can upload it using:

The AWS Console or AWS CLI

Any method that creates an object in the bucket

The moment the file is uploaded, the event is triggered.

Behind the scenes:

Lambda starts
The image is processed
Five new images are generated
All processed files appear in the destination bucket
If we open CloudWatch, we can also see the logs generated by the Lambda function, helpful for understanding what happened and for troubleshooting if something goes wrong.

Conclusion:

And with that, we’ve completed Day 18 of the 30 Days of Terraform Challenge.

We have gone a deep dive into a serverless project of Image processing using AWS Lambda involving S3 Buckets and CloudWatch.

Day 17: AWS Blue/Green with Terraform and Elastic Beanstalk

Anil KUMAR — Fri, 12 Dec 2025 12:09:53 +0000

As part of 30-Day AWS Terraform Challenge, Today is Day 17 which will be focusing on one of the most critical topics in modern infrastructure: achieving zero-downtime updates. This deep dive covered implementing a Blue-Green Deployment strategy on AWS using Terraform and Elastic Beanstalk.

What is Blue-Green Deployment:

Blue-Green Deployment is an infrastructure strategy designed to virtually eliminate downtime during application updates. It operates on a simple, powerful premise. It is also a release technique that keeps two identical production environments - nicknamed “blue” and “green.”

Blue Environment (The Live): This is your current, live production environment, actively receiving all user traffic (e.g., Application Version 1.0).

Green Environment (The Standby): This is an exact, identical copy of the Blue environment. We deploy the new application version (e.g., Application Version 2.0) here. Since it's not receiving live traffic, we can test it thoroughly without impacting the live users.

The real magic is the Swap Process:

Once the Green environment (v2.0) is fully tested and stable, we simply swap the DNS pointer that routes traffic from the Blue environment to the Green environment.

The Green environment immediately becomes the new live "Blue" environment (v2.0).

The old Blue environment (v1.0) becomes the standby "Green," ready for instant rollback or eventual decommissioning.

Key Benefits:

Zero/Minimal Downtime: The only "downtime" is the minimal DNS propagation time.
Safe Testing: Changes are tested on a non-live environment.
Instant Rollback: If v2.0 has an issue, you can instantly revert by swapping the DNS back to the original v1.0 environment.

This also have disadvantages as we are maintaining 2 live servers, there will be a lot of costs involved.

In this project, we will be packaging the application -> deploying the application -> uploading the zip file to S3 and then deploying that file to multiple environments using Blue/Green Environments.

Terraform Implementation:

make sure you clone the github_repo to your local and go to the folder: lessons/day-17

main.tf: Defines the AWS Provider, IAM roles (EC2 profile, Service role), and the private S3 bucket to store the application zips.

blue-environment.tf: Defines the initial production environment (Blue) using app-v1.zip.

green-environment.tf: Defines the initial production environment (Blue) using app-v1.zip.

Blue Environment Setup

We set up the Blue environment with version 1.0 in AWS Elastic Beanstalk.

1. Uploads the app code to S3:

resource "aws_s3_object" "app_v1" {
  bucket = aws_s3_bucket.app_versions.id
  key    = "app-v1.zip"
  source = "${path.module}/app-v1/app-v1.zip"
  etag   = filemd5("${path.module}/app-v1/app-v1.zip")

  tags = var.tags
}

Takes app-v1.zip from your local folder or from repo folder.
Uploads it to an S3 bucket (named app_versions), make sure bucket is created first.
Calculates MD5 hash to detect changes
1. Creates an Elastic Beanstalk app version

resource "aws_elastic_beanstalk_application_version" "v1" {
  name        = "${var.app_name}-v1"
  application = aws_elastic_beanstalk_application.app.name
  description = "Application Version 1.0 - Initial Release"
  bucket      = aws_s3_bucket.app_versions.id
  key         = aws_s3_object.app_v1.id

  tags = var.tags
}

Registers app-v1.zip as version your-app-name-v1
Links it to your main EB application
Stores it in S3 with the version label
1. Launches the production environment (Blue)

resource "aws_elastic_beanstalk_environment" "blue" {
  name                = "${var.app_name}-blue"
  application         = aws_elastic_beanstalk_application.app.name
  solution_stack_name = var.solution_stack_name
  tier                = "WebServer"
  version_label       = aws_elastic_beanstalk_application_version.v1.name

  # IAM Settings
  setting {
    ....
}

Creates an environment called your-app-name-blue
Deploys version 1.0 to it
Uses your specified platform (solution_stack_name)
Sets it as a WebServer tier (production-ready)

In simple terms: It bundles the app → uploads to S3 → registers as EB version 1.0 → deploys to production (blue environment).

Green Environment Setup:

We will use the same above setup for green environment too with just change in names of the environment from blue to green.

# Application Version 2.0 (Green Environment - Staging)
resource "aws_s3_object" "app_v2" {
  bucket = aws_s3_bucket.app_versions.id
  key    = "app-v2.zip"
  source = "${path.module}/app-v2/app-v2.zip"
  etag   = filemd5("${path.module}/app-v2/app-v2.zip")

  tags = var.tags
}

resource "aws_elastic_beanstalk_application_version" "v2" {
  name        = "${var.app_name}-v2"
  application = aws_elastic_beanstalk_application.app.name
  description = "Application Version 2.0 - New Feature Release"
  bucket      = aws_s3_bucket.app_versions.id
  key         = aws_s3_object.app_v2.id

  tags = var.tags
}

# Green Environment (Staging/Pre-production)
resource "aws_elastic_beanstalk_environment" "green" {
  name                = "${var.app_name}-green"
  application         = aws_elastic_beanstalk_application.app.name
  solution_stack_name = var.solution_stack_name
  tier                = "WebServer"
  version_label       = aws_elastic_beanstalk_application_version.v2.name

  # IAM Settings
  ...
}

EC2 Instance Role Creation:

We will be creating EC2 Instance Roles for our app servers.

# IAM Role for Elastic Beanstalk EC2 instances
resource "aws_iam_role" "eb_ec2_role" {
  name = "${var.app_name}-eb-ec2-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })

  tags = var.tags
}

Creates an IAM role ${your-app}-eb-ec2-role
Allows EC2 instances to "assume" this role
Gives your running app permissions to access AWS services (databases, S3, etc.), Currently we gave only permissions to access EC2 service for this project.

Elastic Beanstalk Service Role:

We will also create Elastic Beanstalk Service Role for AWS control plane.

# IAM Role for Elastic Beanstalk Service
resource "aws_iam_role" "eb_service_role" {
  name = "${var.app_name}-eb-service-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "elasticbeanstalk.amazonaws.com"
        }
      }
    ]
  })

  tags = var.tags
}

Creates an IAM role ${your-app}-eb-service-role
Allows the Elastic Beanstalk service itself to manage your environments
AWS needs this to deploy, scale, and monitor your app and all required permissions of Elastic BeanStalk.

Locks down S3 bucket security:

# Block public access to S3 bucket
resource "aws_s3_bucket_public_access_block" "app_versions" {
  bucket = aws_s3_bucket.app_versions.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

Blocks all public access to your app_versions S3 bucket as we will be making the bucket private without having access to public Internet.
Prevents accidental public exposure of app-v1.zip as it shoudld not delete accidentally.
All 4 public access settings = maximum security

Final Deployment:

sudo chmod 755 package-apps.sh
./package-apps.sh 
terraform init
terraform plan
terraform apply

Once the deployment was complete, Terraform will provide us the URLs for both environments as outputs:

Blue URL (Production): Showed "Welcome to blue-green demo v1.0"

Green URL (Staging): Showed "New features in v2.0"

The next step was the actual, zero-downtime Blue-Green swap.
You could also get the URL in AWS console of Elastic BeanStack.

Doing SWAP:

While the process can be scripted using the AWS CLI or advanced Terraform modules, the simplest way is via the AWS Console:

Navigate to the Elastic Beanstalk service.
Select the Actions dropdown for the environment (e.g., the Blue one).
Click "Swap Environment Domains."
Select the Green environment as the target for the swap.
Similarly go to the Green service of Elastic BeanStalk and do the swap selecting the Blue Environment.

The DNS swap took effect quickly. After the propagation time:

The Blue app URL now showed the new content: "New features in v2.0" (It became the new production site).

The Green app URL now showed the old content: "Welcome to blue-green demo v1.0" (It became the standby environment).

Conclusion:

Today blog will be a bit complex compared to the previous ones because we have now moved from noob stage, We also need to have knowledge about Blue-Green Deployment and Elastic BeanStalk. We have understood the key-benefits of Blue-Green Deployment and how it will be preferred in the Production of all the companies utilizing key benefits of Zero-Downtime and Instant Rollback.

Day 16/30: Mastering AWS IAM User Management with Terraform

Anil KUMAR — Thu, 11 Dec 2025 13:43:32 +0000

Today marks the Day 16 of 30 Days of AWS Terraform challenge Initiative by Piyush Sachdeva. Today, we will be mastering a mini-project on AWS main services such as IAM: bulk-creating AWS IAM users from a CSV file, assigning them dynamically to groups based on their attributes (like department and job title), and enabling secure console login.

Architecture:

As we all know that we will use AWS IAM resource for creating Users and managing their access. For any large organizations, IAM is a key service as it helps in creating the users with adequate permissions and grouping them into the respective groups so that no authorized access into other services of AWS will take place.

In this demo, I will show how to manage AWS IAM users, groups, and group memberships using Terraform. It's an AWS equivalent of Azure AD user management, demonstrating Infrastructure as Code (IaC) best practices. This project will solidify your understanding of Terraform's powerful iteration constructs: for_each, for expressions, and conditional filtering.

What This Demo Does

Retrieves AWS Account Information - Gets your current AWS Account ID
Reads User Data from CSV - Loads user information from a CSV file
Creates IAM Users - Automatically creates IAM users with proper naming conventions
Sets Up Login Profiles - Configures console access with password reset requirement
Creates IAM Groups - Sets up organizational groups (Education, Managers, Engineers)
Manages Group Memberships - Automatically assigns users to appropriate groups.

Mini Project Overview

Goal: Bulk create 26 IAM users (based on a users.csv).

Key Features:

Dynamic user assignment to groups (education, engineers, managers).

Enable console login with mandatory temporary passwords.

Utilize an S3 remote backend for state management.

Username Format: First initial + Last name (e.g., Michael Scott -> mscott).

What Gets Created:

26 IAM Users with console access
3 IAM Groups (Education, Managers, Engineers)
Group Memberships based on user attributes
User Tags with metadata (DisplayName, Department, JobTitle)

Project SetUP:

Below is the project structure for this mini-project:

day16/
├── backend.tf          # S3 backend configuration for state 
├── provider.tf         # AWS provider configuration
├── versions.tf         # Terraform version and required providers
├── main.tf            # Main user creation logic
├── groups.tf          # IAM groups and membership management
├── users.csv          # User data source

1. CSV Processing:

The csvdecode() function is the magic here. It takes the CSV file and transforms it into a list of maps, which is a perfect data structure for Terraform to iterate over.

In the CSV file we will have data coming in the format:
first_name,last_name,department,job_title Michael,Scott,Education,Regional Manager Dwight,Schrute,Sales,Assistant to the Regional Manager

So to make that into our usable format, We will convert that into a list of maps so that we can get whatever value based on that key.

locals {
  users = csvdecode(file("users.csv"))
}

Make sure you have the users.csv located in the same folder.

2. IAM User Creation:

This is the main block where we will write main code for creation of IAM users.

resource "aws_iam_user" "users" {
  for_each = { for user in local.users : user.first_name => user }

  name = lower("${substr(each.value.first_name, 0, 1)}${each.value.last_name}")
  path = "/users/"

  tags = {
    "DisplayName" = "${each.value.first_name} ${each.value.last_name}"
    "Department"  = each.value.department
    "JobTitle"    = each.value.job_title
  }
}

Users are created with a username format: {first_initial}{lastname} (e.g., mscott)

This is where the for_each pattern shines. We transform the list of maps from our CSV into a map of objects keyed by the desired username.

for_each Key: The key is dynamically generated using lower() and substr() to enforce our first initial+lastname format (e.g., mscott).
We have used 2 Terraform functions named lower and substr here:
Substr is used to cut short first_name into the first letter followed by last_name and then using the lower function to make the entire string lowercase.

Tags: We propagate the user attributes (department, job_title) into AWS tags. This is critical for the next step: dynamic group assignment.

3. Console Login Profiles:

We need to provide console access and enforce a password change on first login for security.

resource "aws_iam_user_login_profile" "users" {
  for_each = aws_iam_user.users

  user                    = each.value.name
  password_reset_required = true

  lifecycle {
    ignore_changes = [password_reset_required, password_length]
  }
}

Again we will use for_each block to iterate all the users from the aws_iam_users block.

We have used lifecycle meta argument here because it Prevents Terraform from destroying/recreating the resource every time the user changes their password.

4: Create Groups and Memberships:

This is the most advanced section, showcasing conditional logic to manage group membership.

Create IAM Groups:

resource "aws_iam_group" "education" {
  name = "Education"
  path = "/groups/"
}

resource "aws_iam_group_membership" "education_members" {
  name  = "education-group-membership"
  group = aws_iam_group.education.name

  users = [
    for user in aws_iam_user.users : user.name 
    if user.tags.Department == "Education"
  ]
}

Like the similar above block for education, we can create separate blocks for engineering, manager.

In the above block, we could see we have created aws_iam_group block to create groups with the name and path.

In aws_iam_group_membership, we will associate the users with the required groups based on the Department value. We have used a for expression with an if clause to filter the list of all created users, assigning them to a specific group based on their tags.

Manager Group Logic (Advanced):

For the managers group, you might need more complex logic and error handling:

resource "aws_iam_group_membership" "managers_members" {
  name  = "managers-group-membership"
  group = aws_iam_group.managers.name

  users = [
    for user in aws_iam_user.users : user.name if contains(keys(user.tags), "JobTitle") && can(regex("Manager|CEO", user.tags.JobTitle))
  ]
}

The 'can()' function is excellent for safely checking if an attribute exists for Members include those with job_title "manager" OR "CEO".
In Simple, it searches for key named "JobTitle" and it tries to find the names like Manager or CEO inside the Job Title and assigns them to Manager group.

Execution Commands:

# Initialize terraform
terraform init

# Review the changes (check for 58 resources created)
terraform plan 

# Apply the configuration
terraform apply

With this configuration, you will provision 58 resources in total:

(26{Users} + 26{Login Profiles} + 3{Groups} + 3 {Group Memberships}$)

Plan: 58 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + account_id     = "716145636736"
  + user_names     = [
      + "Michael Scott",
      + "Dwight Schrute",
      + "Jim Halpert",
      + "Pam Beesly",
      + "Ryan Howard",
      + "Andy Bernard",
      + "Robert California",
      + "Stanley Hudson",
      + "Kevin Malone",
      + "Angela Martin",
      + "Oscar Martinez",
      + "Phyllis Vance",
      + "Toby Flenderson",
      + "Kelly Kapoor",
      + "Darryl Philbin",
      + "Creed Bratton",
      + "Meredith Palmer",
      + "Erin Hannon",
      + "Gabe Lewis",
      + "Jan Levinson",
      + "David Wallace",
      + "Holly Flax",
      + "Charles Miner",
      + "Jo Bennett",
      + "Clark Green",
      + "Pete Miller",
    ]
  + user_passwords = (sensitive value)

You know before terraform apply, only 2 IAM users were there.

terraform apply
.
.
.
Apply complete! Resources: 58 added, 0 changed, 0 destroyed.

Outputs:

account_id = "716145636736"
user_names = [
  "Michael Scott",
  "Dwight Schrute",
  "Jim Halpert",
  "Pam Beesly",
  "Ryan Howard",
  "Andy Bernard",
  "Robert California",
  "Stanley Hudson",
  "Kevin Malone",
  "Angela Martin",
  "Oscar Martinez",
  "Phyllis Vance",
  "Toby Flenderson",
  "Kelly Kapoor",
  "Darryl Philbin",
  "Creed Bratton",
  "Meredith Palmer",
  "Erin Hannon",
  "Gabe Lewis",
  "Jan Levinson",
  "David Wallace",
  "Holly Flax",
  "Charles Miner",
  "Jo Bennett",
  "Clark Green",
  "Pete Miller",
]
user_passwords = <sensitive>

Now you could see that the 26 new users have been created as per the terraform code.

Summary:

Ingesting external data using csvdecode().
Dynamic resource creation using for_each.
Enabling secure console access for users.
Using tags to propagate data for later use.
Creating dynamic group membership lists using for/if expressions.
Configuring a resilient resource using lifecycle ignore_changes.

Don't forgot to destroy all the resources you have created using the terraform destroy command.

Conclusion:

AWS IAM User Management with Terraform offers a robust and scalable solution for handling access control within your AWS environment. By embracing Infrastructure as Code (IaC) principles, organizations can move beyond manual configurations and achieve a more secure, efficient, and auditable management of user identities and permissions.
This concludes the Day 16 of 30 Days of AWS Terraform challeneg. See you in the next blog.

Below is the Youtube Video for Reference:

Day15: Mini-Project: 2 VPC-Peering

Anil KUMAR — Wed, 10 Dec 2025 11:57:20 +0000

Today marks the Day 15 of 30days of aws terraform challenge initiative by Piyush Sachdev. Today we will deep into the concept of VPC Peering. What exactly is VPC-Peering, what are its use-cases and why we need that in the first place.

VPC-Peering:

As we all know VPC is nothing but a Virtual Private Cloud where we will be hosting our apps or services within the assigned IP addresses of that VPC. Suppose you have 2 VPC's in 2 different regions and you want to have communication between them.

Like app service inside a EC2 instance of one region VPC needs to communicate with DB service of another region VPC. Then you need to establish VPC-Peering for communication to happen, else there will be no communication happening between them.

Simply, In the world of cloud infrastructure, network isolation is the default. But what happens when your App service in us-east-1 needs to fetch data from a database in us-west-2

Routing this traffic over the public internet is slow, insecure, and expensive (NAT Gateway costs add up!). The solution is VPC Peering - a networking connection that allows two VPCs to communicate as if they are in the same network.

Low-Level Architecture (LLD):

Before we write code, let's visualize the packet flow. Below is the architecture we are building. Note how traffic flows privately through the AWS backbone, bypassing the public internet entirely for inter-VPC communication.

The Traffic Flow: EC2 Instance (East) sends a packet to 10.1.x.x (West).
Route Table (East): sees the destination is the Peering Connection (pcx-id), not the Internet Gateway.
AWS Backbone routes traffic securely across regions.
Security Group (West) validates the inbound request (is it from 10.0.0.0/16?).
EC2 Instance (West) processes the request.

Below are the resources that will be created during this project:

Networking Components

Two VPCs:

Primary VPC in us-east-1 (10.0.0.0/16)
Secondary VPC in us-west-2 (10.1.0.0/16)

Subnets:

One public subnet in each VPC
Configured with auto-assign public IP

Internet Gateways:

One for each VPC to allow internet access

Route Tables:

Custom route tables with routes to internet and peered VPC
Routes for VPC peering traffic.

VPC Peering Connection:

Cross-region peering between the two VPCs
Automatic acceptance configured

Compute Resources

EC2 Instances:

One t2.micro instance in each VPC
Running Amazon Linux 2
Apache web server installed
Custom web page showing VPC information

Security Groups:

SSH access from anywhere (port 22)
ICMP (ping) allowed from peered VPC
All TCP traffic allowed between VPCs

The Terraform Implementation(Step-by-Step):

Creating 2 new key-pairs for both Ec2 Instances which will be created before:

# For us-east-1
aws ec2 create-key-pair --key-name vpc-peering-demo --region us-east-1 --query 'KeyMaterial' --output text > vpc-peering-demo.pem

# For us-west-2
aws ec2 create-key-pair --key-name vpc-peering-demo --region us-west-2 --query 'KeyMaterial' --output text > vpc-peering-demo.pem

# Set permissions (on Linux/Mac)
chmod 400 vpc-peering-demo.pem

provider.tf:

provider "aws" {
  region = "us-east-1"
  alias  = "primary"
}

provider "aws" {
  region = "us-west-2"
  alias  = "secondary"
}

Creating 2 VPC's:

# Primary VPC in us-east-1
resource "aws_vpc" "primary_vpc" {
  provider             = aws.primary
  cidr_block           = var.primary_vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "Primary-VPC-${var.primary_region}"
    Environment = "Demo"
    Purpose     = "VPC-Peering-Demo"
  }
}

# Secondary VPC in us-west-2
resource "aws_vpc" "secondary_vpc" {
  provider             = aws.secondary
  cidr_block           = var.secondary_vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "Secondary-VPC-${var.secondary_region}"
    Environment = "Demo"
    Purpose     = "VPC-Peering-Demo"
  }
}

The above block creates 2 new VPC's in 2 different regions with different CIDR's block.

Creating 2 Subnets:

# Subnet in Primary VPC
resource "aws_subnet" "primary_subnet" {
  provider                = aws.primary
  vpc_id                  = aws_vpc.primary_vpc.id
  cidr_block              = var.primary_subnet_cidr
  availability_zone       = data.aws_availability_zones.primary.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name        = "Primary-Subnet-${var.primary_region}"
    Environment = "Demo"
  }
}

# Subnet in Secondary VPC
resource "aws_subnet" "secondary_subnet" {
  provider                = aws.secondary
  vpc_id                  = aws_vpc.secondary_vpc.id
  cidr_block              = var.secondary_subnet_cidr
  availability_zone       = data.aws_availability_zones.secondary.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name        = "Secondary-Subnet-${var.secondary_region}"
    Environment = "Demo"
  }
}

The above block creates 2 subnets in 2 different regions within the VPC CIDR block.

Creating Internet Gateways:

# Internet Gateway for Primary VPC
resource "aws_internet_gateway" "primary_igw" {
  provider = aws.primary
  vpc_id   = aws_vpc.primary_vpc.id

  tags = {
    Name        = "Primary-IGW"
    Environment = "Demo"
  }
}

# Internet Gateway for Secondary VPC
resource "aws_internet_gateway" "secondary_igw" {
  provider = aws.secondary
  vpc_id   = aws_vpc.secondary_vpc.id

  tags = {
    Name        = "Secondary-IGW"
    Environment = "Demo"
  }
}

The above block creates 2 Internet Gateways associating with the already existing VPC in 2 regions.

Creating Route Tables:

# Route table for Primary VPC
resource "aws_route_table" "primary_rt" {
  provider = aws.primary
  vpc_id   = aws_vpc.primary_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.primary_igw.id
  }

  tags = {
    Name        = "Primary-Route-Table"
    Environment = "Demo"
  }
}

# Route table for Secondary VPC
resource "aws_route_table" "secondary_rt" {
  provider = aws.secondary
  vpc_id   = aws_vpc.secondary_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.secondary_igw.id
  }

  tags = {
    Name        = "Secondary-Route-Table"
    Environment = "Demo"
  }
}

The above block demonstrates aws_route_table for creating Route tables and creating a route for internet gateway.

Creating Route_table_association

# Associate route table with Primary subnet
resource "aws_route_table_association" "primary_rta" {
  provider       = aws.primary
  subnet_id      = aws_subnet.primary_subnet.id
  route_table_id = aws_route_table.primary_rt.id
}

# Associate route table with Secondary subnet
resource "aws_route_table_association" "secondary_rta" {
  provider       = aws.secondary
  subnet_id      = aws_subnet.secondary_subnet.id
  route_table_id = aws_route_table.secondary_rt.id
}

In this block, we are associating route tables with the subnets.

VPC Peering Connection:

# VPC Peering Connection (Requester side - Primary VPC)
resource "aws_vpc_peering_connection" "primary_to_secondary" {
  provider    = aws.primary
  vpc_id      = aws_vpc.primary_vpc.id
  peer_vpc_id = aws_vpc.secondary_vpc.id
  peer_region = var.secondary_region
  auto_accept = false

  tags = {
    Name        = "Primary-to-Secondary-Peering"
    Environment = "Demo"
    Side        = "Requester"
  }
}

# VPC Peering Connection Accepter (Accepter side - Secondary VPC)
resource "aws_vpc_peering_connection_accepter" "secondary_accepter" {
  provider                  = aws.secondary
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id
  auto_accept               = true

  tags = {
    Name        = "Secondary-Peering-Accepter"
    Environment = "Demo"
    Side        = "Accepter"
  }
}

VPC Peering is a two-step process: Request and Accept.

The Requester (aws_vpc_peering_connection): Initiates the call from the Primary VPC. We must specify the peer_region because this is a cross-region peer.

The Accepter (aws_vpc_peering_connection_accepter): Lives in the Secondary region. It "picks up the phone" and establishes the tunnel.

Adding routes for VPC-Peering

# Add route to Secondary VPC in Primary route table
resource "aws_route" "primary_to_secondary" {
  provider                  = aws.primary
  route_table_id            = aws_route_table.primary_rt.id
  destination_cidr_block    = var.secondary_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id

  depends_on = [aws_vpc_peering_connection_accepter.secondary_accepter]
}

# Add route to Primary VPC in Secondary route table
resource "aws_route" "secondary_to_primary" {
  provider                  = aws.secondary
  route_table_id            = aws_route_table.secondary_rt.id
  destination_cidr_block    = var.primary_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id

  depends_on = [aws_vpc_peering_connection_accepter.secondary_accepter]
}

Creating the peering connection is not enough. You must tell the VPCs how to use it. We modify the Route Tables in both VPCs to point traffic destined for the other VPC's CIDR block to the peering connection.

Many engineers create the peering link but forget to update the Route Tables. Without these routes, packets will be dropped.

Now create terraform blocks for Security Group and EC2 Instances creation too. I was not including that here as it would make blog bigger. We will be using Data Sources block in EC2 instance creation for availability zones and AMI.

Deployment & Verification

Once all the files were run, run the below 3 commands for creation of Infra.

terraform init
terraform plan
terraform apply

After running terraform plan or apply, you could see 18 resources to be created.

Plan: 18 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + primary_instance_id           = (known after apply)
  + primary_instance_private_ip   = (known after apply)
  + primary_instance_public_ip    = (known after apply)
  + primary_vpc_cidr              = "10.0.0.0/16"
  + primary_vpc_id                = (known after apply)
  + secondary_instance_id         = (known after apply)
  + secondary_instance_private_ip = (known after apply)
  + secondary_instance_public_ip  = (known after apply)
  + secondary_vpc_cidr            = "10.1.0.0/16"
  + secondary_vpc_id              = (known after apply)
  + test_connectivity_command     = (known after apply)
  + vpc_peering_connection_id     = (known after apply)
  + vpc_peering_status            = (known after apply)

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

The below images shows the creation of VPC Peering Connection from Primary to Secondary.

You could see the creation of 2 EC2 instances created through Terraform.

ubuntu@ip-10-0-1-126:~$ ping 10.1.1.113
PING 10.1.1.113 (10.1.1.113) 56(84) bytes of data.
64 bytes from 10.1.1.113: icmp_seq=1 ttl=64 time=63.0 ms
64 bytes from 10.1.1.113: icmp_seq=2 ttl=64 time=62.8 ms
64 bytes from 10.1.1.113: icmp_seq=3 ttl=64 time=62.9 ms
64 bytes from 10.1.1.113: icmp_seq=4 ttl=64 time=62.8 ms
64 bytes from 10.1.1.113: icmp_seq=5 ttl=64 time=62.8 ms
^C
--- 10.1.1.113 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 62.752/62.830/63.021/0.101 ms
ubuntu@ip-10-0-1-126:~$ 
ubuntu@ip-10-0-1-126:~$ 
ubuntu@ip-10-0-1-126:~$ curl 10.1.1.113
<h1>Secondary VPC Instance - us-west-2</h1>
<p>Private IP: 10.1.1.113 </p

You could see that I have Pinged the secondary EC2 instance from Primary Ec2 instance and we are able to connect between them.

You could also see while trying to access Primary EC2 instance from Secondary one.

Make sure to terminate all the resources once the project is done.

terraform destroy --auto-approve

Conclusion

This marks the Day 15 of 30 Days of AWS Terraform challenge. Today we have done a mini-project on VPC Peering Connection by trying to establish connection between 2 VPC's in 2 different regions.

Day 14— AWS Terraform Static Website Hosting

Anil KUMAR — Tue, 09 Dec 2025 10:33:19 +0000

This blog will mark the Day 14 of 30 days of AWS Terraform Challenge Initiative by Piyush Sachdeva. In this blog, we will be doing a hands-on exercise of hosting a Static website on AWS S3 and accessing that through CloudFront Distribution.

This mini project demonstrates how to deploy a static website on AWS using Terraform. We'll create a complete static website hosting solution using S3 for storage and CloudFront for global content delivery.

Architecture:

Internet → CloudFront Distribution → S3 Bucket (Static Website)

Components:

S3 Bucket: Hosts static website files (HTML, CSS, JS)
CloudFront Distribution: Global CDN for fast content delivery
Public Access Configuration: Allows public reading of website files.

Workflow:

Prepare your static website files: Create your HTML, CSS, JavaScript, and image files.
Write Terraform configuration: Define the AWS resources (S3 bucket, CloudFront, Route 53) in .tf files.
Initialize Terraform: Run terraform init to download necessary providers.
Plan changes: Run terraform plan to see what changes will be applied.
Apply changes: Run terraform apply to provision the resources in AWS.
Upload content: If not using Terraform to upload objects, upload your static files to the S3 bucket.
Access your website: Access your website using the S3 static website endpoint or your custom domain if configured.

CloudFront with S3 — Why It is Needed and How It Works:

Hosting a static website directly on Amazon S3 is simple and cost-effective, but it has significant limitations when serving real users at scale.

Problems with S3-only Hosting:

High latency for global users:
S3 buckets exist in a single AWS region. Users located far from that region experience slower load times because every request must travel long physical distances to reach the S3 data center.
Higher data-transfer costs
When traffic grows, serving all content from one region increases inter-region data transfer costs. S3 also does not cache responses, which means every request hits the origin, increasing overall expenses.
Security challenges
Traditional S3 static hosting requires making the bucket public, exposing your content directly to the internet. This introduces risks such as:
Direct object access outside your website
Potential misuse of public URLs
Difficulty enforcing fine-grained access control

Amazon CloudFront, AWS’s global CDN, solves these issues by acting as a secure, fast, distributed caching layer in front of your S3 bucket.

Key Benefits

Global low-latency delivery
CloudFront uses edge locations worldwide to cache content close to users.
First request: Served from S3
Subsequent requests: Served from nearest CloudFront edge
This dramatically reduces page load time for users anywhere in the world.
Lower cost with caching
Since CloudFront serves most requests from its cache, fewer requests reach S3. This reduces S3 data-transfer and request costs. CloudFront’s global data transfer rates are also cheaper than S3’s regional egress charges.
Enhanced security with Origin Access Control (OAC)
Modern deployments use OAC, which includes:
Gives CloudFront permission to read from your private S3 bucket
Removes the need for ANY public bucket permissions
Ensures users can ONLY access S3 content through CloudFront

Complete Architecture Diagram

Key Components:

Private S3 Bucket with static files (HTML, CSS, JS, images).
Origin Access Control (OAC) - The modern, secure way to authorize CloudFront to access the private S3 bucket. This replaces the deprecated Origin Access Identity (OAI).
S3 Bucket Policy - Explicitly authorizes the CloudFront Distribution's service principal and restricts access using the OAC condition.
CloudFront Distribution - Caches content and serves it globally over HTTPS.

Terraform Code:

1. Setting the static files and variables.tf

mkdir www
# Add static files: index.html, style.css, script.js

variable "bucket_name" {
  default = "my-static-website-bucket-anilkumar"
}

locals {
  origin_id = "s3-static-site-origin"
}

2. Creating S3 with public access blocked

The bucket is defined as a standard S3 bucket, and then an access block resource is used to explicitly block all public access, ensuring it can only be accessed by CloudFront.

// first we create s3 resource 
resource "aws_s3_bucket" "my_first_bucket" {
  bucket = var.bucket_name
}

// here we make the bucket private
resource "aws_s3_bucket_public_access_block" "example" {
  bucket = aws_s3_bucket.my_first_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

3. Origin Access Control (OAC)

This resource creates the secure identity that CloudFront will use when communicating with S3.

// now we allow the origin to access the bucket
resource "aws_cloudfront_origin_access_control" "my_origin_access_control" {
  name                              = "my_origin_access_control"
  description                       = "OAC for S3 Static Site"
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

Purpose: Secure identity for CloudFront S3 communication, using the modern SigV4 signing protocol.

4. S3 Bucket Policy (Important):

This policy is the crucial authorization step. It only allows s3:GetObject (read-access for files) and only from the CloudFront service, specifically the ARN of our distribution using the AWS:SourceArn condition.

// bucket policy
resource "aws_s3_bucket_policy" "allow_access_from_cloudfront" {
  bucket = aws_s3_bucket.my_first_bucket.id

  depends_on = [aws_s3_bucket_public_access_block.example]

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "AllowCloudFrontS3Access"
        Effect = "Allow"

        # FIXED → Correct Principal for CloudFront Service
        Principal = {
          Service = "cloudfront.amazonaws.com"
        }

        Action = [
          "s3:GetObject",
        ]

        Resource = [
          "${aws_s3_bucket.my_first_bucket.arn}/*"
        ]

        # FIXED → Required OAC condition to ensure only THIS distribution can access
        Condition = {
          StringEquals = {
            "AWS:SourceArn" = aws_cloudfront_distribution.s3_distribution.arn
          }
        }
      }
    ]
  })
}

5. Uploading Static files:

This resource uses the fileset function to iterate over all files in the local www directory and uploads them to the S3 bucket with the correct MIME type (Content-Type) based on the file extension.

// s3 bucket object 
resource "aws_s3_object" "object" {

  bucket   = aws_s3_bucket.my_first_bucket.id
  for_each = fileset("${path.module}/www", "**/*")
  key      = each.value
  source   = "${path.module}/www/${each.value}"

  etag = filemd5("${path.module}/www/${each.value}")
  content_type = lookup({
    # Common MIME types. Simplified lookup logic using `regex` in the previous blog, but this is clear.
    "html"         = "text/html"
    "css"          = "text/css"
    "js"           = "application/javascript"
    "jpeg"         = "image/jpeg"
    "png"          = "image/png"
    "gif"          = "image/gif"
    "jpg"          = "image/jpg"
    # ... other types
  }, split(".", each.value)[length(split(".", each.value)) - 1], "application/octet-stream")
}

6. CloudFront Distribution

This is the main resource that ties everything together. It defines the caching rules, specifies the S3 bucket as the origin, and links the OAC for secure access.

resource "aws_cloudfront_distribution" "s3_distribution" {
  origin {
    # Use bucket_regional_domain_name for private S3 origin with OAC/OAI
    domain_name              = aws_s3_bucket.my_first_bucket.bucket_regional_domain_name
    origin_access_control_id = aws_cloudfront_origin_access_control.my_origin_access_control.id
    origin_id                = local.origin_id
  }

  enabled             = true
  is_ipv6_enabled     = true
  comment             = "CloudFront distribution for static site"
  default_root_object = "index.html" # Important for serving the index file

  default_cache_behavior {
    allowed_methods    = ["GET", "HEAD"]
    cached_methods     = ["GET", "HEAD"]
    target_origin_id   = local.origin_id

    forwarded_values {
      query_string = false

      cookies {
        forward = "none" # Static sites don't need cookies forwarded
      }
    }

    # IMPROVED: Changed from "allow-all" to "redirect-to-https" for security best practice
    viewer_protocol_policy = "redirect-to-https" 
    min_ttl                = 0
    default_ttl            = 3600 # 1 hour default cache
    max_ttl                = 86400 # 24 hours max cache
  }

  price_class = "PriceClass_100" # Lowest cost, basic global coverage

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  viewer_certificate {
    # Use AWS's default certificate for HTTPS
    cloudfront_default_certificate = true 
  }
}

7. Outputs Declaration:

output "website_url" {
  description = "The URL of the static website"
  value       = "https://${aws_cloudfront_distribution.s3_distribution.domain_name}"
}

output "cloudfront_distribution_id" {
  description = "The ID of the CloudFront distribution"
  value       = aws_cloudfront_distribution.s3_distribution.id
}

output "s3_bucket_name" {
  description = "The name of the S3 bucket"
  value       = aws_s3_bucket.website.bucket
}

8. Deploy & Test

terraform init
terraform plan
terraform apply --auto-approve

terraform apply 


Plan: 8 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloudfront_distribution_id = (known after apply)
  + s3_bucket_name             = (known after apply)
  + website_url                = (known after apply)
aws_cloudfront_origin_access_control.oac: Creating...
aws_s3_bucket.website: Creating...
aws_cloudfront_origin_access_control.oac: Creation complete after 2s [id=E1KGAVQXOCS06C]
aws_s3_bucket.website: Creation complete after 7s [id=my-static-website-anilkumar20251209102509049200000001]
aws_s3_bucket_public_access_block.website: Creating...
aws_s3_object.website_files["style.css"]: Creating...
aws_s3_object.website_files["script.js"]: Creating...
aws_s3_object.website_files["index.html"]: Creating...
aws_cloudfront_distribution.s3_distribution: Creating...
aws_s3_bucket_public_access_block.website: Creation complete after 1s [id=my-static-website-anilkumar20251209102509049200000001]
aws_s3_object.website_files["script.js"]: Creation complete after 1s [id=script.js]
aws_s3_object.website_files["style.css"]: Creation complete after 1s [id=style.css]
aws_s3_object.website_files["index.html"]: Creation complete after 1s [id=index.html]
aws_cloudfront_distribution.s3_distribution: Still creating... [10s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [20s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [30s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [40s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [50s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m0s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m10s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m20s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m30s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m40s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [1m50s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m0s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m10s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m20s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m30s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m40s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [2m50s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [3m0s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [3m10s elapsed]
aws_cloudfront_distribution.s3_distribution: Still creating... [3m20s elapsed]
aws_cloudfront_distribution.s3_distribution: Creation complete after 3m27s [id=E3PI31IGA77HJ]
aws_s3_bucket_policy.website: Creating...
aws_s3_bucket_policy.website: Creation complete after 5s [id=my-static-website-anilkumar20251209102509049200000001]

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.

Outputs:

cloudfront_distribution_id = "E3PI31IGA77HJ"
s3_bucket_name = "my-static-website-anilkumar20251209102509049200000001"
website_url = "https://d2u2vdb3lrnvju.cloudfront.net"

Here the outputs of S3 bucket creation and CloudFront Distribution creation

You could see the creation of S3 bucket with the bucket-prefix we set.

You could see the creation of CloudFront Distribution.

You could see that we are serving the static website through S3 bucket hosting.

Imp Tip: Always pin provider versions in your configurations to prevent unexpected changes in resource behavior.

Conclusion:

This concludes the Day 14 of 30 days of terraform challenge. Today we have done a mini project on Static Website hosting through AWS S3 and CloudFront Distribution. See you tomorrow in a new blog

Day 13: Terraform Data Sources

Anil KUMAR — Sun, 07 Dec 2025 17:34:09 +0000

Today marks the Day 3 of 30 Days of AWS Terraform Challenge Initiative by Piyush Sachdev. Today we will do deep dive into the Terraform Data Sources, what exactly is a Data Source and how it will help us in writing terraform code for AWS resources such as EC2, VPC, subnet and so..

Data Source:

Think of Data source like a phone directory with usename and phone number like key and value with an api, so whenever you want any value with the help of API, you can retrieve that using key instead of hardcoding.

In Short, Terraform data sources with AWS allow you to retrieve information about existing AWS resources or external data that can then be referenced within your Terraform configurations.

For Example, While Creating a EC2 instance, we need and AMI id for which we need to go the AMI release page and find the latest AMI ID and then fetch that which is not the best approach for our current projects, So we need to find a way to automatically get that AMI ID while creating EC2 instances without any manual intervention or hardcoding. So for this we will be using a resource in Terraform named "Data Source". Almost all resources in AWS have data Source and you can get their ID and all details for that resource using the Data Source Like ask for linux_2 OS based ami and it will fetch you that AMI ID.

In short, Data sources allow Terraform to read information about existing infrastructure. They:

Don't create, update, or delete resources
Allow you to reference resources managed elsewhere
Enable sharing infrastructure between teams
Are defined with data blocks instead of resource blocks

How to Use AWS Data Sources:

You define a data source using the data block in your Terraform configuration. The syntax is as follows:

data "provider_type" "name" {
  # Configuration settings for filtering or identifying the data source
}

provider_type: Specifies the type of AWS data source (e.g., aws_ami, aws_vpc, aws_s3_bucket).

name: A local name to reference this data source within your configuration or for internal communication.

Configuration Settings: These vary depending on the data source and are used to filter or identify the specific resource you want to retrieve information about. This often includes id, name, or filter blocks with name and values arguments.

Examples between Data vs. Resource Block:

# Resource Block - Terraform MANAGES this
resource "aws_vpc" "my_vpc" {
  cidr_block = "10.0.0.0/16"
}

# Data Block - Terraform READS this
data "aws_vpc" "existing_vpc" {
  filter {
    name   = "tag:Name"
    values = ["shared-network-vpc"]
  }
}

In the above code block, you can see some key differences between resources block and data sources block.

In the resource block, you can see that the Terraform entirely manages that VPC based on our CIDR block, whereas in Data source block, we are just referencing the existing vpc in our AWS account with the name "existing_vpc" and we will be using that while creating other resources.

In filters section, we have given some values telling Terraform to retreive data from the VPC which is matching with the filters provided.

Task:

In this task, we will first create a VPC using terraform and then we will try to use data source to create EC2 instance using ami data source to fetch ami_id first and then try to create EC2 in the VPC block we created initially.

Creation of VPC :

# This simulates infrastructure created by another team
provider "aws" {
  region = "us-east-1"
}

resource "aws_vpc" "shared" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "shared-network-vpc"  
  }
}

resource "aws_subnet" "shared" {
  vpc_id     = aws_vpc.shared.id
  cidr_block = "10.0.1.0/24"
  tags = {
    Name = "shared-primary-subnet"  # ← This tag is important!
  }
}

Initially, we didn't have any VPC with the CIDR block: 10.0.0.0/16, now we will create a VPC with the above CIDR with a tag: shared-network-vpc.

terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

Terraform will perform the following actions:

  # aws_subnet.shared will be created
  + resource "aws_subnet" "shared" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = (known after apply)
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.0.1.0/24"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = false
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + region                                         = "us-east-1"
      + tags                                           = {
          + "Name" = "shared-primary-subnet"
        }
      + tags_all                                       = {
          + "Name" = "shared-primary-subnet"
        }
      + vpc_id                                         = (known after apply)
    }

  # aws_vpc.shared will be created
  + resource "aws_vpc" "shared" {
      + arn                                  = (known after apply)
      + cidr_block                           = "10.0.0.0/16"
      + default_network_acl_id               = (known after apply)
      + default_route_table_id               = (known after apply)
      + default_security_group_id            = (known after apply)
      + dhcp_options_id                      = (known after apply)
      + enable_dns_hostnames                 = (known after apply)
      + enable_dns_support                   = true
      + enable_network_address_usage_metrics = (known after apply)
      + id                                   = (known after apply)
      + instance_tenancy                     = "default"
      + ipv6_association_id                  = (known after apply)
      + ipv6_cidr_block                      = (known after apply)
      + ipv6_cidr_block_network_border_group = (known after apply)
      + main_route_table_id                  = (known after apply)
      + owner_id                             = (known after apply)
      + region                               = "us-east-1"
      + tags                                 = {
          + "Name" = "shared-network-vpc"
        }
      + tags_all                             = {
          + "Name" = "shared-network-vpc"
        }
    }

Now we can see that a new VPC named/tagged "shared-network-vpc" is created.

Now we will be creating an EC2 instance, using the ami data source and vpc, subnet data source.

Below are the Data source blocks for VPC and Subnet resources:

# Data source to get the existing VPC
data "aws_vpc" "shared" {
  filter {
    name   = "tag:Name"
    values = ["shared-network-vpc"]
  }
}

# Data source to get the existing subnet
data "aws_subnet" "shared" {
  filter {
    name   = "tag:Name"
    values = ["shared-primary-subnet"]
  }
  vpc_id = data.aws_vpc.shared.id  # ← Using aws_vpc data source
}

data "aws_vpc" - reads VPC information from AWS
filter - searches for VPC with specific tag
shared - local name to reference this data source
Returns: VPC ID, CIDR block, and other attributes
Searches for subnet with specific tag
vpc_id - narrows search to our VPC

Now we will go through the Data source for AMI which helps us in fetching the AMI_ID for amazon_linux_2 OS.

# Data source for the latest Amazon Linux 2 AMI
data "aws_ami" "amazon_linux_2" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

most_recent = true - gets latest matching AMI
owners = ["amazon"] - only official Amazon AMIs
Multiple filters for precise matching with tag of names and values matching amzn2 ami values.
Wildcards (*) allow flexible pattern matching

Now in main.tf, we will create an EC2 instance utilizing all the above Data Sources of VPC, Subnet and AMI ID.

resource "aws_instance" "main" {
  ami           = data.aws_ami.amazon_linux_2.id    # AMI - Data source
  instance_type = "t2.micro"
  subnet_id     = data.aws_subnet.shared.id           # Subnet - Data source
  private_ip    = "10.0.1.50"

  tags = {
    Name = "day13-instance"
  }
}

data.aws_ami.amazon_linux_2.id - references AMI data source matching with Linux_2 OS.
data.aws_subnet.shared.id - references subnet data source
Instance will be created in existing infrastructure
Private IP must be within subnet's CIDR range

terraform plan
data.aws_vpc.shared: Reading...
data.aws_ami.amazon_linux_2: Reading...
aws_vpc.shared: Refreshing state... [id=vpc-09527ed20e76d002e]
data.aws_ami.amazon_linux_2: Read complete after 3s [id=ami-0156001f0548e90b1]
data.aws_vpc.shared: Read complete after 3s [id=vpc-09527ed20e76d002e]
data.aws_subnet.shared: Reading...
data.aws_subnet.shared: Read complete after 1s [id=subnet-0e8357d0d5a07c57b]
aws_subnet.shared: Refreshing state... [id=subnet-0e8357d0d5a07c57b]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.main will be created
  + resource "aws_instance" "main" {
      + ami                                  = "ami-0156001f0548e90b1"
      + arn                                  = (known after apply)
      + associate_public_ip_address          = (known after apply)
      + availability_zone                    = (known after apply)
      + disable_api_stop                     = (known after apply)
      + disable_api_termination              = (known after apply)
      + ebs_optimized                        = (known after apply)
      + enable_primary_ipv6                  = (known after apply)
      + force_destroy                        = false
      + get_password_data                    = false
      + host_id                              = (known after apply)
      + host_resource_group_arn              = (known after apply)
      + iam_instance_profile                 = (known after apply)
      + id                                   = (known after apply)
      + instance_initiated_shutdown_behavior = (known after apply)
      + instance_lifecycle                   = (known after apply)
      + instance_state                       = (known after apply)
      + instance_type                        = "t2.micro"
      + ipv6_address_count                   = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + key_name                             = (known after apply)
      + monitoring                           = (known after apply)
      + outpost_arn                          = (known after apply)
      + password_data                        = (known after apply)
      + placement_group                      = (known after apply)
      + placement_group_id                   = (known after apply)
      + placement_partition_number           = (known after apply)
      + primary_network_interface_id         = (known after apply)
      + private_dns                          = (known after apply)
      + private_ip                           = "10.0.1.50"
      + public_dns                           = (known after apply)
      + public_ip                            = (known after apply)
      + region                               = "us-east-1"
      + secondary_private_ips                = (known after apply)
      + security_groups                      = (known after apply)
      + source_dest_check                    = true
      + spot_instance_request_id             = (known after apply)
      + subnet_id                            = "subnet-0e8357d0d5a07c57b"
      + tags                                 = {
          + "Name" = "day13-instance"
        }
      + tags_all                             = {
          + "Name" = "day13-instance"
        }
      + tenancy                              = (known after apply)
      + user_data_base64                     = (known after apply)
      + user_data_replace_on_change          = false
      + vpc_security_group_ids               = (known after apply)

      + capacity_reservation_specification (known after apply)

      + cpu_options (known after apply)

      + ebs_block_device (known after apply)

      + enclave_options (known after apply)

      + ephemeral_block_device (known after apply)

      + instance_market_options (known after apply)

      + maintenance_options (known after apply)

      + metadata_options (known after apply)

      + network_interface (known after apply)

      + primary_network_interface (known after apply)

      + private_dns_name_options (known after apply)

      + root_block_device (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

In the above terraform plan execution, you could see it is first referencing data sources of ami, vpc and subnet and then it started creating EC2 instance.

After running terraform apply, we can see that the EC2 instance is created.

As everything has been created using Data sources of AMI, VPC and subnet, we can finally delete the resources using terraform destroy.

terraform destroy


Plan: 0 to add, 0 to change, 3 to destroy.
aws_subnet.shared: Destroying... [id=subnet-0e8357d0d5a07c57b]
aws_instance.main: Destroying... [id=i-057687d8f123d3e00]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 10s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 10s elapsed]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 20s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 20s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 30s elapsed]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 30s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 40s elapsed]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 40s elapsed]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 50s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 50s elapsed]
aws_instance.main: Still destroying... [id=i-057687d8f123d3e00, 1m0s elapsed]
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 1m0s elapsed]
aws_instance.main: Destruction complete after 1m10s
aws_subnet.shared: Still destroying... [id=subnet-0e8357d0d5a07c57b, 1m10s elapsed]
aws_subnet.shared: Destruction complete after 1m15s
aws_vpc.shared: Destroying... [id=vpc-09527ed20e76d002e]
aws_vpc.shared: Destruction complete after 1s

Conclusion:

This marks the conclusion of Day 13 of 30 days of Terraform Challenge by Piyush Sachdev and we have deep dived into Terraform Data Sources. We have understood what exactly is a Data source and how it helps us to create resources efficiently.

Below is the Youtube Video for reference:

Day 12 - Terraform Functions - Part 2

Anil KUMAR — Fri, 05 Dec 2025 18:03:56 +0000

Today marks the day 12 of 30 days of Terraform Challenge initiative by Piyush Sachdev. This blog is in continuation from previous blog of Day 11 - Terraform Functions of Part 1. In Part 1, we have discussed about String Functions, Numeric Functions, Collection Functions, Type Conversion, Date/Time Functions.

In this blog, we will discuss about remaining inbuilt Functions in terraform such as File Functions, Validation Functions and Lookup Functions.

Validation Functions:

In Terraform, you can define validation functions for variables to ensure that inputs confirm to your specific requirements. This is especially useful when you want to enforce constraints like valid string lengths, specific patterns, or predefined allowed values.

Validation blocks allow you to define strict rules for your variables. Instead of waiting for AWS to reject a resource creation (which takes time), Terraform can catch errors immediately during the plan phase.

Examples:

Example 1: Validating Instance Type

Let’s start by looking at a common validation scenario — checking the instance_type variable to ensure that only certain values are allowed. You might want to validate that the instance type name is between 2 and 10 characters and matches a specific pattern (e.g., t2 or t3 instances).

data "aws_ami" "validated_ami" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

resource "aws_instance" "validated_instance" {
  ami           = data.aws_ami.validated_ami.id
  instance_type = var.instance_type
  tags = {
    Name = "validated-instance"
    Type = var.instance_type
  }
}
variable "instance_type" {
  default = "t2.micro"

  validation {
    condition     = length(var.instance_type) >= 2 && length(var.instance_type) <= 20
    error_message = "Instance type must be between 2 and 10 characters"
  }

In the above example, you can see we have set a validation that Instance type length must be between 2 and 10 accomodating something like t2.micro, t3.micro such that, Not larger instance types will be allowed.
I have given a larger instance_type and it leads to below error.

terraform plan
data.aws_ami.validated_ami: Reading...
data.aws_ami.validated_ami: Read complete after 6s [id=ami-02610f36df0c59544]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Invalid value for variable
│
│   on valid.tf line 19:
│   19: variable "instance_type" {
│     ├────────────────
│     │ var.instance_type is "t2.micromediaughgfcghjfd"
│
│ Instance type must be between 2 and 20 characters
│
│ This was checked by the validation rule at valid.tf:22,3-13.

data "aws_ami" "validated_ami" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

resource "aws_instance" "validated_instance" {
  ami           = data.aws_ami.validated_ami.id
  instance_type = var.instance_type
  tags = {
    Name = "validated-instance"
    Type = var.instance_type
  }
}
variable "instance_type" {
  default = "t2.micro"

  validation {
    condition     = can(regex("^t[2-3]\\.", var.instance_type))
    error_message = "Instance type must start with t2 or t3"
  }
}

When I set instance_type as t4.large, then:

terraform plan
data.aws_ami.validated_ami: Reading...
data.aws_ami.validated_ami: Read complete after 5s [id=ami-02610f36df0c59544]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Invalid value for variable
│
│   on valid.tf line 19:
│   19: variable "instance_type" {
│     ├────────────────
│     │ var.instance_type is "t4.micro"
│
│ Instance type must start with t2 or t3
│
│ This was checked by the validation rule at valid.tf:27,3-13.
╵

Example 2: Validating String Suffix

Another common validation is checking if a string ends with a specific suffix, such as validating backup names.

variable "backup_name" {
  default = "daily_backup"

  validation {
    condition     = endswith(var.backup_name, "_backup")
    error_message = "Backup name must end with '_backup'."
  }
}

In the above example, you can see that when you give any value which are not ending with _backup, then it will leads to an error.

Sensitive Variables:

There were times where you will be giving a sensitive value to a terraform and you dont want that value to be displayed anywhere in the terraform console or needs to be stored in the terraform statefile, In those cases, you can add the parameter of "sensitive=true", which will not make that variable visisble to others in console after executing terraform apply.

variable "db_password" {
  type      = string
  default = password
  sensitive = true
}

output "db_pass_output" {
  value     = var.db_password
  sensitive = true   # By enabling here, the Output must also be marked sensitive
}

terraform plan

Changes to Outputs:
  + db_pass_output = (sensitive value)

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

If you didn't enable sensitive=true flag in output blog, then you will see like below:

 terraform plan
╷
│ Error: Output refers to sensitive values
│
│   on main.tf line 7:
│    7: output "db_pass_output" {
│
│ To reduce the risk of accidentally exporting sensitive data that was intended to be only internal, Terraform requires that any
│ root module output containing sensitive data be explicitly marked as sensitive, to confirm your intent.
│
│ If you do intend to export this data, annotate the output value as sensitive by adding the following argument:
│     sensitive = true

Type Conversion and Concat:

The main point to remember here is we will be using merge for combining maps and we will be using concat for strings.

We will be using Type Conversions when there is a use case of below:

Cleans up data automatically
Makes further operations easier
Prevents mistakes when working with lists that shouldn’t have duplicates
Bridges differences between Terraform’s list and set types

variable "user_locations" {
  default = ["us-east-1", "us-west-2", "us-east-1"]  
}

variable "default_locations" {
  default = ["us-west-1"]
}

locals {
  all_locations    = concat(var.user_locations, var.default_locations)
  unique_locations = toset(local.all_locations)
}

output "location" {
  value = local.unique_locations
}

terraform plan

Changes to Outputs:
  + location = [
      + "us-east-1",
      + "us-west-1",
      + "us-west-2",
    ]

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure

You could see that in the above example by using concat we have
merged 2 variables into a single one and also did a type conversion from list to set eliminating duplicate ones.

Number Functions: Sum, Max, Min, Absolute, and Average:

Terraform provides several handy number functions that help us perform calculations on variables, especially lists of numbers.

These functions are useful for things like monthly costs, resource counts, or any scenario where we need math on our infrastructure data.

variable "monthly_costs" {
  default = [-50, 100, 75, 200]  # -50 is a credit
}

locals {
  positive_costs = [for cost in var.monthly_costs : abs(cost)]
  max_cost       = max(local.positive_costs...)
  total_cost     = sum(local.positive_costs)
  avg_cost       = local.total_cost / length(local.positive_costs)
}

output "max_cost" {
  value = local.max_cost
}

output "positive_cost" {
  value = local.positive_costs
}

output "total_costs" {
  value = local.total_cost
}

output "avg_costs" {
  value = local.avg_cost
}

terraform plan

Changes to Outputs:
  + avg_costs     = 106.25
  + max_cost      = 200
  + positive_cost = [
      + 50,
      + 100,
      + 75,
      + 200,
    ]
  + total_costs   = 425

File Handling Functions:

Terraform provides file handling functions that let us read, decode, and use data from files directly in our configurations.

This is especially useful when working with JSON configuration files or other structured data.

locals {
  config_file_exists = fileexists("./config.json")
  config_data        = local.config_file_exists ? jsondecode(file("./config.json")) : {}
}

output "config" {
  value = local.config_data
}

Breaking it down
fileexists("config.json")
Checks if the file exists. Returns true or false.

file("config.json")
Reads the contents of the file.

jsondecode(...)
Converts the JSON text into a Terraform map or object.

Conditional operator (? :)
If the file exists → decode it
If not → return an empty map {}

terraform plan

Changes to Outputs:
  + config = {
      + api      = {
          + endpoint = "https://api.example.com"
          + timeout  = "30"
        }
      + database = {
          + host     = "db.example.com"
          + password = "super-secret"
          + port     = "5432"
          + username = "admin"
        }
    }

Conclusion:

And with this, This marks the end of Day 12 in our 30-days Terraform journey by Piyush Sachdev.
Today we closed the remaining Terraform functions, everything we couldn’t finish in Part 1 finally found its place here in Part 2.

We explored validations, sensitive values, type conversions, numeric functions, timestamps and file handling.

Below is the youtube Video for reference: