Forem: Chinmay Tonape

AWS CDK Constructs Explained: L1, L2, and L3 with Python

Chinmay Tonape — Mon, 16 Mar 2026 17:42:14 +0000

If you've ever looked at two CDK stacks doing the same thing and wondered why one is 200 lines and the other is 20, the answer is almost always which level of construct the author chose. This post breaks down all three levels — L1, L2, and L3 — using a consistent real-world example so you can see exactly what each abstraction buys you.

Background: What Is AWS CDK?

The AWS Cloud Development Kit (CDK) is an open-source framework that lets you define cloud infrastructure using familiar programming languages — Python, TypeScript, Java, and more. Under the hood, CDK synthesizes your code into AWS CloudFormation templates, which are then used to provision resources.

The key advantage over raw CloudFormation YAML is that you get the full power of a programming language: loops, conditionals, type checking, reusable classes, and IDE support. Instead of copying and pasting 300 lines of YAML every time you need a Lambda function, you write a Python class once and reuse it everywhere.

CDK apps are organized around three concepts:

App — the root of your CDK application
Stack — maps to a CloudFormation stack; a unit of deployment
Construct — the building block. Everything in CDK is a construct.

What Are CDK Constructs?

Constructs are the core abstraction in CDK. Every resource you define — a Lambda function, an S3 bucket, an API Gateway — is a construct. They form a tree: your App contains Stacks, Stacks contain constructs, and constructs can contain other constructs.

CDK ships with three levels of constructs, each offering a different trade-off between control and convenience:

Level	Class prefix	What it represents	Boilerplate
L1	`Cfn*`	Raw CloudFormation resource, 1:1 mapping	Maximum
L2	e.g. `Function`, `RestApi`	Curated resource with defaults and helpers	Medium
L3	Custom class or `aws-solutions-constructs`	Opinionated, reusable multi-resource pattern	Minimum

Understanding when to use each level is one of the most practical skills you can develop as a CDK user.

The Example: A Hello World API

To make the comparison concrete, we'll build the same thing three times — once per construct level:

A REST API with a single GET /hello endpoint, backed by a Lambda function, with CloudWatch Logs capturing both API Gateway access logs and Lambda execution logs.

Prerequisites

Install the required packages

pip install -r requirements.txt

L1 — Raw CloudFormation Wrappers

L1 constructs are the lowest level. Every class is prefixed with Cfn (short for CloudFormation) and maps directly to a CloudFormation resource type. If a property exists in CloudFormation, it exists on the L1 construct — nothing more, nothing less.

Working with L1 feels a lot like writing CloudFormation YAML, just in Python. You are responsible for everything: IAM roles, resource dependencies, Lambda invoke permissions, deployment ordering.

When to reach for L1:

A property you need isn't exposed by the L2 construct yet
You're migrating an existing CloudFormation template to CDK
You need precise control over every resource attribute

# Every piece is explicit — IAM role, log group, function, permissions...
lambda_role = iam.CfnRole(
    self, "LambdaRole",
    assume_role_policy_document={
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {"Service": "lambda.amazonaws.com"},
            "Action": "sts:AssumeRole",
        }],
    },
    managed_policy_arns=[
        "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
    ],
)

lambda_log_group = CfnLogGroup(
    self, "LambdaLogGroup",
    log_group_name="/aws/lambda/l1-hello-world",
    retention_in_days=7,
)

lambda_fn = CfnFunction(
    self, "HelloFunction",
    runtime="python3.12",
    handler="index.handler",
    role=lambda_role.attr_arn,
    code=CfnFunction.CodeProperty(zip_file="..."),
    logging_config=CfnFunction.LoggingConfigProperty(
        log_group=lambda_log_group.log_group_name,
        log_format="JSON",
    ),
)
lambda_fn.add_dependency(lambda_log_group)  # you manage ordering manually

# API Gateway — every piece by hand
rest_api = CfnRestApi(self, "HelloApi", name="l1-hello-world-api")
hello_resource = CfnResource(
    self, "HelloResource",
    rest_api_id=rest_api.ref,
    parent_id=rest_api.attr_root_resource_id,
    path_part="hello",
)
# + CfnMethod, CfnDeployment, CfnStage, CfnPermission, CfnAccount...

Pros: Total control. No hidden behavior. Great for auditing exactly what gets deployed.

Cons: Extremely verbose. You own every IAM permission, every add_dependency(), every Fn.sub() for ARN construction. Easy to miss something.

L2 — Curated Constructs with Defaults

L2 constructs are the sweet spot for most CDK users. They wrap L1 constructs with sensible defaults, helper methods, and automatic wiring. CDK generates IAM roles, adds Lambda invoke permissions, links log groups, and creates deployments — all without you asking.

The mental model shifts from "how do I configure this resource" to "what do I want this resource to do."

When to reach for L2:

Day-to-day infrastructure work (this should be your default)
You want secure defaults without writing boilerplate
You want IDE autocomplete and type safety on resource props

# CDK creates the execution role automatically
hello_fn = _lambda.Function(
    self, "HelloFunction",
    runtime=_lambda.Runtime.PYTHON_3_12,
    handler="index.handler",
    code=_lambda.Code.from_inline("..."),
    timeout=Duration.seconds(10),
    log_group=lambda_log_group,   # just pass the log group, CDK links it
)

# LambdaIntegration automatically adds the resource-based policy
api = apigw.RestApi(
    self, "HelloApi",
    deploy_options=apigw.StageOptions(
        access_log_destination=apigw.LogGroupLogDestination(apigw_log_group),
        access_log_format=apigw.AccessLogFormat.json_with_standard_fields(...),
        logging_level=apigw.MethodLoggingLevel.INFO,
    ),
)
hello_resource = api.root.add_resource("hello")
hello_resource.add_method("GET", apigw.LambdaIntegration(hello_fn))

Pros: Dramatically less code. Secure defaults. Typed props with IDE support. Handles the boring stuff.

Cons: Occasionally a property isn't exposed yet. When that happens, you escape to L1 via .node.default_child — a bit awkward but workable.

L3 — Reusable Patterns

L3 constructs are where CDK really shines for teams. They're custom constructs you build by composing L2 constructs into a higher-level, opinionated pattern. The goal is to encode your team's standards — logging, security, tagging, naming conventions — once, and reuse them everywhere.

Think of L3 as your internal infrastructure library. Instead of every developer wiring up Lambda + API Gateway + CloudWatch from scratch, they instantiate your LambdaApiService and get all of that for free.

AWS also ships a set of official L3 patterns via the AWS Solutions Constructs library — pre-built, well-architected combinations like aws-apigateway-lambda or aws-lambda-dynamodb.

When to reach for L3:

You find yourself copy-pasting the same L2 pattern across multiple stacks
You want to enforce team standards (logging, retention, tagging) in one place
You're building a platform that other teams consume

class LambdaApiService(Construct):
    """
    One construct = Lambda + API Gateway + CloudWatch Logs.
    All wiring is internal. Consumers just pass props.
    """
    def __init__(self, scope: Construct, id: str, *, props: LambdaApiServiceProps):
        super().__init__(scope, id)

        # All the L2 wiring happens here, once, for everyone
        lambda_log_group = logs.LogGroup(self, "LambdaLogs", ...)
        self._function = _lambda.Function(self, "Function", log_group=lambda_log_group, ...)

        apigw_log_group = logs.LogGroup(self, "ApiGwLogs", ...)
        self._api = apigw.RestApi(self, "Api", deploy_options=apigw.StageOptions(
            access_log_destination=apigw.LogGroupLogDestination(apigw_log_group), ...
        ))

        resource = self._api.root.add_resource(props.path)
        resource.add_method("GET", apigw.LambdaIntegration(self._function))

        self.endpoint_url = f"{self._api.url}{props.path}"

Using it in a stack becomes trivial:

class L3HelloWorldStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        # One line gets you Lambda + API GW + CloudWatch, fully wired
        service = LambdaApiService(
            self, "HelloService",
            props=LambdaApiServiceProps(
                function_name="l3-hello-world",
                handler_code=HANDLER_CODE,
                api_name="l3-hello-world-api",
                path="hello",
            ),
        )
        CfnOutput(self, "ApiUrl", value=service.endpoint_url)

Pros: Maximum reuse. New services inherit logging, IAM, and API Gateway automatically. Standards enforced at the construct level, not by convention.

Cons: Upfront design investment. Over-abstraction can hide details that matter. Needs versioning if shared across teams.

Side-by-Side Comparison

	L1	L2	L3
Lines of code (this example)	~130	~60	~40 stack + ~60 construct
IAM roles	Manual	Auto-generated	Auto-generated
Lambda permissions	Manual `CfnPermission`	`LambdaIntegration` handles it	Encapsulated
Log group linking	Manual `add_dependency`	`log_group=` prop	Encapsulated
Reusability	None	Low	High
Escape hatch needed	Never	Rarely	Rarely
Best for	Migration / edge cases	Day-to-day work	Platform / shared libraries

Deploying

# Bootstrap your AWS account (once per account/region)
# Run from inside any of the construct folders
cd l2_construct $&& cdk bootstrap
# Deploy the constructs
cd l1_construct && cdk deploy
cd l2_construct && cdk deploy
cd l3_construct && cdk deploy

Test the endpoint:

L1 Construct:

L2 Construct:

L3 Construct:

Check logs in CloudWatch:

Lambda logs: /aws/lambda/l2-hello-world
API Gateway logs: /aws/apigateway/l2-hello-world

Cleanup

To avoid ongoing charges, destroy the stacks when you're done:

cd l1_construct && cdk destroy
cd l2_construct && cdk destroy
cd l3_construct && cdk destroy

CDK will prompt for confirmation before deleting.

Note: The CDK bootstrap stack (CDKToolkit) is not removed by cdk destroy. You can delete it manually from the CloudFormation console if you no longer need CDK in that account/region.

Conclusion

CDK constructs aren't a hierarchy where L3 is always better — they're tools for different situations. L1 gives you the full CloudFormation surface area with no magic. L2 handles the 80% case cleanly with sensible defaults. L3 is where you invest once and multiply the value across your entire team.

A practical rule of thumb: start with L2, drop to L1 when you hit a wall, and promote repeated L2 patterns into L3 constructs when you find yourself copy-pasting. That progression naturally leads to a clean, maintainable infrastructure codebase.

The real power of CDK isn't any single construct level — it's that all three levels compose together seamlessly in the same stack.

References

GitHub repo with all code from this post: https://github.com/chinmayto/aws-cdk-constructs-python
AWS CDK Python API Reference
AWS CDK Construct Library
AWS Solutions Constructs — official L3 patterns
CDK Patterns — community-contributed patterns

Building Scalable Web Application on AWS with CDK in TypeScript

Chinmay Tonape — Tue, 24 Feb 2026 18:45:26 +0000

Introduction

Infrastructure as Code (IaC) has revolutionized how we provision and manage cloud resources. In our previous blog post, we built a highly available web infrastructure using AWS CDK with Python. We explored the fundamentals of CDK, CloudFormation integration, and deployed a production-ready architecture with EC2 instances behind an Application Load Balancer.

In this tutorial, we'll deploy the exact same architecture, but this time using TypeScript as our CDK language. This gives you the opportunity to:

Compare Python and TypeScript approaches to infrastructure as code
Understand how CDK provides a consistent experience across different programming languages
Choose the language that best fits your team's expertise and preferences
See how type safety and IDE support differ between languages

Whether you're a JavaScript/TypeScript developer looking to manage infrastructure in a familiar language, or you're exploring different CDK language options, this guide will show you how TypeScript brings strong typing, excellent IDE support, and a robust ecosystem to your infrastructure code.

Architecture Overview

We'll build a highly available web infrastructure with the following components:

┌─────────────────────────────────────────────────────────────────────┐
│                            Internet                                 │
└────────────────────────────────┬────────────────────────────────────┘
                                 │
                                 ▼
                    ┌────────────────────────┐
                    │   Internet Gateway     │
                    └────────────────────────┘
                                 │
┌────────────────────────────────┼────────────────────────────────────┐
│                                │                  VPC               │
│                                ▼                                    │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │          Application Load Balancer (ALB)                    │    │
│  │                  (Public Subnets)                           │    │
│  └──────────────────────┬──────────────────┬───────────────────┘    │
│                         │                  │                        │
│  ┌──────────────────────┼──────────────────┼─────────────────────┐  │
│  │  Availability Zone 1 │                  │ Availability Zone 2 │  │
│  │                      │                  │                     │  │
│  │  ┌───────────────────▼────────────┐  ┌──▼──────────────────┐  │  │
│  │  │   Public Subnet (AZ-1)         │  │ Public Subnet (AZ-2)│  │  │
│  │  │                                │  │                     │  │  │
│  │  │  ┌──────────────────┐          │  │                     │  │  │
│  │  │  │  NAT Gateway     │          │  │                     │  │  │
│  │  │  └────────┬─────────┘          │  │                     │  │  │
│  │  └───────────┼────────────────────┘  └─────────────────────┘  │  │
│  │              │                                                │  │
│  │  ┌───────────▼────────────────┐  ┌────────────────────────┐   │  │
│  │  │   Private Subnet (AZ-1)    │  │  Private Subnet (AZ-2) │   │  │
│  │  │                            │  │                        │   │  │
│  │  │  ┌──────────────────┐      │  │  ┌──────────────────┐  │   │  │
│  │  │  │  EC2 Instance 1  │      │  │  │  EC2 Instance 2  │  │   │  │
│  │  │  │                  │      │  │  │                  │  │   │  │
│  │  │  │                  │      │  │  │                  │  │   │  │
│  │  │  └──────────────────┘      │  │  └──────────────────┘  │   │  │
│  │  │                            │  │                        │   │  │
│  │  └────────────────────────────┘  └────────────────────────┘   │  │
│  │                                                               │  │
│  └───────────────────────────────────────────────────────────────┘  │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Building the Infrastructure

Step 1: Project Setup

First, ensure you have the prerequisites installed:

# Install Node.js and npm (if not already installed)
# Install AWS CDK CLI globally
npm install -g aws-cdk

# Verify installation
cdk --version

Step 2: Initialize the Project

# Initialize CDK project (TypeScript example)
cdk init app --language=typescript

# Install dependencies
npm install

Step 3: Bootstrap Your AWS Environment

Before deploying CDK applications, you need to bootstrap your AWS environment. This creates necessary resources like an S3 bucket for storing templates and IAM roles for deployments.

# Bootstrap CDK (one-time per account/region)
cdk bootstrap

>cdk bootstrap
Deploying to account: 197317184204, region: us-east-1
 ⏳  Bootstrapping environment aws://197317184204/us-east-1...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...
CDKToolkit |  0/12 | 11:46:00 pm | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit User Initiated
.
.
.
CDKToolkit | 12/12 | 11:46:51 pm | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 ✅  Environment aws://197317184204/us-east-1 bootstrapped.

Step 4: Review the Infrastructure Code

Creating the VPC

The VPC is configured with 2 Availability Zones, public subnets for the load balancer, and private subnets for EC2 instances:

// Create VPC with public and private subnets
const vpc = new ec2.Vpc(this, 'WebsiteVPC', {
  maxAzs: 2,
  natGateways: 1,
  subnetConfiguration: [
    {
      name: 'PublicSubnet',
      subnetType: ec2.SubnetType.PUBLIC,
      cidrMask: 24
    },
    {
      name: 'PrivateSubnet',
      subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
      cidrMask: 24
    }
  ]
});

Security Groups

We create two security groups: one for the ALB (allowing internet traffic) and one for EC2 instances (allowing traffic only from the ALB):

// Security group for ALB (public-facing)
const albSecurityGroup = new ec2.SecurityGroup(this, 'ALBSecurityGroup', {
  vpc: vpc,
  description: 'Security group for Application Load Balancer',
  allowAllOutbound: true
});

albSecurityGroup.addIngressRule(
  ec2.Peer.anyIpv4(),
  ec2.Port.tcp(80),
  'Allow HTTP traffic from internet'
);

// Security group for EC2 instances (private)
const ec2SecurityGroup = new ec2.SecurityGroup(this, 'EC2SecurityGroup', {
  vpc: vpc,
  description: 'Security group for EC2 instances',
  allowAllOutbound: true
});

ec2SecurityGroup.addIngressRule(
  albSecurityGroup,
  ec2.Port.tcp(80),
  'Allow HTTP traffic from ALB'
);

IAM Role for EC2

The IAM role allows EC2 instances to be managed via AWS Systems Manager:

// IAM role for EC2 instances
const ec2Role = new iam.Role(this, 'EC2Role', {
  assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonSSMManagedInstanceCore')
  ]
});

User Data Script

The user data script installs Apache and creates a dynamic webpage:

// User data script to install and configure web server
const userData = ec2.UserData.forLinux();
userData.addCommands(
  'yum update -y',
  'yum install -y httpd',
  'systemctl start httpd',
  'systemctl enable httpd',
  'TOKEN=$(curl --request PUT "http://169.254.169.254/latest/api/token" --header "X-aws-ec2-metadata-token-ttl-seconds: 3600")',
  'instanceId=$(curl -s http://169.254.169.254/latest/meta-data/instance-id --header "X-aws-ec2-metadata-token: $TOKEN")',
  'echo "<h1>AWS Linux VM Deployed with CDK</h1>" > /var/www/html/index.html',
  'echo "<p>Instance ID: $instanceId</p>" >> /var/www/html/index.html'
);

Creating EC2 Instances

We create 2 EC2 instances distributed across different Availability Zones:

// Create EC2 instances in different AZs
const instance1 = new ec2.Instance(this, 'WebInstance1', {
  vpc: vpc,
  instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.MICRO),
  machineImage: ec2.MachineImage.latestAmazonLinux2023(),
  securityGroup: ec2SecurityGroup,
  role: ec2Role,
  userData: userData,
  vpcSubnets: {
    subnets: [privateSubnets.subnets[0]]
  },
  requireImdsv2: true
});

const instance2 = new ec2.Instance(this, 'WebInstance2', {
  vpc: vpc,
  instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.MICRO),
  machineImage: ec2.MachineImage.latestAmazonLinux2023(),
  securityGroup: ec2SecurityGroup,
  role: ec2Role,
  userData: userData,
  vpcSubnets: {
    subnets: [privateSubnets.subnets[1]]
  },
  requireImdsv2: true
});

Application Load Balancer

Finally, we create the ALB with a target group and listener:

// Application Load Balancer
const alb = new elbv2.ApplicationLoadBalancer(this, 'WebsiteALB', {
  vpc: vpc,
  internetFacing: true,
  securityGroup: albSecurityGroup,
  vpcSubnets: {
    subnetType: ec2.SubnetType.PUBLIC
  }
});

// Target group with health checks
const targetGroup = new elbv2.ApplicationTargetGroup(this, 'WebsiteTargetGroup', {
  vpc: vpc,
  port: 80,
  protocol: elbv2.ApplicationProtocol.HTTP,
  targets: [
    new elbv2_targets.InstanceTarget(instance1, 80),
    new elbv2_targets.InstanceTarget(instance2, 80)
  ],
  healthCheck: {
    path: '/',
    interval: cdk.Duration.seconds(30)
  }
});

// Listener
alb.addListener('WebsiteListener', {
  port: 80,
  protocol: elbv2.ApplicationProtocol.HTTP,
  defaultTargetGroups: [targetGroup]
});

CloudFormation Outputs

We define outputs to easily access important resource information:

// Outputs
new cdk.CfnOutput(this, 'LoadBalancerDNS', {
  value: alb.loadBalancerDnsName,
  description: 'DNS name of the load balancer'
});

Step 5: Deploy the Infrastructure

Preview the CloudFormation template that CDK will generate:

cdk synth

Deploy the stack to AWS:

cdk deploy

CDK will show you the changes and ask for confirmation. Type y to proceed.

The deployment takes approximately 5–10 minutes. Once complete, you’ll see outputs including:

The deployment process:

CDK synthesizes your code into CloudFormation templates
Templates are uploaded to the bootstrap S3 bucket
CloudFormation creates a change set
Resources are provisioned in dependency order
Outputs are displayed (including the load balancer URL)

>cdk deploy
Deploying to account: 197317184204, region: us-east-1

✨  Synthesis time: 10.91s

WebsiteStack: start: Building WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code
WebsiteStack: success: Built WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code
WebsiteStack: start: Building WebsiteStack Template
WebsiteStack: success: Built WebsiteStack Template
WebsiteStack: start: Publishing WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code (197317184204-us-east-1-78ffe0b8)
WebsiteStack: start: Publishing WebsiteStack Template (197317184204-us-east-1-0ae3e5c9)
WebsiteStack: success: Published WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code (197317184204-us-east-1-78ffe0b8)
WebsiteStack: success: Published WebsiteStack Template (197317184204-us-east-1-0ae3e5c9)
Stack WebsiteStack
IAM Statement Changes
┌───┬──────────────────────────────────────────────────────────────┬────────┬──────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┬───────────┐
│   │ Resource                                                     │ Effect │ Action                                                       │ Principal                                                      │ Condition │
├───┼──────────────────────────────────────────────────────────────┼────────┼──────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┼───────────┤ 
│ + │ ${Custom::VpcRestrictDefaultSGCustomResourceProvider/Role.Ar │ Allow  │ sts:AssumeRole                                               │ Service:lambda.amazonaws.com                                   │           │ 
│   │ n}                                                           │        │                                                              │                                                                │           │ 
├───┼──────────────────────────────────────────────────────────────┼────────┼──────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┼───────────┤
│ + │ ${EC2Role.Arn}                                               │ Allow  │ sts:AssumeRole                                               │ Service:ec2.amazonaws.com                                      │           │ 
├───┼──────────────────────────────────────────────────────────────┼────────┼──────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┼───────────┤ 
│ + │ arn:aws:ec2:us-east-1:197317184204:security-group/${WebsiteV │ Allow  │ ec2:AuthorizeSecurityGroupEgress                             │ AWS:${Custom::VpcRestrictDefaultSGCustomResourceProvider/Role} │           │ 
│   │ PC.DefaultSecurityGroup}                                     │        │ ec2:AuthorizeSecurityGroupIngress                            │                                                                │           │ 
│   │                                                              │        │ ec2:RevokeSecurityGroupEgress                                │                                                                │           │ 
│   │                                                              │        │ ec2:RevokeSecurityGroupIngress                               │                                                                │           │ 
└───┴──────────────────────────────────────────────────────────────┴────────┴──────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┴───────────┘ 
IAM Policy Changes
┌───┬────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                                                   │ Managed Policy ARN                                                                           │
├───┼────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${Custom::VpcRestrictDefaultSGCustomResourceProvider/Role} │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"} │
├───┼────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${EC2Role}                                                 │ arn:${AWS::Partition}:iam::aws:policy/AmazonSSMManagedInstanceCore                           │
└───┴────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────┘
Security Group Changes
┌───┬─────────────────────────────┬─────┬────────────┬─────────────────────────────┐
│   │ Group                       │ Dir │ Protocol   │ Peer                        │
├───┼─────────────────────────────┼─────┼────────────┼─────────────────────────────┤
│ + │ ${ALBSecurityGroup.GroupId} │ In  │ TCP 80     │ Everyone (IPv4)             │
│ + │ ${ALBSecurityGroup.GroupId} │ In  │ TCP 443    │ Everyone (IPv4)             │
│ + │ ${ALBSecurityGroup.GroupId} │ Out │ Everything │ Everyone (IPv4)             │
├───┼─────────────────────────────┼─────┼────────────┼─────────────────────────────┤
│ + │ ${EC2SecurityGroup.GroupId} │ In  │ TCP 80     │ ${ALBSecurityGroup.GroupId} │
│ + │ ${EC2SecurityGroup.GroupId} │ Out │ Everything │ Everyone (IPv4)             │
└───┴─────────────────────────────┴─────┴────────────┴─────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)


"--require-approval" is enabled and stack includes security-sensitive updates: 'Do you wish to deploy these changes' (y/n) y
WebsiteStack: deploying... [1/1]
WebsiteStack: creating CloudFormation changeset...
WebsiteStack |  0/39 | 11:50:52 pm | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack                | WebsiteStack User Initiated
.
.
.
WebsiteStack | 39/39 | 11:54:14 pm | CREATE_COMPLETE      | AWS::CloudFormation::Stack                | WebsiteStack 

 ✅  WebsiteStack

✨  Deployment time: 213.29s

Outputs:
WebsiteStack.Instance1Id = i-09fcc8032aee32aa0
WebsiteStack.Instance2Id = i-0c0fa16e8fbfbbbc3
WebsiteStack.LoadBalancerDNS = Websit-Websi-Ii3nPppS97V3-1650643368.us-east-1.elb.amazonaws.com
WebsiteStack.VPCId = vpc-0caf658a5e29d540d
Stack ARN:
arn:aws:cloudformation:us-east-1:197317184204:stack/WebsiteStack/8d7fe6a0-11ad-11f1-abb0-0affee6835c3

✨  Total time: 224.2s

Step 6: Test Your Application

Open the LoadBalancerDNS URL in your browser to see your website. Refresh multiple times to see traffic distributed across different instances in different availability zones.

Step 7: Explore Your Infrastructure

View the CloudFormation stack in the AWS Console:

Navigate to CloudFormation service
Find the WebsiteStack stack
Explore Resources, Events, and Outputs tabs

Cleanup

To avoid ongoing AWS charges, destroy the infrastructure when you're done:

# Delete all resources
cdk destroy

# Confirm deletion when prompted

This removes all resources created by the stack. CloudFormation ensures clean deletion in the correct order, handling dependencies automatically.

Note: The bootstrap resources (S3 bucket, IAM roles) are not deleted by cdk destroy. These are shared across all CDK applications in your account and should be kept for future use.

Project Structure

Understanding the CDK project structure helps you navigate and organize your infrastructure code effectively:

aws-cdk-website/
├── bin/
│   └── app.ts                    # CDK app entry point - defines stacks
├── lib/
│   └── website-stack.ts          # Main stack with VPC, EC2, ALB resources
├── node_modules/                 # npm dependencies (auto-generated)
├── cdk.out/                      # Synthesized CloudFormation templates
├── cdk.json                      # CDK toolkit configuration
├── cdk.context.json              # CDK context values (auto-generated)
├── package.json                  # Node.js project dependencies and scripts
├── package-lock.json             # Locked dependency versions
├── tsconfig.json                 # TypeScript compiler configuration

Key Files Explained:

bin/app.ts - Application entry point where you instantiate your CDK app and stacks:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { WebsiteStack } from '../lib/website-stack';

const app = new cdk.App();
new WebsiteStack(app, 'WebsiteStack', {
  env: { 
    account: process.env.CDK_DEFAULT_ACCOUNT, 
    region: process.env.CDK_DEFAULT_REGION 
  }
});

lib/website-stack.ts - Contains your infrastructure definitions (VPC, EC2, ALB, security groups, etc.)

cdk.json - Configures how the CDK toolkit executes your app:

{
  "app": "npx ts-node --prefer-ts-exts bin/app.ts",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": true,
    "aws-cdk:enableDiffNoFail": true
  }
}

package.json - Defines project metadata, dependencies, and useful scripts:

{
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "deploy": "npm run build && cdk deploy",
    "synth": "cdk synth",
    "diff": "cdk diff",
    "destroy": "cdk destroy"
  }
}

cdk.out/ - Generated directory containing CloudFormation templates after running cdk synth. This is what actually gets deployed to AWS.

Conclusion

We've successfully deployed the same highly available web infrastructure from our previous Python CDK tutorial, but this time using TypeScript. As you've seen, AWS CDK provides a consistent and powerful experience regardless of your language choice.

TypeScript brings several advantages to infrastructure as code:

Strong type safety: Catch errors at compile time before deployment
Superior IDE support: Autocomplete, inline documentation, and refactoring tools
Familiar syntax: JavaScript/TypeScript developers can leverage existing knowledge
Rich ecosystem: Access to npm packages and TypeScript tooling
Better collaboration: Type definitions serve as living documentation

Whether you choose Python, TypeScript, or another supported language, CDK transforms infrastructure provisioning from a tedious, error-prone process into an enjoyable development experience. The combination of CDK's developer-friendly interface and CloudFormation's robust deployment engine provides the best of both worlds.

References

GitHub Repository: aws-cdk-typescript
AWS CDK Documentation: https://docs.aws.amazon.com/cdk/

Building Scalable Web Application on AWS with CDK and Python

Chinmay Tonape — Tue, 24 Feb 2026 17:49:41 +0000

Introduction

Infrastructure as Code (IaC) has revolutionized how we provision and manage cloud resources. In this tutorial, we'll explore AWS Cloud Development Kit (CDK) and build a production-ready web application infrastructure with high availability built in.

What is AWS CDK?

AWS Cloud Development Kit (CDK) is an open-source software development framework that lets you define cloud infrastructure using familiar programming languages like Python, TypeScript, Java, and C#. Instead of writing verbose JSON or YAML templates, you can use the full power of programming languages to define your infrastructure.

Why Use AWS CDK?

Use familiar programming languages: Write infrastructure code in Python, TypeScript, or other supported languages
Leverage IDE features: Get autocomplete, type checking, and inline documentation
Reusable components: Create and share infrastructure patterns as libraries
Higher-level abstractions: CDK constructs provide sensible defaults and best practices
Faster development: Write less code compared to raw CloudFormation templates

CDK and CloudFormation: The Perfect Partnership

AWS CDK doesn't replace CloudFormation—it enhances it. Here's how they work together:

You write CDK code in your preferred programming language (Python in our case)
CDK synthesizes CloudFormation templates from your code using cdk synth
CloudFormation deploys the infrastructure to AWS
CloudFormation manages the stack lifecycle (updates, rollbacks, deletions)

This means you get the power of programming languages for authoring and the reliability of CloudFormation for deployment. CDK handles the complexity of generating correct CloudFormation templates, while CloudFormation ensures safe, predictable infrastructure changes.

Architecture Overview

We'll build a highly available web application with the following architecture:

┌─────────────────────────────────────────────────────────────────────┐
│                            Internet                                 │
└────────────────────────────────┬────────────────────────────────────┘
                                 │
                                 ▼
                    ┌────────────────────────┐
                    │   Internet Gateway     │
                    └────────────────────────┘
                                 │
┌────────────────────────────────┼────────────────────────────────────┐
│                                │                  VPC               │
│                                ▼                                    │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │          Application Load Balancer (ALB)                    │    │
│  │                  (Public Subnets)                           │    │
│  └──────────────────────┬──────────────────┬───────────────────┘    │
│                         │                  │                        │
│  ┌──────────────────────┼──────────────────┼─────────────────────┐  │
│  │  Availability Zone 1 │                  │ Availability Zone 2 │  │
│  │                      │                  │                     │  │
│  │  ┌───────────────────▼────────────┐  ┌──▼──────────────────┐  │  │
│  │  │   Public Subnet (AZ-1)         │  │ Public Subnet (AZ-2)│  │  │
│  │  │                                │  │                     │  │  │
│  │  │  ┌──────────────────┐          │  │                     │  │  │
│  │  │  │  NAT Gateway     │          │  │                     │  │  │
│  │  │  └────────┬─────────┘          │  │                     │  │  │
│  │  └───────────┼────────────────────┘  └─────────────────────┘  │  │
│  │              │                                                │  │
│  │  ┌───────────▼────────────────┐  ┌────────────────────────┐   │  │
│  │  │   Private Subnet (AZ-1)    │  │  Private Subnet (AZ-2) │   │  │
│  │  │                            │  │                        │   │  │
│  │  │  ┌──────────────────┐      │  │  ┌──────────────────┐  │   │  │
│  │  │  │  EC2 Instance 1  │      │  │  │  EC2 Instance 2  │  │   │  │
│  │  │  │                  │      │  │  │                  │  │   │  │
│  │  │  │                  │      │  │  │                  │  │   │  │
│  │  │  └──────────────────┘      │  │  └──────────────────┘  │   │  │
│  │  │                            │  │                        │   │  │
│  │  └────────────────────────────┘  └────────────────────────┘   │  │
│  │                                                               │  │
│  └───────────────────────────────────────────────────────────────┘  │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Prerequisites

Before we begin, ensure you have:

Node.js (v14 or later) for CDK CLI
Python 3.8+ installed

Project Structure

aws-cdk-python/
│
├── app.py                    # CDK app entry point - initializes the stack
├── website_stack.py          # Main infrastructure stack definition
├── deploy.py                 # Deployment helper script
│
├── cdk.json                  # CDK configuration file
├── cdk.context.json          # CDK context values (auto-generated)
│
├── requirements.txt          # Python dependencies
├── setup.py                  # Python package setup file
│
└── cdk.out/                 # CDK synthesized CloudFormation templates (auto-generated)
    ├── WebsiteStack.template.json
    ├── manifest.json
    └── tree.json

Step 1: Install AWS CDK CLI

Install the CDK CLI globally using npm:

npm install -g aws-cdk

Verify the installation:

cdk --version

Step 2: Set Up the Project

Clone the repository or create a new directory:

git clone https://github.com/chinmayto/aws-cdk-python.git
cd aws-cdk-python

Create a Python virtual environment:

# Windows
python -m venv .venv
.venv\Scripts\activate

# Linux/Mac
python3 -m venv .venv
source .venv/bin/activate

Install Python dependencies:

pip install -r requirements.txt

Step 3: Bootstrap Your AWS Environment

CDK requires a one-time bootstrap process to create resources it needs for deployment (S3 bucket for assets, IAM roles, etc.):

cdk bootstrap

This command creates a CloudFormation stack called CDKToolkit in your AWS account.

>cdk bootstrap
Deploying to account: 197317184204, region: us-east-1
 ⏳  Bootstrapping environment aws://197317184204/us-east-1...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...
CDKToolkit |  0/12 | 10:53:19 pm | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit User Initiated
.
.
.
CDKToolkit | 12/12 | 10:54:11 pm | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 ✅  Environment aws://197317184204/us-east-1 bootstrapped.

Step 4: Review the Infrastructure Code

Our infrastructure is defined in website_stack.py. Let's break down the key components:

Creating the VPC

The VPC is configured with 2 Availability Zones, public subnets for the load balancer, and private subnets for EC2 instances:

# Create VPC with public and private subnets
vpc = ec2.Vpc(
    self, "WebsiteVPC",
    max_azs=2,
    nat_gateways=1,
    subnet_configuration=[
        ec2.SubnetConfiguration(
            name="PublicSubnet",
            subnet_type=ec2.SubnetType.PUBLIC,
            cidr_mask=24
        ),
        ec2.SubnetConfiguration(
            name="PrivateSubnet",
            subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS,
            cidr_mask=24
        )
    ]
)

Security Groups

We create two security groups: one for the ALB (allowing internet traffic) and one for EC2 instances (allowing traffic only from the ALB):

# Security group for ALB (public-facing)
alb_security_group = ec2.SecurityGroup(
    self, "ALBSecurityGroup",
    vpc=vpc,
    description="Security group for Application Load Balancer",
    allow_all_outbound=True
)

alb_security_group.add_ingress_rule(
    ec2.Peer.any_ipv4(),
    ec2.Port.tcp(80),
    "Allow HTTP traffic from internet"
)

# Security group for EC2 instances (private)
ec2_security_group = ec2.SecurityGroup(
    self, "EC2SecurityGroup",
    vpc=vpc,
    description="Security group for EC2 instances",
    allow_all_outbound=True
)

ec2_security_group.add_ingress_rule(
    alb_security_group,
    ec2.Port.tcp(80),
    "Allow HTTP traffic from ALB"
)

IAM Role for EC2

The IAM role allows EC2 instances to be managed via AWS Systems Manager:

# IAM role for EC2 instances
ec2_role = iam.Role(
    self, "EC2Role",
    assumed_by=iam.ServicePrincipal("ec2.amazonaws.com"),
    managed_policies=[
        iam.ManagedPolicy.from_aws_managed_policy_name("AmazonSSMManagedInstanceCore")
    ]
)

User Data Script

The user data script installs Apache and creates a dynamic webpage:

# User data script to install and configure web server
user_data = ec2.UserData.for_linux()
user_data.add_commands(
    "yum update -y",
    "yum install -y httpd",
    "systemctl start httpd",
    "systemctl enable httpd",
    "",
    "# Get instance metadata using IMDSv2",
    'TOKEN=$(curl --request PUT "http://169.254.169.254/latest/api/token" --header "X-aws-ec2-metadata-token-ttl-seconds: 3600")',
    'instanceId=$(curl -s http://169.254.169.254/latest/meta-data/instance-id --header "X-aws-ec2-metadata-token: $TOKEN")',
    'instanceAZ=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone --header "X-aws-ec2-metadata-token: $TOKEN")',
    'privHostName=$(curl -s http://169.254.169.254/latest/meta-data/local-hostname --header "X-aws-ec2-metadata-token: $TOKEN")',
    'privIPv4=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4 --header "X-aws-ec2-metadata-token: $TOKEN")',
    "",
    "# Create HTML page",
    'echo "<font face = \\"Verdana\\" size = \\"5\\">"                               > /var/www/html/index.html',
    'echo "<center><h1>AWS Linux VM Deployed with CDK using Python</h1></center>"  >> /var/www/html/index.html',
    'echo "<center> <b>EC2 Instance Metadata</b> </center>"                        >> /var/www/html/index.html',
    'echo "<center> <b>Instance ID:</b> $instanceId </center>"                     >> /var/www/html/index.html',
    'echo "<center> <b>AWS Availablity Zone:</b> $instanceAZ </center>"            >> /var/www/html/index.html',
    'echo "<center> <b>Private Hostname:</b> $privHostName </center>"              >> /var/www/html/index.html',
    'echo "<center> <b>Private IPv4:</b> $privIPv4 </center>"                      >> /var/www/html/index.html',
    'echo "</font>"                                                                >> /var/www/html/index.html'
)

Creating EC2 Instances

We create 2 EC2 instances distributed across different Availability Zones:

# Create 2 EC2 instances in private subnets across different AZs
instances = []
private_subnets = vpc.select_subnets(subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS).subnets

for i in range(2):
    instance = ec2.Instance(
        self, f"WebInstance{i+1}",
        vpc=vpc,
        instance_type=ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.MICRO),
        machine_image=ec2.MachineImage.latest_amazon_linux2023(),
        security_group=ec2_security_group,
        role=ec2_role,
        user_data=user_data,
        vpc_subnets=ec2.SubnetSelection(
            subnets=[private_subnets[i % len(private_subnets)]]
        ),
        require_imdsv2=True
    )
    instances.append(instance)

Application Load Balancer

Finally, we create the ALB with a target group and listener:

# Application Load Balancer
alb = elbv2.ApplicationLoadBalancer(
    self, "WebsiteALB",
    vpc=vpc,
    internet_facing=True,
    security_group=alb_security_group,
    vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC)
)

# Target group
target_group = elbv2.ApplicationTargetGroup(
    self, "WebsiteTargetGroup",
    vpc=vpc,
    port=80,
    protocol=elbv2.ApplicationProtocol.HTTP,
    targets=[targets.InstanceIdTarget(instance.instance_id) for instance in instances],
    health_check=elbv2.HealthCheck(
        path="/",
        interval=Duration.seconds(30)
    )
)

# Listener
listener = alb.add_listener(
    "WebsiteListener",
    port=80,
    protocol=elbv2.ApplicationProtocol.HTTP,
    default_target_groups=[target_group]
)

CloudFormation Outputs

We define outputs to easily access important resource information:

# Outputs
CfnOutput(
    self, "LoadBalancerDNS",
    value=alb.load_balancer_dns_name,
    description="DNS name of the load balancer"
)

CfnOutput(
    self, "Instance1Id",
    value=instances[0].instance_id,
    description="Instance ID of the first EC2 instance"
)

CfnOutput(
    self, "Instance2Id",
    value=instances[1].instance_id,
    description="Instance ID of the second EC2 instance"
)

Step 5: Deploy the Infrastructure

Preview the CloudFormation template that CDK will generate:

cdk synth

Deploy the stack to AWS:

cdk deploy

CDK will show you the changes and ask for confirmation. Type y to proceed.

The deployment takes approximately 5-10 minutes. Once complete, you'll see outputs including:

>cdk deploy
Deploying to account: 197317184204, region: us-east-1

✨  Synthesis time: 16.13s

WebsiteStack: start: Building WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code
WebsiteStack: success: Built WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code
WebsiteStack: start: Building WebsiteStack Template
WebsiteStack: success: Built WebsiteStack Template
WebsiteStack: start: Publishing WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code (197317184204-us-east-1-78ffe0b8)
WebsiteStack: start: Publishing WebsiteStack Template (197317184204-us-east-1-2ed930b7)
WebsiteStack: success: Published WebsiteStack Template (197317184204-us-east-1-2ed930b7)
WebsiteStack: success: Published WebsiteStack/Custom::VpcRestrictDefaultSGCustomResourceProvider Code (197317184204-us-east-1-78ffe0b8)
Stack WebsiteStack
IAM Statement Changes
┌───┬─────────────────────────────────────────────────┬────────┬─────────────────────────────────────────────────┬─────────────────────────────────────────────────┬───────────┐
│   │ Resource                                        │ Effect │ Action                                          │ Principal                                       │ Condition │
├───┼─────────────────────────────────────────────────┼────────┼─────────────────────────────────────────────────┼─────────────────────────────────────────────────┼───────────┤ 
│ + │ ${Custom::VpcRestrictDefaultSGCustomResourcePro │ Allow  │ sts:AssumeRole                                  │ Service:lambda.amazonaws.com                    │           │ 
│   │ vider/Role.Arn}                                 │        │                                                 │                                                 │           │ 
├───┼─────────────────────────────────────────────────┼────────┼─────────────────────────────────────────────────┼─────────────────────────────────────────────────┼───────────┤ 
│ + │ ${EC2Role.Arn}                                  │ Allow  │ sts:AssumeRole                                  │ Service:ec2.amazonaws.com                       │           │ 
├───┼─────────────────────────────────────────────────┼────────┼─────────────────────────────────────────────────┼─────────────────────────────────────────────────┼───────────┤
│ + │ arn:aws:ec2:us-east-1:197317184204:security-gro │ Allow  │ ec2:AuthorizeSecurityGroupEgress                │ AWS:${Custom::VpcRestrictDefaultSGCustomResourc │           │ 
│   │ up/${WebsiteVPC.DefaultSecurityGroup}           │        │ ec2:AuthorizeSecurityGroupIngress               │ eProvider/Role}                                 │           │ 
│   │                                                 │        │ ec2:RevokeSecurityGroupEgress                   │                                                 │           │ 
│   │                                                 │        │ ec2:RevokeSecurityGroupIngress                  │                                                 │           │ 
└───┴─────────────────────────────────────────────────┴────────┴─────────────────────────────────────────────────┴─────────────────────────────────────────────────┴───────────┘ 
IAM Policy Changes
┌───┬────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                                                   │ Managed Policy ARN                                                                           │
├───┼────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${Custom::VpcRestrictDefaultSGCustomResourceProvider/Role} │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"} │
├───┼────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${EC2Role}                                                 │ arn:${AWS::Partition}:iam::aws:policy/AmazonSSMManagedInstanceCore                           │
└───┴────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────┘
Security Group Changes
┌───┬─────────────────────────────┬─────┬────────────┬─────────────────────────────┐
│   │ Group                       │ Dir │ Protocol   │ Peer                        │
├───┼─────────────────────────────┼─────┼────────────┼─────────────────────────────┤
│ + │ ${ALBSecurityGroup.GroupId} │ In  │ TCP 80     │ Everyone (IPv4)             │
│ + │ ${ALBSecurityGroup.GroupId} │ In  │ TCP 443    │ Everyone (IPv4)             │
│ + │ ${ALBSecurityGroup.GroupId} │ Out │ Everything │ Everyone (IPv4)             │
├───┼─────────────────────────────┼─────┼────────────┼─────────────────────────────┤
│ + │ ${EC2SecurityGroup.GroupId} │ In  │ TCP 22     │ ${WebsiteVPC.CidrBlock}     │
│ + │ ${EC2SecurityGroup.GroupId} │ In  │ TCP 80     │ ${ALBSecurityGroup.GroupId} │
│ + │ ${EC2SecurityGroup.GroupId} │ Out │ Everything │ Everyone (IPv4)             │
└───┴─────────────────────────────┴─────┴────────────┴─────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)


"--require-approval" is enabled and stack includes security-sensitive updates: 'Do you wish to deploy these changes' (y/n) y
WebsiteStack: deploying... [1/1]
WebsiteStack: creating CloudFormation changeset...
WebsiteStack |  0/39 | 10:58:15 pm | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack                | WebsiteStack User Initiated
.
.
.
WebsiteStack | 39/39 | 11:01:35 pm | CREATE_COMPLETE      | AWS::CloudFormation::Stack                | WebsiteStack

 ✅  WebsiteStack

✨  Deployment time: 213.18s

Outputs:
WebsiteStack.Instance1Id = i-089c31f66c61482a3
WebsiteStack.Instance2Id = i-0af48aefd5e4efcc2
WebsiteStack.LoadBalancerDNS = Websit-Websi-5kyWZuGmscD3-1445557942.us-east-1.elb.amazonaws.com
WebsiteStack.VPCId = vpc-002b8b79c7f0e2f49
Stack ARN:
arn:aws:cloudformation:us-east-1:197317184204:stack/WebsiteStack/33bc7c70-11a6-11f1-8888-0affc0f3836d

✨  Total time: 229.31s

Step 6: Test Your Application

Copy the LoadBalancerDNS value from the outputs and open it in your browser:

Websit-Websi-5kyWZuGmscD3-1445557942.us-east-1.elb.amazonaws.com

You should see a webpage displaying EC2 instance metadata. Refresh the page multiple times to see the load balancer distributing traffic between the two instances (notice the Instance ID and Availability Zone changing).

Step 7: Explore Your Infrastructure

View the CloudFormation stack in the AWS Console:

Navigate to CloudFormation service
Find the WebsiteStack stack
Explore Resources, Events, and Outputs tabs

Cleanup

To avoid ongoing charges, destroy the infrastructure when you're done:

cdk destroy

Type y to confirm. CDK will delete all resources created by the stack.

>cdk destroy
Deploying to account: 197317184204, region: us-east-1
Are you sure you want to delete: WebsiteStack (y/n) y
WebsiteStack: destroying... [1/1]
WebsiteStack |   0 | 11:08:57 pm | DELETE_IN_PROGRESS   | AWS::CloudFormation::Stack                | WebsiteStack User Initiated
.
.
.
WebsiteStack |  37 | 11:10:22 pm | DELETE_IN_PROGRESS   | AWS::EC2::VPC                             | WebsiteVPC (WebsiteVPCD8B49DC8)

 ✅  WebsiteStack: destroyed

Conclusion

AWS CDK transforms infrastructure provisioning from a tedious, error-prone process into an enjoyable development experience. By leveraging Python and CloudFormation together, we've built a production-ready, highly available web application infrastructure with just a few hundred lines of code.

The combination of CDK's developer-friendly abstractions and CloudFormation's robust deployment engine gives you the best of both worlds: rapid development and reliable operations.

References

GitHub Repository: aws-cdk-python
AWS CDK Documentation: https://docs.aws.amazon.com/cdk/

GitOps with ArgoCD on Amazon EKS using Terraform: A Complete Implementation Guide

Chinmay Tonape — Fri, 12 Dec 2025 04:35:31 +0000

In the rapidly evolving world of DevOps and cloud-native applications, GitOps has emerged as a revolutionary approach to continuous deployment and infrastructure management. This blog post explores how to implement a complete GitOps workflow using ArgoCD on Amazon Elastic Kubernetes Service (EKS), providing you with a production-ready setup that follows industry best practices.

GitOps represents a paradigm shift where Git repositories serve as the single source of truth for both application code and infrastructure configuration. By leveraging ArgoCD as our GitOps operator, we can achieve automated, reliable, and auditable deployments while maintaining the declarative nature of Kubernetes.

What is GitOps?

GitOps is a modern approach to continuous deployment that uses Git as the single source of truth for declarative infrastructure and applications. The core principles include:

Declarative Configuration: Everything is described declaratively in Git
Version Control: All changes are tracked and auditable
Automated Deployment: Changes in Git trigger automatic deployments
Continuous Monitoring: The system continuously ensures the desired state matches the actual state

Why ArgoCD?

ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes that offers:

Application Management: Centralized management of multiple applications
Multi-Cluster Support: Deploy to multiple Kubernetes clusters
Rich UI: Intuitive web interface for monitoring deployments
RBAC Integration: Fine-grained access control
Rollback Capabilities: Easy rollback to previous versions

Architecture Overview

Our implementation creates a robust, scalable architecture that includes:

┌──────────────────────────────────────────────────────────────┐
│                        AWS Cloud                             │
│  ┌──────────────────────────────────────────────────────────┐│
│  │                    VPC                                   ││
│  │  ┌─────────────────┐    ┌──────────────────────────────┐ ││
│  │  │  Public Subnets │    │      Private Subnets         │ ││
│  │  │                 │    │  ┌─────────────────────────┐ │ ││
│  │  │  ┌───────────┐  │    │  │      EKS Cluster        │ │ ││
│  │  │  │    NAT    │  │    │  │  ┌─────────────────────┐│ │ ││
│  │  │  │  Gateway  │  │    │  │  │   NGINX Ingress     ││ │ ││
│  │  │  └───────────┘  │    │  │  │    Controller       ││ │ ││
│  │  │                 │    │  │  └─────────────────────┘│ │ ││
│  │  └─────────────────┘    │  │  ┌─────────────────────┐│ │ ││
│  │                         │  │  │     ArgoCD          ││ │ ││
│  │                         │  │  │     Server          ││ │ ││
│  │                         │  │  └─────────────────────┘│ │ ││
│  │                         │  │  ┌─────────────────────┐│ │ ││
│  │                         │  │  │   Application       ││ │ ││
│  │                         │  │  │   Workloads         ││ │ ││
│  │                         │  │  └─────────────────────┘│ │ ││
│  │                         │  └─────────────────────────┘ │ ││
│  │                         └──────────────────────────────┘ ││
│  └──────────────────────────────────────────────────────────┘│
│                                                              │
│  ┌──────────────────────────────────────────────────────────┐│
│  │                   Route53                                ││
│  │  argocd.chinmayto.com → NGINX Ingress NLB                ││
│  │  app.chinmayto.com → NGINX Ingress NLB                   ││
│  └──────────────────────────────────────────────────────────┘│
└──────────────────────────────────────────────────────────────┘

Prerequisites

Before starting, ensure you have:

AWS CLI configured with appropriate permissions
Terraform installed (version >= 1.0)
kubectl installed
A registered domain name in Route53
Helm installed (version >= 3.0)

Implementation Steps

Step 1: Create VPC and EKS Cluster

We start by creating the foundational infrastructure using AWS community Terraform modules. First, let's define our variables:

# infrastructure/variables.tf
variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "cluster_name" {
  description = "Name of the EKS cluster"
  type        = string
  default     = "CT-EKS-Cluster"
}

variable "cluster_version" {
  description = "Kubernetes version for the EKS cluster"
  type        = string
  default     = "1.33"
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "public_subnet_cidrs" {
  description = "CIDR blocks for public subnets"
  type        = list(string)
  default     = ["10.0.1.0/24", "10.0.2.0/24"]
}

variable "private_subnet_cidrs" {
  description = "CIDR blocks for private subnets"
  type        = list(string)
  default     = ["10.0.10.0/24", "10.0.20.0/24"]
}

Now, let's create the VPC and EKS cluster:

# infrastructure/main.tf
####################################################################################
# Data source for availability zones
####################################################################################
data "aws_availability_zones" "available" {
  state = "available"
}

####################################################################################
### VPC Module Configuration
####################################################################################
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.cluster_name}-VPC"
  cidr = var.vpc_cidr

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  private_subnets = var.private_subnet_cidrs
  public_subnets  = var.public_subnet_cidrs

  enable_nat_gateway = true
  enable_vpn_gateway = false
  single_nat_gateway = true

  enable_dns_hostnames = true
  enable_dns_support   = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = "1"
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = "1"
  }

  tags = {
    Name      = "${var.cluster_name}-VPC"
    Terraform = "true"
  }
}

####################################################################################
###  EKS Cluster Module Configuration
####################################################################################
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = var.cluster_name
  cluster_version = var.cluster_version

  vpc_id                                   = module.vpc.vpc_id
  subnet_ids                               = module.vpc.private_subnets
  cluster_endpoint_public_access           = true
  enable_cluster_creator_admin_permissions = true

  # EKS Managed Node Groups
  eks_managed_node_groups = {
    EKS_Node_Group = {
      min_size     = 1
      max_size     = 3
      desired_size = 2

      instance_types = ["t3.medium"]
      capacity_type  = "ON_DEMAND"

      subnet_ids = module.vpc.private_subnets
    }
  }

  # EKS Add-ons
  cluster_addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
    eks-pod-identity-agent = {
      most_recent = true
    }
  }

  tags = {
    Name      = var.cluster_name
    Terraform = "true"
  }
}

####################################################################################
###  Null Resource to update the kubeconfig file
####################################################################################
resource "null_resource" "update_kubeconfig" {
  provisioner "local-exec" {
    command = "aws eks --region ${var.aws_region} update-kubeconfig --name ${var.cluster_name}"
  }

  depends_on = [module.eks]
}

Step 2: Deploy NGINX Ingress Controller

The NGINX Ingress Controller provides external access to our services through an AWS Network Load Balancer:

# infrastructure/nginx-ingress.tf
################################################################################
# Create ingress-nginx namespace
################################################################################
resource "kubernetes_namespace" "ingress_nginx" {
  metadata {
    name = "ingress-nginx"
    labels = {
      name = "ingress-nginx"
    }
  }
  depends_on = [module.eks]
}

################################################################################
# Install NGINX Ingress Controller using Helm
################################################################################
resource "helm_release" "nginx_ingress" {
  name       = "ingress-nginx"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
  namespace  = kubernetes_namespace.ingress_nginx.metadata[0].name
  version    = "4.8.3"

  values = [
    yamlencode({
      controller = {
        service = {
          type = "LoadBalancer"
          annotations = {
            "service.beta.kubernetes.io/aws-load-balancer-type"                              = "nlb"
            "service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled" = "true"
          }
        }
        metrics = {
          enabled = true
          serviceMonitor = {
            enabled = false
          }
        }
      }
    })
  ]

  depends_on = [kubernetes_namespace.ingress_nginx]
}

################################################################################
# Get the NLB hostname from nginx ingress controller
################################################################################
data "kubernetes_service" "nginx_ingress_controller" {
  metadata {
    name      = "ingress-nginx-controller"
    namespace = kubernetes_namespace.ingress_nginx.metadata[0].name
  }
  depends_on = [helm_release.nginx_ingress]
}

Step 3: Deploy ArgoCD Server

First, let's define the ArgoCD-specific variables:

# infrastructure/variables.tf (ArgoCD Variables)
variable "argocd_namespace" {
  description = "Kubernetes namespace for ArgoCD"
  type        = string
  default     = "argocd"
}

variable "argocd_chart_version" {
  description = "ArgoCD Helm chart version"
  type        = string
  default     = "5.51.6"
}

variable "argocd_hostname" {
  description = "Hostname for ArgoCD ingress"
  type        = string
  default     = "argocd.chinmayto.com"
}

variable "argocd_admin_password" {
  description = "Custom admin password for ArgoCD (leave empty for auto-generated)"
  type        = string
  default     = ""
  sensitive   = true
}

variable "domain_name" {
  description = "Domain name for the hosted zone"
  type        = string
  default     = "chinmayto.com"
}

variable "argocd_subdomain" {
  description = "Subdomain for ArgoCD"
  type        = string
  default     = "argocd"
}

Now, deploy ArgoCD using Helm with custom configuration for ingress access:

# infrastructure/argocd.tf
####################################################################################
### Route53 Hosted Zone
####################################################################################
data "aws_route53_zone" "main" {
  name         = var.domain_name
  private_zone = false
}

####################################################################################
### ArgoCD Namespace
####################################################################################
resource "kubernetes_namespace" "argocd" {
  metadata {
    name = var.argocd_namespace
  }

  depends_on = [module.eks]
}

####################################################################################
### ArgoCD Helm Release
####################################################################################
resource "helm_release" "argocd" {
  name       = "argocd"
  repository = "https://argoproj.github.io/argo-helm"
  chart      = "argo-cd"
  version    = var.argocd_chart_version
  namespace  = kubernetes_namespace.argocd.metadata[0].name

  values = [
    yamlencode({
      server = {
        service = {
          type = "ClusterIP"
        }
        ingress = {
          enabled          = true
          ingressClassName = "nginx"
          hosts            = [var.argocd_hostname]
          annotations = {
            "nginx.ingress.kubernetes.io/ssl-redirect"       = "false"
            "nginx.ingress.kubernetes.io/force-ssl-redirect" = "false"
            "nginx.ingress.kubernetes.io/backend-protocol"   = "HTTP"
          }
        }
      }
      configs = {
        params = {
          "server.insecure" = true
        }
      }
    })
  ]

  depends_on = [kubernetes_namespace.argocd]
}

####################################################################################
### ArgoCD Admin Password Secret (Optional - for custom password)
####################################################################################
resource "kubernetes_secret" "argocd_admin_password" {
  count = var.argocd_admin_password != "" ? 1 : 0

  metadata {
    name      = "argocd-initial-admin-secret"
    namespace = kubernetes_namespace.argocd.metadata[0].name
  }

  data = {
    password = bcrypt(var.argocd_admin_password)
  }

  depends_on = [kubernetes_namespace.argocd]
}

Step 4: Configure Route53 DNS

Set up DNS records to access ArgoCD via a custom subdomain:

# infrastructure/argocd.tf (continued)
####################################################################################
### Route53 DNS Record for ArgoCD (pointing to NGINX Ingress NLB)
####################################################################################
resource "aws_route53_record" "argocd" {
  zone_id = data.aws_route53_zone.main.zone_id
  name    = var.argocd_subdomain
  type    = "A"

  alias {
    name                   = data.kubernetes_service.nginx_ingress_controller.status.0.load_balancer.0.ingress.0.hostname
    zone_id                = "Z26RNL4JYFTOTI" # NLB zone ID for us-east-1
    evaluate_target_health = true
  }

  depends_on = [helm_release.argocd, data.kubernetes_service.nginx_ingress_controller]
}

For application DNS records, create a separate file:

# infrastructure/app-dns.tf
####################################################################################
### Route53 DNS Record for Node.js App (pointing to NGINX Ingress NLB)
####################################################################################
resource "aws_route53_record" "nodejs_app" {
  zone_id = data.aws_route53_zone.main.zone_id
  name    = var.app_subdomain
  type    = "A"

  alias {
    name                   = data.kubernetes_service.nginx_ingress_controller.status.0.load_balancer.0.ingress.0.hostname
    zone_id                = "Z26RNL4JYFTOTI" # NLB zone ID for us-east-1
    evaluate_target_health = true
  }

  depends_on = [helm_release.nginx_ingress]
}

Step 5: Create ArgoCD Project

Define an ArgoCD project to manage application deployments:

# argocd/project.yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: chinmayto-apps
  namespace: argocd
spec:
  description: Project for chinmayto applications
  sourceRepos:
    - 'https://github.com/chinmayto/terraform-aws-eks-argocd.git'
  destinations:
    - namespace: '*'
      server: https://kubernetes.default.svc
  clusterResourceWhitelist:
    - group: '*'
      kind: '*'
  namespaceResourceWhitelist:
    - group: '*'
      kind: '*'

Step 6: Implement App-of-Apps Pattern

Create a root application that manages other applications:

# argocd/app-of-apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: app-of-apps
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/chinmayto/terraform-aws-eks-argocd.git
    targetRevision: HEAD
    path: argocd
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Create individual application definitions:

# argocd/nodejs-app-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nodejs-app
  namespace: argocd
spec:
  project: chinmayto-apps
  source:
    repoURL: https://github.com/chinmayto/terraform-aws-eks-argocd.git
    targetRevision: HEAD
    path: k8s-manifests
  destination:
    server: https://kubernetes.default.svc
    namespace: simple-nodejs-app
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Deployment Instructions

1. Deploy Infrastructure

cd infrastructure
terraform init
terraform plan
terraform apply -auto-approve

2. Verify Deployment

# Check EKS cluster
kubectl get nodes

# Check ArgoCD pods
kubectl get pods -n argocd

# Check NGINX Ingress
kubectl get svc -n ingress-nginx

# Get ArgoCD admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

3. Access ArgoCD UI

Navigate to https://argocd.yourdomain.com and login with:

Username: admin
Password: (from step 4 above)

4. Deploy Applications

Apply the ArgoCD application manifests:

kubectl apply -f argocd/app-of-apps.yaml

Testing

Accessing argocd using the domain: http://argocd.chinmayto.com

Deployed app-of-apps:

Deployed application with components:

$ kubectl get all -n simple-nodejs-app
NAME                                        READY   STATUS    RESTARTS   AGE        
pod/deployment-nodejs-app-bfb4c4d56-pdwtt   1/1     Running   0          10m        
pod/deployment-nodejs-app-bfb4c4d56-rfxfq   1/1     Running   0          10m        

NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/service-nodejs-app   ClusterIP   172.20.120.15   <none>        80/TCP    10m

NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE        
deployment.apps/deployment-nodejs-app   2/2     2            2           10m        

NAME                                              DESIRED   CURRENT   READY   AGE   
replicaset.apps/deployment-nodejs-app-bfb4c4d56   2         2         2       10m

Update the k8s-manifests, commit the code and see argocd picking up the changes and deploying them.

Deployment complete:

$ kubectl get all -n simple-nodejs-app
NAME                                         READY   STATUS    RESTARTS   AGE
pod/deployment-nodejs-app-66684f5945-fdwqp   1/1     Running   0          2m42s
pod/deployment-nodejs-app-66684f5945-h6pv9   1/1     Running   0          2m16s
pod/deployment-nodejs-app-66684f5945-tfs45   1/1     Running   0          2m30s

NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/service-nodejs-app   ClusterIP   172.20.120.15   <none>        80/TCP    13m

NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/deployment-nodejs-app   3/3     3            3           13m

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/deployment-nodejs-app-66684f5945   3         3         3       2m43s
replicaset.apps/deployment-nodejs-app-bfb4c4d56    0         0         0       13m

Cleanup Steps

# Verify applications present
kubectl get applications -n argocd

# Delete specific applications by name (NEVER use --all with projects)
kubectl delete application app-of-apps -n argocd
kubectl delete application nodejs-app -n argocd

# Verify applications are deleted
kubectl get applications -n argocd

# Verify custom projects
kubectl get appprojects -n argocd

# Only delete the custom project, NEVER delete default project
kubectl delete appproject chinmayto-apps -n argocd

# Verify only custom project is deleted (default should remain)
kubectl get appprojects -n argocd

# Delete the resources and namespace for application
kubectl delete all --all-namespaces -l argocd.argoproj.io/instance=nodejs-app
kubectl delete ns -l argocd.argoproj.io/instance=nodejs-app

# Destroy terraform infrastructure
cd infrastructure
terraform destroy -auto-approve

Conclusion

This implementation demonstrates a production-ready GitOps workflow using ArgoCD on Amazon EKS. By following this guide, you've created:

A secure, scalable Kubernetes cluster on AWS
Automated application deployment pipeline
Centralized application management through ArgoCD
DNS-based access to your GitOps platform

The GitOps approach with ArgoCD provides numerous benefits including improved deployment reliability, enhanced security through Git-based workflows, and simplified application lifecycle management. This foundation can be extended to support multiple environments, advanced deployment strategies, and comprehensive monitoring solutions.

References and Further Reading

GitHub Repository: terraform-aws-eks-argocd
ArgoCD Documentation: https://argo-cd.readthedocs.io/

Deploying Applications to Amazon ECS Using GitHub Actions CI/CD

Chinmay Tonape — Sun, 28 Sep 2025 08:41:09 +0000

In the earlier module, we saw how to create an Amazon ECS cluster with a task definition using AWS Community Terraform modules. Now, let’s take the next step—automating application deployment using GitHub Actions CI/CD.

By the end of this blog, you’ll understand how to set up a complete CI/CD pipeline for deploying new versions of your application to ECS, all triggered by source code changes in your repository.

What We’ll Cover

I’ve created GitHub workflows that handle the following:

initial-ecr-setup.yml – Create an Amazon ECR public repository, build the Docker image, and push it to ECR.
deploy-infrastructure.yml – Build the ECS infrastructure (cluster, services, task definitions, ALB) using Terraform.
build-and-deploy.yml – Deploy an updated version of the app whenever source code changes are pushed.
deploy-infrastructure.yml – Destroy ECS infrastructure using Terraform.

In this blog, we’ll mainly focus on the CI/CD workflow for application version upgrades.

CI/CD Pipeline for ECS Deployment

When a new version of your application is committed to the repository, the following GitHub Actions workflow ensures it is automatically built and deployed to ECS.

Secrets in GitHub Actions

For the above workflows, you’ll need to store sensitive information as GitHub Secrets. Navigate to your repository → Settings → Secrets and variables → Actions and add:

AWS_ACCESS_KEY_ID – IAM user or role access key.
AWS_SECRET_ACCESS_KEY – Corresponding secret key.

This ensures that no credentials are exposed in your workflow files.

The DevOps IAM user that will create the ECS infrastructure and run CICD pipelines will need following accesses. It will also need access to terraform state bucket and dynamodb table used for terraform state locking. Following Principle of least privilege is the need of the day!

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr-public:*",
                "ecr:*",
                "ecs:*",
                "ec2:*",
                "elasticloadbalancing:*",
                "logs:*",
                "iam:*",
                "application-autoscaling:*",
                "sts:GetServiceBearerToken",
                "sts:GetCallerIdentity"
            ],
            "Resource": "*"
        }
    ]
}

Initial Setup

To create the public ECR repository and push initial verison of the application to it use initial-ecr-setup.yml workflow and then use deploy-infrastructure.yml workflow to create ECS infrastructure with baseic version of the application.

Step 1: Build and Push Docker Image

The workflow first builds a new Docker image from your application source code and tags it with the commit SHA or a version tag. The image is then pushed to the Amazon ECR public repository.
This ensures that ECS always uses the latest available image for deployment.

name: Build and Deploy Application

on:
  push:
    paths:
      - 'app/**'
    branches:
      - main
      - develop
  pull_request:
    paths:
      - 'app/**'
    branches:
      - main

jobs:
  build-and-push:
    name: Build and Push to ECR
    runs-on: ubuntu-latest
    outputs:
      image-tag: ${{ steps.build-image.outputs.image-tag }}
      image-uri: ${{ steps.build-image.outputs.image-uri }}

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ${{ env.AWS_REGION }}

    - name: Login to Amazon ECR Public
      id: login-ecr-public
      uses: aws-actions/amazon-ecr-login@v2
      with:
        registry-type: public
        mask-password: 'true'

    - name: Build and push Docker image
      id: build-image
      working-directory: ./app
      run: |
        # Generate image tag using git commit SHA and timestamp
        IMAGE_TAG="${GITHUB_SHA:0:8}-$(date +%s)"
        IMAGE_URI="$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

        echo "Building image: $IMAGE_URI"

        # Build the Docker image
        docker build -t $IMAGE_URI .

        # Push the image to ECR
        docker push $IMAGE_URI

        # Also tag and push as latest
        docker tag $IMAGE_URI $ECR_REGISTRY/$ECR_REPOSITORY:latest
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest

Step 2: Update ECS Task Definition

ECS services run based on task definitions. To deploy a new version, we first fetch the current task definition JSON using AWS CLI. The workflow updates the container image inside the task definition JSON with the newly built image from Step 1.

    - name: Download current task definition
      run: |
        aws ecs describe-task-definition \
          --task-definition $TASK_DEFINITION \
          --query taskDefinition > task-definition.json

    - name: Update task definition with new image
      id: task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-definition.json
        container-name: ${{ env.CONTAINER_NAME }}
        image: ${{ needs.build-and-push.outputs.image-uri }}

Step 3: Deploy to ECS Service

The new task definition is deployed to ECS using aws-actions/amazon-ecs-deploy-task-definition.

    - name: Deploy to Amazon ECS service
      uses: aws-actions/amazon-ecs-deploy-task-definition@v2
      with:
        task-definition: ${{ steps.task-def.outputs.task-definition }}
        service: ${{ env.ECS_SERVICE }}
        cluster: ${{ env.ECS_CLUSTER }}
        wait-for-service-stability: true

Accessing the Application

Once the workflow finishes, navigate to the Application Load Balancer (ALB) DNS name associated with your ECS service. You should see the new version of the application running seamlessly.

Initial ECR setup and push image:

Build ECS infrastructure:

Access initial version of application using ALB:

Now update the code, do git push and watch the build and deploy workflow run automatically!:

New image built and pushed to ECR:

ECS Service Deployemnt in progress:

Deployment complete!

Yay!!

Cleanup

When you’re done testing, you can destroy the ECS infrastructure using the Terraform cleanup workflow in destroy-infrastructure.yml

Conclusion

We successfully set up a CI/CD pipeline using GitHub Actions to deploy applications to Amazon ECS. With this automation in place:

Developers just push code, and deployment happens automatically.
ECS services always stay up-to-date with the latest version of your app.
Infrastructure and deployments are consistent, repeatable, and easy to clean up.

This approach enables faster iteration cycles and ensures your applications remain production-ready.

References

Deploying a Simple App on ECS with Fargate & Terraform using AWS Community Modules

Chinmay Tonape — Fri, 26 Sep 2025 12:10:20 +0000

Amazon Elastic Container Service (ECS) makes it easy to run containerized applications on AWS without managing servers. With AWS Fargate and Fargate Spot, you can run workloads in a serverless fashion while optimizing costs.

In this guide, we’ll use Terraform AWS Community Modules to provision the infrastructure and deploy a simple Node.js app hosted in a public Amazon ECR repository.

Architecture

At the core, we provision a VPC with both public and private subnets across multiple Availability Zones for high availability. The ECS Cluster itself is a regional construct and does not directly reside in the VPC. Instead, the actual workloads — called ECS tasks — run inside the private subnets of our VPC. Each ECS task is deployed with security groups to control inbound and outbound traffic.

To make the application accessible to users on the internet, we use an Application Load Balancer (ALB). The ALB is deployed in the public subnets, where it can receive internet traffic. It forwards requests to a target group that points to the ECS tasks running in the private subnets. This separation of public and private networking ensures security while still allowing external access through the load balancer.

The ECS task definition specifies the Node.js container image (stored in a public Amazon ECR repository), along with CPU and memory settings. It also defines port mappings so the application can listen on port 8080. The ECS service ensures the desired number of tasks are always running, and it integrates with the ALB to register and deregister tasks dynamically.

For scalability, we configure autoscaling policies based on CPU and memory utilization. This means if the load increases, ECS will automatically launch more Fargate or Fargate Spot tasks to handle the traffic, and scale down during low demand to save costs.

Step 1: Create VPC and ECS Cluster with Terraform

We’ll use the AWS community VPC and ECS modules.

################################################################################
# VPC Module
################################################################################
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.project_name}-vpc"
  cidr = var.vpc_cidr

  azs             = var.availability_zones
  private_subnets = var.private_subnets
  public_subnets  = var.public_subnets

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_vpn_gateway   = false
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = var.common_tags
}
################################################################################
# ECS Cluster Module
################################################################################
module "ecs_cluster" {
  source  = "terraform-aws-modules/ecs/aws//modules/cluster"
  version = "~> 5.0"

  cluster_name = "${var.project_name}-cluster"

  # Fargate capacity providers
  fargate_capacity_providers = {
    FARGATE = {
      default_capacity_provider_strategy = {
        weight = 50
        base   = 20
      }
    }
    FARGATE_SPOT = {
      default_capacity_provider_strategy = {
        weight = 50
      }
    }
  }

  tags = var.common_tags
}

Step 2: Create ECS Execution & Task Roles

We need IAM roles so ECS can pull images from ECR and write logs.

################################################################################
# ECS Execution Role
################################################################################
resource "aws_iam_role" "ecs_execution_role" {
  name = "${var.project_name}-ecs-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ecs-tasks.amazonaws.com"
        }
      }
    ]
  })

  tags = var.common_tags
}

resource "aws_iam_role_policy_attachment" "ecs_execution_role_policy" {
  role       = aws_iam_role.ecs_execution_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

################################################################################
# Additional policy for ECR public registry access
################################################################################
resource "aws_iam_role_policy" "ecs_execution_ecr_policy" {
  name = "${var.project_name}-ecs-execution-ecr-policy"
  role = aws_iam_role.ecs_execution_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ecr-public:GetAuthorizationToken",
          "ecr-public:BatchCheckLayerAvailability",
          "ecr-public:GetDownloadUrlForLayer",
          "ecr-public:BatchGetImage"
        ]
        Resource = "*"
      }
    ]
  })
}

################################################################################
# ECS Task Role
################################################################################
resource "aws_iam_role" "ecs_task_role" {
  name = "${var.project_name}-ecs-task-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ecs-tasks.amazonaws.com"
        }
      }
    ]
  })

  tags = var.common_tags
}

################################################################################
# CloudWatch Logs Group
################################################################################
resource "aws_cloudwatch_log_group" "ecs_logs" {
  name              = "/ecs/${var.project_name}"
  retention_in_days = var.log_retention_days

  tags = var.common_tags
}

Step 3: Create ALB with Target Group

The ALB will live in public subnets and route traffic to ECS tasks running in private subnets.

################################################################################
# Security Group for ALB
################################################################################
resource "aws_security_group" "alb" {
  name_prefix = "${var.project_name}-alb"
  vpc_id      = module.vpc.vpc_id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = merge(var.common_tags, {
    Name = "${var.project_name}-alb-sg"
  })
}
################################################################################
# Application Load Balancer
################################################################################
resource "aws_lb" "main" {
  name               = "${var.project_name}-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb.id]
  subnets            = module.vpc.public_subnets

  enable_deletion_protection = false

  tags = var.common_tags
}
################################################################################
# Target Group
################################################################################
resource "aws_lb_target_group" "ecs_tg" {
  name        = "${var.project_name}-tg"
  port        = var.app_port
  protocol    = "HTTP"
  vpc_id      = module.vpc.vpc_id
  target_type = "ip"

  health_check {
    enabled             = true
    healthy_threshold   = 2
    interval            = 30
    matcher             = "200"
    path                = "/"
    port                = "traffic-port"
    protocol            = "HTTP"
    timeout             = 5
    unhealthy_threshold = 2
  }

  tags = var.common_tags
}
################################################################################
# ALB Listener
################################################################################
resource "aws_lb_listener" "web" {
  load_balancer_arn = aws_lb.main.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.ecs_tg.arn
  }

  tags = var.common_tags
}

Step 4: ECS Task Definition with Node.js Image

We’ll use a public ECR image of our Node.js app.

################################################################################
# ECS Task Definition
################################################################################
resource "aws_ecs_task_definition" "nodejs_app" {
  family                   = "${var.project_name}-task"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = tostring(var.task_cpu)
  memory                   = tostring(var.task_memory)
  execution_role_arn       = aws_iam_role.ecs_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = jsonencode([
    {
      name  = "nodejs-app"
      image = var.ecr_repository_url

      portMappings = [
        {
          containerPort = var.app_port
          protocol      = "tcp"
        }
      ]

      logConfiguration = {
        logDriver = "awslogs"
        options = {
          "awslogs-group"         = aws_cloudwatch_log_group.ecs_logs.name
          "awslogs-region"        = var.aws_region
          "awslogs-stream-prefix" = "ecs"
        }
      }

      environment = [
        {
          name  = "NODE_ENV"
          value = "production"
        },
        {
          name  = "PORT"
          value = tostring(var.app_port)
        }
      ]

      essential = true
    }
  ])

  tags = var.common_tags
}

Step 5: ECS Service with Autoscaling

Attach the ECS service to the ALB target group and configure autoscaling.

################################################################################
# Security Group for ECS Service
################################################################################
resource "aws_security_group" "ecs_service" {
  name_prefix = "${var.project_name}-ecs-service"
  vpc_id      = module.vpc.vpc_id

  ingress {
    from_port       = var.app_port
    to_port         = var.app_port
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = merge(var.common_tags, {
    Name = "${var.project_name}-ecs-service-sg"
  })
}
################################################################################
# ECS Service
################################################################################
module "ecs_service" {
  source  = "terraform-aws-modules/ecs/aws//modules/service"
  version = "~> 5.0"

  name        = "${var.project_name}-service"
  cluster_arn = module.ecs_cluster.arn

  # Use the separate task definition
  task_definition_arn            = aws_ecs_task_definition.nodejs_app.arn
  desired_count                  = var.min_capacity
  ignore_task_definition_changes = false
  force_new_deployment           = true
  create_task_definition         = false

  # Built-in Auto Scaling
  autoscaling_min_capacity = var.min_capacity
  autoscaling_max_capacity = var.max_capacity

  autoscaling_policies = {
    cpu = {
      policy_type = "TargetTrackingScaling"
      target_tracking_scaling_policy_configuration = {
        predefined_metric_specification = {
          predefined_metric_type = "ECSServiceAverageCPUUtilization"
        }
        target_value       = var.cpu_target_value
        scale_in_cooldown  = var.scale_in_cooldown
        scale_out_cooldown = var.scale_out_cooldown
      }
    }
    memory = {
      policy_type = "TargetTrackingScaling"
      target_tracking_scaling_policy_configuration = {
        predefined_metric_specification = {
          predefined_metric_type = "ECSServiceAverageMemoryUtilization"
        }
        target_value       = var.memory_target_value
        scale_in_cooldown  = var.scale_in_cooldown
        scale_out_cooldown = var.scale_out_cooldown
      }
    }
  }

  load_balancer = {
    service = {
      target_group_arn = aws_lb_target_group.ecs_tg.arn
      container_name   = "nodejs-app"
      container_port   = var.app_port
    }
  }

  subnet_ids = module.vpc.private_subnets
  security_group_rules = {
    alb_ingress = {
      type                     = "ingress"
      from_port                = var.app_port
      to_port                  = var.app_port
      protocol                 = "tcp"
      description              = "Service port"
      source_security_group_id = aws_security_group.alb.id
    }
    egress_all = {
      type        = "egress"
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
    }
  }

  service_tags = var.common_tags
}

Accessing the Application

Once deployed, Terraform will output the ALB DNS name. Open it in your browser:

Service updated to have 10 tasks:

ALB target group showing 10 IP addresse:

Cleanup

When finished, always clean up to avoid unnecessary costs:

terraform destory

Conclusion

We successfully deployed a Node.js application to ECS with Fargate and Fargate Spot using Terraform community modules.
This setup ensures:

Scalability with autoscaling policies
Cost efficiency with Fargate Spot
Security by running tasks in private subnets behind an ALB

Using Terraform makes the process repeatable, modular, and version-controlled.

References

My GitHub Repo

Terraform AWS VPC Module

Terraform AWS ECS Module

Amazon ECS Documentation

Observability on Amazon EKS Cluster: A Complete Guide to Prometheus and Grafana with Helm

Chinmay Tonape — Sun, 21 Sep 2025 18:30:08 +0000

When deploying applications on Amazon Elastic Kubernetes Service (EKS), ensuring that you can observe, monitor, and troubleshoot workloads effectively is crucial. This is where observability comes into play. In this blog, we’ll set up an EKS cluster using AWS Community Terraform modules, install Prometheus and Grafana for monitoring and visualization, configure Route53 DNS records, and deploy a sample microservices application (Voting App).## Why Prometheus and Grafana?

What is Observability in EKS?

Observability in Kubernetes (and EKS) refers to the ability to understand the internal state of your cluster and applications by analyzing the outputs such as logs, metrics, and traces. It goes beyond traditional monitoring:

Monitoring answers “Is my system working?”
Observability answers “Why is my system not working?”

In Kubernetes on AWS EKS, observability provides insights into:

Cluster health (nodes, pods, deployments, daemonsets, etc.)
Application performance (latency, error rates, resource utilization)
Infrastructure metrics (EC2, VPC, networking, EBS/EFS storage, etc.)
User experience tracking (via service-level objectives and traces)

Key pillars of observability:

Metrics – quantitative data collected over time (CPU, memory, request counts).
Logs – textual event records (pod logs, system logs).
Traces – request paths across microservices (distributed tracing).

By combining these three, DevOps teams can detect, debug, and optimize Kubernetes workloads effectively.

What is Prometheus and Grafana

Prometheus is an open-source monitoring and alerting toolkit designed for reliability and scalability. It collects time-series metrics by periodically scraping targets such as Kubernetes nodes, pods, services, and applications. Using its query language PromQL, Prometheus enables detailed analysis of cluster performance, application behavior, and infrastructure health. It also supports alerting rules, making it useful for proactive issue detection.

Grafana is an open-source visualization and analytics platform that integrates with data sources like Prometheus. It allows users to create interactive dashboards and visualizations for monitoring metrics in real time. Grafana turns raw data into meaningful insights by providing charts, graphs, and alerts that help teams detect anomalies and optimize performance.

Together, Prometheus and Grafana form a powerful observability stack on EKS. Prometheus handles metrics collection and storage, while Grafana provides visualization and analysis. This combination allows Kubernetes operators and developers to not only monitor cluster health but also understand why issues occur, enabling faster debugging and better capacity planning.

Architecture Overview

Our monitoring architecture consists of several key components:

┌─────────────────────────────────────────────────────────────┐
│                        EKS Cluster                          │
│                                                             │
│  ┌─────────────────┐    ┌─────────────────┐                 │
│  │   Prometheus    │◄───┤  Node Exporter  │                 │
│  │     Server      │    │                 │                 │
│  │                 │    └─────────────────┘                 │
│  │  - Scrapes      │                                        │
│  │  - Stores       │    ┌─────────────────┐                 │
│  │  - Alerts       │◄───┤ kube-state-     │                 │
│  └─────────────────┘    │ metrics         │                 │
│           │             └─────────────────┘                 │
│           │                                                 │
│           ▼              ┌─────────────────┐                │
│  ┌─────────────────┐◄────┤   Application   │                │
│  │    Grafana      │     │    Metrics      │                │
│  │                 │     └─────────────────┘                │
│  │  - Dashboards   │                                        │
│  │  - Visualization│     ┌─────────────────┐                │
│  │  - Alerting     │◄────┤  AlertManager   │                │
│  └─────────────────┘     │                 │                │
│                          └─────────────────┘                │
└─────────────────────────────────────────────────────────────┘

Architecture of Prometheus and Grafana on EKS

When deploying Prometheus and Grafana on EKS, the architecture typically looks like this:

Prometheus runs inside the cluster and is responsible for:
- Scraping metrics from Kubernetes components, node exporters, pods, and application endpoints.
- Storing time-series data in its own storage backend.
- Exposing APIs that allow querying using PromQL.
Grafana runs alongside Prometheus and is responsible for:
- Connecting to Prometheus as a data source.
- Querying metrics with PromQL queries.
- Creating dashboards that visualize cluster health, workload performance, and application-specific data.

Data Flow:

Kubernetes Metrics Sources:

Kubelet – node and pod metrics.
Kube-state-metrics – state of Kubernetes objects (deployments, pods, nodes, services).
cAdvisor – container-level CPU, memory, filesystem, and network usage.
ETCD metrics – key-value store performance.
Application endpoints – any service exposing /metrics in Prometheus format.

Prometheus Scraping:

Prometheus periodically queries these endpoints using service discovery (Kubernetes API, annotations, labels).
Metrics are collected and stored as time-series data.

Grafana Visualization:

Grafana queries Prometheus using PromQL.
Dashboards are built to visualize CPU usage, memory utilization, pod restarts, latency, and custom application metrics.
Dashboards can be customized or imported from Grafana’s community templates.

Step 1: Create EKS Cluster using AWS Community Terraform Modules

The first step in integrating Amazon EFS with EKS is to provision a Kubernetes cluster that runs securely inside a dedicated VPC. We will use the widely adopted AWS Terraform community modules for both the VPC and EKS setup. Please refer to main module of GitHub repo.

Step 2: Add Helm Repositories

First, let's add the necessary Helm repositories:

# Add Prometheus community Helm repository
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

# Add Grafana Helm repository  
helm repo add grafana https://grafana.github.io/helm-charts

# Add Nginx Ingress Helm repository  
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

# Update repositories
helm repo update

Step 3: Install NGINX Ingress Controller with Helm

To expose Prometheus, Grafana, and other applications securely outside the cluster, we need an Ingress Controller. We’ll use the NGINX Ingress Controller, installed via Helm in its own namespace.

################################################################################
# Create ingress-nginx namespace
################################################################################
resource "kubernetes_namespace" "ingress_nginx" {
  metadata {
    name = "ingress-nginx"
    labels = {
      name = "ingress-nginx"
    }
  }
  depends_on = [module.eks]
}

################################################################################
# Install NGINX Ingress Controller using Helm
################################################################################
resource "helm_release" "nginx_ingress" {
  name       = "ingress-nginx"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
  namespace  = kubernetes_namespace.ingress_nginx.metadata[0].name
  version    = "4.8.3"

  values = [
    yamlencode({
      controller = {
        service = {
          type = "LoadBalancer"
          annotations = {
            "service.beta.kubernetes.io/aws-load-balancer-type"                              = "nlb"
            "service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled" = "true"
          }
        }
        metrics = {
          enabled = true
          serviceMonitor = {
            enabled = false
          }
        }
      }
    })
  ]

  depends_on = [kubernetes_namespace.ingress_nginx]
}

################################################################################
# Get the NLB hostname from nginx ingress controller
################################################################################
data "kubernetes_service" "nginx_ingress_controller" {
  metadata {
    name      = "ingress-nginx-controller"
    namespace = kubernetes_namespace.ingress_nginx.metadata[0].name
  }
  depends_on = [helm_release.nginx_ingress]
}

Step 4: Create Route53 Records for Prometheus and Grafana

To make these tools accessible via a domain, configure Route53 DNS records. This allows you to access Prometheus at https://prometheus.chinmayto.com and Grafana at https://grafana.chinmayto.com.

################################################################################
# Get Route53 hosted zone for chinmayto.com
################################################################################
data "aws_route53_zone" "main" {
  name         = "chinmayto.com"
  private_zone = false
}

################################################################################
# Create Route53 A records for Prometheus and Grafana
################################################################################
resource "aws_route53_record" "prometheus" {
  zone_id = data.aws_route53_zone.main.zone_id
  name    = "prometheus.chinmayto.com"
  type    = "A"

  alias {
    name                   = data.kubernetes_service.nginx_ingress_controller.status.0.load_balancer.0.ingress.0.hostname
    zone_id                = "Z26RNL4JYFTOTI" # NLB zone ID for us-east-1
    evaluate_target_health = true
  }

  depends_on = [helm_release.nginx_ingress]
}

resource "aws_route53_record" "grafana" {
  zone_id = data.aws_route53_zone.main.zone_id
  name    = "grafana.chinmayto.com"
  type    = "A"

  alias {
    name                   = data.kubernetes_service.nginx_ingress_controller.status.0.load_balancer.0.ingress.0.hostname
    zone_id                = "Z26RNL4JYFTOTI" # NLB zone ID for us-east-1
    evaluate_target_health = true
  }

  depends_on = [helm_release.nginx_ingress]
}

Step 5: Install Prometheus Stack

We’ll use the Terraform Helm provider to install both Prometheus (for metrics collection) and Grafana (for visualization) in its own namespace.

The kube-prometheus-stack Helm chart is an umbrella chart maintained by the Prometheus community. It includes Prometheus, Grafana, and kube-state-metrics all bundled together. That’s why deploying just this chart is usually sufficient for Kubernetes observability.

################################################################################
# Create monitoring namespace
################################################################################
resource "kubernetes_namespace" "monitoring" {
  metadata {
    name = "monitoring"
    labels = {
      name = "monitoring"
    }
  }
  depends_on = [module.eks]
}

################################################################################
# Install Prometheus using Helm (after nginx ingress)
################################################################################
resource "helm_release" "prometheus" {
  name       = "prometheus"
  repository = "https://prometheus-community.github.io/helm-charts"
  chart      = "kube-prometheus-stack"
  namespace  = kubernetes_namespace.monitoring.metadata[0].name
  version    = "55.5.0"

  values = [
    yamlencode({
      prometheus = {
        prometheusSpec = {
          retention = "30d"
        }
        service = {
          type = "ClusterIP"
        }
        ingress = {
          enabled          = true
          ingressClassName = "nginx"
          hosts            = ["prometheus.chinmayto.com"]
          paths            = ["/"]
          annotations = {
            "nginx.ingress.kubernetes.io/rewrite-target" = "/"
          }
        }
      }
      grafana = {
        enabled       = true
        adminPassword = "admin123"
        service = {
          type = "ClusterIP"
        }
        ingress = {
          enabled          = true
          ingressClassName = "nginx"
          hosts            = ["grafana.chinmayto.com"]
          path             = "/"
          annotations = {
            "nginx.ingress.kubernetes.io/rewrite-target" = "/"
          }
        }
        persistence = {
          enabled = false
        }
      }
      alertmanager = {
        enabled = true
      }
    })
  ]

  depends_on = [kubernetes_namespace.monitoring, helm_release.nginx_ingress]
}

Step 6: Deploy Sample Microservices Application (Voting App)

Now that observability is in place, let’s deploy a sample microservices app: the Voting App (a simple app with a frontend, backend, and database).

Apply manifests:

cd example-voting-app/k8s-specifications
kubectl apply -f .

Accessing Your Monitoring Stack

You can access Prometheus at https://prometheus.chinmayto.com and Grafana at https://grafana.chinmayto.com.

Validation Steps

1. Verify Prometheus Targets

In Prometheus UI:

Go to Status → Targets
Ensure all targets are "UP"
Check for any scraping errors

2. Test Prometheus Queries

Try these sample queries in Prometheus:

# CPU usage by node
100 - (avg by (instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# Memory usage
(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100

# Pod count by namespace
count by (namespace) (kube_pod_info)

3. Verify Grafana Dashboards

Grafana comes with pre-built dashboards:

Navigate to Dashboards
Check "Kubernetes / Compute Resources / Cluster"
Verify data is displaying correctly

Few Best Grafana Dashboards for EKS Monitoring

Grafana provides a wide range of prebuilt dashboards for Kubernetes and Prometheus. Some of the most popular ones for EKS monitoring include:

Kubernetes / Compute Resources Cluster (ID: 315): Visualizes cluster-wide CPU and memory usage.
Kubernetes / Compute Resources Namespace (Workloads) (ID: 3146): Breaks down metrics by namespace, useful for multi-team environments.
Kubernetes / Compute Resources Pod (ID: 7633): Pod-level monitoring (CPU, memory, restarts).
Kubernetes API Server Metrics (ID: 12006): Helps track API server performance, latency, and error rates.
Node Exporter Full (ID: 1860): Deep-dive into node-level metrics such as disk I/O, CPU, memory, and network.
ETCD Metrics (ID: 3070): Monitoring the control plane’s key-value store.
Kube-State-Metrics Dashboards (ID: 13332): Tracks states of Kubernetes objects like deployments, daemonsets, jobs, and cronjobs. These dashboards can be directly imported into Grafana using their Dashboard IDs from Grafana Labs Dashboard Repository

Cleanup

Delete the k8s resources created

cd example-voting-app/k8s-specifications
kubectl delete -f .

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

Setting up Prometheus and Grafana on EKS provides powerful observability into your Kubernetes clusters. This monitoring stack enables you to:

Track cluster and application performance
Set up proactive alerting
Visualize metrics through beautiful dashboards
Make data-driven decisions about scaling and optimization

The combination of Prometheus's robust metric collection and Grafana's visualization capabilities creates a comprehensive monitoring solution that scales with your infrastructure needs.

Remember to regularly review and update your monitoring configuration as your applications and infrastructure evolve. Monitoring is not a "set it and forget it" solution – it requires ongoing attention and refinement.

References

Kubernetes Storage Playlist - Part 5: Choosing the Right Storage for EKS: EBS vs. EFS vs. S3

Chinmay Tonape — Thu, 11 Sep 2025 02:30:01 +0000

When running applications on Amazon EKS (Elastic Kubernetes Service), one of the first architectural questions you’ll face is: Which storage solution should I use?

Picking the right storage isn’t just about saving data—it directly impacts scalability, reliability, and cost-efficiency of your workloads. AWS provides multiple storage services that integrate with Kubernetes through CSI drivers, and each serves a different purpose.

In this post, we’ll compare the three primary options — Amazon EBS, Amazon EFS, and Amazon S3 (via the Mountpoint CSI driver) — to help you choose the right fit for your workloads.

Amazon EBS (Elastic Block Store)

Amazon EBS is block-level storage attached to a specific EC2 instance or pod through the EBS CSI driver. Think of it like attaching a hard drive to your VM — it’s fast, consistent, and designed for single-pod workloads.

Best suited for:

Relational & NoSQL Databases (PostgreSQL, MySQL, Cassandra)
Stateful applications needing low-latency, high IOPS
Single-instance apps requiring persistence across pod restarts

Key Characteristics:

AZ-bound: An EBS volume lives in one Availability Zone, and pods must be scheduled in the same AZ.
Single-pod access: Most volumes are ReadWriteOnce (RWO) — meaning one pod at a time.
Dynamic provisioning: Kubernetes can auto-create EBS volumes using PersistentVolumeClaims.

If your workload is a database or something that needs dedicated, fast disk storage, EBS is the go-to.

Amazon EFS (Elastic File System)

Amazon EFS is a fully managed, elastic NFS storage that multiple pods can mount simultaneously — even across different Availability Zones. With the EFS CSI driver, it becomes a shared, scalable storage layer for Kubernetes workloads.

Best suited for:

Web content management (WordPress, Drupal)
CI/CD pipelines (shared build artifacts, code repos)
Applications needing shared data across multiple pods
Persistent storage with ReadWriteMany (RWX) access

Key Characteristics:

Shared access: Thousands of pods can read/write to the same file system.
Elastic & serverless: Automatically grows/shrinks with usage.
Multi-AZ durability: Data is stored across multiple AZs for resilience.

If you need shared storage accessible by many pods at once, EFS is the right choice.

Amazon S3 with Mountpoint CSI Driver

Amazon S3 is object storage, but with the Mountpoint for S3 CSI driver, you can mount an S3 bucket into your pod as if it were a file system. This is great for workloads that want S3’s scale and durability but expect a file system interface.

Best suited for:

Data analytics & ML jobs reading/writing large datasets
Centralized log aggregation from multiple pods
Backups & archival storage
Media processing and high-throughput workloads

Key Characteristics:

Object store with file-like access: File operations are translated into S3 API calls.
High throughput: Great for parallel data processing.
Static provisioning only: You can’t dynamically create S3 buckets — only mount existing ones.
Not POSIX-compliant: Doesn’t support all file operations like EBS/EFS.

If your workload is data-intensive and benefits from S3’s scale and economics, the Mountpoint driver is an excellent fit.

Feature	Amazon EBS	Amazon EFS	Amazon S3
Type	Block storage	File system (NFS)	Object storage (via CSI)
Access Mode	Single pod (RWO)	Multi-pod (RWX)	Multi-pod (limited)
Performance	Low latency, high IOPS	Shared, elastic, multi-AZ	High throughput, sequential
Use Case	Databases, stateful apps	Shared web content, CI/CD, dev tools	Data analytics, logs, backups
Provisioning	Static & dynamic	Static & dynamic	Static only
AZ Scope	Single AZ	Multi-AZ	Regional (bucket-level)

Conclusion

Choosing the right storage solution in EKS depends on your workload needs:

Use EBS for high-performance, single-pod workloads like databases.
Use EFS for shared, scalable storage where multiple pods need simultaneous access.
Use S3 (Mountpoint) when you need massive, cost-effective object storage for data-heavy applications.

By aligning your storage choice with your workload requirements, you’ll build an EKS architecture that’s resilient, cost-efficient, and scalable.

References

Refer to my detailed blogs for implementing each of the storage type

AWS Storage with EFS

AWS Storage with EBS

AWS Storage with S3 with Mountpoint

Building an Amazon EKS Cluster with raw Terraform Resources

Chinmay Tonape — Wed, 10 Sep 2025 18:55:42 +0000

Most guides for provisioning Amazon EKS use the AWS community Terraform module for convenience. While this is great for speed, it often abstracts away what’s actually happening behind the scenes.

In this blog, we will build an EKS cluster from scratch using raw Terraform resources (without using the community module). By doing this, you’ll gain a clear understanding of the AWS networking, IAM, and compute resources that make up a Kubernetes cluster on AWS.

Finally, we’ll deploy a sample microservices application (Voting App) on the cluster and access it using port-forwarding.

Architecture

Here’s how the setup works at a high level:

VPC is created with 2 Availability Zones for high availability.
Each AZ contains both a public and a private subnet.
EKS worker nodes (EC2 instances) are launched in private subnets for better security.
A NAT Gateway is provisioned in a public subnet to allow worker nodes in private subnets to pull images and updates from the internet (e.g., from ECR, Docker Hub).
EKS control plane (managed by AWS) communicates with the worker nodes securely within the VPC. This setup ensures that your nodes are not directly exposed to the internet while still having outbound internet access via the NAT gateway.

Step 1: Create a VPC with Public and Private Subnets

EKS requires a properly configured VPC with both public and private subnets. Worker nodes generally live in private subnets, while the NAT gateway in public subnets provides outbound internet connectivity.

We also add Internet Gateway (IGW), NAT Gateway, and route tables to handle connectivity between public/private subnets.

################################################################################
# VPC
################################################################################
resource "aws_vpc" "custom_vpc" {
  cidr_block = var.networking.cidr_block

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-VPC"
  })
}

################################################################################
# PUBLIC SUBNETS
################################################################################
resource "aws_subnet" "public_subnets" {
  count                   = var.networking.public_subnets == null || var.networking.public_subnets == "" ? 0 : length(var.networking.public_subnets)
  vpc_id                  = aws_vpc.custom_vpc.id
  cidr_block              = var.networking.public_subnets[count.index]
  availability_zone       = var.networking.azs[count.index]
  map_public_ip_on_launch = true

  tags = merge(var.common_tags, {
    Name                                        = "${var.naming_prefix}-public-subnet-${count.index}"
    "kubernetes.io/role/elb"                    = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  })
}

################################################################################
# PRIVATE SUBNETS
################################################################################
resource "aws_subnet" "private_subnets" {
  count                   = var.networking.private_subnets == null || var.networking.private_subnets == "" ? 0 : length(var.networking.private_subnets)
  vpc_id                  = aws_vpc.custom_vpc.id
  cidr_block              = var.networking.private_subnets[count.index]
  availability_zone       = var.networking.azs[count.index]
  map_public_ip_on_launch = false

  tags = merge(var.common_tags, {
    Name                                        = "${var.naming_prefix}-private-subnet-${count.index}"
    "kubernetes.io/role/internal-elb"           = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  })
}

################################################################################
# INTERNET GATEWAY
################################################################################
resource "aws_internet_gateway" "i_gateway" {
  vpc_id = aws_vpc.custom_vpc.id

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-i-gateway"
  })
}

################################################################################
# EIPs
################################################################################
resource "aws_eip" "elastic_ip" {
  count      = var.networking.private_subnets == null || var.networking.nat_gateways == false ? 0 : length(var.networking.private_subnets)
  depends_on = [aws_internet_gateway.i_gateway]

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-eip-${count.index}"
  })
}

################################################################################
# NAT GATEWAYS
################################################################################
resource "aws_nat_gateway" "nats" {
  count             = var.networking.private_subnets == null || var.networking.nat_gateways == false ? 0 : length(var.networking.private_subnets)
  subnet_id         = aws_subnet.public_subnets[count.index].id
  connectivity_type = "public"
  allocation_id     = aws_eip.elastic_ip[count.index].id
  depends_on        = [aws_internet_gateway.i_gateway]

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-nat-gw-${count.index}"
  })
}

################################################################################
# PUBLIC ROUTE TABLE
################################################################################
resource "aws_route_table" "public_table" {
  vpc_id = aws_vpc.custom_vpc.id
}

resource "aws_route" "public_routes" {
  route_table_id         = aws_route_table.public_table.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.i_gateway.id
}

resource "aws_route_table_association" "assoc_public_routes" {
  count          = length(var.networking.public_subnets)
  subnet_id      = aws_subnet.public_subnets[count.index].id
  route_table_id = aws_route_table.public_table.id
}

################################################################################
# PRIVATE ROUTE TABLES
################################################################################
resource "aws_route_table" "private_tables" {
  count  = length(var.networking.azs)
  vpc_id = aws_vpc.custom_vpc.id
}

resource "aws_route" "private_routes" {
  count                  = length(var.networking.private_subnets)
  route_table_id         = aws_route_table.private_tables[count.index].id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.nats[count.index].id
}

resource "aws_route_table_association" "assoc_private_routes" {
  count          = length(var.networking.private_subnets)
  subnet_id      = aws_subnet.private_subnets[count.index].id
  route_table_id = aws_route_table.private_tables[count.index].id
}

################################################################################
# SECURITY GROUPS
################################################################################
resource "aws_security_group" "sec_groups" {
  for_each    = { for sec in var.security_groups : sec.name => sec }
  name        = "${var.naming_prefix}-${each.value.name}"
  description = each.value.description
  vpc_id      = aws_vpc.custom_vpc.id

  dynamic "ingress" {
    for_each = try(each.value.ingress, [])
    content {
      description      = ingress.value.description
      from_port        = ingress.value.from_port
      to_port          = ingress.value.to_port
      protocol         = ingress.value.protocol
      cidr_blocks      = ingress.value.cidr_blocks
      ipv6_cidr_blocks = ingress.value.ipv6_cidr_blocks
    }
  }

  dynamic "egress" {
    for_each = try(each.value.egress, [])
    content {
      description      = egress.value.description
      from_port        = egress.value.from_port
      to_port          = egress.value.to_port
      protocol         = egress.value.protocol
      cidr_blocks      = egress.value.cidr_blocks
      ipv6_cidr_blocks = egress.value.ipv6_cidr_blocks
    }
  }
}

Step 2: Create IAM Roles and Policies

EKS needs roles for both the control plane and worker nodes.

Cluster Role – Grants permissions for EKS to manage AWS resources.
Node Role – Allows EC2 instances (worker nodes) to join the cluster.

################################################################################
# EKS CLUSTER ROLE
################################################################################
resource "aws_iam_role" "EKSClusterRole" {
  name = "${var.naming_prefix}-EKSClusterRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "eks.amazonaws.com"
        }
      },
    ]
  })
}

// This policy provides Kubernetes the permissions it requires to manage resources on your behalf. 
// Kubernetes requires Ec2:CreateTags permissions to place identifying information on EC2 resources
// including but not limited to Instances, Security Groups, and Elastic Network Interfaces
resource "aws_iam_role_policy_attachment" "AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.EKSClusterRole.name
}

################################################################################
# NODE GROUP ROLE
################################################################################
resource "aws_iam_role" "NodeGroupRole" {
  name = "${var.naming_prefix}-EKSNodeGroupRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

//This policy allows Amazon EKS worker nodes to connect to Amazon EKS Clusters.
resource "aws_iam_role_policy_attachment" "AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.NodeGroupRole.name
}

// Provides read-only access to Amazon EC2 Container Registry repositories.
resource "aws_iam_role_policy_attachment" "AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.NodeGroupRole.name
}

// This policy provides the Amazon VPC CNI Plugin (amazon-vpc-cni-k8s) the permissions
// it requires to modify the IP address configuration on your EKS worker nodes. 
// This permission set allows the CNI to list, describe, and modify ENIs on your behalf
resource "aws_iam_role_policy_attachment" "AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.NodeGroupRole.name
}

Step 3: Provision the EKS Cluster

Following creates the EKS control plane.

################################################################################
# EKS Cluster
################################################################################
resource "aws_eks_cluster" "eks-cluster" {
  name     = var.cluster_config.name
  role_arn = aws_iam_role.EKSClusterRole.arn
  version  = var.cluster_config.version

  vpc_config {
    subnet_ids         = flatten([var.public_subnets_id, var.private_subnets_id])
    security_group_ids = flatten(var.security_groups_id)
  }

  depends_on = [
    aws_iam_role_policy_attachment.AmazonEKSClusterPolicy
  ]

}

At this point, we create a managed set of EC2 worker nodes with basic required addons

################################################################################
# NODE GROUP
################################################################################
resource "aws_eks_node_group" "node-ec2" {
  for_each        = { for node_group in var.node_groups : node_group.name => node_group }
  cluster_name    = aws_eks_cluster.eks-cluster.name
  node_group_name = "${var.naming_prefix}-${each.value.name}"
  node_role_arn   = aws_iam_role.NodeGroupRole.arn
  subnet_ids      = flatten(var.private_subnets_id)

  scaling_config {
    desired_size = try(each.value.scaling_config.desired_size, 2)
    max_size     = try(each.value.scaling_config.max_size, 3)
    min_size     = try(each.value.scaling_config.min_size, 1)
  }

  update_config {
    max_unavailable = try(each.value.update_config.max_unavailable, 1)
  }

  ami_type       = each.value.ami_type
  instance_types = each.value.instance_types
  capacity_type  = each.value.capacity_type
  disk_size      = each.value.disk_size
  version        = var.cluster_config.version

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-nodes-${each.value.name}"
  })

  depends_on = [
    aws_iam_role_policy_attachment.AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.AmazonEC2ContainerRegistryReadOnly,
    aws_iam_role_policy_attachment.AmazonEKS_CNI_Policy
  ]
}

resource "aws_iam_openid_connect_provider" "eks_oidc_provider" {
  url             = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = ["9e99a48a9960b14926bb7f3b02e22da2b0ab7280"]
}


################################################################################
# EKS Addons
################################################################################
data "aws_eks_addon_version" "eks_addons" {
  for_each = var.addons

  addon_name         = each.value.name
  kubernetes_version = aws_eks_cluster.eks-cluster.version
  most_recent        = coalesce(each.value.most_recent, true)
}

resource "aws_eks_addon" "addons" {
  for_each = var.addons

  cluster_name                = aws_eks_cluster.eks-cluster.name
  addon_name                  = each.value.name
  addon_version               = coalesce(each.value.version, data.aws_eks_addon_version.eks_addons[each.key].version)
  resolve_conflicts_on_create = "OVERWRITE"

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-addon-${each.value.name}"
  })

  depends_on = [aws_eks_node_group.node-ec2]
}

Step 4: Configure kubectl and Access Cluster

################################################################################
# Update kubeconfig with created cluster
################################################################################
resource "null_resource" "update_kubeconfig" {
  provisioner "local-exec" {
    command = "aws eks --region ${var.aws_region} update-kubeconfig --name ${var.cluster_config.name}"
  }
  depends_on = [module.eks]
}

Step 5: Deploy a Sample Microservices Application

We will deploy famous exmaple voting app from the https://github.com/dockersamples/example-voting-app. We will just deploy the manifests and not look into building the docker images at this moment.

A front-end web app in Python which lets you vote between two options
A Redis which collects new votes
A .NET worker which consumes votes and stores them in…
A Postgres database backed by a Docker volume
A Node.js web app which shows the results of the voting in real time

Note: The voting application only accepts one vote per client browser. It does not register additional votes if a vote has already been submitted from a client.

Apply the manifests and verify

cd /example-voting-app/k8s-specifications
kubectl apply -f .
deployment.apps/db created
service/db created
deployment.apps/redis created
service/redis created
deployment.apps/result created
service/result created
deployment.apps/vote created
service/vote created
deployment.apps/worker created


kubectl port-forward service/vote 8080:8080
kubectl port-forward service/result 8081:8081

Access app using:

Vote:   http://localhost:8080
Result: http://localhost:8081

Cleanup

Delete the k8s resources created

cd /example-voting-app/k8s-specifications
kubectl delete -f .

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

By building an EKS cluster from scratch with Terraform—without relying on the AWS community module—we gain a clear understanding of the essential AWS resources required to run Kubernetes in production.

We saw how networking forms the foundation, how the cluster and node groups tie everything together, and finally how workloads are deployed and accessed. This exercise provides not only a strong foundation in AWS and Kubernetes but also the confidence to customize infrastructure for real-world production use cases.

References

Kubernetes Storage Playlist - Part 4: Implementing Amazon S3 Storage with EKS using Terraform and and Kubernetes Manifests

Chinmay Tonape — Mon, 08 Sep 2025 19:07:09 +0000

In this blog, we’ll explore how to integrate Amazon S3 as a storage solution with Amazon EKS using Terraform and Kubernetes YAML manifests. We will run a simple Nginx container that serves website files stored in an S3 bucket.

This approach leverages the Mountpoint for S3 CSI driver, which provides Kubernetes workloads access to Amazon S3 objects using standard POSIX interfaces.

Understanding Amazon S3 for Kubernetes

What is Amazon S3?

Amazon Simple Storage Service (Amazon S3) is an object storage service designed for scalability, durability, and availability. Unlike traditional block storage (like EBS) or file storage (like EFS), S3 stores data as objects inside buckets, which makes it ideal for static content, logs, and backups.

Architecture Diagram

The architecture of attaching Amazon S3 storage to an EKS cluster revolves around the S3 CSI (Container Storage Interface) driver and the Mountpoint for S3 integration.

At the base layer, an S3 bucket acts as the backend object store where application data is kept. Inside Kubernetes, a StorageClass defines how storage is provisioned and consumed. With S3, however, only static provisioning is currently supported—meaning a PersistentVolume (PV) must be manually created and mapped to an existing S3 bucket, and then a PersistentVolumeClaim (PVC) binds to that PV so workloads can access it.

Once a pod is deployed, Kubernetes mounts the PVC to the container’s filesystem through the S3 CSI driver, which internally uses Mountpoint for S3 to provide file system-like access to objects in the bucket.

While dynamic provisioning—where PVCs automatically trigger creation of new storage volumes—is common for drivers like EBS and EFS, it is not yet available for S3. This makes static provisioning the only option, requiring administrators to pre-define the mapping between PVs and S3 buckets. Security is handled through IAM Roles for Service Accounts (IRSA), ensuring pods have only the minimum required permissions to access specific S3 buckets.

Key Benefits of Using S3 with EKS

Scalability: Virtually unlimited storage capacity.
Durability: 11 9’s durability guarantees.
Cost-Effective: Pay for what you use with no upfront provisioning.
Integration: Easy integration with AWS services like Athena, Glue, and CloudFront.

Important Considerations for EKS

CSI Driver Requirements: The Mountpoint for S3 CSI driver must be installed.
Pod Identity Support: Currently, IRSA (IAM Roles for Service Accounts) is required. Pod Identity is not supported yet.
Limitations: Only static provisioning is supported as of now. Dynamic provisioning is on the roadmap.
Resource Quotas: Watch for limits like open file descriptors and network throughput when scaling workloads.

Step 1: Provisioning EKS Cluster with Terraform in a VPC

The first step in integrating Amazon S3 with EKS is to provision a Kubernetes cluster that runs securely inside a dedicated VPC. We will use the widely adopted AWS Terraform community modules for both the VPC and EKS setup. Please refer to main module of GitHub repo.

Step 2: Creating the S3 bucket

S3 Bucket: Encrypted S3 bucket with versioning and permissions

####################################################################################
# S3 Bucket for Kubernetes Storage
####################################################################################
resource "aws_s3_bucket" "main" {
  bucket = var.bucket_name

  tags = {
    Name        = var.bucket_name
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# S3 Bucket Versioning
####################################################################################
resource "aws_s3_bucket_versioning" "main" {
  bucket = aws_s3_bucket.main.id
  versioning_configuration {
    status = var.versioning_enabled ? "Enabled" : "Disabled"
  }
}

####################################################################################
# S3 Bucket Encryption
####################################################################################
resource "aws_s3_bucket_server_side_encryption_configuration" "main" {
  bucket = aws_s3_bucket.main.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
    bucket_key_enabled = true
  }
}

####################################################################################
# S3 Bucket Public Access Block
####################################################################################
resource "aws_s3_bucket_public_access_block" "main" {
  bucket = aws_s3_bucket.main.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

IAM Role: For Mountpoint for S3 CSI Driver

####################################################################################
# IAM Role for S3 CSI Driver (IRSA)
####################################################################################
resource "aws_iam_role" "s3_csi_driver_role" {
  name = "${var.cluster_name}-s3-csi-driver-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Federated = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${replace(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://", "")}"
        }
        Action = "sts:AssumeRoleWithWebIdentity"
        Condition = {
          StringEquals = {
            "${replace(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://", "")}:sub" = "system:serviceaccount:kube-system:s3-csi-driver-sa"
            "${replace(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://", "")}:aud" = "sts.amazonaws.com"
          }
        }
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-s3-csi-driver-role"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# S3 CSI Driver Policy (Based on AWS Documentation)
####################################################################################
resource "aws_iam_policy" "s3_csi_driver_policy" {
  name        = "${var.cluster_name}-s3-csi-driver-policy"
  description = "IAM policy for S3 CSI Driver based on AWS documentation"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:ListBucket"
        ]
        Resource = aws_s3_bucket.main.arn
      },
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject",
          "s3:AbortMultipartUpload",
          "s3:DeleteObject"
        ]
        Resource = "${aws_s3_bucket.main.arn}/*"
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-s3-csi-driver-policy"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# Attach S3 CSI Driver Policy
####################################################################################
resource "aws_iam_role_policy_attachment" "s3_csi_driver_policy" {
  role       = aws_iam_role.s3_csi_driver_role.name
  policy_arn = aws_iam_policy.s3_csi_driver_policy.arn
}

Mountpoint for S3 Add-on: AWS Mountpoint for S3 CSI Driver for Kubernetes integration

####################################################################################
###  S3 Mountpoint CSI Driver Addon (deployed after S3 module)
####################################################################################
resource "aws_eks_addon" "s3_mountpoint_csi_driver" {
  cluster_name             = module.eks.cluster_name
  addon_name               = "aws-mountpoint-s3-csi-driver"
  service_account_role_arn = module.s3.s3_csi_driver_role_arn
  # Using IRSA for CSI driver, Pod Identity for application pods

  # Ensure this addon is created after the S3 module creates the IAM role and pod identity association
  depends_on = [module.s3]

  tags = {
    Name        = "${var.cluster_name}-s3-mountpoint-csi-driver"
    Environment = var.environment
    Terraform   = "true"
  }
}

Step 3: S3 Storage Implementation Patterns

Since only static provisioning is supported for S3 today, we’ll define PersistentVolumes (PV) that reference an S3 bucket.

storage-class.yaml - StorageClass for S3 CSI driver (requires ${S3_BUCKET_NAME} placeholder)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: s3-csi-sc
provisioner: s3.csi.aws.com
parameters:
  bucketName: ${S3_BUCKET_NAME}
  prefix: "k8s-storage/"
volumeBindingMode: Immediate
allowVolumeExpansion: false

persistent-volume.yaml - PersistentVolume for S3 bucket (static provisioning only)

apiVersion: v1
kind: PersistentVolume
metadata:
  name: s3-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: s3-csi-sc
  csi:
    driver: s3.csi.aws.com
    volumeHandle: s3-csi-driver-volume
    volumeAttributes:
      bucketName: ${S3_BUCKET_NAME}
      prefix: "k8s-storage/"

persistent-volume-claim.yaml - PVC using S3 CSI driver

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: s3-pvc
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: s3-csi-sc
  resources:
    requests:
      storage: 1Gi

nginx-pod.yaml - Nginx pod with S3 storage mounted and content creation

apiVersion: v1
kind: Pod
metadata:
  name: nginx-s3-pod
  namespace: default
  labels:
    app: nginx-s3
spec:
  securityContext:
    runAsUser: 0
    runAsGroup: 0
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    command: ["/bin/sh", "-c"]
    args:
    - |
      echo '<h1>Hello from S3 Mountpoint!</h1><p><b>Pod:</b> '$POD_NAME'</p>' > /usr/share/nginx/html/index.html || true
      sed -i 's/user  nginx;/user  root;/' /etc/nginx/nginx.conf
      nginx -g 'daemon off;'
    volumeMounts:
    - name: s3-storage
      mountPath: /usr/share/nginx/html
  volumes:
  - name: s3-storage
    persistentVolumeClaim:
      claimName: s3-pvc

nginx-service.yaml - ClusterIP service to expose nginx on port 80

apiVersion: v1
kind: Service
metadata:
  name: nginx-s3-service
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx-s3

Deployment Steps For Static Provisioning:

Get S3 bucket name from Terraform:

cd infrastructure
S3_BUCKET_NAME=$(terraform output -raw s3_bucket_name 2>/dev/null || echo "")

Update manifests with S3 values:

sed "s/\${S3_BUCKET_NAME}/$S3_BUCKET_NAME/g" storage-class.yaml > storage-class-final.yaml

sed "s/\${S3_BUCKET_NAME}/$S3_BUCKET_NAME/g" persistent-volume.yaml > persistent-volume-final.yaml

Apply manifests:

kubectl apply -f storage-class-final.yaml
kubectl apply -f persistent-volume-final.yaml
kubectl apply -f persistent-volume-claim.yaml
kubectl apply -f nginx-pod.yaml
kubectl apply -f nginx-service.yaml

Refer to deploy.sh for automated deployment script

$ kubectl get sc,pv,pvc
NAME                                    PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/s3-csi-sc   s3.csi.aws.com          Delete          Immediate              false                  10m

NAME                     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM            STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/s3-pv   1Gi        RWX            Retain           Bound    default/s3-pvc   s3-csi-sc      <unset>                          10m

NAME                           STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/s3-pvc   Bound    s3-pv    1Gi        RWX            s3-csi-sc      <unset>                 10m

S3 Bucket:

S3 Bucket Contents:

Nginx pod accessing S3 for index.html

Verification

# Check S3 CSI driver status:
kubectl get pods -n kube-system -l app=aws-mountpoint-s3-csi-driver

# Check deployment status:
kubectl get pod nginx-s3-pod
kubectl get service nginx-s3-service
kubectl get pvc s3-pvc
kubectl get pv

# Test nginx web server:
kubectl port-forward service/nginx-s3-service 8084:80
# Then visit http://localhost:8084

# Check S3 File Persistence
kubectl exec nginx-s3-pod -- ls -la /usr/share/nginx/html/
kubectl exec nginx-s3-pod -- cat /usr/share/nginx/html/index.html

# Check S3 bucket content:
aws s3 ls s3://$S3_BUCKET_NAME/k8s-storage/

Cleanup

kubectl delete -f nginx-service.yaml
kubectl delete -f nginx-pod.yaml
kubectl delete -f persistent-volume-claim.yaml
kubectl delete -f persistent-volume.yaml
kubectl delete -f storage-class.yaml

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

Amazon S3 provides a scalable and cost-effective storage solution for workloads running on EKS. With the Mountpoint for S3 CSI driver, Kubernetes pods can directly mount S3 buckets and serve static files seamlessly. While currently limited to static provisioning, this integration is a powerful way to manage object storage in containerized environments.

In this blog, we provisioned an EKS cluster using Terraform, set up IAM roles with IRSA, deployed the S3 CSI driver, and ran an Nginx container backed by S3 storage.

References

Kubernetes Storage Playlist - Part 3: Implementing Amazon EBS Storage with Amazon EKS Using Terraform and Kubernetes Manifests

Chinmay Tonape — Mon, 08 Sep 2025 18:33:23 +0000

In this blog, we’ll explore how to integrate Amazon Elastic Block Store (EBS) with Amazon Elastic Kubernetes Service (EKS). We’ll provision an EKS cluster with Terraform, configure the EBS CSI driver, and run an Nginx container that uses EBS storage to persist website files.

This is a practical guide for anyone building stateful workloads on EKS.

Understanding Amazon EBS for Kubernetes

What is Amazon EBS?

Amazon Elastic Block Store (EBS) provides persistent block-level storage volumes that can be attached to Amazon EC2 instances. Within Kubernetes, EBS volumes can be exposed to Pods through the EBS CSI (Container Storage Interface) driver, allowing workloads to persist data beyond pod lifecycles.

Architecture Diagram

When you use Amazon EBS with Amazon EKS, your application asks for storage through a PersistentVolumeClaim (PVC). Kubernetes then either connects this request to an existing EBS volume (static provisioning) or automatically creates a new one (dynamic provisioning) with the help of a StorageClass and the EBS CSI driver.

The CSI driver talks to AWS to create and attach the volume to the worker node where your Pod is running, and the volume gets mounted inside the container at the specified path (like /usr/share/nginx/html). One important thing to remember is that EBS volumes work only within a single Availability Zone (AZ). This means your Pod and the EBS volume must be in the same AZ. Dynamic provisioning usually takes care of this automatically, but with static provisioning, you need to make sure the volume is created in the right AZ.

Key Benefits for EKS

Durability: Data persists beyond pod restarts.
Performance: Multiple volume types (gp3, io2, st1, etc.) for optimized throughput or IOPS.
Elasticity: Volumes can be resized without downtime.
Cost-effectiveness: Pay only for the storage provisioned.

Important Considerations with EKS

EBS volumes are AZ-scoped – a Pod using EBS must run in the same Availability Zone as the volume.
Requires Kubernetes v1.17+ with CSI driver support.
Pods requiring EBS must use StatefulSets or carefully scheduled Deployments.
Resource quotas should be monitored to avoid exhausting storage or hitting API limits.

Step 1: Provisioning EKS Cluster with Terraform in a VPC

The first step in integrating Amazon EBS with EKS is to provision a Kubernetes cluster that runs securely inside a dedicated VPC. We will use the widely adopted AWS Terraform community modules for both the VPC and EKS setup. Please refer to main module of GitHub repo.

Step 2: Creating the EFS File System

EBS Storage: Encrypted EBS Storage with optional encryption

####################################################################################
# Static EBS Volume for Testing
####################################################################################
resource "aws_ebs_volume" "static_volume" {
  count = var.create_static_volume ? 1 : 0

  availability_zone = var.availability_zones[0]
  size              = var.static_volume_size
  type              = var.ebs_volume_type
  encrypted         = var.ebs_encrypted
  kms_key_id        = var.ebs_kms_key_id

  # Configure IOPS for gp3, io1, io2 volumes
  iops = var.ebs_volume_type == "gp3" || var.ebs_volume_type == "io1" || var.ebs_volume_type == "io2" ? var.ebs_volume_iops : null

  # Configure throughput for gp3 volumes
  throughput = var.ebs_volume_type == "gp3" ? var.ebs_volume_throughput : null

  tags = {
    Name        = "${var.cluster_name}-static-ebs-volume"
    Environment = var.environment
    Terraform   = "true"
    Purpose     = "Static EBS volume for Kubernetes testing"
  }
}

####################################################################################
# KMS Key for EBS Encryption (Optional)
####################################################################################
resource "aws_kms_key" "ebs_encryption" {
  count = var.ebs_kms_key_id == null ? 1 : 0

  description             = "KMS key for EBS volume encryption in ${var.cluster_name}"
  deletion_window_in_days = 7
  enable_key_rotation     = true

  tags = {
    Name        = "${var.cluster_name}-ebs-encryption-key"
    Environment = var.environment
    Terraform   = "true"
  }
}

resource "aws_kms_alias" "ebs_encryption" {
  count = var.ebs_kms_key_id == null ? 1 : 0

  name          = "alias/${var.cluster_name}-ebs-encryption"
  target_key_id = aws_kms_key.ebs_encryption[0].key_id
}

IAM Role: For EFS CSI driver with pod identity

####################################################################################
# IAM Role for EBS CSI Driver (Pod Identity)
####################################################################################
resource "aws_iam_role" "ebs_csi_driver_role" {
  name = "${var.cluster_name}-ebs-csi-driver-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "pods.eks.amazonaws.com"
        }
        Action = [
          "sts:AssumeRole",
          "sts:TagSession"
        ]
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-ebs-csi-driver-role"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# Create Custom EBS CSI Driver Policy
####################################################################################
resource "aws_iam_policy" "ebs_csi_driver_policy" {
  name        = "${var.cluster_name}-ebs-csi-driver-policy"
  description = "Policy for EBS CSI driver"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ec2:CreateSnapshot",
          "ec2:AttachVolume",
          "ec2:DetachVolume",
          "ec2:ModifyVolume",
          "ec2:DescribeAvailabilityZones",
          "ec2:DescribeInstances",
          "ec2:DescribeSnapshots",
          "ec2:DescribeTags",
          "ec2:DescribeVolumes",
          "ec2:DescribeVolumesModifications"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:CreateTags"
        ]
        Resource = [
          "arn:aws:ec2:*:*:volume/*",
          "arn:aws:ec2:*:*:snapshot/*"
        ]
        Condition = {
          StringEquals = {
            "ec2:CreateAction" = [
              "CreateVolume",
              "CreateSnapshot"
            ]
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:DeleteTags"
        ]
        Resource = [
          "arn:aws:ec2:*:*:volume/*",
          "arn:aws:ec2:*:*:snapshot/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:CreateVolume"
        ]
        Resource = "*"
        Condition = {
          StringLike = {
            "aws:RequestedRegion" = data.aws_region.current.name
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:DeleteVolume"
        ]
        Resource = "*"
        Condition = {
          StringLike = {
            "ec2:ResourceTag/ebs.csi.aws.com/cluster" = "true"
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:DeleteSnapshot"
        ]
        Resource = "*"
        Condition = {
          StringLike = {
            "ec2:ResourceTag/CSIVolumeSnapshotName" = "*"
          }
        }
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-ebs-csi-driver-policy"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# Attach EBS CSI Driver Policy
####################################################################################
resource "aws_iam_role_policy_attachment" "ebs_csi_driver_policy" {
  role       = aws_iam_role.ebs_csi_driver_role.name
  policy_arn = aws_iam_policy.ebs_csi_driver_policy.arn
}

####################################################################################
# Pod Identity Association for EBS CSI Driver
####################################################################################
resource "aws_eks_pod_identity_association" "ebs_csi_driver" {
  cluster_name    = var.cluster_name
  namespace       = "kube-system"
  service_account = "ebs-csi-controller-sa"
  role_arn        = aws_iam_role.ebs_csi_driver_role.arn

  tags = {
    Name        = "${var.cluster_name}-ebs-csi-pod-identity"
    Environment = var.environment
    Terraform   = "true"
  }
}

EKS Add-on: AWS EFS CSI driver for Kubernetes integration

####################################################################################
###  EBS CSI Driver Addon (deployed after EBS module)
####################################################################################
resource "aws_eks_addon" "ebs_csi_driver" {
  cluster_name = module.eks.cluster_name
  addon_name   = "aws-ebs-csi-driver"

  # Ensure this addon is created after the EBS module creates the IAM role and pod identity association
  depends_on = [module.ebs]

  tags = {
    Name        = "${var.cluster_name}-ebs-csi-driver"
    Environment = var.environment
    Terraform   = "true"
  }
}

Step 3: EBS Storage Implementation Patterns

Static Provisioning

In static provisioning, the PersistentVolume (PV) is created manually by the administrator. The PV explicitly points to an existing EBS volume ID and directory.

How it works:

You first provision an EBS Volume using Terraform/CLI.
You then define a Kubernetes PersistentVolume (PV) resource that references the EBS Volume ID.
Then youcreate a PersistentVolumeClaim (PVC) that requests storage matching the PV’s specifications.
Deploy your Pod (e.g., Nginx) and mount the PVC as a volume to store files.

static-storage-class.yaml - StorageClass for static EBS volumes (no parameters needed)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-static-sc
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  fsType: ext4
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

static-persistent-volume.yaml - PersistentVolume pointing to existing EBS volume (requires ${EBS_VOLUME_ID})

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ebs-static-pv
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: ebs-static-sc
  csi:
    driver: ebs.csi.aws.com
    volumeHandle: ${EBS_VOLUME_ID}
    fsType: ext4

static-persistent-volume-claim.yaml - PersistentVolumeClaim for static volume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-static-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-static-sc
  volumeName: ebs-static-pv
  resources:
    requests:
      storage: 10Gi

static-nginx-pod.yaml - Test pod using static EBS volume

apiVersion: v1
kind: Pod
metadata:
  name: nginx-ebs-static-pod
  namespace: default
  labels:
    app: nginx-ebs-static
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
    volumeMounts:
    - name: ebs-storage
      mountPath: /usr/share/nginx/html
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    command: ["/bin/sh"]
    args: ["-c", "echo '<h1>Hello from EBS Static Volume!</h1><p><b>Pod:</b> '$POD_NAME'</p>' > /usr/share/nginx/html/index.html && nginx -g 'daemon off;'"]
  volumes:
  - name: ebs-storage
    persistentVolumeClaim:
      claimName: ebs-static-pvc

static-nginx-service.yaml - Service to expose the static test pod

apiVersion: v1
kind: Service
metadata:
  name: nginx-ebs-static-service
  namespace: default
  labels:
    app: nginx-ebs-static
spec:
  selector:
    app: nginx-ebs-static
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: ClusterIP

Deployment Steps For Static Provisioning:

Get EBS values from Terraform:

cd infrastructure
EBS_VOLUME_ID=$(terraform output -raw ebs_volume_id 2>/dev/null || echo "")

Update manifests with EFS values:

sed "s/\${EBS_VOLUME_ID}/$EBS_VOLUME_ID/g" static-persistent-volume.yaml > static-persistent-volume-final.yaml

Apply static manifests:

kubectl apply -f static-storage-class.yaml
kubectl apply -f static-persistent-volume-final.yaml
kubectl apply -f static-persistent-volume-claim.yaml
kubectl apply -f static-nginx-pod.yaml
kubectl apply -f static-nginx-service.yaml

Refer to static-deploy.sh for deployment script for static provisioning

$ kubectl get sc,pv,pvc
NAME                                        PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/ebs-static-sc   ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   6m39s

NAME                             CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                    STORAGECLASS    VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/ebs-static-pv   10Gi       RWO            Retain           Bound    default/ebs-static-pvc   ebs-static-sc   <unset>                          6m31s

NAME                                   STATUS   VOLUME          CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/ebs-static-pvc   Bound    ebs-static-pv   10Gi       RWO            ebs-static-sc   <unset>                 6m28s

EBS Volume:

Nginx pod accessing EBS for index.html

Dynamic Provisioning (Recommended)

In dynamic provisioning, Kubernetes automatically creates EBS volumes on demand using the EBS CSI driver and a StorageClass.

How it works:

Define a StorageClass that specifies the EBS Volume (e.g., gp3 type, retention policy, binding mode)..
When an application requests storage using a PVC, Kubernetes dynamically creates:
- A new PV backed by a new EBS volume
- Pods mount the dynamically provisioned PV through the PVC.

dynamic-storage-class.yaml - StorageClass with gp3 volume configuration

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-dynamic-sc
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  fsType: ext4
  encrypted: "true"
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete

dynamic-persistent-volume-claim.yaml - PVC for dynamic volume creation

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-dynamic-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-dynamic-sc
  resources:
    requests:
      storage: 10Gi

dynamic-nginx-pod.yaml - Test pod using dynamic EBS volume

apiVersion: v1
kind: Pod
metadata:
  name: nginx-ebs-dynamic-pod
  namespace: default
  labels:
    app: nginx-ebs-dynamic
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
    volumeMounts:
    - name: ebs-storage
      mountPath: /usr/share/nginx/html
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo '<h1>Hello from EBS Dynamic Volume!</h1><p><b>Pod:</b> '$POD_NAME'</p>' > /usr/share/nginx/html/index.html"]
  volumes:
  - name: ebs-storage
    persistentVolumeClaim:
      claimName: ebs-dynamic-pvc

dynamic-nginx-service.yaml - Service to expose the dynamic test pod

apiVersion: v1
kind: Service
metadata:
  name: nginx-ebs-dynamic-service
  namespace: default
  labels:
    app: nginx-ebs-dynamic
spec:
  selector:
    app: nginx-ebs-dynamic
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: ClusterIP

Deployment Steps For Dynamic Provisioning:

Apply static manifests:

kubectl apply -f dynamic-storage-class.yaml
kubectl apply -f dynamic-persistent-volume-claim.yaml
kubectl apply -f dynamic-nginx-pod.yaml
kubectl apply -f dynamic-nginx-service.yaml

Refer to dynamic-deploy.sh for deployment script for dynamic provisioning

$ kubectl get sc,pv,pvc
NAME                                         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/ebs-dynamic-sc   ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   3m31s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                     STORAGECLASS     VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/pvc-7057dfad-6961-49cd-bf4f-db2f0bd9727f   10Gi       RWO            Delete           Bound    default/ebs-dynamic-pvc   ebs-dynamic-sc   <unset>                          3m22s

NAME                                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/ebs-dynamic-pvc   Bound    pvc-7057dfad-6961-49cd-bf4f-db2f0bd9727f   10Gi       RWO            ebs-dynamic-sc   <unset>                 3m29s

EBS Volume:

Nginx pod accessing EBS for index.html

Verification

Check that everything is working:

# Check EFS CSI driver pods
kubectl get pods -n kube-system -l app=efs-csi-controller

# Check storage classes
kubectl get storageclass ebs-sc

# For Static Provisioning:
kubectl get pv ebs-static-pv
kubectl get pvc ebs-static-pvc
kubectl get pod nginx-ebs-static
kubectl get service nginx-ebs-static-service

# For Dynamic Provisioning:
kubectl get pvc ebs-dynamic-pvc
kubectl get pod nginx-ebs-dynamic
kubectl get service nginx-ebs-dynamic-service

# Check volume mount
kubectl exec nginx-ebs-static-pod -- df -h /usr/share/nginx/html
kubectl exec nginx-ebs-dynamic-pod -- df -h /usr/share/nginx/html

# View content
kubectl exec nginx-ebs-static-pod -- cat /usr/share/nginx/html/index.html
kubectl exec nginx-ebs-dynamic-pod -- cat /usr/share/nginx/html/index.html

# Test nginx web servers
kubectl port-forward service/nginx-ebs-static-service 8082:80   # Static
# Then visit http://localhost:8082

kubectl port-forward service/nginx-ebs-dynamic-service 8083:80  # Dynamic
# Then visit http://localhost:8083

Cleanup

Static Provisioning Cleanup

kubectl delete -f static-storage-class.yaml
kubectl delete -f static-persistent-volume-final.yaml
kubectl delete -f static-persistent-volume-claim.yaml
kubectl delete -f static-nginx-pod.yaml
kubectl delete -f static-nginx-service.yaml

Dynamic Provisioning Cleanup

kubectl delete -f dynamic-nginx-service.yaml
kubectl delete -f dynamic-nginx-pod.yaml
kubectl delete -f dynamic-persistent-volume-claim.yaml
kubectl delete -f dynamic-storage-class.yaml

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

Amazon EBS provides reliable block storage for stateful workloads on Amazon EKS. Using Terraform for infrastructure provisioning and Kubernetes manifests for storage configuration gives you both automation and flexibility. With proper setup of IAM roles, CSI drivers, and best practices, you can run workloads like Nginx with persistent storage on EKS confidently.

References

Kubernetes Storage Playlist - Part 2: Implementing Amazon EFS Storage with EKS using Terraform and Kubernetes Manifests

Chinmay Tonape — Mon, 08 Sep 2025 06:10:28 +0000

In this blog, we will walk through how to integrate Amazon Elastic File System (EFS) with Amazon Elastic Kubernetes Service (EKS) using Terraform for infrastructure provisioning and Kubernetes manifests for workloads.

As a demo, we will deploy an NGINX container that uses EFS-backed storage to persist and serve website files. This approach is common for workloads that require shared storage across multiple pods.

Understanding Amazon EFS for Kubernetes

What is Amazon EFS?

Amazon Elastic File System (EFS) is a fully managed, scalable, and serverless network file system that can be mounted concurrently by multiple EC2 instances, Lambda functions, and Kubernetes pods.

For Kubernetes workloads, EFS provides persistent volumes that can be shared across multiple pods, even across Availability Zones (AZs) in a VPC.

Architecture Diagram

The architecture of how Amazon EFS integrates with an EKS cluster revolves around Kubernetes storage resources and AWS networking components. First, an EFS file system is created with mount targets in each Availability Zone of the VPC, ensuring that EKS worker nodes across zones can connect to the storage. The EFS CSI driver, deployed as a DaemonSet in the EKS cluster, acts as the bridge between Kubernetes and EFS. To use this storage, a StorageClass is defined, which tells Kubernetes how to provision PersistentVolumes (PVs) either statically (by pointing to an existing file system/directory) or dynamically (by creating sub-directories through EFS Access Points). Applications then request storage using PersistentVolumeClaims (PVCs), which bind to the appropriate PV. When pods are deployed (e.g., an NGINX container), the CSI driver mounts the EFS file system onto the worker node through the mount targets, and the pod accesses it as a standard directory path. This setup ensures that any pod in the cluster, regardless of node or AZ, can reliably attach to the same EFS-backed storage for shared and persistent data.

Key Benefits for EKS

Shared storage: Multiple pods can read and write simultaneously.
Elastic scaling: Automatically grows and shrinks with demand.
Multi-AZ access: Ensures high availability across EKS worker nodes.
Use cases: Content management, logs aggregation, web apps, home directories.

Important Considerations with EKS

Kubernetes version: Ensure you are using a supported version (v1.23+ recommended).
EFS CSI driver: Must be installed in the cluster to enable storage integration.
Pod Identity/IRSA: Required to grant fine-grained IAM permissions.
Performance: Suitable for general-purpose and throughput-heavy workloads, but not ideal for low-latency databases.
Quotas: File system size and performance modes must be considered when designing workloads.

Step 1: Provisioning EKS Cluster with Terraform in a VPC

Step 2: Creating the EFS File System

EFS File System: Encrypted EFS with configurable performance mode

####################################################################################
# EFS File System
####################################################################################
resource "aws_efs_file_system" "main" {
  creation_token = "${var.cluster_name}-efs"
  encrypted      = true

  performance_mode = var.performance_mode
  throughput_mode  = var.throughput_mode

  tags = {
    Name        = "${var.cluster_name}-efs"
    Environment = var.environment
    Terraform   = "true"
  }
}

EFS Mount Targets: One per private subnet for high availability and EFS access point

####################################################################################
# EFS Mount Targets (one per private subnet)
####################################################################################
resource "aws_efs_mount_target" "main" {
  count           = length(var.private_subnet_ids)
  file_system_id  = aws_efs_file_system.main.id
  subnet_id       = var.private_subnet_ids[count.index]
  security_groups = [aws_security_group.efs.id]
}

####################################################################################
# EFS Access Point for Pod
####################################################################################
resource "aws_efs_access_point" "pod_access_point" {
  file_system_id = aws_efs_file_system.main.id

  posix_user {
    gid = var.posix_user_gid
    uid = var.posix_user_uid
  }

  root_directory {
    path = "/app-data"
    creation_info {
      owner_gid   = var.posix_user_gid
      owner_uid   = var.posix_user_uid
      permissions = "755"
    }
  }

  tags = {
    Name        = "${var.cluster_name}-efs-access-point"
    Environment = var.environment
    Terraform   = "true"
  }
}

Security Group: Allows NFS traffic (port 2049) from EKS nodes

####################################################################################
# Security Group for EFS
####################################################################################
resource "aws_security_group" "efs" {
  name_prefix = "${var.cluster_name}-efs-"
  vpc_id      = var.vpc_id

  ingress {
    description = "NFS from EKS nodes"
    from_port   = 2049
    to_port     = 2049
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name        = "${var.cluster_name}-efs-sg"
    Environment = var.environment
    Terraform   = "true"
  }
}

IAM Role: For EFS CSI driver with pod identity

####################################################################################
# IAM Role for EFS CSI Driver (Pod Identity)
####################################################################################
resource "aws_iam_role" "efs_csi_driver_role" {
  name = "${var.cluster_name}-efs-csi-driver-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "pods.eks.amazonaws.com"
        }
        Action = [
          "sts:AssumeRole",
          "sts:TagSession"
        ]
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-efs-csi-driver-role"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# Create Custom EFS CSI Driver Policy
####################################################################################
resource "aws_iam_policy" "efs_csi_driver_policy" {
  name        = "${var.cluster_name}-efs-csi-driver-policy"
  description = "Policy for EFS CSI driver"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "elasticfilesystem:DescribeAccessPoints",
          "elasticfilesystem:DescribeFileSystems",
          "elasticfilesystem:DescribeMountTargets",
          "ec2:DescribeAvailabilityZones"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "elasticfilesystem:CreateAccessPoint",
          "elasticfilesystem:TagResource"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "elasticfilesystem:DeleteAccessPoint"
        ]
        Resource = "*"
        Condition = {
          StringEquals = {
            "elasticfilesystem:AccessedViaMountTarget" = "true"
          }
        }
      }
    ]
  })

  tags = {
    Name        = "${var.cluster_name}-efs-csi-driver-policy"
    Environment = var.environment
    Terraform   = "true"
  }
}

####################################################################################
# Attach EFS CSI Driver Policy
####################################################################################
resource "aws_iam_role_policy_attachment" "efs_csi_driver_policy" {
  role       = aws_iam_role.efs_csi_driver_role.name
  policy_arn = aws_iam_policy.efs_csi_driver_policy.arn
}
####################################################################################
# Pod Identity Association for EFS CSI Driver
####################################################################################
resource "aws_eks_pod_identity_association" "efs_csi_driver" {
  cluster_name    = var.cluster_name
  namespace       = "kube-system"
  service_account = "efs-csi-controller-sa"
  role_arn        = aws_iam_role.efs_csi_driver_role.arn

  tags = {
    Name        = "${var.cluster_name}-efs-csi-pod-identity"
    Environment = var.environment
    Terraform   = "true"
  }
}

EKS Add-on: AWS EFS CSI driver for Kubernetes integration

####################################################################################
###  EFS CSI Driver Addon (deployed after EFS module)
####################################################################################
resource "aws_eks_addon" "efs_csi_driver" {
  cluster_name = module.eks.cluster_name
  addon_name   = "aws-efs-csi-driver"

  # Ensure this addon is created after the EFS module creates the IAM role and pod identity association
  depends_on = [module.efs]

  tags = {
    Name        = "${var.cluster_name}-efs-csi-driver"
    Environment = var.environment
    Terraform   = "true"
  }
}

Step 3: EFS Storage Implementation Patterns

Static Provisioning

In static provisioning, the PersistentVolume (PV) is created manually by the administrator. The PV explicitly points to an existing EFS File System ID and directory.

How it works:

You first provision an EFS file system and create mount targets in your VPC.
You then define a Kubernetes PersistentVolume (PV) resource that references the EFS File System ID.
Applications request storage through a PersistentVolumeClaim (PVC) that binds to this PV.
The EFS volume is mounted directly into pods through the PVC.

static-storage-class.yaml - StorageClass for static EFS provisioning (no parameters)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-static-sc
provisioner: efs.csi.aws.com
# Static provisioning - no parameters needed

static-persistent-volume.yaml - PV using existing EFS file system (requires ${EFS_FILE_SYSTEM_ID})

apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-static-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-static-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: ${EFS_FILE_SYSTEM_ID}

static-persistent-volume-claim.yaml - PVC for static volume

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-static-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-static-sc
  volumeName: efs-static-pv
  resources:
    requests:
      storage: 5Gi

static-nginx-pod.yaml - Test pod using static EFS volume

apiVersion: v1
kind: Pod
metadata:
  name: nginx-efs-static-pod
  namespace: default
  labels:
    app: nginx-efs-static
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
    volumeMounts:
    - name: efs-storage
      mountPath: /usr/share/nginx/html
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    command: ["/bin/sh"]
    args: ["-c", "echo '<h1>Hello from EFS Static Volume!</h1><p><b>Pod:</b> '$POD_NAME'</p>' > /usr/share/nginx/html/index.html && nginx -g 'daemon off;'"]
  volumes:
  - name: efs-storage
    persistentVolumeClaim:
      claimName: efs-static-pvc

static-nginx-service.yaml - Service to expose the static test pod

apiVersion: v1
kind: Service
metadata:
  name: nginx-efs-static-service
  namespace: default
spec:
  selector:
    app: nginx-efs-static
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

Deployment Steps For Static Provisioning:

1 Get EFS values from Terraform:

   cd infrastructure
   EFS_FILE_SYSTEM_ID=$(terraform output -raw efs_file_system_id)

2 Update manifests with EFS values:

   # Replace placeholder in static-persistent-volume.yaml
   sed "s/\${EFS_FILE_SYSTEM_ID}/$EFS_FILE_SYSTEM_ID/g" static-persistent-volume.yaml > static-persistent-volume-final.yaml

3 Apply static manifests:

   kubectl apply -f static-storage-class.yaml
   kubectl apply -f static-persistent-volume-final.yaml
   kubectl apply -f static-persistent-volume-claim.yaml
   kubectl apply -f static-nginx-pod.yaml
   kubectl apply -f static-nginx-service.yaml

Refer to static-deploy.sh for deployment script for static provisioning

$ kubectl get sc,pv,pvc
NAME                                         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/efs-static-sc    efs.csi.aws.com         Delete          Immediate              false                  27s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                     STORAGECLASS     VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/efs-static-pv                              5Gi        RWX            Retain           Bound    default/efs-static-pvc    efs-static-sc    <unset>                          25s

NAME                                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/efs-static-pvc    Bound    efs-static-pv                              5Gi        RWX            efs-static-sc    <unset>                 21s

EFS Volume:

Nginx pod accessing EFS for index.html

Dynamic Provisioning

In dynamic provisioning, Kubernetes automatically creates PersistentVolumes (PVs) as needed, based on a StorageClass that uses the EFS CSI driver. This approach leverages EFS Access Points, which act like sub-directories with their own permissions inside the file system.

How it works:

Define a StorageClass that specifies the EFS file system and provisioning mode.
When an application requests storage using a PVC, Kubernetes dynamically creates:
- A new PV backed by an EFS Access Point.
- A dedicated directory within the EFS filesystem.
- Pods mount the dynamically provisioned PV through the PVC.

dynamic-storage-class.yaml - StorageClass for dynamic EFS provisioning (requires ${EFS_FILE_SYSTEM_ID})

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-dynamic-sc
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap
  fileSystemId: ${EFS_FILE_SYSTEM_ID}
  directoryPerms: "700"
  gidRangeStart: "1000"
  gidRangeEnd: "2000"
  basePath: "/dynamic_provisioning"
allowVolumeExpansion: true

dynamic-persistent-volume-claim.yaml - PVC for dynamic volume creation

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-dynamic-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-dynamic-sc
  resources:
    requests:
      storage: 5Gi

dynamic-nginx-pod.yaml - Test pod using dynamic EFS volume

apiVersion: v1
kind: Pod
metadata:
  name: nginx-efs-dynamic-pod
  namespace: default
  labels:
    app: nginx-efs-dynamic
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80
    volumeMounts:
    - name: efs-storage
      mountPath: /usr/share/nginx/html
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo '<h1>Hello from EFS Dynamic Volume!</h1><p><b>Pod:</b> '$POD_NAME'</p>' > /usr/share/nginx/html/index.html"]
  volumes:
  - name: efs-storage
    persistentVolumeClaim:
      claimName: efs-dynamic-pvc

dynamic-nginx-service.yaml - Service to expose the dynamic test pod

apiVersion: v1
kind: Service
metadata:
  name: nginx-efs-dynamic-service
  namespace: default
spec:
  selector:
    app: nginx-efs-dynamic
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

Deployment Steps For Dynamic Provisioning:

1 Get EFS values from Terraform:

   cd infrastructure
   EFS_FILE_SYSTEM_ID=$(terraform output -raw efs_file_system_id)

2 Update manifests with EFS values:

   # Replace placeholder in dynamic-storage-class.yaml
   sed "s/\${EFS_FILE_SYSTEM_ID}/$EFS_FILE_SYSTEM_ID/g" dynamic-storage-class.yaml > dynamic-storage-class-final.yaml

3 Apply dynamic manifests:

   kubectl apply -f dynamic-storage-class-final.yaml
   kubectl apply -f dynamic-persistent-volume-claim.yaml
   kubectl apply -f dynamic-nginx-pod.yaml
   kubectl apply -f dynamic-nginx-service.yaml

Refer dynamic-deploy.sh for deployment script for dynamic provisioning

$ kubectl get sc,pv,pvc
NAME                                         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/efs-dynamic-sc   efs.csi.aws.com         Delete          Immediate              true                   14m

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                     STORAGECLASS     VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/pvc-c07eb431-8060-4e19-8fd6-7166e816c4d9   5Gi        RWX            Delete           Bound    default/efs-dynamic-pvc   efs-dynamic-sc   <unset>                          14m

NAME                                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/efs-dynamic-pvc   Bound    pvc-c07eb431-8060-4e19-8fd6-7166e816c4d9   5Gi        RWX            efs-dynamic-sc   <unset>                 14m

Nginx pod accessing EFS for index.html:

Verification

Check that everything is working:

# Check EFS CSI driver pods
kubectl get pods -n kube-system -l app=efs-csi-controller

# Check storage classes
kubectl get storageclass efs-static-sc efs-dynamic-sc

# For Static Provisioning:
kubectl get pv efs-static-pv
kubectl get pvc efs-static-pvc
kubectl get pod nginx-efs-static-pod
kubectl get service nginx-efs-static-service

# For Dynamic Provisioning:
kubectl get pvc efs-dynamic-pvc
kubectl get pod nginx-efs-dynamic-pod
kubectl get service nginx-efs-dynamic-service

# Test nginx web servers
kubectl port-forward service/nginx-efs-static-service 8080:80   # Static
# Then visit http://localhost:8080

kubectl port-forward service/nginx-efs-dynamic-service 8081:80  # Dynamic
# Then visit http://localhost:8081

# Test file persistence
kubectl exec -it nginx-efs-static-pod -- ls -la /usr/share/nginx/html/
kubectl exec -it nginx-efs-dynamic-pod -- ls -la /usr/share/nginx/html/

Cleanup

Static Provisioning Cleanup

   kubectl delete -f static-nginx-service.yaml
   kubectl delete -f static-nginx-pod.yaml
   kubectl delete -f static-persistent-volume-claim.yaml
   kubectl delete -f static-persistent-volume.yaml
   kubectl delete -f static-storage-class.yaml

Dynamic Provisioning Cleanup

   kubectl delete -f dynamic-nginx-service.yaml
   kubectl delete -f dynamic-nginx-pod.yaml
   kubectl delete -f dynamic-persistent-volume-claim.yaml
   kubectl delete -f dynamic-storage-class.yaml

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

Amazon EFS with EKS provides a scalable, shared, and highly available storage solution for Kubernetes workloads. Using Terraform, YAML manifests, and the EFS CSI driver, you can seamlessly integrate cloud-native storage with your workloads.

For our demo, we deployed an NGINX container that serves website files from EFS storage. This pattern is widely applicable to web applications, shared logs, and content management systems running on EKS.