Forem: marunkumar1983

Disaster Recovery on AWS - Part 1: Multi-Site Active/Active (EC2, ECS, Fargate)

marunkumar1983 — Tue, 01 Apr 2025 04:06:43 +0000

Introduction

What is the highest financial loss a company could face due to the absence of a Disaster Recovery (DR) setup? For enterprises, this could amount to millions, depending on their Recovery Time Objective (RTO) and Recovery Point Objective (RPO).
Disaster Recovery is not just an IT concern—it is a business necessity. Organizations must prioritize DR to ensure business continuity and mitigate risks associated with data loss, cyber-attacks, and natural disasters. A lack of a DR strategy can lead to severe financial and reputational damage.

What is Disaster Recovery?

Disaster Recovery (DR) is a strategic framework and set of processes designed to restore IT systems, applications, and data in the event of disruptions. DR ensures business continuity by minimizing downtime and data loss, leveraging backup strategies, failover mechanisms, and automated recovery processes.

What is RTO (Recovery Time Objective)?

RTO defines the maximum acceptable downtime for an application or system after a failure before it significantly impacts business operations. A lower RTO indicates a faster recovery time and typically requires more robust infrastructure and automation.

What is RPO (Recovery Point Objective)?

RPO refers to the maximum tolerable amount of data loss measured in time. It defines how frequently data backups should be performed to ensure minimal loss in case of a disaster. A lower RPO requires more frequent backups and robust data replication strategies.

Why is Disaster Recovery Necessary Beyond Availability Zones?

Each AWS region consists of multiple Availability Zones (AZs). While deploying applications across multiple AZs enhances high availability, it does not qualify as Disaster Recovery. A DR strategy extends beyond AZs to ensure resilience across multiple geographic regions. DR is a critical component of IT General Controls (ITGCs), supporting the reliability of financial applications and compliance with regulatory standards. ITGCs encompass data backup, recovery, system availability, and change management. Audit departments and external firms assess these controls to ensure an organization's DR preparedness.

Disaster Recovery Strategies

Types of DR Strategies

A well-architected DR strategy can be classified into different models based on business requirements:

Backup & Restore: Cost-effective, but recovery time can be slow.
Pilot Light: A minimal setup is always running, with additional resources scaled up during a disaster.
Warm Standby: A scaled-down version of the environment runs continuously, allowing for quick recovery.
Multi-Site Active/Active: Applications are deployed across multiple regions, distributing traffic dynamically to maintain availability and resilience.

Architecture: Multi-Site Active/Active (EC2, ECS, Fargate)

Architecture Flow:
• Route 53: Global traffic routing across AWS regions.
• Application Load Balancer (ALB): Distributes traffic to multiple regions.
• Region 1 & Region 2: Active deployment of EC2, ECS, and Fargate to handle workloads. Best practice should be using IaC (Cloudformation, Terraform) and DevOps tools like Jenkins.

• Traffic Distribution: The percentage of traffic routed to each region can be defined based on business needs.

Key Takeaways

• DR is not optional – It is a fundamental requirement to ensure business continuity.
• Faster Recovery: A well-defined RTO and RPO minimize downtime and data loss.
• Cost Efficiency: A robust DR strategy helps reduce long-term operational costs by ensuring preparedness.
• Automated Rollback & Testing: Regularly test and validate backups to ensure effective failover during disasters.

Call to Action (CTA)

Assess your DR strategy today. Ensure your backups, failover systems, and rollback processes are tested, automated, and ready for the unexpected. Define and review your RTO and RPO to align with business objectives.
Conclusion
Disaster Recovery is more than an IT requirement—it is a business-critical function. Organizations must proactively implement DR measures to avoid financial losses, maintain compliance, and ensure uninterrupted operations.

Action Points:

Implement robust backup and recovery procedures.
Develop and maintain Business Continuity Plans.
Regularly test and optimize DR plans to meet evolving business needs. A well-prepared DR strategy can mean the difference between business resilience and catastrophic failure. Start planning today to safeguard your enterprise’s future.

Retro’ing and Debugging 2024: Lessons, Challenges, and 🏆 Achievements - Driving 🐎 Innovation and 🚚 Delivery

marunkumar1983 — Sat, 01 Feb 2025 06:13:41 +0000

This is a submission for the 2025 New Year Writing challenge: Retro’ing and Debugging 2024.

As I reflect on 2024, it has been a year of immense 🌱 growth, challenges, and 🏆 achievements. This retrospective captures my 🏃‍♂️ journey, the 🚫 hurdles I faced, the lessons I learned, and the milestones I reached.

# The Journey: Driving 🐎 Innovation and 🚚 Delivery

At the start of the year, I embarked on a critical mission—writing 📖 RFPs, designing solution and SOWs to onboard 350+ repositories into a DevOps pipeline with Continuous Delivery (CD) along with new DevOps Infrastructure. This project also involved 🔧 framework upgrades and vault implementation, making it a multi-faceted endeavor. Glad to share one year of great experience which is going to shape my career.

Challenges: Navigating a High-Stakes Environment

Delivering three distinct projects, worth millions, posed several challenges. Success required cohesive teamwork and careful solutioning and coordination.

1. Project Management
• ⌚ Time Allocation: Balancing deliverables across three time-sensitive projects.
• 💪 CEO Oversight: Ensuring on-time updates with the CEO monitoring the progress.

2. Resource Constraints
• 🔍 Finding Experts: Identifying suitable SMEs for technical deliveries.
• 🤝 Right Expertise & Ownership: Finding the right set of expertise SMEs and giving complete responsibility along with freedom was the main reason for success.
• Team Collaboration: Securing application team support for timely execution.

3. Cross-Team Coordination
• Stakeholder Alignment: Getting buy-in from 🤝 support, stakeholders, and various business units.
• Weekend Availability: Coordinating the availability of 5-10 different teams every weekend for seven months.

4. 🔧 Technical Complexity
• 🔄 Pipeline Creation: Developing 24 new 🔧 DevOps pipeline patterns for seven different 💻 technologies along with designing solution.

5. 🏆 Taking Complete Ownership
• Ensuring deliveries were covered end-to-end, including solutioning and timely execution.
• Clearly explaining the process to management for better understanding and alignment.
• Always ensures customer obsession.
• Appreciating the resources with open heart and mind.
• Bringing all into same page and bring up to speed for the on-time quality deliverables.

Lessons Learned: Building Foundations for the Future

Every project provides opportunities for 🌱 growth, but this year’s experiences were particularly 🚀 transformative. These lessons will guide me for years to come:

1. Communication is Key:
• Always obtain email confirmations instead of relying on verbal agreements.
• Document mutual agreements to ensure clarity.

2. Process Discipline:
• Maintain detailed 📝 minutes of meetings (MoM) for accountability.
• Commit to being present during production live windows every weekend.

3. ⌛ Preparedness:
• Ensure AVP (Application Verification Process) involvement during change windows.

4. 📝 What Could Have Been Done?
• Careful planning and mutually agreed 🗓️ date confirmation.

5. What Should We Do?
• Maintain a healthy 🏡 work-life balance.
• I have attended AWS re:Invent in Las Vegas and that experience was energizing in both career as well as personal.
• Healthy lifestyle is always important. Yes. I have successfully completed my first 5K RUN in AWS re:Invent.

## 🏆 Achievements: Celebrating Success

Despite the challenges, the results were 🎉 remarkable. Here are the key milestones:

1. ✨ Seamless Onboarding:
• Onboarded 33 applications from seven business units, integrating 345 repositories within nine months.

2. Innovative Solutions:
• Created 24 CD DevOps pipeline patterns for seven diverse technologies.

3. Recognition:
• Received appreciation from the CIO and senior management for 🏆 outstanding delivery and execution.

Looking Ahead

2024 has been a year of intense effort and significant rewards. The challenges I overcame and the lessons I learned have fortified my skills, making me better equipped to tackle future endeavors. As I move forward, I’ll carry these insights to continue driving impactful 🚀 change and innovation.

Step-by-Step Guide: Highly Available Architecture with ALB and Amazon ECS on AWS Fargate

marunkumar1983 — Fri, 31 Mar 2023 15:59:22 +0000

As cloud computing becomes increasingly important in modern IT infrastructure, containerization has emerged as a powerful tool for managing and deploying applications. And among the various containerization platforms available, AWS Fargate stands out as a particularly promising solution. AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. With its unique blend of scalability, cost-effectiveness, and simplicity, Fargate is poised to dominate the IT container world in the coming years.

I have designed and implemented the following architecture, and successfully completed a proof of concept. To ensure that below architecture flow is accurate and efficient, I plan to utilize three distinct verification methods.

Verification Method 1: Accessing Docker URL

Let's create a EC2 instance. Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the cloud. EC2 provides users with a flexible and cost-effective way to run applications in the cloud, without the need to invest in and manage their own physical hardware.

EC2 instance creation

T2.Micro (AMI: amzn2-ami-kernel-5.10-hvm-2.0.20230207.0-x86_64-gp2)

Create security group

Allow ssh port 22, http port 80 and map with EC2 instance

Docker Installation

Docker is an open platform for developing, shipping, and running applications. Docker image is a read-only template with instructions for creating a Docker container.



    sudo yum update -y
    sudo amazon-linux-extras install docker
    sudo service docker start
    sudo systemctl enable docker
    sudo usermod -a -G docker ec2-user
    sudo docker info

Create a Docker Image and run



 vi Dockerfile
            FROM ubuntu:18.04

            # Install dependencies
            RUN apt-get update && \
            apt-get -y install apache2

            # Install apache and write Fargate POC message
            RUN echo 'Fargate-ECS POC!' > /var/www/html/index.html

            # Configure apache
            RUN echo '. /etc/apache2/envvars' > /root/run_apache.sh && \
            echo 'mkdir -p /var/run/apache2' >> /root/run_apache.sh && \
            echo 'mkdir -p /var/lock/apache2' >> /root/run_apache.sh && \ 
            echo '/usr/sbin/apache2 -D FOREGROUND' >> /root/run_apache.sh && \ 
            chmod 755 /root/run_apache.sh

            EXPOSE 80

            CMD /root/run_apache.sh
 sudo docker build -t hello-fargate-ecs .
 sudo docker images --filter reference=hello-fargate-ecs
 docker run -t -i -p 80:80 hello-fargate-ecs

Accessing direct docker url.

Verification Method 2: Accessing Task URL

How two different services (EC2 --> ECR) communicate each other in AWS cloud?

By using IAM Role.

Create IAM role (IAM --> Roles --> "AmazonEC2ContainerRegistryFullAccess") and map with EC2 instance.

Create ECR and push image

Amazon Elastic Container Registry (Amazon ECR) is an AWS managed container image registry service that is secure, scalable, and reliable. It's similar to Docker Hub.



aws ecr create-repository --repository-name hello-fargate-ecs-repository --region us-west-2
docker tag hello-fargate-ecs <awsaccount>.dkr.ecr.us-west-2.amazonaws.com/hello-fargate-ecs-repository
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin <awsaccount>.dkr.ecr.us-west-2.amazonaws.com
docker push <awsaccount>.dkr.ecr.us-west-2.amazonaws.com/hello-fargate-ecs-repository

Fargate ECS Cluster Creation

An Amazon ECS cluster is a logical grouping of tasks or services.

Task Definitions Creation

Task definition is a template. A task definition is required to run Docker containers in Amazon ECS. The Task definition allows you to specify which Docker image to use, which ports to expose, how much CPU and memory to allocate, how to collect logs, and define environment variables.

Run Task by enabling public IP

An ECS task can consist of one or more containers that work together to perform a specific function. Task definition is used to define the configuration of a task. The task definition includes information such as the Docker image to use, the resources required, the container port mappings, and any data volumes to be mounted.

Make sure Public IP "Turned on" which will be used to hit from browser.

Task Running successfully and we could see public IP.

Hit the public IP from the browser to complete second verification method successfully.

Delete the task once verified. We will use ALB, ECS Service in our next verification method.

Verification Method 3: Accessing ALB URL

ALB (Application Load Balancer) distributes incoming traffic to multiple targets such as EC2 instances, containers, and Lambda functions, based on rules and conditions defined by the user.

ALB Target Group is a logical grouping of targets (such as EC2 instances or containers) that receive traffic from an Application Load Balancer based on the rules and conditions set by the user, allowing for efficient distribution of traffic and improved application availability and scalability.

Target Group Creation

Make sure to select "IP addresses" when creating target group.

ALB Creation

Note: You may incur cost if you are not a new AWS user. New AWS customers receive 750 hours per month as per below URL.
https://aws.amazon.com/elasticloadbalancing/pricing/

Create ALB by mapping created target group.

ECS Service Creation

ECS Service is used to guarantee that we always have some number of Tasks running at all times. Another very important point is that a Service can be configured to use a load balancer. Service will automatically register the container's EC2 instance with the load balancer.
Tasks cannot be configured to use a load balancer, only Services can. If a Task's container exits due to an error, or the underlying EC2 instance fails and is replaced, the ECS Service will replace the failed Task.

Make sure to select Launch type as "FARGATE".

Deployment configurations: Provide service name, task definition and desired task.

We already have ALB and target group ready. Make sure to map existing ALB and target group.

Service deployment is in progress.

Service deployment completed successfully. Desired tasks are running successfully.

Target group is healthy and registered automatically with task's public IP.

We can hit ALB URL from browser now.

Custom domain can be configured with Route53.

Note: Enterprise usually use "Private Subnet" for such configurations. End point configurations are required to establish communication.

I always wanted to connect with like-minded people. Here is my Linkedin. Ping/comment if any suggestions or clarifications required.

https://www.linkedin.com/in/arunkumarmannarmannan/

Thank you!