Forem: Yeshwanth L M

re:Play 2025

Yeshwanth L M — Sun, 07 Dec 2025 14:10:03 +0000

re:Invent Week AWS News Round-Up Summary

This week's AWS news was dominated by re:Invent announcements, with a massive flow of new services and features across multiple domains.

1. Performance and Compute Power

AWS Graviton5 CPU: Unveiled for up to 30% better performance and 40% better price performance than Graviton4 for general-purpose workloads, launching with new Amazon EC2 M9g instances.
Lambda Managed Instances: A new way to run Lambda functions on EC2 compute with serverless simplicity, granting access to specialized hardware and flexible EC2 pricing while AWS manages the infrastructure.

2. Serverless and Data Modernization

Lambda Durable Functions: Allows for the orchestration of multi-step applications and AI workflows directly in Lambda, featuring automatic checkpointing and year-long waits.
Amazon S3 Object Size Increase: The maximum object size limit has been significantly increased from 5 TB to 50 TB, supporting massive datasets like AI training corpora and high-resolution video.

3. Cost Optimization

Database Savings Plans: New, flexible, commit-based discounts of up to 35% for AWS managed databases across engines and regions, simplifying long-term cost optimization.
Amazon RDS for SQL Server: Now supports Microsoft SQL Server 2022 Developer Edition to help cut non-production licensing costs.

4. AI, Agents, and Foundation Models

New Foundation Models in Amazon Bedrock:
- Mistral AI models (fast, cost-effective options).
- Amazon Nova 2 Lite and Nova 2 Pro (Preview) with advanced step-by-step reasoning and a large 1M-token context window.
Agentic Services (AI-Powered Teammates - Previews):
- AWS Security Agent: Frontier agentic approach for AppSec, code analysis, and on-demand penetration testing.
- AWS DevOps Agent: Autonomous on-call teammate to accelerate incident response, correlate metrics/logs, and recommend resilience improvements.
- Kiro’s New Autonomous Agent: An AI dev teammate that runs multi-repo tasks and ships coordinated pull requests.

5. Security and Observability

Security Hub & GuardDuty Enhancements:
- AWS Security Hub: Now Generally Available with near real-time risk analytics and unified exposure views.
- GuardDuty Extended Threat Detection: Adds unified, AI-powered attack sequence findings for EC2 and ECS.
CloudWatch Unification: Now unifies log data management and analytics (operations, security, compliance) with new features like OCSF/OTel normalization and AI-powered queries.
X-Ray Transition to OpenTelemetry: Encouraging customers to adopt the open, vendor-neutral tracing standard.

Key Takeaways for Your Stream:

The Big Theme: A massive acceleration in AI and Agents across every part of the AWS stack (DevOps, Security, and App Dev).
The Performance Jump: Graviton continues to lead with major performance and cost efficiency gains.
Simplicity and Scale: The S3 limit increase and Lambda's move to durable functions are game-changers for large-scale data and complex serverless workflows.

🤖 AWS Outage (Oct 2025): Breakdown & Lessons

Yeshwanth L M — Wed, 22 Oct 2025 06:38:51 +0000

🌎 The Big Picture: What Happened?

When: Monday, October 20, 2025 (for about 15 hours).
What: A massive portion of the internet stopped working. Apps like Roblox, Snapchat, Duolingo, and even services like Alexa and Ring doorbells went offline.
Where: The failure started in AWS US-EAST-1 (Northern Virginia). This is the oldest, largest, and most important AWS data center region in the world.
Why: It was not a hack. It was an internal technical failure that caused a massive chain reaction (a cascading failure).

💥 The Story: A Cascade of Failures

The failure was like a set of dominos falling.

The First Domino (The Monitor): A tiny, internal AWS system that monitors the health of its own Network Load Balancers (NLBs) glitched.
The Second Domino (The Traffic Cop): Because the monitor failed, the NLBs (the "traffic cops" that direct data) also failed.
The Third Domino (The Phonebook): DynamoDB (a critical database used by thousands of apps) relied on those "traffic cops" for its DNS (the internet's "phonebook"). When the cops failed, the phonebook entry for DynamoDB went blank.
The Final Collapse: Apps across the internet tried to "call" DynamoDB but couldn't find its "phone number." This caused them to fail. The failure then spread to other core services in the region, like EC2 (servers) and IAM (logins), bringing down the entire region's "management layer."

Simple Analogy: A tiny fuse for the airport's control tower blew. This made the air traffic controllers go blind. Because they were blind, they couldn't tell planes which runway to land on. Soon, no planes could land (DynamoDB), and this caused the entire airport to shut down (the whole region).

🧑‍💻 3 Hard Lessons for Engineers

The us-east-1 Trap: We all use US-EAST-1 as our default. This outage proved many global services (like IAM logins) are still secretly controlled from this one region. A failure there can break your app everywhere.
The Cloud is Not Magic: The cloud is just someone else's computer. We must design our apps to survive cloud failures. We share responsibility for resilience.
Your App is Only as Strong as its Weakest Link: Thousands of apps failed because their entire system was in one region, or they "hardcoded" a dependency (like dynamodb.us-east-1.amazonaws.com) into their app.

🛠️ Your 5-Step Survival Guide (DevOps Action Plan)

Here are the concrete actions to prevent this from happening to you.

1. Stop Confusing Multi-AZ and Multi-Region
- Multi-AZ (multiple data centers in one city) is the minimum. It would not have saved you from this outage.
- Action: Use a Multi-Region architecture (e.g., US-EAST-1 and US-WEST-2) for critical apps. This can be Active-Passive (warm standby) or Active-Active (running in both places at once).
2. Use DNS Failover (Your Best Friend)
- This is non-negotiable for a multi-region setup.
- Action: Use Amazon Route 53 DNS Failover. It automatically detects a failing region and sends all your users to the healthy one, like a smart GPS rerouting traffic around a crash.
3. Design for Graceful Degradation
- Your app shouldn't be "all or nothing."
- Action: Ask: "If the 'upload' feature breaks, can I just disable the button and let the user keep browsing?" Decouple your services (e.g., with SQS queues) so a failure in one part doesn't crash the whole system.
4. Banish Hardcoded Endpoints
- Never, ever write us-east-1 directly in your application's code.
- Action: Audit your code. Use environment variables or a parameter store (like AWS SSM) to manage endpoints. Your app shouldn't care where it's running.
5. Practice Failing (Chaos Engineering)
- The companies that survived weren't lucky; they were prepared.
- Action: Run a GameDay (a simulated disaster). Intentionally break things in your test environment to find weaknesses. Ask your team, "What happens if I shut down the primary database right now?" and test it.

AWS Bedrock Powered VPC Flow Log Analyzer 🔍

Yeshwanth L M — Wed, 15 Oct 2025 11:46:27 +0000

Supercharge Your VPC Flow Log Analysis with Amazon Bedrock

In today's complex and dynamic cloud environments, understanding network traffic is crucial for security, troubleshooting, and performance optimization. AWS VPC Flow Logs provide a wealth of information about the IP traffic going to and from network interfaces in your VPC. However, manually analyzing these logs can be a daunting and time-consuming task.

What if you could use the power of generative AI to analyze your VPC Flow Logs using natural language? This is where the Amazon Bedrock-Powered VPC Flowlogs Analyzer comes in. This solution, available on GitHub, leverages the capabilities of Amazon Bedrock to provide a powerful and intuitive way to query and understand your network traffic.

The Challenge with VPC Flow Logs

VPC Flow Logs are a critical source of information for network monitoring and security analysis. They can help you:

Diagnose overly restrictive or permissive security group and NACL rules.
Monitor traffic that is reaching your instances.
Understand traffic patterns and identify anomalies.

However, the raw data from Flow Logs is verbose and can be difficult to parse. To get meaningful insights, you often need to use specialized tools or write complex queries, which can be a barrier for many users.

The Solution: A Generative AI-Powered Approach

The Amazon Bedrock-Powered VPC Flowlogs Analyzer provides a new paradigm for interacting with your network data. Instead of writing complex queries, you can simply ask questions in plain English. For example, you could ask:

IP Address Analysis:

"What source IP addresses do you see?"
"List all destination IP addresses"
"Which IP has the most traffic?"

Port and Protocol Analysis:

"What destination ports are being accessed?"
"Show me all TCP connections"
"Which protocols are being used?"

Security Analysis:

"Which connections were rejected?"
"Show me suspicious activities"
"Are there any failed connection attempts?"

Traffic Analysis:

"What's the largest data transfer?"
"Show me connections to external IPs"
"Which interface has the most traffic?"

The solution uses Amazon Bedrock, a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies, to understand your natural language queries and generate the appropriate code and queries to retrieve the information from your VPC Flow Logs.

Key Features

Natural Language Queries: Ask questions about your VPC Flow Logs in plain English.
Serverless and Scalable: The solution is built on a serverless architecture that can scale to handle large volumes of data.
Easy to Deploy: The entire infrastructure can be deployed using a single python script.
Extensible: The solution can be extended to support additional data sources and analysis capabilities.

Getting Started

To get started with the Amazon Bedrock-Powered VPC Flowlogs Analyzer, you will need:

An AWS account with access to Amazon Bedrock.
Python 3.10+ installed on your local machine.

The solution can be deployed using a simple script that sets up all the necessary AWS resources. Once deployed, you can start querying your VPC Flow Logs using natural language through the provided interface.

Example Usage

Here are a few examples of how you can use the solution to analyze your VPC Flow Logs:

Identify suspicious traffic: "Show me all traffic from IP address 192.0.2.1"
Troubleshoot connectivity issues: "Is there any traffic being blocked by a security group?"
Monitor application traffic: "What are the top 10 most active IP addresses?"

Conclusion

The Amazon Bedrock-Powered VPC Flowlogs Analyzer is a powerful tool that can help you unlock the full potential of your VPC Flow Logs. By leveraging the power of generative AI, you can gain deeper insights into your network traffic, improve your security posture, and optimize your cloud environment.

To learn more and get started, check out the GitHub repository.

Demo of the project: Hands-on Demo.

4 Ways to Transfer Files/Code From Your Local Computer to a Remote Cloud Server

Yeshwanth L M — Tue, 07 Oct 2025 09:35:13 +0000

File and Code Transfer: Local Machine → Cloud Server

This guide explains four popular methods to transfer files or code from your local computer to a remote cloud server, such as AWS EC2 running Ubuntu.

1. SCP (Secure Copy Protocol)
Quickly copy files or folders from your local machine to a remote server over SSH.

Copy a single file:

scp -i /path/to/private/key.pem /local/file/path user@SERVER_PUBLIC_IP:/PATH/INSIDE/SERVER

Copy an entire folder:

scp -i /path/to/private/key.pem /local/FOLDER/path user@SERVER_PUBLIC_IP:/PATH/INSIDE/SERVER

2. S3 Method (with IAM Role on EC2)
Upload files to an S3 bucket from your local machine. Then, grant your EC2 instance permission to access S3, and transfer files directly on the instance.

Sample IAM Policy for EC2 IAM Role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::file-transfer-demo-server-bucket",
        "arn:aws:s3:::file-transfer-demo-server-bucket/*"
      ]
    }
  ]
}

Upload from local to S3:

aws s3 cp /path/to/file s3://file-transfer-demo-server-bucket/

Download from S3 to EC2:

aws s3 cp s3://file-transfer-demo-server-bucket/IMG_8366.jpg /home/ubuntu/

The EC2 instance must have the proper IAM role attached with this policy.

3. WinSCP or Other SFTP GUI Tools
For Windows or those who prefer GUIs, use WinSCP (or FileZilla, Cyberduck) for drag-and-drop file transfer:

Connect using SFTP, your server’s IP, username (e.g., ubuntu), and your .pem SSH private key.
Drag files or folders from local to remote panel.
Make sure SFTP/SSH (port 22) is open.

4. GitHub: Clone Code from Repository
For code and structured projects, push your content to a GitHub (or GitLab) repository, then on the cloud server:

git clone https://github.com/yourusername/your-repo.git

Note:

Requires git installed on the cloud server.
Best for transferring source code, not large binary files.
For all SSH-based methods (SCP, SFTP, WinSCP), ensure the correct username and private key.
Always protect your credentials and configure least-privilege access for IAM roles and bucket policies.
For large files or many files, S3 or SCP may be more efficient than GitHub.

Enhance Your Cloud Development Workflow with Amazon Q CLI and MCP Servers

Yeshwanth L M — Tue, 07 Oct 2025 07:19:02 +0000

In the rapidly evolving landscape of cloud development, tools that streamline workflows and enhance productivity are invaluable. Amazon Q CLI, a command-line interface tool, brings intelligent assistance directly to your terminal with features like IDE-style autocomplete and agentic capabilities. When paired with Model Context Protocol (MCP) servers, Amazon Q CLI transforms into a powerful ally, offering a rich "toolbox" of functionalities for diverse development tasks. This guide will walk you through the complete setup of Amazon Q CLI and MCP servers, empowering you to automate and accelerate your AWS projects.

Understanding Amazon Q CLI and MCP Servers
Amazon Q CLI acts as a smart assistant within your terminal, understanding your context and providing intelligent suggestions or executing tasks based on your input. It aims to reduce manual effort and cognitive load during development.

Model Context Protocol (MCP) Servers are specialized applications that extend the capabilities of Amazon Q CLI. They provide specific functionalities, acting as plugins or modules that Amazon Q CLI can leverage to perform more complex operations, such as generating infrastructure diagrams, writing code, or managing Kubernetes resources. Think of them as a collection of experts that Amazon Q CLI can consult for specialized tasks.

Installing Amazon Q CLI
Before you begin, ensure you have an AWS Builder ID, as it's a prerequisite for using Amazon Q CLI.
The installation process for Amazon Q CLI is straightforward and varies slightly depending on your operating system:

For macOS:
The simplest way to install Amazon Q CLI on macOS is by using Homebrew:

brew install amazon-q

Alternatively, you can download the installer directly and follow the on-screen prompts.

For Windows (via WSL - Windows Subsystem for Linux):
If you're a Windows user, WSL offers a seamless Linux environment to run Amazon Q CLI.

Download the Amazon Q CLI zip file.
Unzip the contents to your desired location.
Open your WSL terminal and navigate to the unzipped directory.
Run the installation program provided within the unzipped files.

For Linux (e.g., Ubuntu):
For Linux distributions like Ubuntu, follow these steps:

First, ensure your system is up-to-date:

sudo apt update && sudo apt upgrade -y

Install libfuse2, which is often a dependency:

sudo apt install libfuse2 -y

Download the Amazon Q CLI .deb package. Install the .deb file using dpkg:

sudo dpkg -i amazon-q-cli.deb (Replace amazon-q-cli.deb) with the actual filename

After successful installation, you can log in to Amazon Q CLI using your AWS Builder ID:

q login

Once logged in, you can start interacting with Amazon Q CLI by simply typing q in your terminal.

Setting up MCP Servers Locally
MCP servers can be run locally using various tools like npx, uvx, or docker. This guide will demonstrate using uvx for its simplicity and efficiency.

Install uvx:
If you don't have uvx installed, you can typically install it via npm (Node Package Manager):

npm install -g uvx

Configure mcp.json:
Create or modify the mcp.json file located in your Amazon Q CLI configuration directory: ~/.aws/amazonq/mcp.json. This file defines the MCP servers that Amazon Q CLI will recognize and utilize.

Here's an example of how your mcp.json might look, including configurations for an AWS CDK MCP server and an AWS Diagram MCP server:

{
  "servers": [
    {
      "name": "awslabs.cdk-mcp-server",
      "command": "uvx",
      "args": ["awslabs.cdk-mcp-server"]
    },
    {
      "name": "awslabs.aws-diagram-mcp-server",
      "command": "uvx",
      "args": ["awslabs.aws-diagram-mcp-server"]
    }
  ]
}

In this configuration:

"name": A unique identifier for the MCP server.
"command": The executable command to run the MCP server (e.g., uvx).
"args": Any arguments required to run the specific MCP server.

By defining these entries, you're essentially telling Amazon Q CLI where to find and how to launch these powerful extensions.

Powerful Use Cases with Amazon Q CLI and MCP Servers
With Amazon Q CLI and MCP servers set up, you unlock a realm of possibilities for automating and enhancing your AWS development tasks:

Creating Architectural Diagrams: The awslabs.aws-diagram-mcp-server is incredibly useful for visually representing your cloud infrastructure. You can describe your desired architecture in natural language, and Amazon Q CLI, leveraging the MCP server, will generate a professional diagram.

Example Prompt:

q chat "Create a fault-tolerant, highly available real-time GPS tracking system using AWS ECS Fargate, Redis (ElastiCache), Aurora Serverless, and API Gateway. Use ALB for socket routing, integrate CloudWatch for logging, and S3 for archival storage. CI/CD should be managed by CodePipeline."

Amazon Q CLI will then process this prompt and output a detailed architectural diagram, saving you hours of manual drawing.
Generating Terraform Code:
For Infrastructure as Code (IaC) enthusiasts, MCP servers can assist in generating Terraform configurations. Describe the AWS resources you need, and Amazon Q CLI can provide the corresponding Terraform code, significantly accelerating your IaC development.
Managing Kubernetes Resources:
If you're working with Kubernetes on AWS (e.g., EKS), MCP servers can help you generate Kubernetes manifests (YAML files) based on your requirements, simplifying the deployment and management of containerised applications.

Conclusion
Integrating Amazon Q CLI with MCP servers transforms your command-line experience into an intelligent and highly efficient development environment. From generating complex architectural diagrams to automating infrastructure provisioning and managing Kubernetes resources, these tools empower you to work smarter, not just harder. By following the steps outlined in this guide, you can unlock a new level of productivity in your AWS cloud

Automate Your AWS MSK Kafka Cluster with Terraform: A Complete Guide

Yeshwanth L M — Tue, 07 Oct 2025 07:18:39 +0000

In a previous post, we walked through setting up an AWS MSK cluster manually using the AWS Console. While that's great for learning, it's not repeatable, scalable, or easy to manage. Today, we're taking it to the next level with Infrastructure as Code (IaC).

We'll use Terraform to define our entire MSK environment—VPC, subnets, security groups, IAM roles, the MSK cluster, and even a client EC2 instance—in a single set of configuration files. With a few simple commands, you can create, update, or destroy the whole setup.

What We'll Build:

A new VPC with public subnets across three Availability Zones.
All necessary networking (Internet Gateway, Route Tables).
A secure MSK Kafka cluster.
An EC2 instance pre-configured with Kafka tools and the correct authentication settings.
IAM roles and security groups that allow the EC2 instance to securely communicate with the MSK cluster.

Let's get started!

🛠️ Prerequisites

Before you begin, make sure you have the following:

An AWS Account: You'll need an AWS account with programmatic access. If you haven't already, configure your credentials locally using the AWS CLI:
```
aws configure
```
Terraform Installed: You'll need the Terraform CLI installed on your machine.

How to Install Terraform

Terraform is easy to install. Here are instructions for common operating systems.

On macOS (using Homebrew):

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

On Windows (using Chocolatey):

choco install terraform

On Linux (Debian/Ubuntu):

wget -O- [https://apt.releases.hashicorp.com/gpg](https://apt.releases.hashicorp.com/gpg) | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] [https://apt.releases.hashicorp.com](https://apt.releases.hashicorp.com) $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

After installation, verify it's working by running:

terraform --version

📜 Understanding the Terraform Code

Save the code from the prompt into a file named main.tf. Let's break down what each section of our Terraform configuration does.

1. Networking (VPC, Subnets, IGW)

This section builds the foundational network for our resources. We create a new VPC and then provision three public subnets, one in each available AWS Availability Zone for high availability. The Internet Gateway and Route Tables ensure our EC2 instance can reach the internet.

2. Security (Security Groups & SSH Key)

SSH Key: Terraform dynamically generates an RSA key pair. The public key is uploaded to AWS (aws_key_pair), and the private key is saved locally as msk-client-key.pem so you can SSH into the EC2 instance.
Security Groups: We create two security groups: one for the MSK cluster and one for the EC2 client. The rules are configured to allow the EC2 instance and the MSK cluster to communicate freely with each other on any port, while the EC2 instance only accepts incoming SSH traffic from the internet.

3. IAM Role for EC2

Instead of hardcoding AWS keys, we use an IAM Role. This block creates an aws_iam_role that the EC2 instance can "assume." The attached aws_iam_policy grants the instance specific permissions to connect, describe, read from, and write to the MSK cluster topics. This is the most secure way to grant AWS permissions to services.

4. The MSK Cluster

This is the core resource. We define an aws_msk_cluster with three small broker nodes (kafka.t3.small). Critically, the client_authentication block is configured to use IAM (iam = true), which allows our EC2 instance to authenticate using its assigned IAM role.

5. The EC2 Client Instance

This is where the magic happens! We launch a t2.micro EC2 instance. The user_data script is a powerful feature that runs automatically on the first boot. This script:

Installs Java.
Downloads and extracts the correct version of Kafka.
Downloads the AWS MSK IAM Auth library, which is required for IAM authentication.
Creates the client.properties file with the exact configuration needed for our Kafka tools to authenticate with MSK via IAM.

This means that as soon as the instance is ready, it's already fully configured to act as a Kafka client!

🚀 Deploying the Infrastructure

Note: You can find the complete terraform script here: https://github.com/yeshwanthlm/AWS-MSK-Crash-Course/blob/main/terraform/msk-cluster-with-vpc-ec2-client.tf

With the msk-cluster-with-vpc-ec2-client.tf file saved, running the deployment is as simple as three commands.

Initialize Terraform

This command downloads the necessary AWS provider plugin.

terraform init

Plan the Deployment

This is a dry run. Terraform shows you exactly what resources it will create, change, or destroy.

terraform plan

Apply the Configuration

This command executes the plan and builds everything in your AWS account. Type yes when prompted.

terraform apply

Grab a coffee! ☕ The MSK cluster creation is the longest step and will take around 20-30 minutes. Once finished, Terraform will display the outputs, including the public IP of your EC2 instance.

✅ Connecting and Testing Your Cluster

Once terraform apply is complete, let's verify everything works.

Get the EC2 Public IP

Find the public IP from the Terraform output. You can also run terraform output ec2_public_ip.

SSH into the EC2 Instance

The private key msk-client-key.pem was saved in your project directory.

# Make sure to set correct permissions if needed
chmod 400 msk-client-key.pem

ssh -i "msk-client-key.pem" ec2-user@<YOUR_EC2_PUBLIC_IP>

Get Your Bootstrap Brokers String

In your local terminal (not the SSH session), get the connection string from the Terraform outputs.

terraform output bootstrap_brokers

Copy this long string. You'll need it for the next steps.

Create a Kafka Topic

Inside your EC2 SSH session, navigate to the Kafka bin directory and create a topic.

bin/kafka-topics.sh --create \
  --bootstrap-server <bootstrapServerString> \
  --command-config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties \
  --replication-factor 3 \
  --partitions 1 \
  --topic my-first-topic

Start a Producer

In the same terminal, start the console producer. This will give you a > prompt.

bin/kafka-console-producer.sh \
  --broker-list <bootstrapServerString> \
  --producer.config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties \
  --topic my-first-topic

Type a message like Hello from Terraform! and press Enter.

Start a Consumer (in a new terminal)

Open a second terminal window and SSH into your EC2 instance again. Navigate to the same bin directory and run the consumer:

bin/kafka-console-consumer.sh \
  --bootstrap-server <bootstrapServerString> \
  --consumer.config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties \
  --topic my-first-topic \
  --from-beginning

You should instantly see Hello from Terraform! appear in your consumer window. Success! 🎉

🧹 Cleaning Up

Don't forget to tear down your infrastructure to avoid ongoing AWS charges! The beauty of Terraform is that this is a single, simple command.

terraform destroy

Type yes when prompted, and Terraform will neatly remove all the resources it created.

Conclusion

You've successfully automated a complete AWS MSK environment using Terraform. With Infrastructure as Code, you now have a repeatable, version-controlled, and reliable way to manage your Kafka clusters on AWS. This is the foundation for building powerful, event-driven applications at scale.

A Beginner's Guide to AWS MSK: From Cluster Setup to Your First Message

Yeshwanth L M — Mon, 06 Oct 2025 09:48:49 +0000

🚀 Getting Started with AWS MSK: Your First Kafka Cluster

Ever wondered how massive, data-driven apps handle real-time event streams for things like live analytics, log aggregation, or IoT data? A key technology behind this is Apache Kafka, a powerful open-source distributed event streaming platform.

However, setting up and managing Kafka on your own can be a complex and time-consuming task. This is where AWS Managed Streaming for Apache Kafka (MSK) comes in.

In this guide, we'll cover:

What AWS MSK is and why it's useful.
The different cluster types available.
A full, hands-on tutorial to create your own MSK cluster and send your first messages from an EC2 instance.

Let's dive in!

## 🤔 What is AWS MSK?

Think of Apache Kafka as a high-speed, central post office for your application's data. 📮 Applications can send messages (produce) to different mailboxes (topics), and other applications can pick them up (consume) when they're ready.

AWS MSK is a fully managed service that runs this post office for you. It handles the heavy lifting so you don't have to:

Provisioning Servers: No need to pick, set up, or configure EC2 instances.
Kafka Software Management: AWS handles the installation, patching, and upgrades of Kafka.
High Availability: MSK automatically distributes your cluster across multiple data centers (Availability Zones) to ensure it's resilient to failure.

In short, you get the full power of Apache Kafka without the operational overhead.

## ✨ Why Do You Need AWS MSK?

So, why choose MSK over managing Kafka yourself?

✅ Simplified Operations: Spend your time building applications, not managing infrastructure.
🌐 Highly Available & Scalable: MSK is built for resilience. You can easily scale your cluster's compute and storage with a few clicks and no downtime.
🔒 Secure by Default: Integrates seamlessly with AWS services like IAM for authentication, VPC for network isolation, and KMS for encrypting your data at rest and in transit.
💯 Fully Compatible: It's 100% compatible with open-source Apache Kafka. You can migrate existing applications, tools, and plugins without changing your code.

## Cluster Types: Provisioned vs. Serverless

When creating a cluster, MSK gives you two options:

### Provisioned Clusters

This is the traditional model. Think of it like leasing a fleet of trucks. 🚚 You choose the size and number of trucks (broker types and count), and you have full control over the configuration.

Best for: Predictable, high-volume workloads where you want fine-grained control.
You pay for: The resources you provision, 24/7.

### Serverless Clusters

This is a newer, more flexible option. Think of it like a pay-per-package delivery service. 📦 You don't manage any trucks; you just send your data, and the service automatically scales to handle the load.

Best for: New apps, or workloads with variable or unpredictable traffic.
You pay for: The data you stream and retain (throughput and storage).

For this tutorial, we'll use a Provisioned cluster to see all the underlying configurations.

## 🛠️ Hands-On Demo: Creating Your Cluster and Sending Messages

Time to build! We'll create an MSK cluster and an EC2 instance to communicate with it.

### Part 1: Create the MSK Cluster

In the AWS Console, navigate to MSK.
Click Create cluster and choose the Custom create method.
Cluster settings:
- Cluster name: my-demo-msk-cluster
- Cluster type: Provisioned
- Apache Kafka version: Use the recommended default.
Networking:
- Select your desired VPC.
- Choose at least two Availability Zones and select a subnet in each. For a simple demo, public subnets are fine, but use private subnets for production.
Security:
- Under Access control methods, check the box for IAM role-based authentication. This is the most secure and straightforward way to connect from other AWS services.
Review and Create. The cluster will take 20-30 minutes to become Active.

### Part 2: Set Up the EC2 Client & Security Groups

While the cluster is creating, let's set up our client machine.

Launch an EC2 Instance:

Go to the EC2 service and click Launch instance.
Name: MSK-Client-EC2
AMI: Amazon Linux 2
Instance Type: t2.micro (Free Tier eligible)
Network: Crucially, select the same VPC and one of the subnets you used for your MSK cluster.

IAM Role: Attach an IAM role to the instance with a policy that allows it to connect to MSK. A simple policy for this demo would be:

{
"Version": "2012-10-17",
"Statement": [
{
    "Effect": "Allow",
    "Action": [
        "kafka-cluster:Connect",
        "kafka-cluster:AlterCluster",
        "kafka-cluster:DescribeCluster"
    ],
    "Resource": "*"
},
{
    "Effect": "Allow",
    "Action": [
        "kafka-cluster:*Topic*",
        "kafka-cluster:WriteData",
        "kafka-cluster:ReadData"
    ],
    "Resource": "*"
},
{
    "Effect": "Allow",
    "Action": [
        "kafka-cluster:AlterGroup",
        "kafka-cluster:DescribeGroup"
    ],
    "Resource": "*"
}
]
}

Configure Security Groups (The Important Part!):
- MSK Cluster Security Group: Find the security group attached to your MSK cluster. Add an inbound rule to allow All traffic from the security group of your MSK-Client-EC2 instance.
- EC2 Instance Security Group: Find the security group for your EC2 instance. Add an inbound rule to allow All traffic from the MSK cluster's security group. Also, make sure you have a rule to allow SSH from your IP.

This two-way rule allows the EC2 instance and the MSK brokers to communicate freely within the VPC.

### Part 3: Connect and Send Messages

Once your MSK cluster is Active and your EC2 instance is running, SSH into the instance.

Install Tools:

    # Update and install Java
    sudo yum update -y
    sudo yum -y install java-11

# Download and extract Apache Kafka tools
wget https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz

tar -xzf kafka_2.13-3.6.0.tgz

cd kafka_2.13-3.6.0/libs

wget https://github.com/aws/aws-msk-iam-auth/releases/download/v1.1.1/aws-msk-iam-auth-1.1.1-all.jar

Create Client Properties File:
We need to tell the Kafka tools to use IAM for authentication.

# Create the config file
cat << EOF > client.properties
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
EOF

Get Bootstrap Servers:
In the MSK console, click your cluster, then View client information. Copy the Bootstrap servers endpoint for IAM.

Create a Topic:
Let's create a "mailbox" called my-first-topic. Replace <YOUR_BOOTSTRAP_SERVERS> with the endpoint you just copied.

bin/kafka-topics.sh --create --bootstrap-server <bootstrapServerString> --command-config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties --replication-factor 3 --partitions 1 --topic my-first-topic

Start a Producer:
This command gives you a prompt where you can type messages to send.

bin/kafka-console-producer.sh --broker-list <bootstrapServerString> --producer.config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties --topic my-first-topic

Type Hello MSK! and hit Enter. Type a few more messages.

Start a Consumer (in a new terminal):
Open a second SSH session to your EC2 instance, navigate to the same bin directory, and run:

bin/kafka-console-consumer.sh --bootstrap-server <bootstrapServerString> --consumer.config /home/ec2-user/kafka_2.13-3.6.0/bin/client.properties --topic my-first-topic --from-beginning

You should see the messages you typed in the producer terminal appear instantly! 🎉

## Conclusion

Congratulations! You've successfully deployed a highly available Apache Kafka cluster using AWS MSK, configured secure access from an EC2 instance, and sent your first real-time messages.

By using MSK, you get to leverage the power of Kafka for your event-driven applications without the headache of managing the underlying infrastructure.

Thanks for reading! Let me know in the comments if you have any questions.

aws #kafka #cloud #devops #tutorial

Automate AWS RDS & Aurora Recommendations with Lambda and EventBridge

Yeshwanth L M — Mon, 06 Oct 2025 05:49:58 +0000

Never Miss an AWS Database Tune-Up Again: Your Automated Alert System

As developers, we're constantly juggling a million things. The last thing we want to worry about is whether our Amazon RDS and Aurora databases are running at their peak. AWS provides a ton of great recommendations to boost performance, security, and reliability, but who has the time to check for them manually? 😥

What if you could get these crucial recommendations delivered straight to your inbox, automatically? In this post, we'll walk you through a simple yet powerful solution to automate this process using AWS Lambda, Amazon EventBridge, and Amazon Simple Email Service (SES).

Let's dive in! 🚀

The Big Picture: How It Works

We'll be creating a serverless workflow that does the following:

Scheduled Trigger: An Amazon EventBridge rule will kick things off on a schedule you define (e.g., once a day, once a week).
Recommendation Fetching: The EventBridge rule will trigger an AWS Lambda function. This function's job is to go and fetch all the latest recommendations for your RDS and Aurora instances.
Email Notification: Once the Lambda function has the recommendations, it will format them into a neat HTML email and send it to you and your team using Amazon SES.

This "set it and forget it" system ensures you're always in the loop about potential optimizations for your databases.

Let's Get Building! The Step-by-Step Guide

Here’s a high-level overview of the steps we’ll take to bring this solution to life:

1. Tag, You're It! Tag Your Database Instances

First things first, you'll need a way to identify the database instances you want to monitor. The easiest way to do this is by applying a tag. For example, you could create a tag with the key send-recommendations and the value true for all the databases you want to receive notifications for.

2. Get Your Email Ready with Amazon SES

To send emails, you'll need to set up Amazon SES. This involves verifying your email address or domain to ensure you're authorized to send emails from it.

3. Permissions, Permissions, Permissions: The IAM Role

Our Lambda function needs permission to access other AWS services (like RDS and SES). We'll create an IAM (Identity and Access Management) role with a policy that grants the necessary permissions. This is a crucial security step to ensure our function only has access to what it needs.

4. The Brains of the Operation: The Lambda Function

This is where the magic happens! We'll write a Lambda function (you can use your favorite language, like Python or Node.js) that will:

Fetch recommendations for your RDS and Aurora instances (using the tag we created earlier to filter them).
Format the recommendations into a user-friendly HTML email.
Send the email using Amazon SES.

5. Set the Clock with EventBridge

Finally, we'll create an Amazon EventBridge rule that runs on a schedule. This rule will be configured to trigger our Lambda function, kicking off the entire process automatically.

Get the Code

You can find the complete source code, including the Lambda function and IAM policy, on GitHub. Feel free to clone it and get started right away!

➡️ GitHub Repo: https://github.com/yeshwanthlm/RDS-Automation/tree/main

Why You Should Do This Right Now

By automating your RDS and Aurora recommendations, you'll:

Save Time and Effort: No more manually checking for recommendations.
Stay Proactive: Address potential issues before they become major problems.
Boost Performance and Security: Keep your databases running smoothly and securely.
Never Miss a Beat: Ensure critical recommendations are never overlooked.

The best part? This solution is incredibly flexible. You can customize the filtering logic to match your organization's specific needs and priorities.

So what are you waiting for? Take an hour to set this up today and thank yourself later. Happy coding! 🎉

How to Convert AWS Clicks into CDK/CloudFormation (The EASY Way)

Yeshwanth L M — Sun, 05 Oct 2025 13:28:51 +0000

The Manual Grind is Over

Let's be real: setting up infrastructure in the AWS Management Console is a right of passage. We've all been there—clicking through menus, configuring settings, and trying to remember every single step we took. It works for a one-off task, but what happens when you need to do it again? And again? It's slow, error-prone, and doesn't scale.

For years, the answer has been Infrastructure as Code (IaC) using tools like AWS CloudFormation or the AWS Cloud Development Kit (CDK). But this comes with a steep learning curve. What if you could get the best of both worlds? What if you could perform your setup once in the console and have AI generate the code for you?

That's exactly what Console-to-Code, a powerful feature in Amazon Q Developer, does. It watches your actions in the console and magically transforms them into clean, ready-to-use IaC.

Why Console-to-Code is a Game-Changer 🚀

This isn't just another tool; it's a fundamental shift in how we can approach cloud automation.

Bridge the Skills Gap: New to IaC? No problem. Use the visual console you're comfortable with and get high-quality code as your output. It's an incredible way to learn by doing.
Massive Time Saver: Instead of spending hours writing YAML or TypeScript from scratch, you can generate a solid baseline in minutes. This dramatically speeds up prototyping and deployment.
Enforce Consistency and Best Practices: By converting manual setups into code, you create a repeatable, reliable process. This ensures every environment you spin up is identical, eliminating the "it worked on my machine" problem for infrastructure.
Multi-Language Support: Whether you're a fan of Python, Java, TypeScript with the AWS CDK, or prefer the simplicity of CloudFormation (JSON/YAML), Console-to-Code has you covered.

From Clicks to Code: Your 3-Step Guide

Ready to try it out? The process is incredibly straightforward. Here’s how to turn your console actions into reusable code.

Step 1: Hit Record

First, you need to tell Amazon Q to start watching.

Log in to your AWS Management Console.
Navigate to a supported service like VPC, RDS, or EC2.
On the far right edge of your browser window, you'll see the Console-to-Code icon. Click it.
Simply click "Start recording."

That's it! The tool is now actively recording every infrastructure-related action you take.

Step 2: Build Your Infrastructure

Now for the fun part. Go through the console and perform the tasks you want to automate.

Launch an EC2 instance.
Create a new S3 bucket and configure its policies.
Set up a VPC with subnets and route tables.

You can even move between different services during a single recording session. The Console-to-Code panel will keep track of everything you do.

Step 3: Generate Your Code

Once you've completed your setup, it's time to get your code.

In the Console-to-Code panel, you'll see a list of all the actions you performed.
Review and select the specific actions you want to include in your IaC template.
At the bottom of the panel, choose your desired output format from the dropdown menu (e.g., AWS CDK - Python or CloudFormation - YAML).
Click the "Generate chosen language" button.

Instantly, the panel will display the generated code. You'll not only get the IaC template but also the equivalent CLI commands for your reference.

Final Thoughts & Best Practices

Console-to-Code is an incredibly powerful feature for accelerating your cloud journey. To get the most out of it:

Plan Your Steps: Have a clear idea of what you want to build before you start recording to keep your generated code clean and focused.
Review the Output: The generated code is a fantastic starting point, but always review it to understand what's happening and see if you can make any custom tweaks.
Use it as a Learning Tool: If you're new to IaC, generate code for simple tasks first. Study the output to understand how resources are defined and linked. It’s one of the most practical ways to learn.

The era of choosing between the easy-to-use console and powerful automation is over. With Console-to-Code, you can finally have both.

Happy building!

Connect with the Author

Thanks for reading! I'm Yeshwanth L M, an AWS Community Builder passionate about making cloud and DevOps accessible to everyone. If you found this article helpful, let's connect!

YouTube: Subscribe to TechWithYeshwanth for more tutorials.
Community: Join the Channel Membership for exclusive perks.
GitHub: Follow my projects and contribute.
Blog: Read more articles on dev.to.
Instagram: Follow @techwithyeshwanth for daily tech content.
LinkedIn: Connect on LinkedIn.
Book a 1:1 Call: Schedule a mentoring session with me on TopMate.

Conquering the AWS Certified Solutions Architect Associate Exam: Your Essential Study Guide and Tips from A Monk in Cloud

Yeshwanth L M — Fri, 06 Jun 2025 16:42:05 +0000

Hey Cloud Enthusiasts! Yeshwanth here, ready to guide you through the journey of preparing for one of the most coveted certifications in the cloud computing landscape: the AWS Certified Solutions Architect Associate (SAA-C03) exam. This certification is a powerful testament to your ability to design and implement well-architected infrastructures within the AWS Cloud.

Understanding the Role of an AWS Solutions Architect:
As a Solutions Architect, your core responsibility is to bridge customer requirements with effective, scalable, and secure AWS solutions. This means not only knowing which services to use but also understanding how to combine them to create infrastructure that adheres to key principles: efficiency, security, reliability, fault tolerance, and cost-effectiveness. These are the pillars upon which your SAA-C03 exam will be built.

Your Primary Study Resources:
When it comes to official study materials, consider the following your go-to sources:

AWS Whitepapers: These provide in-depth technical details and best practices.
AWS FAQs: Often overlooked, but they contain crucial Q&A on service functionalities and limitations.
AWS Documentation: The authoritative source for all AWS services.
Best video tutorials by Adrian Cantrill.

Beyond reading, hands-on experience in building systems on AWS is incredibly beneficial. The exam includes many scenario-based questions, and practical knowledge will greatly enhance your understanding. For the most precise and up-to-date information on the exam structure and content, always consult the official SAA-C03 Exam Guide directly on the AWS Certification website. Give it a quick read to set your expectations!

Starting Your SAA-C03 Journey: A Recommended First Step
If you're relatively new to AWS, I highly recommend kicking off your studies with the FREE AWS Certified Cloud Practitioner Essentials digital course. This interactive course covers fundamental AWS Cloud concepts, services, security, architecture, pricing, and support plans. It's an excellent foundation before you delve into the more complex topics of the SAA-C03.

Navigating Study Materials: Staying Up-to-Date
The internet is awash with resources claiming to be the "best" for the SAA-C03 exam. However, it's critical to be discerning. Some resources might be outdated and won't cover the latest services or features introduced in the SAA-C03 exam version.

My golden rule: Always check the official AWS Certification website for the most current information. The official AWS Certified Solutions Architect Associate SAA-C03 exam page is your single source of truth. Here, you'll find the official exam guide, sample questions, and the link to schedule your exam.

Key Services for the SAA-C03 Exam (SAA-C03 Version Specific)
For the SAA-C03 exam version, be aware of services like:

AWS Global Accelerator
Elastic Fabric Adapter (EFA)
Elastic Network Adapter (ENA)
AWS ParallelCluster
Amazon FSx (for Windows File Server and Lustre)
AWS DataSync
AWS Directory Service
High Performance Computing concepts
Aurora Serverless

Core AWS Services to Master for the SAA-C03 Exam
While the list above highlights specific updates, a deep understanding of the following core AWS services is non-negotiable for the SAA-C03:

EC2 (Elastic Compute Cloud): This is foundational. Understand instance types, AMIs, storage options (EBS), and networking.
Lambda: The heart of serverless computing. Know its integrations with other AWS services for building complete serverless applications.
Elastic Load Balancer (ELB): Crucial for high availability. Study the different types (Application, Network, Gateway) and their features.
Auto Scaling: Understand what services can be auto-scaled, the triggers for scaling, and how it manages instance counts.
Elastic Block Store (EBS): Primary storage for EC2. Familiarize yourself with volume types, security, backup, and restore procedures.
S3 / Glacier: Explore the various S3 storage classes, their use cases, and capabilities like static website hosting, access policies, and lifecycle management. S3 is a heavily tested service!
Storage Gateway: Understand its purpose and when to use it versus direct S3 or EBS. Differentiate between DataSync and Storage Gateway.
EFS (Elastic File System): Often compared with other storage solutions. Know when EFS is the right choice, considering cost and efficiency trade-offs.
RDS / Aurora: Understand the differences between various RDS databases and what makes Aurora unique. Learn about parameter groups, option groups, and subnet groups.
DynamoDB: A frequently tested NoSQL database. Compare it with RDS, ElastiCache, and Redshift. It's often paired with Lambda for serverless applications.
ElastiCache: Focus on Redis and its functions. Identify scenarios where caching can improve performance, such as managing ELB session state or optimizing RDS.
VPC / NACL / Security Groups: Master the components of a Virtual Private Cloud (subnets, route tables, internet gateways, NAT gateways, VPN gateways). Crucially, understand the distinct roles of Network Access Control Lists (NACLs) and Security Groups.
Route 53: Study the different record types and routing policies. Be familiar with hosted zones and domains.
IAM (Identity and Access Management): IAM Users, Groups, Policies, and Roles are fundamental. Understand how IAM integrates with other services for secure applications and be aware of best practices.
CloudWatch: Learn about monitoring in AWS, metrics, CloudWatch Logs, CloudWatch Alarms, and custom metrics using the CloudWatch Agent.
CloudTrail: Understand how CloudTrail works and the types of logs it stores, differentiating them from CloudWatch Logs.
Kinesis: Have a high-level understanding of Kinesis Data Streams, including sharding, and how the different Kinesis services operate.
CloudFront: Learn how CloudFront speeds up content delivery, its content sources, and supported SSL certificates.
SQS (Simple Queue Service): Understand how SQS decouples systems, message management (standard, FIFO, dead-letter queues), and the differences between SQS, SNS, SES, and Amazon MQ.
SNS (Simple Notification Service): Study its function, integrations, and supported notification recipients.
SWF (Simple Workflow Service) / CloudFormation / OpsWorks: Understand the functions, capabilities, and typical use cases for each of these orchestration and automation services.

Crucial Comparison Scenarios
Based on my exam experience, pay special attention to the nuances and appropriate use cases when comparing these services:

AWS DataSync vs. Storage Gateway
FSx (considerations for cold and hot storage)
Cross-Region Read Replicas vs. Multi-AZ RDS (focus on high-availability aspects)
Amazon Object Key vs. Object Metadata
Direct Connect vs. Site-to-Site VPN
AWS Config vs. AWS CloudTrail
Security Group vs. NACL
NAT Gateway vs. NAT Instance
Geolocation routing policy vs. Geoproximity routing policy on Route 53

Your Path to Success: Practice and Hands-On Experience
Beyond the official documentation, consider leveraging reputable study aids like practice exams. Aim for consistent high scores to ensure you're truly prepared for the exam's format and difficulty.

Most importantly, get hands-on! Sign up for an AWS Free Tier account and perform lab exercises. Experiencing these services directly – spinning up EC2 instances, creating S3 buckets, configuring VPCs – will deeply embed the concepts and help you remember what each service is capable of. It's an invaluable part of the learning process.

I wish you all the best on your AWS Certified Solutions Architect Associate exam journey! If you have any questions or want to share your own tips, drop them in the comments below.

Happy Architecting!

Yeshwanth AKA A Monk in Cloud

Simplifying Multi-Region EC2 Management with AWS EC2 Instance Manager

Yeshwanth L M — Thu, 22 May 2025 13:59:11 +0000

Introduction

Managing EC2 instances across multiple AWS regions can be a challenging task. As your cloud infrastructure grows, switching between regions in the AWS Console becomes time-consuming and inefficient.
Today, I'm excited to introduce the AWS EC2 Instance Manager - a lightweight, browser-based tool I've developed to solve this exact problem.

🔗 Try the app here:
https://gray-plant-037bead10.6.azurestaticapps.net/

📂 GitHub Repo:
https://github.com/yeshwanthlm/AWS-EC2-Instance-Manager

The Challenge of Multi-Region Management

If you're managing AWS infrastructure, you've likely encountered these pain points:

• Constantly switching between AWS regions to check instance status
• Difficulty getting a consolidated view of all running instances
• Time wasted navigating through the AWS Console for simple operations
• Need for a quick way to stop or terminate instances across regions

These challenges inspired me to create a simple yet powerful solution that runs entirely in your browser.

Introducing AWS EC2 Instance Manager

The AWS EC2 Instance Manager is a static web application that provides a unified interface for managing EC2 instances across all AWS regions. What makes this tool special is its simplicity - there's no
backend server, no complex setup, and no additional infrastructure required.

Key Features

Cross-Region Visibility

The application automatically discovers and queries all AWS regions, providing a single dashboard view of your EC2 instances. This eliminates the need to manually switch between regions in the AWS
Console.

Focus on Active Resources

The tool shows only running instances by default, helping you focus on active resources that might be incurring costs. This filtering makes it easier to identify instances that could potentially be
stopped to optimize your AWS spending.

Streamlined Instance Management

With just a single click, you can stop or terminate instances directly from the interface. This streamlined approach saves valuable time compared to navigating through the AWS Console for each action.

Security-First Design

Security is a top priority. The application handles AWS credentials with care - they're stored only in memory and never persisted or sent to any server other than AWS directly. This client-side only
architecture minimizes security risks.

Responsive Interface

Whether you're at your desk or on the go, the responsive design ensures the tool works seamlessly across desktop and mobile browsers.

Technical Implementation

The AWS EC2 Instance Manager is built with simplicity in mind:

• Pure Web Technologies: The application uses vanilla HTML, CSS, and JavaScript without any frameworks, keeping it lightweight and fast.
• AWS SDK for JavaScript: Leverages the AWS SDK v2 to interact with AWS services directly from the browser.
• Client-Side Architecture: All processing happens in your browser, with no server-side components required.

This approach makes the tool incredibly portable - you can run it from any computer with a modern web browser.

Getting Started

Using the EC2 Instance Manager is straightforward:

Clone or download the repository from GitHub
Open the index.html file in your web browser
Enter your AWS credentials (Account ID, Access Key ID, and Secret Access Key)
Click "Connect to AWS" to fetch your running instances
Use the provided buttons to stop or terminate instances as needed

Required Permissions

To use the tool effectively, your AWS credentials need these specific permissions:
• ec2:DescribeRegions - To discover all available AWS regions
• ec2:DescribeInstances - To list EC2 instances in each region
• ec2:StopInstances - To stop running instances
• ec2:TerminateInstances - To terminate instances when needed

Security Considerations

While the tool is designed with security in mind, it's important to note that storing AWS credentials in a browser application is not recommended for production environments. For production use, consider
these more secure alternatives:

• Implement authentication using Amazon Cognito
• Create a backend using API Gateway and Lambda to handle AWS operations
• Use proper IAM roles with least privilege principles

Future Roadmap

This is just the beginning for the EC2 Instance Manager. Future enhancements may include:

• Adding the ability to start stopped instances
• Including more detailed instance information and advanced filtering options
• Expanding support to other AWS resources like RDS databases and Lambda functions
• Implementing secure credential storage with Amazon Cognito

Troubleshooting Tips

If you encounter issues while using the tool:

• Verify your AWS credentials are correct and have the necessary permissions
• Ensure you have running EC2 instances in your account
• Check your browser's console for any error messages
• Make sure your browser isn't blocking scripts from loading

Conclusion

The AWS EC2 Instance Manager demonstrates how a simple tool can significantly improve the efficiency of cloud resource management. By providing a consolidated view of EC2 instances across all regions and
enabling quick actions, it saves valuable time for AWS users.

I built this tool to address a common pain point in my own AWS management workflow, and I hope it proves useful for others facing similar challenges. The project is open-source and available for anyone to
use, modify, and improve.

Whether you're managing a handful of instances or a large fleet across multiple regions, this tool can help streamline your EC2 management tasks and provide better visibility into your AWS resources.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

This project was developed by Yeshwanth L M at A Monk in Cloud. The AWS EC2 Instance Manager is licensed under the MIT License, making it freely available for personal and
commercial use.

Building a Tech Logo Memory Match Game with Pygame

Yeshwanth L M — Wed, 21 May 2025 15:36:34 +0000

Memory games are not only fun but also help improve cognitive skills like concentration and short-term memory. In this blog post, I'll walk you through how I built a tech-themed memory matching card game using Python and Pygame.

Project Overview

The Tech Logo Memory Match Game is a classic card-matching game where players flip cards to find matching pairs. What makes this version special is that it features logos from popular tech companies, making it both entertaining and educational for tech enthusiasts.

Key Features

Tech-themed cards: Featuring logos from companies like AWS, Azure, Docker, GitHub, and more
Multiple difficulty levels: Easy (3x4 grid), Medium (4x4 grid), and Hard (4x5 grid)
Countdown timer: Adding excitement and challenge
Score tracking: Keeping count of moves and completion time
Smooth animations: Card flipping animations for better user experience

The Development Process

1. Setting Up the Project

I started by setting up the basic structure of the project:

import pygame
import random
import time
import sys

# Initialize pygame
pygame.init()

# Set up display
WIDTH, HEIGHT = 800, 600
screen = pygame.display.set_mode((WIDTH, HEIGHT))
pygame.display.set_caption("Tech Logo Memory Match")

2. Creating the Card Class

The Card class is the foundation of the game, handling the state and behavior of each card:

class Card:
    def __init__(self, x, y, width, height, logo_img, back_img):
        self.x = x
        self.y = y
        self.width = width
        self.height = height
        self.logo_img = logo_img
        self.back_img = back_img
        self.is_flipped = False
        self.is_matched = False

    def draw(self, screen):
        if self.is_flipped or self.is_matched:
            screen.blit(self.logo_img, (self.x, self.y))
        else:
            screen.blit(self.back_img, (self.x, self.y))

    def is_clicked(self, mouse_pos):
        return (self.x <= mouse_pos[0] <= self.x + self.width and
                self.y <= mouse_pos[1] <= self.y + self.height)

3. Implementing Game Logic

The core game logic involves:

Shuffling and distributing cards
Handling card flips
Checking for matches
Managing the timer and game state

def check_for_match(flipped_cards):
    if len(flipped_cards) == 2:
        if flipped_cards[0].logo_img == flipped_cards[1].logo_img:
            flipped_cards[0].is_matched = True
            flipped_cards[1].is_matched = True
            return True
    return False

4. Creating the User Interface

A clean, intuitive UI is crucial for a good gaming experience:

def draw_ui(screen, time_left, moves, difficulty):
    # Draw background
    screen.fill((40, 44, 52))

    # Draw timer
    font = pygame.font.SysFont('Arial', 24)
    timer_text = font.render(f"Time: {time_left}s", True, (255, 255, 255))
    screen.blit(timer_text, (20, 20))

    # Draw moves counter
    moves_text = font.render(f"Moves: {moves}", True, (255, 255, 255))
    screen.blit(moves_text, (WIDTH - 120, 20))

    # Draw difficulty
    diff_text = font.render(f"Difficulty: {difficulty}", True, (255, 255, 255))
    screen.blit(diff_text, (WIDTH // 2 - 80, 20))

5. Adding Difficulty Levels

Different difficulty levels keep the game challenging and engaging:

def setup_game(difficulty):
    if difficulty == "Easy":
        rows, cols = 3, 4
        time_limit = 60
    elif difficulty == "Medium":
        rows, cols = 4, 4
        time_limit = 90
    else:  # Hard
        rows, cols = 4, 5
        time_limit = 120

    # Create and return the game board
    return create_board(rows, cols), time_limit

6. Implementing Animations

Smooth animations enhance the gaming experience:

def flip_animation(card, screen, clock, frames=10):
    original_width = card.width

    for i in range(frames):
        # Calculate width for this frame
        current_width = original_width * (frames - i) / frames

        # Clear the card area
        pygame.draw.rect(screen, (40, 44, 52), 
                        (card.x, card.y, original_width, card.height))

        # Draw the card with the current width
        scaled_img = pygame.transform.scale(
            card.back_img if not card.is_flipped else card.logo_img,
            (int(current_width), card.height)
        )
        screen.blit(scaled_img, (card.x + (original_width - current_width)/2, card.y))

        pygame.display.update()
        clock.tick(60)

📷 Screenshots

Challenges and Solutions

Challenge 1: Card Matching Logic

One of the initial challenges was implementing the card matching logic correctly. I needed to ensure that:

Only two cards could be flipped at once
Matched cards stayed face up
Unmatched cards flipped back after a short delay

The solution was to use a list to track flipped cards and implement a state machine to manage the game flow.

Challenge 2: Responsive Layout

Making the game look good on different screen sizes was another challenge. I solved this by calculating card positions dynamically based on the screen dimensions and the number of cards.

Challenge 3: Performance Optimization

With animations and multiple images, performance could be an issue. I optimized by:

Pre-loading all images at startup
Using efficient drawing techniques
Limiting unnecessary screen updates

Lessons Learned

Building this game taught me several valuable lessons:

Planning is crucial: Having a clear structure before coding saved time and reduced bugs
User experience matters: Small details like animations and clear UI make a big difference
Testing is essential: Regular testing with different scenarios helped catch issues early

Future Improvements

There are several ways I plan to enhance this game in the future:

Add sound effects and background music
Implement a high score system
Create custom themes beyond tech logos
Add multiplayer functionality
Optimize for mobile devices

Conclusion

💡 All of this started with just one prompt to Amazon Q:
"Create a memory matching card game using the Pygame library, where players flip cards to match iconic tech logos like AWS, Azure, Docker, and GitHub. Incorporate multiple difficulty levels and a countdown timer to add challenge and urgency. Use clean UI design with subtle animations to enhance the learning experience while keeping it fun and educational."

✅ Amazingly, Amazon Q generated the entire game logic, assets integration, and structure without me having to modify a single line. This is a glimpse into how powerful and production-ready generative AI has become—even for something as interactive as a game!

Creating this Tech Logo Memory Match Game was both fun and educational. Pygame provides a great framework for building simple games, and the project helped me improve my Python skills while creating something enjoyable.

If you're interested in trying the game yourself or contributing to its development, check out the GitHub repository.

Happy coding and happy matching!

Note: Don't forget to replace the GitHub repository link with your actual repository URL once you've created it.