Forem: harsh patel

When to Choose Cassandra in System Design

harsh patel — Wed, 04 Feb 2026 20:35:10 +0000

How It Achieves Massive Write Scalability (and What You Trade Off)

Designing large-scale systems often comes down to one uncomfortable truth:

You cannot optimize reads, writes, consistency, and simplicity all at once.

When your system is write-heavy—logs, metrics, events, feeds, IoT data—traditional databases often become the bottleneck. This is where Apache Cassandra becomes a compelling choice.

This article explains:

When Cassandra is the right database
Why it outperforms traditional databases for writes
How its internal storage model enables that performance

When Should You Choose Cassandra?

Cassandra is a strong choice when writes dominate your workload and availability matters more than strict consistency.

Choose Cassandra if:

You have very high write throughput (10k–100k+ writes/sec)
Data arrives continuously (events, logs, metrics, tracking)
You need horizontal scaling by adding nodes
Downtime is unacceptable (multi-node, multi-DC availability)
Your queries are predictable and can be modeled without JOINs
Eventual or tunable consistency is acceptable

Avoid Cassandra if:

You need complex ad-hoc queries or JOINs
You rely heavily on transactions across multiple rows/tables
Reads must be extremely fast and strongly consistent
Your dataset is small and doesn’t need horizontal scaling

Cassandra is not a “general-purpose” database—it’s a specialized write-scaling machine.

Why Traditional Databases Struggle with Writes

Most relational databases (PostgreSQL, MySQL) use B-tree–based storage engines.

How writes work in traditional databases:

Data is updated in place
Indexes must be updated immediately
Writes trigger random disk I/O
Locks, WAL flushes, and index maintenance add overhead

This works well for mixed workloads, but at scale:

Random disk seeks become expensive
Index-heavy schemas slow down writes
Vertical scaling hits hardware limits
Sharding adds operational complexity

In short: traditional databases optimize for reads first.

Why Cassandra Is Faster for Writes

Cassandra flips the design priorities.

Instead of updating data in place, Cassandra uses a Log-Structured Merge Tree (LSM Tree), which turns almost every write into a sequential append.

Key design choice:

Never modify data in place. Always append.

Sequential disk writes are orders of magnitude faster than random writes, which is why Cassandra can sustain massive write throughput on modest hardware.

Cassandra’s Internal Storage Model (LSM Tree)

Cassandra’s storage engine revolves around three core components.

1. Commit Log (Durability First)

Every write is appended to the commit log
Acts as a write-ahead log
Ensures data isn’t lost if a node crashes
Sequential disk I/O → very fast

This step guarantees durability without slowing down the write path.

2. Memtable (In-Memory Writes)

Writes are stored in an in-memory, sorted structure
Sorted by primary key
Multiple updates to the same key are merged in memory
No disk I/O for each write

This absorbs write bursts efficiently.

3. SSTable (Immutable Disk Storage)

When the Memtable fills up, it is flushed to disk as an SSTable
SSTables are:
- Immutable
- Sorted by primary key
- Written sequentially

Because SSTables are never updated, Cassandra avoids random disk writes entirely.

How Cassandra Handles Updates and Deletes

Cassandra treats every change as a new write:

Updates → new version with a higher timestamp
Deletes → written as tombstones (delete markers)

The “current state” of a row is determined by timestamps, not by overwriting data.

This design enables fast writes but shifts cleanup work to the background.

Reading Data: The Trade-Off

Reads are more complex than writes:

Check the Memtable (latest data)
Use Bloom filters to identify relevant SSTables
Read SSTables from newest to oldest
Merge results to find the latest version

Bloom filters help skip unnecessary disk reads, but reads are still slower than in well-indexed relational databases.

This is the intentional trade-off.

Compaction: Paying the Cost Later

To prevent unlimited growth of SSTables and tombstones, Cassandra runs compaction:

Merges multiple SSTables into fewer ones
Resolves multiple versions of rows
Removes deleted and expired data
Improves read performance over time

Cassandra makes writes cheap now, and pays the cost later via compaction.

Why This Design Scales So Well

Putting it all together:

Traditional DB	Cassandra
In-place updates	Append-only writes
Random disk I/O	Sequential disk I/O
Write blocks reads	Writes isolated from reads
Hard to shard	Built-in partitioning
Vertical scaling	Horizontal scaling

Cassandra scales because:

Writes are fast and predictable
Nodes are independent (shared-nothing)
Adding nodes increases total throughput
Failures don’t stop the system

Final Takeaway

Cassandra is not faster because it’s “better”—it’s faster because it chooses a different set of trade-offs.

If your system is write-heavy, highly available, and horizontally scalable, Cassandra’s LSM-based storage model can outperform traditional databases by an order of magnitude.

But if reads, transactions, or flexibility matter more—choose something else.

Good system design is not about the best database.

It’s about the right database for the workload.

Finding Vulnerabilities on EC2 Instances Using AWS Inspector

harsh patel — Mon, 27 Oct 2025 02:31:48 +0000

Introduction

Amazon EC2 is one of the most widely used services on AWS for deploying applications in the cloud. However, as applications grow, so does the responsibility of keeping them secure.

That’s where Amazon Inspector comes in — an automated security assessment service that identifies vulnerabilities and deviations from best practices. It helps developers and security teams strengthen their application’s security posture before issues turn into real threats.

Architecture Overview

Here’s the high-level architecture of the setup we’ll build in this lab:

In this hands-on lab, we’ll use Amazon Inspector to detect and fix vulnerabilities in an EC2 instance. You’ll:

Launch an EC2 instance with a custom IAM role and security group
Install an outdated version of Node.js (to simulate a vulnerability)
Enable Amazon Inspector and analyze its findings
Apply remediations by closing an open port and updating Node.js
Re-run Inspector to validate the fixes

By the end, you’ll have practical experience identifying and resolving vulnerabilities on AWS — an essential skill for cloud engineers, developers, and anyone serious about cloud security.

Getting Started

Amazon Inspector is a built-in AWS security assessment service that automatically scans workloads such as EC2 instances, container images, and Lambda functions for known vulnerabilities and network exposures.

It continuously monitors your environment and provides actionable findings categorized by severity — allowing you to prioritize and mitigate risks efficiently.

Create a Role and a Security Group

Before launching an EC2 instance, we’ll create:

An IAM role for SSM access, allowing the instance to communicate with AWS Systems Manager.
A security group with an intentionally open port (port 21) to simulate a network vulnerability.

Create the IAM Role

In the AWS Console, search for IAM and open the service.
From the sidebar, select Roles → Create role.
Under Trusted entity type, choose AWS service, and under Use case, select EC2.
Attach the policy AmazonSSMManagedInstanceCore.
Name the role ec2-ssm-role, scroll down, and click Create role.

Create the Security Group

In the AWS Console, search for Security groups under the EC2 service.
Click Create security group.
Name it ec2-sg and add a short description.
Leave the default VPC selected.
Under Inbound rules, add:
- Type: SSH — Port: 22 — Source: Anywhere (IPv4)
- Type: Custom TCP — Port: 21 — Source: Anywhere (IPv4)
Click Create security group.

⚠️ Note: Port 21 is used for FTP, which transmits credentials and data unencrypted. Leaving it open to “Anywhere” exposes your instance to significant security risks. Amazon Inspector will flag this later.

Create an EC2 Instance

We’ll now create an EC2 instance, attach the role and security group, and install an outdated version of Node.js.

Steps to Launch the Instance

In the AWS Console, open EC2 → Instances → Launch instances.
Name and tags: Set the instance name as ec2_inspect.
AMI: Choose Amazon Linux 2023 (kernel 6.1) (64-bit x86).
Instance type: Select t2.micro (Free Tier).
Key pair: Choose Proceed without a key pair (for this lab).
Network settings:
- Select Existing security group
- Choose ec2-sg
Storage: Keep the default 8 GiB gp3 volume.
Advanced details:
- Under IAM instance profile, choose ec2-ssm-role
Click Launch instance, then View all instances and wait until the instance state is Running.

Install Node.js 14.x (Vulnerable Version)

Once the instance is running:

Select your instance → click Connect → Connect again.
Run the following commands:

curl -O https://rpm.nodesource.com/pub_14.x/el/7/x86_64/nodejs-14.21.3-1nodesource.x86_64.rpm
sudo yum install -y nodejs-14.21.3-1nodesource.x86_64.rpm

This installs Node.js v14.x, which is outdated and contains known vulnerabilities — perfect for testing Amazon Inspector.

You now have a running EC2 instance with:

An open port (21)
An outdated Node.js installation Both of which Amazon Inspector should detect as vulnerabilities.

Enable Amazon Inspector

Amazon Inspector continuously scans EC2 instances, ECR repositories, and Lambda functions for vulnerabilities. Let’s enable it for our environment.

Steps to Enable Inspector

Search for Amazon Inspector in the AWS Console and open it.
Click Get Started → Activate Inspector.
- This automatically creates a Service-Linked Role with permissions to scan AWS resources.
Wait 3–5 minutes for activation.
On the left sidebar, click Account management.
- You’ll see scanning statuses for EC2, ECR, and Lambda.
Since we only need EC2 scanning, deactivate the others:
- Click Actions → Amazon ECR scanning → Deactivate
- Click Actions → AWS Lambda scanning → Deactivate

Viewing the Findings

In the left menu, click Instances — Amazon Inspector should automatically detect your EC2 instance (ec2_inspect).
Click the Instance ID to open its findings.

You’ll see results categorized by severity:

Critical
High
Medium

Examples from this lab:

High: “Port 21 is reachable from an Internet Gateway - TCP”
Critical: “CVE-2023-26136 - tough-cookie” (from outdated Node.js 14.x)

Each finding includes a Remediation section — Inspector recommends removing open ports or updating vulnerable packages.

We’ll follow those recommendations next.

Viewing Amazon Inspector Findings

Detailed Finding Analysis

Apply Remediation

Now, let’s fix the vulnerabilities Amazon Inspector identified.

Delete the Open Port (21)

Go to EC2 → Security Groups and open ec2-sg.
Under the Inbound rules tab, click Edit inbound rules.
Click Delete beside the rule for Port 21.
Click Save rules.

This immediately removes external FTP access and mitigates the “Network Reachability” issue.

Update Node.js to the Latest Version

Return to the EC2 → Instances → ec2_inspect dashboard.
Click Connect → Connect again.
Run the following commands:

sudo rm -rf /etc/yum.repos.d/nodesource*
sudo yum remove -y nodejs
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
. ~/.nvm/nvm.sh
nvm install 16

Verify the version:

node -v

You should now see v16.x, which resolves the vulnerability found earlier.

Refresh Amazon Inspector

Inspector may take time to detect the changes, but you can speed this up:

Go to the Amazon Inspector dashboard.
Click General settings → Deactivate Inspector.
- Type deactivate in the confirmation box.
After 5–7 minutes, refresh and Activate it again.
Once reactivated, go to Instances → [your EC2 instance ID].

You’ll see that the vulnerabilities “Port 21 is reachable” and “CVE-2023-26136 - tough-cookie” are no longer present.

Your EC2 instance is now significantly more secure.

Validation and Results

After re-running Amazon Inspector:

The High and Critical findings have disappeared.
The security posture of your instance has improved.
Any remaining minor findings can be further remediated as needed.

This confirms that Amazon Inspector successfully detected and verified the fixes — exactly how it functions in real-world AWS environments.

Conclusion

Through this lab, we explored how Amazon Inspector strengthens cloud security by automatically detecting vulnerabilities in EC2 instances and providing actionable remediation guidance.

We learned how to:

Create IAM roles and security groups safely
Simulate vulnerabilities using outdated software and open ports
Use Amazon Inspector to detect risks
Apply fixes and validate improvements

Key takeaway: Cloud security isn’t a one-time setup — it’s a continuous process.

By integrating Amazon Inspector into your workflows, you can proactively identify and patch vulnerabilities before they escalate.

Thanks for reading!

If you found this guide useful, drop a ❤️ or share your experience with Amazon Inspector in the comments below.

Mastering AWS Key Management Service (KMS): A Practical Guide to Data Encryption

harsh patel — Sat, 25 Oct 2025 19:50:01 +0000

Getting Started with AWS Key Management Service (KMS) — Python Edition

We live in a digital world where protecting sensitive information is more critical than ever. One of the most effective ways to safeguard data is encryption. But encryption is only as strong as your key management. That’s where AWS Key Management Service (KMS) shines.

In this hands-on lab, you’ll create and use a customer-managed KMS key, generate data keys, encrypt/decrypt data in Python, and integrate KMS with DynamoDB. You’ll also see how key policy adds a security layer beyond IAM permissions.

What You’ll Learn

Create a customer-managed KMS key
Generate data keys with KMS (Python + boto3)
Encrypt & decrypt data locally with the plaintext data key
Integrate KMS with DynamoDB (SSE-KMS)
Use key policy to control access beyond IAM

Getting Started

AWS KMS is a fully managed service to create and manage cryptographic keys for encrypting data inside and outside AWS. It integrates with services like S3, EBS, RDS, and DynamoDB, helping you meet compliance while keeping data secure.

Lab Setup (Pre-built Resources)

This lab includes a pre-created IAM user that can list DynamoDB tables and read from a specific table Books. We’ll use this user later to show how KMS encryption prevents access until the key policy allows it.

KMS Key Basics

A KMS key (a.k.a. CMK) is the root key used to:

Generate & encrypt data keys
Encrypt small payloads (≤ 4 KB)
Integrate with AWS services’ encryption

Types of keys:

AWS-managed keys: Created/managed by AWS (limited control)
Customer-managed keys: Created/managed by you (full control, symmetric or asymmetric)

We’ll use a symmetric customer-managed key.

Create a Customer-Managed Key

AWS Console → KMS → Customer managed keys → Create key

Key type: Symmetric
Key usage: Encrypt and decrypt
Key material origin: KMS (recommended)
Regionality: Single-region
Alias: myKey
Grant admin and usage permissions to your main user (e.g., ed-user) Finish the wizard.

Generate a Data Key (Python)

KMS keys don’t encrypt large blobs directly; instead, they generate data keys. You use the plaintext data key locally for encryption, discard it, and store only the encrypted data key (which you can decrypt later via KMS).

Prereqs

pip install boto3 cryptography
aws configure

Fetch your key ARN

Console → KMS → Customer managed keys → myKey → copy ARN.

Generate the key

import boto3, binascii

kms = boto3.client("kms", region_name="us-east-1")
cmk_arn = "<your-kms-key-arn>"

resp = kms.generate_data_key(KeyId=cmk_arn, KeySpec="AES_256")

plaintext_key = resp["Plaintext"]
encrypted_data_key = resp["CiphertextBlob"]

print("Plaintext Data Key (hex):", binascii.hexlify(plaintext_key).decode())
print("Encrypted Data Key (hex):", binascii.hexlify(encrypted_data_key).decode())

Encrypt Data Using the Data Key (Python)

import base64, binascii
from cryptography.hazmat.primitives import padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

plaintext_key_hex = "<your-plaintext-data-key-hex>"
key = binascii.unhexlify(plaintext_key_hex)

data = "Hello, this is my sensitive data.".encode()

padder = padding.PKCS7(128).padder()
padded = padder.update(data) + padder.finalize()

cipher = Cipher(algorithms.AES(key), modes.ECB())
encryptor = cipher.encryptor()
ct = encryptor.update(padded) + encryptor.finalize()

encrypted_b64 = base64.b64encode(ct).decode()
print("Encrypted Data (base64):", encrypted_b64)

Decrypt the Data Key (Python)

import boto3, binascii

kms = boto3.client("kms", region_name="us-east-1")

encrypted_data_key_hex = "<your-encrypted-data-key-hex>"
encrypted_data_key = binascii.unhexlify(encrypted_data_key_hex)

resp = kms.decrypt(CiphertextBlob=encrypted_data_key)
recovered_key = resp["Plaintext"]

print("Recovered Plaintext Key (hex):", binascii.hexlify(recovered_key).decode())

Decrypt the Data (Python)

import base64, binascii
from cryptography.hazmat.primitives import padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

plaintext_key_hex = "<your-plaintext-data-key-hex>"
key = binascii.unhexlify(plaintext_key_hex)

encrypted_b64 = "<your-encrypted-data-base64>"
ct = base64.b64decode(encrypted_b64)

cipher = Cipher(algorithms.AES(key), modes.ECB())
decryptor = cipher.decryptor()
padded_plain = decryptor.update(ct) + decryptor.finalize()

unpadder = padding.PKCS7(128).unpadder()
plain = unpadder.update(padded_plain) + unpadder.finalize()

print("Decrypted Data:", plain.decode())

Integrate KMS with DynamoDB (SSE-KMS)

Now we’ll integrate KMS with DynamoDB to encrypt table data at rest.

Create an encrypted table

Console → DynamoDB → Create table

Table name: Books
Partition key: Author (String)
Sort key: Book_Title (String)
Table settings: Customize → Encryption at rest: Customer managed key → pick myKey Create and wait until status is Active.

Add an item

Explore table items → Create item

Author: Seth Godin
Book_Title: The Dip

Test Access with a Different IAM User

Even with read permissions, the user can’t read the table yet because the table is encrypted with your customer-managed key, and the key policy doesn’t permit this user to use the key.

Modify the Key Policy to Grant Access

Get the user’s ARN

Console → IAM → Users → IAMLabUser → copy ARN.

Edit the KMS key policy

Console → KMS → Customer managed keys → myKey → Switch to policy view → Edit

Add this statement (replace <IAM_ARN>):

{
  "Sid": "Allow use of the key",
  "Effect": "Allow",
  "Principal": { "AWS": "<IAM_ARN>" },
  "Action": ["kms:Encrypt", "kms:Decrypt"],
  "Resource": "*"
}

Save changes.

Verify as IAM user

Sign back in as IAMLabUser and Scan the Books table.

Now it works — because the user is allowed to use the KMS key.

Wrap-Up

You just built a practical, secure workflow with AWS KMS — end to end and fully in Python:

Created a customer-managed KMS key
Generated, stored, and recovered data keys
Encrypted/decrypted data locally with AES-256
Secured DynamoDB with SSE-KMS
Used key policy to gate access beyond IAM

Getting Started with AWS Config: Monitor, Detect, and Remediate Non-Compliant Resources

harsh patel — Thu, 23 Oct 2025 21:48:32 +0000

Getting Started with AWS Config

AWS Config is a powerful management service that continuously monitors and records the configuration of your AWS resources. It helps ensure compliance by evaluating these resources and their relationships within a specific AWS Region. Whenever a resource deviates from the defined configuration rules, AWS Config flags it as non-compliant, allowing you to identify and remediate issues proactively.

In this Cloud Lab, you’ll learn how to use AWS Config to enforce compliance on EC2 security groups. You’ll begin by creating an IAM role that grants AWS Config the necessary permissions to perform its operations. Next, you’ll configure AWS Config to monitor security groups in the us-east-1 (N. Virginia) Region. After that, you’ll define and apply Config Rules to evaluate compliance across your resources.

Once AWS Config is set up, you’ll intentionally create non-compliant resources to observe how it detects and reports them. You’ll then configure remediation actions that automatically bring these resources back into compliance.

By the end of this lab, you’ll be able to confidently use AWS Config to:

Monitor AWS resources in real time
Detect configuration drift or non-compliance
Automatically enforce your organization’s security and governance policies

Architecture diagram

1. Create an IAM Role

An IAM role is an AWS identity with a set of defined permissions. Unlike a user, a role can be assumed by trusted entities, including AWS services, to gain temporary access for specific actions.

Every role consists of:

A trust policy — defines who can assume the role.
A permissions policy — defines what those entities can do.

In this step, you’ll create an IAM role for AWS Config, granting it permissions to monitor and manage AWS resources.

Steps to Create the IAM Role

Open the IAM Console

Search for “IAM” in the AWS Console and select it to open the dashboard.
Create a New Role

Go to Roles → Create role.
Select the Trusted Entity

Under Use case, search and select Config.

Click Next.
Attach Permissions

The policy AWSConfigServiceRolePolicy will be automatically selected.

Click Next.
Review and Create

The default name AWSServiceRoleForConfig will appear automatically.

Click Create role.

⚠️ Note: If the error “A role with this name already exists” appears, use the existing service role instead of creating a new one.

2. Start the Configuration Recorder

The Configuration Recorder is the core of AWS Config. It continuously tracks configuration changes for selected AWS resources.

In this step, you’ll enable it to monitor EC2 security groups within your Region.

Steps to Start the Configuration Recorder

Open AWS Config

Search for “Config” and select the service.

You’ll land on the AWS Config Home page.
Set Up AWS Config

Click Set up AWS Config in the sidebar.
Select Specific Resources

Under Recording method, choose Specific resource types.

Add:
- AWS EC2 SecurityGroup → Frequency: Continuous
Assign the IAM Role

Under Data governance, choose Select a role from your account and pick the previously created role.
Configure Delivery Method

Under Delivery method, select Create a bucket.

This S3 bucket will store all configuration logs.

Click Next.
Skip Rule Selection

On the Rules page, click Next without selecting any rules (we’ll add them later).
Review and Confirm

Review your setup and click Confirm.

AWS Config will now record configuration changes for all security groups in your region.

💡 If Config Is Already Enabled:

Go to Settings → Edit, select Specific resource types, and configure only AWS EC2 SecurityGroup for Continuous recording.

3. Add AWS Config Rules

AWS Config Rules allow AWS Config to evaluate resources against compliance standards. AWS provides managed rules, and you can also create custom rules.

In this lab, you’ll define managed rules to enforce your organization’s policy:

restricted-ssh → Ensures no public SSH access (port 22)

Steps to Add AWS Config Rules

Add the “restricted-ssh” Rule

From the AWS Config dashboard, select Rules → Add rule.
Choose Add AWS managed rule.
Search for and select restricted-ssh.
Click Next, leave defaults, and click Next again.
Review and click Save.

This rule marks any EC2 security group that allows public SSH access as non-compliant.

AWS Config will now begin evaluating resources against these rules.

4. Provision EC2 Resources

Now, you’ll create EC2 resources that AWS Config can evaluate.

We’ll intentionally make one of them non-compliant to observe AWS Config’s detection capabilities.

Create a Security Group

Open the EC2 Dashboard → Search “EC2” in the console.
Go to Network & Security → Security Groups.
Click Create security group.
Set:
- Name: MySecurityGroup
- Description: Security group allowing SSH access
Under Inbound rules, click Add rule:
- Type: SSH
- Source: Anywhere-IPv4 (0.0.0.0/0)
Click Create security group.

This configuration allows public SSH access — intentionally non-compliant per your rule.

Create an EC2 Instance

Go to Instances → Launch instances.
Set the following:
- Name: client
- AMI: Amazon Linux 2023 AMI
- Instance type: t2.micro
- Key pair: Proceed without key pair
- Network settings: Select existing security group → MySecurityGroup
- Storage: Default gp3
Click Launch instance.
Wait for the instance status to show Running.

AWS Config will now start evaluating these resources.

5. Check the Resources’ Compliance

Now let’s verify if your security group comply with the defined rules.

Steps to Verify Compliance

Open AWS Config → Rules.
Under restricted-ssh, look for 1 Noncompliant resource(s). If none appear, wait about a minute and refresh.

Click the restricted-ssh rule → view Resources in scope.
Click the Resource ID to confirm it’s your MySecurityGroup.
To view its details, click View Configuration Item (JSON). You’ll find:

json
"ipPermissions": [
    {
        "fromPort": 22,
        "ipProtocol": "tcp",
        "toPort": 22,
        "ipv4Ranges": [
            {
                "cidrIp": "0.0.0.0/0"
            }
        ]
    }
]

This shows that port 22 (SSH) is open to the public (0.0.0.0/0) — making the security group non-compliant.

6. Add and Use a Remediation Action

Once AWS Config identifies non-compliance, you can fix it using remediation actions.

AWS Config integrates with AWS Systems Manager Automations to perform predefined remediation workflows.

Here, you’ll create and execute a remediation action to automatically remove public SSH access from your EC2 security group.

Add the Remediation Action

Open AWS Config → Rules → restricted-ssh.
Click Actions → Manage remediation.
Under Select remediation method, choose Manual remediation.
In Remediation action details, select AWS-DisablePublicAccessForSecurityGroup.
For Resource ID parameter, choose GroupId.
In Parameters, set:
- IpAddressToBlock: 0.0.0.0/0
Click Save changes.

Use the Remediation Action

Open the restricted-ssh rule again.
In Resources in scope, select the non-compliant security group.
Click Remediate.
Once the process completes, you’ll see “Action executed successfully.”
Refresh the page — the resource should disappear from the non-compliant list.
Finally, verify from the EC2 Dashboard → Security Groups that the inbound SSH rule allowing public access has been removed.

You’ve successfully remediated your non-compliant resource.

Conclusion

In this Cloud Lab, you learned how to:

Configure AWS Config to monitor and record resource changes
Define compliance rules for security groups
Detect and review non-compliant resources
Apply remediation actions to automatically fix violations

With AWS Config, you can continuously track, audit, and enforce configuration compliance — ensuring your AWS environment remains secure and aligned with organizational policies.

AWS CloudFront Security Implementation Guide

harsh patel — Wed, 08 Oct 2025 18:42:39 +0000

Combining Origin Access Control (OAC) and Signed URLs with Trusted Key Groups

Executive Summary

This guide provides a comprehensive implementation framework for securing Amazon S3 static assets through AWS CloudFront using Origin Access Control (OAC) and Signed URLs with Trusted Key Groups. This architecture establishes multiple security layers to protect sensitive content while maintaining optimal delivery performance.

Architecture Overview

This architecture establishes a two-layer security model where both end-user access and origin access are independently verified and protected.

User → CloudFront (Signed URLs)

Users must present a cryptographically signed URL containing an expiration timestamp.
CloudFront validates the signature using public keys from trusted Key Groups.
Any invalid or expired requests are rejected at the edge, preventing unauthorized access.

CloudFront → S3 (Origin Access Control)

CloudFront authenticates to S3 using AWS Signature Version 4.
S3 bucket policies ensure that requests only come from authorized CloudFront distributions.
Direct access to S3 is blocked, enforcing all data delivery through CloudFront.

Security Layers

Viewer Authentication: Signed URL validation using RSA key pairs
Edge Security: CloudFront distribution with restricted access policies
Origin Authentication: OAC with SigV4 signing for S3 access
Bucket Security: S3 bucket policies restricting access to specific CloudFront distributions

Implementation Specifications

Phase 1: S3 Bucket Configuration

1.1 Create Private S3 Bucket (Console)

Navigate to Amazon S3 Console
Click Create bucket
Enter Bucket name: demo-oac-secure-assets
Select AWS Region
Under Block Public Access, select Block all public access
(Optional) Enable Versioning for audit trail
Click Create bucket

1.2 Upload Sample Content (Console)

Select your newly created bucket
Click Upload → Add files
Select your static assets (HTML, CSS, JS, images)
Click Upload
Verify objects show No public access in the permissions column

Phase 2: Origin Access Control Setup

2.1 Create OAC Configuration (Console)

Navigate to CloudFront Console
In left navigation, select Origin access
Click Create control setting
Configure:
- Name: secure-oac-configuration
- Description: Origin Access Control for secure S3 access
- Signing protocol: SigV4
- Signing behavior: Always
Click Create

Phase 3: Cryptographic Key Management

3.1 Generate Key Pair Locally

# Generate 2048-bit RSA private key
openssl genrsa -out private_key.pem 2048

# Extract public key
openssl rsa -in private_key.pem -pubout -out public_key.pem

# Set secure permissions
chmod 600 private_key.pem

3.2 Create CloudFront Public Key (Console)

In CloudFront Console, open Key management → Public keys
Click Create public key
Configure:
- Name: secure-content-key
- Public key: paste contents of public_key.pem
Click Create public key
Note the generated Key pair ID (format: K123456789ABCDEF)

3.3 Establish Trusted Key Group (Console)

In CloudFront Console, open Key management → Key groups
Click Create key group
Configure:
- Name: trusted-key-group
- Description: Key group for signed URL validation
- Public keys: select your created public key
Click Create key group

Phase 4: CloudFront Distribution Configuration

4.1 Create Distribution (Console)

Navigate to CloudFront Console → Create distribution
Origin settings:
- Origin domain: select your S3 bucket (demo-oac-secure-assets.s3.amazonaws.com)
- Origin path: (Leave blank)
- Name: SecureS3Origin
- Origin access: Origin access control settings (recommended)
- Select control setting: your created OAC
- Bucket policy: Yes, update the bucket policy

4.2 Configure Default Cache Behavior

Cache policy: CachingOptimized
Viewer protocol policy: Redirect HTTP to HTTPS
Restrict viewer access: Yes
Trusted key groups: select your created key group
Additional:
- AWS WAF: Enable if required
- HTTP/2: Enabled
- IPv6: Enabled

4.3 Distribution Settings

Price class: choose based on geographic requirements
Alternate domain name (CNAME): configure if using custom domain
SSL certificate: default or custom ACM certificate
Click Create distribution
Wait for status to change from In Progress → Deployed

Phase 5: S3 Bucket Policy Implementation

5.1 Verify Automatic Policy Creation (Console)

Go to S3 Console → select bucket demo-oac-secure-assets → Permissions tab
Verify an auto-generated bucket policy that allows CloudFront access via OAC
If not present, apply the manual policy below

5.2 Manual Bucket Policy (if required)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowCloudFrontOACAccess",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::demo-oac-secure-assets/*",
      "Condition": {
        "StringEquals": {
          "AWS:SourceArn": "arn:aws:cloudfront::123456789012:distribution/E2ABCD1234EXAMPLE"
        }
      }
    }
  ]
}

Replace placeholders:

123456789012 → your AWS Account ID
E2ABCD1234EXAMPLE → your CloudFront Distribution ID

Signed URL Generation Implementation

Python Implementation

import base64
import json
import time
from urllib.parse import urlencode
from cryptography.hazmat.primitives import serialization, hashes
from cryptography.hazmat.primitives.asymmetric import padding

class CloudFrontSigner:
    def __init__(self, key_pair_id: str, private_key_path: str):
        self.key_pair_id = key_pair_id
        self.private_key_path = private_key_path

    def _b64url_encode(self, data: bytes) -> str:
        """Base64 URL-safe encoding without padding"""
        return base64.b64encode(data).decode().replace('+', '-').replace('=', '_').replace('/', '~')

    def _sign_policy(self, policy: bytes) -> str:
        """Sign policy document using RSA private key"""
        with open(self.private_key_path, 'rb') as key_file:
            private_key = serialization.load_pem_private_key(
                key_file.read(),
                password=None
            )
        signature = private_key.sign(policy, padding.PKCS1v15(), hashes.SHA1())
        return self._b64url_encode(signature)

    def generate_signed_url(self, resource_url: str, expires_in: int = 3600) -> str:
        """Generate CloudFront signed URL with custom policy"""
        expiration = int(time.time()) + expires_in
        policy = {
            "Statement": [
                {
                    "Resource": resource_url,
                    "Condition": {
                        "DateLessThan": { "AWS:EpochTime": expiration }
                    }
                }
            ]
        }
        policy_json = json.dumps(policy, separators=(',', ':')).encode()
        policy_encoded = self._b64url_encode(policy_json)
        signature = self._sign_policy(policy_json)
        return f"{resource_url}?Policy={policy_encoded}&Signature={signature}&Key-Pair-Id={self.key_pair_id}"

# Implementation Example
signer = CloudFrontSigner(
    key_pair_id="K123456789ABCDEF",
    private_key_path="/secure/keys/private_key.pem"
)

signed_url = signer.generate_signed_url(
    resource_url="https://d123456789.cloudfront.net/secure-document.pdf",
    expires_in=7200  # 2 hours validity
)
print("Signed URL:", signed_url)

Phase 6: Testing

Test Scenario	Expected Result	Validation Steps
Direct S3 Access	HTTP 403 Access Denied	1) Copy S3 object URL → 2) Open in browser → 3) Verify Access Denied
Unsigned CloudFront Access	HTTP 403 Access Denied	1) Copy CloudFront URL → 2) Access w/o params → 3) Verify Access Denied
Valid Signed URL	HTTP 200 OK	1) Generate signed URL → 2) Open in browser → 3) Verify content loads
Expired Signed URL	HTTP 403 Access Denied	1) Generate URL with past expiration → 2) Access → 3) Verify rejection
Modified Signature	HTTP 403 Access Denied	1) Alter signature parameter → 2) Access → 3) Verify rejection

Results Comparison

🔒 Access Without Signed URL (Blocked)

)

✅ Access With Signed URL (Successful)

Beyond the Proxy: Building a Secure, Observable Serverless API with Amazon API Gateway

harsh patel — Sun, 05 Oct 2025 18:21:01 +0000

Introduction: Rethinking API Gateway

If you ask many developers what Amazon API Gateway is, they’ll often say,

“It’s a way to trigger Lambda functions with HTTP requests.”

While that’s a common use case, this view undersells the service dramatically.

In reality, API Gateway is a complete API management platform. It’s your first and most critical line of defense, capable of handling validation, transformation, authentication, throttling, and observability before your Lambda function even runs.

It shifts the burden of common API concerns from your code to a managed, scalable layer.

To prove this, we’ll build TrailLog — a serverless API for logging hiking trips.

We won’t just connect endpoints; we’ll use API Gateway’s full potential to create a robust, secure, and observable API.

By the end of this guide, you will learn how to:

Validate & Transform: Enforce data contracts at the gateway, keeping Lambda code clean.
Secure Endpoints: Implement a custom Lambda authorizer for token-based access control.
Achieve Observability: Gain deep insights into API performance with structured CloudWatch logs.
Orchestrate a Governed API: Deploy a production-ready API with full visibility and control.

The Blueprint: TrailLog System Architecture

Our fully serverless architecture is designed for scalability and simplicity.

Each AWS service plays a clear role in this ecosystem.

Component	Responsibility
Amazon API Gateway (REST API)	The unified entry point — handles routing, validation, auth, and logging
AWS Lambda	Executes backend logic for creating and retrieving trips
Amazon DynamoDB	Stores all hiking data in a serverless NoSQL table
Amazon CloudWatch	Centralizes logs and metrics for monitoring and debugging

This stack scales automatically with traffic, without managing or patching servers.

Architecture Diagram

Client application sends incoming HTTPS REST requests to Amazon API Gateway, which triggers AWS Lambda functions for GET, POST, and DELETE operations. Lambda interacts with Amazon DynamoDB for data storage and sends logs to Amazon CloudWatch.

Foundation: Setting Up Permissions with IAM

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role,

configured via an inline policy that grants access to DynamoDB and CloudWatch logs.

IAM Role: `lambda-traillog-role`

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role.

This role is granted access to the DynamoDB table TrailLog through the following inline policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DDBWriteTrailLog",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    },
    {
      "Sid": "DDBReadTrailLog",
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Scan",
        "dynamodb:Query"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    },
    {
      "Sid": "DDBDescribeForChecks",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeTable"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    }
  ]
}

This role is attached to all Lambda functions in the project.

Deep Dive 1: The `POST /trips` Endpoint — Validation & Transformation

The first endpoint demonstrates API Gateway’s ability to offload validation and data transformation

before invoking the backend logic.

DynamoDB Table: TrailLog

Property	Value
Partition Key	`tripId` (String)
Billing Mode	Pay-per-request

Lambda Function: `create-trip` (Python 3.12)

This Lambda function remains simple because API Gateway enforces all validation upfront.

import json, os, uuid, boto3
from datetime import datetime
from decimal import Decimal, InvalidOperation

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def lambda_handler(event, context):
    # API Gateway has already validated input; focus on core logic.
    trip_id = str(uuid.uuid4())
    item = {
        "tripId": trip_id,
        "userId": event["userId"],
        "trailName": event["trailName"],
        "date": event["date"],
        "distanceKm": event["distanceKm"],
        "durationMin": event["durationMin"],
        "elevationGainM": event["elevationGainM"],
        "notes": event.get("notes", ""),
        "createdAt": datetime.utcnow().isoformat() + "Z"
    }
    table.put_item(Item=item)
    return {
        "statusCode": 201,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({"ok": True, "tripId": trip_id})
    }

The API Gateway Configuration: Non-Proxy Integration

Disabling Lambda Proxy Integration gives us control over request handling and validation.

Setting	Value
Integration Type	Lambda Function
Proxy Integration	Disabled
Validator	Body & Params
Model	`TripCreate`

Request Model: `TripCreate`

{
  "type": "object",
  "required": ["userId", "trailName", "date", "distanceKm", "durationMin", "elevationGainM"],
  "properties": {
    "userId": {"type": "string"},
    "trailName": {"type": "string"},
    "date": {"type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"},
    "distanceKm": {"type": "number"},
    "durationMin": {"type": "integer"},
    "elevationGainM": {"type": "integer"},
    "notes": {"type": "string"}
  }
}

Now, any malformed input (for example, an invalid date format or incorrect type)

is rejected at the gateway, saving Lambda invocations and improving reliability.

Successfully created a new hiking trip via the POST /trips endpoint, returning a 201 status and generated tripId.

Deep Dive 2: The `GET /trips/{tripId}` Endpoint — Simple Proxy Integration

For simple reads, we use Lambda Proxy Integration to pass the entire request context directly to Lambda.

Lambda Function: `get-trip`

This function extracts tripId from the path and fetches the corresponding item from DynamoDB.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _to_native(x):
    if isinstance(x, decimal.Decimal):
        return int(x) if x % 1 == 0 else float(x)
    return x

def lambda_handler(event, context):
    trip_id = (event.get("pathParameters") or {}).get("tripId")
    if not trip_id:
        return {"statusCode": 400, "body": "Missing tripId"}

    res = table.get_item(Key={"tripId": trip_id})
    item = res.get("Item")
    if not item:
        return {"statusCode": 404, "body": "Not found"}

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(_to_native(item))
    }

Test Request

curl https://<api-id>.execute-api.us-east-1.amazonaws.com/dev/trips/<tripId>

Fetched detailed hiking trip data successfully using the GET /trips/{tripId} endpoint.

Deep Dive 3: Security and Observability — `/secure/admin/stats`

This administrative endpoint combines two advanced API Gateway features:

custom token authorization and centralized CloudWatch observability.

1. Securing the Route with a Lambda Authorizer

We use a custom Lambda authorizer to protect the /secure/admin/stats route.

Authorizer Function: `lambda-authorizer`

import json

def lambda_handler(event, context):
    DEMO_TOKEN = "Bearer trail123"
    token = event.get("authorizationToken", "").strip()

    if token == DEMO_TOKEN:
        effect, role = "Allow", "admin"
    else:
        effect, role = "Deny", "guest"

    return {
        "principalId": "trail-admin",
        "policyDocument": {
            "Version": "2012-10-17",
            "Statement": [{
                "Action": "execute-api:Invoke",
                "Effect": effect,
                "Resource": event["methodArn"]
            }]
        },
        "context": {"role": role}
    }

Any request without X-Auth-Token: Bearer trail123 will receive a 403 Forbidden response.

2. Stats Aggregation Function: `admin-stats`

This function aggregates trip statistics across all records.

It’s invoked only if the authorizer grants access.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _n(x):
    if isinstance(x, decimal.Decimal):
        return int(x) if x % 1 == 0 else float(x)
    return x

def lambda_handler(event, context):
    scan = table.scan(ProjectionExpression="tripId, distanceKm, durationMin, elevationGainM")
    items = scan.get("Items", [])
    total = len(items)
    km = sum(_n(i.get("distanceKm", 0)) for i in items)
    mins = sum(_n(i.get("durationMin", 0)) for i in items)
    gain = sum(_n(i.get("elevationGainM", 0)) for i in items)

    stats = {
        "totalTrips": total,
        "totalKm": round(float(km), 2),
        "totalHours": round(float(mins) / 60.0, 1),
        "totalElevationM": int(gain)
    }

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(stats)
    }

Test Request

curl -H "Authorization: Bearer trail123" "$BASE/secure/admin/stats"

Access denied without a valid token — API Gateway blocking unauthorized requests using the custom Lambda authorizer.

Authorized admin access to /secure/admin/stats returning aggregated trail statistics.

The Crown Jewel: Deep Observability with CloudWatch

To gain complete visibility into API behavior, enable both Execution Logs and Access Logs in Amazon API Gateway.

These logs provide the foundation for debugging, performance monitoring, and request analytics.

Create a Log Group for Access Logs

Before enabling access logging, create a dedicated CloudWatch Log Group where API Gateway will push structured log entries.

This ensures all API requests are tracked consistently and stored centrally.

Steps:

Open CloudWatch Console → Log groups → Create log group.
Enter the name:

   /apigw/traillog/dev

(Optional) Set the retention period to 7, 30, or 90 days depending on your monitoring needs.
Note down the log group ARN — you’ll use it in your API Gateway stage configuration:

   arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev

Ensure API Gateway has permission to write logs by attaching this managed policy to its execution role:

   arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs

CloudWatch log group /apigw/traillog/dev created to store structured access logs from API Gateway.

1. Execution Logs

Execution logs trace the internal request flow through API Gateway —

from input validation and mapping templates to Lambda invocation and response transformation.

They’re your primary tool for debugging and tracing how API Gateway processes a request.

Property	Description
Log Group	`API-Gateway-Execution-Logs_{restApiId}/dev`
Purpose	Traces internal request flow and mapping templates
Best For	Debugging authorization, validation, or mapping issues

Ensure your API stage has CloudWatch execution logging enabled and that API Gateway can push logs.

Attach the following managed policy to grant permissions:

arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs

2. Access Logs

Access logs summarize each API request and response in a structured JSON format.

These are invaluable for analytics, performance tracking, and auditing user behavior.

Property	Description
Log Group ARN	`arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev`
Purpose	Capture per-request analytics and response metadata
Best For	Monitoring latency, traffic, and client behavior

Access Log Format Template

To enable structured access logs, open your API Gateway stage and set the log format using the following JSON template.

{
  "requestId": "$context.requestId",
  "ip": "$context.identity.sourceIp",
  "time": "$context.requestTime",
  "method": "$context.httpMethod",
  "path": "$context.resourcePath",
  "status": $context.status,
  "integrationStatus": $context.integration.status,
  "userAgent": "$context.identity.userAgent",
  "authorizer": "$context.authorizer.role",
  "error": "$context.error.message"
}

Once redeployed, every request will produce:

Execution Logs: Detailed traces for debugging.
Access Logs: High-level JSON entries for analytics.

API Gateway Execution Logs showing Lambda invocation flow, request mapping, and response transformation details.

Structured JSON access logs capturing requestId, method, path, status, and authorizer context for analytics.

Conclusion: You’ve Built More Than an API

TrailLog isn’t just a CRUD backend. It’s a governed, secure, and observable API platform built with managed AWS services.

Key Takeaways

Pre-Lambda Validation: Offload schema checks to API Gateway for cleaner, cheaper code.
Custom Authorization: Implement fine-grained security without burdening business logic.
Structured Logging: Use CloudWatch for comprehensive visibility and debugging.

By embracing these patterns, you can use Amazon API Gateway not just as a trigger mechanism, but as a complete API management engine — the foundation of scalable, production-ready serverless systems.

Managing Data Access with Amazon S3 Access Points

harsh patel — Thu, 18 Sep 2025 23:48:11 +0000

Amazon S3 Access Points are a powerful feature that simplify how organizations control and manage access to their data. Instead of relying solely on complex bucket policies, Access Points allow you to create dedicated entry points with tailored permissions. This provides more flexibility, especially when working with teams, applications, or VPCs that require different levels of access.

In today's world where secure and efficient data handling is critical, mastering S3 Access Points sets you apart as someone who understands not just storage, but also governance and access control in the cloud.

In this Cloud Lab, we'll explore:

Creating an S3 bucket for sensitive data
Configuring a VPC-bound S3 Access Point
Testing access from both an allowed VPC and a default VPC using AWS Lambda
Understanding how these controls enforce network-level security

By the end, you'll have hands-on experience with S3 Access Points and the skills to design secure data access patterns for real-world cloud applications.

Step 1: Create an S3 Bucket

The first step is to provision an Amazon S3 bucket that will act as the primary storage location for sensitive documents. We'll later use S3 Access Points to control how this data is accessed and processed.

Creating the S3 Bucket

Navigate to the AWS Management Console and search for S3.
Select the S3 service from the results.
Click Create bucket.
Enter a globally unique name in the format: s3-bucket-<random_text>
Replace <random_text> with any string that ensures uniqueness across AWS accounts.
Under Block Public Access settings, make sure Block all public access is selected.
Scroll down and click Create bucket to finalize.

At this point, your S3 bucket is ready to securely store documents.

Uploading the Document

Create a simple text file called sampledata.txt on your local machine (e.g., with content like "This is sensitive organizational data.").
In the S3 Dashboard, select the newly created bucket (s3-bucket-<random_text>).
Click Upload → Add files, and choose the sampledata.txt file.
Click Upload to complete the process.

Outcome: You now have a secure S3 bucket with a sensitive file in place, ready to be accessed and controlled using Amazon S3 Access Points.

Step 2: Create a VPC-Bound Access Point

Amazon S3 Access Points make it easier to define fine-grained permissions, ensuring that specific applications, users, or roles get only the access they need. This is particularly useful in environments where multiple teams or services share the same dataset.

In this step, we'll create an S3 Access Point bound to a specific VPC. This ensures that sensitive data stored in our bucket is accessible only through that VPC.

Step 2.1: Create the VPC

In the AWS Management Console, search for VPC and open the service.
From the left menu, select Your VPCs → Create VPC.
Under Resources to create, choose VPC and more.
In the Name tag auto-generation field, replace the default value with org.
Configure the following:
- Number of public subnets → 0
- Number of private subnets → 2
- VPC endpoints → Select S3 Gateway
Click Create VPC.
Once provisioned, click View VPC and copy the VPC ID for later use.

Step 2.2: Create the Access Point

Open the S3 service in the AWS Console.
From the left menu, select Access Points for general purpose buckets.
Click Create access point.
Name it: vpc-bound-access-point
Under Bucket name, select the S3 bucket created earlier.
Under Network origin, choose Virtual private cloud (VPC).
Paste the VPC ID of the org-vpc.
Ensure Block all public access is enabled.
Add the following policy (replace <Account ID> with your AWS account ID and <VPC ID> with the ID you copied in the previous step):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:us-east-1:<Account ID>:accesspoint/vpc-bound-access-point/object/*",
      "Condition": {
        "StringEquals": {
          "aws:SourceVpc": "<VPC ID>"
        }
      }
    }
  ]
}

Click Create access point.

Outcome: The Access Point now enforces that objects in the bucket can only be retrieved from the org-vpc.

Step 3: Test the VPC-Bound Access Point

To validate the policy behavior, we'll create an AWS Lambda function. First, we'll deploy it inside the org-vpc and confirm access. Then, we'll move it to the default VPC to ensure access is denied.

Note: This step assumes you have a basic IAM role (e.g., LambdaExecutionRole) with permissions for Lambda and S3 access. You may need to create this role first if it doesn't exist.

Step 3.1: Create a Lambda Function in org-vpc

In the AWS Console, search for Lambda.
Go to Functions → Create function → Author from scratch.
Configure:
- Function name: accesspoint_function
- Runtime: Python 3.11
- Execution role: Use an existing role → LambdaExecutionRole
Expand Advanced settings → Enable VPC → Select:
- VPC: org-vpc
- Subnets: Select the two private subnets (e.g., org-subnet-private1-us-east-1a, org-subnet-private2-us-east-1b)
- Security group: Default
Click Create function.

Step 3.2: Add the Lambda Code

Replace the default code in the Lambda function editor with the following Python code.

(Remember to replace <Account ID> with your actual AWS account ID.)

import boto3
from botocore.client import Config

# Initialize S3 client
s3 = boto3.client('s3', config=Config(signature_version='s3v4'))

def getObject(bucket, key):
    response = s3.get_object(Bucket=bucket, Key=key)
    data = response['Body'].read().decode('utf-8')
    print(data)
    return data

def lambda_handler(event, context):
    return getObject(
        "arn:aws:s3:us-east-1:<Account ID>:accesspoint/vpc-bound-access-point",
        "sampledata.txt"
    )

Step 3.3: Test from org-vpc

Click Deploy to save your code changes.
Click Test to configure a test event. Use the default "Hello World" template and give it a name.
Click Test to run the function.

Result: The execution should succeed. The file contents will appear in the function's logs because the Lambda function runs inside the allowed VPC.

Step 3.4: Test from Default VPC

In the Lambda function, go to the Configuration tab and select the VPC section.
Click Edit.
Switch the VPC to the default VPC and select two of its subnets.
Select the default security group.
Click Save.
Wait for the network changes to apply, then click Test again to run the function.

Result: The function execution will now fail with an error like AccessDenied or 403 Forbidden. No data is retrieved because the Access Point policy correctly restricts access to only the org-vpc.

Conclusion

Amazon S3 Access Points provide a powerful mechanism for implementing fine-grained access control to your S3 data. By binding access points to specific VPCs, you can enforce network-level security that complements traditional IAM policies. This approach is particularly valuable in multi-tenant environments, compliance scenarios, or when you need to isolate data access to specific network segments.

This hands-on exercise demonstrated how to:

Create a secure S3 bucket with sensitive data.
Configure a VPC-bound access point with appropriate policies.
Test access patterns from both allowed and restricted networks.

This knowledge empowers you to design more secure and manageable data access patterns in your AWS environments, ensuring that your sensitive data remains protected while still being accessible to authorized services and applications.

Connecting Multi-VPC Applications Using AWS Transit Gateway

harsh patel — Sun, 16 Feb 2025 19:38:00 +0000

Deploying a React App Across Multiple VPCs Using AWS Transit Gateway

Introduction

If you're working with AWS and need to connect multiple VPCs efficiently, AWS Transit Gateway is your best bet! It acts as a centralized hub, making it easier to route traffic between multiple VPCs without the complexity of VPC peering.

In this guide, we'll deploy a React application where the frontend and backend live in separate VPCs. These VPCs will be connected using AWS Transit Gateway, allowing smooth communication between services. Let’s dive in!

Prerequisites

An AWS account with admin access
Basic knowledge of AWS Networking (VPCs, Subnets, Route Tables)
Familiarity with React and MySQL

Architecture Overview

Here’s what we’re building:

Frontend VPC → Hosts the React application
Backend VPC → Runs a MySQL database
AWS Transit Gateway → Connects both VPCs
Transit Gateway Attachments → Link each VPC to the Transit Gateway
Routing Tables → Manage traffic flow between VPCs
Security Groups → Control access

Step 1: Create Two VPCs

Go to AWS VPC Console.
Click Create VPC and name it FrontendVPC (CIDR: 10.0.0.0/16).
Create BackendVPC (CIDR: 10.1.0.0/16).
Add public and private subnets in both VPCs:

FrontendVPC Public: 10.0.1.0/24 FrontendVPC Private: 10.0.2.0/24 BackendVPC Public: 10.1.1.0/24 BackendVPC Private: 10.1.2.0/24

Step 2: Set Up AWS Transit Gateway

Go to AWS Transit Gateway Console.
Click Create Transit Gateway, name it, and enable DNS support.
Once created, note the Transit Gateway ID.

Step 3: Attach VPCs to Transit Gateway

In Transit Gateway Attachments, create:
Attachment for FrontendVPC
Attachment for BackendVPC

Step 4: Configure Route Tables

1.Modify each VPC’s route table:
2.Associate these routes with the correct subnets.

Step 5: Deploy the Frontend Application

Launch an EC2 instance in FrontendVPC public subnet.
Install Node.js and React:
sudo yum install -y mysql-server sudo systemctl start mysqld sudo mysql_secure_installation mysql -u root -p -e "CREATE DATABASE app_db;"
Allow inbound HTTP traffic (port 3000) via security groups.

Step 6: Deploy the Backend and MySQL Database

Launch an EC2 instance in BackendVPC private subnet.
Install MySQL and set up a database: sudo yum install -y mysql-server sudo systemctl start mysqld sudo mysql_secure_installation mysql -u root -p -e "CREATE DATABASE app_db;"
Set up a backend API using Express.js: npm install express mysql node server.js
Allow MySQL traffic (port 3306) from FrontendVPC in security groups.

Step 7: Open and Test the Application

Retrieve the Public IP Address of the frontend EC2 instance.
Open a browser and go to: ping <Backend-EC2-Private-IP> curl http://<Backend-EC2-Private-IP>:<Backend-Port>
Test frontend-backend connectivity:
sudo yum install -y mysql-server sudo systemctl start mysqld sudo mysql_secure_installation mysql -u root -p -e "CREATE DATABASE app_db;"
If something fails, check:

Route Tables for correct routing.
Security Groups for inbound/outbound traffic.
Transit Gateway Attachments for proper linking.

This architecture allows for scalability, security, and efficient communication between VPCs.