Forem: Rajesh Gundeti

Secure Data Sharing: AWS Lambda Writing to S3 Across Accounts

Rajesh Gundeti — Wed, 03 Sep 2025 02:33:08 +0000

Introduction

In the multi account AWS environment. It is common to have services hosted in one account and the data ingested to the different account. For instance, there is a lambda function in one AWS account and read or write data to the S3 bucket in a different AWS account.

This blog explains the detailed steps to configure S3 bucket policy and IAM role for lambda function to achieve cross account S3 access with AWS Lambda.

The Scenario

Account A (1111111111): Hosts the Lambda function.
Account B (2222222222): Owns the target S3 bucket (account_b_bucket).

The goal is to allow AWS lambda function in Account A to put an object in the S3 bucket of Account B.

Architecture Overview

The setup involves three main components

1) IAM role (account_a_lambda_role) in account A and the custom policy attached to the role to access the S3 bucket in account B.

Attach this policy to the Lambda execution role (replace the bucket name):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountS3",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::account_b_bucket/*"]
    }
  ]
}

This will grant permissions on the lambda to request access to the S3 bucket in account B.

2) Trust policy of Lambda Execute Role allows lambda function to assume the role.

Attach this policy to the Lambda execution role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "lambda.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

3) Bucket Policy in Account B that allows IAM role attached to the lambda function in account A to put the object in the bucket.

Configure the bucket policy in account B to trust account A's Lambda role. Replace the account number, lambda role and bucket name.

{
  "Id": "ExamplePolicy",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountBucket",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::account_b_bucket/*"],
      "Principal": {
        "AWS": ["arn:aws:iam::1111111111:role/account_a_lambda_role"]
      }
    }
  ]
}

Execution Flow

The Lambda in Account A is invoked.

It assumes the IAM role account_a_lambda_role.

The role policy allows S3 actions on the target bucket.

The bucket policy in Account B authorizes the role ARN.

The Lambda can now securely access the cross-account bucket.

Sample Lambda code

Replace the "TARGET_BUCKET" in the below code with your target bucket name.

import boto3
import os
from datetime import datetime

# Initialize S3 client
s3_client = boto3.client("s3")

def lambda_handler(event, context):
    # Get bucket name from environment variable
    bucket_name = os.environ["TARGET_BUCKET"]

    # Generate file content
    timestamp = datetime.utcnow().strftime("%Y-%m-%d_%H-%M-%S")
    file_content = f"Hello from Lambda!\nTimestamp: {timestamp}\n"

    # File name to upload
    file_name = f"lambda_output_{timestamp}.txt"

    try:
        # Upload file content to S3
        s3_client.put_object(
            Bucket=bucket_name,
            Key=file_name,
            Body=file_content.encode("utf-8")
        )

        return {
            "statusCode": 200,
            "body": f"File {file_name} successfully uploaded to {bucket_name}"
        }
    except Exception as e:
        return {
            "statusCode": 500,
            "body": f"Error uploading file: {str(e)}"
        }

Conclusion

By configuring the correct IAM role policies and S3 bucket policies, we can ensure the secure communication between the lambda in one AWS account and S3 bucket in a different AWS account.

This pattern is not limited to the Lambda function. It can be applied to other services like EC2, ECS tasks etc.

Configuring Hybrid Authentication (Certificates + Users) in AWS Client VPN

Rajesh Gundeti — Fri, 22 Aug 2025 02:24:53 +0000

Introduction

In this post we are going to look into the point to site VPN configuration using AWS client VPN. The clientVPN enables the endusers connect to securely the AWS hosted network with mutual authentication and user authentication.

Before diving into the configuration steps. Let us discuss the typical network architecture. As illustrated in the cover image. The network architecture has one hub/transit account which is a demilitarized zone (DMZ) for accessing the spoke accounts. Typically hub/transit is uses the inspect services like firewall for any incoming traffic from internet. In our scenario, we are hosting the AWS client VPN. It uses the AWS certificate manager for mutual authentication and managed active directory for user authentication.

Step 1) Generate server and client certificates for mutual authentication

In the hub/transit account. Run the following commands to configure client VPN.

Run the below command to clone the openVPN easy-rsa repo on your laptop/local machine.

git clone https://github.com/OpenVPN/easy-rsa.git

Run the below command to switch to the easyrsa3 directory

cd easy-rsa/easyrsa3

Initialize the new PKI environment.

./easyrsa init-pki

Build a new certificate authority (CA) by running the below command.

./easyrsa build-ca nopass

To generate the server certificate and key run the below command.

./easyrsa --san=DNS:server build-server-full server nopass

Generate the client certificate and key.

./easyrsa build-client-full client1.domain.tld nopass

Copy the server certificate, key, chain and client certificate and key to a custom folder “awsclientvpn”.

mkdir ~/awsclientvpn/
cp pki/ca.crt ~/awsclientvpn/
cp pki/issued/server.crt ~/awsclientvpn/
cp pki/private/server.key ~/awsclientvpn/
cp pki/issued/client1.domain.tld.crt ~/awsclientvpn
cp pki/private/client1.domain.tld.key ~/awsclientvpn/
cd ~/awsclientvpn/

Zip the custom directory and upload the zip file to the cloud shell in the account where client VPN will be configured.In this scenario, upload the zip file to the hub/transit AWS account.

zip -r awsclientvpn.zip awsclientvpn

Upload the zipfile to the cloudshell in the account where client VPN will be configured, unzip.

Run the below command from the directory where files are downloaded and then import the server certificates to AWS certificate manager.

aws acm import-certificate --certificate fileb://server.crt --private-key fileb://server.key --certificate-chain fileb://ca.crt

Run the below command to import the client certificates to AWS certificate manager.

aws acm import-certificate --certificate fileb://client1.domain.tld.crt --private-key fileb://client1.domain.tld.key --certificate-chain fileb://ca.crt

In the AWS console verify that the certificates are imported.

Step 2) Create the directory service for user authentication

In the spoke/shared services account, create a new managed AD for integrating with client VPN for user authentication.

aws ds create-microsoft-ad \
    --name <your-domain-name> \
    --short-name <your-netbios-name> \
    --password <your-admin-password> \
    --vpc-settings VpcId=<your-spoke-vpc-id>,SubnetIds=<spoke-subnet-id-1>,<spoke-subnet-id-2> \
    --edition STANDARD \
    --description "My AWS Managed Microsoft AD Standard"

Step 3) Create client VPN endpoint

Navigate to the VPC section from AWS console and enter the Name, description and CIDR that will be assigned for the client machines by VPN client.

Select the server certificate ARN. Choose authentication both options "mutual authentication" and "user-based authentication". Select client certificate ARN. In the user based authentication select Active directory authentication. Then select the directory ID from the dropdown menu.

Enter the DNS Server details. If you are using route 53 as a DNS. Use VPC+2 IP as the DNS address. If VPC CIDR is 10.10.10.0/24, then the DNS IP address is 10.10.10.2.

Select the transport protocol as UDP. Select the VPC from the dropdown. Create a dedicated security group and open port 443 to allow access for target IP address.

Associate target VPC to the client VPN.

Select the VPC to which the client VPN need to be associated. Choose public subnet for users to access VPN over the internet. The VPC endpoint ENI will be created in the subnet selected.

Add Authorization Rule. In this demo, I am whitelisting for all users. Use the a separate AD group to allow access only to the specific users.

Authorization rule is the instructions that will allow users to access VPN from a specific network. In the below screenshot 0.0.0.0/0 was entered to allow all users from internet. Preferably, select a specific access group.

Now the client VPN configuration was completed.

Step 4) Configure client on local machine

Download the client configuration file to the local machine/laptop.

On your laptop, download and install open VPN. click here to download the client.

open client and upload the profile (*.ovpn file). Make sure to place the client cert, key in the same folder where *.ovpn profile file is located.

After the profile was successfully imported, the endpoint details will appear on the openVPN client.

Connect to the client. When prompted enter the password.

When connected to the VPN, the status on the client will change to connected.

Connect EC2 servers in the spoke accounts using their private IP address.

Reference:

https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/client-auth-mutual-enable.html

Re-Encrypt Your AWS EBS Volumes with a Shared KMS Key (CMK)

Rajesh Gundeti — Fri, 28 Mar 2025 01:34:52 +0000

Introduction
Amazon Elastic Block Store (EBS) allows encryption using AWS-managed or customer-managed keys. However, modifying the encryption key of an existing EBS volume is not straightforward. This blog post walks through a Python script utilizing Boto3 to re-encrypt an EBS volume with a shared AWS Customer Managed key (CMK), ensuring data security compliance. In this scenario the CMK is shared from a different account. We will go through the steps to change the encryption key of the volume from AWS default key to shared CMK.

Required Libraries
The script makes use of the boto3 library, which is the official AWS SDK for Python. It allows programmatic access to AWS services, including EC2, which manages EBS volumes.

To install Boto3, run:

pip install boto3

The script uses the following components from boto3:

boto3.client('ec2'): Creates an EC2 client to interact with AWS EC2 services.
create_snapshot(): Creates a backup snapshot of an EBS volume.
get_waiter('snapshot_completed'): Waits until the snapshot is fully created.
copy_snapshot(): Copies the snapshot with a new encryption key.
get_waiter('snapshot_completed'): Ensures the copied snapshot is ready.
create_volume(): Creates a new EBS volume from the copied snapshot.
get_waiter('volume_available'): Ensures the new volume is available.
detach_volume(): Detaches the old volume from the EC2 instance.
attach_volume(): Attaches the new volume to the EC2 instance.
describe_volumes(): Retrieves details of the newly attached volume.

Understanding the Script
The provided Python script automates the process of changing an EBS volume’s encryption key in five key steps:

Step 1: Create a Snapshot of the Existing Volume

The script begins by creating a snapshot of the existing EBS volume, serving as a backup before re-encryption.

snapshot = ec2.create_snapshot(VolumeId=volume_id, Description="Snapshot before changing encryption key")
snapshot_id = snapshot['SnapshotId']

To ensure data consistency, it waits for the snapshot to complete before proceeding.

waiter = ec2.get_waiter('snapshot_completed')
waiter.wait(SnapshotIds=[snapshot_id])

Step 2: Copy the Snapshot with the Shared CMK

The script then copies the snapshot, applying the new KMS key for encryption.

copied_snapshot = ec2.copy_snapshot(
    SourceRegion=region,
    SourceSnapshotId=snapshot_id,
    DestinationRegion=region,
    KmsKeyId=kms_key_arn,
    Encrypted=True,
    Description="Snapshot with customer-managed KMS key"
)
copied_snapshot_id = copied_snapshot['SnapshotId']

A wait process ensures the copied snapshot is ready for use.

Step 3: Create a New Volume from the Copied Snapshot

The copied snapshot is then used to create a new volume that is encrypted with the new KMS key.

new_volume = ec2.create_volume(
    SnapshotId=copied_snapshot_id,
    AvailabilityZone=availability_zone,
    Encrypted=True,
    KmsKeyId=kms_key_arn
)
new_volume_id = new_volume['VolumeId']

A waiter ensures the new volume is available before proceeding.

Step 4: Replace the Old Volume with the New Volume

The script detaches the old volume from the EC2 instance:

ec2.detach_volume(VolumeId=volume_id)

After confirming the old volume is detached, the new volume is attached in its place.

ec2.attach_volume(VolumeId=new_volume_id, InstanceId=instance_id, Device=device_name)

Step 5: Verify the New Volume

Finally, the script retrieves and prints the details of the newly attached volume to verify the encryption change.

volume_info = ec2.describe_volumes(VolumeIds=[new_volume_id])
print(f"New volume details: {volume_info}")

Script
Below is the full python code to change the encryption key of the EBS volume.

import boto3

def convert_ebs_encryption(volume_id, kms_key_arn, region, availability_zone, instance_id, device_name):
    ec2 = boto3.client('ec2', region_name=region)

    # Step 1: Create a snapshot of the existing volume
    snapshot = ec2.create_snapshot(VolumeId=volume_id, Description="Snapshot before changing encryption key")
    snapshot_id = snapshot['SnapshotId']
    print(f"Snapshot created: {snapshot_id}")

    # Wait for the snapshot to complete
    waiter = ec2.get_waiter('snapshot_completed')
    waiter.wait(SnapshotIds=[snapshot_id])

    # Step 2: Copy the snapshot with the new KMS key
    copied_snapshot = ec2.copy_snapshot(
        SourceRegion=region,
        SourceSnapshotId=snapshot_id,
        DestinationRegion=region,
        KmsKeyId=kms_key_arn,
        Encrypted=True,
        Description="Snapshot with customer-managed KMS key"
    )
    copied_snapshot_id = copied_snapshot['SnapshotId']
    print(f"Copied snapshot created: {copied_snapshot_id}")

    # Wait for the copied snapshot to complete
    waiter.wait(
        SnapshotIds=[copied_snapshot_id],
        WaiterConfig={
            'Delay': 60,  # Wait 60 seconds between each attempt
            'MaxAttempts': 60  # Maximum of 60 attempts
        }
    )

    # Step 3: Create a new volume from the copied snapshot
    new_volume = ec2.create_volume(
        SnapshotId=copied_snapshot_id,
        AvailabilityZone=availability_zone,
        Encrypted=True,
        KmsKeyId=kms_key_arn
    )
    new_volume_id = new_volume['VolumeId']
    print(f"New volume created: {new_volume_id}")

    # Wait for the new volume to become available
    waiter = ec2.get_waiter('volume_available')
    waiter.wait(VolumeIds=[new_volume_id])
    print(f"New Volume is available")

    ## Step 4: Detach the old volume and attach the new volume
    #ec2.detach_volume(VolumeId=volume_id)
    #ec2.attach_volume(VolumeId=new_volume_id, InstanceId=instance_id, Device=device_name)
    #print(f"New volume {new_volume_id} attached to instance {instance_id} as {device_name}")

    # Step 4: Detach the old volume
    ec2.detach_volume(VolumeId=volume_id)

    # Wait for the old volume to be fully detached
    detach_waiter = ec2.get_waiter('volume_available')
    detach_waiter.wait(VolumeIds=[volume_id])
    print(f"Old volume {volume_id} detached successfully")

    # Attach the new volume
    ec2.attach_volume(VolumeId=new_volume_id, InstanceId=instance_id, Device=device_name)
    print(f"New volume {new_volume_id} attached to instance {instance_id} as {device_name}")

    # Step 5: Verify the new volume
    volume_info = ec2.describe_volumes(VolumeIds=[new_volume_id])
    print(f"New volume details: {volume_info}")

# Example usage
#convert_ebs_encryption('<EBS_VOLUME_ID>', '<CUSTOMER_KMS_KEY_ID>', '<REGION>', '<AVAILABILITY_ZONE>', '<INSTANCE_ID>', '<DEVICE_NAME>')

Conclusion
This automated approach enables AWS users to change the encryption key of an EBS volume without downtime for all non root volumes. It ensures compliance with security policies while maintaining data integrity. By leveraging Boto3 and AWS KMS, organizations can efficiently manage encrypted EBS volumes at scale.

#security

Rajesh Gundeti — Thu, 13 Mar 2025 21:55:38 +0000