Internal Details About S3

Harshavardhan S U — Thu, 14 Aug 2025 08:47:58 +0000

We will see about how Amazon S3 works.
S3 is based on object-based storage model, where each and everything you put into S3 is considered as an object.

An entity is considered an object if it has an Key (ID), data (blob), metadata and some attributes.

Here the Key is unique inside the bucket and metadata is data about the contents inside the object. The attributes are data about the object itself rather than the content inside.

And it seems, S3 runs on distributed systems and basically every object uploaded is copied across different nodes, said as multi AZ Replication to improve fault tolerance since even if one node fails, the data is available in remaining nodes.

The S3 follows flat storage model where there is no hierarchy in storage like we have in file storage where a folder can exist inside a folder. Here we cannot create a bucket inside a bucket. You can notice that you can upload a folder to S3, let's see that with an example. When you upload folder with picture1.png inside it. S3 will take it's path as its key which is images/picture1.png where "/" is just part of the object's key. Thereby creating an illusion of folder inside S3.
Every object inside a single bucket is indexed using object's key. When you upload a file into a bucket, the file's name will be taken as the object's ID.

Each operation you do in management console or AWS command line, it will fire a API call to the servers. The API is nothing but an access point for servers to utilize the server functions. The simple way for developers to create a bucket is using AWS CLI. Let's see some commands to get a hands on experience. Please ensure that you have configured the command line with AWS credentials and you have installed AWS CLI.

Bucket Management

# Create a bucket 
aws s3 mb s3://<bucket-name> 

# Delete a bucket: 
aws s3 rb s3://<bucket-name> --force
(Use --force to delete a non-empty bucket)

# List all buckets: 
aws s3 ls

Object Management

# Upload a file: 
aws s3 cp <local-file-path> s3://<bucket-name>/<key>

# Download a file: 
aws s3 cp s3://<bucket-name>/<key> <local-file-path> 

# Delete an object: 
aws s3 rm s3://<bucket-name>/<key>

Synchronization

# Sync local directory to bucket 
aws s3 sync <local-directory> s3://<bucket-name> 

# Sync bucket to local directory: 
aws s3 sync s3://<bucket-name> <local-directory>

Listing objects

# List objects in a bucket: 
aws s3 ls s3://<bucket-name> 

# For generating a presigned URL for an object:
aws s3 presign s3://<bucket-name>/<key> --expires-in <seconds>

I have included the link where you can see all the basic commands with arguments to manage buckets and objects.
s3 -- AWS CLI 2.28.6 Command Reference

Hosting My Portfolio on AWS: A Serverless Journey with S3, CloudFront, and Lambda

Harshavardhan S U — Fri, 04 Apr 2025 13:10:56 +0000

This article describes how I hosted my static portfolio website, which is a static website in the cloud using Amazon Web Services, leveraging serverless technology.

Thanks to Namecheap's free domain for students, I designed the website with Canva and developed it using Bootstrap 5, which has a 12-column Layout for easy mobile responsiveness.

Here's how I did it, the challenges I faced, and why AWS Serverless was the perfect fit.

These are the following services I used to make it possible.

Amazon S3: To store my website files, including HTML files, static folder (CSS, JS files), assets folder, and a resume.
Amazon CloudFront: To make the site globally available at greater speed using caching optimization.
Amazon Certification Manager: To obtain the SSL certificate for enabling https security on my site.
Amazon API Gateway: To make the lambda function available as an endpoint to the client side.
Amazon Lambda: To process the request and send a pre-signed URL of the resume stored in S3.
Amazon DynamoDB: To store the email IDs with timestamps for future use.

The architecture I used:

First, create a bucket in your preferred region and upload all the required files. Block all public access because we going to let only CloudFront access the bucket through OAC (Origin Access Control).

Bucket Items

Block Public Access

Bucket Policy

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::harsha-resume-doc/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::533267145529:distribution/E2NCAYZ67PRE3K"
                }
            }
        }
    ]
}

Create your free SSL certificate using ACM and verify it through your DNS provider. In my case, Namecheap's control panel, adding some DNS records for verification. After verification, the certificate is provided, and we are ready to use it for the next step.

Note: You can do this step only if you are creating the request in N. Virginia region, i.e. us-east-1.

Create a CloudFront Distribution with Origins as the bucket. Attach the SSL certificate you got using the previous step, and more importantly about the price class, I went with all edge locations for best performance.

Redirect your domain to the CloudFront by inserting the CNAME record in the DNS server in Namecheap. Now, you can view your website being hosted by pasting your domain name in the browser.

Let's add functionality to our resume download button.
Create a DynamoDB table with Email ID as a primary key for storing the email IDs of the users.

Create a proper execution role for the Lambda function with access to S3 and DynamoDB using AWS IAM service.

Create a Lambda function to get the request, process it, and store it in the DynamoDB table. Then, return a pre-signed URL to the resume back to the user for downloading it within 5 minutes (300 seconds).
Lambda Code in Python

import json
import boto3
import datetime
import os

# Initialize AWS clients
dynamodb = boto3.resource("dynamodb")
s3 = boto3.client("s3")

# Environment variables (set these in Lambda)
TABLE_NAME = os.environ["DYNAMODB_TABLE"]  # DynamoDB Table Name
S3_BUCKET = os.environ["BUCKET_NAME"]     # S3 Bucket Name
RESUME_KEY = os.environ["RESUME_KEY"]               # File key in S3

def lambda_handler(event, context):
    try:
        # Parse request body
        body = json.loads(event["body"])
        email = body.get("email")

        if not email:
            return {"statusCode": 400, "body": json.dumps({"message": "Email is required"})}

        # Store email & date in DynamoDB
        table = dynamodb.Table(TABLE_NAME)
        table.put_item(
            Item={
                "email": email,
                "date": str(datetime.date.today())
            }
        )

        # Generate a pre-signed URL for the resume
        presigned_url = s3.generate_presigned_url(
            "get_object",
            Params={"Bucket": S3_BUCKET, "Key": RESUME_KEY},
            ExpiresIn=300  # URL valid for 5 minutes
        )

        return {
            "statusCode": 200,
            "body": json.dumps({"message": "Success", "url": presigned_url})
        }

    except Exception as e:
        print(f"Error: {str(e)}")
        return {"statusCode": 500, "body": json.dumps({"message": "Internal server error"})}

Create an HTTP API using API Gateway and add a route request-resume with a POST Request. Integrate the API with the Lambda function and add this API as a form action in the download button submission form.

Ta-da!!! You will get your portfolio hosted completely using Serverless technology using AWS, and you can put this in your resume.

Errors I have to face:

Wrong access control of bucket Items in S3 Solved by attaching a bucket policy allowing CloudFront access via OAC.
Unable to cloudfront distribution Opened an AWS support ticket for account verification (resolved in 24 hours).

The cost this project incurs is based on the number of visitors to the site. If the number is below 10, it will be very low. As the data moving out of the cloud will incur cost. And yes, we still cache the site at the edge locations using CDN.

You can access the site via this link

Future considerations for enhancing the project:

Using AWS SES, sending the resume via the mail to the users, and adding a follow-up mail
Adding AWS WAF protection and its captcha in forms can secure our site against vulnerabilities.

Forem: Harshavardhan S U

Internal Details About S3

Hosting My Portfolio on AWS: A Serverless Journey with S3, CloudFront, and Lambda