Forem: Mbanugo Franklyn

Fast Static Website Deployment Using AWS And Pulumi

Mbanugo Franklyn — Sat, 05 Apr 2025 21:28:04 +0000

This is a submission for the Pulumi Deploy and Document Challenge: Fast Static Website Deployment

What I Built

For this challenge, I successfully deployed a basic web page using Infrastructure as Code (Pulumi), AWS S3 Bucket and CloudFront. This project used the following tool:

AWS S3 Bucket for Static Website Hosting
AWS CloudFront which serves as a Content Delivery Network (CDN) for global distribution, caching, and HTTPS support.
Automating Cloud Resources Using Pulumi

Live Demo Link

Demo Link

Project Repo

Github Repo

My Journey

In this project, I used the following technologies:

Web Page: HTML
Cloud Provider: AWS
Infrastructure as Code: Pulumi
Programming Language: Python

Prerequisites

Before you begin, ensure you have:

AWS Account: You can sign up for a free tier AWS account if you don’t have one. You'll need this to create and access the S3 bucket and CloudFront Services.
AWS CLI installed and configured with your credentials.
IAM User Account: Create an IAM (Identity Access Management) user with full access and permissions to use the S3 and CloudFront services. You’ll need an IAM User instead of using your main account.
Pulumi Account: You can sign up for a Pulumi account if you don't have one. You will need this to be able to access your projects and stacks.
Pulumi CLI: You can install the Pulumi CLI Here. You'll use it to login and automate your resources.
Python3.6 or higher
Basic understanding of python
Basic understanding of AWS services

1. Configure AWS Credential in the CLI:

After installing the AWS CLI, You need to configure your AWS CLI to enable you to connect and access your account. Run the following command to configure your AWS CLI:

aws configure

Enter the following details:

AWS_ACCESS_KEY_ID= # From IAM user security credentials
AWS_SECRET_ACCESS_KEY= # From IAM user security credentials
DEFAULT_REGION=

2. Configure Pulumi CLI:

After configuring the AWS CLI, you need to configure the Pulumi CLI to enable you to access and manage your project and stacks. Run the following command to authenticate your Pulumi CLI:

pulumi login

You will be prompted to manage your stacks by logging in using your personal access token or hitting the <ENTER> key to sign in using the browser.

In this project, I will be using my personal access token to login. To get your personal access token:

navigate to your dashboard

click on the icon of your profile:

Click on the personal access token.

Click on Create token

You will be asked to add a Description and Expiration of your personal access token.

After creating your token, copy and paste your token in the CLI. You should see the following output when you have successfully logged in:

3. Initializing Pulumi Project

After you have successfully logged in, you need to initialize a new Pulumi project template to manage your infrastructure as code. In this project, I will be initializing a pulumi aws-python project template.

Run the following command to initialize a new pulumi project:

pulumi new

Select the aws-python template. This template shows that you will be using AWS as your cloud provider and Python as your programming language.

You will be prompted to enter the following details:

Project name: # Name of the project you want to use
Project Description: # Name of your project description
Stack: # Name of your stack (e.g. dev, stage, prod....)
AWS Region: # The AWS Region you want to deploy your infrastructure

You will also be prompted to select the toolchain for installing dependencies and running the program. In this project, I will be selecting pip toolchain.

After entering the following details, it will automatically save the configuration, Install the necessary dependencies, and create a virtual environment for your project.

4. Building the application

You will need to create a new folder named www in your project directory. This folder will contain the files(index.html) for your website.

The index.html file will contain the following code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Deploying A Static Website</title>
</head>
<body>
    <h1>Pulumi Challenge</h1>
    <h2>Deploying A Static Website</h2>
    <p>Welcome to the Pulumi challenge! This is a simple static website deployed using Pulumi.</p>
</body>
</html>

Next, you will need to update the content __main__.py file. This file will contain the infrastructure to automate your resources to AWS. The __main__.py file will contain the following:

Import Pulumi Configuration Setting

Let's start by creating a variable named config to import the Pulumi Configuration Setting that will:

Invoke and access the configuration value stored in the Pulumi.<stack-name>.yaml file.
Retrieve the value of the path configuration key.

config = pulumi.Config()
path = config.get("path") or "./www"

In your project directory, navigate to the Pulumi.<your-stack-name>.yaml file and add the following code:

Fast-Static-Website:path: ./www

Create S3 Bucket
Next, create a variable named bucket that will:

Create an S3 bucket resource using the newer version of Pulumi

bucket = aws.s3.BucketV2(
    'Static-Website-Bucket'
)

Configure The Bucket To Host A Static Website
Next, create a variable named bucket_website that will:

Configure the s3 bucket to host a static website using S3.BucketWebsiteConfigurationV2.
The S3.BucketWebsiteConfigurationV2 resource enable static website hosting and define the behavior for serving files.

bucket_website = aws.s3.BucketWebsiteConfigurationV2(
    'Static-Website-Bucket-Configuration',
    bucket=bucket.id,
    index_document={'suffix': 'index.html'},
    error_document={'key': 'error.html'},
)

Creating An IAM Policy For The Bucket
Next, create a variable named bucket_policy_resource that will:

Create an S3 Bucket Policy
Allow only CloudFront to access the objects in the bucket

bucket_policy_resource = aws.s3.BucketPolicy(
    "bucketPolicyResource",
    bucket=bucket.id,  # Reference the bucket's ID
    policy=bucket.arn.apply(lambda arn : f"""{{
        "Version": "2012-10-17",
        "Statement": [
            {{
                "Effect": "Allow",
                "Principal": {{
                    "Service": "cloudfront.amazonaws.com"
                }},
                "Action": "s3:GetObject",
                "Resource": "{arn}/*"
            }}
        ]
    }}"""
))

Creating Bucket Ownership Control
Next, create a variable named ownership_controls and public_access_block that will:

Create an S3 bucket ownership control
Create an s3 Bucket Public Access Block

ownership_controls = aws.s3.BucketOwnershipControls(
    "ownership-controls",
    bucket=bucket.bucket,
    rule={
        "object_ownership": "ObjectWriter",
    },
)

public_access_block = aws.s3.BucketPublicAccessBlock(
    "public-access-block",
    bucket=bucket.bucket,
    block_public_acls=False,
)

Using Synced Folder To Sync Local Files to S3 Bucket
Next, create a variable named bucket_folder that will:

Upload local files to S3 bucket

bucket_folder = synced_folder.S3BucketFolder(
    "bucket-folder",
    acl="public-read",
    bucket_name=bucket.bucket,
    path=path,
    opts=pulumi.ResourceOptions(depends_on=[ownership_controls, public_access_block]),
)

Before adding this content, you need to install pulumi-synced-folder>=0.0.0,<1.0.0 package and import the package in your __main__.py file. Run the following command to install the package:

pip install pulumi-synced-folder>=0.0.0,<1.0.0

Add the import at the top of your code in your __main__.py file:

import pulumi_synced_folder as synced_folder

Creating CloudFront Distribution
Next, create a variable named cdn that will:

Create a CloudFront CDN (Content Delivery Network) to distribute and cache the website
Improve Performance
HTTPS Support

cdn = aws.cloudfront.Distribution(
    "cdn",
    enabled=True,
    origins=[
        {
            "origin_id": bucket.arn,
            "domain_name": bucket_website.website_endpoint,
            "custom_origin_config": {
                "origin_protocol_policy": "http-only",
                "http_port": 80,
                "https_port": 443,
                "origin_ssl_protocols": ["TLSv1.2"],
            },
        }
    ],
    default_cache_behavior={
        "target_origin_id": bucket.arn,
        "viewer_protocol_policy": "redirect-to-https",
        "allowed_methods": [
            "GET",
            "HEAD"
        ],
        "cached_methods": [
            "GET",
            "HEAD"
        ],
        "default_ttl": 86400,
        "max_ttl": 31536000,
        "min_ttl": 60,
        "forwarded_values": {
            "query_string": False,
            "cookies": {
                "forward": "none",
            },
        },
    },
    custom_error_responses=[
        {
            "error_code": 404,
            "response_code": 404,
            "response_page_path": "/error.html",
        }
    ],
    restrictions={
        "geo_restriction": {
            "restriction_type": "none",
        },
    },
    viewer_certificate={
        "cloudfront_default_certificate": True,
    },
)

5. Exporting The URLs

Add the following code at the bottom of your code to export the URLs of the static website hosted on S3 bucket and the CloudFront distribution.

pulumi.export('websiteURL', bucket_website.website_endpoint)
pulumi.export("cdnURL", pulumi.Output.concat("https://", cdn.domain_name))

6. Running the script

Run the following command to execute the script:
pulumi up

You should see the following output:

You will be prompted if you want to perform the update, click yes

Challenges Faced

Error-1: Added a wrong argument when creating my S3 bucket

Solution: I used Pulumi copilot to know why i was getting the error and I fixed it by taking out the wrong argument.
Error-2: When trying to create the resources using pulumi up i encountered this error:

Solution: I used Pulumi copilot to check why it was failing then it showed that the error occurred when trying to set ACLs on a bucket that has Object Ownership set to "Bucket owner enforced", which disables ACLs. Then i saw an AWS article on how to control ownership of bucket and disabling ACLs in your bucket

After that i used Pulumi copilot to check on how i can have control of the object and allow acl.

Using Pulumi

In this project, I deployed a static website to AWS using Pulumi with Python, AWS S3, AWS IAM and AWS CloudFront. Pulumi handle the provisioning, configuration and integration of AWS resources efficiently.

Why Pulumi Was Beneficial

Infrastructure as Code: Defining my infrastructure using Python made it easy to understand, build, deploy and manage cloud infrastructure.
Preview: pulumi preview let me see what changes would happen before actually applying them.

Create A Weather Data Collection System Using Python In 7 Steps

Mbanugo Franklyn — Wed, 05 Mar 2025 23:07:57 +0000

Have you ever wondered how weather data is collected or analyzed?

In this article, you'll learn how to create a weather data collection system using Python, Matplotlib, and AWS SDK for Python (boto3). This approach allows you to create and analyze weather data from multiple locations effectively at scale.

Prerequisites

Before you begin, make sure you have:

AWS Account: You can sign up for a free tier AWS account if you don’t have one. You'll need this to create and access the S3 bucket storing your weather data.
AWS CLI: You can Install the AWS Command Line Interface to interact with AWS services. You'll use it to configure your credentials and manage your S3 bucket.
IAM User Account: Create an IAM(Identity Access Management) user with full access and permissions to use the S3 service. You’ll need an IAM User instead of using your main account.
OpenWeather API Account - You can sign up for a free tier account here. You need an API key to request the weather information.
Python3.6 or higher
Basic understanding of Python

Step 1: Configure AWS Credentials in the command line (CLI)

After installing the AWS CLI, You need to configure your AWS CLI to enable you to connect and access your account. Run the following command to configure your AWS CLI:

aws configure

Enter the following details:
AWS_ACCESS_KEY_ID=<your_aws_access_key> AWS_SECRET_ACCESS_KEY=<your_aws_secret_access_key> DEFAULT_REGION=<aws_region>

Step 2: Create a virtual environment

After configuring your AWS CLI, you need to create a virtual environment for Python to enable us to isolate all the packages we’ll use in this project in a separate Python environment.

Run the following command to create a virtual environment:

  python -m venv <name-of-your-virtual-env-folder>

After creating your virtual environment, you need to activate your virtual environment before installing or using any packages in your virtual environment.

To activate the virtual environment on Windows (with Powershell) run the following command:

.\venv\Script\Activate.ps1

On macOS/Linux:

source venv/bin/activate

Step 3: Installing the packages and libraries

After activating the virtual environment. Run the following command to install the packages or library using the requirements.txt file. This file contains a list of all the tools (also called libraries or packages) you need for this project.

The requirements.txt file will contain the following libraries:

boto3==1.26.137
python-dotenv==1.0.0
requests==2.28.2
matplotlib

pip install -r requirements.txt

Step 4: Configuring the environment variables

Create a .env file in the root directory and configure the following contents:

OPENWEATHER_API_KEY=your_api_key
AWS_BUCKET_NAME=your_bucket_name

Step 5: Creating a Python file

After configuring the environment variables. Create a Python file named weather-dashboard.py.
Our python file will contain the following:

1. Setting Up The WeatherDashboard Class

Let’s start by creating our class constructor that will:

Fetch weather data from OpenWeather.
Create a S3 bucket.
Save fetched data to S3

class WeatherDashboard:
    def __init__(self):
        self.api_key = os.getenv('OPENWEATHER_API_KEY')
        self.bucket_name = os.getenv('AWS_BUCKET_NAME')
        self.s3_client = boto3.client('s3')

This code creates a class constructor named WeatherDashboard and stores different methods with each task to be executed. The __init__ method takes a parameter self. self is a reference to the current instance of the class and is used to access variables that belong to the class.

The __init__ method creates various objects that store the self.api_key for the OpenWeather API, bucket_name for the s3 bucket name and self.s3_client initializes an s3 client using boto3

Fetch Weather Data From OpenWeather

def fetch_weather(self, city):
    """Fetch weather data from OpenWeather API"""
    base_url = "http://api.openweathermap.org/data/2.5/forecast?"
    params = {
        "q": city,
        "appid": self.api_key,
        "units": "imperial"
    }

    # Fetch weather data from API and handle errors if any occur
    try:
        response = requests.get(base_url, params=params)
    except requests.exceptions.RequestException as e:
        print(f"Error fetching weather data: {e}")
        return None
    else:
        response.raise_for_status()
        return response.json()

The function fetches weather data from the OpenWeather API, sends the request, and returns the JSON response. This function also includes error handling to manage any potential issues.

Create S3 Bucket

def create_bucket_if_not_exists(self):
    """Create S3 bucket if it doesn't exist"""
    try:
        self.s3_client.head_bucket(Bucket=self.bucket_name)
        print(f"Bucket {self.bucket_name} exists")
    except:
        print(f"Creating bucket {self.bucket_name}")
    try:
        # Simpler creation for us-east-1 region
        if os.getenv('AWS_DEFAULT_REGION') == 'us-east-1':
            self.s3_client.create_bucket(Bucket=self.bucket_name)
            print(f"Successfully created bucket {self.bucket_name}")
        else:
            self.s3_client.create_bucket(
                Bucket=self.bucket_name,
                CreateBucketConfiguration={
                    'LocationConstraint': 'eu-north-1'
                }
            )
            print(f"Successfully created bucket {self.bucket_name}")

    except Exception as e:
        print(f"Error creating bucket: {e}")

This function checks if an S3 bucket with the specified name already exists. If the bucket is not found, it creates a new one. This function also includes error handling to manage any potential issues.

Save Fetched Data To S3

def save_to_s3(self, weather_data, city):
    """Save weather data to S3 bucket"""
    if not weather_data:
        return False

    timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
    file_name = f"weather-data/{city}-{timestamp}.json"

    try:
        """put weather data into S3 bucket"""
        weather_data['timestamp'] = timestamp
        self.s3_client.put_object(
            Bucket=self.bucket_name,
            Key=file_name,
            Body=json.dumps(weather_data),
            ContentType='application/json'
        )
        print(f"Successfully saved data for {city} to S3")
        return True
    except Exception as e:
        print(f"Error saving to S3: {e}")
        return False

This function stores weather data as a JSON file in the S3 bucket. This function also includes error handling to manage any potential issues.
2. Visualize Weather Data Using Matplotlib

Next, Let’s create a function named visualize_weather_data outside our class constructor that:

Creates a bar chart to represent the weather data.
Adds city names on the x-axis and temperatures on the y-axis.
Annotates each bar with the weather conditions.

# create a function that takes three parameters for the cities, temperatures, and conditions

def visualize_weather_data(cities, temperatures, conditions):
    # Visualize the weather data using matplotlib.
    axes = plt.subplots()[1] # create multiple subplots
    # Create a bar chart
    axes.bar(cities, temperatures, color= 'skyblue') # create a bar chart using the cities and temperature values
        # Add labels and title
    axes.set_xlabel('City')
    axes.set_ylabel('Temperature (°F)')
    axes.set_title(f"Weather in {cities} - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")

    # Annotate each bar chart with the weather conditions
    for i, condition in enumerate(conditions):
        if i < len(temperatures):
            axes.text(i, temperatures[i], condition, ha='center', va='bottom')

    # Display the plot
    plt.show()

3. Fetching And Storing The Weather Data In The Main Function

Next, Let's create a function named main outside our class constructor that:

Creates an of Instance of our class constructor WeatherDashboard
Fetches the weather data for our predefined cities
Allows user to add more cities
Saves data to S3
Calls the visualization function


def main():
    dashboard = WeatherDashboard()

    # Create bucket if needed
    dashboard.create_bucket_if_not_exists()

    cities = ["Philadelphia", "Seattle", "New York"]

    while True:
        # create an input that prompt the user if he will like to add another cities and if yes, add the city to the cities list and if no, continue with the existing cities list
        add_more = input(f"Will you like to add another city to the list? (yes/no): ").lower()
        if add_more == "yes":
            city = input("Enter city name: ")
            cities.append(city)
            break
        elif add_more == "no":
            break
        else:
                print("Invalid input. Please enter 'yes' or 'no'.")

    #create an empty list for conditions
    conditions = []
    # create an empty list for temperatures
    temperatures = []

    for city in cities:

        print(f"\nFetching weather for {city}...")

        # create a variable to hold the weather data for the city and print the temperature, feels like, humidity and description
        weather_data = dashboard.fetch_weather(city)
        if weather_data:
            temp = weather_data['list'][0]['main']['temp']
            feels_like = weather_data['list'][0]['main']['feels_like']
            humidity = weather_data['list'][0]['main']['humidity']
            condition = weather_data['list'][0]['weather'][0]['description']

            print(f"Temperature: {temp}°F")
            print(f"Feels like: {feels_like}°F")
            print(f"Humidity: {humidity}%")
            print(f"Conditions: {conditions}")

            # Add temperatures and conditions to respective lists
            temperatures.append(temp)
            conditions.append(condition)

        # Save to S3
        success = dashboard.save_to_s3(weather_data, city)
        if success:
            print(f"Weather data for {city} saved to S3!")
        else:
            print(f"Failed to fetch weather data for {city}")

    # calling the visualize_weather_data function
    visualize_weather_data(cities, temperatures, conditions)

    return "Weather data collected successfully"

If __name__ == “__main__”:
     main()

This function stores the instance of our class constructor in a variable name dashboard. The main function fetches the weather data for predefined cities, saves it to an S3 bucket, prints the status of the weather data, and visualizes the weather data using matplotlib.

STEP 6:

Run the following command to execute the application:

python weather_dashboard.py

You should see the following output:

Step 7: Conclusion

Congratulations! You’ve successfully built a weather data collection system using Python, Matplotlib, and AWS SDK. This application fetches real-time weather data, stores it in an S3 bucket, and creates visual representations of the data. Also, you can further enhance the project by:

Adding automated data collection scheduling
Adding automated testing
Creating a web interface for your dashboard

GitHub Repositiory:

You can find the complete project files in this repository: Github

Resources for further learning:

Matplotlib

Matplotlib Subplot Documentation

OpenWeather API Documentation

AWS S3 Documentation