Forem: Adrian Mudzwiti

Serverless FastAPI Deployment: Actions Speak Louder Than Words

Adrian Mudzwiti — Sun, 30 Nov 2025 12:54:11 +0000

The final chapter of the Serverless FastAPI app tetralogy has arrived, we started with developing our app locally, then we wrote tests and in the last chapter we used native tooling and services within AWS to secure our app from bad actors.

We've reached a fork in the road, we can continue to manually deploy our app by running commands locally or we can incorporate a more traditional approach to automatically test and deploy our app using a CI/CD pipeline.

I initially wanted to use Azure Pipelines, that's what I have been using at work daily for the past 6 years. I appreciate Azure DevOps, lovely platform.

- Cristiano Ronaldo voice (Infamous rant about nothing changing at Man Utd)

To change things up, I then thought why not use GitHub Actions ? The infrastructure and application code already exists in GitHub.

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.

GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository. For example, you can run a workflow to automatically add the appropriate labels whenever someone creates a new issue in your repository.

- GitHub

Adding GitHub as an identity provider

We have identified GitHub Actions as the platform to implement our CI/CD pipeline, before we begin we need to setup an authenticated connection between GitHub and AWS. We will achieve this by setting up GitHub as an identity provider within AWS IAM.

Navigate towards IAM in the AWS management console, select Identity providers under Access management.
Select Add provider.
Select OpenID Connect under Provider type.
Enter https://token.actions.githubusercontent.com under Provider URL.
Enter sts.amazonaws.com under Audience.
Select Add provider.

Creating a custom IAM policy

Next up is defining the permissions for an IAM role, we'll go for an approach that best aligns with the principle of least privilege.

Select Policies under Access management.
Select Create policy.
In the Policy editor, select the JSON button.
Replace the contents within the editor with the below JSON:
Select Next.
Enter GitHubActionsDeploymentPolicy under the Policy name.
Select Create policy.

Creating an IAM role

(IAM) roles are entities you create and assign specific permissions to that allow trusted identities such as workforce identities and applications to perform actions in AWS. When your trusted identities assume IAM roles, they are granted only the permissions scoped by those IAM roles. Using IAM roles is a security best practice because roles provide temporary credentials that do not need to be rotated.

- Amazon Web Services

Select Roles under Access management.
Select Create role.
Select Web identity under Trusted entity.
Select token.actions.githubusercontent.com as the identity provider under Web identity.
Select sts.amazonaws.com as the audience.
The input box for GitHub organization also supports personal GitHub accounts, feel free to enter your GitHub username.
Enter the name of the GitHub repository you are using for this app. Select Next.
Search for and select the policy we created earlier ("GitHubActionsDeploymentPolicy"). Select Next.
Provide a name for the role, i.e GitHub-Actions-Assume-Role
Scroll to the bottom of the page and select Create role.
Copy the arn for the role to your clipboard, you'll need this later.

Creating our first GitHub Actions workflow

Navigate towards your GitHub repo.
Select settings, select secrets and variables, select Actions.
Select New repository secret, enter ROLE_TO_ASSUME as the name and enter the arn for the role you copied earlier as the Secret.
Open your project in Visual Studio Code (or your IDE of choice) and create a new branch.
Create the following directories and files inside the project directory:
Edit the deploy.yml file and add the following code:

At a high level, the deploy.yml file contains 2 jobs, a job to run tests and a job to deploy the app. The Deploy job is dependent on the previous Test job passing. The role-to-assume references the arn we stored as a repository secret earlier on.

Commit the changes and publish the new branch.
Navigate towards your repo in GitHub, select the Pull requests tab, select Compare & pull request.
Select Create pull request.
Navigate towards the Actions tab, you should see the workflow running. You can review the logs to see the progress. If successful you'll see 2 endpoint URLs.

Destroying the cdk stack via GitHub Actions

We've seen how our app is deployed using the deploy.yml workflow, we'll now create a workflow to delete the CDK stack, it prompts the user for confirmation first before initiating the stack deletion.

Navigate back towards your IDE, select the destroy.yml file and add the following code:
Commit the changes and push the changes.
Navigate towards the Actions tab in your browser, you should currently see the Deploy Player FC API workflow that previously ran, navigate towards the main Actions to see all workflows.
You'll see the newly created Destroy Player FC CDK Stack workflow, select the workflow, select the Run workflow dropdown, enter y to initiate the destruction of the CDK stack.

We now have a complete CI/CD pipeline that automatically tests our FastAPI app and deploys it to AWS. You can use this foundation to extend the workflows further or add additional deployment stages.

Serverless FastAPI Security: Unlocked Doors Invite Unwanted Guests

Adrian Mudzwiti — Thu, 24 Jul 2025 20:28:27 +0000

In part 1 and part 2 of our Serverless FastAPI series, we covered the development and testing aspects of our FastAPI app. Now we’ll shift our attention to security.

Security shouldn’t be an afterthought, however, the theme of this series has been to get you up and running in a manner that is beginner-friendly, whilst also exposing you to shortcomings and approaches in an organic way.

Up until now, our Lambda Function URL has been publicly accessible, allowing every opportunist, every bad actor and their collective to have an open pass with an unprotected API provided they can find the endpoint URL.

The thought of an unprotected API in 2025 is starting to sound like a bad low-budget movie that went straight to DVD back in the 2000s. As the saying goes in the UK, it’s not looking good brav.

We’ll need to secure our API so that it’s not abused or compromised. AWS provides several complimentary security services that we can layer together to lock down access to our API.

We’ll implement security in two phases:

Phase 1: AWS IAM Authentication First, we’ll secure our existing Lambda Function URL by changing the authentication type from None to AWS_IAM, this restricts access to authenticated IAM users only.

Phase 2: OAuth 2.0 with API Gateway Then, we’ll implement a more sophisticated OAuth 2.0 flow using API Gateway and Amazon Cognito, suitable for client applications that need programmatic access.

Components that we’ll leverage for full OAuth 2.0 implementation:

Amazon Cognito: Handles user authentication and authorization for your web and mobile apps.
Resource Server: Defines API scopes that control what operations clients can perform.
API Gateway: HTTP API that intercepts requests, validates JWT tokens and forwards authorized request to Lambda.
JWT Authorizer: Built-in API Gateway component that validates JWT tokens against Cognito’s issuer and audience claims.

Below is a high-level sequence diagram that illustrates the OAuth 2.0 flow a client needs to complete to interact with our fully secured API.

Revisiting a familiar cdk stack

From the root of the project, activate the virtual environment, then navigate towards the iac directory and then open the iac_stack.py file.

We need to add a few imports for constructs that we will leverage as we build our security layer. Ensure that imports look like below:

Updating the Lambda Function URL authentication

There’s beauty in simplicity. Let’s start by securing our API, we can achieve this by changing the authentication type from None to AWS_IAM.

Deploy this change with cdk deploy

Now, if we attempt to access our API endpoint via a browser or REST API client like Postman, we’ll get a 403(Forbidden) response. Try it out!

Building the identity foundation with Amazon Cognito

We are going to create an Amazon Cognito User Pool, which is an OpenID Connect (OIDC) identity provider (IdP). Think of this as a guest list at an event, if you’re on the list, you’re allowed entry.

Creating our authentication domain

Next, we’ll create a User Pool Domain, if a User Pool is the guest list, then a User Pool domain is like the venue address where guests check in. It’s the specific location where your application(s) will send users to authenticate.

Setting access boundaries with the use of scopes

Next up is enforcing boundaries with the use of scopes. We have our guest list (User Pool) and our venue (User Pool Domain), but venues often have restricted areas, like a VVIP section cordoned off for elite members of society. That’s exactly what scopes do.

Scopes define the permission level that clients can request, in our case, full read/write access to the API.

Creating a Resource Server

Time to create the Resource Server. This is like officially registering your event with the venue’s management system. It tells the venue ‘this is our event, these are our VIP areas, and here are the access rules’, essentially linking your API to the permission system.

Setting up machine-to-machine access

We’re creating machine-to-machine access with a User Pool Client. Think of this as backstage crew passes, these aren’t for the audience, but for the technical staff, sound engineers or equipment operators who need to access different areas to keep the event running.

These credentials allow applications to authenticate themselves and request the access they need.

Building the fortress entrance

Now we’re building the fortress entrance with API Gateway. Think of this as constructing the main security checkpoint at your event’s entrance, the single point where every guest must pass through.

It’s the official gateway that intercepts everyone trying to enter, checks their credentials and either grants or denies access to the venue.

Scroll to the bottom of the file and add the below:

Create JWT Authorizer

Now we’re creating the JWT Authorizer. Think of this as installing a high-tech security scanner at your entrance checkpoint.

It automatically reads and validates the special security codes on each access pass, checking that they’re genuine, haven’t expired and were issued by the right authority. No human guard is needed to verify every detail.

Building Protected API Routes

Now we’re building protected API routes. Think of this as setting up specific security checkpoints for different areas of your venue, one for the main hall, another for VIP lounges and separate ones for backstage areas.

Each checkpoint knows exactly what credentials to check and directs validated guests to the right location.

This route protects the main players endpoint for add and retrieving players with JWT authorization for GET and POST operations.

Protects the individual player retrieval endpoint with JWT token validation.

Protects player update and removal endpoints (PATCH/DELETE) with JWT authorization.

Run cdk deploy

Testing our API using a REST API client

We’re going to test if our API still works with all the changes we have made. For that we will need a tool that can allow us to test our API, feel free to use any tool you’re comfortable with, I’ll use Postman from here on out.

Let’s retrieve the app credentials that were created for us using the CDK, open a browser and login to the AWS Management Console.

Search for Cognito in the main search bar, select the service.
Select PlayerFCUserPool under User Pools.
Select App clients under Applications from the left-hand side of the window.
Select PlayerFC-M2M-Client, you should see a Client ID and a hidden Client secret, leave the browser window open. We’ll revisit this page shortly.

Creating an environment & variables in Postman

Open Postman, select Environments from the sidebar.
Select the “+“ button to create a new environment.
Enter a name for the environment, perhaps PlayerFC shall suffice.
Create a new variable called client_id, set the type to secret, copy and paste the client id value from the App client in Amazon Cognito and set as the current value.
Create another variable called client_secret, set the type to secret, copy and paste the client secret value from the App client in Amazon Cognito and set as the current value.
Create the the last variable called access_token, set the type to secret, leave the initial value blank.
Select 💾 Save.

Exporting & Importing OpenAPI definition

Instead of manually creating the requests in Postman, we’ll head over to API Gateway export the OpenAPI definition and import the definition into Postman.

Search for API Gateway in the AWS Management console. Select the service.
Select PlayerFCHttpApi under APIs, select Export under Develop from the left sidebar.
Select Latest configuration from the Source drop-down list.
Ensure the toggle is set to on for Include API Gateway extensions.
Select JSON under Output format.
Click the Download button.

Navigate back to Postman and follow the below steps to import the OpenAPI definition:

Select Collections from the left sidebar.
Select Import, drag and drop the OpenAPI definition we downloaded earlier. Alternatively, select files, select the OpenAPI definition.
Select Postman Collection, select Import.

Generating an access token

Add a request under the players.
Enter Generate access token as the name for this request.
Select POST as the method.
Enter https://playerfc.auth.af-south-1.amazoncognito.com/oauth2/token in the URL input box.
Select the Authorization tab and ensure the Auth Type is set to No Auth.
Select the Body tab, select x-www-form-urlencoded, select Bulk Edit, copy and paste the below:

Select the Scripts tab, copy and paste the below JavaScript for Post-response, this will automatically populate the access_token environment variable whenever you execute the Generate access token request:

Select 💾 Save.
If you execute the request, an access token is returned and set as an environment variable.

Now that we have an access token, let's add a player, in the last blog it was Christoper Nkunku, keeping it within the French national team, let’s add Kylian Mbappé.

Select the Post /players request.
Select the Authorization tab, set the empty token field to access_token.
Select the Body tab, select raw, copy and paste the below JSON payload into the request body:

Select Send.

Feel free to test the other endpoints, don’t forget to set the access_token environment variable under the Authorization tab for all the other requests.

If you forget to add the access_token along with the request, you’ll be greeted with a 401(Unauthorized) response.

If you’re looking for the complete code, you’ll find it under my GitHub repo

With these security layers in place, your FastAPI app is protected from unauthorized access.

This project might squeeze another post in the series, only time will tell. Till then, take care.

Serverless FastAPI Testing: Use Moto and Just Mock It!

Adrian Mudzwiti — Tue, 01 Jul 2025 20:02:27 +0000

In my previous blog post Serverless FastAPI Development: Building Player FC API on AWS, we explored creating and deploying a FastAPI application on AWS. In this blog post we’ll take a look at testing our app locally.

We write tests to prove that our code works as designed, however since our code interacts with cloud services it’s somewhat of a challenge to mock tests to the cloud without actually making api calls that traverse the internet, well that is unless you use Moto.

Moto is a Python library that mocks AWS services, allowing you to test without making real API calls.

Why Mock AWS Services?

When it comes to testing applications that interact with cloud services like AWS, mocking becomes essential for a couple of practical reasons.

First, cloud services cost money. Testing against resources deployed in the cloud isn’t free.

Secondly, an active & reliable internet connection is required, it’s not ideal to have your tests bound to the internet. You might find yourself at a conference with slow and limited wifi connectivity or a space with public wifi that shouldn’t be trusted. You could be on a plane or train, you might even find yourself in a remote area.

Mocking allows you to run tests locally without incurring additional costs. Everyone loves to save money after all.

Setting Up Your Test Environment

Some preparation is required to ensure we can run our tests, we need a way for our tests to import modules that we have written as well as letting pytest know where these files are located.

This can be achieved by creating a conftest.py file as well as a pyproject.toml file.

conftest.py file gets the absolute path of the project root directory.
pyproject.toml file sets the path for our app, test paths and silences a deprecation warning for botocore.

Create these files at your project’s root:

Your First Test: The Root Endpoint

Create a directory that will be a home for our tests, name it tests and within this directory create a file named test_player.py.

Let’s create a test for our root endpoint, add the following imports at the top of the file:

Create a TestClient object and pass app as an argument, add a test function named test_root, see below for the complete code snippet:

Run pytest test_player.py::test_root in the terminal window. The test should pass.

Create Pytest Fixtures

We will use Fixtures to provide a defined, reliable and consistent context for our tests. This will include player data, mocked AWS credentials for moto and our mock DynamoDB table.

Let’s add a couple of fixtures to our code, we will start with creating a fixture that contains a single player’s data, add this code directly below the client object we created earlier:

Now we need to take a similar approach for representing all players, however creating a function with all this data will make the code long, a better approach would be to create a separate json file and load the data when the function is called.

Create a file named players.json in the tests directory and populate it with the below:

Add the below code to create a fixture that will load the all players data from the json file when the function is called:

Mocking AWS credentials and DynamoDB service

Create a fixture that will mock AWS credentials for below by adding the below code:

The mocked AWS credentials will be used as an argument for our mock DynamoDB table, add the below code to create another fixture for mocking the AWS DynamoDB service:

CRUD Testing Journey

With all the fixtures created, we are now at a stage that we can begin testing the other endpoints that would normally interact with AWS services, albeit mocked in nature.

We can create a test that will create and return the player data, this function takes in the dynamodb_table and player_data fixtures we created earlier as arguments, add the below code:

Run pytest test_player.py::test_create_and_get_player, this test too shall pass.

Onto the next endpoint, lets test if we can get all players, this will be achieved by loading the players data from a json file and asserting that players names are found and if a certain player is not found.

Run pytest test_player.py::test_get_all_players

We’re on a roll with tests that are passing at this stage, lets test the endpoint for updating a player details, the player in question is Christopher Nkunku, he will be transferring to Bayern Munich and will take up the number 10 jersey.

Run pytest test_player.py::test_update_player

Now let’s create a test for removing a player.

Run pytest test_player.py::test_delete_player

The final test is an edge case, lets create a test when removing a non existent player, an error 404 should be returned since the player does not exist.

Run pytest test_player.py::test_delete_non_existent_player

Conclusion

Testing locally? Sorted.

Next up: Locking down your Lambda Function URL because security isn’t optional. Stay tuned. ⚡️🔐

I’ll cover that in a future post. Until then, happy testing. ⚡️🐍

Serverless FastAPI Development: Building Player FC API on AWS

Adrian Mudzwiti — Sat, 11 Jan 2025 07:00:00 +0000

It's been a while since I've had the opportunity to build something simple, interesting and modern. Towards the backend of 2024 I stumbled across FastAPI and got excited, whilst I've built internal APIs at work before, I hadn't yet created anything public facing.

Hello FastAPI!

FastAPI is a modern, powerful framework for building APIs with Python and it seemed perfect for what I wanted to build, an API for basic football player info. I initially dubbed it "Jugador FC" before settling for "Player FC API".

Configuring Environment.

Before you begin, make sure you have the following requirements in place:

AWS CDK
Docker
Python 3.12.7

Creating the Project

Create a directory on your machine. Name it player_fc_fastapi_app, within this directory create the following subdirectories:

app
    Contains all the FastAPI code
dynamo_db_local
    Contains a python script to create a local version of an Amazon DynamoDB Table
iac
    Contains your stack files to create resources in AWS

I have made it easier by providing the commands that you can run to save time below:

The project directory structure should now look like below:

Setting up the Python environment

After creating the directory structure, create a text file called requirements.txt and insert the following lines in it:

Once you have created the requirements.txt file, create a virtual environment and install the dependencies:

Setting up Amazon DynamoDB Local

Let's begin with setting up a local instance of DynamoDB, this will require Docker to be installed and running.

This will take a few seconds for the image to be pulled and starting a container, once done we can navigate towards the dynamo_db_local directory and create a create_ddb_table.py file, populate the file with the below code:

With this code, you can create a table in the local DynamoDB instance. Run the code snippet.

FastAPI Development

Now that we have a local instance of DynamoDB up and running, let's begin creating our app, navigate towards the app directory and create two files, main.py and requirements.txt.

Populate the requirements.txt with the below:

Create the below subdirectories :

models
Pydantic Player models
routers
Contains routes

Let's create a couple of models using Pydantic, we will use the Player and UpdatePlayer models to define the data structure of player info we can add or modify.

Within the models subdirectory, create an empty __init__.py file and a file named players.py and fill with the below code:

Within the routers subdirectory, create an empty __init__.py file and a file named players.py and fill with the below code:

Creating an empty __init__.py file turns a folder into a Python package.

Create a file named main.py within the app subdirectory and start populating it with the below code:

Test Drive

Time for a quick test drive, ensure you are in the app directory and run the below command to start Uvicorn:

Now that our app is up and running, navigate to http://127.0.0.1:8000/docs/

You will see the automatic interactive API documentation with 6 endpoints available:

Let's try adding a player. Select the POST /players endpoint, select the Try It out button and use the below payload to add the world's best player, "Vinícius Júnior":

Here's what each API operation looks like in action.

Adding a New Player:

Retrieving All Players:

Updating Player Information:

Getting Single Player Details:

Removing a Player:

Deployment using AWS CDK v2

Now that we are comfortable with running and testing our app locally, it's time to deploy our app on AWS. We will use the AWS CDK v2.

Navigate into the iac directory, run the below command to initialize a cdk project:

Modify the requirements.txt file found in the subdirectory, add the below line:

Let's define a DynamoDB Table, Lambda function and a Lambda function url. In the current iac directory, there is another subdirectory that you need to navigate towards (iac). Open the iac_stack.py file and replace the contents of the CDK stack with the code below:

We have one final step before we initiate the deploy, set the flag for local_development: bool to False in the players.py file in the app/routers directory.

Activate the virtual environment within the iac directory and install the dependencies with the below commands:

Deploy the app with the cdk deploy command.

Once the deployment is complete, you'll see a function URL in the terminal output, this is your API endpoint on AWS.

Test all endpoints using the function URL like we did during the local test drive. Once you have added a player it's time to verify if our player data has persisted or vanished into the ether.

To verify everything's working:

Head over to the AWS Management Console
Navigate to DynamoDB
Find the Players Table
Select Explore table items

You should see your player data in the cloud:

💡 Important: Don't forget to clean up resources! When no longer needed, you can run the cdk destroy command to delete all AWS resources that were created.

That wraps up our journey from local FastAPI development to serverless deployment on AWS.

Exploring AWS Serverless Deployments with CDK v2: From RSS to X Posts - Part 3 of the Odyssey

Adrian Mudzwiti — Thu, 01 Aug 2024 17:29:20 +0000

Welcome to part 3 of “Exploring AWS Serverless Deployments with CDK v2”. Firstly I’d like to thank you for your patience as there’s been a bit of a gap since part 2. I was deep into studying and working on serverless projects at work which kept me away, but i’m excited to get back on track and continue our exploration.

In previous posts, we’ve defined our constructs and deployed them to AWS. Today, we’ll focus on an essential practice: testing. Proper testing ensures that our deployments work as expected and can save us from potential issues.

Getting Started With Testing

To get started, you’ll need to add pytest to your project’s dependencies (the main requirements.txt file for our stack).

pip install -r requirements.txt

Within our project, navigate towards the test directory, then unit and open the test_rss_lambda_ddb_socialshare_stack.py file. This auto generated test file includes an example test.

We don’t have an SQS construct in our stack but reviewing the example test provides some level of insight on how to test a construct. Let’s delete the auto generated example test and create our own test.

Setting Up The Testing Function

First, let’s create a reusable function to get the CloudFormation template from the stack:

Testing DynamoDB Table Properties

We’ll start by adding a test to check that the DynamoDB table in our stack has the correct properties. Here’s how to do it:

Testing Lambda Functions

Next, let’s ensure that our stack creates the correct number of Lambda functions and verifies their runtime version:

Running Tests

To run tests you can execute pytest in the terminal:

pytest

Below is the output you should receive:

================================================================================= test session starts =================================================================================
platform darwin -- Python 3.12.4, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/adrian/Developer/Projects/rss-lambda-ddb-socialshare
plugins: typeguard-2.13.3
collected 2 items                                                                                                                                                                     

tests/unit/test_rss_lambda_ddb_socialshare_stack.py ..                                                                                                                          [100%]

================================================================================= 2 passed in 21.86s ==================================================================================

Conclusion

In Part 3 of our series, we’ve learned how to test our CDK constructs. In the final installment, we will explore how to test Lambda functions locally.

Resources

Exploring AWS Serverless Deployments with CDK v2: From RSS to X Posts - Part 2 of the Odyssey

Adrian Mudzwiti — Sun, 31 Mar 2024 13:30:00 +0000

In this blog post, we'll continue our exploration of AWS Serverless deployments with CDK v2 by focusing on Lambda functions.

We'll explore how to create and integrate these functions into our architecture along with a crucial step of granting permissions to resources that are deployed within the stack.

Lambda

The first Lambda function that we will create will periodically query an RSS feed, process the data and store the data in DynamoDB.

To get started with Lambda we will use the Amazon Lambda Python Library, this will provide constructs for Python Lambda functions. This will require Docker to be installed and running.

Modify the requirements file for the stack as below:

Next we will create a directory for our first Lambda function:

mkdir lambda_rss_ddb_func
cd lambda_rss_ddb_func

Lets create a lambda_handler.py file, this file will contain our code that performs the magic:

code lambda_handler.py

The next file that we will create will be the requirements file for all our Python dependencies:

code requirements.txt

We will only be using the requests library in the Lambda function, so make sure to include requests in the newly created requirements.txt.

Time to write some code in the lambda_handler.py file, the code extracts post id, post title and link from a website feed, we will be using the feed from Hypebeast, once we have extracted the data we need, the data will be inserted into our DynamoDB Table.

Below is the code that you can populate your lambda_handler.py file:

Once you have modified your stack to include the construct to create a Lambda function, make sure to change the directory in your terminal to the root folder of the project.

Let's turn our attention back to our stack to define the Lambda construct, add the below code:

Amazon EventBridge rule

To run the lambda function on a schedule, we can make use of an Amazon EventBridge rule that will periodically run our Lambda function. Add the below construct and permissions to the stack:

Another Lambda function

In a few moments time we will create another directory for our second lambda function that is invoked from new records(s) being added to our DynamoDB Table and creates a post on X with the post title and post link.

SSM Parameter Store

To access the X API, you'll need to create an X Developer account, I've included the link in the resources section of this post.

We will need X credentials (consumer key, consumer secret, access token & access token secret).

These credentials need to stored somewhere securely, the SSM Parameter Store is a service that's free and will fulfill our next requirement well.

Nagivate towards the Parameter Store under AWS Systems Manager in the AWS Management Console.
Select the Create parameter button.
In the Name textbox, enter /x/consumer_key, select SecureString under Type and paste your consumer_key in the Value textbox.
Repeat the above process for the remaining credentials (consumer secret, access token & access token secret).

X API stuff out the way, let's create that directory:

mkdir lambda_x_share_func
cd lambda_x_share_func

Within this directory create another lambda_handler_py file and a requirements.txt file.

Below is the code you should insert in the newly created lambda_handler_py file in the lambda_x_share_func directory:

You'll need to specify a region when initializing the ssm_client, early on I noticed that Lambda was unable to access the values in the Parameter Store, this was strange as all the documentation I read seemed to indicate that the Lambda function should have been able to access the Parameter Store in the same region.

In the requirements.txt file make sure to include tweepy, that's the library that we will use in our Lambda function to interact programmatically with X.

Navigate back to the stack, we wil now create a construct for the LambdaShareFunc, add the below code:

In order for our Lambda function to read the Parameter Store values in SSM we can create a SSM Policy statement, this will grant the Lambda function permissions to retrieve the secrets.

We need to allow our Lambda function to act when new items are added to our DynamoDB table, this can be achieved using DynamoDB Streams, lets add another construct.

One last code addition to our stack is to enable DynamoDB Streams on our Table. This is achieved by adding stream=dynamodb.StreamViewType.NEW_IMAGE in the table construct:

We're almost on the final stretch, ensure that Docker is running.

We can now deploy the stack using cdk deploy, this will take a few moments.

Below are screenshots from my terminal window and AWS Management Console of the successful deployment:

I'll navigate towards an X burner account I created a couple of years ago for testing the X API, below are screenshots with Posts created:

Conclusion

In this blog post, we've delved into the intricacies of integrating Lambda functions, adding permissions to newly created constructs and enabling a DynamoDB stream trigger that invokes a Lambda Function to create a Post on X into our AWS serverless architecture deployments using CDK v2.

In an upcoming blog post we will shift our focus on testing constructs and lambda functions locally.

Resources

Exploring AWS Serverless Deployments with CDK v2: From RSS to X Posts - Part 1 of the Odyssey

Adrian Mudzwiti — Sat, 30 Mar 2024 13:30:00 +0000

Weekend projects often come and go, but on the rare occasion, one does stand out that is truly worth documenting.

In this multi-part blog series I invite you to join me on a journey into the realm of AWS serverless architecture deployments with CDK v2.

Together, we'll embark on a fascinating exploration - from harvesting RSS feeds to crafting X Posts.

Throughout this series, we'll explore how to set up a serverless solution that automatically gathers information from an RSS feed, extracts the important details from XML and saves them efficiently in a DynamoDB table.

Plus, we'll see how new entries in our DynamoDB table invokes a Lambda function to create X Posts using DynamoDB Streams.

In my professional sphere, I predominantly rely on the Azure SDK for Python, with occasional use of Terraform, while my personal projects frequently entail Pulumi, Terraform and once in a blue moon CloudFormation (Shocking, I know). However, for this specific endeavor, we are utilizing the AWS CDK as our conduit.

AWS CDK is an Infrastructure as Code tool that empowers us to define and manage resources with code, offering a fresh perspective on serverless architecture deployments.

Before we dive in, lets take a moment to visualize the architecture we are about to construct:

Pre-requisites:

AWS CLI
AWS CDK v2
Python
Docker

Creating a CDK Project

To get started with creating a new CDK project, you'll need to enter the below commands:

mkdir rss-lambda-ddb-socialshare
cd rss-lambda-ddb-socialshare
cdk init --language python

cdk init

Once the cdk init command has executed, you'll need to activate the virtual environment with one of the below commands depending on your OS:

source .venv/bin/activate

.venv\Scripts\activate.bat

Next you will need to install Python packages and dependencies required for our stack and constructs, run the below command:

pip install -r requirements.txt

Feel free to explore the project directory. You'll notice folders and files have been created for you.

┣ 📂.venv
┃ ┣ 📂bin
┃ ┣ 📂include
┃ ┣ 📂lib
┃ ┗ 📜pyvenv.cfg
┣ 📂rss_lambda_ddb_socialshare
┃ ┣ 📜__init__.py
┃ ┗ 📜rss_lambda_ddb_socialshare_stack.py
┣ 📂tests
┃ ┣ 📂unit
┃ ┃ ┣ 📜__init__.py
┃ ┃ ┗ 📜test_rss_lambda_ddb_socialshare_stack.py
┃ ┗ 📜__init__.py
┣ 📜.gitignore
┣ 📜app.py
┣ 📜cdk.json
┣ 📜README.md
┣ 📜requirements-dev.txt
┣ 📜requirements.txt
┗ 📜source.bat

The app.py file is your app's entry point. The code in this file instantiates an instance of the RssLambdaDdbSocialshareStack class from the rss_lambda_ddb_socialshare/rss_lambda_ddb_socialshare_stack.py file.

The most important file that we care about is the rss_lambda_ddb_socialshare/rss_lambda_ddb_socialshare_stack.py file, this is where most of the code to define resources will be added.

We will modify the stack to create the below resources:

DynamoDB Table
2 Lambda Functions
EventBridge rule

The first construct in our stack that we will add is a DynamoDB Table, you'll need to import RemovalPolicy and aws_dynamodb as dynamodb. Then modify the stack, I have included code snippets that you can use to overwrite what is currently in the rss_lambda_ddb_socialshare/rss_lambda_ddb_socialshare_stack.py file.

The removal policy we have defined in the DynamoDB Table construct is destructive, for production environments one would use the RETAIN attribute, e.g. removal_policy=RemovalPolicy.RETAIN

cdk synth

The next command to execute is cdk synth, this will generate a CloudFormation template for our current stack and be used for deployment.

cdk bootstrap

Once synthesized, we need to prepare an environment (target AWS account and AWS Region) for deployment. This is done using the cdk bootstrap command. This will provision an S3 bucket to store files as well as create IAM roles that are required to perform deployments. This command only needs to be run once.

cdk bootsrap aws://{AWS-ACCOUNT-NUMBER}/{REGION}

cdk deploy

The cdk deploy command will deploy our stack to AWS.

We can verify the deployment was successful by viewing the terminal output, alternatively log into the AWS Management Console.

Conclusion

That concludes Part 1, in an upcoming blog post, we will learn how to create constructs for Lambda functions, granting permissions for constructs within the stack and adding the actual code for our Lambda functions that performs the magic.

Resources

A tale of invocation - Using AWS Lambda to transfer files from AWS S3 to Azure Blob Storage

Adrian Mudzwiti — Sat, 25 Mar 2023 19:31:51 +0000

A few days ago I decided to scan a popular freelance site for an interesting problem to solve without the intention of applying for the job, as lady luck would have it I stumbled across an interesting challenge, the client needed to copy data from an S3 bucket to Azure Blob Storage using Power Automate. Seems like a fairly easy task right ?

I started off with reading the Microsoft Docs and reviewing the Amazon S3 Power Platform Connector.

There is a caveat when it comes to using the connector as Microsoft explicitly expresses a known limit of object sizes needing to being less than 3.5 MB, as to be expected with low-code development platforms, there’s always a catch.

This seemed like a perfect opportunity to build a Lambda function with Python.

In this post I will show you how to create a Lambda function that is invoked once a file has been uploaded to S3 and using Lambda layers to package libraries that will be used by the Lambda function.

Pre-requisites:

• AWS Account
• Amazon S3 Bucket
• Azure Subscription
• Azure Storage Account
• Docker

This post assumes that you have already created an S3 bucket in AWS and an Azure Storage account with a container. If you’re familiar with Pulumi, I have included Pulumi programs to create the required resources in both clouds in the below repo:

lambda-s3-to-blob

Step 1: Create an IAM execution role

The Lambda function that we will create in the next step will require permissions to access S3 and write permissions to CloudWatch logs.

Navigate towards the IAM console and select Roles under Access management from the left side-menu, select the Create role button, from the subsequent page that loads select AWS service as the Trusted entity type and Lambda as the Use case. On the Add permissions page you’ll need to search and select the checkbox for the below permissions:

AmazonS3ReadOnlyAccess
AWSLambdaBasicExecutionRole

Provide a name for the role e.g s3ToAzureBlobRole and finalize the creation of the role.

Step 2: Create a Lambda function

Navigate towards the Lambda console and select the Create function button.

Select Author from scratch and under the Basic information section provide a name of the function eg. s3ToAzureBlob and select Python 3.9 under the Runtime drop-down.

Under the Architecture section, if you’re using Apple Silicon like me, make sure to select arm64 otherwise if you’re using an intel based machine, select x86_64.

We’ll need to expand the Permissions and make sure Use an existing role is selected, from the drop-down select the role you created earlier (s3ToAzureBlobRole). Select the Create function button.

Under the code tab, you’ll need to replace the code shown in the console editor with the code shown below:

import boto3
import botocore
import os
import tempfile
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from azure.core.exceptions import ClientAuthenticationError, ServiceRequestError

s3 = boto3.resource('s3')

# Credentials for accessing Azure Blob Storage
storage_account_key = os.environ.get('storage_account_key')
storage_account_name = os.environ.get('storage_account_name')
connection_string = os.environ.get('connection_string')
container_name = os.environ.get('container_name')

def lambda_handler(event, context):
    # Get temp file location when running
    temFilePath = tempfile.gettempdir()

    # Change directory to /tmp folder
    os.chdir(temFilePath)

    for record in event['Records']:
        # Get bucket and key from s3 trigger event
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']

        # Get file name from key, join /tmp folder path and file name
        file_name = key
        upload_file_path = os.path.join(temFilePath, file_name)

        try:
            # Download the object from s3
            s3.meta.client.download_file(bucket, key, key)

            def upload_to_blob_storage(file_path, file_name):
                """Upload file to Azure storage as blob from /tmp folder"""
                blob_service_client = BlobServiceClient.from_connection_string(connection_string)
                blob_client = blob_service_client.get_blob_client(container=container_name, blob=file_name)

                with open(file_path, "rb") as data:
                    blob_client.upload_blob(data, overwrite=True)
                    print(f" {file_name} uploaded to Azure Blob !")

            upload_to_blob_storage(upload_file_path, file_name)

        except FileNotFoundError:
            print(f"The file {key} does not exist")
        except botocore.exceptions.ClientError as error:
            print(error.response['Error']['Code'], error.response['Error']['Message'])
        except ClientAuthenticationError as e:
            print(f"Error uploading file: {e}")
        except ServiceRequestError as e:
            print(f"Error uploading file: {e}")

Save the changes and select the Deploy button.

Step 3: Create an S3 trigger

A trigger is a service or resource that invokes your function. We will be using S3 as a source that invokes our function each time an object is uploaded to the S3 bucket.

Under the Trigger configuration, select S3 as the source and select the bucket, under Event type select Put, ensure the checkbox under Recursive invocation is selected and select the Add button to complete the trigger configuration.

Step 4: Create a Lambda Layer

Lambda layers allow you to package additional libraries that will be used by your function. Our function will make use of Azure libraries (azure-storage-blob, azure-core) to communicate with Azure services from the Python code.

You’ll need to create the following directory structure locally that is compatible with Python 3.9 to create a simulated Lambda environment with Docker.

├── requirements.txt
└── python/
    └── lib/
        ├── python3.9/
        │   └── site-packages/

In the requirements.txt file, specify the Azure Storage Blobs client library as well as the Azure Core shared client library for Python by adding the following lines in the file.

azure-storage-blob
azure-core

Ensure that the Docker daemon is running. You’ll need to install the library dependencies required by the Lambda function to the subfolders created earlier. Enter the below command:

docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.9" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.9/site-packages/; exit"

Once the dependencies have been downloaded you’ll need to zip the contents of the python folder.

zip -r azurelib.zip python > /dev/null

Navigate towards the Lambda console from your browser and expand the side menu by selecting the hamburger menu on the top left corner of the page. Select Layers under Additional resources. Select the Create layer button, you’ll need to provide a name for the layer, upload the zipped file contain the library our function requires to interact with Azure and select Create once you’ve provided the mandatory information.

Navigate towards the function you created earlier, scroll towards the bottom of the page and select Add a layer under the Layers section. Select Custom layers, select the layer you created earlier and select the version.

Step 5: Adding environment variables

Select the Configuration tab, select Environment variables, select the edit button then select Add environment variable, you’ll need to populate the Key/Value pairs with credentials required for accessing Azure Blob Storage.

To test if the solution works, you can upload a file to the S3 bucket, you can then review the CloudWatch logs for all log events.

You can also check if the container within your Azure Storage Account has an uploaded file.

Optional

You might want to increase the Timeout to 15 seconds to negate the function timing out, you can also increase the Ephemeral storage from 512 MB to a higher capacity, up to 10 GB.

Conclusion

The best source for new project ideas can be found looking at freelance sites. This approach leads to a more organic approach to learning.

Thanks for reading, I hope you’ve learnt something new.

Using AWS WAF to protect your WordPress site hosted on AWS

Adrian Mudzwiti — Sat, 18 Feb 2023 08:44:49 +0000

First blog post of the new year. It’s been a while. Throughout my cloud journey and continuous learning of the AWS platform and its services, I often notice that security is often overlooked when deploying WordPress sites, most tutorials will guide you through the steps for deploying a highly available WordPress site and neglect to show ways in which you can protect your WordPress site once deployed.

Disclaimer - The use of an AWS Web Application Firewall (WAF) and managed rules highlighted in this blog post is by no means exhaustive but can help improve your security posture and better protect your WordPress site, in this blog post we’ll take a look at using specific AWS Managed Rules for WAF.

Please take note that an AWS WAF can only be attached to the below resource types:

Amazon CloudFront distributions
Application Load Balancer
Amazon API Gateway
Amazon AppSync

An AWS WAF can be thought of as product that inspects traffic between your web application and the internet, it makes use of rules that allow you to block or allow web requests based on conditions that you define.

We’ll be making use of a Web ACL and a few AWS Managed Rules to protect our WordPress site.

Managed Rules contain pre-defined rules that are designed and managed by AWS and AWS Marketplace sellers to protect your web application.

The AWS Managed Rule Groups that will be used are listed below:

WordPress application managed rule group

The WordPress application rule group contains rules that block request patterns associated with the exploitation of vulnerabilities specific to WordPress sites.

SQL database

The SQL database rule group contains rules to block request patterns associated with exploitation of SQL databases, like SQL injection attacks. This can help prevent remote injection of unauthorized queries.

PHP Application

The PHP application rule group contains rules that block request patterns associated with the exploitation of vulnerabilities specific to the use of the PHP programming language, including injection of unsafe PHP functions. This can help prevent exploitation of vulnerabilities that permit an attacker to remotely run code or commands for which they are not authorized.

Admin protection

The Admin protection rule group contains rules that allow you to block external access to exposed administrative pages.

An important concept to understand is Web ACL rule capacity units (WCU), for each rule group you enable, capacity units are accumulated and should not exceed 1500 WCU’s, this is a calculation done by AWS to control the operating resources required to run your rules, rule groups and Web ACLs.

The estimated cost for creating 1 Web ACL is $ 5.00 per month (prorated hourly). Since we will be only using free rule groups, you will be charged for duration you have a WAF created and a Web ACL in use.

Let’s get started with creating our first AWS WAF with managed rules and attach it to an application load balancer, later on we’ll even restrict access to the WordPress admin area to be only accessible to an IP address that we specify.

Setting up an AWS WAF, Creating a Web ACL & Adding AWS Managed Rule groups

Sign in to the AWS Management Console and navigate towards to the AWS WAF console.
Select Create web ACL.
For the Name field, enter a name for the Web ACL and select the resource type you wish to associate this Web ACL with.
Select Add AWS resources, select an existing resource type (CloudFront / Application Load Balancer etc.) and select Next.
On the Add rules and rule groups page, select the Add rules drop-down from the Rules section.
Select Add managed rule groups, on the Add managed rule groups page, expand AWS managed rule groups.
Under the Free rule groups section, switch the toggle to on for the below rule groups:

Admin protection
PHP Application
SQL database
WordPress application
Select Add rules. On the Add rules and rule groups page, select Next.
On the Set rule priority page, you can move the rules up and down to change the evaluation order.
On the Configure metrics page, ensure that the rules are checked and the Enable sampled requests option is selected.
On the Review and create web ACL page, review the settings you have chosen and then select Create web ACL.

Creating an IP Set

In order to restrict access to the WordPress admin area to be only accessible to an IP address of your choice, you”ll need to first create an IP Set.

Navigate towards IP Sets and select Create IP Set.
Provide a name, choose the AWS region where your other AWS resources have been deployed and provide an IP address.

Restricting access to WordPress Admin area

Navigate towards Web ACLs and select the Web ACL created earlier.
Select the Rules tab, select the AWS-AWSManagedRulesAdminProtectionRuleSet **rule and select **Edit.
Select the Override to Block option from the drop-down in the Admin protection Rules section.
In the Scope-down statement section, ensure the checkbox for Enable scope-down statement is selected.
From the If a request drop-down, select the doesn’t match the statement(NOT) option, from the Inspect drop-down select Originates from an IP address in, from the IP Set drop-down select IP the set that you created earlier and select the Source IP address radio button.
Select Save rule.

After editing the AWS-AWSManagedRulesAdminProtectionRuleSet rule, if you attempt to access the WordPress admin area from an IP that’s not listed in the IP Set, you will be greeted with a 403 Forbidden response.

To test it out, you can attempt to reach your WordPress admin area by using a connection through a VPN or via a mobile device or tablet using cellular data.

That concludes this blog post on how you can better protect your WordPress site using AWS WAF, Web ACLs, IP Sets and AWS Managed Rules.

Connect to your Linux EC2 instance using SSH + Visual Studio Code

Adrian Mudzwiti — Mon, 12 Sep 2022 05:58:02 +0000

In this blog post i'll show you how to configure VS Code to allow remote development for Linux based EC2 instances hosted on AWS.

On the rare occasion that I need to connect to Linux based EC2 instances, I normally use Session Manager through the AWS Management Console or use the Session Manager plugin for AWS CLI.

Recently I had to take part in a web development project, the remote development environment ran on an EC2 instance. I had no desire of using the VI text editor (that's too hardcore) and preferred to use VS Code from my MacBook Pro as it has some pretty useful extensions installed.

Pre-requisites:

• AWS Account
• Visual Studio Code

This blog post assumes the reader has some basic knowledge of SSH and is using macOS or a Linux based OS.

To get started you'll need an SSH Config file, on macOS it can be located in the ~/.ssh directory. I've created a new EC2 instance, new key-pair and moved the private key from the Downloads directory to the ~/.ssh directory.

To modify the config file, use the below commands:

cd ~/.ssh
code config

Let's add a new section containing our EC2 details. The section must conform to the below structure:

Host {Friendly Name Used for identification}
    HostName {Public DNS of EC2 instance}
    User {username}
    IdentityFile {location of private key}

Next you'll need to install the Remote - SSH Extension in VS Code.

Once installed, you'll notice the Remote Explorer icon appear on the Activity Bar, select the icon, this will bring the Primary Side Bar into view.

From the dropdown, select SSH Targets, a list of SSH Targets will be listed.

Right-click the SSH target, you'll be presented with 2 options: Connect to Host in Current Window and Connect to Host in New Window.

You will then be prompted to verify the fingerprint. Select Continue.

On the bottom left corner of VS Code, the Status bar will indicate the connection status.

That concludes this blog post where I showed how to configure VS Code for Remote Development for Linux based EC2 instances hosted on AWS.

Deploy a WordPress Amazon Lightsail instance using Terraform

Adrian Mudzwiti — Sun, 15 May 2022 18:30:51 +0000

Lately I've been learning about Terraform and getting into the habit of deploying infrastructure as code across different cloud service providers, it’s been a while since I found a use case to provision infrastructure on AWS.

For inspiration behind this blog post I took a trip down memory lane circa late 2021, when I was going through a WordPress learning phase I initially used Amazon Lightsail for WordPress hosting, for some reason I used to click through the GUI to provision an Amazon Lightsail instance.

I had previously looked at documentation around using CloudFormation and never bothered to pursue that avenue for Lightsail. With that in the past, this new found confidence and understanding in using infrastructure as code and in particular with Terraform it was time to solve this mini challenge of yester year.

HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share.

If you’re new to Terraform it might be worthwhile to read the official documentation or watch an awesome intro video by Nana Janashia:

What is Terraform | Terraform by HashiCorp

Terraform explained in 15 mins | Terraform Tutorial for Beginners

Pre-requisites:

• AWS Account
• Named profile configured
• Terraform must be installed
• Visual Studio Code
• AWS Toolkit extension for VS Code

All the code in this blog can be found in the repo:

Amazon-Lightsail-Terraform

Once you have cloned the repo, you'll need to run terraform init, this command is used to initialize the working directory that contains our Terraform configuration files.

I have included a variables.tf file to move away from as many hard-coded values in the main.tf file as possible.

This will allow versatility and makes the template reusable for different types of Lightsail blueprints eg. LAMP, Node.js, Joomla, GitLab etc

Official documentation from AWS on blueprints can be found here.

In order to find the available blueprint IDs, type the below command in the AWS CLI:

aws lightsail get-blueprints

When you have entered your desired inputs in the variables.tf file, you can apply your configuration using terraform apply and the Amazon Lightsail instance will be provisioned.

Useful resources:

Resource: aws_lightsail_instance

Regions and Availability Zones in Amazon Lightsail

Encrypting Attached EBS Volumes

Adrian Mudzwiti — Tue, 12 Apr 2022 19:41:23 +0000

Time and time again whenever I'm reviewing an AWS environment I've always noticed that many customers neglect enabling encryption features for data at rest for EC2 volumes and snapshots, RDS and even S3 buckets and their objects.

As with most cloud deployments it's fairly easy to provision resources and run them for quite some time until you realize that a flag or toggle that could improve your security posture should have been selected at the time of provisioning.

Once a resource has been provisioned without enabling encryption at rest, the task of enabling encryption becomes a rather serious challenge.

In this blog post I'll show you how to encrypt EBS volumes that are attached to EC2 instances. This is a fairly manual task that has to be done for each EBS attached volume, plan for downtime... It will take some time.

AWS has a feature that needs to be configured per region to enable EBS encryption by default. This can be found in the Account attributes tile under the EC2 Dashboard page. Now would be the perfect time to enable this feature for future deployments.

Back to the task at hand, encrypting an EBS volume that is attached to a running EC2 instance has a few steps.

Remediation:

Step 1: Navigate towards the EC2 console and select Volumes under the Elastic Block Store section.
Step 2: Select an unencrypted volume and then select the Actions button and from the dropdown select Create snapshot.
Step 3: Navigate towards Snapshots on the left-hand side under the Elastic Block Store section.
Step 4: Select the newly created snapshot, select the Actions button then select Copy snapshot.
Step 5: Under encryption, select the checkbox next to Encrypt this snapshot and proceed to select the Copy snapshot button at the bottom of the screen.

Step 6: Create a volume from the copied snapshot by selecting the snapshot then proceed to select the Actions button then select Create volume from snapshot. From the Create volume page, ensure to select the same Availability Zone where your EC2 instance was originally deployed in (Open another tab and verify the region), scroll to bottom of the page and select Encrypt this snapshot.
Step 7: Navigate towards Volumes, you'll see the newly created encrypted volume in an Available state.
Step 8: Navigate back towards the EC2 Dashboard and select Instances under the Resources tile, select the instance that you intend to detach the unencrypted EBS volume and attach the newly encrypted volume, select the Storage tab and copy the Root device name to your clipboard.

Step 9: Stop the running instance, then navigate towards Volumes under the Elastic Block Store section.

Step 10: Select the unencrypted volume, then select the Actions button and from the dropdown select Detach volume.
Step 11: Now select the encrypted volume then select the Actions button and from the dropdown select Attach volume, from the attach volume page, select your instance and in the Device name textbox paste the Root device name from step 8 and select the Attach volume button at the bottom of the screen.

Step 12: The final step is to start your instance.

In 12 steps I've shown you how to encrypt an EBS volume that is attached to an EC2 instance, If you have a couple of EBS volumes this shouldn't take long, just make a note of Root device name.

I'm having flashbacks of a customer that had over a 100 unencrypted volumes being used in production. That brings the conclusion to this blog post.