Forem: Leszek Ucinski

Finding Your Fit: Crafting Diverse Development Environments on AWS Lightsail

Leszek Ucinski — Fri, 03 Nov 2023 13:02:53 +0000

💡Exploring new development environments? AWS Lightsail offers cost-effective access to Windows and Linux operating systems. While primarily focusing on Windows connecting to Linux, this guide offers pathways for all. Get hands-on and explore a range of development opportunities affordably and flexibly. This guide helps you start, connect, and set up an alternative environment that suits your needs.

📋Table of Contents

Introduction
Working in a Windows Environment: Exploring WSL and Its Limitations
Building Your Own Linux Development Environment
- Prerequisites
- Setting up an AWS Lightsail Instance
- Connecting to Your Lightsail Instance with VS Code
- Configuring the Development Environment
Understanding Costs: AWS Lightsail Billing
Final Thoughts and Considerations

Introduction

My everyday machine is Windows, serving me well for most tasks. However, I found it limiting for code development and AWS interactions. Navigating through PowerShell and the absence of essential CLI tools became obstacles (could you live without these cli tools?). This isn't about pitting one OS against another, but about enabling a smoother development journey. Here, I'll share a straightforward method to establish a Linux-based development environment.

☁️ Utilizing AWS credits, I explored a cost-saving solution. This guide showcases how to set up a Linux environment on AWS Lightsail, offering an alternative OS for developers seeking flexibility. While I focus on enabling Linux for Windows users, Lightsail provides various OS types for diverse developer needs.

❗This solution will incur monthly charges that you have to be aware of (see the Understanding Costs: AWS Lightsail Billing).

Working in a Windows Environment: Exploring WSL and Its Limitations

While an apparent and straightforward alternative is using WSL on Windows (see here for setup instructions), my experience in Python and AWS SAM development revealed potential drawbacks. WSL often becomes unresponsive due to memory management inefficiencies. The memory allocation within WSL, managed by the Windows operating system, can lead to performance degradation and unresponsiveness, particularly evident when handling resource-intensive tasks in development environments. Despite my mighty 32GB RAM machine, every time I unleash sam local start-api with a medley of Lambda function fiestas, my WSL slowly morphs into the 'Snail Subsystem for Linux.' It huffs, puffs, and eventually naps, leaving me in the lurch. 😊

Building Your Own Linux Development Environment

⚓ Prerequisites

An AWS Account
VS Code on your Windows machine

💻 Setting up an AWS Lightsail Instance

Open Lightsail Console. And choose Create instance
Choose the OS Only and your preferred Linux Distribution.
Use either the Default key or create/upload your own. Download the key.
Choose the instance plan. I wouldn't the smallest one, since it has only 512 MB of RAM. Go with a bit larger one. Give it a name, tags, and launch. After a couple of minutes, it'll be in the Running state.💤

Time for some additional setup. Choose your instance and go to the Networking tab. I would recommend attaching a static IP so that you can easily reconnect to your instance. You can remove the HTTP rule from the IPv4 Firewall, we'll be connecting through SSH. Configure the SSH rule to allow connections only from your IP (you can keep the Allow Lightsail browser SSH option enabled so that you can easily connect to your instance through the AWS Console). Check your IP and replace it as Source IP: You might want to disable the IPv6 networking if you're not planning on using it.

Connecting to Your Lightsail Instance with VS Code

Store locally the SSH Key from the Lightsail console. Remember the path to the .pem file.
Start Visual Studio Code.
Install the Remote Explorer extension through the Extensions menu.
You should see the Open a Remote Window icon on the left bottom of VS Code. Click on it and then choose Connect to Host. And then Add New Host. Give it a name like ubuntu@<your.lightsail.static.ip>. Select SSH configuration file or create it (on Windows it's usually stored at C:\Users\{USERNAME}\.ssh\config. The config file will open and then specify the path to the SSH key or rename the host if you want:
Press F1 or Ctrl+Shift+P to open the command palette in VS Code. Type and select "Remote-SSH: Connect to Host...". Choose your Host. Choose Linux. Accept the fingerprint. 🏁 And voila! You're connected!

Using the left menu bar you can choose Explorer -> Open Folder to browse the instances file system:

🚧 Configuring the Development Environment

Access the Terminal within Visual Studio Code: Once connected to your Lightsail instance via SSH in VS Code, go to the top menu and select Terminal > New Terminal. This action will open a new terminal window within VS Code, connected to your Lightsail instance.
Start off with updating and upgrading the packages (below commands for Ubuntu):

sudo apt update

sudo apt upgrade

Depending on your needs you might want to download additional packages. Ubuntu image comes with Python 3.10 installed.
Let's install AWS CLI Install unzip

sudo apt install unzip

Download the AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

Unzip the downloaded package:

unzip awscliv2.zip

Run the installation script:

sudo ./aws/install

Verify the installation:

aws --version

Understanding Costs: AWS Lightsail Billing

💵 One important aspect to note is that charges apply not only to actively running instances but also to stopped instances. Whether your Lightsail instance is actively running or stopped, you'll incur charges. According to Lightsail's documentation:

Q: When am I getting charged for a plan?
Your Lightsail instances are charged only when they're in the running or stopped state. If you delete your Lightsail instance before the end of the month, we only charge you a prorated cost, based on the total number of hours that you used your Lightsail instance.

AWS bills the instances by the hour, but advertises the monthly price by capping it for a whole month. For example, if you use a $5/1GB per month plan for 5 days, you will be charged 120 hours (5 days x 24 hours) at an hourly rate of $0.00672 (10 USD / 720 hours)(depending on the AWS Region). This amounts to $0.81 for 5 days of usage. More details on additional costs can be found here.
⭐ But remember that the first three months are free! ⭐

Final Thoughts and Considerations

📜 Cloud-based development environments offer enhanced productivity and flexibility. You can connect to your Linux instance on Lightsail from any machine supporting SSH and have a separate environment fitting your needs. Since there are many OS options to choose from you can start a Windows machine if that's what you need. While Windows Subsystem for Linux (WSL) is free, AWS Lightsail's Linux environment incurs additional costs. The crucial consideration is enabling both Linux users to initiate Windows and vice versa, balancing the limitations of local setups against the incurred expenses of cloud alternatives.

Cover image generated with AI. Content credentials: Generated with AI ∙ November 3, 2023 at 11:57 AM.

Event-driven file management using S3 Notifications and Step Functions

Leszek Ucinski — Mon, 23 Jan 2023 10:55:41 +0000

If our looking for a solution to handle file management of S3 objects, but listing a Bucket doesn't work for you, maybe this approach will.

Problem and solution:

You want to log S3 objects metadata and expose it through an API backed with superfast DynamoDB. I've chosen to write this simple example without even using any Lambda functions. We'll be using the build in StepFunctions integration with DynamoDB. So no code at all.

In this first step of building that kind of solution I will describe how to log newly uploaded files in the DynamoDB table in an event-driven fashion. In the next step, I’ll describe how to expose that data through a Lambda-backed API.

What will we build:

An event-driven architecture deployed with AWS SAM.

Resources used (might incur costs):

Serverless StateMachine
EventBridge Rule
S3 Bucket
DynamoDB table

Requirements:
An AWS Account obviously
AWS SAM CLI installed and configured

Let’s start! You can either run the console command to initiate SAM App using one of the Quick Start Templates or choose the Custom Template that we will write here:

sam init

I’ve chosen the first option and provided Python as my runtime even though I won't need any code in this example. Then just clear the template.yaml and remove the existing lambda code or else get started with a clean slave.

We’ll start with deploying our Bucket and set up Notifications to be sent to EventBridge:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Event-driven file management

Resources:
  FilesBucket:
    Type: AWS::S3::Bucket
    Properties:
      NotificationConfiguration:
        EventBridgeConfiguration:
          EventBridgeEnabled: true

Step into your App folder and run:

sam deploy --guided

Now that we have our bucket and deployed with EventBridge configuration all S3 event notifications will be sent to the default Event Bridge bus.

In the next step, we will deploy a DynamoDB table. We’ll create two string attributes for now.

FilesTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: 'fileName'
          AttributeType: S
        - AttributeName: 'lastModified'
          AttributeType: S
      KeySchema:
        - AttributeName: fileName
          KeyType: HASH
        - AttributeName: lastModified
          KeyType: RANGE

Now let’s write the definition of our StateMachine in ASL (Amazon States Language) that we’ll be deploying when creating the StateMachine resource.

{
  "Comment": "Triggered on EventBridge S3 Object Created notification",
  "StartAt": "Query",
  "States": {
    "Query": {
      "Type": "Task",
      "Next": "ObjectExists",
      "Parameters": {
        "TableName": <<TABLE_NAME>>,
        "KeyConditionExpression": "fileName = :fileName",
        "ExpressionAttributeValues": {
          ":fileName": {
            "S.$": "$.detail.object.key"
          }
        }
      },
      "Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
      "ResultPath": "$.dynamoDbResult"
    },
    "ObjectExists": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.dynamoDbResult.Items",
          "NumericGreaterThan": 0,
          "Next": "UpdateObject"
        }
      ],
      "Default": "PutObject"
    },
    "UpdateObject": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:updateItem",
      "Parameters": {
        "TableName": <<TABLE_NAME>>,
        "Key": {
          "fileName": {
            "S": "$.detail.object.key"
          }
        },
        "UpdateExpression": "set lastModified : lastModified",
        "ExpressionAttributeValues": {
          ":lastModified": {
            "S": "$.time"
          }
        }
      },
      "Next": "ObjectUpdated"
    },
    "PutObject": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Parameters": {
        "TableName": <<TABLE_NAME>>,
        "Item": {
          "fileName": {
            "S": "$.detail.object.key"
          },
          "lastModified": {
            "S": "$.time"
          }
        }
      },
      "Next": "ObjectAdded"
    },
    "ObjectUpdated": {
      "Type": "Succeed"
    },
    "ObjectAdded": {
      "Type": "Succeed"
    }
  }
}

The visual workflow would look like that:

The StateMachine would be triggered on S3 Object Created notification. In the ‘Query’ step will check if the file already exists in the DynamoDB table. Then will do the ‘ObjectExists’ Choice operation on the output from the query step. If there’s no Attribute with the newly created object the ‘PutObject’ operation will run. If there’s one will update the ‘lastModified’ attribute will the event time.

Now let’s create our StateMachine Resource in our SAM template and specify the definition parameter providing a path to the above JSON file (mine is at ‘definitions/object_created.asl.json’):

ObjectCreatedStateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: definitions/object_created.asl.json
      Policies:
        - CloudWatchLogsFullAccess
        - DynamoDBCrudPolicy:
            TableName: !Ref FilesTable
      Type: STANDARD
      DefinitionSubstitutions:
        TableName: !Ref FilesTable
      Events:
        EBPutRule:
          Type: EventBridgeRule
          Properties:
            Pattern:
              source:
                - aws.s3
              detail-type:
                - Object Created
              detail:
                bucket:
                  name:
                    - !Ref FilesBucket

The above snippet creates the StateMachine for us, an IAM Role with permissions to perform CRUD operations on DynamoDB Table. It even creates EventBridge Rule with patern that will trigger StateMachine on S3 Object Created events in the FilesBucket.

This is just a beginning of a potentially powerful event-driven architecture. Thanks to the power of EventBridge and the flexibility of the Rules and patterns, one might filter the events granularly and route the events to corresponding processes.
This simple app would definitely need some error handling and maybe integration with Lambda functions to get more adjusted data out of events and into DynamoDB. In another part I will describe how one might integrate the data stored in DynamoDB table with serverless HTTP API.

Projects repo:

GitHub

Cloud Resume Challenge - AWS

Leszek Ucinski — Wed, 27 Oct 2021 20:52:55 +0000

Background

I stumbled upon the Cloud Resume Challenge while preparing myself for the AWS Certified Solution Architect exam. I already passed the Cloud Practitioner and was looking for some hands-on labs or challenges to get my hands cloudy...

I've come up with my own challenge 100 days in Cloud during which I go through one lab a day and describe how I solved it. But I needed a bigger project that would demand some more digging and sweating to accomplish, hence the CRC.

Challenge instructions:

✔️ Certification - got my AWS Certified Cloud Practitioner
✔️ HTML - check [my profile](https://myprofile.cloudofthings.link/
✔️ CSS - style.css
✔️ Static Website - AWS SAM template takes care of that
✔️ HTTPS - Amazon CloudFront takes care of that
✔️ DNS - Route 53 points to my profile page as part of cloudofthings.net
✔️ Javascript - there's some script in script.js and in index.html
✔️ Database - DynamoDB deployed using SAM
✔️ API - similarly here
✔️ Python - Lambda functions
✔️ Tests - Integration and End-to-end tests in GitHub actions
✔️ Infrastructure as Code - AWS SAM template
✔️ CI/CD - GitHub actions
✔️ Blog post - obviously... 😏

My challenge experience

I started with creating a sample HelloWorld application using SAM. This way I got Lambda with API Layer as a starting point for my infrastructure. Not everything went smoothly as one might have wished for so I'll focus mainly on my greatest challenges and solutions that thought me something new.

An example of an issue I got stuck on was when making the bucket accessible for the CloudFront distribution so I've decided to make my website bucket private and used CloudFront Origin Access Identity. It somehow felt more secure as well, so win-win situation even though it drove me crazy not being able to figure out what's blocking CloudFront access to the previously public bucket.

Creating the ACM Certificate and attaching it to my CloudFront. I live in Sweden and I thought it's only logical to use resources nearby and to find out if there are any services that I won't be able to use. And Certificates for CloudFront distribution happen to be an example of that. Since CloudFront uses certificates created only in the us-east-1 region I had to create one in AWS Console and use its ARN in the template. I suspect that there is a pure IaC-ish type of solution but this one works too.

The most giving lesson from the challenge was the CI/CD with GitHub Actions. What I revelation that was. Even though that I used Live Server extension for VSC - which I strongly recommend - I was struck by how easy it is to upload and deploy your code with Actions. When I set up my Workflow to use AWS SAM it builds and deploys changes in the infrastructure. Similarly for the website contents making the pushes would trigger the upload of the website content.

Working on the backend consisting of API Gateway, Lambda, DynamoDB was relatively easy since I've done it many times. Here I've learned that you can easily increment values in tables using update expression.

Another challenging part was to write a SAM template, getting the syntax right, using !Ref and Fn::Sub functions. But it was a great practice. Infrastructure as Code feels like the only way to design and deploy resources in AWS.

Summary

I got so much out of this Challenge! I needed that kind of project to get my head around all of the things I've been learning recently while preparing for the Solution Architect Associate exam and put my skills into action. I wrote two separate posts on my blog detailing the steps I took, so if you are curious have a look at the first and second one.

Additional notes

LesUski / AWS-Cloud-Resume-Challenge-SAM

AWS Cloud Resume Challenge using AWS SAM

Check my website if you’re looking for inspiration on AWS hands-on labs.