Forem: Eliana Lam

Rethinking Serverless Application Workflows from a Testing Perspective

Eliana Lam — Thu, 27 Nov 2025 05:22:39 +0000

Speaker: Takumi Abe @ AWS Community Day Hong Kong 2025

Phase 1: Requirements:

Architecture Overview:

Batch processing workflow using AWS Lambda and Step Functions.
Three states in a straight line.
Unit tests for each Lambda function, but manual testing for Step Functions workflow.

Testing Goals:

Automate Step Functions workflow testing.
Ensure input and output values match expected values.
Integrate tests into CI/CD pipeline using GitHub Actions.
Avoid deploying to a real AWS environment for testing.

Phase 2: Plan:

Tools and Technologies:

Step Functions Local: For running and testing Step Functions locally.
GitHub Actions: For CI/CD workflow automation.
Localstack or similar: To simulate AWS Lambda locally.

GitHub Actions Workflow:

Set up workflow to run tests on push or pull requests.
Use Step Functions Local Docker image for local testing environment.
Use Localstack for Lambda simulation.
https://www.localstack.cloud/
Import Step Functions definition and run workflow locally.
Verify input and output values of each state.

Steps in the Workflow:

[ 1 ] Start Containers:
Pull and start Step Functions Local Docker image.
Set up Localstack for Lambda simulation.
[ 2 ] Import Step Functions Definition:
Import state machine definition into local environment.
[ 3 ] Run Workflow:
Trigger Step Functions workflow using local endpoint.
[ 4 ] Validate Results:
Check input and output values against expected values.
[ 5 ] Report Results:
Generate a report or status indicating test success or failure.

How to Test Serverless Architecture?

Introduction:

Serverless architecture offers significant benefits, enabling quick delivery of value to customers.
The focus here is on testing from a testing perspective, rather than system performance or cost.

Why Do We Need Testing?

Testing helps us understand, predict, and control system behavior.
It allows us to identify and quantify system risks.
Writing tests helps us better understand requirements.
Testing ensures reproducibility and explainability of results.
The primary goal of testing is to reduce unknown risks.

Types of Tests:

Unit Tests: Test individual components or functions in isolation.
Integration Tests: Test the interaction between combined components.
Logic Tests: Verify the correctness of business logic.
Workflow Tests: Test the end-to-end flow of a process or workflow.
API Tests: Test the functionality and reliability of APIs.

Recommendation:

It is important to be aware of different test types and their purposes and goals.
Understanding these test types will help in designing effective testing strategies for serverless architectures.

Separate Processing for Easier Testing:

Key Points:

Easy-to-Test Units: Units that are easy to test are also easy to understand.
System Clarity: When a system is designed with separate, testable units, it naturally shows what it does, when it does it, and why it does it.
Avoiding Complexity: If something is hard to test, it likely means there is too much responsibility and logic concentrated in one place.

Example Comparison:

Less Architecture: A single Lambda function handles both getting data from other servers and processing/storing it in DynamoDB.
Right Architecture: Separate Lambda functions for each operation (getting data and processing/storing data).

Benefits of Separation:

Easier to test each part individually.
When an error occurs, it is easier to pinpoint the exact location of the problem.
Reduces the time from error occurrence to error identification and fix.

General Principle:

Dividing Methods: The same principle of dividing methods applies across different programming languages (e.g., Python, Ruby, Go).
Small Units: Identify what you consider to be small, testable units and apply this division principle accordingly.

Testing Perspective in System Design:

Identifying Weak Points: From a testing perspective, it is easy to identify weak points, such as too much responsibility in one place.
Reducing Error Detection Time: High testability helps reduce the time from error occurrence to error detection and fix, similar to reducing the development cycle.
Quick Delivery: High testability contributes to quicker system delivery.

Key Points to Consider When Building Serverless Architecture from a Testing Perspective:

Separate Processing: Easier to test units are easier to understand. Separating processes helps identify weak points and reduces error detection time.
Avoiding Complexity: Concentrated responsibility and logic in one place make testing difficult. Separate processes to simplify testing and troubleshooting.
High Testability: Contributes to quicker system delivery and reduced development cycles.

Final Goal:

The ultimate goal for engineers is to deliver value to users.
Always keep user value in mind to ensure successful outcomes.
Encouraged discussion and collaboration to share insights and improve testing practices.

Practical AWS FinOps for Cloud Success

Eliana Lam — Thu, 27 Nov 2025 05:20:42 +0000

Speaker: James Freeman @ AWS Community Day Hong Kong 2025

On-Premises vs. Cloud Technology Consumption:

On-premises: Procurement cycles, fixed spend, use until hardware fails or contract ends.
Cloud: Pay-as-you-go model, potential for bill shock, cost increases with usage.
Cloud offers low cost of failure, instant procurement, and real-time cost data access.

FinOps Introduction:

FinOps is akin to DevOps, amalgamating finance, technical operations, and technical aspects.
FinOps requires cross-business unit sponsorship and buy-in from executive, finance, procurement, engineering, and operations teams.

Key Considerations for FinOps:

Education is crucial for adopting the FinOps mindset, especially for those transitioning from on-premises models.
Clear accountability and ownership for cloud resource usage and billing are essential.
Emphasizes the need for widespread organizational buy-in for smooth FinOps implementation.

Risk of Losing Control Over Spend:

Without proper FinOps practices, organizations risk losing control over their cloud spending.

Cost Optimization on AWS:

Starting resources on AWS is simple, but optimizing for cost requires different considerations.

Importance of Cost Tracking:

Cost tracking is vital for effective FinOps.
AWS uses a "see, save, run" cycle with customers for cost management.

Problem: Visibility for Costs:

Without visibility into spending and the ability to break down costs meaningfully, meaningful cost management is difficult.

Education and Reviews:

Education about FinOps is key.
Regular reviews are crucial, and AWS TAMs are responsible for organizing these with customers.
Reviews help identify spend patterns and suggest cost-saving options.

Concept of Ownership:

AWS provides constructs to help define ownership and cost accountability.
Using a single AWS account with a corporate credit card for all users is discouraged.

AWS Organizations:

AWS Organizations is a powerful feature for billing, cost management, and security.
It helps in setting up guardrails and managing what users can and can’t do.
It is valuable for new AWS users and has minimal associated cost.

Organizational Units (OUs) and Linked Accounts:

Within AWS Organizations, OUs can be set up similar to directory structures.
Linked accounts can be created for departments or projects to segregate costs and simplify accounting.

Problem: Granular Level of Resources:

At the most detailed level, resources like EC2 instances, Lambdas, and VPCs can become difficult to track.
Organizations may have tens of thousands of these resources, complicating cost management.

Subdivision Within Constructs:

Even with constructs like Organizations, OUs, and linked accounts, further subdivision is often needed.
Cost allocation tagging is used for this purpose.

Cost Allocation Tagging:

Tagging is crucial for cost management, with examples provided (e.g., project, customer, accounts).
There is no right or wrong way to tag, as long as it works for the business.
Emphasis on creating a tagging dictionary to avoid confusion and ensure consistency.
Example: Differentiating between "Project," "project," "project-name," and "project_name" due to case sensitivity and varying formats.

Tagging Dictionary:

A tagging dictionary is simple but vital to prevent mix-ups and ensure clear cost allocation.
Helps decipher spending by project or cost center.

Cost Visibility:

At the end of the month, AWS sends a bill with a total number, which is a starting point for cost visibility.

Ensuring Cost Control:

Importance of oversight to determine if spending is appropriate and to identify potential shadow IT projects.
Questions to ask and monitor regarding AWS spend.

AWS Cost Explorer:

A free tool similar to Excel graphing, allowing detailed filtering and data diving.
Highly recommended for its power and ease of use.

AWS Cost and Usage Reports:

For more detailed data than Cost Explorer, with a small cost associated for storing data in S3 and using tools like Athena.
Useful for complex setups and specific internal accounting metrics (e.g., vCPU hours).

Cloud Intelligence Dashboards:

Free, open-source dashboards that cut data in various ways and present it digestibly.
Allow breakdown of instance, storage, and networking spend.
Available on GitHub for deployment, with cost mainly being the deployment in Quicksight.
Dashboards range from high-level (suitable for executives and finance) to granular (for fin practitioners and engineering).
Value depends on the user's role within the business.
https://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/dashboards.html

Awareness and Usage:

The goal is to raise awareness of these tools and their suitability for different business levels.
Significant engineering effort has been put into creating these dashboards for appropriate use cases.

Reducing AWS Bills and Saving Money:

The presentation covers a runbook for cost-saving strategies, focusing on cost savings impact (y-axis) and technical complexity (x-axis).

Simple Cost-Saving Measures:

Commitments: Using instance savings plans and reserved instances for resources running 365 days a year can provide significant discounts on on-demand rates.
Elastic Workloads: Shutting down resources during periods of low usage (e.g., weekends) can lead to immediate cost savings.
Identifying Underutilized Resources: Tools like Cost Explorer and Cloud Intelligence Dashboards help identify and delete idle resources.

Moderate Complexity Measures:

Serverless Architecture: Moving to serverless platforms (e.g., RDS) can reduce costs by paying only for compute time needed.
Graviton Instances: Transitioning from x86 to ARM 64 architectures (Graviton) offers significant cost savings with no impact on software.
EBS Volumes: Upgrading from GP2 to GP3 EBS volumes can save money.

High Complexity Measures:

Right Sizing: Optimizing instance types and resources to match actual usage.
Cloud-Native Architecture: Moving away from static resources (e.g., EC2, RDS) to fully serverless, pay-per-use models.
Long-Term Strategy: Adopting cloud-native architectures is the most powerful lever for long-term cost savings, though it requires significant engineering and development effort.

Unit Cost as a Metric:

Measuring unit cost (e.g., cost per transaction) rather than overall spend.
Decreasing unit cost over time as a sign of efficiency.

Key Performance Indicators (KPIs):

Suggest picking three KPIs tailored to business requirements.
Examples: unit cost, customer satisfaction, and efficiency metrics.
Importance of defining and understanding unit cost for optimization.

Feedback Loops:

Continuous feedback loops for ongoing optimization.
Feedback should be a regular, iterative process, not a one-time event.

Prioritization and Communication:

Businesses should determine what to prioritize based on customer satisfaction and efficiency.
Effective communication is crucial for complete feedback loops.
Ensure open conversations between finance, procurement, and engineering teams.
Concerns and insights should be communicated and integrated into the roadmap.
Incomplete loops (lack of communication) indicate problems in the process.

Reviewing with Cloud Intelligence Dashboards:

Use dashboards for customer reviews, tailoring data to stakeholders’ needs.
C-level executives require top-level data and KPIs, not detailed instance breakdowns.
Consider the audience and format when communicating data.

Row-Level Security:

Out-of-the-box dashboards provide universal access; use row-level security for granular control.
Restrict access to relevant data for specific departments to maintain security and relevance.

Education on Cost Management Tools:

Emphasize the importance of educating teams responsible for cost, especially engineers.
Tools like Cost Explorer, budget settings, commitment recommendations, anomaly detection, and right-sizing recommendations are available.
Anomaly detection and right-sizing recommendations help optimize resource usage.
Cost and Usage Reports provide detailed data, though they incur a small cost.

Additional AWS Tools for Cost Optimization:

Compute Optimizer: A free tool worth exploring to further optimize EC2 instance costs.
Trusted Advisor: Provides numerous recommendations, not just for cost optimization but also for security.
AWS Config: Often needed by customers for various purposes.
CloudWatch: Critical for gathering metrics within EC2 instances, as AWS cannot see into customer instances by default.
S3 Lens: A powerful tool for visualizing and optimizing the use of S3 object storage.

Shared Responsibility Model:

AWS is responsible for the integrity of the cloud, while customers are responsible for what they run on the cloud.

Cost of Chargeable Services:

Tools like CloudWatch are essential for customers to gather metrics and optimize their resources, as AWS cannot access this data directly.
Even chargeable services like Cost and Usage Reports have negligible costs, primarily due to S3 data storage.
These tools are valuable for detailed cost optimization and should be considered despite the small associated costs.

Disaster and Emergency Response with TAK and AWS

Eliana Lam — Thu, 27 Nov 2025 04:50:32 +0000

Speaker: Uriel Alonso @ AWS Community Day Hong Kong 2025

Disaster Exposure in the Philippines:

The Philippines is one of the most disaster-prone countries, ranked as such for 16 consecutive years.
Experiences an average of 20 tropical storms annually, along with floods, earthquakes, landslides, and volcanic events.
These hazards overwhelm first responders due to scale, frequency, mass casualties, and communication challenges.

Global Challenge:

Disasters are not limited to the Philippines; they are a global issue affecting many countries.

Solution: Reliable Communication During Disasters:

Real-time situational maps.
Drone feeds to locate victims, hazards, and damage.
Disconnected operations for remote or offline areas.

Introduction to Team Awareness Kit (TAC):

TAC, also known as Team Awareness Kit, is a geospatial visualization and collaboration tool.
Designed for real-time situational awareness in critical environments.
Enables secure data sharing, mapping, and communication across diverse teams and devices.

Core Capabilities of TAC:

Real-time mapping and geolocation.
Secure messaging and data overlays for maps.
Integration with sensors, drones, and IoT devices.
Open-source and customizable via plugins.

Platform Compatibility:

TAC can be used on iOS, Android, Linux, and Windows.

Why TAC?

Successfully deployed in disaster response operations globally.
Used in Kenya, Peru, Somalia, Gaza, Mexico, Poland, Brazil, and the Philippines.
Field-tested in the 2017 Marawi crisis under extreme conditions.
Scalable and secure with AWS.

Detailed Breakdown of Disaster Response Architecture:

Main Services Used for TAC Server:

EC2: Deploys the TAC server.
S3: Stores assets and user data.
DynamoDB: Manages user credentials.
Lambda: Automates the sending of user credentials and facilitates clear, instant messaging.

Communication Features:

Text and Alerts: TAC can send text messages and alerts to all responders.
AWS Wickr Integration: Provides secure communication.
One-on-One Chat: Enables private conversations.
Team-Specific Updates: Shares updates only with relevant teams.
Group/All Chats: Keeps everyone informed about the disaster situation.

IoT Integration:

T100A IoT Trackers: Connected via LoRaWAN gateways to deliver encrypted GPS and sensor data directly into TAC Map.
Secure Location Sharing: Sends location data to TAC Map for all connected users.
LoRaWAN (Long Range Wide Area Network) is an open wireless communication protocol for the Internet of Things (IoT) that connects low-power devices over long distances. It is built on LoRa (Long Range) technology, which uses a specific modulation technique for long-range, low-bandwidth, and low-power communication. This makes LoRaWAN ideal for applications like smart cities, smart agriculture, and industrial monitoring, where devices need to send small amounts of data with long battery life.

Domain Services for IoT Integration:

IoT Core: Fetches data from trackers (sensor data, sensor ID, coordinates).
Lambda: Decodes the data.
EC2: Deploys Node-RED, which sends location data to TAC devices/clients.
Node-RED: Low-code programming for event-driven applications
Node-RED's goal is to enable anyone to build applications that collect, transform and visualize their data; building flows that can automate their world.

https://nodered.org/

Drone Integration:

Live Aerial Views: Streams live drone footage to responders.
Hazard and Victim Detection: Helps locate victims, assess damage, and improve search and rescue speed.

CCTV Streaming Feature:

Live Ground-Level Monitoring: Provides continuous monitoring of key areas.
Tracks People and Vehicles: Enhances site security and operational awareness.

Live Streaming Services:

EC2: Installs the streamer for live features.

Additional Details on Live Streaming and Disconnected Operations:

Live Streaming from CCTV to IVS:

GStreamer: Used to push live streams from CCTV cameras to IVS.
https://gstreamer.freedesktop.org/
IVS: Handles live streaming. (Amazon Interactive Video Service)
https://aws.amazon.com/ivs/
S3: Used for auto-recording and auto-saving of live streams.
Kinesis Video Streams (KVS): Live streams from cameras are also pushed to KVS.
Object Detection: Recognition service is used for object detection, connected to both S3 and Kinesis Video Streams.

Operations in Disconnected Areas:AWS Snowball Edge

Used for offline TAC operations.
Allows TAC to run in environments with no internet.
Ideal for remote areas or disaster-hit zones without electricity or internet.
Supports mapping, messaging, and location sharing features.

Benefits:

Faster rescues and better coordination.
Helps save more victims.
Operates in degraded or disconnected environments.
No more outdated information.
Can be deployed for small incidents or nationwide responses.

"Disasters, whether man-made or natural, are inevitable. The Philippines is battered by calamities every year. Our capability to communicate during calamities can significantly impact those responding and the affected community. While we can't prevent calamities, we can improve our communication and coordination to help first responders save more lives during disasters."