Forem: Nikitas Gargoulakis

Complete Guide to AWS Monitoring and Observability for DevOps Teams

Nikitas Gargoulakis — Thu, 22 Jan 2026 20:03:18 +0000

In today’s cloud-first world, many organisations find themselves wrestling with a common challenge: monitoring fragmentation. If you’re migrating to AWS from on-premises infrastructure, you’ve likely accumulated a collection of monitoring tools, Grafana here, Zabbix there, maybe some Prometheus, Scrutinizer, and a dash of CloudWatch. Each tool serves a purpose, but together they create operational chaos.

This article walks through a real-world architecture for consolidating multiple monitoring tools into a unified, AWS-native observability platform. Whether you’re monitoring EKS clusters, Active Directory, firewalls, or a hybrid infrastructure, this guide will help you build a single panel of glass for your entire estate.

The Problem: Death by a Thousand Dashboards

Let’s paint a familiar picture:

3 AM : Your phone rings. Production is down.
3:02 AM : You check CloudWatch. Nothing obvious.
3:05 AM : Switch to Grafana. Some weird metrics.
3:10 AM : Check Zabbix. Server CPU is spiking.
3:15 AM : But why? Check logs. wait, where are those logs again?
3:25 AM : Finally correlate the issue across four different systems.
MTTR : 45 minutes (30 of which were spent context-switching between tools)

Sound familiar? You’re not alone.

The Core Requirements

When consolidating monitoring infrastructure, we need to solve for:

Unified Visibility : One place to see everything
Proactive Detection : Catch issues before users do
Fast Root Cause Analysis : Correlate events across layers
Compliance Ready : Query data for audits without panic
Operational Efficiency : Stop paying for five tools when one will do

The Solution: AWS-Native Observability Stack

After extensive research and real-world implementation, here’s the architecture that actually works:


┌─────────────────────────────────────────────────────────┐
│ Visualization Layer │
│ CloudWatch Dashboards | Managed Grafana | QuickSight │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ Analytics &amp; Investigation Layer │
│ CloudWatch Insights | Athena | OpenSearch Service │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ Centralized Data Lake (Optional) │
│ AWS Security Lake (OCSF) │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ Monitoring &amp; Security Services │
│ CloudWatch | Security Hub | GuardDuty | Config │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ Your Infrastructure │
│ EKS | EC2 | Lambda | RDS | On-Prem (Logs Only) │
└─────────────────────────────────────────────────────────┘

The Core AWS Services

Let’s break down each component:

1. Amazon CloudWatch: Your Foundation

CloudWatch is unavoidable when working with AWS. Instead of fighting it, embrace it as your foundation.

What You Get:

Metrics : CPU, memory, disk, network, custom application metrics
Logs : Centralized log aggregation with retention policies
Alarms : Threshold-based and anomaly detection alerting
Dashboards : Pre-built and custom operational views
Insights : SQL-like queries for log analysis

Real-World Setup:


{
  "agent": {
    "metrics_collection_interval": 60
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/application/*.log",
            "log_group_name": "/aws/application/myapp",
            "log_stream_name": "{instance_id}",
            "retention_in_days": 30
          }
        ]
      }
    }
  },
  "metrics": {
    "namespace": "CustomApp/Metrics",
    "metrics_collected": {
      "cpu": {
        "measurement": [
          {"name": "cpu_usage_idle", "unit": "Percent"}
        ]
      },
      "mem": {
        "measurement": [
          {"name": "mem_used_percent", "unit": "Percent"}
        ]
      }
    }
  }
}

2. Container Insights for EKS

If you’re running Kubernetes on AWS, Container Insights is a game-changer.

Deployment:


# Enable control plane logging
aws eks update-cluster-config \
  --name my-cluster \
  --logging '{"clusterLogging":[{"types":["api","audit","authenticator"],"enabled":true}]}'

# Deploy FluentBit DaemonSet
kubectl apply -f https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/fluent-bit/fluent-bit.yaml

What You See:

Cluster-level metrics (CPU, memory, network)
Namespace and pod-level breakdowns
Node performance and capacity
Application logs automatically collected from stdout/stderr

This replaces your standalone Prometheus + Grafana setup for most use cases.

3. AWS Security Hub: Your Security Command Center

Think of Security Hub as your security findings aggregator. It’s like having a security operations assistant that never sleeps.

What It Aggregates:

GuardDuty : AI-powered threat detection
AWS Config : Configuration compliance
IAM Access Analyzer : Permission issues
Macie : Sensitive data discovery
Inspector : Vulnerability scanning

Compliance Made Easy:


# Enable Security Hub with CIS AWS Foundations Benchmark
aws securityhub enable-security-hub \
  --enable-default-standards

# Get compliance summary
aws securityhub get-findings \
  --filters '{"ComplianceStatus": [{"Value": "FAILED", "Comparison": "EQUALS"}]}'

4. Amazon OpenSearch: Your SIEM Replacement

Replacing Microsoft Sentinel? OpenSearch Service is your answer.

Why OpenSearch Over Sentinel?

Use OpenSearch’s anomaly detection feature. It’s surprisingly good at catching unusual patterns you’d miss manually.

5. AWS Security Lake: The Long-Term Play

Here’s where things get interesting. Security Lake is AWS’s answer to the question: “Where do I store petabytes of security data without going bankrupt?”

The OCSF Advantage

Security Lake automatically normalizes logs to the Open Cybersecurity Schema Framework (OCSF). This means:

Standardized queries across all log sources
Multi-cloud ready (Azure, GCP logs can be normalized too)
Future-proof (vendor-agnostic format)

When to Use Security Lake:

YES if you need:

1 year log retention
Compliance with strict audit requirements
Multi-cloud strategy
Cost-effective long-term storage (S3 is cheap!)

NO if you need:

Real-time alerting (use CloudWatch + OpenSearch instead)
Simple single-account setup
Quick implementation (<4 weeks)

Use Security Lake for retention, OpenSearch for hot analytics (last 30 days).

6. The On-Premises Challenge

Let’s address the elephant in the room: on-premises monitoring in a cloud-native world.

What’s Realistic:

You CAN:**

Forward logs via CloudWatch Agent
Send syslogs via Kinesis Firehose
Store and search on-prem logs in AWS
Create basic alerts on log patterns

You CANNOT (easily):**

Get real-time metrics dashboards
Automated remediation for on-prem resources
Full observability parity with AWS resources

The Pragmatic Approach:


# On-premises server → CloudWatch Logs
# Install agent
wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i amazon-cloudwatch-agent.deb

# Configure to send logs only (no metrics)
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config \
  -m onPremise \
  -s \
  -c file:/opt/aws/amazon-cloudwatch-agent/etc/config.json

For true on-premises monitoring, you might need to keep Zabbix or Prometheus for a while.

Architecture Decision: Two Approaches

Approach 1: With Security Lake (Compliance-First)

Best For : Healthcare, finance, government, or anyone with >1 year log retention requirements


AWS Services → Security Lake (S3/OCSF) → Athena (SQL queries)
                    ↓
              OpenSearch (Last 30 days hot analytics)
                    ↓
              CloudWatch Dashboards + Managed Grafana

Pros :

Cost-effective long-term retention
OCSF standardization
Multi-cloud ready
Compliance-friendly

Cons :

More complex setup
Longer implementation (16-20 weeks)
Requires OCSF knowledge

Approach 2: Direct CloudWatch/OpenSearch (Speed-First)

Best For : Startups, lower compliance reqs, quick wins


AWS Services → CloudWatch Logs → OpenSearch (direct)
                    ↓
              CloudWatch Dashboards + Managed Grafana
                    ↓
              S3 (archived logs via export)

Pros :

Faster implementation
Simpler architecture
Real-time everything
Lower learning curve

Cons :

Higher CloudWatch Logs costs at scale
No OCSF normalization
OpenSearch storage costs

Real-World Implementation: Step-by-Step

Let’s build this thing. Here’s the actual deployment sequence:

Week 1-2: Foundation


# 1. Enable AWS Organizations (if not already)
aws organizations create-organization

# 2. Enable CloudTrail (all regions, all accounts)
aws cloudtrail create-trail \
  --name organization-trail \
  --s3-bucket-name my-cloudtrail-bucket \
  --is-organization-trail \
  --is-multi-region-trail

# 3. Enable GuardDuty
aws guardduty create-detector --enable

# 4. Enable Security Hub
aws securityhub enable-security-hub

# 5. Enable AWS Config
aws configservice put-configuration-recorder \
  --configuration-recorder name=default,roleARN=arn:aws:iam::ACCOUNT:role/aws-service-role/config.amazonaws.com/AWSServiceRoleForConfig
aws configservice start-configuration-recorder \
  --configuration-recorder-name default

Week 3-4: EKS Monitoring


# fluent-bit-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-config
  namespace: amazon-cloudwatch
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush 5
        Log_Level info

    [INPUT]
        Name tail
        Path /var/log/containers/*.log
        Parser docker
        Tag kube.*

    [FILTER]
        Name kubernetes
        Match kube.*
        Kube_URL https://kubernetes.default.svc:443
        Merge_Log On

    [OUTPUT]
        Name cloudwatch_logs
        Match kube.*
        region us-east-1
        log_group_name /aws/eks/my-cluster
        log_stream_prefix app-
        auto_create_group true



kubectl apply -f fluent-bit-config.yaml

Week 5-6: OpenSearch SIEM


# Create OpenSearch domain
aws opensearch create-domain \
  --domain-name security-analytics \
  --engine-version "OpenSearch_2.11" \
  --cluster-config InstanceType=r6g.large.search,InstanceCount=3 \
  --ebs-options EBSEnabled=true,VolumeType=gp3,VolumeSize=100 \
  --encryption-at-rest-options Enabled=true \
  --node-to-node-encryption-options Enabled=true \
  --advanced-security-options Enabled=true,InternalUserDatabaseEnabled=false

Week 7-8: Dashboards and Alerts


// cloudwatch-dashboard.json
{
  "widgets": [
    {
      "type": "metric",
      "properties": {
        "metrics": [
          ["AWS/EC2", "CPUUtilization", {"stat": "Average"}]
        ],
        "period": 300,
        "stat": "Average",
        "region": "us-east-1",
        "title": "EC2 CPU Overview"
      }
    },
    {
      "type": "log",
      "properties": {
        "query": "SOURCE '/aws/eks/my-cluster' | fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 20",
        "region": "us-east-1",
        "title": "Recent Errors"
      }
    }
  ]
}



aws cloudwatch put-dashboard \
  --dashboard-name "Production-Overview" \
  --dashboard-body file://cloudwatch-dashboard.json

Automated Incident Response

Here’s where it gets interesting. Let’s automate security responses:


# lambda/security_response.py
import boto3

ec2 = boto3.client('ec2')
sns = boto3.client('sns')

def lambda_handler(event, context):
    """
    Responds to GuardDuty findings automatically
    """
    finding = event['detail']
    finding_type = finding['type']

    # SSH Brute Force detected
    if 'SSHBruteForce' in finding_type:
        instance_id = finding['resource']['instanceDetails']['instanceId']

        # Quarantine instance
        ec2.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=['sg-quarantine'] # Pre-created quarantine security group
        )

        # Notify team
        sns.publish(
            TopicArn='arn:aws:sns:us-east-1:ACCOUNT:security-alerts',
            Subject=f'CRITICAL: Instance {instance_id} Quarantined',
            Message=f'Detected SSH brute force attack. Instance automatically isolated.\n\nFinding: {finding}'
        )

        return {'status': 'quarantined', 'instance': instance_id}

EventBridge Rule :


{
  "source": ["aws.guardduty"],
  "detail-type": ["GuardDuty Finding"],
  "detail": {
    "severity": [7, 8, 8.9],
    "type": ["UnauthorizedAccess:EC2/SSHBruteForce"]
  }
}

Result : Threat detected → Instance isolated → Team notified. All in <30 seconds.

Cost Optimization Tips

Let’s talk money. Here’s how to keep costs reasonable:

1. CloudWatch Logs: The Got you by Surprise Cost


# Set appropriate retention periods
import boto3

logs = boto3.client('logs')

# Development logs: 7 days
logs.put_retention_policy(
    logGroupName='/aws/lambda/dev-functions',
    retentionInDays=7
)

# Production logs: 30 days
logs.put_retention_policy(
    logGroupName='/aws/lambda/prod-functions',
    retentionInDays=30
)

# Compliance logs: Export to S3, then delete
logs.put_retention_policy(
    logGroupName='/aws/cloudtrail',
    retentionInDays=90
)

2. Use Log Sampling

Not every log line needs immediate indexing:


# Sample 10% of high-volume logs
import random

def lambda_handler(event, context):
    if random.random() &lt; 0.1: # 10% sampling
        # Send to OpenSearch
        pass

    # Always send to S3 (cheap storage)
    # Send everything

3. OpenSearch Reserved Instances


# Save 30-40% with 1-year reserved capacity
aws opensearch purchase-reserved-instance-offering \
  --reserved-instance-offering-id offering-id \
  --instance-count 3

4. S3 Intelligent-Tiering


# Automatic cost optimization for Security Lake
aws s3api put-bucket-intelligent-tiering-configuration \
  --bucket security-lake-bucket \
  --id intelligent-tiering \
  --intelligent-tiering-configuration '{
    "Id": "intelligent-tiering",
    "Status": "Enabled",
    "Tierings": [
      {"Days": 90, "AccessTier": "ARCHIVE_ACCESS"},
      {"Days": 180, "AccessTier": "DEEP_ARCHIVE_ACCESS"}
    ]
  }'

Migration Strategy: The Practical Path

Don’t try to do everything at once. Here’s the battle-tested sequence:

Phase 1: AWS Resources

Start with EKS (highest ROI)
Add EC2 instances
Enable RDS Enhanced Monitoring
Configure Lambda logging

Win : 60% of your monitoring consolidated

Phase 2: Security

Enable Security Hub
Deploy GuardDuty
Set up OpenSearch SIEM
Migrate from Sentinel

Win : Security team has single console

Phase 3: Dashboards

Build CloudWatch operational dashboards
Deploy Managed Grafana
Recreate critical legacy dashboards
Train operations team

Win : Ops team stops using old tools

Phase 4: On-Premises

Deploy CloudWatch Agent to servers
Configure syslog forwarding
Archive on-prem logs in S3

Phase 5: Decommission

Parallel run validation (2 weeks)
Export historical data
Turn off Zabbix, Prometheus
Reclaim licenses and infrastructure

Common Issues (And How to Avoid Them)

Issue #1: CloudWatch Logs Cost Explosion

Problem : Someone enables debug logging in production

Solution :


# Implement log sampling and filtering at source
import logging
import watchtower

# Only send WARNING and above to CloudWatch
handler = watchtower.CloudWatchLogHandler(log_group='/aws/app')
handler.setLevel(logging.WARNING)

logger = logging.getLogger( __name__ )
logger.addHandler(handler)

Issue #2: Alert Fatigue

Problem : 500 alerts per day, all marked “critical”

Solution :


# Implement alert prioritization
def calculate_severity(metric_value, threshold):
    if metric_value > threshold * 1.5:
        return 'CRITICAL' # Page on-call
    elif metric_value > threshold * 1.2:
        return 'WARNING' # Slack notification
    else:
        return 'INFO' # Log only

Issue #3: The “We’ll Monitor Everything” Trap

Problem : Monitoring 10,000 metrics per instance

Solution : Start with the Golden Signals :

Latency : How long requests take
Traffic : Request volume
Errors : Failure rate
Saturation : Resource utilization


# Focused metric collection
CRITICAL_METRICS = [
    'CPUUtilization',
    'MemoryUtilization',
    'NetworkIn',
    'NetworkOut',
    'DiskReadOps',
    'DiskWriteOps'
]

Issue #4: Forgetting About Cardinality

Problem : OpenSearch cluster dies from high-cardinality fields

Solution :


# Don't index user IDs, session IDs, or timestamps as keywords!
PUT /logs/_mapping
{
  "properties": {
    "user_id": {
      "type": "text", # Don't use "keyword" for high-cardinality
      "index": false # Don't index if you won't search it
    },
    "timestamp": {
      "type": "date"
    }
  }
}

Success Metrics: Measuring Your Win


Old Way: "Let me check 5 systems..."
Time to answer: 15-30 minutes

New Way: "Here's the CloudWatch dashboard..."
Time to answer: 30 seconds

Troubleshooting Guide

Issue: CloudWatch Agent Not Sending Logs


# Check agent status
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a query -m ec2 -c default

# Check agent logs
sudo tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

# Common fix: IAM permissions
# Ensure instance role has CloudWatchAgentServerPolicy

Issue: OpenSearch “Cluster Red” Status


# Check cluster health
curl -XGET 'https://your-domain.region.es.amazonaws.com/_cluster/health?pretty'

# Common causes:
# 1. Unassigned shards (need more nodes)
# 2. Disk space >85% used (scale storage)
# 3. JVM pressure (scale instance type)

# Quick fix: Delete old indices
curl -XDELETE 'https://your-domain.region.es.amazonaws.com/old-index-*'

Issue: High CloudWatch Costs


# Find expensive log groups
aws logs describe-log-groups \
  --query 'logGroups[*].[logGroupName,storedBytes]' \
  --output table | sort -k2 -rn

# Check for debug logs in production
aws logs filter-log-events \
  --log-group-name /aws/lambda/my-function \
  --filter-pattern "DEBUG" \
  --limit 10

Best Practices Checklist

Document current log volumes (GB/day)
List all alert rules from legacy systems
Identify compliance retention requirements
Get buy-in from security and ops teams
Set realistic budget expectations
Start with non-production environment
Run legacy and new systems in parallel (2+ weeks)
Train ops team before cutover
Have rollback plan ready
Document everything (future you will thank you)
Monitor CloudWatch costs daily (first month)
Review alert effectiveness weekly
Gather user feedback from ops team
Optimise based on actual usage patterns
Schedule quarterly reviews

The Bottom Line

Consolidating from multiple monitoring tools to a unified AWS native stack isn’t just about reducing complexity, it’s about operational excellence :

Faster incident response : 15 minutes instead of 45
Better security posture : Automated threat response
Compliance confidence : Query any log in seconds
Cost savings : £5-10k+/year in eliminated tools
Happier ops team : One system to master, not five

Getting Started

If you’re ready to begin:

Week 1 : Audit current tools and costs
Week 2 : Estimate AWS costs with AWS Pricing Calculator
Week 3 : POC with non-prod EKS cluster
Week 4 : Build business case
Week 5+ : Execute phased migration

Resources

AWS Documentation

Tools

Community

Conclusion

Building a unified AWS monitoring solution is a journey, not a destination. Start small, prove value quickly, and iterate based on real-world usage.

The goal isn’t monitoring perfection, it’s operational sanity. When your phone rings at 3 AM, you want answers in minutes, not a hunt across five different tools.

Tags: #aws #monitoring #observability #cloudwatch #devops #sre #kubernetes #eks #security #siem

The post Complete Guide to AWS Monitoring and Observability for DevOps Teams first appeared on Allaboutcloud.co.uk.

Kiro CLI – Transform Your Terminal into an AI-Powered AWS Architecture Studio

Nikitas Gargoulakis — Tue, 06 Jan 2026 19:55:34 +0000

Creating AWS architecture diagrams has traditionally been one of those tasks that developers and solutions architects love to procrastinate on. You know the drill: dragging and dropping icons in tools like Lucidchart or draw.io, hunting for the latest AWS service icons, spending hours on layout and alignment, and then starting all over again when requirements change. What if I told you there’s a better way?

Here comes Kiro CLI with Model Context Protocol (MCP) support, a game-changing approach that lets you generate professional AWS architecture diagrams using natural language prompts, right from your terminal.

What is Kiro cli?

Kiro CLI is a command-line interface that brings AI-powered development capabilities directly to your terminal. Built on Claude’s frontier models, it’s designed to help developers write code, debug issues, automate workflows, and yes, create architecture diagrams – all through natural conversation.

Unlike traditional CLI tools that require memorizing specific commands and syntax, Kiro CLI understands what you want to accomplish and helps you get there through an interactive dialogue. It’s like having a senior developer sitting next to you, ready to help with any task.

Key Features about Kiro cli

1. Custom Agents for Specialized Tasks

You can create task-specific agents optimized for your workflows. Want a DevOps agent that knows your infrastructure patterns? Or a diagram specialist that follows your company’s architecture standards? Kiro CLI lets you build and deploy these specialized agents with pre-defined tool permissions, context, and prompts.

2. Advanced Context Management

Kiro CLI maintains project-specific conversation history and understands your codebase through directory-based persistence. It automatically associates chat sessions with your working directories, ensuring relevant context is always available.

3. Native MCP Support

This is where the magic happens for diagram generation. The Model Context Protocol allows Kiro CLI to connect to external tools and data sources, including AWS documentation and diagram generation libraries.

4. Seamless Cross-Platform Experience

If you’re already using Kiro IDE, your configurations transfer seamlessly. Your MCP servers, steering files, and project documentation work in both environments – configure once, use everywhere.

Understanding Model Context Protocol (MCP)

Before diving into diagram creation, let’s understand what makes this possible. The Model Context Protocol, developed by Anthropic, is an open standard that enables AI tools to securely connect to external data sources, tools, and custom servers.

Think of MCP as a universal adapter for your AI assistant:

MCP Client : The host application (Kiro CLI in our case) that communicates with MCP servers
MCP Server : Lightweight programs that expose specific tools or resources
MCP Tools : Model-controlled functions that the AI can automatically discover and invoke

For AWS diagram generation, we’ll use two critical MCP servers:

AWS Diagram MCP Server : Provides tools to create diagrams using Python’s diagrams library
AWS Documentation MCP Server : Searches and fetches AWS documentation for best practices

Setting Up Your Environment

Let’s get everything configured so you can start generating diagrams.

Step 1: Install Kiro CLI

Installation is straightforward and takes less than a minute:


curl -fsSL https://cli.kiro.dev/install | bash

Verify the installation:


kiro --version

You should see output like kiro-cli x.x.x

Step 2: Configure Authentication

Kiro CLI supports multiple authentication methods. For quick experimentation, AWS Builder ID is recommended:


kiro login

Follow the prompts to complete authentication.

Step 3: Install Required Dependencies

The diagram generation relies on Python tooling. First, install uv (a fast Python package installer):


pip install uv

If you’re on macOS, you’ll also need Graphviz:


brew install graphviz

For Linux:


sudo apt-get install graphviz

Step 4: Configure MCP Servers

Kiro CLI automatically discovers MCP servers from the configuration file. Create or edit ~/.kiro/settings/mcp.json:


{
  "mcpServers": {
    "awslabs.aws-diagram-mcp-server": {
      "command": "uvx",
      "args": ["awslabs.aws-diagram-mcp-server"],
      "env": {
        "FASTMCP_LOG_LEVEL": "ERROR"
      },
      "autoApprove": [],
      "disabled": false
    },
    "awslabs.aws-documentation-mcp-server": {
      "command": "uvx",
      "args": ["awslabs.aws-documentation-mcp-server@latest"],
      "env": {
        "FASTMCP_LOG_LEVEL": "ERROR"
      },
      "autoApprove": [],
      "disabled": false
    }
  }
}

Step 5: Verify MCP Configuration

Start a Kiro session:


kiro-cli

Check your configured MCP servers:


/mcp

You should see both AWS Diagram and AWS Documentation MCP servers listed.

Creating Your First AWS Diagram

Let’s start with a simple three-tier web application architecture.

Basic Three-Tier Architecture

In your Kiro chat session, simply describe what you want:


Create a three-tier web application architecture diagram with:
1. Application Load Balancer in a public subnet
2. Auto Scaling group with EC2 instances in private subnets
3. RDS MySQL database in private subnets
4. S3 bucket for static assets
Include VPC, availability zones, and security groups

Kiro will use the AWS Diagram MCP to generate Python code using the diagrams library and create a visual representation. The diagram will be saved as an image file (typically PNG or SVG).

Understanding the Generated Code

Behind the scenes, Kiro generates code similar to this:


from diagrams import Diagram, Cluster
from diagrams.aws.compute import EC2, AutoScaling
from diagrams.aws.network import ELB, VPC
from diagrams.aws.database import RDS
from diagrams.aws.storage import S3

with Diagram("Three-Tier Web Application", show=False, direction="TB"):
    with Cluster("VPC"):
        with Cluster("Public Subnet"):
            alb = ELB("Application\nLoad Balancer")

        with Cluster("Private Subnet - App Tier"):
            app_group = [EC2("Web Server 1"),
                        EC2("Web Server 2"),
                        EC2("Web Server 3")]
            asg = AutoScaling("Auto Scaling")

        with Cluster("Private Subnet - Data Tier"):
            db = RDS("MySQL\nDatabase")

        static = S3("Static Assets")

    alb >> app_group >> db
    alb >> static

The main feature of Kiro CLI is that you don’t need to know this syntax – just describe what you want, and the AI handles the rest.

Advanced Diagram Patterns

Data Processing Pipeline

For more complex architectures, Kiro CLI really shines. Here’s how to create a sophisticated data processing pipeline:


Create a data processing pipeline diagram using AWS services:

1. Data Ingestion cluster:
   - Kinesis Data Streams for real-time ingestion
   - API Gateway for REST endpoints

2. Data Processing cluster:
   - Lambda functions for transformation
   - AWS Glue for ETL jobs
   - Step Functions for orchestration

3. Data Storage cluster:
   - S3 buckets (Raw, Processed, Curated)
   - DynamoDB for metadata

4. Analytics cluster:
   - Athena for querying
   - QuickSight for visualization

Include data flow arrows, IAM role boundaries, and CloudWatch monitoring.
Ensure AWS best practice visualization.

Serverless Microservices Architecture


Design a serverless microservices architecture with:
- API Gateway as the entry point
- Multiple Lambda functions for different services (auth, orders, inventory)
- DynamoDB tables for each service
- EventBridge for inter-service communication
- Cognito for user authentication
- CloudFront for global distribution
- S3 for frontend hosting

Show clear service boundaries and event flows.

Container-Based Architecture


Create an EKS-based microservices diagram showing:
- VPC with public and private subnets across 3 AZs
- Application Load Balancer
- EKS cluster with multiple node groups
- ECR for container images
- RDS Aurora for database
- ElastiCache for caching
- CloudWatch for logging and monitoring

Include networking flow and security group relationships.


 Perfect! I've created a comprehensive EKS microservices architecture diagram
that follows AWS best practices. Here's what the diagram includes:

## Architecture Components:

VPC Design (Multi-AZ):
- VPC with CIDR 10.0.0.0/16 across 3 Availability Zones
- Public subnets (10.0.1-3.0/24) for internet-facing resources
- Private subnets (10.0.11-13.0/24) for EKS worker nodes
- Database subnets (10.0.21-23.0/24) for RDS and ElastiCache

Networking & Load Balancing:
- Internet Gateway for public internet access
- NAT Gateways in each AZ for outbound internet access from private subnets
- Application Load Balancer in public subnets for incoming traffic

EKS Cluster:
- Managed EKS control plane
- Two node groups with different instance types:
  - General purpose (t3.medium) for standard workloads
  - Compute optimized (c5.large) for CPU-intensive tasks
- Microservices deployed as containers (Frontend, API Gateway, User Service,
Order Service, Payment Service)

Data Layer:
- RDS Aurora with primary (writer) and replica (reader) instances
- ElastiCache Redis cluster for caching

Supporting Services:
- ECR for container image storage
- CloudWatch for logging and monitoring

Security:
- Security groups with appropriate port restrictions:
  - ALB SG: Ports 80/443 (HTTP/HTTPS)
  - EKS SG: Ports 443, 10250 (Kubernetes API, kubelet)
  - DB SG: Ports 3306/5432 (MySQL/PostgreSQL)
  - Cache SG: Port 6379 (Redis)

## AWS Best Practices Implemented:

1. High Availability: Multi-AZ deployment across 3 availability zones
2. Security: Private subnets for compute and database tiers, security groups with
least privilege
3. Scalability: Multiple node groups, auto-scaling capabilities
4. Monitoring: CloudWatch integration for logs and metrics
5. Container Management: ECR for secure image storage
6. Network Segmentation: Separate subnets for different tiers
7. Managed Services: Using managed EKS control plane, RDS Aurora, and ElastiCache

The diagram shows the complete traffic flow from users through the internet
gateway, load balancer, to the microservices, and their connections to the
database and cache layers. The orange dashed lines represent security group
associations for each component.

Best Practices for Diagram Generation

1. Be Specific About Your Requirements

The more detailed your prompt, the better the result. Instead of “create an AWS diagram,” specify:

Which AWS services to include
How they connect to each other
Clustering and organisation preferences
Network boundaries and security groups
Data flow directions

2. Iterate on Your Diagrams

Kiro CLI excels at iteration. After generating an initial diagram, you can refine it:


Add a CloudFront distribution in front of the ALB
Include an AWS WAF for security
Show the connection to Route 53 for DNS
Don't over engineer it. Keep it simple

3. Leverage AWS Documentation MCP

When you’re unsure about best practices, ask Kiro to consult AWS documentation:


Search AWS documentation for best practices on securing RDS databases,
then update the diagram to reflect those recommendations.

Real-World Use Cases

1. Documentation for Stakeholders

When you need to explain your infrastructure to non-technical stakeholders, generate clean, professional diagrams that focus on business flows rather than technical details.

2. Compliance and Audit Requirements

Many compliance frameworks require architecture documentation. Kiro CLI can quickly generate diagrams showing security controls, data flows, and network segmentation.

3. Infrastructure Planning

Before implementing new features, use Kiro to explore different architectural approaches visually. Generate multiple diagram variations to compare trade-offs.

4. Onboarding New Team Members

Create comprehensive architecture diagrams as part of your onboarding documentation. New developers can quickly understand system design and component relationships.

5. Disaster Recovery Planning

Generate diagrams showing your DR setup, including cross-region replication, backup strategies, and failover processes.

Comparing Traditional vs. AI-Powered Diagram Tools

Traditional Approach

Time : A few hours for a complex architecture
Learning Curve : Days to weeks to master diagramming tools
Maintenance : Manual updates for every change
Consistency : Difficult to maintain across multiple diagrams
Cost : Subscription fees for professional tools

Kiro CLI Approach

Time : 5-15 minutes for a complex architecture
Learning Curve : Minutes – just describe what you want
Maintenance : Natural language updates
Consistency : AI ensures standard practices
Cost : Included in Kiro subscription

Advanced Tips and Tricks

1. Multi-Region Architectures


Create a multi-region architecture showing:
- Primary region in us-east-1 with full stack
- DR region in eu-west-1 with warm standby
- Route 53 health checks and failover routing
- Cross-region replication for S3 and DynamoDB


## Architecture Components

Primary Region (us-east-1) - Active:
- Application Load Balancer distributing traffic
- Multiple EC2 web servers for high availability
- RDS with Multi-AZ deployment for database resilience
- DynamoDB primary table
- S3 primary bucket for object storage
- CloudWatch for monitoring and alerting

DR Region (eu-west-1) - Warm Standby:
- ALB in standby mode (can be activated quickly)
- EC2 instance in warm standby (minimal capacity, can scale up)
- RDS Read Replica for database failover
- DynamoDB Global Tables for automatic replication
- S3 replica bucket with cross-region replication
- CloudWatch for monitoring the DR environment

## Key AWS Best Practices Implemented

Route 53 Health Checks & Failover:
- DNS-based failover routing
- Health checks monitor primary region endpoints
- Automatic failover to DR region when primary fails
- Green arrows show normal traffic flow, red dashed shows failover

Cross-Region Replication:
- **RDS:** Read replicas provide near real-time data replication
- **DynamoDB:** Global Tables enable automatic multi-region replication
- **S3:** Cross-region replication ensures data durability across regions

Warm Standby Strategy:
- Cost-effective approach with minimal resources in DR region
- Can be scaled up quickly during failover
- Balances cost with recovery time objectives (RTO)

This architecture provides robust disaster recovery capabilities while following
AWS Well-Architected principles for reliability and cost optimisation

2. Hybrid Cloud Scenarios


Design a hybrid architecture connecting on-premises data center to AWS:
- Direct Connect connection
- VPN as backup
- AWS Transit Gateway
- On-premises resources shown separately

The Future of Infrastructure Documentation

Kiro CLI represents a shift in how we can create and maintain technical documentation. By combining the power of large language models with specialized tools through MCP, we’re moving toward a future where:

Documentation stays in sync with code automatically
Architecture diagrams update as infrastructure evolves
Best practices are enforced consistently
Knowledge sharing becomes frictionless

Conclusion

Traditional diagramming tools still have their place, but for developers and architects, Kiro CLI with MCP support offers an unprecedented productivity boost. What used to take hours now takes minutes. What required specialised tool knowledge now just requires clear communication.

The real power isn’t just in generating diagrams faster, it’s in lowering the barrier to creating and maintaining quality documentation. When documentation becomes this easy, teams are more likely to keep it current, and that benefits everyone.

Whether you’re building a simple three-tier application or a complex microservices architecture spanning multiple regions, Kiro CLI can help you create professional AWS architecture diagrams that communicate your vision clearly and accurately.

https://kiro.dev/docs/cli

This article reflects my personal experience using AWS Kiro for side projects and creating diagrams, that can be used as starting point, following AWS Best Practices

The post Kiro CLI – Transform Your Terminal into an AI-Powered AWS Architecture Studio first appeared on Allaboutcloud.co.uk.

How to Protect VMware VMs from Ransomware with AWS Backup

Nikitas Gargoulakis — Wed, 03 Sep 2025 17:43:00 +0000

AWS Backup ransomware protection helps secure VMware workloads against modern ransomware attacks by using immutable backups, logical air-gapped vaults, and cross-account isolation.

This guide details the entire process of deploying an enterprise-level backup solution that safeguards against ransomware attacks, by using three important security layers:

Immutable Storage – Backups cannot be deleted by anyone, including attackers with administrative access
Logical Air-Gap Vault – Securely isolated storage environment that uses AWS controls (like separate accounts, restrictive IAM, and immutability) to prevent access or tampering
External Approval Authority – Approval team located in separate AWS account outside your organisation

Architecture Overview

Security Model

Production Environment

Your production AWS accounts and VMware environment
Daily automated backups with Changed Block Tracking
Standard backup vault for operational restores

External Approval Authority

Completely separate AWS account (not part of your organisation)
Contains approval team of 3+ trusted senior executives
MFA enforcement for all team members
Only function: approve emergency backup access

Recovery Environment

Used exclusively during disaster recovery
Can request access to protected backups
Access granted only after multi-party approval from external team

How This Stops Ransomware Attacks

Traditional Attack Pattern:

Attacker gains access to production systems
Escalates privileges to administrator level
Deletes or encrypts all backups
Encrypts production data
Demands ransom

How Our Architecture Prevents This:

Step 3 Fails: Compliance-mode locks prevent backup deletion by anyone
Air-Gapping: Backups logically isolated, not directly accessible
External Approval: Attacker cannot compromise approval team (separate organization)
Multi-Party Requirement: Need 2+ trusted individuals to approve access
Result: Backups remain protected; organization can recover without paying ransom

Prerequisites

Production Environment Requirements

AWS Infrastructure:

AWS Organisations with minimum 2 accounts
One account designated as backup delegate administrator
Site-to-Site VPN between on-premises datacenter and AWS / Internet connectivity from VMware environment to AWS

VMware Environment:

VMware vSphere 6.7 or later
vCenter Server operational
Service account with backup permissions
Available resources: 4 vCPUs, 8GB RAM, 80GB disk for gateway VM

Network Requirements:

VPN bandwidth: 100 Mbps minimum (1 Gbps recommended)
Network latency: Under 50ms recommended
Firewall permits: HTTPS (port 443) to AWS endpoints
DNS resolution for AWS service endpoints

External Approval Account Requirements

New AWS Account:

Must be completely separate from your organisation
Cannot be member of any AWS Organisations
Dedicated email address (example: backup-approvals@yourcompany.com)
Root account secured with hardware MFA token

Trusted Approvers (3 minimum):

Senior executives with authority to approve emergencies
Examples: CTO, CISO, Infrastructure Director, etc
Must be available 24/7 for emergency response

VMware Backup Gateway Setup

Configure AWS Organisations Delegation

Purpose: Designates your backup account as central backup administrator for entire organization.

Instructions:

Sign in to your AWS Organizations management account
Navigate to AWS Organizations service
Go to Services → AWS service access
Enable access for AWS Backup service
Navigate to Delegated administrators
Register your backup account as delegated administrator for AWS Backup service
Verify delegation appears as “Active”

What This means:

Backup account can manage backups across all organization accounts
Doesn’t require management account access for daily operations
Centralizes backup management and policies
Download and Deploy Backup Gateway

Installs virtual appliance that connects AWS Backup to your VMware environment.

In AWS Backup Console (Backup Account):

Select your region (example: EU-WEST-2 for London)
Navigate to AWS Backup → External resources → Gateways
Click Create gateway
Download the OVF template file (approximately 1.2 GB)
Save as: aws-appliance-latest.ova

In VMware vSphere Client:

Connect to vCenter Server
Right-click parent object (datacenter or cluster)
Select Deploy OVF Template
Choose Local file and select downloaded OVA file
Provide gateway name: Backup-Gateway-Production
Select compute resource (cluster or host)
Critical: Select storage disk format: Thick Provision Lazy Zeroed
Select management network (must have internet access)
Complete deployment wizard

Configure VM Settings Before Power-On:

Right-click deployed gateway VM → Edit Settings
Verify configuration:
- CPU: 4 vCPUs
- Memory: 8 GB (set memory reservation to 8192 MB)
- Hard Disk: 80 GB
Go to VM Options → VMware Tools
Enable: Synchronize Time with Host
Enable: Synchronize at startup and resume
Save settings

Configure Gateway Network Settings

Power On Gateway VM:

Right-click gateway VM → Power → Power On
Open VM console

Initial Login:

Default username: admin
Default password: password
You’ll be prompted to change password immediately
Create strong password (minimum 12 characters, mixed case, numbers, symbols)

Configure Static IP Address:

At main menu, select Configure Network
Choose Static IP configuration
Enter network details:
- IP Address: (assign from your management network range)
- Subnet Mask: (example: 255.255.255.0)
- Default Gateway: (your network gateway)
- Primary DNS: (your internal DNS server)
- Secondary DNS: (backup DNS, can use 8.8.8.8)
Save configuration

Test Network Connectivity:

At main menu, select Test Network Connectivity
Gateway tests:
- Basic network connectivity
- DNS resolution
- Internet access
- AWS endpoint reachability
- Time synchronization
All tests should show “OK” or “PASS”
Record the gateway IP address for next step

Firewall Requirements:

If you have firewalls between gateway and internet, allow outbound traffic:

Destination: *.backup.[your-region].amazonaws.com (port 443)
Destination: *.s3.[your-region].amazonaws.com (port 443)
Destination: time.aws.com (port 123 UDP)
No inbound rules required (all connections are outbound)

Register Gateway with AWS

In AWS Backup Console:

Navigate to External resources → Gateways
Click Register gateway
Enter gateway details:
- Gateway IP Address: (IP from previous step)
- Gateway Name: Production-VMware-Gateway
- Gateway Timezone: (select your timezone)
Add tags for organization:
- Environment: Production
- Purpose: VMware-Backup
- Location: On-Premises
Click Register gateway

Verify Connection:

Wait 2-5 minutes for connection
Status should change from “Registering” to “Connected”
Green indicator shows healthy connection
If connection fails, verify firewall rules and network connectivity

Integrate VMware vCenter

Create vCenter Service Account:

In vCenter Server, create a service account for AWS Backup with these permissions:

Required Permissions:

Virtual Machine: All inventory, configuration, state, and provisioning operations
Datastore: Browse datastore, allocate space
Network: Assign network
Apply at: Datacenter or Cluster level
Propagate to child objects: Yes

Add Hypervisor in AWS Backup:

Navigate to External resources → Hypervisors
Click Add hypervisor
Select your registered gateway
Choose Hypervisor Type: VMware vCenter
Enter connection details:
- Host: (vCenter IP address or hostname)
- Port: 443 (default)
- Username: (service account created above)
- Password: (service account password)
Provide hypervisor name: Production-vCenter
Click Test Connection to verify
Click Add hypervisor

Wait for VM Discovery:

AWS Backup automatically discovers all VMs (5-10 minutes)
Progress shown in console
After completion, view discovered VMs under External resources → Virtual machines

Create VMware Tags for Backup Selection

In vSphere Client:

Navigate to Tags & Custom Attributes → Tags
Click New Tag Category

Create Tag Category:

Category Name: backup
Description: Backup schedule
Cardinality: Single value per object

Create Tags Under ‘backup’ Category:

Tag: daily
- For VMs requiring daily backups
- Example: Production databases, critical applications
Tag: weekly
- For VMs requiring weekly backups
- Example: Development servers, secondary systems
Tag: monthly
- For VMs requiring monthly backups only
- Example: Archive systems, long-term storage
Tag: none
- For VMs excluded from backups
- Example: Temporary VMs, easily recreated systems

Apply Tags to VMs:

Right-click each VM in vSphere inventory
Select Tags & Custom Attributes → Assign Tag
Choose appropriate backup tag
VM will now be automatically included in matching backup plan

Tagging Strategy Example:

Mission-critical database servers: backup:daily
Application servers: backup:daily
File servers: backup:daily
Development servers: backup:weekly
Test environments: backup:none

Create Backup Plan

In AWS Backup Console:

Navigate to Backup plans
Click Create backup plan
Select Build a new plan

Backup Plan Configuration:

Plan Name: VMware-Production-Daily-Backup

Backup Rule Configuration:

Rule Name: DailyBackupRule
Backup Vault: Default (temporary; will add air-gapped vault in Phase 2)
Schedule:
- Frequency: Daily
- Time: 3:00 AM (choose off-peak time for your organization)
- Timezone: Your local timezone
- Backup window start: Within 1 hour
- Completion window: Within 3 hours

Lifecycle Settings:

Transition to cold storage: 30 days
Expire/Delete: 90 days
(Air-gapped vault will have longer retention)

Tags for Recovery Points:

BackupType: Daily
Environment: Production
Automated: True

Create Backup Selection:

After creating plan, immediately create backup selection:

Selection Name: Tagged-VMs-Daily-Production
IAM Role: Select Default role (AWS creates automatically)
Resource Selection: Include specific resource types
Resource Type: Select VM (Virtual Machine)

Define Selection by Tags:

Tag Key: backup
Tag Value: daily
Condition: Equals

Optional Additional Filter:

Tag Key: environment
Tag Value: production
This ensures only production VMs with daily tag are backed up

Click Assign resources

What Happens Now:

AWS Backup automatically discovers all VMs with backup:daily tag
First backup runs at next scheduled time (3:00 AM)
You can also trigger manual backup immediatel, for testing

Execute First Backup

Trigger Manual Backup (Don’t Wait for Schedule):

Navigate to Protected resources
Locate a test VM (non-production, with sample data)
Click the VM name
Click Create on-demand backup
Select backup vault: Default
Use default IAM role
Start backup immediately
Click Create on-demand backup

Monitor Backup Progress:

Navigate to Jobs → Backup jobs
Find your job at top of list
Watch status progression:
- Created → Job queued
- Running → Backup in progress (shows percentage)
- Completed → Backup successful

First Backup Timing:

Full backup typically takes 1-3 hours depending on VM size
Shows progress percentage throughout
Backup size approximately equals VM disk usage

Verify Backup Completed:

Navigate to Backup vaults → Default
Click Recovery points tab
Find your VM’s recovery point
Verify:
- Status: Completed (green)
- Backup size: Reasonable for your VM
- Creation date: Today

Important: Incremental Backups

First backup is always full snapshot
Second and subsequent backups use Changed Block Tracking (CBT)
Incremental backups are 90-95% smaller
Complete in minutes instead of hours
Automatic – no configuration needed

Test Restore

Initiate Test Restore:

Navigate to Backup vaults → Default
Click Recovery points tab
Select your test VM’s recovery point
Click Restore

Restore Configuration:

VMware Destination Settings:

Target Hypervisor: Select your vCenter
Resource Pool: Select appropriate pool
Datastore: Select storage location
VM Folder: Create RestoredVMs folder for test restores
Network: Map networks appropriately

VM Settings:

VM Name: TestVM-Restored-Validation
Power On: Yes (to immediately test functionality)

IAM Role: Select default role

Click Restore

Monitor Restore:

Navigate to Jobs → Restore jobs
Watch status: Running → Completed
Typical restore time: 30 minutes – 2 hours

Validate Restored VM:

In vSphere Client:

Navigate to RestoredVMs folder
Verify VM exists: TestVM-Restored-Validation
Confirm VM is powered on
Open console and verify:
- Guest operating system boots normally
- All disks present and accessible
- Applications start correctly
- Data integrity is intact (compare sample files)

Document Recovery Metrics:

Recovery Point Objective (RPO): Time difference between restore point and actual data
Recovery Time Objective (RTO): Time from restore initiation to VM fully operational
These metrics are critical for disaster recovery planning

Delete Test Restore:

After validation, delete restored test VM
Prevents confusion and saves storage
Keep documented results for reference

Your organisation now has:

Operational backup gateway connected to AWS
VMware vCenter fully integrated
Tag-based backup policies configured
First full backup completed successfully
Incremental backup capability verified
Restore process tested and validated

Air-Gapped Vault with Compliance Locks

Compliance-Mode Locks

What is Compliance-Mode Lock:

Makes backup vault permanently immutable
Backups cannot be deleted before retention period expires
Not even root account owner can bypass
Not even AWS support can override
Once grace period expires, lock is irreversible

During grace period you can:

Test backup and restore operations
Verify retention policies work correctly
Delete vault if you change your mind (last chance)

After grace period expires:

Lock becomes permanent
No changes possible
Vault exists until all backups expire naturally

*\ **Warning**:*

This is a point of no return decision. Before proceeding:

Get approval from senior management
Understand financial commitment (vault accumulates costs for entire retention period)
Test thoroughly during grace period
Document retention requirements clearly

Create Dedicated KMS Encryption Key

In AWS Key Management Service (KMS):

Navigate to KMS service
Select same region as backup vault (example: eu-west-2)
Click Customer managed keys → Create key

Key Configuration:

Step 1 – Configure Key:

Key Type: Symmetric
Key Usage: Encrypt and decrypt
Key Material Origin: KMS
Regionality: Single-Region key

Step 2 – Add Labels:

Alias: backup-airgapped-vault-encryption
Description: Encryption key for air-gapped backup vault - Production environment
Tags:
- Purpose: Backup-Encryption
- Environment: Production
- VaultType: AirGapped

Step 3 – Define Key Administrators:

Select your IAM user or role as key administrator
This allows you to manage key policies
Key administrators cannot use key to encrypt/decrypt

Step 4 – Define Key Usage Permissions:

Select: AWS Backup service (allows AWS Backup to use key)
Select: Your backup administrator IAM role
This grants permission to encrypt and decrypt backup data

Step 5 – Review and Create:

Review all settings carefully
Click Finish

Document Key Information:

Copy Key ID (format: a1b2c3d4-…)
Copy Key ARN (format: arn:aws:kms:eu-west-2:account:key/…)
Store in secure location
You’ll need this later for vault creation

Create Air-Gapped Backup Vault

In AWS Backup Console:

Navigate to Backup vaults
Click Create backup vault
Important: Select Create logically air-gapped vault option

Vault Configuration:

Basic Information:

Vault Name: Production-AirGapped-Vault
Vault Type: Logically air-gapped vault
Description: Immutable backup vault for ransomware protection

Encryption:

Select: Choose a custom encryption key
Select the KMS key you created in Step 2.1
Key alias: backup-airgapped-vault-encryption

Retention Configuration:

Minimum Retention Days: 30
Maximum Retention Days: 365
Adjust those based on your compliance requirements

Tags:

Environment: Production
Purpose: Ransomware-Protection
ComplianceMode: True
CreatedDate: (today’s date)

Click Create vault

Document Vault ARN

After creation:

Copy the full Vault ARN (format: arn:aws:backup:eu-west-2:account:backup-vault:Production-AirGapped-Vault)
Print this ARN and store in physical safe
Store digital copy in password manager
You will need this ARN for disaster recovery
Without this ARN, you cannot request access during emergency

Step 2.3: Apply Compliance-Mode Vault Lock

FINAL WARNING – READ CAREFULLY:


╔════════════════════════════════════════════════════════╗
║ POINT OF NO RETURN ║
╠════════════════════════════════════════════════════════╣
║ ║
║ You are about to enable COMPLIANCE MODE VAULT LOCK ║
║ ║
║ After grace period (default 3 days) expires: ║
║ • Lock becomes PERMANENT and IMMUTABLE ║
║ • Vault CANNOT be deleted by anyone ║
║ • Settings CANNOT be changed or modified ║
║ • Even AWS support CANNOT bypass this lock ║
║ • Vault exists until retention period expires ║
║ ║
║ Financial Commitment: ║
║ • Estimated monthly cost: $325 ║
║ • Commitment period: Retention period (365 days) ║
║ • Cannot be canceled or refunded ║
║ ║
║ Required Approvals: ║
║ □ Management approval obtained ║
║ □ Backup/restore tested successfully ║
║ □ Retention requirements verified ║
║ □ Budget approval secured ║
║ □ Vault ARN documented offline ║
║ □ All implications understood ║
║ ║
╚════════════════════════════════════════════════════════╝

If you have all approvals and understand implications:

In AWS Backup Console:

Navigate to Backup vaults
Select Production-AirGapped-Vault
Click Actions → Configure vault lock

Vault Lock Configuration:

Lock Mode: Compliance mode (recommended for ransomware protection)
Minimum Retention Days: 30
Maximum Retention Days: 365
Grace Period: 3 (72 hours to test)

Review warning dialog carefully
Type confirm to acknowledge
Click Apply vault lock

Grace Period Begins:

You now have 3 days to test thoroughly
Mark calendar for when lock becomes permanent
Use this time to validate restore operations
Last chance to delete vault if needed

Update Backup Plan for Air-Gapped Copy

Modify Existing Backup Plan:

Navigate to Backup plans
Select VMware-Production-Daily-Backup
Click Edit
Find DailyBackupRule
Click Edit rule

Add Copy Destination:

Scroll to Copy to destination section:

Enable: Yes, copy backups to another vault
Destination Vault: Select Production-AirGapped-Vault

Lifecycle for Copied Backups:

Transition to cold storage: 90 days
Expire: 365 days

Why Different Lifecycle:

Primary vault: Short retention (90 days) for quick operational restores
Air-gapped vault: Long retention (365 days) for ransomware recovery
Cold storage after 90 days saves approximately 90% on storage costs

Click Save changes

How Copy Jobs Work:

Primary backup runs at scheduled time (3:00 AM) to Default vault
After primary backup completes, copy job starts automatically
Backup copied to air-gapped vault (typically 30 minutes – 2 hours)
Both copies exist independently:
- Primary can be deleted after 90 days (operational use)
- Air-gapped copy protected for 365 days (ransomware protection)
- If primary corrupted, air-gapped copy remains safe

Monitor Copy Job Execution

Wait for Next Scheduled Backup or Trigger Manual Backup:

After next backup completes, copy job automatically starts.

Monitor Copy Jobs:

Navigate to Jobs → Copy jobs
Locate most recent copy job
Watch status progression:
- Created: Job queued
- Running: Copy in progress (shows percentage)
- Completed: Copy successful

Typical Timeline:

Source backup size: 500 GB
Copy duration: 45-90 minutes
Network: Internal AWS (no egress charges)

Verify Copy in Air-Gapped Vault:

Navigate to Backup vaults
Select Production-AirGapped-Vault
Click Recovery points tab
Verify:
- Recovery point from test VM exists
- Status: Completed
- Size: Matches primary backup
- Retention: 365 days
- Locked: Yes

Check Vault Statistics:

View vault summary:

Number of recovery points: Should match expected backup count
Total storage: Sum of all backup sizes
Locked status: Yes (with grace period countdown or “Locked” if expired)
Lock date: When lock becomes or became permanent

Test Restore from Air-Gapped Vault

Test during grace period while you can still delete vault if problems occur

Post-Lock Verification

After Grace Period Expires:

Verify Permanent Lock Status:

Navigate to Backup vaults
Select Production-AirGapped-Vault
Verify vault details:
- Locked: Yes (no grace period remaining)
- Lock Mode: Compliance
- Lock Date: (date when lock became permanent)
- Immutable: True

Test Lock Protection (Should Fail – Proves It Works):

Attempt 1: Try to Delete Vault

Select vault
Click Actions → Delete
Expected: Error message “Cannot delete vault – protected by compliance-mode lock”
This proves protection is working correctly

Attempt 2: Try to Modify Lock Settings

Select vault
Click Actions → Configure vault lock
Expected: All options greyed out / disabled
This proves lock is truly immutable

Attempt 3: Try to Delete Individual Backup

Select a recovery point
Try to delete
Expected: Deletion blocked by retention policy
Backup can only be deleted after retention period expires naturally

Document Vault Lock Status:

Record in your documentation:

Lock Status: Permanent / Immutable
Lock Applied Date: [date]
Earliest Possible Deletion: [date + 30 days minimum retention]
Verified By: [your name]
Next Review: [quarterly review date]

Your organisation now has:

Dedicated KMS encryption key for air-gapped vault
Logically air-gapped vault with compliance-mode lock
Automated copy jobs from primary to air-gapped vault
Tested restore capability from air-gapped vault
Permanent immutable protection active
Vault ARN documented in secure offline location

External Approval Team Configuration

Understanding External Approval Teams

Why External Account is Critical:

Vulnerable Setup (What NOT To Do):


Approval Team → Located in your AWS Organization
↓
Attacker compromises organization
↓
Attacker can compromise approval team
↓
Result: Backups accessible to attacker

Secure Setup (What We’re Building):


Approval Team → Separate AWS account (outside organization)
↓
Attacker compromises organization
↓
Approval team remains isolated and secure
↓
Result: Backups protected, attacker cannot approve access

Key Security Principle:

Even if ransomware attackers gain root access to every account in your organization, they cannot access air-gapped vault without approval from external team members who are outside the compromised environment.

Create External Approval Account

Create Completely Separate AWS Account:

Critical Requirements:

Must be standalone AWS account
Cannot be member of your AWS Organization
Cannot be part of any organizational structure
Managed by separate administrators
Dedicated email address (not shared with production accounts)

Account Creation Process:

Go to https://aws.amazon.com/
Click Create an AWS Account
Use dedicated email address
- Example: backup-approvals@yourcompany.com
- Create new mailbox if needed (don’t reuse existing)
Account name: Backup-Approval-Authority
Complete registration process
Provide payment method (monthly cost will be ~$0)

Immediate Security Configuration:

1. Secure Root Account:

Sign in as root user immediately
Navigate to Security credentials
Enable MFA using hardware token (strongly recommended) or authenticator app
Create strong root password (20+ characters)
Store credentials in password manager
Record recovery codes securely

2. Set Account Alias:

Navigate to IAM → Dashboard
Create account alias: backup-approval-authority
This creates friendly URL: https://backup-approval-authority.signin.aws.amazon.com

3. Enable CloudTrail Logging:

Navigate to CloudTrail service
Create trail: approval-team-audit-trail
Apply to all regions: Yes
Log file validation: Enabled
Create new S3 bucket for logs
Enable log encryption (optional but recommended)

Why CloudTrail is Critical:

Logs every approval action
Provides complete audit trail
Required for compliance
Cannot be disabled (ransomware protection)
Helps forensics if incident occurs

Enable IAM Identity Center

What is IAM Identity Center:

Centralised user management for AWS
Built-in MFA enforcement
Required for approval team functionality

Enable Identity Center

Configure Multi-Factor Authentication:

In IAM Identity Center, click Settings
Navigate to Authentication tab
Under Multi-factor authentication section:
- Enable MFA: Yes
- Prompt users for MFA: Every time (most secure)
- Allow these MFA types:
- Authenticator apps (Google Authenticator, Authy, 1Password)
- Security keys (YubiKey, other FIDO2 devices)
- Built-in authenticators
Click Save changes

Configure Password Policy:

Still in Settings → Authentication :

Password requirements:

Minimum length: 14 characters
Require uppercase letters: Yes
Require lowercase letters: Yes
Require numbers: Yes
Require symbols: Yes
Password expiration: 90 days
Prevent password reuse: Last 24 passwords
Account lockout: 5 failed attempts
Lockout duration: 15 minutes

Configure Session Duration:

In Settings :

Session duration: 8 hours
Idle timeout: 1 hour
This balances security with usability

Create Approval Team Members

Identify Trusted Approvers:

Selection Criteria:

Senior executive level (C-suite)
Technical understanding of disaster recovery
Authority to approve emergency actions
Available 24/7 for emergency response
Trusted with company-critical decisions
Ideally not IT administrators (separation of duties)

Example Approval Team Composition:

Approver 1: Chief Technology Officer (CTO)

Role: Technical authority and infrastructure oversight
Responsibility: Verify technical legitimacy of requests

Approver 2: Chief Information Security Officer (CISO)

Role: Security authority and incident response
Responsibility: Verify security implications and threats

Approver 3: Chief Financial Officer (CFO) or IT Manager

Role: Business continuity and operational authority
Responsibility: Authorize business impact decisions

Create Users in IAM Identity Center:

In IAM Identity Center, navigate to Users
Click Add user

For Each Approver, Configure:

User Details:

Username: (first.last format, example: john.smith)
Email address: (work email, must be valid and monitored)
First name: (example: John)
Last name: (example: Smith)
Display name: (example: John Smith)

Optional but Recommended:

Job title: (example: Chief Technology Officer)
Department: (example: C- Leadership)
Phone number: (for verification)

Click Next
Skip group assignment (for now)
Click Next
Review details
Click Add user

Repeat for all approval team members (minimum 3 recommended)

Users Receive Setup Emails:

Each user receives invitation email:


Subject: Set up your AWS IAM Identity Center account

You've been invited to join the Backup-Approval-Authority 
AWS account.

Click here to complete setup: [Link expires in 7 days]

Setup Requirements:
1. Create password (minimum 14 characters)
2. Configure MFA device (required)
3. Save recovery codes
4. Complete profile

Important: Complete setup within 7 days or invitation expires.

User Setup Process

Each Approval Team Member Must Complete:

Step 1: Create Password

Click invitation link received via email
Create strong password meeting requirements:
- Minimum 14 characters
- Mix of uppercase, lowercase, numbers, symbols
- Example: MySecure$Backup#Approval#2025!
Confirm password
Click Continue

Step 2: Register MFA Device

Choose MFA device type:

Recommended: Authenticator app (Google Authenticator, 1Password)
Alternative: Hardware security key (YubiKey or similar)

For Authenticator App:

Open authenticator app on smartphone
Select Add account or scan QR code option
Scan QR code displayed in AWS console
App generates 6-digit codes every 30 seconds
Enter two consecutive codes to verify
Click Assign MFA device

Important MFA Setup Notes:

Save backup codes in secure location
Test MFA before closing setup
If smartphone lost, recovery codes allow access

Step 3: Complete User Profile

Verify display name is correct
Verify email address
Add phone number (used for out-of-band verification)
Review profile details
Click Complete setup

Step 4: Test Initial Sign-In

Sign out from setup session
Navigate to: https://backup-approval-authority.signin.aws.amazon.com
Enter username
Enter password
Enter current MFA code from authenticator app
Should successfully sign in and see approval portal

Enable Multi-Party Approval in Backup Account

Switch Context to Production Backup Account:

Now configure your production backup account to accept external approval teams.

In Production Backup Account:

Sign in to backup administrator account
Select your backup region (example: eu-west-2)
Navigate to AWS Backup service
Click Settings in left navigation menu

Enable Required Features:

Enable all three cross-account features:

Cross-Account Backup: Enable
Cross-Account Monitoring: Enable
Multi-Party Approval: Enable

What These Settings Enable:

Cross-Account Backup: Allows backup sharing between AWS accounts
Cross-Account Monitoring: View backup status across multiple accounts
Multi-Party Approval: Required for external approval team integration

Click Save changes

Verification:

All three settings should show “Enabled” status
Green checkmarks next to each feature
If any setting fails to enable, check IAM permissions

Create Approval Team

Return to External Approval Account:

In External Approval Account (us-east-1 region):

Navigate to AWS Backup service
Click Approval teams in left menu
Click Create approval team

Approval Team Configuration:

Team Details:

Team Name: Production-Recovery-Team
Description: External approval team for emergency backup access.

Add Team Members:

For each IAM Identity Center user created earlier:

Member 1:

User: Select first user from dropdown
Email: (auto-populated from Identity Center)
Role: Approver

Member 2:

User: Select second user
Email: (auto-populated)
Role: Approver

Member 3:

User: Select third user
Email: (auto-populated)
Role: Approver

Approval Threshold:

Minimum approvals required: 2
This means 2 out of 3 members must approve any access request
Prevents single point of failure
Balances security with availability (if 1 member unavailable, can still proceed)

Tags for Organization:

Purpose: Emergency-Recovery
Organisation: External-Authority
CriticalityLevel: High
Environment: Production

Click Create approval team

***Document Approval Team ARN ***

After creation:

Copy Approval Team ARN (format: arn:aws:backup:us-east-1:account:approval-team/Production-Recovery-Team)
Store in password manager
You will need this ARN for disaster recovery
Without this ARN, cannot request or grant access during emergency

Team Activation Process

Automatic Invitation Emails Sent:

Immediately after approval team creation, each member receives:

From: AWS Backup Multi-Party Approval
Subject: [ACTION REQUIRED] Join Approval Team: Production-Recovery-Team

You've been invited to join an approval team for emergency 
backup access.

Team: Production-Recovery-Team
Your Role: Approver
Approval Threshold: 2 of 3 members must approve

CRITICAL REQUIREMENTS:
• ALL team members must accept within 24 hours
• If ANY member declines, team becomes inactive
• MFA is REQUIRED (already configured during setup)
• You must be available 24/7 for emergency approvals

To Accept Invitation:
1. Click invitation link: [Link expires in 24 hours]
2. Sign in with IAM Identity Center credentials
3. Complete MFA verification
4. Review team details
5. Accept membership

Approval Portal: https://backup-approvals.aws.amazon.com/

Questions? Contact your security team immediately.

Each Team Member Must:

Click invitation link (within 24 hours)
Sign in to IAM Identity Center:
- Username: (their username)
- Password: (their password)
- MFA Code: (6-digit code from authenticator app)
Review approval team invitation details:
- Team name: Production-Recovery-Team
- Team purpose: Emergency backup access approval
- Minimum approvals: 2 of 3
- Responsibilities: 24/7 availability, out-of-band verification
Click Accept invitation
Confirm understanding of role and responsibilities

Test Approval Portal Access

Each Member Should Independently Test:

Access Approval Portal:

Navigate to: https://backup-approvals.aws.amazon.com/
Sign in with IAM Identity Center credentials
Complete MFA verification
Should see: Approval Portal Dashboard

Document Portal Access:

Record in documentation:

Portal URL: https://backup-approvals.aws.amazon.com/
All 3 members tested access successfully: Yes
Date tested: [today’s date]
Next access test: [quarterly drill date]

Your organisation now has:

External approval account (completely separate from production organization)
IAM Identity Center configured with strong MFA enforcement
Three trusted approval team members with active accounts and MFA
Approval team created and fully activated
All members can access approval portal
Approval team ARN documented in secure offline location

Cross-Account Integration

Cross-Account Resource Sharing

AWS Resource Access Manager (RAM)

AWS service for secure resource sharing between accounts
Works across AWS Organizations and external accounts
Provides audited, secure sharing
Required for external approval team integration

The Integration Flow:


External Approval Account (has approval team)
↓
[Share via AWS RAM]
↓
Production Backup Account (has air-gapped vault)
↓
[Associate approval team with vault]
↓
Vault requires approval team authorization for access

Share Approval Team via AWS RAM

In External Approval Account:

Sign in to external approval account
Navigate to AWS Resource Access Manager (RAM) service

Create Resource Share:

Click Create resource share

Step 1 – Resource Share Details:

Name: Backup-Approval-Team-Share
Description: Shares Production-Recovery-Team with production backup account for emergency vault access authorization

Step 2 – Select Resources to Share:

Resource type: Select Backup: Approval Team
Select resources: Check Production-Recovery-Team
This is the approval team created in Phase 3

Step 3 – Grant Access to Principals:

Critical Setting:

Principal type: Select AWS account
Enter AWS account ID: (your production backup account ID)
Allow external principals: ✓ CHECK THIS BOX
- This is critical for cross-organization sharing
- Without this, sharing will fail
- Required because approval account is outside your organization

Step 4 – Add Tags:

Purpose: Emergency-Recovery-Authorisation
TargetAccount: (backup account ID)
SecurityLevel: Critical
SharedResource: ApprovalTeam

Review all settings
Click Create resource share

Resource Share Status:

Status: Pending (waiting for acceptance by backup account)
Share ARN: (document this for reference)
Shared resources: 1 (approval team)
Principals: 1 (backup account)

Accept Resource Share in Backup Account

Switch to Production Backup Account:

Sign in to production backup account
Region: eu-west-2 (or your backup region)
Navigate to AWS Resource Access Manager (RAM)

View Pending Invitations:

Click Shared with me in left navigation
Click Resource shares tab
You should see:
- Resource share name: Backup-Approval-Team-Share
- Status: Pending
- Shared from: (external approval account ID)
- Resources: 1 Backup approval team
- Received: (timestamp)

Accept the Resource Share:

Select the resource share
Click Accept resource share button
Confirmation dialog appears
Read the confirmation message
Click Accept

Verification After Acceptance:

Status changes from Pending to Active
Resource share now appears under “Accepted” section
Shared resources become available to use

Verify Approval Team Accessibility

In Production Backup Account:

Navigate to AWS Backup service
Region: eu-west-2 (your backup region)
Click Approval teams in left navigation menu

You Should Now See:


Approval Teams

Name: Production-Recovery-Team
Shared From: [External Account ID]
Status: Active
Members: 3
Minimum Approvals: 2 of 3
Type: External (Cross-Account)

This Confirms:

Cross-account sharing successful
Approval team accessible from production account
Ready to be assigned to air-gapped vault
External approval team will protect your backups

If Approval Team Doesn’t Appear:

Verify resource share was accepted (check AWS RAM)
Confirm you’re in correct region (eu-west-2 or your backup region)
Wait 2-3 minutes for propagation
Check AWS Service Health Dashboard for any issues

Create and Apply Vault Access Policy

Purpose: Configure vault to require external approval for all restore operations.

In AWS Backup Console:

Navigate to Backup vaults
Select Production-AirGapped-Vault
Click Access policy tab
Click Edit policy

Select Policy Builder Option:

Create policy with three statements:

Statement 1 – Allow Backup Operations:

Effect: Allow
Principal: Service → backup.amazonaws.com
Actions:
- backup:CopyIntoBackupVault (allows copy jobs)
Resources: * (all)
Conditions: None

Statement 2 – Require Approval for Restore:

Effect: Allow
Principal: * (anyone)
Actions:
- backup:StartRestoreJob (restore operations)
- backup:GetRecoveryPointRestoreMetadata (restore metadata)
Resources: * (all)
Condition (Critical):
- Condition key: backup:ApprovalTeamArn
- Operator: String equals
- Value: (paste your approval team ARN from Phase 3)

Statement 3 – Deny Dangerous Operations:

Effect: Deny
Principal: * (anyone, including root)
Actions:
- backup:DeleteRecoveryPoint (prevent backup deletion)
- backup:DeleteBackupVault (prevent vault deletion)
- backup:PutBackupVaultAccessPolicy (prevent policy modification)
Resources: * (all)
Conditions: None

Review all three statements
Click Save policy

What This Policy Enforces:

✓ Allows: AWS Backup service to copy backups into vault (normal operations) ✓ Requires: External approval team approval for any restore operation ✗ Denies: Anyone (including root) from deleting backups or vault ✗ Denies: Anyone from modifying this vault access policy (prevents tampering)

Policy Protection:

Once applied, this policy cannot be modified without:

Deleting and recreating vault (impossible due to compliance lock)
Or having specific IAM permissions (which should be tightly controlled)

Associate Approval Team with Vault

In Vault Configuration:

While viewing Production-AirGapped-Vault
Click Approval team tab
Click Assign approval team

Association Settings:

Select approval team: Production-Recovery-Team (from external account)
Review association notice:
- “This approval team is from an external account”
- “All vault access will require approval from this team”
- “Minimum 2 of 3 approvals required”

Confirm you understand implications
Click Assign approval team

Confirmation After Assignment:

You should see:


Approval Team Assignment

Team Name: Production-Recovery-Team
Team ARN: arn:aws:backup:us-east-1:[EXTERNAL-ACCOUNT]:approval-team/...
Status: Assigned
Shared From: [External Account ID]
Members: 3
Minimum Approvals: 2 of 3
Type: External (Cross-Organization)
Assignment Date: [Today's Date]

Document Complete Configuration:

Record all details in secure documentation:


Production Air-Gapped Vault - Complete Configuration
═══════════════════════════════════════════════════

Vault Information:
─────────────────
Name: Production-AirGapped-Vault
ARN: arn:aws:backup:eu-west-2:[BACKUP-ACCOUNT]:backup-vault:Production-AirGapped-Vault
Region: eu-west-2
Account: [Backup Account ID]

Security Configuration:
──────────────────────
Compliance Lock: Active (Permanent)
Lock Date: [Date]
Minimum Retention: 30 days
Maximum Retention: 365 days
Encryption: Custom KMS Key

Approval Team:
─────────────
Name: Production-Recovery-Team
ARN: arn:aws:backup:us-east-1:[EXTERNAL-ACCOUNT]:approval-team/Production-Recovery-Team
Location: External Account (Outside Organization)
Account: [External Approval Account ID]

Team Members:
────────────
1. [Name] ([Title]) - [Email] - [Phone]
2. [Name] ([Title]) - [Email] - [Phone]
3. [Name] ([Title]) - [Email] - [Phone]

Approval Requirements:
─────────────────────
Minimum Approvals: 2 of 3
MFA Required: Yes (all members)
Out-of-Band Verification: Required before approval

Access Details:
──────────────
Approval Portal: https://backup-approvals.aws.amazon.com/
Emergency Contact Card: [Physical Location]
Vault ARN Document: [Physical Safe Location]

Integration Status:
──────────────────
RAM Resource Share: Active
Approval Team Assignment: Complete
Access Policy: Applied and Enforced
Testing: Completed Successfully

Verification:
────────────
Configuration Date: [Date]
Verified By: [Your Name]
Next Review: [Quarterly Review Date]
Last Tested: [DR Drill Date]

Your organisation now has:

Approval team shared from external account via AWS RAM
Production backup account accepted and integrated shared team
Air-gapped vault configured with comprehensive access policy
Policy enforces multi-party approval requirement from external team
Approval team assigned to vault
Integration verified through comprehensive testing
Complete configuration documented

Your backups are now protected by multiple security layers:

Immutable compliance-mode locks (cannot be deleted by anyone)
Logical air-gapping (isolated from your production environment)
External approval authority (outside your organisation)
Multi-party approval requirement (2+ trusted individuals)
MFA enforcement (all approvers)
Complete CloudTrail audit trail

Even if ransomware attackers compromise your entire AWS organisation with root access, they can’t access or delete your immutable backups without approval from external team members who are completely outside the compromised environment.

Conclusion

You have now successfully implemented enterprise-grade ransomware protection.

Your organisation is now protected against:

Ransomware attacks targeting backups
Insider threats attempting backup deletion
Administrative account compromise
Organisational-level security breaches
Accidental deletion of critical backups

This implementation guide is designed to be used by any organisation implementing AWS Backup for VMware with air-gapped vaults and external approval teams. All examples should be adapted to your specific environment, account IDs, and organisational requirements.

The post How to Protect VMware VMs from Ransomware with AWS Backup first appeared on Allaboutcloud.co.uk.

How to detect Forest fires using Kinesis Video Streams and Amazon Rekognition

Nikitas Gargoulakis — Wed, 05 Jun 2024 22:01:55 +0000

Introduction

On a hot summer night, while we were enjoying our food and drinks, the dogs suddenly began barking and staring at a certain direction. We got outside to have a better look and noticed that the sky had started to turn orange. We immediately knew what it was happening, there was a huge fire at a beautiful forest a few miles away. This was happening almost every summer, at different places, wiping out forests and destroying homes, with a massive impact on the environment and people's lives.

Having seen the aftermath and the years it took for the burnt areas and people to recover, I decided to build something to detect smoke and fire and help reduce the destructive impact. After all, early detection plays a crucial role when it comes to forest fires.

Challenges

Waiting for a Real-time scenario, like the one described above, was not an option or desirable for testing my solution. To overcome this challenge, i decided to simulate the required conditions.

I used my laptop and played YouTube videos of forest fires as the source. This approach allowed me to consistently recreate the visual characteristics of forest fires, use specific scenes, thus ensuring that my solution was tested thoroughly under different conditions. This approach provided a reliable and efficient way to validate my solution and demonstrate how it could possibly handle similar real-time scenarios.

Prerequisites

Here is a brief overview of the AWS services and components used in the solution:

RTSP Camera
An IP/CCTV camera

Raspberry Pi
This acts as a local gateway to connect the camera and manage the video stream up to Amazon Kinesis Video Streams. It is using certificates generated by AWS IoT Core to authenticate itself securely to AWS services.

AWS IoT
Set up an IoT Thing to represent my IP camera. This involved configuring the certificates and policies for secure communication between the IP camera and AWS IoT. It is an important component in creating a secure and manageable architecture for streaming video from an RTSP camera through a Raspberry Pi to Kinesis Video Streams.

Kinesis Video Stream KVS
Kinesis Video Stream to ingest live video from the RTSP camera (with a matching name to the IoT Thing).

Amazon Rekognition
Trained a Rekognition Custom Labels model to detect smoke and fire in images. Training takes some time, depending on the size of the dataset. (The ARN is used in Lambda functions).

S3
Created an S3 bucket to store the extracted images from the IP camera, with the appropriate bucket policies to allow read/write access from the AWS services used.

Lambda
Wrote a Lambda function to processes images stored in S3, detect smoke and fire using Rekognition, and trigger an SNS notification.

SNS
If smoke or fire is detected by the Rekognition Custom Labels model, the Lambda function triggers a notification using Amazon Simple Notification Service (SNS). SNS can then deliver the notification to subscribed endpoints, such as email, SMS, or mobile push notifications.

IAM Roles
Created the required IAM roles and policies for Kinesis Video Streams, Rekognition, Lambda, IoT, S3, and SNS. As per best practices, least privilege principles were applied.

Producer SDK - GStreamer plugin
The GStreamer plugin for Kinesis Video Streams is a component that integrates GStreamer with Amazon Kinesis Video Streams.

Solution Overview and walkthrough

Here is a brief overview about how the solution works.

The first thing to do is to start the Amazon Rekognition Model that we trained.

Next, we need to setup the RTSP camera and test the stream, using VLC. Then we move on and configure the GStreamer plugin in the Raspberry-Pi.

We have to transfer the certificates to the Raspberry Pi and place them in a specific directory.

Obtain the IoT credential endpoint using AWS CloudShell or awscli:

aws iot describe-endpoint --endpoint-type iot:CredentialProvider

The next step is to set the environment variables for the region, certificate paths, and role alias:

export AWS_DEFAULT_REGION=eu-west-1
export CERT_PATH=certs/certificate.pem.crt
export PRIVATE_KEY_PATH=certs/private.pem.key
export CA_CERT_PATH=certs/AmazonRootCA1.pem
export ROLE_ALIAS=CameraIoTRoleAlias
export IOT_GET_CREDENTIAL_ENDPOINT=cxxxxxxxxxxs.credentials.iot.eu-west-1.amazonaws.com

Now we can execute the GStreamer command and start streaming to Kinesis Video Streams:

./kvs_gstreamer_sample FireDetection rtsp://username:password@192.168.1.100/stream1

With the video feed successfully streaming to Kinesis Video Streams, it's time to start extracting the images from the stream.

Kinesis Video Streams simplifies this process by automatically transcoding and delivering images. It extracts images from video data in real-time based on tags and delivers them to a specified S3 bucket.
To use that feature, we need to create a JSON file named update-image-generation-input.json with the required config.

{
 "StreamName": "FireDetection",
 "ImageGenerationConfiguration":
 {
  "Status": "ENABLED",
  "DestinationConfig":
  {
   "DestinationRegion": "eu-west-1",
   "Uri": "s3://images-bucket-name"
  },
  "SamplingInterval": 200,
  "ImageSelectorType": "PRODUCER_TIMESTAMP",
  "Format": "JPEG",
  "FormatConfig": {
                "JPEGQuality": "80"
       },
  "WidthPixels": 1080,
  "HeightPixels": 720
 }
}

and run the following command in awscli

aws kinesisvideo update-image-generation-configuration \
--cli-input-json file://./update-image-generation-input.json \

If we check our S3 bucket we can see the extracted images

Our Lambda function is now going to be triggered and will start processing them using Amazon Rekognition. This allows for identifying smoke/fire objects within the images and triggering notifications based on detected objects.

Conclusion

We now have a solution where our IP camera streams video to a Kinesis Video Stream. AWS Lambda processes frames from this stream, using Amazon Rekognition Custom Labels to detect smoke and fire. Detected events are then triggering SNS.
By integrating Amazon Rekognition with custom labels, Kinesis Video Streams, S3, and AWS IoT, we can create a powerful image recognition system for many use cases.

For a more detailed walkthrough, feel free to contact me.

Amazon Forecast - Empowering Accurate and Data-Driven Predictions

Nikitas Gargoulakis — Mon, 03 Jul 2023 18:37:20 +0000

Amazon Forecast is a fully managed service that utilizes machine learning algorithms to generate highly accurate forecasts. It helps businesses to make informed decisions by predicting future outcomes based on historical data. Whether you are planning inventory levels, optimizing resource allocation, or predicting customer demand, Amazon Forecast provides the tools to derive actionable insights and improve decision-making.

Configuring Amazon Forecast

To effectively configure Amazon Forecast, you need to follow these steps:

Data Preparation: Prepare your historical data, ensuring it meets the required format for Amazon Forecast. This includes having a time series dataset with timestamped data points and associated values. Amazon Forecast supports CSV, JSON, and Parquet file formats.
Create a Dataset: In the Amazon Management Console, create a dataset in Amazon Forecast. Specify the dataset type, schema, and import the historical data prepared in the previous step.
Choose a Predictor: Select an appropriate predictor from the available algorithms in Amazon Forecast. AWS offers a range of algorithms, including ARIMA, Prophet, DeepAR+, and others, each suited for specific use cases. Consider the nature of your data and the desired forecast horizon when choosing a predictor.
Train the Model: Initiate the training process by providing the dataset and predictor configuration to Amazon Forecast. The service automatically trains the model using machine learning techniques to learn patterns and relationships within the data.
Evaluate and Fine-tune: After training the model, evaluate its performance using statistical metrics such as Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE). Fine-tune the model by adjusting hyperparameters, choosing different algorithms, or modifying the dataset to improve forecast accuracy.
Generate Forecasts: When you are satisfied with the model's performance, use Amazon Forecast to generate forecasts for future time periods. These forecasts provide valuable insights for planning, decision-making, and resource allocation.

Use Cases and Advantages of Amazon Forecast

Amazon Forecast finds application in a wide range of industries and use cases. Let's explore some common scenarios where Amazon Forecast can be leveraged and the advantages it brings:

Use Case 1: Retail and Demand Planning

In the retail industry, accurately predicting customer demand is crucial for optimizing inventory levels, managing supply chains, and reducing costs. With Amazon Forecast, retailers can analyze historical sales data, account for seasonality and trends, and generate accurate demand forecasts. This enables proactive inventory management, reduces stockouts, and improves overall operational efficiency.

For instance, a large e-commerce platform can leverage Amazon Forecast to predict demand for specific products during peak shopping seasons like Black Friday. By using historical sales data, customer behavior patterns, and external factors, such as marketing campaigns and economic indicators, the platform can optimize inventory, allocate resources efficiently, and ensure a seamless shopping experience for customers.

Use Case 2: Energy and Capacity Planning

Energy providers face the challenge of accurately forecasting energy demand to optimize capacity planning, resource allocation, and pricing strategies. Amazon Forecast assists in analyzing historical energy consumption patterns, weather data, and market conditions to predict future energy demand accurately. This enables energy companies to optimize their generation, transmission, and distribution operations, ensuring efficient utilization of resources and reducing costs.

For example, a renewable energy company can leverage Amazon Forecast to predict electricity demand in a particular region. By considering historical energy consumption, weather forecasts, and upcoming events, the company can optimize the generation mix, schedule maintenance activities, and manage energy contracts effectively, leading to cost savings and improved reliability.

Use Case 3: Financial Planning and Revenue Forecasting

In the financial industry, accurate revenue forecasting and financial planning are critical for strategic decision-making and investment strategies. Amazon Forecast allows financial institutions to analyze historical revenue data, market trends, economic indicators, and customer behavior to predict future revenue accurately. This helps organizations make data-driven decisions, allocate resources effectively, and adapt to market changes.

For instance, a fintech startup can utilize Amazon Forecast to forecast monthly revenue based on historical transaction data and user activity. By accurately predicting revenue streams, the startup can make informed decisions regarding marketing budgets, product development, and expansion plans, ensuring sustainable growth and profitability.

Custom Case

A few years ago during the COVID outbreak, i spent some time testing the service. At first, I created the relevant historical data sets.

Then started testing all the algorithms and training the models.

At the end, the predictions matched closely those published by the health authorities.

Amazon Forecast Algorithms

Amazon Forecast provides a range of advanced algorithms to cater to different use cases and data characteristics. Let's briefly explore some of the key algorithms available:

ARIMA (AutoRegressive Integrated Moving Average): A widely used algorithm that models the time series data by considering the auto-regressive, integrated, and moving average components. ARIMA is effective for data with linear trends and seasonal patterns.
Prophet: Developed by Facebook, Prophet is a powerful algorithm that incorporates seasonality, holidays, and trend components in its forecasts. It handles missing data and outliers effectively, making it suitable for datasets with irregularities.
DeepAR+: DeepAR+ is a deep learning algorithm that utilizes recurrent neural networks (RNNs) to capture complex patterns in time series data. It can handle long-term dependencies and non-linear relationships, making it suitable for datasets with significant variations and multiple seasonalities.

Every algorithm possesses unique strengths and is tailored to specific use cases and as result, it is recommended to experiment with multiple algorithms and fine-tune them based on the characteristics of your data to achieve the most accurate forecasts.

Conclusion

By configuring Amazon Forecast and leveraging its range of algorithms, organizations can make accurate predictions, optimize operations, and make data-driven decisions.

Whether it's demand planning in retail, capacity optimization in energy, or revenue forecasting in finance, Amazon Forecast empowers businesses to harness the power of predictive analytics. The step-by-step configuration process, along with a variety of algorithms, ensures flexibility and accuracy in generating forecasts tailored to your specific use cases.

If you require additional guidance on configuring and utilising Amazon Forecast, feel free to reach out. I can assist you with unlocking the potential of Amazon Forecast and driving informed decision-making through accurate predictions.

Useful Resources

Amazon Forecast
Amazon-Forecast-samples on Github. Information about workshops, useful code and samples, to get you started.

Secure Remote Access - EC2 Instance Connect Endpoint

Nikitas Gargoulakis — Mon, 19 Jun 2023 20:13:21 +0000

AWS recently launched a new feature called Amazon EC2 Instance Connect (EIC) Endpoint

EIC Endpoint provides a secure solution to connect to your instances via SSH or RDP in private subnets without IGWs, public IPs, agents, and bastion hosts. By configuring an EIC Endpoint for your VPC, you can securely connect using your existing client tools or the Console/AWS CLI.

Connect to private EC2 instances through an EIC Endpoint - Image Copyright AWS

In this post i am going to show you how you can create an EIC Endpoint and connect to an instance in a Private subnet, by using the AWS console and AWS CLI.

Create the EC2 Instance Connect (EIC) Endpoint

Login to the AWS Console and Click on VPC. Then at the menu on the left, click Endpoints and then on Create Endpoint

In the next screen select the Instance Connect Endpoint option, your VPC, Security Group and Subnet.

When done click on Create Endpoint

Wait a few minutes, then hit refresh on the next screen. Your Endpoint should be now shown as Available.

If you prefer to create it using the AWS CLI, run the following command and replace SUBNET and SG-ID.

aws ec2 create-instance-connect-endpoint \
    --subnet-id [_SUBNET_] \
    --security-group-id [_SG-ID_]

Connect to your instance through AWS Console

For the purpose of this tutorial, i have created an EC2 instance in a Private Subnet

Click on Connect

Select Connect using EC2 Instance Connect Endpoint and then pick your Endpoint from the list. Next click Connect

You have now successfully connected to your instance

Connect using the AWS CLI

This option requires some extra steps. At first you need to attach a policy to your user. You can use an AWS Managed one, to start and test the service.

But for best practises and security you can refer to this link about how to create a custom policy.
Once done you can proceed.

To connect to your instance from the AWS CLI, you can run the following command where [INSTANCE] is the instance ID of your EC2 instance:

aws ec2-instance-connect ssh --instance-id [INSTANCE]

The new EC2 Instance Connect Endpoint feature has been added to AWS CLI v2.12.0. If you are having issues, you just need to update your AWS CLI to the latest version.

Conclusion

EC2 Instance Connect Endpoint offers several significant benefits to remote access management. As we can see, it eliminates the need to manage SSH key pairs manually, reducing the chances of key exposure or unauthorised access. Additionally, it allows you to grant temporary access to users by specifying the duration of their access, adding an extra layer of security. With EC2 Instance Connect Endpoint, you can also audit and track all remote access requests for compliance and governance purposes in CloudTrail.

You can read more about this great feature at this AWS post:Secure Connectivity from Public to Private: Introducing EC2 Instance Connect Endpoint

Deploy Amazon Workspaces using Service Catalog

Nikitas Gargoulakis — Thu, 15 Jun 2023 20:29:22 +0000

In this post we are going to discuss about Amazon Workspaces and how you can automate the deployment.

But first let's have a brief introduction about that service.

Amazon WorkSpaces is a cloud-based virtual desktop service that allows you to provision virtual desktops in the cloud and access them from anywhere. It provides a fully managed, secure, and scalable desktop computing environment without the need for you to manage any hardware or software and you can access the desktop from any supported device.

WorkSpaces requirements

In order to deploy Amazon Workspaces a few things need to be in place.

Active Directory to authenticate users and provide access to their WorkSpace. This can be AWS Managed Microsoft AD or On-premises AD. Or you can use an AWS AD Connector that will act as a proxy service for an existing Active Directory. If you're using AWS Managed Microsoft AD or Simple AD, your directory can be in a dedicated private subnet, as long as the directory has access to the VPC where the WorkSpaces are located. (To allow WorkSpaces to use an existing AWS Directory Service, you must first register it with WorkSpaces. After you register a directory, you can start launching WorkSpaces.)
VPC You’ll need a minimum of two subnets for an Amazon WorkSpaces deployment because each AWS Directory Service construct requires two subnets in a multi-AZ deployment.

For more details about the requirements and deployments scenarios, you may refer to this link:

Best Practices for Deploying Amazon WorkSpaces

Assumptions

In this guide we are going to focus on Automating the Workspaces deployment and AD configuration is out of scope. We are going to consider that AD and users are already configured.

Directory Registration

The first step is to register the Directory in Amazon Workspaces.
In the AWS Console click on Workspaces and then Directories, on the left.
Select your Directory, Click on Actions and then Register

Now, you have to select 2 subnets in your Workspaces VPC and click on Register again.

The Directory Registration process has begun and few minutes later the Registered status will be shown as True.

Service Catalog Configuration

Clone the following Github repo to your PC.
Amazon Workspaces.
It contains 2 files:

workspaces.yaml
sc-workspaces.yaml

Update the required values in workspaces.yaml (pDirectory, pUsername, pEncryptionKey, pWorkstationType) and then upload it, in your artefacts bucket (or a S3 bucket of your choice)
Update sc-workspaces.yaml with the S3 URL for that file
In AWS console, navigate to Cloudformation and deploy sc-workspaces.yaml

When deployment is complete, you are going to have a new Portfolio and Product in the Service Catalog.

Workspaces Deployment

Now you are ready to deploy your first Workspace by using SC.
Under Products, select Workspaces and lick on Launch Product

Select your product version (There will be just one. More will be visible if you update the CF template in the future)

Fill any required values and click Launch Product
(In WorkSpace User field enter the AD username of the Workspace owner. That user must exist in AD)

In the next screen you can now see that Service catalog has started provisioning your Workspace.

You can also check the progress in Cloudformation

Wait for a few minutes and then in AWS Console, click on Workspaces. Your newly provisioned workspace will now be visible

Click on the workspace to view it's details and take a note of the Registration Code, as you are going to need it at the next step

Connect to your Workspace

Download the Amazon Workspaces Client
Run the client, enter the Registration Code and click Continue

Now enter the AD Username and Password and click on Sign In

You have now successfully logged in your Amazon Workspace

Terminate your Workspace

In Service Catalog click on Provisioned Products.
Select the Workspace that you want to Terminate
Click on Actions and select Terminate

How to implement a Mesh Network on AWS

Nikitas Gargoulakis — Tue, 11 Apr 2023 20:07:20 +0000

It was sometime ago, that i was working in a complex Greenfield project.

We had to design a secure infrastructure (in many aspects), make sure that all traffic was encrypted at Rest and in Transit and deploy a large number of services in AWS. While the Dev teams were working on building the applications, i was focusing on those requirements.

The main requirement, was to design and implement a flat Mesh Network on AWS (with encrypted traffic). All servers deployed, should have a point-to-point connection to every other peer in the network. On top of that, some servers hosted on Azure and GCP should be able to join the Mesh. And to add more complexity, external clients like Laptops or Mobile Phones should be able to access securely, specific servers/services in the Mesh.

After gathering all the information and speaking with a number of people to make sure that all requirements were properly documented and added to the backlog, it was time to start working for the PoC (Proof of Concept)

One of the tools that seemed like the right candidate was Wireguard.

What is Wireguard

WireGuard is a secure network tunnel, operating at layer 3, implemented as a kernel virtual network interface for Linux. It utilizes state-of-the-art cryptography and it aims to be faster, simpler, leaner, and more useful than IPsec. Wireguard is currently under heavy development, but already it might be regarded as the most secure, easiest to use, and simplest VPN solution in the industry. That’s why a lot of VPN service providers have started to using it..
Source: wireguard.com

After reading the documentation and running some tests, i decided to proceed with that. The process of setting up a Wireguard as a VPN is straight forward. You have to install it, generate the required keys, create a wg0.conf file for each server, configure the relevant Security groups on AWS and your VPN is up and running quickly.

But in our case, we had to build a Mesh consisting of 100s of servers, most of them being part of AutoScaling groups. As a result, we didn’t have the option to configure wg0.conf manually, every time a server had to join the mesh.

How Wireguard works

WireGuard works by encrypting the connection using a pair of cryptographic keys, each server needs to have it’s own private and public keys and then exchange public keys with the rest.

The wg0.conf file contains all the necessary configuration parameters for the WireGuard interface

Here are some of the main parameters that can be configured in wg0.conf:

PrivateKey: This parameter defines the private key for the WireGuard interface. It is used to authenticate and encrypt traffic between peers.
ListenPort: This parameter defines the port that WireGuard will listen on for incoming connections ( default is UDP 51820).
Address: This parameter defines the IP address and subnet mask for the WireGuard interface.
Peer: This parameter defines the configuration for a peer on the WireGuard network. It includes the public key of the peer, its IP address, allowed IPs (the IP ranges that the peer can access), and other options such as endpoint configurations.

Mesh Solution Overview

I started working on my Spike and it was a challenge to find the best way to implement such a Mesh topology. At first i deployed a number of EC2 instances, by using Terraform, in multiple AWS regions. After that i was going through my list and started building and trying things:

Terraform and Ansible: Successfully created a Mesh, but it was really difficult to manage any new peers and auto update the wg0.conf when they joined. Came to the conclusion that it was fine for static setups but not for dynamic.
Terraform, Hashicorp Vault and a ton of bash scripts: That looked promising, let’s see how it works. When connecting nodes via wireguard, each node has to know the public key and endpoint ip of all peers. In this scenario, nodes with proper authentication in Vault were allowed to publish their own data and also to read connection data from other peers. They could all read the meeting point data for our mesh (data structure containing basic information about our mesh network), publish their own configuration to vault, query vault for other nodes known to the meeting point and add a wireguard peer for each of them. Although it worked, it was really complex to support it and troubleshoot, especially after the handover.

Then i came across a tool called Netmaker. It was at the early stages of development but looked really promising (Since then, i have tested all versions, including the current one 0.18.5 that was released a few days ago, with big improvements and fixes).

What is Netmaker

Netmaker is a platform for creating fast and secure virtual networks with WireGuard. It is a tool for creating and managing virtual overlay networks. If you have at least two machines with internet access that you need to connect with a secure tunnel or thousands of servers spread across multiple locations or cloud providers, Netmaker is the perfect “tool”. It connects machines securely, wherever they are.
Source: Netmaker.org

Now, after this intro, let’s see how we can create a secure Mesh Network on AWS using Netmaker and Wireguard.

How to Install Netmaker

Start by Launching a VM with Ubuntu 20.04 or latest with a public IP. (Ubuntu is the one currently supported)
Open ports 443, 80, and 51821-51830 (UDP) on the security group. You can make this range smaller, but keep in mind that you need have a port for each network you create. (I ‘am going to explain more about Networks later)
Run the following script:

sudo wget -qO /root/nm-quick-interactive.sh https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick-interactive.sh && sudo chmod +x /root/nm-quick-interactive.sh && sudo /root/nm-quick-interactive.sh

You need to answer a number of simple questions and at the the end you are going to presented with the login URL.

After typing the URL, you are going to be asked to create a username and password and when you login this is what you are going to see.

Create a network

The first thing we have to do afterwards, is to create a Network and enter the IP ranges that our servers would use for secure cross-communication. (Wireguard interface wg0, is going to use an IP address for that range)

Click the ‘Networks’ tile on the dashboard, or in the left navigation panel click ‘Networks’.

On the Networks screen, click on the ‘Create Network’ button.

Give you network a name, and then enter your preferred CIDR. Or click on the ‘Autofill’ button and then change the name and the CIDR generated by the autofill option.

Create the Keys

Then proceed by creating the required keys. When done, we can see that there multiple ways to add a peer to our Mesh Network

Create and configure the Nodes

Most of the hard work is done. And now it’s time to launch a few instances in AWS in multiple regions and spread them across Public and Private subnets. In our case almost all instances are in Private subnets, with the exception of Netmaker server and Azure instance.

I like to use Terraform with Gitlab Runners for my test deployments and for this demo i had about 10 EC2 instances up and running really fast (Was using spot instances to minimise costs). Just remember that you need to deploy a standalone (on-demand) EC2 instance for Netmaker.

All the the Security Groups, for the Nodes, were configured to allow incoming traffic (UDP) to ports 51820–51830.

And with with the help of User Data and the command shown below, we can configure the nodes to join the Mesh during the launch process.
(Need to replace eyJzZXJxxxyxxxxxxxxxxxxxxxxcccccccccccvvvvvvv0000000== with your token)

#!/bin/bash

sudo curl -Lo /etc/yum.repos.d/wireguard.repo https://copr.fedorainfracloud.org/coprs/jdoss/wireguard/repo/epel-7/jdoss-wireguard-epel-7.repo
sudo yum install epel-release
sudo amazon-linux-extras install -y epel && yum install -y wireguard-dkms wireguard-tools
curl -sL 'https://rpm.netmaker.org/gpg.key' | sudo tee /tmp/gpg.key
curl -sL 'https://rpm.netmaker.org/netclient-repo' | sudo tee /etc/yum.repos.d/netclient.repo
sudo rpm --import /tmp/gpg.key
sudo yum check-update
sudo yum install -y netclient
netclient register -t eyJzZXJxxxyxxxxxxxxxxxxxxxxcccccccccccvvvvvvv0000000==

After a few minutes we have our instances up and running, fully configured with Wireguard and Netclient (All of the them have automatically joined our Mesh network).

Now let’s launch one more server but this time in… Azure

Time to check our Netmaker GUI and make sure that all nodes have joined. If they don’t show immediately, there is no need to worry. It could take up to 5 mins to show up. In our case all Nodes are now visible with a Healthy status.

At this point we have successfully deployed and configured a flat Mesh network, not only between AWS instances but also with a server in a different cloud provider. All traffic between them is encrypted in transit, by using Wireguard.

Mesh Graph / Visualisation

Let’s see how our Mesh Network looks like at this stage

Wireguard Mesh

Netmaker server used as Ingress Node

Test our Mesh Network

How about running some tests to confirm that everything is working as expected?

Access Control Lists

By default, Netmaker creates a “full mesh,” meaning every node in our network can talk to every other node. But there is a nice feature that you can use in order to enable/disable any peer-to-peer connection in the network.

The ACL feature can be accessed by either clicking on “ACLs” in the sidebar, or by clicking on a Node in the Node List.

Add External Clients

There are cases that external clients need to access some services running in the nodes. That can be a Mobile phone, a laptop/tablet or an IoT device.

We can achieve that by creating an Ingress. (And once connected to the Ingress, we can reach all servers in the network.)

The next step is to generate the client configs. Clients can then join our mesh, either by scanning a QR code or by importing the Wireguard config (Please note, that Wireguard client must be installed in the mobile, laptop etc)

In our case i have download the config in my laptop and have connected using the Wireguard client.

For this demo, i have installed Apache in an AWS EC2 instance and in an Azure VM. As you can see, i can access both from my laptop, through a secure tunnel, using the 10.141.x.x IPs ( Mesh network CIDR)

Apache running on AWS EC2

Apache running on Azure instance

Conclusion

This is just a use case of using Netmaker and Wireguard to create a secure Mesh Network on AWS. There are more as you can see below and we are going to discuss some of them in future posts.

Automate the creation of a large WireGuard-based (Mesh) network
Secure access to a home or office network
Provide remote access to resources like an AWS VPC, or K8S cluster
Create clusters that span environments
Remotely access a cluster from an external source
Remotely access an external source from a cluster
Manage a secure mesh of IoT devices

Hope you found this post useful. Feel free to reach to me for any questions.

Useful links:
Wireguard
Netmaker