Forem: William Mukoyani

Building a Serverless Data Analytics Pipeline with AWS: Premier League Dashboard

William Mukoyani — Tue, 16 Dec 2025 15:51:11 +0000

Inspired by AWS Cookbook by John Culkin & Mike Zazon - Chapter 7: Big Data

My Journey into Data Analytics

While exploring AWS AI/ML services, I realized that artificial intelligence and machine learning are fundamentally built upon quality data foundations. This insight led me to step back and master the data analytics fundamentals first. What better way than to build a complete serverless pipeline?

In this post, I'll share how I created a scalable analytics solution using AWS S3, Athena, and QuickSight to analyze Premier League data.

Why Premier League data? As a passionate football enthusiast, I'm fascinated by the rich statistical narratives that unfold each season. Every match generates meaningful data points, from goals and assists to tactical formations and player performance metrics. This abundance of structured, real-world data makes football analytics an ideal playground for learning data engineering concepts while working with something I genuinely care about.

What we'll build:

Serverless data storage with Amazon S3
SQL querying with Amazon Athena
Interactive dashboards with Amazon QuickSight

⚠️ Disclaimer: The Premier League data used in this project is completely fictional and for demonstration purposes only. If you see Manchester City with 150 points or Tottenham actually winning something, that's just my creative data generation at work! 😄 Please don't use this for your fantasy football decisions - you've been warned! For real Premier League data, check the official sources (and prepare for more realistic disappointment).

Architecture Overview

This serverless architecture eliminates infrastructure complexity while providing:

Scalability: Automatic scaling without server management
Cost-efficiency: Pay-per-query pricing model
Speed: Query results in seconds

Implementation Highlights

1. S3 Data Lake Setup

I stored Premier League CSV files in S3, creating a scalable data foundation:

# Create bucket and upload data
aws s3api create-bucket --bucket premier-league-data-$(openssl rand -hex 3)
aws s3 cp data/ s3://your-bucket/raw-data/ --recursive

2. Athena SQL Querying

Created External Tables:
Athena's power lies in querying data directly from S3 without moving it. Here's how I created the tables:

-- Create standings table
CREATE EXTERNAL TABLE IF NOT EXISTS standings (
    team_name STRING,
    matches_played INT,
    wins INT,
    draws INT,
    losses INT,
    goals_for INT,
    goals_against INT,
    goal_difference INT,
    points INT
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 's3://your-bucket/raw-data/'
TBLPROPERTIES ('skip.header.line.count'='1');

-- Create match results table
CREATE EXTERNAL TABLE IF NOT EXISTS match_results (
    team_name STRING,
    result_type STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 's3://your-bucket/raw-data/'
TBLPROPERTIES ('skip.header.line.count'='1');

Sample Analytics Queries:

Once tables were created, I ran insightful queries:

-- Top 6 teams analysis
SELECT team_name, points, goal_difference 
FROM standings 
ORDER BY points DESC 
LIMIT 6;

-- Win percentage calculation
SELECT team_name, 
       ROUND((wins * 100.0 / matches_played), 2) as win_percentage
FROM standings
ORDER BY win_percentage DESC;

-- Verify data integrity
SELECT *  FROM standings;
SELECT *  FROM match_results;

Key Athena Benefits:

No data movement required
Standard SQL interface
Pay only for data scanned ($5/TB)
Results in seconds

3. QuickSight Dashboards

Built interactive visualizations, including:

League standings table
Points comparison charts
Goal difference analysis
Team performance metrics

Business Value for Management

QuickSight delivers immediate ROI through:

Decision Speed: Real-time dashboards eliminate waiting for IT reports
Cost Savings: $9/user vs $70+ for traditional BI tools like Tableau
Self-Service Analytics: Business users create their own insights without technical dependencies
Mobile Access: Executive dashboards available anywhere, anytime
Scalability: Handles 10 users or 10,000 users with the same architecture
Security: Enterprise-grade AWS security and compliance built in

ImageSource amazon.com QuickSight page

Management Benefits:

Reduce reporting cycle from weeks to minutes
Democratize data access across all departments
Lower total cost of ownership by 60-80% vs traditional solutions
Eliminate server maintenance and upgrade costs

Results & Insights

Cost Breakdown:

S3 Storage: ~$0.05/month
Athena Queries: ~$0.25/month
QuickSight: $9/user/month

Total: ~$9.30/month for enterprise-grade analytics!

Key Learnings:
✅ Setup completed in under 2 hours
✅ Serverless = zero infrastructure management
✅ SQL familiarity accelerated development
⚠️ QuickSight permissions required for initial troubleshooting

Next Steps

This foundation opens doors to:

Real-time data integration
Machine learning predictions
Advanced ETL pipelines with AWS Glue

Final Reflections

Starting with data fundamentals before diving into AI/ML proved invaluable. This serverless analytics pipeline demonstrates that powerful data solutions don't require complex infrastructure - just the right AWS services working together.

The S3 + Athena + QuickSight combination delivers enterprise-grade analytics at startup costs, making it perfect for both learning and production use cases.

Resources

GitHub Repository: AWS-Analytics Project
AWS Cookbook: Chapter 7 - Big Data
AI-Powered BI Tool: Amazon Quick Sight
Athena SQL: Amazon Athena
Serverless S3: Amazon S3

Building your own data pipeline? Connect with me on LinkedIn I'd love to hear about your experience!

GitHub Actions for Modern Deployments

William Mukoyani — Thu, 11 Dec 2025 18:15:04 +0000

Published: December 11, 2024 • 6 min read

Introduction

GitHub Actions has revolutionized how we approach Continuous Integration and Continuous Deployment (CI/CD). With its native integration into GitHub repositories and powerful workflow capabilities, it has become the go-to choice for modern deployment pipelines.

What's the Business Value

Accelerated Development Velocity

Faster Release Cycles: Automated pipelines reduce deployment time from hours to minutes
Reduced Manual Bottlenecks: Eliminate waiting for manual approvals and deployments
Parallel Development: Multiple teams can work simultaneously without deployment conflicts
Quick Feedback Loops: Developers get immediate feedback on code quality and functionality

Quality Assurance & Risk Reduction

Automated Testing: Catch bugs before they reach production, reducing customer impact
Security Scanning: Identify vulnerabilities early in the development process
Consistent Deployments: Eliminate human error through standardized deployment processes
Rollback Capabilities: Quickly revert problematic releases to maintain service availability

Cost Efficiency

Reduced DevOps Overhead: Less manual intervention means lower operational costs
Infrastructure Optimization: Automated scaling and resource management reduce cloud costs
Developer Productivity: Teams spend more time building features, less time on deployment tasks
Reduced Downtime: Automated monitoring and rollbacks minimize revenue-impacting outages

Competitive Advantage

Market Responsiveness: Deploy features and fixes faster than competitors
Innovation Focus: Engineering teams can focus on product development rather than operations
Customer Satisfaction: Reliable deployments lead to better user experiences
Scalability: Handle increased demand without proportional increases in operational complexity

In this comprehensive guide, we'll explore how to build robust, secure, and efficient deployment pipelines using GitHub Actions.

Understanding GitHub Actions

Core Concepts

Workflows: Automated processes defined in YAML files
Jobs: Groups of steps that execute on the same runner
Steps: Individual tasks within a job
Actions: Reusable units of code
Runners: Servers that execute workflows

Workflow Structure

name: CI/CD Pipeline
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: npm test

Building a Complete CI/CD Pipeline

1. Code Quality and Testing

name: Quality Checks
on: [push, pull_request]

jobs:
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run linting
        run: npm run lint

      - name: Run tests
        run: npm test -- --coverage

      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage/lcov.info

2. Security Scanning

  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Snyk to check for vulnerabilities
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'

3. Build and Package

  build:
    needs: [lint-and-test, security]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Build application
        run: npm run build

      - name: Upload build artifacts
        uses: actions/upload-artifact@v4
        with:
          name: build-files
          path: dist/

Docker Integration

Building and Pushing Images

  docker:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: myapp/web
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=sha,prefix={{branch}}-

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Multi-Environment Deployments

Environment-Specific Workflows

name: Deploy to Environments
on:
  push:
    branches:
      - develop    # Deploy to staging
      - main       # Deploy to production

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
    steps:
      - name: Deploy to ${{ github.ref == 'refs/heads/main' && 'Production' || 'Staging' }}
        run: |
          echo "Deploying to ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}"
          # Deployment commands here

Using Environment Secrets

  deploy-to-aws:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-west-2

      - name: Deploy to S3
        run: |
          aws s3 sync ./dist s3://${{ secrets.S3_BUCKET_NAME }} --delete
          aws cloudfront create-invalidation --distribution-id ${{ secrets.CLOUDFRONT_ID }} --paths "/*"

Advanced Patterns

Matrix Builds

  test-matrix:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [16, 18, 20]

    steps:
      - uses: actions/checkout@v4
      - name: Setup Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm test

Conditional Deployments

  deploy:
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    needs: [test, build]
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to production
        run: echo "Deploying to production"

Reusable Workflows

# .github/workflows/reusable-deploy.yml
name: Reusable Deploy
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      deploy-token:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - name: Deploy
        run: echo "Deploying to ${{ inputs.environment }}"

# .github/workflows/main.yml
jobs:
  deploy-staging:
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
    secrets:
      deploy-token: ${{ secrets.STAGING_TOKEN }}

Infrastructure as Code Integration

Terraform Deployment

  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.6.0

      - name: Terraform Init
        run: terraform init
        working-directory: ./infrastructure

      - name: Terraform Plan
        run: terraform plan -no-color
        working-directory: ./infrastructure

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve
        working-directory: ./infrastructure

AWS CDK Deployment

  cdk-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'

      - name: Install CDK
        run: npm install -g aws-cdk

      - name: Install dependencies
        run: npm ci
        working-directory: ./infrastructure

      - name: CDK Deploy
        run: cdk deploy --require-approval never
        working-directory: ./infrastructure

Monitoring and Notifications

Slack Notifications

  notify:
    runs-on: ubuntu-latest
    if: always()
    needs: [test, build, deploy]
    steps:
      - name: Slack Notification
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          channel: '#deployments'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}

Custom Metrics

  metrics:
    runs-on: ubuntu-latest
    steps:
      - name: Send deployment metrics
        run: |
          curl -X POST ${{ secrets.METRICS_ENDPOINT }} \
            -H "Content-Type: application/json" \
            -d '{
              "deployment": {
                "status": "success",
                "duration": "${{ github.event.head_commit.timestamp }}",
                "commit": "${{ github.sha }}"
              }
            }'

Security Best Practices

Secret Management

# Use environment-specific secrets
- name: Deploy
  env:
    API_KEY: ${{ secrets.API_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}
  run: ./deploy.sh

OIDC Authentication

permissions:
  id-token: write
  contents: read

steps:
  - name: Configure AWS credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
      aws-region: us-west-2

Performance Optimization

Caching Strategies

- name: Cache dependencies
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

Parallel Jobs

jobs:
  test:
    runs-on: ubuntu-latest
    # ... test steps

  lint:
    runs-on: ubuntu-latest
    # ... lint steps

  security:
    runs-on: ubuntu-latest
    # ... security steps

  deploy:
    needs: [test, lint, security]
    runs-on: ubuntu-latest
    # ... deploy steps

Troubleshooting Common Issues

Debug Mode

- name: Enable debug logging
  run: echo "ACTIONS_STEP_DEBUG=true" >> $GITHUB_ENV

Artifact Debugging

- name: Upload logs
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: debug-logs
    path: |
      logs/
      *.log

Summary

GitHub Actions provides a powerful, flexible platform for building modern CI/CD pipelines. By following the patterns and practices outlined in this guide, you can create robust deployment workflows that scale with your team and projects.

Key takeaways:

Start simple and iterate
Use reusable workflows and actions
Implement proper security practices
Monitor and optimize performance
Plan for failure scenarios

The future of software deployment is automated, secure, and fast. GitHub Actions gives you the tools to achieve all three.

References

GitHub Actions Documentation - GitHub Actions Doc
GitHub Actions Marketplace - Marketplace
GitHub Actions Workflow Syntax - GitHub Actions Workflow
GitHub Actions Security Hardening - GitHub Actions Security Guides
Continuous Delivery: Reliable Software Releases - Jez Humble & David Farley, Addison-Wesley
The DevOps Handbook - Gene Kim, Jez Humble, Patrick Debois, John Willis
GitHub Actions Runner Images - GitHub Actions Runner Images
Awesome GitHub Actions - Awesome Actions
GitHub Actions Toolkit - Actions Toolkit
Act: Run GitHub Actions Locally - Act
GitHub Actions Best Practices - Essential Features of Github Actions

Ready to optimize your deployment pipeline? Check out my other articles on Infrastructure as Code and cloud best practices.

Terraform Best Practices for Production

William Mukoyani — Thu, 11 Dec 2025 18:00:07 +0000

Published: December 11, 2025 • 4 min read

Introduction

Terraform has become the de facto standard for Infrastructure as Code (IaC) across multiple cloud providers. However, moving from simple proofs of concept to production-ready infrastructure requires following established best practices and patterns.

Business Value

Cost Optimization

Multi-Cloud Strategy: Avoid vendor lock-in and leverage competitive pricing across providers
Resource Lifecycle Management: Automatically destroy unused resources to minimize waste
Infrastructure Standardization: Reduce operational overhead through consistent deployment patterns
Cost Visibility: Track infrastructure costs through consistent tagging and resource grouping

Risk Mitigation

Infrastructure Drift Detection: Identify and correct configuration drift before it causes issues
Disaster Recovery: Rapidly rebuild infrastructure in alternate regions or providers
Change Management: Review and approve infrastructure changes through code review processes
Compliance Automation: Ensure security policies are consistently applied across all environments

Business Agility

Faster Market Entry: Deploy new products and features with infrastructure provisioned in minutes
Developer Self-Service: Enable development teams to provision their own environments safely
Experimentation: Quickly spin up test environments for proof-of-concepts and A/B testing
Global Expansion: Replicate infrastructure patterns across multiple regions and markets

Project Structure

Recommended Directory Layout

terraform/
├── environments/
│   ├── dev/
│   ├── staging/
│   └── prod/
├── modules/
│   ├── vpc/
│   ├── ec2/
│   └── rds/
├── shared/
│   └── variables.tf
└── README.md

Environment Separation

Keep environments completely separate:

# environments/prod/main.tf
module "vpc" {
  source = "../../modules/vpc"

  environment = "prod"
  cidr_block  = "10.0.0.0/16"

  tags = {
    Environment = "production"
    Project     = "my-app"
  }
}

State Management

Remote State Storage

Always use remote state for production:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

State Locking

Implement state locking to prevent concurrent modifications:

resource "aws_dynamodb_table" "terraform_locks" {
  name           = "terraform-locks"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

Module Design

Input Variables

Define clear, well-documented variables:

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"

  validation {
    condition = contains([
      "t3.micro", "t3.small", "t3.medium"
    ], var.instance_type)
    error_message = "Instance type must be t3.micro, t3.small, or t3.medium."
  }
}

Output Values

Expose necessary information:

output "vpc_id" {
  description = "ID of the VPC"
  value       = aws_vpc.main.id
}

output "private_subnet_ids" {
  description = "IDs of private subnets"
  value       = aws_subnet.private[*].id
}

Security Best Practices

Sensitive Data Management

Never hardcode secrets:

# ❌ Bad
resource "aws_db_instance" "main" {
  password = "hardcoded-password"
}

# ✅ Good
resource "aws_db_instance" "main" {
  manage_master_user_password = true
}

IAM Policies

Follow the least privilege principle:

data "aws_iam_policy_document" "app_policy" {
  statement {
    effect = "Allow"
    actions = [
      "s3:GetObject",
      "s3:PutObject"
    ]
    resources = [
      "${aws_s3_bucket.app_bucket.arn}/*"
    ]
  }
}

Resource Naming and Tagging

Consistent Naming

locals {
  name_prefix = "${var.project}-${var.environment}"

  common_tags = {
    Project     = var.project
    Environment = var.environment
    ManagedBy   = "terraform"
  }
}

resource "aws_vpc" "main" {
  cidr_block = var.cidr_block

  tags = merge(local.common_tags, {
    Name = "${local.name_prefix}-vpc"
  })
}

Version Management

Provider Versions

Pin provider versions:

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

Module Versions

Use semantic versioning for modules:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  # module configuration
}

Testing and Validation

Pre-commit Hooks

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.83.5
    hooks:
      - id: terraform_fmt
      - id: terraform_validate
      - id: terraform_docs

Automated Testing

# test/vpc_test.go
func TestVPCModule(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../modules/vpc",
        Vars: map[string]interface{}{
            "cidr_block": "10.0.0.0/16",
        },
    }

    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)

    vpcId := terraform.Output(t, terraformOptions, "vpc_id")
    assert.NotEmpty(t, vpcId)
}

CI/CD Integration

GitHub Actions Workflow

name: Terraform
on:
  push:
    branches: [main]
  pull_request:

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: 1.6.0

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        run: terraform plan -no-color

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve

Monitoring and Observability

Resource Tagging for Cost Tracking

locals {
  cost_tags = {
    CostCenter = var.cost_center
    Owner      = var.owner
    Purpose    = var.purpose
  }
}

CloudWatch Integration

resource "aws_cloudwatch_log_group" "app_logs" {
  name              = "/aws/lambda/${local.name_prefix}"
  retention_in_days = var.log_retention_days

  tags = local.common_tags
}

Performance Optimization

Parallel Execution

resource "aws_instance" "web" {
  count = var.instance_count

  ami           = data.aws_ami.ubuntu.id
  instance_type = var.instance_type

  tags = merge(local.common_tags, {
    Name = "${local.name_prefix}-web-${count.index + 1}"
  })
}

Resource Dependencies

resource "aws_security_group" "web" {
  name_prefix = "${local.name_prefix}-web"
  vpc_id      = aws_vpc.main.id
}

resource "aws_instance" "web" {
  depends_on = [aws_internet_gateway.main]

  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  vpc_security_group_ids = [aws_security_group.web.id]
}

Common Pitfalls to Avoid

Hardcoding Values: Use variables and data sources
Large State Files: Break into smaller, focused modules
No State Locking: Always implement state locking
Ignoring Drift: Regularly run terraform plan to detect drift
Poor Documentation: Document modules and complex configurations

Parting Thoughts

Following these best practices will help you build maintainable, secure, and scalable Terraform infrastructure. Remember that Infrastructure as Code is not just about automation, it's about bringing software engineering practices to infrastructure management.

Start implementing these practices incrementally in your existing projects, and make them standard for all new infrastructure deployments.

References

Terraform Documentation - Terraform Docs
Terraform Best Practices Guide - Terraform Best Practices
HashiCorp Learn Terraform - Learn Hashicorp Terraform
Terraform AWS Provider Documentation - AWS Provider Doc
Terraform: Up & Running - Yevgeniy Brikman, O'Reilly Media
Terratest: Automated Testing for Infrastructure - Terratest Gruntwork IO
Terraform Registry - Terraform Registry
Terraform Cloud Documentation - Terraform Cloud Doc
Pre-commit Terraform Hooks - Terraform Hooks
Checkov: Static Analysis for Terraform - CheckOv

Want to learn more about cloud infrastructure? Check out my articles on AWS CloudFormation and CI/CD best practices.

Getting Started with AWS CloudFormation

William Mukoyani — Thu, 11 Dec 2025 17:39:55 +0000

Published: December 11, 2025 • 3 min read

Introduction

Infrastructure as Code (IaC) has revolutionized how we manage and deploy cloud resources. AWS CloudFormation stands as one of the most powerful tools in this space, allowing developers and DevOps engineers to define their entire infrastructure using declarative templates.

What is AWS CloudFormation?

AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so you can spend less time managing those resources and more time focusing on your applications.

Business Value

Cost Reduction

Reduced Manual Errors: Eliminate costly mistakes from manual infrastructure provisioning
Faster Time-to-Market: Deploy environments in minutes instead of days or weeks
Resource Optimization: Automatically terminate unused resources to prevent cost overruns
Predictable Budgeting: Template-based deployments provide consistent cost estimates

Operational Efficiency

Team Productivity: DevOps teams can focus on innovation rather than repetitive tasks
Compliance Assurance: Built-in governance ensures all deployments meet security standards
Disaster Recovery: Rapidly recreate entire environments from templates during outages
Audit Trail: Complete change history for regulatory compliance and troubleshooting

Scalability Benefits

Multi-Region Deployment: Easily replicate infrastructure across global regions
Environment Parity: Ensure development, staging, and production environments are identical
Rapid Scaling: Automatically provision resources based on demand patterns

Why CloudFormation

Consistency: Deploy identical infrastructure across multiple environments
Version Control: Track changes to your infrastructure over time
Rollback Capability: Automatically rollback failed deployments
Cost Management: Easily estimate costs and track resource usage

CloudFormation Template Structure

AWSTemplateFormatVersion: '2010-09-09'
Description: 'A sample CloudFormation template'

Parameters:
  # Input parameters for the template

Resources:
  # AWS resources to create

Outputs:
  # Values to return after stack creation

Your First CloudFormation Template

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Simple S3 bucket creation'

Parameters:
  BucketName:
    Type: String
    Description: Name for the S3 bucket
    Default: my-cloudformation-bucket

Resources:
  MyS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      VersioningConfiguration:
        Status: Enabled

Outputs:
  BucketName:
    Description: 'Name of the created S3 bucket'
    Value: !Ref MyS3Bucket

Actionable Tips

Use Parameters and Mappings - Make templates flexible and reusable
Implement Proper Naming - Use consistent naming conventions
Add Documentation - Include detailed descriptions
Use Cross-Stack References - Export values between stacks
Implement Rollback Triggers - Set up CloudWatch alarms

Deployment Strategies

Blue-Green Deployments

Resources:
  BlueEnvironment:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: blue-targets

Rolling Updates

Resources:
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: 1
        MaxBatchSize: 2

Wrapping it Up

AWS CloudFormation brings consistency, reliability, and scalability to infrastructure management. Start small with simple templates and gradually build complexity as you become more comfortable with CloudFormation's capabilities.

References

AWS CloudFormation Documentation - Cloudformation Doc
AWS CloudFormation Template Reference - AWS CloudFormation template reference
AWS CloudFormation Best Practices - UserGuide Best Practices
AWS Well-Architected Framework - Well Architected
Infrastructure as Code: Managing Servers in the Cloud - Kief Morris, O'Reilly Media
AWS CloudFormation Sample Templates - AWS labs CloudFormation Templates
AWS CloudFormation Linter (cfn-lint) - AWS CloudFormation cfn-lint
AWS CloudFormation Guard - CloudFormation Guard

Ready to dive deeper into AWS infrastructure? Check out my other articles on Terraform and AWS best practices.

Getting Ready for Online Tech Jobs: What You Need to Know

William Mukoyani — Tue, 29 Jul 2025 15:07:37 +0000

How to prepare, position, and power your way into the global tech workforce

The global tech industry is evolving, and so is how we work. Remote opportunities are no longer the exception; they're fast becoming the standard. At the recent Online Work Africa Summit, I had the opportunity to speak on a topic close to my heart: helping aspiring tech professionals navigate the remote job landscape.

Whether you're just getting started or pivoting from another field, this blog recaps key takeaways from that talk and gives you practical steps to kickstart or boost your online tech career.

Understanding the Online Tech Job Landscape

Online tech work isn’t limited to just developers in Silicon Valley. Today, companies around the world are hiring talent based on skills, not geography.

Here are some in-demand remote-friendly roles:

Development: Frontend, backend, mobile apps, full-stack
Cloud & Infrastructure: AWS, Azure, GCP, DevOps
Data & AI: Data analysts, data engineers, machine learning practitioners
Cybersecurity: Ethical hackers, cloud security engineers, SOC analysts
User Experience: UI/UX designers, product managers
IT Operations & Support: IT helpdesk, network admins

Tip: Start by identifying which area matches your strengths or interests, then go deep on the skills needed for that path.

Skills You Need to Thrive

Succeeding in an online tech job requires more than just technical know-how. It’s a blend of hard skills, soft skills, and remote readiness.

Technical Skills:

Coding: Python, JavaScript, Java, SQL
Cloud platforms: AWS, Azure, Google Cloud
Data literacy: Excel, SQL, visualization tools, ML basics
Security awareness: Basic security hygiene and tooling

Soft Skills:

Communication: Clear, concise, and proactive messaging
Time Management: Prioritizing and tracking tasks without micromanagement
Adaptability: Tech changes fast, so should you

Remote Skills:

Self-direction, accountability, asynchronous collaboration (Slack, Zoom, Trello)
Ability to work across time zones

Being great remotely is a skill in itself. Treat it as part of your professional growth plan.

Building an Online Presence That Gets You Noticed

When you're applying for remote roles, your digital footprint is your first impression. Here's how to make it shine:

GitHub: Share projects, contribute to open source, document your code clearly
LinkedIn: Use relevant keywords, post consistently, and engage with tech communities
Personal Portfolio: A simple website with your skills, resume, and past work adds huge credibility. A tech community I'd recommend Developers to check out is AWS Community Builders.

Hiring managers will Google you. Make sure they find something valuable.

The Power of Certifications (Backed by Real Skills)

Certifications can help you signal capability, especially when you're early in your journey or changing fields. Some standout options include:

Cloud:

AWS Certified Cloud Practitioner (great for beginners)
AWS Solutions Architect Associate (for aspiring cloud engineers)
Google Cloud Digital Leader
Microsoft Azure Fundamentals

Data & AI:

Security:

CompTIA Security+
AWS Security Specialty

Remember, certs are useful when backed by hands-on practice. Combine learning with labs, projects, or internships.

Where to Find Remote Tech Jobs

You don’t need to wait for opportunities to come to you; know where to look:

Remote Job Boards:

Freelancing Platforms:

Look for clients who value long-term relationships and have verified payment methods. Start small, build reviews, and grow gradually.

Resume & Interview Tips for Remote Roles

Your resume should showcase more than just technical skills. Show that you're remote-ready:

Mention tools like Slack, Zoom, GitHub
Highlight remote work experience or team collaboration
Link to your portfolio and GitHub
Prepare for video interviews and live coding sessions (HackerRank, CodeSignal, etc.)

Remote interviews are often asynchronous or over video, practice speaking, sharing your screen, and managing your environment.

Common Challenges in Remote Tech Work

Working online isn’t all sunshine and flexibility. Here are real challenges to prepare for:

Isolation & Burnout: Build routines and connect with online communities
Distractions: Design a focused work environment
Connectivity: Invest in stable internet and backups (like mobile hotspots, UPS)
Time Zones: Learn to collaborate asynchronously and manage calendars smartly

Create Your Personal Growth Plan

Set yourself up for long-term success by having a roadmap:

Define 3–6 month goals (example, finish a course, build a project, apply to X jobs)
Join tech communities (example, Slack groups, Discord, Twitter/X, Meetup)
Track your learning and reflect regularly

“Consistency beats intensity. Progress compounds over time.”

You’re Closer Than You Think

The future of work is already here. Companies want talent, not geography. If you’re willing to learn, build, and show up, there’s space for you in the global tech economy.

Whether you're in Nairobi, Accra, Kigali, or anywhere else you belong in tech. Your skills, story, and effort matter.

Bonus Resources

Here are a few resources I mentioned during the session:

AWS Skill Builder
AWS Educate
Coursera
FreeCodeCamp
Buildspace
LinkedIn Learning
Kaggle

I’d love to hear your thoughts, whether it’s a question, insight, or your own experience. Let’s connect on LinkedIn and keep the conversation going!

“Opportunities are global. Your skills can be too.” – William Mukoyani

image source: Behance.net (Kelly KTKL & Travis Hunt)

Educating Machines.

William Mukoyani — Thu, 16 Jan 2025 06:01:06 +0000

Teaching Machines Intelligence

We live in a time where we can teach machines to think and learn. It’s not just about technology; it’s about changing how we work and live. But what does it mean to teach machines? And why is it so important?

Teaching machines refers to training algorithms and models to recognize patterns, make decisions, and solve problems using data. This process, known as machine learning (ML), forms the backbone of modern AI applications, allowing machines to perform tasks once thought exclusive to human intelligence, from recommending your next favorite song on Spotify, YouTube, Apple Music, and Amazon Music to detecting fraudulent transactions. Educated machines are impacting every aspect of our lives.

The Inspiration Behind Educating Machines

This blog was inspired by a book I've been reading by Professor John C. Lennox 2084, my experiences attending a Machine Learning cohort as an AWS Authorized Instructor, leading the AWS Usergroup Kampala boot camp, where I had the opportunity to present Machine Learning on AWS, and supporting an AWS Generative AI Workshop Pre-summit Masterclass in the Connected Africa Summit 2024 with a focus on Amazon Bedrock for building and scaling generative AI Applications with foundational models and Amazon Q Developer for increased developer productivity. It was amazing to see how passionate people are about learning! and got to play around with PartyRock, an Amazon Bedrock playground. Here are artistic images generated by Partyrock Futuristic Renaissance Robot and Grand Canyon

Interestingly, deciding on the title for this blog was almost as challenging as training an ML model, which proves that creativity and technology share more similarities than we might think!

The Foundations of Machine Learning

Let's break down the basics of machine learning. To educate a machine, we need three things:

Data: This is what machines learn from. Machines rely on vast datasets to recognize patterns and build intelligence.
Algorithms: The teaching methods that guide machines in interpreting the data. These algorithms define how machines learn and adapt.
Models: The result of this education is machines that can perform tasks like predicting outcomes, identifying images, or generating text.

Think of it as teaching a student. Data acts as the curriculum, algorithms are the teaching style, and the model is the educated student ready to tackle real-world problems. So how does all this begin?

How Machines Learn

The process of educating machines is a mix of art and science. It begins with data and culminates in models ready to perform specific tasks. Here are the key steps involved:

Data Collection: For a machine to learn effectively, it needs high-quality, diverse datasets as it's widely stated The more the data the better the AI. For example, training a fraud detection model requires transactional data with clear examples of fraudulent and legitimate activities.
Preprocessing: Raw data is messy. Preprocessing involves cleaning, transforming, and organizing data into a format suitable for training. This step ensures the machine learns from accurate and meaningful information.
Training: Algorithms are applied to the data to create a model. During this phase, the machine identifies patterns and relationships in the data, learning to perform specific tasks.
Testing: Education isn’t complete without evaluation. Machines are tested using unseen data to measure their performance and ensure they generalize well to new scenarios.
Deployment and Retraining: The real world is ever-changing, and so is data. Once deployed, models require continuous monitoring and retraining to stay relevant and accurate.

As Dr. Werner Vogels aptly puts it, "Everything fails all the time." Even educating machines comes with its fair share of challenges!

Challenges in Educating Machines

While the steps seem straightforward, the process comes with its challenges. Data bias, for instance, can lead to biased outcomes, raising ethical concerns. If a dataset lacks diversity, the model might struggle with inclusivity, leading to unintended consequences. Overfitting is another common challenge, where a model learns too much from the training data and fails to perform well in the real world.
Another significant factor is computational power. Training sophisticated models often requires substantial resources, such as high-performance GPUs or cloud-based solutions like Amazon SageMaker AI , which provide scalable infrastructure for machine learning. Overcoming these hurdles requires careful planning, innovation, and collaboration.

Real-World Applications

Educated machines are reshaping industries across the globe. Here are some examples:

Fraud Detection: Banks use ML models to spot fraudulent activities in real-time. For instance, a SageMaker project I worked on involved predicting credit fraud by leveraging patterns in transactional data to enhance security. For practice purposes, you can look into this Fraud Detection using ML on AWS
E-commerce Personalization: Platforms analyze user behavior to recommend products, creating personalized shopping experiences. Here is a service I can recommend for recommendations Amazon Personalize
Healthcare: AI-powered tools assist in diagnosing diseases, and predicting patient outcomes. Here are some notable Healthcare Solutions
Manufacturing: Predictive maintenance ensures machines are running smoothly prevents breakdowns and optimizes operations. You can gain some hands-on with Predictive maintenance.
Formula 1 Performance: Formula 1 utilizes AWS Machine Learning services to revolutionize racing. ML powers every aspect of the sport from accelerating car performance to enhancing fan engagement through real-time data insights. AWS helps the F1 broadcast team gain a clearer picture of on-track action with live driver battles, championship predictions, and top speeds. This real-time data storytelling elevates the viewing experience and showcases how ML can drive innovation and excitement. If looking to read more on the F1 Use Case F1 On AWS would be a great place to check out.

Looking into Machine learning?

Free Learning Resources

Machine learning is an exciting field! If you want to learn, there are many free resources. Below are a few recommended options:

Amazon Machine Learning University(MLU) - On YouTube is a fantastic resource for anyone looking to dive into machine learning concepts, regardless of their experience level.
Coursera (Machine Learning by Andrew Ng) - One of the most famous and beginner-friendly courses in machine learning, Andrew Ng’s course offers a solid foundation. It covers algorithms, supervised and unsupervised learning, and more. It’s perfect for someone looking to understand the theory behind ML.
edX (Introduction to AI and ML) - A great option for those who prefer learning from universities. edX offers various courses on AI and machine learning, many of which are free to audit. The curriculum is designed for learners at all stages.
Kaggle - Known for its data science competitions, Kaggle is also an excellent platform for learning. It provides free access to datasets and a range of hands-on tutorials to help you practice real-world machine-learning problems.
Google AI - Google’s AI website offers various free courses and tutorials that span the basics of machine learning to advanced topics. It’s a useful starting point for those who want to explore ML concepts in depth.
AWS SageMaker Studio Lab - AWS SageMaker Studio Lab is a free, fully integrated development environment (IDE) for machine learning. It allows you to run Jupyter notebooks and develop machine-learning models without the need for a complex setup. It provides free compute resources, making it an excellent tool for practicing machine learning without incurring costs. Whether you're experimenting with algorithms or building your own models, SageMaker Studio Lab offers a low-barrier entry point into cloud-based machine learning.
AWS Live on Twitch - Where AWS occasionally streams live training and workshops on Twitch to help individuals learn about various AWS services and gain knowledge on different domains. These live sessions are a great way to engage with AWS experts in real-time and build cloud knowledge interactively.

Also if you looking to solidify your knowledge of Machine learning and put yourself out there as an ML Expert here are some ML certifications I can recommend AWS Certified Machine Learning Engineer - Associate and AWS Certified Machine Learning - Specialty.

A Note on Expertise

While I’ve shared these resources based on my own experience and research, I am by no means an expert in machine learning. This field is vast and rapidly changing, so I’m always open to hearing from experts and those with deeper insights. If you have any recommendations or insights that can further enrich this learning, feel free to share!

The Future of Educating Machines

Looking ahead, the methods and applications of machine learning will only expand. Techniques like generative AI and reinforcement learning good example AWS DeepRacer are already pushing boundaries. However, as we innovate, we must also focus on ensuring responsible and ethical learning for machines.

Data scientists and AI engineers play a key role in this. By focusing on fairness and transparency, we can create systems that help people without creating problems.

Educating machines is more than just tech; it is a partnership between human ingenuity and machine capability. As we teach machines to think, let us guide them to learn responsibly, ensuring they become tools for progress and not pitfalls of unintended consequences.

A Reflection on Humanity and Machines.

As we let machines to learn and adapt, it is worth reflecting on the mystery of human nature. Fyodor Dostoevsky once said:

"Man is a mystery. It needs to be unraveled, and if you spend your whole life unraveling it, don't say that you've wasted your time. I am studying that mystery because I want to be a human being."

Perhaps, in educating machines, we are also uncovering the intricate patterns of what it means to be human.

Amazon CloudFront

William Mukoyani — Thu, 19 Jan 2023 14:27:51 +0000

What is Amazon CloudFront?

Amazon CloudFront is a content delivery network (CDN) service provided by Amazon Web Services (AWS). It is designed to deliver content, such as videos, images, and other static files, to users with low latency and high transfer speeds.

Latency

Latency is the delay between the user's request for content and the delivery of that content from the nearest edge location.
Latency is measured in milliseconds.

It is worth noting that CDN services are continuously growing among organizations such as Netflix, Facebook, and Amazon as their major web traffic is served through CDN

Benefits of Amazon CloudFront

One of the key benefits of using CloudFront is its global network of edge locations. This network includes more than 200 points of presence (PoPs) around the world, which allows for content to be delivered quickly and efficiently to users, regardless of their location. This is particularly useful for delivering large files, such as videos, as it ensures that users can access the content with minimal buffering or waiting times.
Another benefit of CloudFront is its integration with other AWS services. For example, it can be used in conjunction with Amazon S3 to deliver content stored in S3 buckets, or with Amazon Elastic Compute Cloud (EC2) to deliver content from your own web servers. This allows for a seamless and efficient workflow for delivering content to users.

Features of Amazon CloudFront

CloudFront provides a range of features that help to ensure the security and performance of your content delivery.

It supports HTTPS, which encrypts data in transit, and it also allows you to configure custom SSL certificates. Additionally, it allows you to control access to your content by using signed URLs or signed cookies, which can be used to restrict access to specific users or groups.

Difference between signed URLs and signed cookies.

A signed URL is a special URL that includes a unique signature, which is used to authenticate the user and grant access to private content. The signature is generated using the user's security credentials and the URL's parameters. When a user requests the signed URL, CloudFront checks the signature to make sure it is valid and, if so, grants access to the private content.
Signed cookies, on the other hand, are a way to grant access to private content by including authentication information in a cookie. The authentication information is a signature that is generated using the user's security credentials and the cookie's parameters. When a user requests private content, CloudFront checks the signature in the cookie to make sure it is valid and, if so, grants access to the content.
signed URLs are more suitable for content that is available for a limited time, like access to a live event
signed cookies are more suitable for content that is intended for a specific user or group of users, like a personal dashboard.

Another important feature of CloudFront is availability and its ability to handle high-traffic loads. With CloudFront, you can scale your content delivery to handle millions of requests per second, which is particularly useful for handling traffic spikes during peak times.

How CloudFront Works

A user requests a piece of content (such as an image, video, or web page) from a website that is using CloudFront.
The request is sent to the nearest CloudFront edge location, which is determined by the global network of AWS edge locations.
The edge location checks to see if it has a copy of the requested content in its cache.
If the edge location has a cached copy of the content, it is delivered to the user.

If the edge location does not have a cached copy of the content, it sends a request to the origin server (such as an S3 bucket or EC2 instance) to retrieve the content.

The origin server sends the content back to the edge location.

The edge location caches the content and delivers it to the user.

Subsequent requests for the same content from users in the same geographical area are served from the edge location's cache, reducing the load on the origin server and improving the delivery time for the users.

It's worth mentioning that CloudFront also supports various features such as:

Customizable SSL certificate
Live streaming
Geo-restriction
DDoS protection
Customized error pages
, which allows you to customize the behavior of the content delivery network to better suit your needs.

In conclusion, Amazon CloudFront is a powerful and flexible content delivery network that allows you to deliver content to users quickly and efficiently. Its global network of edge locations, integration with other AWS services, and range of security and performance features make it an ideal solution for delivering content to users around the world. Whether you are delivering videos, images, or other types of files, CloudFront can help you to ensure that your content is delivered quickly and securely.

Amazon Athena- Serverless

William Mukoyani — Thu, 19 Jan 2023 09:31:21 +0000

What is Amazon Athena?

Amazon Athena is a serverless, interactive query service that allows you to analyze data in Amazon S3 using standard SQL. With Athena, there is no infrastructure to manage, and you pay only for the queries that you run. Since you only pay for the queries that you run, it can be a cost-effective solution for running ad-hoc queries or for data analysis projects that are not part of a regular workflow.

Athena's key feature is the ability to analyze data stored in S3 without having to move or transform it first. This makes it an ideal solution for analyzing large, raw data sets such as logs, sensor data, and other type of unstructured data.

One of the key benefits of Athena is its ability to work with a variety of data formats, including CSV, JSON, ORC, Parquet, and Avro. This makes it easy to work with data from different sources and in different formats, without the need for data transformation or ETL processes.

In addition, Athena integrates with other AWS services such as Amazon QuickSight for data visualization, Amazon Glue for data cataloging, and Amazon CloudWatch for monitoring and troubleshooting. This allows you to build a complete data analytics solution on the AWS platform.

Another great feature of Athena is its ability to work with partitioned data. This means that you can partition your data based on specific attributes, such as date or location, which improves query performance and reduces costs. This makes Athena a great option for working with large, partitioned datasets.

Athena also supports a variety of SQL functions and operators, including aggregate functions, filtering, and sorting. This makes it easy to perform complex data analysis and gain insights from your data.

How to use Amazon Athena

Set up an S3 bucket: Before you can use Athena, you will need to have your data stored in an S3 bucket. This can be done by manually uploading your data or using a service like Amazon Kinesis or Amazon Glue to automatically move your data into S3.
Create a table: Once your data is in S3, you can create a table in Athena that references the location of your data in S3. You will need to define the schema of your data, including the column names and data types. Athena supports a variety of data formats, including CSV, JSON, ORC, Parquet, and Avro.
Run queries: Once your table is set up, you can use the Athena query editor to run SQL queries against your data. You can also use the Athena API to run queries programmatically.
Analyze your data: Athena allows you to perform a wide range of data analysis tasks, including filtering, sorting, and aggregating your data. You can also use the results of your queries to create visualizations using Amazon QuickSight.
Monitor and Optimize: You can monitor the performance of your queries and optimize them using the Athena Query Monitoring and the Query Details page. You can also use the AWS Glue Data Catalog, to improve the performance and optimize the cost of your queries.

It's worth noting that Athena is a serverless service, so there are no servers or infrastructure to manage or provision. Additionally, Athena only charges you for the amount of data scanned per query, so you only pay for what you use.

Ways to Interact with Amazon Athena

It's worth noting that there are several ways to interact with Athena, you can use various AWS SDKs, the AWS Management Console, the Command line Interface, and the Athena API to interact with Athena, Additionally, you can use third-party tools such as Presto CLI or Dataform to interact with Athena.
In this post, I'll share some insights on how you can use the Command line Interface to interact with Amazon Athena.

You can access Amazon Athena using the command line interface (CLI) by using the AWS Command Line Interface (AWS CLI) tool or you can use AWS Cloudshell on the AWS Management console.
AWS Cloudshell comes with other pre-installed tools like Python, AWS CLI, Node.js etc..

Here are the steps to take when you opt to use the AWS Command Line Interface(AWS CLI) tool.

First, you will need to install the AWS CLI tool on your computer. You can do this by following the instructions on the AWS documentation
Once the AWS CLI tool is installed, you will need to configure it by running the aws configure command and providing your AWS access key and secret key.
Now you can run Athena commands using the aws athena command.

For example, to list the tables in your Athena database you can use the command:

aws athena list-tables --query 'TableList[*].Name' --output text

To run a query on Athena using the CLI, you can use the start-query-execution command. For example, to run a query that select all rows from a table named 'mytable':

aws athena start-query-execution --query-string "SELECT * FROM mytable" --result-configuration OutputLocation=s3://mybucket/queryresults/

To get the results of the query, you can use the get-query-results command. For example, to get the results of the query execution with the execution ID '7890547':

aws athena get-query-results --query-execution-id 7890547

SQL queries you run on Amazon Athena

Count the number of rows in a table: SELECT COUNT(*) FROM mytable;
Select specific columns from a table: SELECT column1, column2 FROM mytable;
Filter rows based on a specific condition: SELECT column1, column2 FROM mytable WHERE column2 > 100;
Sort the results of a query: SELECT column1, column2 FROM mytable ORDER BY column2 DESC;
Group data by a specific column: SELECT column1, COUNT(*) FROM mytable GROUP BY column1;
Join two tables: SELECT a.column1, b.column2 FROM table1 a JOIN table2 b ON a.id = b.id;
Use aggregate functions: SELECT AVG(column1), SUM(column2) FROM mytable;
Use a subquery: SELECT column1 FROM mytable WHERE column2 IN (SELECT column2 FROM mytable2);

In conclusion, if you are looking for an easy and cost-effective way to analyze large data sets in S3, Amazon Athena is an excellent option to consider. Its serverless architecture, standard SQL support, and integration with other AWS services make it a powerful and flexible solution for data analysis.