Forem: Timur Galeev

ECS vs EKS: When You DON'T Need Kubernetes - A Practical Guide to Choosing AWS Container Services

Timur Galeev — Sun, 04 Jan 2026 16:20:58 +0000

Introduction

You know what? I see teams spinning up Kubernetes clusters for three microservices all the time. Then they spend two months figuring out pods, ingress controllers, and all that magic. And then they pay $70 per month just for three clusters in different regions, not counting the actual servers.

Here's the honest truth: Kubernetes is a powerful tool but you don't always need it. Amazon ECS is a simpler alternative that handles most tasks faster and cheaper.

In this article I'll show you:

When ECS beats EKS (and saves you tons of money)
Real scenarios with numbers and examples
Ready-to-use code snippets for deploying to both platforms
How to make the decision without headaches

Let's dive in!

Quick Comparison: ECS vs EKS

First let's look at the main differences in a simple table:

Feature	AWS ECS	AWS EKS
Cluster Cost	$0	$0.10/hour (~$70/month)
Setup Complexity	Low (2-4 hours)	High (1-2 days)
Learning Curve	Few days	Several weeks
Management	AWS Console/CLI	kubectl + AWS Console
Ecosystem	AWS services	Entire Kubernetes world
Portability	AWS only	Any cloud/on-prem
Updates	Automatic	Manual (control plane)
Best For	1-10 services	10-100+ services

Architecture: How It Works

ECS Architecture:

Your Application
    ↓
Docker Image (you need this!)
    ↓
Task Definition (container description)
    ↓
ECS Service (manages launch)
    ↓
EC2 or Fargate (where it runs)
    ↓
Container running

EKS Architecture:

Your Application
    ↓
Docker Image
    ↓
Kubernetes Pod specification
    ↓
Deployment/StatefulSet
    ↓
Kubernetes Control Plane ($$$)
    ↓
Worker Nodes
    ↓
Container in Pod

See the difference? ECS has two fewer steps and each one is easier to understand.

When ECS is Your Best Choice

This is where it gets interesting. Many people think Kubernetes is always needed but that's not true. Let's break down real situations where ECS wins.

Scenario 1: Multi-Regional Deployment (3-5 Services)

Imagine: you have a simple API and a couple supporting services. You need to deploy them in three regions - Europe, Asia, USA. For redundancy, you know.

With EKS you pay:

Europe cluster: $70/month
Asia cluster: $70/month
USA cluster: $70/month
Total: $210/month just for the right to run containers

With ECS you pay:

Cluster is free: $0
Total: $0 for management

In other words, save $2,520 per year just on the control plane! And you still gotta pay for the actual servers.

Real Example

I had a project - e-commerce backend. Five services:

API Gateway (Node.js)
Order Service (Python)
Payment Service (Go)
Notification Service (Node.js)
Analytics Worker (Python)

Each service needed a Docker image. Here's a simple Dockerfile example for the Node.js API:

# Dockerfile for API Gateway
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm install --production

COPY . .

EXPOSE 3000
CMD ["node", "server.js"]

We deployed across three regions using ECS Fargate. Setup time: 4 hours including Terraform code. If we'd done it with EKS - that's minimum a week with Helm charts, ingress controllers and all that kitchen.

Here's how we defined the task in ECS (simplified):

# ECS Task Definition - just the container part
resource "aws_ecs_task_definition" "api_gateway" {
  family                   = "api-gateway"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  container_definitions = jsonencode([{
    name      = "api"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/api-gateway:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    environment = [
      { name = "NODE_ENV", value = "production" },
      { name = "PORT", value = "3000" }
    ]
  }])
}

Compare this to Kubernetes - you'd need Deployment YAML, Service YAML, maybe Ingress, ConfigMaps... it adds up.

Scenario 2: Quick Start and Simplicity

You're a startup. You have an MVP that needs to ship yesterday. Team of three people nobody knows Kubernetes deeply.

ECS gives you:

Launch in couple hours (not days!)
AWS integration out of the box
No need to hire Kubernetes expert
Less moving parts = less things break

Look I'm not saying Kubernetes is bad. It's awesome! But do you need it when you just wanna run a container? It's like buying a truck to get bread from the store.

Time to learn:

ECS: 2-3 days to work comfortably
EKS: 2-3 weeks minimum (or even a month)

Here's a complete minimal ECS setup with Terraform:

# Minimal ECS cluster
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# ECS Service - runs 2 copies of your container
resource "aws_ecs_service" "app" {
  name            = "my-app"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.api_gateway.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = ["subnet-xxx", "subnet-yyy"]
    security_groups  = ["sg-xxx"]
    assign_public_ip = true
  }
}

That's it! No Helm, no kubectl, no YAML soup.

Scenario 3: AWS-Native Project

Your project is fully in AWS:

Database - RDS
Files - S3
Queues - SQS
Cache - ElastiCache
Logs - CloudWatch

Why Kubernetes here? ECS integrates with these services natively and simpler.

Example - S3 access:

ECS Task Role (simple):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "s3:*",
    "Resource": "arn:aws:s3:::my-bucket/*"
  }]
}

Attach the role to Task Definition - done.

In EKS you do the same through IRSA (IAM Roles for Service Accounts):

Setup OIDC provider
Create ServiceAccount in Kubernetes
Link with IAM role
Annotate the pod

More steps = more places to mess up.

When EKS Becomes Necessary

Alright enough praising ECS. Let's be honest - there are situations where EKS is really better.

Scenario 1: Large Microservices Architecture (20+ Services)

When you have 20, 30, 50 microservices - that's different math.

Why EKS wins:

$70 per cluster is fixed price (whether 5 services or 50)
Kubernetes scales complexity better
Ecosystem: Helm, Operators, service mesh (Istio, Linkerd)
Centralized management of all services

Cost example:

With 30 services in one region:

ECS: 30 separate ECS Services = lots of config hard to manage
EKS: One cluster all services in namespaces manage through GitOps

Here $70/month pays for convenience.

A typical Kubernetes deployment:

# Kubernetes Deployment - simpler at scale
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-gateway
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-gateway
  template:
    metadata:
      labels:
        app: api-gateway
    spec:
      containers:
      - name: api
        image: my-registry/api-gateway:v1.2.3
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"

With Kubernetes you get built-in health checks, rolling updates, easy rollbacks.

Scenario 2: Multi-Cloud or Hybrid Infrastructure

Your company wants:

Work in AWS and GCP simultaneously
Keep some workloads on-premise
Have ability to migrate between clouds

EKS (Kubernetes) gives portability:

Same YAML manifests work everywhere
Can move applications between clouds
Standardization across all infra

ECS is AWS only. Can't move it anywhere. (ECS anywhere?! :) )

Scenario 3: Advanced Features

GPU workloads for ML/AI: EKS supports GPU nodes out of the box + all tooling like Kubeflow.

Complex networking policies: Network Policies in Kubernetes give precise traffic control between pods.

# Network Policy example
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
spec:
  podSelector:
    matchLabels:
      app: api-gateway
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 3000

Stateful applications: StatefulSets Persistent Volumes - all this works better in Kubernetes.

Practical Deployment Examples

Enough theory let's get hands dirty. I'll show how to deploy a simple application to both ECS and EKS. Same application to compare.

Our application: Nginx + simple Node.js API (both need Docker images)

Building Docker Images First

Before deploying anywhere you need Docker images. Here's our setup:

# Dockerfile for our Node.js app
FROM node:18-alpine

WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .

EXPOSE 3000
CMD ["node", "index.js"]

Build and push:

# Build image
docker build -t my-app:latest .

# Tag for ECR
docker tag my-app:latest 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

# Push to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789.dkr.ecr.us-east-1.amazonaws.com
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

ECS Deployment with Terraform

Let's start with the simpler one - ECS.

Step 1: VPC Setup

# Create VPC for containers
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true
}

# Public subnets
resource "aws_subnet" "public" {
  count                   = 2
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.${count.index}.0/24"
  availability_zone       = data.aws_availability_zones.available.names[count.index]
  map_public_ip_on_launch = true
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
}

Step 2: ECS Cluster and Service

# Create ECS cluster
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# Task Definition - describes your Docker container
resource "aws_ecs_task_definition" "app" {
  family                   = "my-app"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  execution_role_arn = aws_iam_role.ecs_execution.arn
  task_role_arn      = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([{
    name      = "app"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    logConfiguration = {
      logDriver = "awslogs"
      options = {
        "awslogs-group"         = "/ecs/my-app"
        "awslogs-region"        = "us-east-1"
        "awslogs-stream-prefix" = "app"
      }
    }
  }])
}

# ECS Service - runs and maintains containers
resource "aws_ecs_service" "app" {
  name            = "my-app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = aws_subnet.public[*].id
    security_groups  = [aws_security_group.ecs_tasks.id]
    assign_public_ip = true
  }
}

Step 3: IAM Roles

# Role for ECS to pull Docker images and write logs
resource "aws_iam_role" "ecs_execution" {
  name = "ecs-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ecs_execution_policy" {
  role       = aws_iam_role.ecs_execution.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

# Role for your application (e.g., S3 access)
resource "aws_iam_role" "ecs_task" {
  name = "ecs-task-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

Deploy It

terraform init
terraform plan
terraform apply

Done! Container is running.

EKS Deployment with Terraform

Now the same thing but in EKS.

Step 1: EKS Cluster

# EKS cluster
resource "aws_eks_cluster" "main" {
  name     = "my-eks-cluster"
  role_arn = aws_iam_role.eks_cluster.arn
  version  = "1.28"

  vpc_config {
    subnet_ids = concat(aws_subnet.public[*].id, aws_subnet.private[*].id)
  }
}

# Worker nodes
resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "main-nodes"
  node_role_arn   = aws_iam_role.eks_node.arn
  subnet_ids      = aws_subnet.private[*].id

  scaling_config {
    desired_size = 2
    max_size     = 4
    min_size     = 1
  }

  instance_types = ["t3.medium"]
}

Step 2: IAM for EKS

# Cluster role
resource "aws_iam_role" "eks_cluster" {
  name = "eks-cluster-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster.name
}

# Node role
resource "aws_iam_role" "eks_node" {
  name = "eks-node-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_worker_node" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node.name
}

resource "aws_iam_role_policy_attachment" "eks_cni" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node.name
}

Step 3: Kubernetes Manifests

After cluster is created deploy application with kubectl:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 3000

Deploy It

# 1. Apply Terraform
terraform init
terraform apply

# 2. Configure kubectl
aws eks update-kubeconfig --name my-eks-cluster --region us-east-1

# 3. Check nodes
kubectl get nodes

# 4. Deploy application
kubectl apply -f deployment.yaml

# 5. Check status
kubectl get pods
kubectl get svc

Difference:

ECS: one terraform apply and done
EKS: terraform apply + kubectl commands + wait for everything to come up

Complexity Comparison

Action	ECS	EKS
Config files	3-4 Terraform files	4-5 Terraform + YAML manifests
First deploy time	5-7 minutes	15-20 minutes
Commands to run	2 (init apply)	5+ (terraform + kubectl)
Need to know	AWS Terraform Docker	AWS Terraform Kubernetes kubectl Docker

Real Cases and Economics

Let's calculate concrete numbers for typical scenarios.

Case 1: Startup with 5 Microservices in 3 Regions

Requirements:

5 services (API Workers Background Jobs)
3 regions: US EU Asia
2 instances each service
All need Docker images built and stored in ECR

ECS Fargate:

Cluster cost: $0
ECR storage: ~$5/month (for Docker images)
Compute (Fargate):
  - 5 services × 2 instances × 3 regions = 30 tasks
  - Each task: 0.25 vCPU 512 MB
  - $0.04048/hour per vCPU $0.004445/hour per GB
  - (~0.25 × $0.04048 + 0.5 × $0.004445) × 730 hours = ~$9/task/month
  - 30 tasks × $9 = $270/month

Total: ~$275/month

EKS:

Cluster cost: $70 × 3 regions = $210/month
ECR storage: ~$5/month (same Docker images)
Compute (EC2 nodes):
  - Minimum 2× t3.medium per region = 6 instances
  - t3.medium = $0.0416/hour × 730 = ~$30/month
  - 6 × $30 = $180/month

Total: $210 + $5 + $180 = $395/month

Savings with ECS: $120/month or $1440/year

Plus with ECS you don't pay DevOps engineer to manage Kubernetes :)

Case 2: Large Project with 30 Services in 1 Region

ECS:

Cluster: $0
Management: 30 separate ECS Services (hard to manage!)
Compute: depends on load

EKS:

Cluster: $70/month
Management: One namespace GitOps Helm (easier!)
Compute: same + better resource utilization

Here EKS wins on management convenience. $70 pays for itself.

Time for Setup and Maintenance

Real numbers from my experience:

Initial setup:

ECS: 4 hours (Terraform + tests + Docker builds)
EKS: 2 days (cluster + addons + monitoring setup + Docker builds)

Weekly maintenance:

ECS: ~30 minutes (check logs updates)
EKS: ~2 hours (updates cluster checks monitoring)

Platform updates:

ECS: automatic
EKS: need to update control plane once a year (takes half a day with tests)

Decision Checklist: What to Choose?

So here's a simple flowchart for decision making:

Choose ECS if:

✅ You have less than 10-15 microservices
✅ Project is AWS only (no multi-cloud plans)
✅ Team doesn't know Kubernetes (and doesn't want to learn)
✅ Need to launch quickly (MVP startup)
✅ Budget is limited
✅ Simple application without complex dependencies
✅ Multi-regional deploy (save on clusters)
✅ Comfortable with Docker basics

Choose EKS if:

✅ More than 20+ microservices
✅ Need portability (multi-cloud hybrid)
✅ Team knows Kubernetes
✅ Need advanced features (service mesh operators)
✅ GPU workloads for ML/AI
✅ Already using Kubernetes elsewhere
✅ Complex microservices architecture
✅ Want access to Kubernetes ecosystem

Middle Ground

You can start with ECS and migrate later!

Many companies do this:

Start on ECS (fast and cheap)
Grow to 15-20 services
Kubernetes developers join the team
Gradually migrate to EKS

This is normal evolution. Don't go Kubernetes just "because it's cool".

Conclusions

Here's what's important to remember:

ECS is not a "second-rate" option. It's a full-fledged solution that handles many tasks excellently. Yes EKS is more powerful in capabilities but most projects simply don't need those capabilities.

Main ECS advantages:

Free control plane (save $70-210+ per month)
Simplicity and launch speed
Less operational overhead
Native AWS integration
Perfect for multi-regional deployment of small services

When EKS is really needed:

Large scale (20+ services)
Code portability
Advanced Kubernetes features
Already have expertise in team

My advice: Don't chase the hype. Start with ECS if the task allows. Save time money and nerves. And when you really grow into Kubernetes - then migrate.

Kubernetes is like a Ferrari - cool car but for a trip to the store a regular Toyota works fine. And uses less gas 😄

Final Thoughts

The choice between ECS and EKS isn't about "better" or "worse" - it's about right tool for the job.

Start simple. ECS lets you ship fast without the Kubernetes learning curve. Your Docker skills transfer directly. AWS handles the orchestration.

As you grow, reassess. When you hit 15-20 services or need multi-cloud, EKS makes sense. But many successful companies run production on ECS for years.

Remember: Complexity is a cost. Every abstraction layer you add costs time money and mental overhead. Sometimes the best architecture is the simplest one that works.

Both platforms use Docker. Both run containers. Both scale. The question is: how much complexity do you actually need?

Choose wisely!

Sources

Building an AI-Optimized Platform on Amazon EKS with NVIDIA NIM and OpenAI Models

Timur Galeev — Wed, 18 Dec 2024 22:46:10 +0000

Introduction

The rise of artificial intelligence (AI) has brought about an unprecedented demand for infrastructure that can handle large-scale computations, support GPU acceleration, and provide scalable, flexible management of workloads. Kubernetes has emerged as a leading platform for orchestrating these workloads, and Amazon Elastic Kubernetes Service (EKS) extends Kubernetes’ capabilities by simplifying deployment and scaling in the cloud.

NVIDIA Infrastructure Manager (NIM) complements Kubernetes by optimizing GPU workloads, a critical need for training large language models (LLMs), computer vision, and other computationally intensive AI tasks. Additionally, OpenAI models can be integrated into this ecosystem to unlock cutting-edge AI capabilities, such as text generation, image recognition, and decision-making systems.

This article provides an in-depth guide to building a complete AI platform using EKS, NVIDIA NIM, and OpenAI models, with Terraform automating the deployment. Whether you are an AI researcher or a business looking to adopt AI, this guide outlines how to build a robust and scalable platform. Complete code for this setup is available on GitHub https://github.com/timurgaleev/eks-nim-llm-openai.

Why Choose NVIDIA NIM and EKS for AI Workloads?

Challenges of AI Workloads

AI applications, especially those involving LLMs, have unique challenges:

GPU Resource Management: Training and inference rely on GPUs, which are scarce and expensive resources. Efficient allocation and monitoring are crucial.

Scalability: AI workloads often need to scale dynamically based on user demand or data processing requirements.

Storage for Large Datasets: AI models and datasets can require hundreds of gigabytes, necessitating persistent, shared, and scalable storage.

Observability: Monitoring system performance, especially GPU utilization and latency, is essential for optimizing workloads.

NVIDIA NIM: A Solution for GPU Workloads

NVIDIA NIM addresses these challenges by providing:

GPU Scheduling: Maximizes GPU usage across workloads.

Integration with Kubernetes: Leverages Kubernetes to manage pods, jobs, and resources efficiently.

AI Model Management: Simplifies deployment and scaling of AI models with Helm charts and Kubernetes CRDs (Custom Resource Definitions).

Support for Persistent Storage: Integrates with shared storage solutions like AWS EFS for storing datasets and models.

Amazon EKS: A Scalable Kubernetes Solution

Amazon EKS adds value by:

Managed Kubernetes: Reduces operational overhead by handling Kubernetes cluster setup, updates, and management.

Elastic Compute Integration: Dynamically provisions GPU-enabled instances, such as g4dn and p4d, to handle AI workloads. Ensure that your AWS account has sufficient quotas and availability for these instance types to avoid provisioning issues.

Built-in Security: Integrates with AWS IAM and VPC for secure access and network segmentation.

Together, NVIDIA NIM and Amazon EKS create a powerful platform for AI model training, inference, and experimentation.

Architecture Overview

The platform architecture integrates NVIDIA NIM and OpenAI models into an EKS cluster, combining compute, storage, and monitoring components.

Key Components

EKS Cluster: Manages Kubernetes workloads and scales GPU-enabled nodes.

Karpenter: Dynamically provisions and scales nodes (CPU and GPU) based on workload demands, optimizing resource utilization and cost.

GPU Node Groups: Nodes equipped with NVIDIA GPUs for ML and AI inference tasks.

NVIDIA NIM: Deploys GPU workloads, manages AI pipelines, and integrates with Kubernetes.

OpenAI Web UI: Provides a user-friendly interface for interacting with AI models.

Persistent Storage: AWS EFS supports shared storage for datasets and models.

Observability Tools: Prometheus and Grafana offer real-time monitoring of system metrics, including GPU utilization and pod performance.

Deployment Guide

This guide provides step-by-step instructions to deploy the architecture using Terraform. While the focus is on essential components like EKS, GPU workloads, and observability, we skip detailed VPC configuration to allow flexibility based on your specific requirements.

For a VPC example that fits this deployment, refer to the repository: https://github.com/timurgaleev/eks-nim-llm-openai.

Step 1: Provisioning the EKS Cluster

Provisioning an Amazon EKS cluster is the foundation for Kubernetes workloads. Below is the EKS Cluster Configuration with key highlights to focus on scalability, system add-ons, and Karpenter integration.

EKS Cluster Configuration

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = compact([
    for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
    substr(cidr_block, 0, 4) == "100." ? subnet_id : null
  ])

  manage_aws_auth_configmap = true
  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups = [
        "system:bootstrappers",
        "system:nodes"
      ]
    }
  ]

  eks_managed_node_group_defaults = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
    ebs_optimized = true
    block_device_mappings = {
      xvda = {
        device_name = "/dev/xvda"
        ebs = {
          volume_size = 100
          volume_type = "gp3"
        }
      }
    }
  }

  eks_managed_node_groups = {
    core_node_group = {
      name            = "core-node-group"
      description     = "EKS Core node group for hosting system add-ons"
      subnet_ids      = compact([
        for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
        substr(cidr_block, 0, 4) == "100." ? subnet_id : null
      ])
      ami_type        = "AL2_x86_64"
      instance_types  = ["m5.xlarge"]
      capacity_type   = "SPOT"
      desired_size    = 2
      min_size        = 2
      max_size        = 4
      labels = {
        WorkerType    = "SPOT"
        NodeGroupType = "core"
      }
      tags = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key Highlights

1. Networking:

Subnets are filtered to include only CIDR blocks starting with 100. to ensure specific subnet assignment for nodes.

2. IAM and Auth:

Integration with Karpenter is configured via the aws_auth_rolesblock, allowing Karpenter to dynamically provision nodes.

3. Managed Node Groups:

Core Node Group:
- Optimized for system-level workloads.
- Configured with m5.xlarge spot instances for cost efficiency.
- Labels such as NodeGroupType: core and taints can be used to restrict workloads to this node group.

4. Storage:

Nodes are configured with gp3 root volumes (100 GiB) for system usage. Additional storage for workloads should be configured separately.

5. Scaling:

Use Karpenter for workload-based scaling instead of additional managed node groups. The eks_managed_node_groups block here is only for critical system workloads.

Step 2: Deploying NVIDIA NIM for AI Workloads

Deploying NVIDIA NIM (NVIDIA Inference Manager) requires configuring persistent storage for large datasets and allocating GPU resources for optimal performance. Here's an expanded guide breaking down the essential steps.

1. Persistent Storage with AWS EFS

AI workloads often require storage that exceeds local node capacity. AWS EFS (Elastic File System) provides a shared and scalable storage solution across multiple pods. Below is the configuration for creating a Persistent Volume Claim (PVC) backed by EFS:

Code: Persistent Volume Claim (PVC)

kubernetes_persistent_volume_claim_v1 "efs_pvc" {
  metadata {
    name      = "efs-storage"
    namespace = "nim"
  }
  spec {
    access_modes       = ["ReadWriteMany"] # Enables sharing storage across multiple pods.
    storage_class_name = "efs"             # Links the PVC to an EFS storage class.
    resources {
      requests = {
        storage = "200Gi" # Reserves 200 GiB of scalable storage.
      }
    }
  }
}

Key Points:

Access Mode: "ReadWriteMany" allows simultaneous access by multiple pods, critical for parallel workloads.
Storage Class: Must correspond to an EFS provisioner configured in the Kubernetes cluster.
Capacity: Start with 200 GiB and scale as per your dataset requirements.

2. Deploying NVIDIA NIM Helm Chart

After configuring storage, deploy NVIDIA NIM using Helm. The Helm chart simplifies GPU allocation and links the persistent storage to NIM-managed workloads.

Configure the NGC API Key

Before deploying NVIDIA NIM, you need to retrieve your NGC API Key from NVIDIA’s cloud platform and set it as an environment variable. This key enables secure authentication with NVIDIA’s container registry and services.

Steps to Retrieve the NGC API Key:

Log in to your NGC account.
Navigate to Setup > API Keys.
Click Generate API Key if you don’t already have one.
Copy the generated key to use in your deployment process.

Set the NGC API Key as an Environment Variable:

Run the following command in your terminal to make the key accessible to Terraform during deployment:

export TF_VAR_ngc_api_key=<replace-me>

Replace <replace-me> with your actual API key. This key will be passed to NVIDIA NIM to enable seamless model deployment.

Code: Helm Release for NVIDIA NIM

helm_release "nim_llm" {
  name      = "nim-llm"
  chart     = "./nim-llm"                # Points to the NIM Helm chart location.
  namespace = "nim"
  values = [
    templatefile("nim-llm-values.yaml", {
      model_id    = var.model_id            # Specifies the LLM model (e.g., GPT-like models).
      num_gpu     = var.num_gpu             # Allocates GPU resources for inference tasks.
      ngc_api_key = var.ngc_api_key
      pvc_name    = kubernetes_persistent_volume_claim_v1.efs_pvc.metadata[0].name
    })
  ]
}

Key Points:

model_id: The identifier of the model being deployed (e.g., GPT-3, BERT).
num_gpu: Configures GPU resources for inference tasks. The value should align with the instance type used in your cluster (e.g., g4dn.xlarge for one GPU).
pvc_name: Links the EFS-backed PVC to the workload for storing large datasets or models.

3. Configuration Highlights

Why Persistent Storage?

AI models and datasets are often larger than the node's local storage. Using EFS ensures:
- Scalability: Adjust storage as required without downtime.
- High Availability: Accessible across multiple Availability Zones.

GPU Allocation

NVIDIA NIM optimizes GPU usage for inference. Use the num_gpu variable to specify the number of GPUs for your workload, ensuring efficient resource utilization.

Summary

Storage Configuration: Use AWS EFS with Kubernetes PVC for shared, scalable storage across pods.
GPU Allocation: NVIDIA NIM enables efficient GPU resource management for AI inference tasks.
Helm Chart Deployment: Leverage Helm for streamlined deployment, linking GPU resources and persistent storage.

Step 3: Adding OpenAI Web UI

The OpenAI Web UI provides an interface for users to interact with deployed AI models.

"helm_release" "openai_webui" {
  name       = "openai-webui"
  chart      = "open-webui"
  repository = "https://helm.openwebui.com/"
  namespace  = "openai-webui"
  values = [
    jsonencode({
      replicaCount = 1,
      image = {
        repository = "ghcr.io/open-webui/open-webui"
        tag        = "main"
      }
    })
  ]
}

Step 4: Observability with Prometheus, Grafana, and Custom Metrics

Prometheus and Grafana are essential tools for monitoring AI workloads. Prometheus collects resource metrics, including GPU-specific data, while Grafana visualizes these metrics through tailored dashboards. These tools help ensure that AI operations are running smoothly and efficiently.

To extend observability, the Prometheus Adapter is configured with custom rules for tracking AI-specific metrics. Key configurations include:

Tracking Active Requests: Using the num_requests_running metric, Prometheus monitors the number of ongoing requests, providing insights into workload intensity.
Inference Queue Monitoring: The nv_inference_queue_duration_us metric tracks NVIDIA inference queue times, converted into milliseconds for enhanced readability.

Sample Configuration for Prometheus Adapter:

prometheus:
  url: http://kube-prometheus-stack-prometheus.${prometheus_namespace}
  port: 9090
rules:
  default: false
  custom:
  - seriesQuery: '{__name__=~"num_requests_running"}'
    resources:
      template: <<.Resource>>
    name:
      matches: "num_requests_running"
      as: ""
    metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)
  - seriesQuery: 'nv_inference_queue_duration_us{namespace!="", pod!=""}'
    resources:
      overrides:
        namespace:
          resource: "namespace"
        pod:
          resource: "pod"
    name:
      matches: "nv_inference_queue_duration_us"
      as: "nv_inference_queue_duration_ms"
    metricsQuery: 'avg(rate(nv_inference_queue_duration_us{<<.LabelMatchers>>}[1m])/1000) by (<<.GroupBy>>)'

These configurations enable Prometheus to expose meaningful custom metrics that are critical for scaling and optimizing AI workloads. By integrating these metrics into Grafana dashboards, users gain actionable insights into system performance and bottlenecks.

Step 5: Scaling and Optimization with Karpenter

In large-scale AI deployments, workload demands fluctuate significantly. Dynamic scaling is essential for managing these workloads effectively while minimizing costs. Karpenter, a Kubernetes-native cluster autoscaler, provides powerful mechanisms for optimizing resource utilization. It dynamically provisions nodes tailored to the specific demands of applications, including GPU-heavy AI workloads.

This section integrates Karpenter into the EKS Blueprint framework, highlighting its configuration for both CPU and GPU workloads. The full implementation and configurations are available in the https://github.com/timurgaleev/eks-nim-llm-openai.

Deploying Karpenter with EKS Blueprints

Karpenter is added to the EKS cluster as a Blueprint add-on. Below is an example of the configuration block for enabling Karpenter, focusing on both CPU and GPU workload optimization:

module "eks_blueprints_addons" {
  source  = "aws-ia/eks-blueprints-addons/aws"
  version = "~> 1.2"

  enable_karpenter                  = true
  karpenter_enable_spot_termination = true
  karpenter_node = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
  }
  karpenter = {
    chart_version = "0.37.0"
  }
}

This configuration enables Karpenter with support for Spot instance termination handling and assigns additional IAM policies for managing nodes.

Configuring Karpenter for CPU and GPU Workloads

For effective scaling, Karpenter relies on Provisioner configurations tailored to workload requirements. The following examples showcase how Karpenter dynamically provisions CPU and GPU nodes.

CPU Workloads

name: cpu-karpenter
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[2]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: cpu-karpenter
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["m5"]
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["xlarge", "2xlarge", "4xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

GPU Workloads

name: gpu-workloads
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[1]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: gpu-workloads
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["g5", "p4", "p5"]  # GPU instances
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["2xlarge", "4xlarge", "8xlarge", "12xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

Terraform Automation Scripts

To streamline the deployment and teardown of resources, the project includes two utility scripts: install.sh and cleanup.sh.

install.sh: Automates the deployment process. It initializes Terraform, applies modules sequentially (e.g., VPC and EKS), and ensures all resources are provisioned successfully. A final Terraform apply captures any remaining dependencies.

cleanup.sh: Safely destroys the deployed infrastructure. It handles dependencies like Kubernetes services, Load Balancers, and Security Groups, ensuring proper teardown order. Each module is destroyed sequentially, with a final pass to catch residual resources.

These scripts enhance operational efficiency and minimize errors during deployment and cleanup phases, making the workflow more robust and reproducible.

Key Features of Karpenter in AI Ecosystems

Dynamic Node Provisioning: Automatically provisions CPU or GPU nodes based on real-time workload needs.
Cost Optimization: Leverages Spot instances while ensuring reliable on-demand scaling for critical workloads.
Enhanced Resource Utilization: Consolidates underutilized nodes and removes idle resources with disruption policies.
Tailored Scaling Policies: Supports node pools for diverse workload types, such as inference tasks or data preprocessing.

Karpenter’s integration with GPU-optimized workloads ensures that demanding AI models benefit from high-performance compute nodes while maintaining cost efficiency.

Use Cases

1. AI Model Training

NVIDIA NIM’s GPU optimizations allow for efficient training of models like BERT or GPT, reducing runtime and costs.

2. Real-Time Inference

Deploy models for real-time applications such as fraud detection, image recognition, or natural language understanding.

3. Experimentation and Research

With the OpenAI Web UI, data scientists can quickly test and iterate on models.

Conclusion

This platform enables the scalable and efficient deployment of AI workloads by integrating NVIDIA NIM with Amazon EKS. Terraform automates the process, ensuring repeatable and reliable setups. With GPU optimization, persistent storage, and observability tools, the platform is well-suited for businesses and researchers alike.

By following this guide, you can build a scalable and efficient AI platform. For detailed code and further exploration, visit the GitHub repository https://github.com/timurgaleev/eks-nim-llm-openai.

Deploying AWS EKS with Terraform and Blueprints Addons

Timur Galeev — Thu, 07 Nov 2024 09:07:18 +0000

After a pause from covering AWS and infrastructure management, I’m back with insights for those looking to navigate the world of AWS containers and Kubernetes with ease. For anyone new to deploying Kubernetes in AWS, leveraging Terraform for setting up an EKS (Elastic Kubernetes Service) cluster can be a game-changer. By combining Terraform’s infrastructure-as-code capabilities with AWS’s EKS Blueprints Addons, users can create a scalable, production-ready Kubernetes environment without the usual complexity.

In this article, I'll guide you through using Terraform to deploy EKS with essential add-ons, which streamline the configuration and management of your Kubernetes clusters. With these modular add-ons, you can quickly incorporate features like CoreDNS, the AWS Load Balancer Controller, and other powerful tools to customize and enhance your setup. Whether you’re new to container orchestration or just seeking an efficient AWS solution, this guide will help you build a resilient EKS environment in a few straightforward steps.

So let’s start.

Setting Up the VPC for EKS

The VPC configuration is foundational for your EKS cluster, establishing a secure, isolated environment with both public and private subnets. Private subnets are typically used to host your Kubernetes nodes, keeping them inaccessible from the internet. Here’s the configuration provided in the vpc.tf file, which sets up both public and private subnets along with NAT and Internet Gateway options for flexible networking.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name                 = local.name
  cidr                 = var.vpc_cidr
  azs                  = local.azs
  secondary_cidr_blocks = var.secondary_cidr_blocks
  private_subnets      = concat(local.private_subnets, local.secondary_ip_range_private_subnets)
  public_subnets       = local.public_subnets
  enable_nat_gateway   = true
  single_nat_gateway   = true
  public_subnet_tags   = {"kubernetes.io/role/elb" = 1}
  private_subnet_tags  = {
    "kubernetes.io/role/internal-elb" = 1
    "karpenter.sh/discovery" = local.name
  }
  tags = local.tags
}

This setup:

Creates private and public subnets across multiple availability zones.
Configures a secondary CIDR block for the EKS data plane, which is crucial for large-scale deployments.
Enables a NAT gateway for private subnets, ensuring secure internet access for internal resources.
Tags subnets for Kubernetes service and discovery, essential for integration with other AWS services like load balancers and Karpenter.

Deploying EKS with Managed Node Groups

Now that the VPC is configured, let’s move on to deploying the EKS cluster with the eks.tf file configuration. This setup includes defining managed node groups within the EKS cluster, specifying node configurations, security rules, and IAM roles.

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true
  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = compact([for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) : substr(cidr_block, 0, 4) == "100." ? subnet_id : null])

  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups   = ["system:bootstrappers", "system:nodes"]
    }
  ]

  eks_managed_node_groups = {
    core_node_group = {
      name             = "core-node-group"
      ami_type         = "AL2_x86_64"
      min_size         = 2
      max_size         = 8
      desired_size     = 2
      instance_types   = ["m5.xlarge"]
      capacity_type    = "SPOT"
      labels           = { WorkerType = "SPOT", NodeGroupType = "core" }
      tags             = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key components:

VPC and Subnets: The vpc_id and subnet_ids reference the private subnets, providing a secure foundation for EKS nodes.
Managed Node Groups: This setup defines a core node group with spot instances (capacity_type = "SPOT") to optimize cost, with configurable instance types, sizes, and labels for workload placement.
Security Rules and IAM Roles: Configures additional security rules to manage access between nodes and clusters, along with IAM roles to control permissions for Karpenter and node management.

Configuring EKS Add-ons

Add-ons enhance your EKS cluster by integrating additional AWS services and open-source tools. With the EKS Blueprints, you can easily set up these add-ons, which range from storage solutions to observability and monitoring tools.

Setting Up the EBS CSI Driver for Persistent Storage

The Amazon EBS CSI Driver is essential for persistent storage on EKS. This module configures the necessary IAM roles for the driver, enabling it to provision and manage EBS volumes.

module "ebs_csi_driver_irsa" {
  source                = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version               = "~> 5.20"
  role_name_prefix      = format("%s-%s-", local.name, "ebs-csi-driver")
  attach_ebs_csi_policy = true
  oidc_providers = {
    main = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:ebs-csi-controller-sa"]
    }
  }
  tags = local.tags
}

This configuration creates an IAM role for the EBS CSI Driver using IAM Roles for Service Accounts (IRSA), which allows the driver to interact with EBS securely.

Enabling Amazon CloudWatch Observability

The amazon-cloudwatch-observability add-on integrates CloudWatch for monitoring and logging, providing insights into your cluster’s performance.

eks_addons = {
  amazon-cloudwatch-observability = {
    preserve                 = true
    service_account_role_arn = aws_iam_role.cloudwatch_observability_role.arn
  }
}

This snippet specifies the IAM role required for CloudWatch, enabling detailed observability for your workloads.

Integrating AWS Load Balancer Controller

The AWS Load Balancer Controller allows you to provision and manage Application Load Balancers (ALBs) for Kubernetes services. Here’s how it’s configured:

enable_aws_load_balancer_controller = true
aws_load_balancer_controller = {
  set = [{
    name  = "enableServiceMutatorWebhook"
    value = "false"
  }]
}

The enableServiceMutatorWebhook setting is disabled to avoid automatic modification of service annotations, making it ideal for custom configurations.

Adding Karpenter for Autoscaling

Karpenter is an open-source autoscaler designed for Kubernetes, enabling efficient and dynamic scaling of EC2 instances based on workload requirements. This configuration sets up Karpenter with support for spot instances, reducing costs for non-critical workloads.

enable_karpenter                  = true
karpenter_enable_spot_termination = true
karpenter_node = {
  iam_role_additional_policies = {
    AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  }
}
karpenter = {
  chart_version       = "0.37.0"
  repository_username = data.aws_ecrpublic_authorization_token.token.user_name
  repository_password = data.aws_ecrpublic_authorization_token.token.password
}

This configuration includes additional IAM policies for Karpenter nodes, making it easier to integrate with AWS services like EC2 for flexible scaling.

These add-ons, configured through the AWS EKS Blueprints and Terraform, help streamline Kubernetes management on AWS while offering enhanced storage, observability, and autoscaling.

To explore the complete configuration, you can find the full code in the GitHub repository https://github.com/timurgaleev/aws-eks-terraform-addons. The repository includes install.sh to deploy the EKS cluster and configure the add-ons seamlessly, along with cleanup.sh to tear down the environment when it’s no longer needed.

Conclusion

This Terraform setup provides a powerful framework for deploying EKS with essential add-ons, such as storage, observability, and autoscaling, to support scalable applications. Specifically, this configuration is designed to enable deployment of applications like OpenAI Chat, showcasing Kubernetes' flexibility for real-time, interactive workloads. With this setup, you’re ready to deploy and manage robust, production-grade EKS clusters in AWS.

Beginning the Journey into ML, AI and GenAI on AWS

Timur Galeev — Mon, 22 Jan 2024 22:08:13 +0000

Machine Learning (ML), Artificial Intelligence (AI), and Generative Artificial Intelligence (GenAI) are transformative technologies that have the potential to revolutionize industries across the globe.

At the last AWS re:Invent, there were numerous updates related to ML/AI and everything associated with these technologies. I also decided to delve into these topics and immerse myself in this field.

I won't delve into explaining the meanings of ML, AI, DL(Deep Learning), and GenAI. However, I'd like to touch upon FMs and LLM as we will focus our attention there. I found myself losing the same question when I came across this topic in my reading or listening. :)

Foundational Models (FMs) within the AWS ecosystem represent fundamental structures and algorithms essential for diverse AI applications. These models, often created by industry-leading AI companies, are integral to the development and functionality of AWS services, shaping the landscape of artificial intelligence on the platform. In the context of Amazon Bedrock, Language Models (LMs) play a pivotal role. These LMs contribute to the service's linguistic capabilities, facilitating advanced language understanding and content generation within the AWS environment.

AWS provides various services for Machine Learning and Artificial Intelligence, including Amazon SageMaker, AWS DeepLens, AWS DeepComposer, Amazon Forecast and more. Familiarize yourself with the services available to determine which ones suit your specific needs.

Generative Artificial Intelligence (GenAI) is a type of artificial intelligence that can generate text, images, or other media using generative models. AWS offers a range of services for building and scaling generative AI applications, including Amazon SageMaker, Amazon Rekognition, AWS DeepRacer, and Amazon Forecast. AWS has also invested in developing foundation models (FMs) for generative AI, which are ultra-large machine learning models that generative AI relies on. AWS has also launched the Generative AI Innovation Center, which connects AWS AI and ML experts with customers around the world to help them envision, design, and launch new generative AI products and services. Generative AI has the potential to revolutionize the way we create and consume media, but it is important to use it responsibly and ethically.

Some examples GenAI: One of the most well-known examples of GenAI is ChatGPT, launched by OpenAI, which became wildly popular overnight and galvanized public attention. Another model from OpenAI, called text-embedding-ada-002, is specifically designed to work with embeddings—a type of database specifically designed to feed data into large language models (LLM). However, it’s important to note that generative AI creates artifacts that can be inaccurate or biased, making human validation essential and potentially limiting the time it saves workers. Therefore, end users should be realistic about the value they are looking to achieve, especially when using a service as is.

I've also delved a bit deeper into Broad AI when learning GenAI and I'd like to show this in the form of the following picture as it explains a lot.

Layers from broad artificial Intelligence to generative AI

Broad AI includes task-specific algorithms, Machine Learning (ML), and Deep Learning. These layers enable AI to perform tasks like image recognition, natural language processing, and complex pattern modeling.

The transition to GenAI involves Transfer Learning, Reinforcement Learning, and Autonomous Learning. These layers allow AI to apply knowledge across contexts, learn from interactions, and independently gather and learn from information.

So, the journey from Broad AI to GenAI represents significant leaps in AI capabilities, moving towards AI systems that can truly understand, learn, and adapt like a human brain.

Let's explore a couple of AWS services that, from my perspective, are among the more popular today.

Amazon SageMaker:

Amazon SageMaker is a comprehensive platform that simplifies the machine learning workflow. It covers everything from data labeling and preparation to model training and deployment. Take advantage of SageMaker's Jupyter notebook integration for interactive data exploration and model development. The platform also supports popular ML frameworks like TensorFlow and PyTorch.

Amazon Q is a groundbreaking Generative AI assistant crafted with a focus on security and privacy. Its purpose is to unleash the transformative capabilities of this technology for employees within organizations of varying sizes and across diverse industries.

Introduces robust enhancements to the generative AI service, Amazon Bedrock.

Amazon Bedrock, an entirely managed service on AWS, provides access to extensive language models and other foundational models (FMs) from prominent artificial intelligence (AI) companies such as AI21, Anthropic, Cohere, Meta, and Stability AI, all consolidated through a unified API.

I would also like to share more information about Amazon Bedrock here about the innovations that were announced at the latest AWS re:Invent.

Fine-tuning for Amazon Bedrock:
Now, there are increased opportunities for model customization in Amazon Bedrock, featuring fine-tuning support for Cohere Command Lite, Meta Llama 2, and Amazon Titan Text models, with Anthropic Claude's support expected soon.

These recent enhancements to Amazon Bedrock significantly reshape how organizations, regardless of their size or industry, can leverage generative AI to drive innovation and redefine customer experiences.

AWS is compatible with all the leading deep-learning frameworks, facilitating their deployment. The deep-learning Amazon Machine Image, accessible on both Amazon Linux and Ubuntu, allows for the creation of managed, auto-scalable GPU clusters. This enables training and inference processes to be conducted at any scale. Also, AWS offers a range of AI services that allow you to integrate pre-trained models into your applications without the need for deep expertise in machine learning. Services like Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing, and Amazon Polly for text-to-speech can enhance your applications with AI capabilities.

The best way to solidify your understanding of ML, AI, and GenAI on AWS is through hands-on projects. Start with simple projects and gradually increase complexity as you gain confidence. Use datasets available on platforms like Kaggle or create your own to train and test models.

Conclusion:

Embarking on a journey into Machine Learning, Artificial Intelligence, and Generative Artificial Intelligence on AWS is an exciting endeavor. By following these steps, you can lay a solid foundation for your understanding and proficiency in leveraging AWS services for ML and AI applications. Remember, the key to success is a combination of hands-on experience, continuous learning, and active engagement with the AWS community. Happy training!

Domain-Driven Design (DDD) in AWS. Find Your Business Domains.

Timur Galeev — Wed, 29 Mar 2023 15:27:12 +0000

This article is an introduction to Domain-Driven Design and how it can be used with AWS. I will provide guidance on how to define business domains in legacy monolithic applications and decompose them into a set of microservices step by step. By starting with Domain-Driven Design for your microservices, you can get the benefits of cloud scaling in your new refactored application.

Is Domain-Driven Design usefull for me?
The purpose of Domain-Driven Design is to free the domain code from technical details to have more room to work with its complexity. It is well suited to work with very complex domains and projects that are starting to dive into legacy.

Domain-Driven Design requires an understanding of the business idea or understanding of the final 'business product'. It requires time and commitment from both business experts and technical implementers. Domain-Driven Design should not be used in situations where you need 'quick solutions'. Instead, use Domain-Driven Design for software that supports the core business area rather than supporting areas. Running Domain-Driven Design can be achieved through an event-storming session. However, as mentioned above, this is a commitment worth making. It will allow you to develop software that is more tailored to the needs of your end clients. It will also help create decoupled services that are more scalable and maintainable. The combination will result in greater business agility.

Event Storming

Event Storming helps teams of business and technical people come to a consensus on what the solution should be. This happens without being distracted by the specific implementation details of how it will be implemented. This means, that it may take longer for the teams to start providing source code. However, all teams will be better aligned as to what each microservice should be responsible for. The event storming workshop is a brainstorming session. In this session all stakeholders in the solution work together to define the business events that correspond to the domains.
Suppose, we have a commerce development task where the business event might be a customer who applies for a new product. During the workshop, the group will begin to identify the object that triggered the event, the processes that should occur as a result, and any subsequent event triggered by the original event.

To do this, a team brainstorming session takes place where the event groups can identify areas of their business and then the contexts in which they operate. These can be used to define the usage and the relationship that occurs between each microservice and it’s context. Once the domains have been defined with the help of the business experts, the technical implementers can start designing the solution.

The result of the event-storming session is a domain model for development. The domain model can be used to define a number of bounded contexts.

Bounded Contexts

A bounded context is the boundary where each domain applies. The order contract opening example can be thought of as the 'Order Contract Opening Context' in a shop. In a complete system, there may be other contexts such as the product context, the description context and the manufacturing context. Identifying the business events that cause interactions between the different constrained contexts helps to determine how your microservices will interact with each other in the new architecture.

The example context map is just a sample of the core domains. There are also a number of supporting and subsequent domains. Although it is necessary to have a service that manages these, this is not part of the core application domain for sending products.

Core Domains and refactoring

When you start defining "Core Domains and Subsequent Domains" the question that usually arises is how to manage requests between domains.To do this, we'll look at options for using AWS services for how domains can be implemented.
Containerisation or serverless diagram of such solutions would rather be a 'modern architecture' than a good old-fashioned network and virtual machine deployment diagram. The advantage of these solutions is that the diagram itself helps to actually outline, what the logical functionality is, since we can be more expressive and fine-grained with resource usage.
The undisputed king of serverless computing platforms has been AWS LambdaAWS Lambda for several years now. It satisfies all the aforementioned conditions and can be used in a number of languages/implementations, including TypeScript.
Other viable options might include some of the more well-known container services such as AWS ECS or AWS EKS wrapped Fargate. However, they require considerably more setup and configuration, and also require that containerization actually takes place. It doesn't mean that containerisation is bad, in general containerisation can be good, it all depends on your development idea whether it's refactoring into microservices or starting a new application.
If you need Eventing then here it is Simple Notification Service (SNS). It is a push-based service, i.e. it automatically handles the distribution of the event to the recipients. SNS uses a pay-per-use model and it is essentially serverless as the only infrastructure you need is the SNS subject.
The modern cloud is about using its own API products to expose its applications, rather than building something of its own with Fastify, Kong or the like.
The API gateway acts as the only public interface connected to any other infrastructure, in our case primarily our Lambda compute functions, which will respond to paths defined in the gateway. In the case of AWS the service of interest, unsurprisingly, is called the AWS API Gateway.

The following diagram shows how monolith receives some of the traffic during the gradual addition of new microservices in the example application.

There is also AWS Migration Hub, it will help you in finding your domains and even offer AWS services that you can implement. This will help you to plan a refactoring or plan for migrating from your old OnPrem solutions to AWS with all modern solutions.

Conclusion

To summarise, people just don't tend to talk about 'domains' all day. Most employees do not pay attention to the implementation of domains in the organisation.
It is also worth noting that dividing systems into domains after they have been fully designed is also useless. DDD should be done, at least approximately, at the initial design stage.
But in any case, DDD is the place to be. In the example of using DDD in AWS it looks simple but when you start to go deeper here you find a lot of services where they all have to be interconnected and here comes the methods and dependencies. That's why it's very important to create a structure at the beginning of the work.

AWS, Terraform ,WordPress. Step-by-Step Guide Example

Timur Galeev — Tue, 24 Jan 2023 12:52:15 +0000

I've looked at many ways to run Wordpress on AWS but they are all expensive or unstable for me. So I decided to make a stable version of Wordpress installation that suited me.

Infrastructure management has changed a lot over the years. So much so that the traditional system administrator managed a rack full of servers. In many cases, the initial setup required manual intervention at the console.

Why AWS for WordPress might be a great choice
To begin with, AWS is a big deal. It's the cloud hosting provider with the largest market share. AWS is such a successful platform that it accounts for half of Amazon's operating income, which is in the billions of dollars.

AWS offers high scalability, making it ideal for sites with thousands of daily visitors. The platform also allows for any server configuration. This is ideal for high-performance sites such as online stores.

In this article, I will show you how to use Terraform to install Wordpress on AWS.

Our objectives

Create a ‘tf’ files which will hold all of our relevant configuration information (main.tf, …)
Define which provider we will be using in the Terraform config. (aws, cloudflare)
Certificate handling (AWS Certificate Manager)
Define security group rules and names.
Define the EC2 instances we want to create. (AWS Auto Scaling Group)
Run Terraform to plan and apply our configuration.

Before you get started, you’ll need to sign up for AWS. During the process, you’ll need to verify your account using a credit card – onto which they’ll charge $1 – and receive a verification code via SMS.
When you’re ready, select the Free support plan and you’ll get access to your console, which is where the magic happens

So, let's start creating our wordpress in AWS

I will assume that you have already configured all the necessary tools to run such as Terraform, aws-cli

The first thing you need to do is clone the repository:
$ git clone https://github.com/timurgaleev/wordpress-ec2-rds-alb-vpc.git

Once you've taken the templates from the repository, you now need to configure the 'tf' files, which will hold all of the relevant Terraform configuration. You can encapsulate it in one file, but for simplicity and convenience we will work within multiple files.

The structure of the repository is as follows:

acm.tf - AWS Certificate Manager Terraform module
alb.tf - AWS Application and Network Load Balancer Terraform module
asg.tf - AWS Auto Scaling Group (ASG) Terraform module
cloudflare.tf - Cloudflare Provider
efs.tf - Provides an Elastic File System (EFS) File System resource
output.tf - Terraform Output Values
rds.tf - AWS RDS Terraform module
security_group.tf - AWS EC2-VPC Security Group Terraform module
vpc.tf - AWS VPC Terraform module
variables.tf - variables used in Terraform.

Prerequisites needed for creating a envirenment
provider.tf you will need a change.

backend "s3" {
    bucket         = "ecs-terraform-examplecom-state"
    key            = "example/com.tfstate"
    region         = "eu-west-1"
    encrypt        = "true"
    dynamodb_table = "ecs-terraform-remote-state-dynamodb"
  }

It will also use an S3 bucket that will be used as a remote store for our Terraform state. This allows multiple users to work with one set of Infrastructure as Code without causing conflicts.

Then change the variables according to your needs.
variables.tf like environment, Domain, Cloudflare API Token

variable "asg_min_size" {
  description = "AutoScaling Group Min Size "
  default     = 1
}

variable "asg_max_size" {
  description = "AutoScaling Group Max Size "
  default     = 2
}

variable "asg_desired_capacity" {
  description = "AutoScaling Group Desired Capacity"
  default     = 1
}

variable "rds_engine" {
  description = "RDS engine"
  default     = "mariadb"
}

variable "rds_engine_version" {
  description = "RDS engine version"
  default     = "10.6.7"
}

variable "rds_instance_class" {
  description = "RDS instance class"
  default     = "db.t3.micro"
}

variable "site_domain" {
  description = "Domain"
  default     = "example.com"
}

variable "cloudflare_zone" {
  description = "cloudflare Zone Id"
}

variable "dns_ttl" {
  description = "cloudflare for dns = 1 is automatic."
  default     = 1
}

variable "dns_allow_overwrite_records" {
  description = "cloudflare allow overwrite records."
  default     = true
}

variable "cloudflare_api_token" {
  description = "cloudflare api token"
}

variable "ssh_key_name" {
  description = "SSH Key"
}

Now we can run our WordPress.

Run terraform init
Run terraform plan and review
Run terraform apply

Now enter your server domain, and you'll see the following WordPress installation screen. If that's the case, great job! If you don't see that screen, you may need to double-check the steps you followed.

You can destroy this WordPress by running:

terraform plan -destroy
terraform destroy  --force

Conclusion
We've only had a superficial look at how Terraform can be used in AWS, but I think a simple introduction is the best part! We created a database in RDS and plugged it into Wordpress as well as created and wired the necessary data into Cloudflare.