Forem: Kehinde Ogunlowo

The Complete Guide to Production EKS with Terraform

Kehinde Ogunlowo — Sun, 08 Mar 2026 18:56:56 +0000

"Production-ready EKS deployment with Terraform — Karpenter autoscaling, self-healing nodes, pod security standards, and multi-AZ high availability."

EKS is the most popular managed Kubernetes service, but most deployments I've seen in production audits are dangerously under-configured. Missing node auto-remediation, no pod security standards, manual scaling — the list goes on.

This guide covers everything you need for production EKS.

EKS vs AKS vs GKE

Feature	EKS	AKS	GKE
Control Plane Cost	$0.10/hr	Free	Free (Standard)
Autopilot Mode	No (use Karpenter)	No	Yes
Node Auto-Repair	Manual/Lambda	Built-in	Built-in
Service Mesh	App Mesh / Istio	Istio	Anthos / Istio
GPU Support	p4d, g5	NC, ND series	T4, A100

Terraform Module

module "eks" {
  source = "github.com/kogunlowo123/terraform-aws-auto-healing-eks"

  cluster_name    = "production-cluster"
  cluster_version = "1.29"
  vpc_id          = module.vpc.vpc_id
  subnet_ids      = module.vpc.private_subnet_ids

  node_groups = [{
    name           = "general"
    instance_types = ["m6i.xlarge", "m6i.2xlarge"]
    min_size       = 3
    max_size       = 20
    desired_size   = 5
  }]

  enable_karpenter                = true
  enable_cluster_autoscaler       = false  # Use Karpenter instead
  enable_node_termination_handler = true
  enable_auto_remediation         = true
}

Best Practices

Use Karpenter over Cluster Autoscaler for faster scaling and better bin-packing
Enable pod disruption budgets for every production workload
Use node termination handler for spot instance graceful shutdown
Implement network policies with Calico or Cilium
Enable control plane logging to CloudWatch
Use IRSA (IAM Roles for Service Accounts) instead of node-level IAM

Open Source

terraform-aws-auto-healing-eks — Self-healing EKS
terraform-aws-eks — Standard EKS module
terraform-aws-vpc-complete — VPC for EKS

Full guide: kogunlowo123.github.io

Building Production RAG Pipelines on AWS with Bedrock and OpenSearch

Kehinde Ogunlowo — Sun, 08 Mar 2026 18:54:25 +0000

RAG (Retrieval-Augmented Generation) is how enterprises are deploying LLMs without fine-tuning. But most tutorials stop at the demo stage. Production RAG is a different beast entirely.

Here's what production RAG actually requires — and how to build it on AWS.

RAG vs Fine-Tuning vs Prompt Engineering

Approach	Cost	Data Freshness	Accuracy	Complexity
RAG	Medium	Real-time	High (with good retrieval)	Medium
Fine-Tuning	High	Static (retraining needed)	High	High
Prompt Engineering	Low	Static	Variable	Low

Architecture

The pipeline: Documents → Chunking → Embeddings → Vector Store → Query → Retrieval → LLM → Response.

Python Implementation

import boto3
import json

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
opensearch = boto3.client("opensearchserverless")

def query_knowledge_base(question: str, collection_id: str) -> str:
    # Generate embedding for the question
    embed_response = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": question})
    )
    query_embedding = json.loads(embed_response["body"].read())["embedding"]

    # Search OpenSearch vector store
    results = search_vectors(query_embedding, collection_id, k=5)
    context = "\n".join([r["text"] for r in results])

    # Generate answer with context
    prompt = f"""Based on the following context, answer the question.

Context: {context}

Question: {question}

Answer:"""

    response = bedrock.invoke_model(
        modelId="anthropic.claude-3-sonnet-20240229-v1:0",
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "messages": [{"role": "user", "content": prompt}],
            "max_tokens": 1024
        })
    )
    return json.loads(response["body"].read())["content"][0]["text"]

Hallucination Mitigation

Chunk size matters — 512 tokens with 50-token overlap works best
Hybrid search — combine semantic + keyword search (BM25)
Citation grounding — force the model to cite source chunks
Confidence scoring — filter low-relevance retrievals (cosine similarity < 0.7)
Guardrails — use Bedrock Guardrails to block off-topic responses

Open Source Modules

terraform-aws-rag-pipeline — Complete RAG infrastructure
terraform-aws-bedrock-platform — Bedrock foundation
terraform-aws-bedrock-agents — Autonomous Bedrock agents

Full article with architecture diagrams: kogunlowo123.github.io

Kehinde Ogunlowo — Principal Multi-Cloud DevSecOps Architect at Citadel Cloud Management

# Building a Production-Ready AWS VPC with Terraform: Multi-Tier Subnets, NAT Gateways, and VPC Endpoints

Kehinde Ogunlowo — Sat, 07 Mar 2026 21:20:40 +0000

A network topology diagram showing a multi-tier VPC with public, private, database, cache, and management subnets across three availability zones, with NAT gateways and VPC endpoints illustrated.

If you've ever inherited an AWS account where everything lives in the default VPC, you know the pain. Security groups used as the only network boundary. No flow logs. Public IP addresses on database instances. It's the kind of setup that keeps security teams awake at night.

A well-architected VPC is the foundation of everything you build on AWS. Get it wrong, and you're retrofitting network isolation into a running production system — one of the least enjoyable exercises in cloud engineering.

In this article, I'll walk through a production-grade VPC architecture using Terraform, based on a module I've been refining across multiple enterprise deployments. The full module is available at terraform-aws-vpc-complete.

Why VPC Architecture Matters More Than You Think

Most teams start with a simple public/private subnet split and call it a day. That works for a proof of concept, but production workloads demand more:

Regulatory compliance often requires network-level isolation between application tiers
Cost optimization depends on keeping traffic within the AWS network via VPC endpoints
Blast radius reduction means a compromised web server shouldn't have a network path to your database
Operational visibility requires flow logs that actually capture meaningful traffic patterns

Let's build something that addresses all of these.

The 5-Tier Subnet Strategy

Instead of the typical two-tier model, I use five distinct subnet tiers:

Tier	Purpose	Internet Access	Example Workloads
Public	Internet-facing resources	Direct (IGW)	ALBs, NAT Gateways, Bastion hosts
Application	Compute workloads	Outbound via NAT	ECS tasks, EC2 instances, Lambda
Data	Databases and storage	None	RDS, ElastiCache, OpenSearch
Cache	In-memory data stores	None	Redis clusters, Memcached
Management	Ops and monitoring tools	Outbound via NAT	Jenkins, Prometheus, VPN endpoints

Each tier gets its own subnet in every availability zone, its own route table, and its own set of network ACLs.

Defining the Network Layout

Here's how the module structures the CIDR allocation:

module "vpc" {
  source = "github.com/kogunlowo123/terraform-aws-vpc-complete"

  vpc_name   = "production"
  vpc_cidr   = "10.0.0.0/16"
  azs        = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]

  # 5-tier subnet strategy
  public_subnets      = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  application_subnets = ["10.0.10.0/24", "10.0.11.0/24", "10.0.12.0/24"]
  data_subnets        = ["10.0.20.0/24", "10.0.21.0/24", "10.0.22.0/24"]
  cache_subnets       = ["10.0.30.0/24", "10.0.31.0/24", "10.0.32.0/24"]
  management_subnets  = ["10.0.40.0/24", "10.0.41.0/24", "10.0.42.0/24"]

  enable_nat_gateway     = true
  single_nat_gateway     = false  # one per AZ for HA
  enable_vpn_gateway     = false
  enable_dns_hostnames   = true
  enable_dns_support     = true

  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
    Project     = "core-infrastructure"
  }
}

The key design decision here is the CIDR allocation. I leave gaps between tiers (/24 blocks starting at .1, .10, .20, .30, .40) so there's room to add subnets later without re-addressing. In a /16 VPC, you have 65,536 addresses — don't be stingy with the spacing.

NAT Gateway Strategy: Cost vs. Availability

NAT gateways are one of the most expensive networking components in AWS. Each one costs roughly $32/month plus data processing charges. With three AZs, that's nearly $100/month just for NAT — before any traffic flows.

For production, I always recommend one NAT gateway per AZ:

# High-availability NAT configuration
enable_nat_gateway   = true
single_nat_gateway   = false
one_nat_gateway_per_az = true

For development and staging environments, a single NAT gateway is acceptable:

# Cost-optimized NAT for non-production
enable_nat_gateway   = true
single_nat_gateway   = true

The module handles the route table associations automatically. Each private subnet's route table points to the NAT gateway in its own AZ, so if one AZ goes down, the other AZs continue to function.

Route Table Isolation

Each subnet tier gets its own route table. This is critical — it means you can have application subnets that route through NAT while data subnets have no route to the internet at all:

resource "aws_route_table" "data" {
  count  = length(var.azs)
  vpc_id = aws_vpc.this.id

  # No default route — these subnets are fully isolated
  # Only VPC-local traffic (10.0.0.0/16) is routable

  tags = {
    Name = "${var.vpc_name}-data-${var.azs[count.index]}"
    Tier = "data"
  }
}

No NAT gateway route. No internet gateway route. The database subnets literally cannot reach the internet, and the internet cannot reach them. This is defense in depth at the network layer.

VPC Endpoints: Keeping Traffic Private and Cheap

Every call to an AWS API from within your VPC traverses the NAT gateway by default. That means S3 downloads, CloudWatch metrics, ECR image pulls — all going out through NAT and incurring data processing charges.

VPC endpoints solve this by creating private paths to AWS services:

# Gateway endpoints (free)
enable_s3_endpoint       = true
enable_dynamodb_endpoint = true

# Interface endpoints (cost per hour + per GB)
enable_interface_endpoints = true
interface_endpoints = [
  "ecr.api",
  "ecr.dkr",
  "ecs",
  "ecs-telemetry",
  "logs",
  "monitoring",
  "ssm",
  "ssmmessages",
  "ec2messages",
  "sts",
  "secretsmanager",
  "kms"
]

Gateway endpoints (S3 and DynamoDB) are free and should always be enabled. There's no reason not to.

Interface endpoints cost about $7.20/month each (in us-east-1), plus $0.01/GB of data processed. The math works out in your favor when you're pulling container images from ECR multiple times a day or shipping thousands of CloudWatch metrics per minute.

A cluster pulling 50GB of container images monthly from ECR through NAT costs roughly $2.25 in NAT data processing. The ECR interface endpoints cost $14.40/month. At scale, though — say 500GB — the NAT cost is $22.50 and the endpoint cost is still $14.40. The crossover point depends on your traffic patterns, but the security benefit of keeping traffic off the public internet is worth the cost regardless.

VPC Flow Logs: Visibility Into Your Network

Flow logs are non-negotiable in production. They capture metadata about every network flow — source, destination, ports, protocol, action (accept/reject), and byte counts.

# Flow log configuration
enable_flow_logs          = true
flow_log_destination_type = "cloud-watch-logs"
flow_log_retention_days   = 90

flow_log_cloudwatch_log_group = "/vpc/production/flow-logs"

flow_log_traffic_type = "ALL"  # capture both accepted and rejected

# Custom log format for richer data
flow_log_format = "$${version} $${account-id} $${interface-id} $${srcaddr} $${dstaddr} $${srcport} $${dstport} $${protocol} $${packets} $${bytes} $${start} $${end} $${action} $${log-status} $${vpc-id} $${subnet-id} $${az-id} $${sublocation-type} $${sublocation-id}"

I recommend shipping flow logs to CloudWatch Logs for real-time alerting and to S3 for long-term storage and analysis with Athena:

# Dual destination for flow logs
enable_flow_logs_s3          = true
flow_log_s3_bucket_arn       = aws_s3_bucket.flow_logs.arn
flow_log_s3_key_prefix       = "vpc-flow-logs/production"

Querying Flow Logs with Athena

Once flow logs land in S3, you can create an Athena table and run SQL queries:

-- Find rejected traffic to data subnets
SELECT srcaddr, dstaddr, dstport, protocol, action,
       SUM(bytes) as total_bytes, COUNT(*) as flow_count
FROM vpc_flow_logs
WHERE action = 'REJECT'
  AND dstaddr LIKE '10.0.2%'
  AND date = '2026/03/06'
GROUP BY srcaddr, dstaddr, dstport, protocol, action
ORDER BY flow_count DESC
LIMIT 20;

This is how you catch misconfigured security groups, unexpected traffic patterns, and potential intrusion attempts.

Network ACLs: The Second Layer

Security groups are stateful and operate at the instance level. Network ACLs are stateless and operate at the subnet level. Using both gives you defense in depth:

# Data tier NACLs - restrict to only application tier
data_subnet_nacl_rules = {
  inbound = [
    {
      rule_number = 100
      protocol    = "tcp"
      action      = "allow"
      cidr_block  = "10.0.10.0/24"  # app subnet AZ-a
      from_port   = 5432
      to_port     = 5432
    },
    {
      rule_number = 101
      protocol    = "tcp"
      action      = "allow"
      cidr_block  = "10.0.11.0/24"  # app subnet AZ-b
      from_port   = 5432
      to_port     = 5432
    },
    {
      rule_number = 102
      protocol    = "tcp"
      action      = "allow"
      cidr_block  = "10.0.12.0/24"  # app subnet AZ-c
      from_port   = 5432
      to_port     = 5432
    }
  ]
  outbound = [
    {
      rule_number = 100
      protocol    = "tcp"
      action      = "allow"
      cidr_block  = "10.0.10.0/0"
      from_port   = 1024
      to_port     = 65535
    }
  ]
}

This ensures that even if someone misconfigures a security group, the database subnets only accept PostgreSQL traffic from the application tier.

Tagging Strategy for Cost Allocation

Every subnet, route table, NAT gateway, and endpoint gets tagged consistently:

subnet_tags = {
  "kubernetes.io/cluster/production" = "shared"
}

public_subnet_tags = {
  "kubernetes.io/role/elb" = "1"
  Tier                     = "public"
}

application_subnet_tags = {
  "kubernetes.io/role/internal-elb" = "1"
  Tier                              = "application"
}

These tags serve double duty: they enable Kubernetes auto-discovery of subnets for load balancers, and they allow cost allocation reports to break down networking costs by tier.

Putting It All Together

Here's a complete example that ties everything together:

module "vpc" {
  source = "github.com/kogunlowo123/terraform-aws-vpc-complete"

  vpc_name = "production"
  vpc_cidr = "10.0.0.0/16"
  azs      = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]

  # Subnet tiers
  public_subnets      = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  application_subnets = ["10.0.10.0/24", "10.0.11.0/24", "10.0.12.0/24"]
  data_subnets        = ["10.0.20.0/24", "10.0.21.0/24", "10.0.22.0/24"]
  cache_subnets       = ["10.0.30.0/24", "10.0.31.0/24", "10.0.32.0/24"]
  management_subnets  = ["10.0.40.0/24", "10.0.41.0/24", "10.0.42.0/24"]

  # NAT
  enable_nat_gateway     = true
  single_nat_gateway     = false
  one_nat_gateway_per_az = true

  # VPC Endpoints
  enable_s3_endpoint       = true
  enable_dynamodb_endpoint = true
  enable_interface_endpoints = true
  interface_endpoints = ["ecr.api", "ecr.dkr", "logs", "monitoring", "ssm", "sts", "kms"]

  # Flow Logs
  enable_flow_logs       = true
  flow_log_traffic_type  = "ALL"
  flow_log_retention_days = 90

  # DNS
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
  }
}

Lessons Learned

After deploying this pattern across dozens of AWS accounts, here are the things that consistently trip teams up:

Don't use /16 for everything. If you ever need VPC peering or Transit Gateway, overlapping CIDRs will block you. Plan your IP address space across all accounts upfront.
Enable flow logs from day one. Retroactively enabling them means you have no baseline to compare against when something goes wrong.
Test failover. Kill a NAT gateway and verify that the AZ-local route table doesn't send traffic to a NAT in another AZ. If it does, your route table associations are wrong.
Review VPC endpoint policies. An S3 gateway endpoint with a default policy allows access to every S3 bucket in every AWS account. Lock it down to your buckets.
Monitor NAT gateway metrics. BytesOutToDestination and PacketsDropCount will tell you when you're approaching throughput limits (45 Gbps per gateway).

Wrapping Up

A production VPC isn't just about creating subnets. It's about building a network foundation that enforces security boundaries, optimizes costs, and provides the visibility you need when things go wrong at 3 AM.

The module I've walked through here is available on GitHub. Fork it, adapt it, and build on it:

terraform-aws-vpc-complete — The full VPC module with all configurations discussed in this article

Check out my other infrastructure modules at github.com/kogunlowo123 for complementary resources covering WAF, AKS, and multi-cloud landing zones.

Have questions about VPC design or want to share your own patterns? Drop a comment below — I'd love to hear how other teams approach this.