Forem: Prasad P

How We Built an AI Terraform Co-Pilot That Actually Works (And Made It Free)

Prasad P — Wed, 19 Nov 2025 11:42:27 +0000

The Problem We Kept Hitting

Every DevOps engineer has been here: you need to spin up infrastructure, but Terraform syntax is fighting you. You know what you want—"an RDS instance with read replicas in us-east-1"—but translating that to HCL takes 30 minutes of documentation diving.

Existing AI tools? They hallucinate provider versions. They forget required arguments. They generate code that looks right but fails on terraform plan.

We spent 18 months building something better for Realm9, and I want to share the technical approach that made it actually useful.

Why Most AI-to-Terraform Tools Fail

Before diving into our solution, here's why the naive approach doesn't work:

1. Context Window Limitations

Terraform configurations reference modules, variables, and state from across your project. GPT-4 can't see your entire codebase.

2. Version Drift

The AI was trained on Terraform 0.12 syntax but you're running 1.6. Provider APIs change constantly.

3. State Blindness

The AI doesn't know what resources already exist. It'll suggest creating a VPC when you already have three.

4. No Validation Loop

Most tools generate code and hope for the best. No terraform validate, no plan check, no iteration.

Our Architecture: How We Solved It

Here's the technical breakdown of how Realm9's Terraform Co-Pilot actually works:

Layer 1: Project Context Injection

Before any prompt hits the LLM, we build a context package:

├── Current provider versions (from .terraform.lock.hcl)
├── Existing resource inventory (from state)
├── Variable definitions and current values
├── Module interfaces you've defined
└── Your naming conventions (parsed from existing code)

This context gets injected as system prompt, so the AI knows:

You use aws provider 5.31.0, not 4.x
You already have a VPC named main-vpc
Your naming convention is ${project}-${env}-${resource}

Layer 2: Retrieval-Augmented Generation (RAG)

We maintain a vector database of:

Official Terraform provider documentation
AWS/Azure/GCP API specifications
Common patterns and anti-patterns

When you ask "create an S3 bucket with versioning", we retrieve the current S3 resource documentation—not whatever was in GPT's training data 18 months ago.

Layer 3: Validation Loop

Here's where most tools stop. We don't.

User prompt
    ↓
Generate HCL
    ↓
terraform fmt (syntax check)
    ↓
terraform validate (semantic check)
    ↓
If errors → feed errors back to LLM → regenerate
    ↓
terraform plan (dry run)
    ↓
Show plan diff to user

The AI sees its own mistakes and fixes them. Usually takes 1-2 iterations to get valid code.

Layer 4: BYOK (Bring Your Own Key)

We don't lock you into our API costs. On the free tier, you plug in your own OpenAI/Anthropic/Azure OpenAI key. You control:

Which model (GPT-4, Claude, etc.)
Rate limits
Cost

Paid tiers include API credits so you don't have to manage keys.

Real Example: What This Looks Like

User input:

Create an RDS PostgreSQL instance for production with:
- Multi-AZ deployment
- 100GB storage with autoscaling
- Private subnet only
- 7-day backup retention

What the AI generates (after validation loop):

resource "aws_db_instance" "production_postgres" {
  identifier     = "${var.project}-${var.environment}-postgres"
  engine         = "postgres"
  engine_version = "15.4"
  instance_class = "db.r6g.large"

  allocated_storage     = 100
  max_allocated_storage = 500
  storage_type          = "gp3"

  multi_az               = true
  db_subnet_group_name   = aws_db_subnet_group.private.name
  vpc_security_group_ids = [aws_security_group.rds.id]

  backup_retention_period = 7
  backup_window          = "03:00-04:00"
  maintenance_window     = "Mon:04:00-Mon:05:00"

  skip_final_snapshot = false
  final_snapshot_identifier = "${var.project}-${var.environment}-postgres-final"

  tags = local.common_tags
}

Notice it:

Used your existing naming convention (var.project, var.environment)
Referenced your existing subnet group and security group
Picked appropriate engine version for current provider
Added sensible defaults you didn't specify (maintenance window, final snapshot)

Why We Made the AI Free

The free tier includes:

5 users
10 environments
1 Terraform project with 3 workspaces
Full AI co-pilot with BYOK

Why give away the AI? Because:

AI is table stakes now - Charging for basic AI features feels wrong in 2025
BYOK means no margin anyway - You're paying OpenAI directly
The value is the complete platform - AI alone isn't useful; AI integrated with full Terraform lifecycle management is

Our paid tiers ($9.2k-$48k/year) are for teams that need more capacity, enterprise security (SSO/SAML), and included API credits.

Beyond AI: Complete Terraform Lifecycle Management

The AI co-pilot is just one part. Realm9 provides end-to-end Terraform lifecycle management:

Projects & Workspaces

Organize infrastructure into projects with multiple workspaces (dev, staging, prod)
GitOps integration with GitHub/GitLab for version control
Automatic plan/apply workflows with approval gates

Enterprise-Grade Security

End-to-end encryption for all credentials and secrets
Cloud provider credentials stored with AES-256 encryption
No plaintext secrets ever touch disk

Compliance & Audit Trail

SOC 2 Type II compliant controls
ISO 27001 security framework
Complete audit logging of every action
Who ran what, when, and what changed
Exportable audit reports for compliance reviews

State Management

Secure remote state storage
State locking to prevent conflicts
State versioning and rollback capabilities
Drift detection between state and actual infrastructure

This isn't just an AI wrapper—it's a complete Terraform platform that happens to have AI built in.

The Bigger Picture: Environment Management

The AI co-pilot is part of Realm9, a platform that also handles:

Environment booking - No more spreadsheets or Slack wars over who's using staging
Built-in observability - Logs/metrics/traces at 1/10th the cost of Datadog
Drift detection - Know when infrastructure doesn't match code

We built it because we were spending $150k+/year on Plutora + Terraform Cloud + Datadog, and they didn't even talk to each other.

Try It Yourself

Option 1: Self-host free tier

Installation guide - Deploy on your Kubernetes cluster in 30 minutes
Bring your own LLM API key
Full AI co-pilot included

Option 2: Evaluate enterprise features

14-day evaluation - Test Terraform automation, SSO/SAML, advanced AI
No credit card required

Option 3: Explore the code

GitHub: realm9-platform - Star the repos to follow development

What's Next

We're working on:

Multi-cloud support - Same AI, different providers (Azure, GCP)
Cost estimation - "This change will add ~$45/month"
Policy as Code - AI suggests compliant configurations

Follow our GitHub or check realm9.app for updates.

Questions? Drop them in the comments. I'll answer everything about the architecture, AI approach, or why we made certain decisions.

Introducing Realm9: Solving Enterprise Environment Chaos with AI

Prasad P — Sun, 02 Nov 2025 23:01:28 +0000

Introducing Realm9: Solving Enterprise Environment Chaos with AI

After spending years working with platform engineering teams, I kept hearing the same frustrations:

"QA booked the staging environment, but dev team also needs it for a critical demo."

"We're spending $60,000/year on Datadog for just 10GB/day of logs."

"Our engineers waste 40% of their time managing Terraform changes manually."

Sound familiar? That's why we built Realm9 - an AI-powered platform that addresses all three problems in a single, integrated solution.

The Problem: Environment Management is Broken

Most enterprise organizations manage 50-200+ environments across development, testing, and production. The coordination nightmare includes:

Problem 1: Booking Conflicts

Double-bookings: Two teams book the same environment
Idle waste: Environments sit unused while teams wait in queue
No visibility: Spreadsheets and email chains don't scale
Manual approvals: Managers become bottlenecks

Problem 2: Observability Costs

Datadog: $5,000+/month for 10GB/day
Splunk: $6,000+/month
Elastic Cloud: $2,000+/month
Total: $60K-200K/year for mid-sized teams

Problem 3: Terraform Workflow Friction

Manual editing: Error-prone, slow
Context switching: Engineers lose flow
No AI assistance: Unlike modern code editors
Git complexity: PR workflows add overhead

Why Existing Solutions Fall Short

ServiceNow CMDB: Complex enterprise software, not developer-friendly. Teams revolt against using it.

Plutora / Enov8: Enterprise pricing ($50K+/year licenses), heavyweight processes that slow down agile teams.

Spreadsheets: Everyone starts here. Breaks down at 50+ environments. No API integration, no automation.

DIY Solutions: Teams build custom tools, then spend 20% of engineering time maintaining them.

The Realm9 Architecture: Three Integrated Solutions

1. Smart Environment Booking System

Key Features:

Queue Management: Automatic prioritization with fairness algorithms
Multi-level Approvals: Role-based workflows (team lead → manager → director)
Shared Environments: Multiple teams can use same environment concurrently
Auto-release: Environments automatically freed when booking expires
Real-time Dashboard: See all environments, bookings, and availability

Example Workflow:

1. Developer requests staging-us-west for 4 hours
2. System checks availability and conflicts
3. If occupied, adds to queue with priority
4. Manager approves (if policy requires)
5. Developer gets access + Slack notification
6. Auto-release after 4 hours (or manual extension)

2. Built-in Observability (RO9)

This is where we get aggressive on cost.

Architecture: Multi-Tier Storage

┌─ Hot Tier (Redis)    → Last 15 min  → Zero latency
├─ Warm Tier (NVMe)    → Last 24 hours → Sub-10ms queries
├─ Cold Tier (S3)      → Last 30 days → Sub-100ms queries
└─ Archive (Glacier)   → 7 years      → 99% cost reduction

Technology Stack:

Apache Arrow IPC: Zero-copy data transfer, 10x compression
DuckDB: Vectorized query engine for analytical workloads
Parquet Format: Columnar storage with aggressive compression (15-25:1)
Bloom Filters: Sub-millisecond filtering across billions of events

Performance Design Goals:

Targeting 200K logs/second ingestion
Sub-50ms query latency (P99)
15-25:1 compression ratio
Estimated cost: from $75/month (vs $5,000+ for Datadog)

How We Achieve the Cost Savings:

Intelligent Tiering: Recent data hot, old data cold automatically
Columnar Compression: Store only what you query frequently
S3 Economics: Leverage cloud storage pricing (pennies per GB)
Zero Marketing Budget: We pass savings to customers

3. AI Terraform Co-Pilot (BYOK Model)

The standout feature: Bring Your Own Key (BYOK) for LLM providers.

Why BYOK?

Data Sovereignty: Your infrastructure conversations stay in your LLM account
Cost Control: You manage and optimize LLM spending directly
Provider Choice: Switch between OpenAI, Anthropic, Azure OpenAI
Compliance: Meet data residency requirements

Supported LLM Providers:

OpenAI (GPT-4o, GPT-4o-mini, GPT-5)
Anthropic (Claude 4.5 Sonnet, Claude 4.1 Opus)
Azure OpenAI (all OpenAI models via Azure)
Google Vertex AI (coming Q1 2025)
AWS Bedrock (coming Q1 2025)

What It Does:

You: "Create a VPC with public and private subnets across 3 AZs"

AI: [Reads your existing terraform files]
    [Generates HCL following best practices]
    [Updates files in editor]
    [Validates configuration]
    [Creates commit with descriptive message]

You: "Add a NAT gateway to the private subnets"

AI: [Understands context from previous changes]
    [Updates only relevant files]
    [Preserves existing resources]

Architecture: Model Context Protocol (MCP)

We built the AI on Model Context Protocol, an emerging standard for AI tool access. This gives the agent 45+ tools:

Database Tools: Project details, workspace info, cloud credentials
File Tools: Terraform file operations, Git status, file tree
Execution Tools: terraform plan, terraform apply, run logs
Git Tools: Commit, push, PR creation

Security Model:

Agent cannot bypass tool interface
All queries filtered by organization (multi-tenant isolation)
Redis TTL auto-cleanup prevents data leakage
No cross-project or cross-organization access

Technical Innovations

Innovation 1: Frontend/Backend Tool Separation

Traditional AI agents execute all operations immediately. This is dangerous for infrastructure.

Our Approach:

Backend Tools: Execute server-side (database queries, file reads)
Frontend Tools: Pause agent, request UI confirmation, resume with result

Example: terraform apply is a frontend tool. Agent generates plan, shows diff in UI, waits for human approval, then executes.

Innovation 2: Redis-Centric Ephemeral State

All agent session state lives in Redis (not PostgreSQL):

Fast Access: Sub-millisecond latency
Auto-Cleanup: TTL-based (no manual garbage collection)
Horizontal Scaling: Redis Cluster for high availability
Separation of Concerns: Persistent data in Postgres, ephemeral state in Redis

Innovation 3: Polling-Based Agent Communication

For Kubernetes observability agents:

Agents Make Outbound Calls Only: No inbound firewall rules needed
No Webhooks: Backend never calls agent directly
Simple Deployment: No load balancer, ingress, certificates required
Works Everywhere: NAT, firewalls, air-gapped environments

Security & Compliance

We designed Realm9 from day one with enterprise compliance in mind. While actual certification depends on your specific deployment and audit requirements, our architecture aligns with:

SOC 2 Type II Design:

✅ Logical access controls (MFA, RBAC)
✅ Comprehensive audit logging
✅ Encryption at rest and in transit
✅ Secure development lifecycle
✅ Incident response procedures

ISO 27001 Alignment:

✅ Information security management system (ISMS) design
✅ Access control policies (A.9)
✅ Cryptography controls (A.10)
✅ Operations security (A.12)

GDPR Compliance Architecture:

✅ Privacy by design
✅ Data minimization
✅ Right to erasure (data deletion APIs)
✅ Data portability (export functions)

HIPAA Ready (Healthcare):

✅ Access controls and audit logs
✅ Encryption standards (AES-256)
✅ Transmission security
✅ Business Associate Agreement (BAA) capable

Key Security Features:

API Key Security: SHA-256 hashed storage, HTTPS-only transmission
Multi-tenant Isolation: Organization-scoped access, no cross-contamination
BYOK Model: Your LLM keys, your data sovereignty
Network Security: Agents make outbound calls only

Cost Comparison: 3-Year TCO

Here's what we're seeing with early adopters:

Cost Category	Traditional Stack	Realm9	Estimated Savings
Environment Management	$70K-90K/year (Plutora/Enov8 license)	Included	$70-90K/year
Observability	$60K-120K/year (Datadog/Splunk)	From $900/year	$59-119K/year
Terraform Cloud	$20K-40K/year (Enterprise plan)	Included	$20-40K/year
Total Annual	$150K-250K	From $50K	$100-200K/year savings
3-Year TCO	$450K-750K	From $150K	$300-600K savings

Estimates based on mid-sized organizations (50-100 engineers). Your results may vary.

Real-World Use Case: Platform Engineering Team

Before Realm9:

120 environments across 5 cloud regions
Google Sheets for booking (broke down at 80 environments)
$84,000/year Datadog bill
8 hours/week managing Terraform changes manually
2-3 environment booking conflicts per week

After Realm9:

All 120 environments in unified dashboard
Zero booking conflicts (queue management + auto-release)
~$1,200/year observability costs (estimated 98% reduction)
AI handles 80% of Terraform changes (engineers review only)
Team freed up 32 hours/week for feature work

ROI Calculation:

Annual savings: ~$82,800 ($84K Datadog → ~$1.2K RO9)
Time savings: 32 hours/week × 52 weeks × $100/hour = $166,400/year
Total value: $249,200/year
Realm9 cost: ~$50K/year (estimated)
Net benefit: $199,200/year

Getting Started

GitHub Repositories (Open Source)

All our code is on GitHub under the realm9-platform organization:

realm9 - Main platform
ro9-observability - Log analytics
realm9-ai-agent - AI system
realm9-terraform - Terraform integration
realm9-multi-cloud - Cloud management
realm9-enterprise-security - Security architecture

Self-Hosted Deployment

# Deploy with Helm
helm install realm9 oci://public.ecr.aws/m0k6f4y3/realm9/realm9 \
  --namespace realm9 \
  --create-namespace \
  --set global.domain=your-domain.com \
  --set postgresql.auth.password=your-secure-password