Forem: anitaalicloud

How I Built SwiftDeploy: A Tool That Writes Its Own Infrastructure

anitaalicloud — Wed, 06 May 2026 19:28:10 +0000

A deep dive into declarative deployments, OPA policy gates, and chaos engineering from Stage 4A to 4B

Introduction

Most DevOps tasks ask you to configure infrastructure manually. This one asked me to build the tool that does it for me.

The result is SwiftDeploy which is a CLI tool that reads a single manifest.yaml file and generates your entire deployment stack from it. Nginx configs, Docker Compose files, policy checks, live metrics dashboards are all derived from one source of truth.

This post covers the full journey: the design decisions, the guardrails, the chaos, and the lessons learned.

The Architecture

Here is how all the pieces connect:

┌─────────────────────────────────────────────────────┐
│                    manifest.yaml                     │
│              (single source of truth)                │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
              ./swiftdeploy init
                       │
          ┌────────────┴────────────┐
          ▼                         ▼
     nginx.conf              docker-compose.yml
   (generated)                 (generated)
          │                         │
          ▼                         ▼
┌─────────────────────────────────────────────────────┐
│                   Docker Stack                       │
│                                                      │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐      │
│   │  Nginx   │───▶│   App    │    │   OPA    │      │
│   │  :8080   │    │  :3000   │    │  :8181   │      │
│   └──────────┘    └──────────┘    └──────────┘      │
│   (public)        (internal)      (internal)         │
└─────────────────────────────────────────────────────┘
          │                         ▲
          ▼                         │
     curl :8080              CLI queries OPA
    (your browser)          before deploy/promote

The key insight: you only ever touch manifest.yaml. The tool handles everything else.

Part 1 — The Design: A Tool That Writes Its Own Files

The Problem with Handwritten Config

When you write nginx.conf and docker-compose.yml by hand, you introduce drift. Change a port in one place and forget to update it in another. After a few weeks, nobody knows which file is the source of truth.

SwiftDeploy solves this with a three-layer system:

manifest.yaml          →    templates/*.tmpl    →    generated files
(VALUES)                    (STRUCTURE)              (VALUES + STRUCTURE)

The manifest.yaml holds all the values — ports, image names, modes, timeouts. The templates hold the structure — how nginx.conf and docker-compose.yml should look. The CLI combines them at runtime.

How `swiftdeploy init` Works

# Read manifest into a Python dict
m = yaml.safe_load(open("manifest.yaml"))

# Build a replacements map
replacements = {
    "{{NGINX_PORT}}":   str(m["nginx"]["port"]),
    "{{SERVICE_PORT}}": str(m["services"]["port"]),
    # ... etc
}

# Read template, replace placeholders, write output
with open("templates/nginx.conf.tmpl") as f:
    content = f.read()

for placeholder, value in replacements.items():
    content = content.replace(placeholder, value)

with open("nginx.conf", "w") as f:
    f.write(content)

Simple string replacement. No Jinja2, no templating engine — just Python's built-in str.replace(). The grader can delete nginx.conf and docker-compose.yml, run ./swiftdeploy init, and they regenerate perfectly every time.

The API Service

The API is a Python HTTP server using only the standard library — no Flask, no FastAPI. This keeps the Docker image under 60MB (well under the 300MB limit).

It runs in two modes controlled by a MODE environment variable injected by Docker Compose:

stable mode  →  normal behaviour
canary mode  →  adds X-Mode: canary header + activates /chaos endpoint

The same image runs both modes. The only difference is the environment variable.

The Nginx Reverse Proxy

Nginx sits in front of the app and adds:

X-Deployed-By: swiftdeploy header on every response
JSON error bodies on 502/503/504 (instead of ugly HTML)
Structured access logs in the required format
Forwards X-Mode header from the upstream app

Critically, the app port is never exposed directly. Only Nginx's port is mapped to the host. All traffic must flow through it.

Part 2 — The Guardrails: OPA Policy Enforcement

Why OPA?

The task required that the CLI never make allow/deny decisions itself. All logic must live in OPA (Open Policy Agent).

This matters because it separates concerns cleanly:

CLI  →  collects data, calls OPA, surfaces the result
OPA  →  owns all decision logic, never called by the app

If you want to change a policy, you edit a .rego file. You never touch the CLI. If you want to change a threshold, you edit data.json. You never touch the Rego files.

Policy Structure

Each policy domain owns exactly one question:

Infrastructure policy — Is the host healthy enough to deploy?

package infrastructure

import rego.v1

default allow := false

allow if {
    count(violations) == 0
}

violations contains msg if {
    input.disk_free_gb < data.infrastructure.min_disk_free_gb
    msg := sprintf(
        "Disk free (%.1fGB) is below minimum threshold (%.1fGB)",
        [input.disk_free_gb, data.infrastructure.min_disk_free_gb]
    )
}

violations contains msg if {
    input.cpu_load > data.infrastructure.max_cpu_load
    msg := sprintf(
        "CPU load (%.2f) exceeds maximum threshold (%.2f)",
        [input.cpu_load, data.infrastructure.max_cpu_load]
    )
}

Canary safety policy — Is the canary healthy enough to promote?

package canary

import rego.v1

default allow := false

allow if {
    count(violations) == 0
}

violations contains msg if {
    input.error_rate > data.canary.max_error_rate
    msg := sprintf(
        "Error rate (%.2f%%) exceeds maximum threshold (%.2f%%)",
        [input.error_rate * 100, data.canary.max_error_rate * 100]
    )
}

Threshold values live in data.json — never hardcoded in Rego:

{
  "infrastructure": {
    "min_disk_free_gb": 10.0,
    "max_cpu_load": 16.0,
    "min_mem_free_percent": 10.0
  },
  "canary": {
    "max_error_rate": 0.01,
    "max_p99_latency_ms": 500
  }
}

The Hard Gate in Action

When the CPU load exceeded the threshold, swiftdeploy deploy was blocked:

[deploy] Running OPA pre-deploy policy checks...
  Host stats: disk=80.08GB free, cpu_load=12.88, mem_free=50.0%
[policy] Checking Infrastructure...
  [BLOCK] Infrastructure policy FAILED:
    x CPU load (12.88) exceeds maximum threshold (2.00)
[deploy] BLOCKED by policy. Fix violations above before deploying.

The deploy never started. The CLI surfaced the exact violation reason from OPA — no guessing required.

OPA Isolation

OPA is intentionally isolated from public Nginx ingress. It runs on port 8181 inside the Docker network. It is NOT behind Nginx, and its port is only accessible to the CLI running on the host. A user hitting localhost:8080 cannot reach OPA.

Failure Handling

The CLI handles every distinct OPA failure mode differently:

except urllib.error.URLError as e:
    # OPA unreachable — warn but don't crash
    return {"allowed": False, "violations": [], 
            "error": f"OPA unreachable: {e.reason}"}

except json.JSONDecodeError:
    # OPA returned garbage — different message
    return {"allowed": False, "violations": [], 
            "error": "OPA returned invalid JSON"}

except Exception as e:
    # Catch-all — still doesn't crash
    return {"allowed": False, "violations": [], 
            "error": f"Unexpected OPA error: {e}"}

The CLI never crashes or hangs when OPA is unavailable. It warns the operator and continues.

Part 3 — The Chaos: Breaking Things on Purpose

The /metrics Endpoint

The API exposes a /metrics endpoint in Prometheus text format:

# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",path="/",status_code="200"} 42

# HELP http_request_duration_seconds Request latency
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{method="GET",path="/",le="0.005"} 38

# HELP app_mode Current deployment mode (0=stable, 1=canary)
# TYPE app_mode gauge
app_mode 1

# HELP chaos_active Current chaos state (0=none, 1=slow, 2=error)
# TYPE chaos_active gauge
chaos_active 0

No third-party libraries — pure Python calculating histogram buckets manually.

Injecting Chaos

After promoting to canary mode, chaos was injected:

# Slow mode — every request sleeps 3 seconds
curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "slow", "duration": 3}'

# Error mode — 50% of requests return HTTP 500
curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "error", "rate": 0.5}'

The Status Dashboard Capturing the Failure

With error mode active at 50%, the status dashboard showed:

=======================================================
  SwiftDeploy Status Dashboard
  2026-05-06T14:18:43Z
=======================================================

  Mode:        CANARY
  Uptime:      892s
  Req/s:       2.40
  P99 Latency: 250ms
  Error Rate:  48.20%

  Policy Compliance:
    + Infrastructure: PASSING
    x Canary Safety: FAILING
      -> Error rate (48.20%) exceeds maximum threshold (1.00%)

The canary safety policy immediately flagged the failure. Attempting to promote to stable at this point would have been blocked by OPA.

Recovery

curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "recover"}'

Within one scrape cycle the dashboard showed error rate back to 0% and canary safety back to PASSING.

Part 4 — The Audit Trail

Every significant event is written to history.jsonl:

{"event": "deploy_success", "timestamp": "2026-05-06T13:53:39Z"}
{"event": "promote_success", "target": "canary", "timestamp": "2026-05-06T14:01:22Z"}
{"event": "status_scrape", "mode": "canary", "error_rate": 0.482, "timestamp": "2026-05-06T14:18:43Z"}

Running ./swiftdeploy audit parses this file and generates a clean Markdown report:

## Timeline
| Timestamp | Event | Details |
|---|---|---|
| 2026-05-06T13:53:39Z | Deploy | Stack deployed successfully |
| 2026-05-06T14:01:22Z | Promote | Mode switched to canary |

## Policy Violations
| Timestamp | Policy | Reason |
|---|---|---|
| 2026-05-06T14:18:43Z | Canary Safety | error_rate=48.20%, p99=250ms |

Lessons Learned

1. Single source of truth is worth the extra complexity
It felt like overkill to build a template engine just to generate two config files. But when the grader deletes your generated files and reruns init, you're grateful every value comes from one place.

2. OPA's syntax changes between versions
The latest OPA image requires import rego.v1 and the if/contains keywords. Older Rego syntax silently fails to load. Always check your OPA container logs first.

3. Start OPA before running policy checks
OPA is part of the stack, so it doesn't exist before docker compose up. The fix was to start OPA first as a separate step, wait 3 seconds for it to load policies, then run the pre-deploy check.

4. Chaos engineering reveals what metrics matter
Before injecting chaos, the /metrics endpoint felt like box-ticking. After watching the error rate spike to 48% in real time on the status dashboard while OPA simultaneously flagged the canary safety policy — the value became obvious.

5. Policy as code beats policy as documentation
A README saying "don't deploy if CPU load is above 2.0" gets ignored. A Rego file that blocks the deploy enforces it automatically.

The Full Subcommand Reference

./swiftdeploy init              # generate nginx.conf + docker-compose.yml
./swiftdeploy validate          # 5 pre-flight checks
./swiftdeploy deploy            # OPA check + start stack + health wait
./swiftdeploy promote canary    # switch to canary mode
./swiftdeploy promote stable    # switch back to stable
./swiftdeploy status            # live metrics + policy compliance dashboard
./swiftdeploy audit             # generate audit_report.md
./swiftdeploy teardown          # stop all containers
./swiftdeploy teardown --clean  # stop + delete generated files

Source Code

The full project is available on GitHub: https://github.com/AnitaAliCloud/hng4-devops

Built as part of the HNG DevOps Track — Stage 4A and 4B

How I Built an Anomaly Detection Engine for DDoS Protection

anitaalicloud — Tue, 28 Apr 2026 03:56:36 +0000

Introduction
Imagine you run a busy website. On a normal day, about 50 people visit per second. Then suddenly, 5,000 requests flood in every second from a single IP address. Your server crashes. Your real users can't access anything. This is a DDoS (Distributed Denial of Service) attack.
In this post, I'll explain how I built a tool that watches incoming traffic in real time, learns what "normal" looks like, and automatically blocks attackers before they can cause damage.

What Does the Tool Do?
My anomaly detection engine does 6 things automatically:

Watches every HTTP request coming into the server
Learns what normal traffic looks like over time
Detects when traffic becomes abnormal
Blocks the attacker using the Linux firewall
Alerts me on Slack within 10 seconds
Unbans the IP automatically after a timeout

The Architecture
Internet Traffic
↓
Nginx (logs every request as JSON)
↓
Nextcloud (the actual app)

Detector Daemon reads Nginx logs
↓
Sliding Window → tracks request rates
Rolling Baseline → learns normal traffic
Z-score Detection → spots anomalies
iptables → blocks attackers
Slack → sends alerts
Dashboard → shows live metrics

Part 1 — How the Sliding Window Works
Think of the sliding window like a 60 second camera 🎥
Every request that comes in gets a timestamp. We store these timestamps in a Python deque (double-ended queue) — one for each IP address and one for global traffic.

from collections import deque
import time

One deque per IP

ip_windows = {}

def record_request(ip):
now = time.time()

if ip not in ip_windows:
    ip_windows[ip] = deque()

# Add this request
ip_windows[ip].append(now)

# Remove requests older than 60 seconds from the LEFT
cutoff = now - 60
while ip_windows[ip] and ip_windows[ip][0] < cutoff:
    ip_windows[ip].popleft()

# Current rate = how many requests in last 60 seconds
current_rate = len(ip_windows[ip]) / 60
return current_rate

from collections import deque
import time

One deque per IP

ip_windows = {}

def record_request(ip):
now = time.time()

if ip not in ip_windows:
    ip_windows[ip] = deque()

# Add this request
ip_windows[ip].append(now)

# Remove requests older than 60 seconds from the LEFT
cutoff = now - 60
while ip_windows[ip] and ip_windows[ip][0] < cutoff:
    ip_windows[ip].popleft()

# Current rate = how many requests in last 60 seconds
current_rate = len(ip_windows[ip]) / 60
return current_rate

The magic is the eviction — old timestamps get removed from the left side of the deque automatically. So the deque always contains only the last 60 seconds of requests. The current rate is simply the length of the deque divided by 60.

Part 2 — How the Baseline Learns from Traffic
The baseline answers one question: "What is normal?"
We can't hardcode this because every website is different. A news site might normally get 1000 req/s. A small blog might get 2 req/s. So we let the system learn.
Every second we record how many requests came in. Every 60 seconds we look at the last 30 minutes of data and calculate:

import math

def recalculate_baseline(per_second_counts):
# Calculate average requests per second
mean = sum(per_second_counts) / len(per_second_counts)

# Calculate how much it normally varies
variance = sum((x - mean) ** 2 for x in per_second_counts) / len(per_second_counts)
stddev = math.sqrt(variance)

# Apply floors to prevent false alarms on quiet traffic
effective_mean = max(mean, 1.0)
effective_stddev = max(stddev, 1.0)

return effective_mean, effective_stddev

We also maintain per-hour slots — the system prefers the current hour's data when it has enough samples. This means the baseline adapts to time-of-day patterns. Rush hour traffic looks different from 3 AM traffic!

Part 3 — How the Detection Logic Makes a Decision
Once we have the baseline we use a Z-score to decide if current traffic is anomalous.
The Z-score answers: "How many standard deviations away from normal is this?"

def is_anomalous(current_rate, mean, stddev):
# Z-score calculation
z_score = (current_rate - mean) / stddev

# Rate multiplier
rate_multiplier = current_rate / mean

# Flag as anomalous if EITHER condition fires
if z_score > 3.0:
    return True, "z-score exceeded 3.0"

if rate_multiplier > 5.0:
    return True, "rate exceeded 5x baseline"

return False, None

Example:

Normal traffic: 50 req/s (mean=50, stddev=10)
Attack traffic: 5000 req/s from one IP
Z-score = (5000 - 50) / 10 = 495
495 > 3.0 → ANOMALY DETECTED! 🚨

We also detect error surges — if an IP is getting lots of 404/500 errors it might be scanning for vulnerabilities. In that case we tighten the thresholds automatically.

Part 4 — How iptables Blocks an IP
iptables is Linux's built-in firewall. It runs in the kernel and can drop packets before they even reach your application.
When we detect an attack:

import subprocess

def ban_ip(ip):
# Add a DROP rule — silently discard all packets from this IP
subprocess.run([
"iptables", "-A", "INPUT",
"-s", ip,
"-j", "DROP"
])
print(f"Banned {ip}")

def unban_ip(ip):
# Remove the DROP rule
subprocess.run([
"iptables", "-D", "INPUT",
"-s", ip,
"-j", "DROP"

])
print(f"Unbanned {ip}")
The -j DROP means "jump to DROP action" — the packet is silently discarded. The attacker doesn't even get an error message back. From their perspective the server just stopped responding.
Bans lift automatically on a backoff schedule:

1st offence → 10 minutes
2nd offence → 30 minutes
3rd offence → 2 hours
4th+ → permanent

Part 5 — The Live Dashboard
The dashboard is a simple web page that refreshes every 3 seconds showing:

Global requests per second
Currently banned IPs
Top 10 source IPs
CPU and memory usage
Current baseline mean and stddev
System uptime

It's built using Python's built-in http.server — no web framework needed!

Part 6 — Slack Alerts
When an IP gets banned, a Slack message arrives within 10 seconds:
🚨 IP BANNED
IP: 192.168.1.100
Condition: Anomalous request rate
Current Rate: 450.00 req/s
Baseline: 12.00 req/s
Ban Duration: 10 minutes
Timestamp: 2026-04-28 03:22:36 UTC
And when the ban expires:
✅ IP UNBANNED
IP: 192.168.1.100
Reason: ban-expired
Timestamp: 2026-04-28 03:32:36 UTC

What I Learned
Building this project taught me:

Z-scores are powerful — a simple maths formula can detect attacks that would be impossible to catch with hardcoded thresholds
Baselines must be dynamic — hardcoding "block if > 100 req/s" is wrong because normal traffic varies by time of day
iptables is incredibly fast — kernel-level packet dropping happens before the request even reaches Python
Threading needs care — shared data structures need locks to prevent race conditions
Deques are perfect for sliding windows — O(1) append and popleft make them ideal for real-time rate tracking

Try It Yourself
The full source code is available at:
https://github.com/AnitaAliCloud/hng-stage3-devops
The live dashboard is running at:
http://anitacloud.duckdns.org:8080

Built as part of the HNG14 DevOps internship programme

Forem: anitaalicloud

How I Built SwiftDeploy: A Tool That Writes Its Own Infrastructure

Introduction

The Architecture

Part 1 — The Design: A Tool That Writes Its Own Files

The Problem with Handwritten Config

How swiftdeploy init Works

The API Service

The Nginx Reverse Proxy

Part 2 — The Guardrails: OPA Policy Enforcement

Why OPA?

Policy Structure

The Hard Gate in Action

OPA Isolation

Failure Handling

Part 3 — The Chaos: Breaking Things on Purpose

The /metrics Endpoint

Injecting Chaos

The Status Dashboard Capturing the Failure

Recovery

Part 4 — The Audit Trail

Lessons Learned

The Full Subcommand Reference

Source Code

How I Built an Anomaly Detection Engine for DDoS Protection

One deque per IP

One deque per IP

How `swiftdeploy init` Works