How I Built a Real-Time DDoS Detection Engine as a DevOps Intern

Doris Okereke — Wed, 29 Apr 2026 19:22:38 +0000

I am Doris Okereke, a DevOps intern at HNG Internship Stage 3. When I saw this task, my first reaction was panic. I had never heard of iptables, I had never written Go code, and I barely understood what DDoS meant. But I built it anyway, and this post is my honest story of how I did it, what broke, and what I learned.

What Is This Project and Why Does It Matter?

Imagine you run an online shop. Normally about 10 customers walk through your door every minute. Suddenly 5,000 people flood in at once, blocking the entrance so real customers cannot get in. That is a DDoS attack.

DDoS stands for Distributed Denial of Service. Attackers use thousands of computers to flood your server with requests until it crashes or becomes too slow to serve real users. Companies lose millions of dollars to these attacks every day.

My task was to build a system that watches all traffic coming into a Nextcloud server, learns what normal traffic looks like, detects when something is wrong, and automatically blocks attackers without any human intervention.

My Setup

Before I could write a single line of code, I had to set up my environment. I used a Google Cloud Platform virtual machine running Ubuntu 22.04 with 2 vCPUs and 4GB RAM. I chose GCP because I already had free credits from previous HNG stages.

I used Docker and Docker Compose to run all my services in containers. Containers mean each service runs in isolation without interfering with others.

I chose Go as my programming language because it compiles to a single binary, uses very little memory, and has goroutines which are lightweight threads perfect for running multiple things at the same time.

I used DuckDNS for a free domain name pointing to my server.

My Architecture

Here is how everything connects:

External Traffic
      ↓
Nginx (Reverse Proxy on Port 80)
      ↓ writes JSON logs
HNG-nginx-logs (Shared Docker Volume)
      ↓ reads logs
My Detector Daemon (Go Service)
      ↓ when attack detected
iptables (Linux Kernel Firewall) blocks the IP
      ↓
Slack Webhook sends me an alert
      ↓
Live Dashboard on Port 9090 shows everything in real time

Part 1: Setting Up Nginx as a Reverse Proxy with JSON Logging

Nginx sits in front of everything. Every request from the internet hits Nginx first before reaching Nextcloud. This is called a reverse proxy.

The most important thing I configured was the log format. By default Nginx writes logs in plain text that is hard to parse programmatically. I changed it to JSON so my Go detector could easily read and understand every log line.

Here is my nginx.conf file:

events { worker_connections 1024; }

http {
    log_format json_combined escape=json
        '{'
        '"source_ip":"$remote_addr",'
        '"timestamp":"$time_iso8601",'
        '"method":"$request_method",'
        '"path":"$request_uri",'
        '"status":$status,'
        '"response_size":$body_bytes_sent'
        '}';

    access_log /var/log/nginx/hng-access.log json_combined;

    server {
        listen 80;
        server_name doris-stage3.duckdns.org;

        location /dashboard {
            proxy_pass http://172.18.0.1:9090/;
        }

        location / {
            proxy_pass http://nextcloud:80;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $remote_addr;
        }
    }
}

The log_format defines what information is captured for every request. source_ip is the real IP address of the visitor. method is GET or POST. status is the HTTP response code like 200 for success or 404 for not found.

The access_log line tells Nginx to write logs to a file inside a Docker volume shared with my detector.

The difference between access log and error log is this. The access log records every HTTP request; every visitor, every page load. The error log records problems with Nginx itself. My detector only reads the access log because it cares about traffic patterns.

Part 2: Docker Compose — Orchestrating All Services

Here is my complete docker-compose.yml with explanations:

version: '3.8'

volumes:
  HNG-nginx-logs:
    name: HNG-nginx-logs

networks:
  hng-net:
    driver: bridge

services:
  nextcloud:
    image: kefaslungu/hng-nextcloud
    restart: always
    networks:
      - hng-net
    volumes:
      - HNG-nginx-logs:/var/log/nginx:ro

  nginx:
    image: nginx:alpine
    restart: always
    ports:
      - "80:80"
    networks:
      - hng-net
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - HNG-nginx-logs:/var/log/nginx

  detector:
    build: ./detector
    restart: always
    network_mode: host
    cap_add:
      - NET_ADMIN
    volumes:
      - HNG-nginx-logs:/var/log/nginx:ro

The HNG-nginx-logs volume is a shared hard drive. Nginx writes logs to it. My detector reads from it. They never talk to each other directly; they share this common storage.

The hng-net network is a private internal network. Nginx can reach Nextcloud by name because they are both on this network. Docker's internal DNS resolves container names automatically.

The restart: always means Docker automatically restarts any service that crashes.

The :ro at the end of volume mounts means read-only. Nextcloud and the detector can read logs but cannot write to them. Only Nginx writes logs.

The most important part is the detector configuration. I used network_mode: host instead of the Docker network and added cap_add: NET_ADMIN. This is because iptables rules only apply to the network namespace they run in. If my detector ran inside Docker's isolated network, its iptables commands would only block traffic inside the container. By using host networking, when my detector runs iptables it modifies the actual server firewall that real external traffic passes through.

Part 3: The Detector: My Go Daemon

monitor.go: Reading the Log File

My monitor opens the Nginx log file and seeks to the end. It then reads new lines as they appear in real time. Each line is parsed from JSON into a Go struct containing IP, timestamp, method, path, status, and response size.

func (m *Monitor) tailFile() error {
    f, _ := os.Open(m.logPath)
    f.Seek(0, io.SeekEnd)
    reader := bufio.NewReader(f)

    for {
        line, err := reader.ReadString('\n')
        if err == io.EOF {
            time.Sleep(100 * time.Millisecond)
            continue
        }
        var entry LogEntry
        json.Unmarshal([]byte(line), &entry)
        m.detector.Process(entry)
    }
}

baseline.go: Learning What Normal Looks Like

Every second my baseline records how many requests came in. After 30 minutes it has 1800 data points. Every 60 seconds it calculates the mean (average requests per second) and standard deviation (how much traffic varies from the mean).

I also maintain separate baselines per hour of the day because traffic at 2am looks different from traffic at 2pm.

The floor values prevent division by zero. Mean cannot go below 0.1 and standard deviation cannot go below 0.1.

detector.go: The Sliding Window

I maintain a list of timestamps for each IP address. When an IP makes a request, I add the current timestamp to the right side of the list. I then remove all timestamps from the left that are older than 60 seconds. The length of this list divided by 60 gives me requests per second for that IP.

win.requests = append(win.requests, now)
win.requests = evictOld(win.requests, cutoff)

func evictOld(times []time.Time, cutoff time.Time) []time.Time {
    i := 0
    for i < len(times) && times[i].Before(cutoff) {
        i++
    }
    return times[i:]
}

I maintain two windows: one per IP and one globally. The per-IP window catches a single aggressive attacker. The global window catches distributed attacks spread across many IPs.

My detection uses three methods. Z-score detection calculates Z = (current_rate - mean) / stddev. If Z exceeds 3.0 the IP is flagged. A Z-score of 3.0 means the traffic is statistically anomalous — something that only happens 0.3% of the time in normal conditions. Rate multiplier detection flags any IP whose rate exceeds 5 times the baseline mean. Error surge detection tightens thresholds by 30% when an IP generates 4xx and 5xx errors at 3 times the baseline error rate.

blocker.go — Blocking with iptables

cmd := exec.Command("iptables", "-I", "INPUT", "-s", ip, "-j", "DROP")
cmd.Run()

iptables is Linux's built-in packet filtering firewall. This command inserts a rule into the INPUT chain telling the kernel to DROP all packets from the attacker's IP before they reach any application. This is kernel-level blocking — extremely fast.

unbanner.go — Automatic Release

My system checks every 10 seconds if any bans have expired. The release schedule is progressive. First ban lasts 10 minutes in case it was accidental. Second ban lasts 30 minutes. Third ban lasts 2 hours. Fourth ban is permanent.

notifier.go — Slack Alerts

Every ban and unban sends a message to my Slack channel. I call SendSlack using the go keyword which means it runs in a background goroutine. The main detector never waits for Slack to respond. Banning happens instantly even if Slack is slow.

config.yaml: All Thresholds in One Place

zscore_threshold: 3.0
rate_multiplier_threshold: 5.0
error_rate_multiplier: 3.0
sliding_window_seconds: 60
baseline_window_minutes: 30
baseline_recalc_interval: 60
unban_schedule:
  - 600
  - 1800
  - 7200
  - -1

Nothing is hardcoded in my Go files. Every threshold and window size lives in this file. To change the Z-score threshold I edit this file and restart the detector with no recompilation needed.

The Challenges I Faced and How I Overcame Them

My biggest struggle was SSH authentication. I could not connect to my server from my laptop for a long time. The problem was GCP creates users based on the SSH key comment, not the username I typed. My key was registered as user bluechip but I kept trying to log in as dorisjenny27. Once I ran ssh bluechip@35.232.253.197 it connected immediately.

I spent hours trying to get the dashboard accessible via domain name. The problem turned out to be that my DuckDNS domain was pointing to my laptop IP address instead of the server IP. On top of that my detector had banned my laptop IP which was blocking all my connections. I cleared the bans with sudo iptables -F and updated DuckDNS to the correct server IP.

The Go build failed initially because my server had Go 1.18 installed but my code needed Go 1.21. I downloaded and installed Go 1.21 manually from the official Go website.

Docker networking confused me deeply at first. I could not understand why the detector needed network_mode: host. After understanding that iptables rules only apply within their network namespace it made complete sense.

Testing: Did It Actually Work?

I tested my system by running a load test from my laptop using k6:

export const options = {
  stages: [
    { duration: '10s', target: 100 },
    { duration: '30s', target: 500 },
  ],
};

export default function () {
  http.get('http://35.232.253.197/');
}

This sent 3002 requests ramping up to 500 virtual users. My detector flagged my laptop IP within seconds, added an iptables DROP rule, and sent a Slack notification showing the IP, rate of 0.42 req/s against a baseline of 0.10 req/s, and a ban duration of 600 seconds. After 10 minutes the ban was automatically released and another Slack notification confirmed the unban.

How to Run This Project From Scratch

Provision an Ubuntu 22.04 server with at least 2 vCPU and 2GB RAM.

Install dependencies:

sudo apt update && sudo apt install -y docker.io docker-compose-v2 git wget
wget https://go.dev/dl/go1.21.0.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.0.linux-amd64.tar.gz
export PATH=/usr/local/go/bin:$PATH

Clone and configure:

git clone https://github.com/dorisjenny27/hng-stage3-devops.git
cd hng-stage3-devops

Edit detector/config.yaml and set your Slack webhook URL. Then start everything:

sudo docker compose up -d

Access the dashboard at http://your-server-ip:9090

GitHub Repository

https://github.com/dorisjenny27/hng-stage3-devops

Conclusion

Building this project taught me more in three days than I learned in months of reading. I now understand how reverse proxies work, how Docker volumes enable inter-container communication, how Linux firewalls operate at the kernel level, and how statistical methods detect anomalies in real time.

The most important thing I learned is that DevOps is not just about running commands. It is about understanding why each component exists and how they connect to form a complete system.

If you are a beginner reading this, start with the docker-compose.yml to understand the architecture. Then read monitor.go for log tailing. Then baseline.go for statistics. Then detector.go for decision making. Each file is small and focused on one job. You can build this too.

Forem: Doris Okereke