Forem: Ian alex

How I Built a DDoS Detection Tool From Scratch Using Go

Ian alex — Wed, 29 Apr 2026 22:27:39 +0000

A beginner-friendly guide to building an anomaly detection engine that watches web traffic in real time and automatically blocks attackers.

What This Project Does (And Why It Matters)
Imagine you run a website. Most days, you get maybe 50 visitors per hour. One afternoon, a single computer starts sending 10,000 requests per second. Your server slows to a crawl. Real users can't load the page. This is a DDoS attack — Distributed Denial of Service — and it's one of the most common threats on the internet.
My project is a daemon (a program that runs continuously in the background) that watches all incoming HTTP traffic to a Nextcloud server, learns what "normal" looks like, and automatically blocks IPs that behave abnormally. Think of it like a bouncer at a club who knows how busy a normal Friday night is and starts turning people away when the crowd gets suspiciously large — or kicks out the one person who keeps trying to barge through the door 100 times a minute.
The entire system is built in Go and runs alongside the web server inside Docker containers.

The Architecture at a Glance
Here's how the pieces fit together:

Internet Traffic
       |
       v
+------------------+
|      Nginx       |  ──> writes JSON access log
|   (reverse proxy)|       to /var/log/nginx/hng-access.log
+--------+---------+
         |
         v
+------------------+
|    Nextcloud     |   (the actual web app)
+------------------+

Meanwhile, running alongside...

+------------------------------------------+
|       Anomaly Detector Daemon            |
|                                          |
|  1. monitor.go   → reads the log file   |
|  2. detector.go  → counts requests      |
|  3. baseline.go  → learns "normal"      |
|  4. blocker.go   → bans bad IPs         |
|  5. unbanner.go  → lifts bans later     |
|  6. notifier.go  → alerts via Slack     |
|  7. dashboard.go → live web UI          |
+------------------------------------------+

Nginx writes every request as a JSON line to a log file. My detector tails that file (like running tail -f in a terminal), parses each line, and feeds it into the detection engine. Let me walk through each core concept.

How the Sliding Window Works
The first question the detector needs to answer is: how fast is this IP sending requests right now?

To answer that, I use a sliding window — a list that only keeps events from the last 60 seconds. Here's how it works in plain language:

Every time a request comes in from an IP, I record the timestamp.
Before counting, I throw away any timestamps older than 60 seconds.
The number of timestamps left = the number of requests in the last minute.
Divide by 60 = requests per second.

In code, this looks like:

type SlidingWindow struct {
    windowSeconds int
    events        []float64  // timestamps
}

func (sw *SlidingWindow) Add(timestamp float64) {
    sw.events = append(sw.events, timestamp)
    sw.evict(timestamp)  // remove old entries
}

func (sw *SlidingWindow) evict(now float64) {
    cutoff := now - float64(sw.windowSeconds)
    i := 0
    for i < len(sw.events) && sw.events[i] < cutoff {
        i++
    }
    if i > 0 {
        sw.events = sw.events[i:]  // chop off the old ones
    }
}

func (sw *SlidingWindow) Rate(now float64) float64 {
    sw.evict(now)
    if len(sw.events) == 0 {
        return 0
    }
    return float64(len(sw.events)) / float64(sw.windowSeconds)
}

I maintain two sliding windows:

Per-IP: Each IP address gets its own window. This catches a single attacker.
Global: One window for all traffic combined. This catches distributed attacks where many IPs flood you at once.

Why not just count requests per minute? Because a per-minute counter resets at minute boundaries. If an attacker sends 500 requests in the last 5 seconds of one minute and 500 in the first 5 seconds of the next, a per-minute counter would show 500 each minute. The sliding window correctly shows 1000 in 10 seconds — much more suspicious.

How the Baseline Learns From Traffic
Knowing the current rate isn't enough. Is 10 requests per second a lot? For Google, that's nothing. For a personal blog, that's an attack. The detector needs to learn what your normal traffic looks like.
This is where the rolling baseline comes in.
Every second, I record the total request count. I keep 30 minutes of these per-second counts. Every 60 seconds, I calculate two numbers:

Mean: The average requests per second over the last 30 minutes.
Standard deviation (stddev): How much the traffic varies from that average.

If your server normally gets 2 requests per second, with occasional bumps to 5, the mean might be 2.5 and the stddev might be 1.2.
Here's the clever part: I also maintain per-hour slots. Traffic at 3am is very different from traffic at 3pm. If I have enough data for the current hour, I use that hour's baseline instead of the global 30-minute window. This prevents the detector from thinking normal afternoon traffic is an attack just because the morning was quiet.

// Prefer current hour's slot if it has enough data
if slot, ok := b.hourlySlots[currentHour]; ok && slot.Size() >= 10 {
    rawMean = slot.Mean()
    rawStddev = slot.StdDev()
} else if len(b.window) >= 10 {
    // Fall back to full rolling window
    // ... calculate from all data
}

I also enforce floor values — the mean never goes below 1.0 and stddev never below 0.5. Without these floors, a server with zero traffic would have a mean of 0, and any single request would look like an anomaly. That would be useless.

How the Detection Logic Makes a Decision
Now we have two pieces of information:

Current rate for an IP (from the sliding window)
Normal rate for the server (from the baseline)

The detector flags traffic as anomalous if either of these conditions is true:
Condition 1: Z-Score > 3.0
The z-score answers: "How many standard deviations away from normal is this?"

z-score = (current_rate - mean) / stddev

If the mean is 2.5 req/s and stddev is 1.2, and an IP is sending 8 req/s:

z-score = (8 - 2.5) / 1.2 = 4.58

That's way above 3.0 — anomaly detected.
In statistics, a z-score above 3 means the value is in the top 0.1% — it almost certainly isn't normal traffic.

Condition 2: Rate > 5x the Baseline Mean
This is a simpler check. If normal traffic is 2 req/s and an IP is sending 15 req/s, that's 7.5x the baseline — clearly suspicious.
Why have both? Because if the stddev is very low (traffic is extremely consistent), the z-score threshold becomes very sensitive. The 5x multiplier acts as a sanity check.

Bonus: Error Surge Tightening
If an IP is generating a lot of 4xx and 5xx errors (failed login attempts, scanning for vulnerabilities), the detector automatically halves the thresholds. Now the z-score threshold drops from 3.0 to 1.5, and the rate multiplier drops from 5x to 2.5x. This catches attackers who are probing your server even if their request rate isn't extremely high.

How iptables Blocks an IP
When the detector identifies a bad IP, it needs to actually stop that traffic from reaching the server. This is where iptables comes in.
iptables is the Linux firewall. It's a set of rules that tell the operating system what to do with incoming network packets. The two commands that matter:

Ban an IP:

iptables -A INPUT -s 203.0.113.42 -j DROP

plaintext
This says: "Append a rule to the INPUT chain: any packet from source 203.0.113.42, DROP it." The traffic never reaches Nginx, never reaches Nextcloud. It's as if that IP doesn't exist.

Unban an IP:

iptables -D INPUT -s 203.0.113.42 -j DROP

plaintext

Same rule, but -D (delete) instead of -A (append).

In Go, I run these commands using os/exec:

func addIptablesRule(ip string) bool {
    cmd := exec.Command("iptables", "-A", "INPUT", "-s", ip, "-j", "DROP")
    _, err := cmd.CombinedOutput()
    return err == nil
}

plaintext

The Ban Escalation Schedule
Not every attacker deserves a permanent ban. Maybe it was a misconfigured bot, not a malicious attack. So bans follow an escalating schedule:

Ban	Duration
1st	2 minutes
2nd	30 minutes
3rd	2 hours
4th+	Permanent

An unbanner goroutine checks every 10 seconds for expired bans and removes them. If the same IP triggers detection again, the next ban is longer. By the 4th offense, they're blocked permanently.

The Dashboard
Everything above runs silently. To see what's happening in real time, the daemon serves a web dashboard that auto-refreshes every 3 seconds. It shows:

Global requests per second — how busy the server is right now
Banned IPs — who's blocked and for how long
Top 10 source IPs — who's sending the most traffic
CPU and memory usage — system health
Baseline stats — the current effective mean and standard deviation
Hourly slots — how traffic patterns differ by hour

The dashboard is built with Go's built-in net/http server and plain HTML with inline CSS. No JavaScript frameworks, no build tools. The auto-refresh is a single HTML meta tag:

<meta http-equiv="refresh" content="3">.

What I Learned
Building this project taught me several things:

Statistics has practical uses. Mean, standard deviation, and z-score aren't just textbook concepts — they're the foundation of anomaly detection in production systems. 2.** Go's concurrency model is elegant.** Each component (log monitor, unbanner, dashboard) runs in its own goroutine. They communicate through channels. No complex threading code needed.
Docker networking is tricky. The detector needs network_mode: host to run iptables on the host's network stack. Without it, iptables rules only affect the container's isolated network — useless for blocking real attackers.
Disk space matters. Access logs grow fast under heavy traffic. I learned this the hard way when my server ran out of space multiple times during testing.
Don't hardcode baselines. A static threshold like "ban anyone over 100 req/s" would either miss slow attacks or block legitimate traffic spikes. The rolling baseline adapts to whatever your actual traffic looks like.

Try It Yourself
The full source code is available on GitHub: https://github.com/ik-alex/HNG14-Stage3-Devops.git
The live dashboard is at: http://ikalex.duckdns.org:5000
If you want to understand security tooling, I'd recommend starting with just the sliding window. Write a small program that counts events per second using a deque. Once that clicks, everything else builds on top of it.

My First Task In HNG Internship

Ian alex — Wed, 03 Jul 2024 00:40:39 +0000

I signup for an internship program named HNG. It is expected that the intern should have an intermediate to advance experience for any track they wish to participate in. For more information regarding the internship, your can follow this link https://hng.tech/internship and applying for a job at HNG you can also checkout this link https://hng.tech/hire.

Task 1: We were tasked to write a script named create_user.sh for creating a user and adding the user to a group via reading from an input file.

#!/bin/bash
# Log file and password file
PASSWORD_FILE="/var/secure/user_passwords.txt"
LOG_FILE="/var/log/user_management.log"
# ensure to check if the number of argument provided is 1
# if !true exit running the entire codebase
if [ $# -ne 1 ]; then
    echo "Usage: $0 <input_textfile>" | sudo tee -a $LOG_FILE
    exit 1
fi

Considering the above code block;
#!/bin/bash the shebang declaration specifying that this file is a bash script.

PASSWORD_FILE="/var/secure/user_passwords.txt"
LOG_FILE="/var/log/user_management.log"

the above block of code assigns the path /var/secure/user_passwords.txt to variable PASSWORD_FILE and path /var/log/user_management.log to variable LOG_FILE

if [ $# -ne 1 ]; then
    echo "This is how to run the script: $0 <input_textfile>" | sudo tee -a $LOG_FILE
    exit 1
fi

The above block of code checks if only argument is passed to the script.

$# -ne 1 checks if the number of argument passed is not equal to one and prints the output to the terminal and also log the data.
else if the condition doesn't hold true it exits the block of the code.

if [ ! -f "$input_textfile" ]; then
    echo "Error: The file $input_textfile does not exists" | sudo tee -a $LOG_FILE
    exit 1
fi

! -f "$input_textfile this checks if an input file is not passed to the script it exits

sudo chown root:root $PASSWORD_FILE
sudo mkdir -p /var/secure
sudo touch $PASSWORD_FILE
sudo chmod 600 $PASSWORD_FILE
sudo touch $LOG_FILE
sudo chmod 640 $LOG_FILE

This Create necessary directories such as $LOG_FILE $PASSWORD_FILE and set permissions such as making the $PASSWORD_FILE have root administrative privilege and setting the permission to read and write privilege.
sudo chmod 640 $LOG_FILE this ensure that the user has a read and write privilege and the group has only read privilege.

generate_password() {
    < /dev/urandom tr -dc 'A-Za-z0-9!@#$%&*' | head -c 12
}

This function is responsible for generating random password

Read File

while IFS=';' read -r user groups; do
    if [ -z "$user" ] || [ -z "$groups" ]; then
        echo "Skipping invalid line: $user;$groups" | sudo tee -a $LOG_FILE
        continue
    fi

Start a loop that reads a line from the $FILENAME, splits it into two parts separated by ; based on IFS.
read -r user groups Assign the first part to username and the remaining parts to groups.
-z "$user" -z "$groups" checks to see if the user and group name is empty.

Creating users

if id -u "$user" >/dev/null 2>&1; then
        echo "This particular User $user exists" | sudo tee -a $LOG_FILE
    else
        sudo useradd -m "$user"
        if [ $? -eq 0 ]; then
            echo "User $user created" | sudo tee -a $LOG_FILE

            # Generating the random password for each user 
            password=$(generate_password)
            echo "$user,$password" | sudo tee -a $PASSWORD_FILE >/dev/null
            echo "$user:$password" | sudo chpasswd
            echo "User $user password is set" | sudo tee -a $LOG_FILE

            # Set appropriate permissions for the home directory
            sudo chmod 700 /home/$user
            sudo chown $user:$user /home/$user
            echo "Home directory for user $user set up with appropriate permissions" | sudo tee -a $LOG_FILE
        else
            echo "Failed to create user $user" | sudo tee -a $LOG_FILE
            continue
        fi
    fi

id -u "$user" >/dev/null 2>&1 this looks for the user id and suppress the standard output and error to /dev/null
sudo useradd -m "$user"
useradd: This is the command used to add a new user
-m: This option tells useradd to create a home directory for the new user if it does not already exist. The home directory will be created in the /home/ directory and named after the user.
"$user": This is the username of the new user being created. The $user variable should contain the name of the user
[ $? -eq 0 ] this checks if the previous command successfully executed and 0 indicates success.
password=$(generate_password) calls the generate_password function and assigns the result to the password variable.
echo "$user,$password" | sudo tee -a $PASSWORD_FILE >/dev/null this suppresses the output due to the /dev/null
echo "$user:$password" | sudo chpasswd this allows the user to change password.
echo "User $user password is set" | sudo tee -a $LOG_FILE displays the output to the terminal. sudo chmod 700 /home/$user gives the user a full privileged. sudo chown $user:$user /home/$user gives the owner of the directory to the user. else if the condition doesn't hold true it print the output of failed user creation to the terminal.

Adding Users to Group

IFS=',' read -r -a group_array <<< "$groups"
    for group in "${group_array[@]}"; do
        if getent group "$group" >/dev/null 2>&1; then
            sudo usermod -aG "$group" "$user"
            echo "User $user added to existing group $group" | sudo tee -a $LOG_FILE
        else
            sudo groupadd "$group"
            sudo usermod -aG "$group" "$user"
            echo "Group $group created and user $user added to it" | sudo tee -a $LOG_FILE
        fi
    done
done < "$input_textfile"

-IFS=',' read -r -a group_array <<< "$groups"

IFS=',': Sets the Internal Field Separator to a comma. This means the read command will split the input string based on commas.
read -r -a group_array <<< "$groups" Reads the group variable, splits it by comma and stores the value to the group_array.
group=$(echo "$group" | xargs) this removes any leading whitespace in the group.
for group in "${group_array[@]}" this loops through the group_array array and stores each iteration to group.
if getent group "$group" >/dev/null 2>&1 if the group exists in the system; also suppress the standard output and error.
sudo usermod -aG "$group" "$user" adds users to the existing group
sudo groupadd "$group" this creates a new group.
sudo usermod -aG "$group" "$user" this adds user to the group
echo "Group $group created and user $user added to it" | sudo tee -a $LOG_FILE prints the output and log it into the log file.
done ends the for loop
done < "$input_textfile" ends the while loop that reads from the input file.
echo "User creation and group assignment created." | sudo tee -a $LOG_FILE outputting the finished the creation of users and group.

Running The Script

created the file named called name-of-text-file.txt

nano name-of-text-file.txt

#file content of the file
kachi; security, crypto, signals
dika; werey, genuis, smartkid
diamond; werey, soc, faith
chimummy; boss, theboss, smartguy
david; psycho, funny, jovial
faith; babe, babygirl, fine

execute the script create_userss.sh with the text file name-of-text-file.txt

# making the script file to be executable
chmod +x create_userss.sh
# running the script
./create_userss.sh name-of-text-file.txt

checking the LOG_FILE

sudo cat /var/log/user_management.log

The display output for the log file

checking the password file PASSWORD_FILE

sudo cat /var/secure/user_passwords.txt

This displayed output for the password file