Forem: Ebi Soroush

A simple api gateway from scratch written in golang

Ebi Soroush — Mon, 30 Mar 2026 20:01:14 +0000

Building an API Gateway From Scratch in Go

API gateways are one of those infrastructure components that feel intimidating from the outside. You know they handle auth, routing, rate limiting, but the internals are a black box. I decided to fix that by building one from scratch in Go, using nothing but the standard library's net/http. No frameworks. Just code.

This post walks through the design of simple-api-gateway: what it does, how it's structured, and the key implementation decisions along the way.

The Core Idea: A Gateway is Just a Middleware Chain

Before diving in, it's worth naming the central insight: an API gateway is fundamentally a configurable http.Handler. Every feature, auth, rate limiting, tracing, is just middleware wrapped around a reverse proxy.

In Go, that pattern looks like this:

type Middleware func(http.Handler) http.Handler

A middleware takes a handler and returns a new handler that does something before or after calling the next one. Chain enough of these together and you have a gateway.

The Gateway struct implements http.Handler directly:

type Gateway struct {
    config     *types.Config
    router     *Router
    middleware map[string]Middleware
}

func (gw *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request) {

    route, params := gw.router.Match(r)
    if route == nil {
        http.NotFound(w, r)
        return
    }

    handler := gw.buildChain(route)
    handler.ServeHTTP(w, r)
}

No magic, just a struct with a method. This also means the gateway is fully testable without a live server. You can call ServeHTTP directly in a test.

Routing

I designed the router to handle three path patterns: exact (/users), parameterised (/users/:id), and prefix wildcards (/users/*). Routes are sorted by specificity at startup so more precise matches always win.

func (ro *Router) Match(r *http.Request) (*types.Route, map[string]string) {
    for _, route := range ro.routes { // sorted: exact > param > prefix
        if matched, params := matchPath(route.Path, r.URL.Path); matched {
            if methodAllowed(route.Methods, r.Method) {
                return &route, params
            }
        }
    }
    return nil, nil
}

One deliberate limitation: :param segments are matched and used for routing, but the extracted values aren't injected into the request context. For the purpose of this project, learning the gateway pattern, that tradeoff was acceptable, but it's a clear gap for production use.

The Middleware Chain

When a route is matched, its configured middleware is assembled into a chain and applied around the upstream proxy:

func (gw *Gateway) buildChain(route *types.Route) http.Handler {
    handler := gw.proxy.Forward(route.Target)

    // Apply in reverse so the first listed middleware is outermost
    for i := len(route.Middleware) - 1; i >= 0; i-- {
        name := route.Middleware[i]
        if mw, ok := gw.middleware[name]; ok {
            handler = mw(handler)
        }
    }
    return handler
}

The order middleware is listed in config.yaml is the order requests traverse the chain, so if you list [tracing, auth, ratelimit, logging, metrics], tracing wraps everything, auth runs next, and so on. Getting this ordering right matters a lot in practice.

This was designed so each built-in is registered at startup and custom middleware can be added before calling Start():

gw := core.NewGateway(cfg)
gw.Register("my-middleware", func(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // do something before
        next.ServeHTTP(w, r)
        // do something after
    })
})
gw.Start()

Authentication

The auth middleware validates JWT Bearer tokens and injects verified claims into the request context:

func NewAuthMiddleware(verifier TokenVerifier) Middleware {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            token := extractBearer(r)
            if token == "" {
                writeError(w, http.StatusUnauthorized, "missing token")
                return
            }

            claims, err := verifier.Verify(token)
            if err != nil {
                writeError(w, http.StatusUnauthorized, "invalid token")
                return
            }

            ctx := context.WithValue(r.Context(), claimsKey{}, claims)
            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }
}

The middleware itself doesn't care how a token is verified, that's delegated to a TokenVerifier interface:

type TokenVerifier interface {
    Verify(token string) (Claims, error)
}

This separation means swapping in a different authentication backend is a matter of implementing one interface, with no changes to the middleware itself. The active verifier is selected at startup based on the auth.type field in config — currently jwt is the built-in implementation, but plugging in something like a Keycloak OIDC verifier would just mean implementing Verify against Keycloak's token introspection endpoint and setting type: keycloak in your config:

auth:
  enabled: true
  type: keycloak # swap without touching middleware code
  # keycloak-specific config would go here

Downstream handlers retrieve the injected claims with middleware.ClaimsFromContext(ctx), regardless of which verifier produced them.

Rate Limiting

Rate limiting uses a per-IP token bucket from golang.org/x/time/rate:

func (rl *rateLimiter) getLimiter(ip string) *rate.Limiter {
    rl.mu.Lock()
    defer rl.mu.Unlock()

    if lim, ok := rl.clients[ip]; ok {
        return lim
    }

    lim := rate.NewLimiter(rate.Limit(rl.rps), rl.burst)
    rl.clients[ip] = lim
    return lim
}

This works fine for a single instance but has two limitations worth knowing: the map has no eviction, so memory grows unbounded over time; and there's no shared state across instances, so it breaks under horizontal scaling. A production system would push this to Redis. For a learning project, the simplicity is worth it.

Observability

Three layers of observability are built in.

Logging uses Go 1.21's slog with configurable JSON or text output. Each request gets a structured log entry with method, path, status, and duration.

Metrics are exposed via Prometheus at a configurable path (default /metrics). Three instruments are tracked:

gateway_requests_total{method, path, status}
gateway_request_duration_seconds{method, path}
gateway_requests_in_flight

The /metrics endpoint itself is served before route matching, bypassing all middleware, otherwise you'd need to auth-exempt your own scraper.

Distributed tracing uses OpenTelemetry. The tracing middleware creates a server span for each request and propagates a traceparent header to the upstream, so traces stitch together across service boundaries. Spans are exported to Jaeger over gRPC OTLP.

One subtle gotcha: tracing must be explicitly listed in each route's middleware array. It is not in the internal isKnownMiddleware registry, so if you omit it, there's no warning — requests just won't be traced.

Configuration

Everything is controlled by a single config.yaml, parsed strictly with gopkg.in/yaml.v3's KnownFields — unknown keys are rejected at startup rather than silently ignored:

server:
  port: 8080

routes:
  - path: /users
    methods: [GET]
    target: http://localhost:9002
    middleware: [tracing, auth, ratelimit, logging, metrics]

auth:
  enabled: true
  type: jwt
  secret: "your-secret"

rate_limit:
  enabled: true
  requests_per_second: 5
  burst: 4

metrics:
  prometheus:
    enabled: true
    path: /metrics
  otel:
    enabled: true
    endpoint: localhost:4317 # host:port only, no scheme

Running It

# Start Jaeger for tracing
docker compose up -d

# Start the gateway + mock upstream services
make run-example

# Generate a dev JWT and hit an endpoint
make get-token
# Run this a couple of time to have some request with tracing
curl -H "Authorization: Bearer <token>" http://localhost:8080/users

# Inspect traces and metrics
open http://localhost:16686       # Jaeger UI
curl http://localhost:8080/metrics

Observability in Action

Here's what the observability stack looks like when the gateway is running.

Prometheus metrics endpoint

Hitting /metrics exposes all three instruments in standard Prometheus text format — request counts broken down by method, path, and status code, latency histograms, and the current in-flight count.

Structured JSON logs

With logging.format: json set in config, every request produces a structured log entry. This makes logs trivially parseable by tools like Loki, Datadog, or any log aggregator.

Jaeger trace view

Each request generates a server span with timing and metadata. Because traceparent is propagated to the upstream, the full trace stitches together across the gateway and the backend service in a single waterfall view.

Jaeger service graph

The service graph shows the call relationships between the gateway and its upstream services, making it easy to see which services are being hit and at what rate.

What I Took Away

Building this clarified something I'd always glossed over: the complexity in production gateways (Kong, Traefik, AWS API Gateway) isn't in the core concept, it's entirely in the operational concerns layered on top. Distributed rate limiting, hot config reloading, circuit breaking, mTLS, dynamic service discovery. The fundamental pattern, a middleware chain in front of a reverse proxy, is surprisingly approachable.

If you want to genuinely understand a piece of infrastructure, build a limited version of it. The gaps in your mental model become impossible to ignore once you're responsible for filling them.

The full source is on GitHub at github.com/realEbi/simple-api-gateway.

A simple load balancer from scratch written in golang

Ebi Soroush — Sun, 08 Feb 2026 10:56:56 +0000

I implemented a simple HTTP load balancer from scratch in Golang to deepen my understanding of a load balancer's internal processes and to brush up on my Go skills.
This project is designed to be simple yet cover the main objectives of a load balancer. In the architecture design, I extracted its modules to have separate responsibilities, simplifying testing and future development.

Load Balancer

The core functionality and responsibility of a load balancer is to handle incoming network traffic. In terms of network layers, there are two main types of load balancers:

HTTP and gRPC-based, which operate at Layer 7 (the Application Layer).
TCP/UDP-based, which operate at Layer 4 (the Transport Layer).

We can summarize the core functionality of a load balancer as follows:

Accepts an incoming request from a client.
Forwards the request to a backend server from its pool of backends.
Returns the backend's response to the client.

There are many more details to the functionality of a load balancer, such as:

Synchronous vs. streaming responses
Different algorithms to pick the backend for each request
Error handling
Session affinity and sticky sessions
Monitoring and observability techniques

And more, all of which are crucial for any production system. However, for this simple load balancer project, we will not focus on these advanced topics.

For production use cases, it's recommended to use one of the open-source, cloud-native, or managed solutions:

Solution / Service	Type / Role	OSI Layer	Notes
NGINX	Reverse proxy, web server, LB	L7	Path routing is a core configuration feature (`location` blocks).
Envoy	L7 proxy / service proxy	L7	Core model: Listener → Route → Cluster; routing is fundamental.
AWS Application Load Balancer	Managed application LB	L7	Supports path and host-based routing rules.
AWS Network Load Balancer	Managed network LB	L4	TCP/UDP only; no HTTP awareness.
GCP HTTP(S) Load Balancer	Managed global application LB	L7	Supports host and path routing globally.
GCP TCP/UDP Load Balancer	Managed network LB	L4	No Layer-7 inspection or routing.
Azure Application Gateway	Managed application LB	L7	Provides host/path routing and WAF integration.
Azure Load Balancer	Managed network LB	L4	Basic TCP/UDP distribution only.

Simple Load balancer

In this project, I implemented a simple HTTP load balancer that has the following core features:

Traffic Proxying: HTTP request proxying with different load balancing strategies.
Load Balancing Strategies: The load balancer supports multiple strategies for selecting backend servers:
- Round Robin: Distributes requests to backend servers in a circular order.
- Weighted Round Robin: Distributes requests based on the weight assigned to each server.
- Least Connections: Sends requests to the server with the fewest active connections.
- Random: Selects a backend server randomly.
Health Checks: The load balancer periodically checks the health of backend servers using a TCP dial. Unhealthy servers are temporarily removed from the rotation.
Request Retry: If a request to a backend server fails, the load balancer will retry the request on a different server.
Configuration: The load balancer can be configured using a JSON file (config.json) or command-line flags. The configuration includes the load balancer's port, request timeout, health check interval, load balancing strategy, and a list of backend servers.

Architecture and Intent

The primary goal of this project's architecture is to be modular, testable, and easy to understand. The design follows the principle of Separation of Concerns, where each part of the system has a single, well-defined responsibility. This not only makes the code cleaner but also mimics patterns found in production systems, making it a valuable learning exercise.

By isolating logic—for instance, completely separating the logic for choosing a backend from the logic for proxying a request—we can test each part independently and add new features with minimal side effects.

High-Level Overview

At its core, the load balancer's job is to receive an HTTP request and decide which backend server should handle it. The interaction between the main components can be visualized like this:

The process is as follows:

The LoadBalancer receives a request.
It asks the current Strategy to choose a backend from the pool.
The LoadBalancer then uses the chosen Backend's reverse proxy to forward the request.

Let's look at each component in more detail.

Component Breakdown

`strategy` Module: The Brains

The most important architectural decision was to define backend selection logic using an interface. This is an application of the classic Strategy Design Pattern.

type LoadBalancingStrategy interface {
    SelectBackend(pool []*Backend) *Backend
}

The LoadBalancer holds a variable of this interface type. This means we can create any number of strategies (Round Robin, Least Connections, etc.) that fulfill this contract. To change the load balancer's behavior, we simply swap out the implementation, without touching the LoadBalancer's core code. This makes the system incredibly flexible and extensible.

`backend` Module: The Worker

A Backend is not just a URL string; it's a stateful object responsible for its own state.

type Backend struct {
    Url               url.URL
    proxy             *httputil.ReverseProxy
    isHealthy         bool
    activeConnections int64
    // ...
    mu                sync.RWMutex
}

It tracks its health, the number of active connections, and its configured weight. It also holds the httputil.ReverseProxy instance that performs the actual request forwarding. The mutex ensures that its state can be safely updated and read by multiple concurrent requests and health checks.

`loadbalancer` Module: The Coordinator

This is the central component that ties everything together.

type LoadBalancer struct {
    pool                []*Backend
    strategy            LoadBalancingStrategy
    // ...
}

Its responsibilities include:

Managing the pool of Backend objects.
Handling incoming HTTP traffic and using the current Strategy to pick a Backend.
Performing periodic health checks on all backends in its pool.
Implementing retry logic if a request to a chosen backend fails.

`config` Module: The Blueprint

A simple but important module responsible for loading and parsing the application's configuration from a config.json file. This decouples the load balancer's logic from hard-coded settings, allowing you to reconfigure it without changing the code.

Conclusion

In this post, we walked through the process of building a simple yet functional HTTP load balancer from scratch in Go. We implemented several core features, including multiple load balancing strategies, periodic health checks, and dynamic configuration. The result is a working application that demonstrates the fundamental principles of traffic management.

More importantly, this project was a practical exploration of key software design patterns. By using an interface for our balancing algorithms (the Strategy Pattern), we created a system that is flexible and easy to extend. The emphasis on modularity and separation of concerns resulted in a codebase that is clean, testable, and easier to reason about.

While our load balancer is simple, it provides a solid foundation that could be extended in many ways. Future improvements could include:

More Advanced Strategies: Implementing IP hashing for session affinity (sticky sessions).
Enhanced Observability: Adding Prometheus metrics for monitoring request latency, error rates, and active connections.
HTTPS Support: Adding TLS termination for secure communication.
Dynamic Configuration: Implementing hot-reloading of the configuration file without restarting the service.

I hope this article has been an insightful look into the internals of a load balancer and has inspired you to build your own. Feel free to explore the complete source code on GitHub, try it out for yourself, and even contribute your own ideas!