Forem: Francis Awuor

Thinking in Pipelines: A Better Way to Structure Go Systems

Francis Awuor — Sun, 03 May 2026 16:55:14 +0000

I was working on a personal project recently.

A job scraper.

And in the process, I came across a pattern that’s genuinely changed how I think about structuring backend systems in Go.

It's called the Pipeline Pattern. And as it turns out, it actually shows up in a lot of places - payments, analytics, APIs, etc.

In this article, I’ll be walking you through it using my job scraper project. Which is actually a perfect use case for this pattern, as you’ll see.

The Mess We're Trying to Avoid

Now before I show you the pattern, let me show you what the code could look like without it.

Now essentially what my scraper does is this:

Scrape job listings from multiple sources
Normalize them (i.e., clean them up)
Score them based on keywords (The most relevant to my skillset get the highest scores)
Save them to a database.

So initially what I might have done, was something like this (simplified):

for _, raw := range rawJobs {
    // normalize
    raw.Title = strings.TrimSpace(raw.Title)
    raw.Location = strings.ReplaceAll(raw.Location, "NYC", "New York")

    // score
    score := 0
    for _, keyword := range keywords {
        if strings.Contains(raw.Title, keyword) {
            score++
        }
    }

    // save
    s.Repo.Create(raw.Title, raw.Location, score)
}

Which technically works. But it creates a few problems:

No clear stages. Where does normalization end and scoring begin? You have to read everything to understand anything.
Hard to test. How do you test just the scoring logic? You can't. It's glued to everything else inside that loop.
Hard to change. Want a new scoring rule? You're digging through the loop, touching normalization, maybe breaking the save logic. Everything is coupled.
Hard to reuse. If you need that normalization logic somewhere else, you have to copy-paste it. And now you’re duplicating code.
Concurrency feels impossible. Now imagine you want to process jobs concurrently. Where do you even start? The loop? The scoring part? The saving part? Everything is tangled, so it’s hard to determine what to run concurrently.

The Realization

Now here’s the thing you might notice when you look at that loop.

It's not really one problem. It's the same set of steps, repeated for every job. Scrape a job, clean it up, evaluate it, save it. Every single time.

So the question becomes — what if we made those steps explicit?

The Pipeline

That's the core idea. Instead of one big loop doing everything, you break the work into stages:

Scrape → Normalize → Score → Store

Each stage does one thing. Data flows through, gets transformed, moves on.

Here's what that looks like in practice. This is the actual pipeline from my scraper:

type Pipeline struct {
    scorer         scoring.Scorer
    jobService     JobService
    companyService CompanyService
    logger         *slog.Logger
}

func NewPipeline(
    scorer scoring.Scorer,
    jobService JobService,
    companyService CompanyService,
    logger *slog.Logger,
) *Pipeline {
    return &Pipeline{
        scorer:         scorer,
        jobService:     jobService,
        companyService: companyService,
        logger:         logger,
    }
}

Notice what's happening in NewPipeline. You're not hardcoding any specific scraper or store. You're passing them in. We'll come back to why that matters in a second.

Now here's the Run() method. This is where the actual pipeline executes:

func (p *Pipeline) Run(ctx context.Context, scraper Scraper) error {
    // 1. Scrape
    rawJobs, err := scraper.Scrape(ctx)
    if err != nil {
        return fmt.Errorf("scraping %s: %w", scraper.Source(), err)
    }

    for _, rawJob := range rawJobs {
        // 2. Normalize
        normalizedJob, err := normalize.Normalize(rawJob)
        if err != nil {
            failed++
            continue
        }

        // 3. Score
        job.Score = p.scorer.Score(job)

        // 4. Save
        if err := p.jobService.Save(ctx, job); err != nil {
            failed++
            continue
        }
        saved++
    }
    return nil
}

Same logic as the messy version. But now every stage is cleanly separated into it’s own separate function.

You can read this top to bottom as it is now and immediately understand what the system does. Scrape, normalize, score, save. No digging or guessing. The structure tells you the story.

And that's before we even get to the best part.

Swappability

Here's where it gets interesting.

Because each stage is separate and wired through interfaces, you can swap any part of the pipeline without touching the rest. Think about what that means in practice.

Swap the scraper. During testing, I use a FakeScraper that returns hardcoded jobs. In production, that becomes a real scraper hitting actual job sites.

// testing
scraper := &FakeScraper{}

// production
scraper := &RemotiveScraper{}

The pipeline doesn't change. Not a single line.

Swap the store. In development, an in-memory store is fast and easy to work with. In production, that's a real database.

// development
store := NewInMemoryStore()

// production
store := NewPostgresStore()

Same pipeline. Different backend.

Swap the scorer. Right now I'm using a simple keyword-based scorer. Later, that could become something smarter. Maybe ML-based, maybe something else entirely.

scorer := &KeywordScorer{keywords: []string{"Go", "backend", "remote"}}

// later...
scorer := &MLScorer{}

Still the same pipeline.

This really made it click for me: You don’t rewrite the system. You just swap parts. Go back to that messy loop for a second. Try swapping the scraper there. You can't. The logic is baked in, everything depends on everything, there's no clean boundary to swap across.

The pipeline gives you those boundaries.

Concurrency - The Worker Pool

So now the system is clean and flexible. But what about performance?

Right now, everything runs one job at a time. Scrape one, normalize it, score it, save it. Then move to the next. For small datasets that's fine. For a thousand jobs, that's slow.

So let's level it up.

The idea is simple: instead of processing one job at a time, spin up a pool of workers (goroutines) and let them process jobs concurrently. If you haven't used goroutines before, they're basically lightweight threads in Go. You can spin up many of them without much overhead. The key word there is many, not unlimited. More on that in a second.

Here's the shape of it:

func (p *Pipeline) Run(ctx context.Context, scraper Scraper, numWorkers int) error {
    rawJobs, err := scraper.Scrape(ctx)
    if err != nil {
        return fmt.Errorf("scraping %s: %w", scraper.Source(), err)
    }

    jobs := make(chan RawJob)
    var wg sync.WaitGroup

    // spin up workers
    for i := 0; i < numWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for raw := range jobs {
                // normalize, score, save
            }
        }()
    }

    // feed jobs into the channel
    for _, raw := range rawJobs {
        jobs <- raw
    }
    close(jobs)

    wg.Wait()
    return nil
}

Think of it like a queue and a team. The channel is the queue. Raw jobs go in one end, workers pull from the other. Each worker runs a job through the pipeline stages independently. sync.WaitGroup just makes sure we don't return until every worker is done.

The numWorkers parameter is the important part. You decide how many workers run. Not Go, not the runtime. You. That matters because unbounded concurrency has a real cost. A thousand goroutines all trying to write to your database at the same time will hurt more than help. Three to ten workers, controlled, is usually the right call.

The pipeline didn't change conceptually. It just runs in parallel now.

Pipelines Are Everywhere

Now what’s interesting… once you internalize this pattern, you start seeing it everywhere.

Payments: validate the card → charge it → save the transaction. Analytics: collect the event → clean it → store it. APIs: receive the request → process it → send the response.

Different domains, same shape. Data comes in, moves through stages, comes out the other side transformed. That's the pipeline pattern. And the reason it keeps showing up is because it maps well onto how a lot of real work actually flows, i.e., step by step, with clear handoffs between stages.

Which means if you learn to recognize it, you can apply it. Not just in job scrapers, but anywhere you find yourself writing code that takes something, does a series of things to it, and produces a result.

So, Why Bother?

You could write a system without any of this. Plenty of working software is just one big loop doing everything. And honestly, for something small and throwaway, that's fine.

But the moment your system needs to grow, that big loop starts fighting you. You want to add a new data source, but the scraping logic is tangled with the normalization. You want to test scoring in isolation, but it's buried three levels deep. You want to swap your in-memory store for a real database, but there's no clean seam to grab onto.

I’ve run into all this while working on my scraper and other recent builds. And the pipeline pattern has made those problems manageable.

That's really what this is about. Not clean code for the sake of clean code. But about building something you can actually come back to, change, and grow without it fighting you the whole way.

The four ideas worth keeping:

Break into stages. Each stage does one thing.
Keep stages focused. If a stage is hard to name, it's probably doing too much.
Make parts swappable. Wire through interfaces, not concrete types.
Control concurrency. A worker pool, not unlimited goroutines.

Thinking in pipelines, I feel, makes a system easier to reason about. Which matters a lot when you’re in the middle of building something or making updates.

That's the pattern. Hope it's useful.

Where Does This Code Go? Layers in Go

Francis Awuor — Wed, 18 Mar 2026 19:20:41 +0000

Until not too long ago, when I wanted to change one thing in my code, I'd end up having to touch a bunch of other things. Or I'd be staring at a function trying to figure out why it's doing three completely different jobs.

Turns out there's a name for what I was doing wrong, and a pattern that fixes it.

This article is essentially what I wish someone had shown me earlier. We're going to walk through a way of organising your Go code into three layers: Service, Repository, and Delivery…

So that each part of your code has one job, and changing one thing doesn't ripple into everything else.

I'll be using two of my own projects as examples: a netcat-based group chat app, and a URL shortener.

What is Separation of Concerns?

Before we get into the layers, it's worth naming the idea behind all of this.

Separation of concerns basically means: each part of your code should have one clear job, and should know as little as possible about everything else. That's it.

The reason this matters is that when one part of your code knows too much about another part, they get tangled. And tangled code is code that's hard to change. You touch one thing and break another.

The three-layer pattern we're about to walk through is one concrete way to apply this idea in a Go backend. Each layer gets one job. Each layer talks to the next one through a clean boundary. That boundary is what keeps things from bleeding into each other.

The Three Layers

Here's the model we need to internalize before we get into the details:

Service - this is what your app does. Your business rules. The reason the app exists. If someone asked "what does this app actually do?", the answer lives here.

Repository - this is where your app stores and retrieves data. It doesn't care about business rules. It just reads and writes.

Delivery - this is how users interact with your app. An HTTP server, a CLI, a netcat listener. It receives input, calls the service, and returns a response. It's the outermost layer, the plumbing that connects your app to the outside world.

Each layer only talks to the one below it. Delivery calls the service. The service calls the repository. None of them reach past their neighbour, and that constraint is exactly what keeps your code clean.

We'll start with the most important one: the service layer.

The Service Layer

The service layer is the core of your app. It's where your business rules live (the logic that makes your app what it is, not just plumbing or storage.)

A good way to think about it: if someone asked "what does this app actually do?", the service layer is where you'd point them.

Here's what my group chat service handles:

Registering clients when they connect
Creating groups and letting clients join them
Broadcasting messages to everyone in a group
Unregistering clients when they disconnect
Keeping a chat history per group, and sending it to new clients when they join
Notifying everyone when someone joins or leaves

All of that is business logic. Those are decisions the app makes. Not the database, not the HTTP handler.

Here's the public surface of that service:

func (s *Service) Register(client *Client) error
func (s *Service) Unregister(client *Client)
func (s *Service) Broadcast(sender *Client, content string)
func (s *Service) Rename(client *Client, newName string) error
func (s *Service) JoinGroup(client *Client, groupName string) error

Notice what's missing. There's no mention of netcat, no mention of how the client is connected, no mention of how messages are being sent over the wire. The service just does the work. How users are actually reaching the app is someone else's problem, i.e., the delivery layer's problem, which we'll get to.

One thing worth pointing out about this particular service: it uses channels and a Run() loop internally for concurrency safety, which is pretty idiomatic Go for this kind of problem. That's an implementation detail though. The outside world just calls Register, Broadcast, JoinGroup and so on. It has no idea what's happening inside.

That's the point. The service's internals are its own business.

The Repository Layer

The repository layer handles storage. Reading, writing, that's it. The service doesn't care how data is stored. It just calls clean, intention-revealing methods and expects results back.

Here's the Store interface from the URL shortener:

type Store interface {
    Save(link ShortLink) error
    Get(id string) (ShortLink, error)
    IncrementHits(id string) error
}
}

The service calls Save. The service calls Get. It has no idea whether that's hitting a Postgres database or an in-memory map. That's the whole point of defining it as an interface.

And here's what makes that useful in practice. I actually have two implementations of this interface. An in-memory one:

func (store *MemStore) Save(link ShortLink) error { ... }
func (store *MemStore) Get(id string) (ShortLink, error) { ... }
func (store *MemStore) IncrementHits(id string) error { ... }

And a Postgres one:

func (store *PGStore) Save(link ShortLink) error { ... }
func (store *PGStore) Get(id string) (ShortLink, error) { ... }
func (store *PGStore) IncrementHits(id string) error { ... }

Both implement the same interface. Both are completely valid stores as far as the service is concerned. The service just calls Save. It doesn't know and doesn't care which one it's talking to. So you can easily swap one for another, or even swap in a mock version for tests.

We'll see how you actually swap between them in a minute. But first, the delivery layer.

The Delivery Layer

The delivery layer is the outermost layer. It's how users actually reach your app, e.g., via an HTTP server, a CLI, a netcat listener. Its job is simple: receive input, call the service, return a response.

That's it. It should contain no business logic.

A useful test for this: if you replaced your HTTP server with a CLI tomorrow, would you need to rewrite this code? If yes, that logic belongs in the service, not here.

Here's what a handler in my URL shortener looks like:

func (h *Handler) HandleShorten(w http.ResponseWriter, r *http.Request) {
    var req ShortenRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "invalid request", http.StatusBadRequest)
        return
    }

    link, err := h.shortener.Shorten(req.URL)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(link)
}

The handler decodes the request, calls h.shortener.Shorten(), and writes the response. That's all it does. It has no idea how the shortener generates a short code, which store it's using, or any of the logic involved. It just calls the service and handles the result.

Things the delivery layer should not be doing:

Generating short codes
Validating business rules (e.g., checking if a URL already exists)
Knowing what database you're using
Knowing how the service implements anything internally

If you catch yourself writing that kind of logic in a handler, it's a sign it belongs one layer down.

Interfaces: The Glue That Makes It Work

We've mentioned interfaces a couple of times now. Let's actually talk about why they matter here.

In Go, an interface defines what something can do, without caring about what it is. And that distinction is what makes the whole swappability idea practical rather than theoretical.

Here's the Store interface again:

type Store interface {
    Save(link ShortLink) error
    Get(id string) (ShortLink, error)
    IncrementHits(id string) error
}

The service depends on this interface. Not on MemStore. Not on PGStore. It just knows it's talking to something that can Save, Get, and IncrementHits. What that something actually is, is decided elsewhere.

Here's what the shortener (service) constructor looks like:

func NewShortener(store Store, generator IDGenerator) *Shortener {
    return &Shortener{store: store, generator: generator}
}

It takes a Store. Not a *MemStore. Not a *PGStore. A Store. So you can pass in either, and it'll work exactly the same.

This is sometimes called programming to an interface rather than an implementation. The service doesn't care what's behind the interface, and that's precisely why you can swap things out without touching the service at all.

Encapsulation: A Goal, Not a Side Effect

There's another benefit to this pattern that's worth naming explicitly, because it's easy to miss.

When each layer only exposes a clean interface to the layer above it, the internals of each layer are hidden. And that's not just a nice accident. I’d say it’s something we should be actively aiming for.

Here's why it matters practically: if your service hides its internals, you can refactor it freely. Change how you generate short codes, swap out your concurrency strategy, restructure your internal logic, etc. None of that affects the delivery layer, because the delivery layer only ever saw the interface. It never knew what was inside.

Same goes for the repository. You could migrate from Postgres to SQLite, and as long as your new implementation satisfies the Store interface, nothing else in your codebase needs to change.

Your teammates (or future you), can work on one layer without needing to understand or touch the others. I think that's a genuinely good thing to have, even on small projects.

Putting It All Together

Here's where it all clicks. Look at the main.go from the URL shortener:

// store := shorten.NewMemStore()
store := shorten.NewPGStore(sqlDB)
generator := shorten.NewBase62Generator()
shortener := shorten.NewShortener(store, generator)

mux := http.NewServeMux()
shorten.RegisterRoutes(mux, shortener)

server := http.Server{
    Addr:    ":8080",
    Handler: mux,
}

This is the wiring. main.go is where all three layers meet (and it's really the only place they meet.)

Notice the commented out line at the top. Switching from in-memory storage to Postgres is literally a one line change. That's the payoff of having defined a Store interface and having both implementations satisfy it.

The flow is:

main creates a store and passes it to the service
main creates the service and passes it to the delivery layer
The delivery layer handles requests and calls the service
The service does the work and calls the repository
Nobody reaches past their neighbour

Each layer was built independently. Each layer can be changed independently. And main is just the place that plugs them all together.

Conclusion

So to recap. Three layers, three jobs:

Service: what your app does
Repository: where your app stores data
Delivery: how users reach your app

Each layer talks to the one below it through a clean boundary. None of them know more than they need to. That's separation of concerns in practice.

The payoff isn't always obvious on a small project. But even on small projects, I've found it makes the code easier to reason about, easier to change, and easier for someone else to pick up. You always know where something belongs. And when you need to change something, you know exactly where to go.

It's one of those things that feels like a bit of extra work upfront, but saves you a lot of pain later.

A Beginner’s Guide to Testing in Go

Francis Awuor — Wed, 11 Feb 2026 19:33:37 +0000

I’ve been writing tests in Go for a few weeks now. And I’ve had more than a few moments of uncomfortable confusion.

So here’s a few things I wish someone had told me upfront.

Examples are in Go, but the ideas aren’t really language dependent.

Tests Enforce Contracts

Now right off the bat, you don’t want to get this wrong.

When writing tests, I’ve found the key thing to keep in mind is we should treat what we’re testing like a black box.

Give it specific inputs, expect specific outputs. For example, let’s take a look at function I had in a recent project (simplified). The project was a simple URL shortener I built.

func Create(url string) (shortCode string, err error)

The idea is, the function should take a URL string, e.g., https://exampleurl.com/this/is/really/long. That’s the input.

And it should return a short code that represents the long URL, e.g., bdcagl. Or an error if things go wrong. Those are the outputs.

That right there, that’s the contract. We’re saying if you give me a URL (a valid one), I should give you back a short code. Unless something goes wrong. In which case, I should give you an error.

This is what we test. So our test might look like this:

t.Run("Valid URL", func(t *testing.T) {
    url := "https://example.com"

    shortCode, err := Create(url)
    if err != nil {
        t.Fatalf("expected nil err, got %v", err) // if there's an unexpected error, fail the test
    }

    // Check that shortCode was generated
    if shortCode == "" {
        t.Error(`expected short code, got ""`) // after Creation, short code should not be empty
    }
})

Now the test above is what we call the happy path. It’s what we expect to happen when everything goes right. But what happens if something goes wrong?

Errors Are Part of the Contract Too

Of the outputs we expect, specific error cases are included.

What I mean is, if, for example, I pass in bad input, I should get back one error. But if something fails internally, I should get back a different error.

This is part of the contract. And it took me quite a while to realize.

Cause see what I’d often do is, in cases where I expected an error, I’d simply check for the presence of errors, e.g.,

url := "https:invalid-url//example./com" // invalid input should fail and return an error

shortCode, err := Create(url)
if err == nil {
        t.Fatalf("expected error, got nil") // if there's no error where you expect one, fail the test
    }

This is not enough!

I’ve since found that a better way to handle this, would be to check for specific errors, e.g,

if !errors.Is(err, ErrInvalidURL) { // Check for invalid url error
        t.Fatalf("expected ErrInvalidURL, got %v", err)
}

The contract should specify what error you get back.

NOTE: Testing error cases is just as important as testing happy paths. DO NOT sleep on it.

And we’re not done yet. Cause important as it is to have this mental model of contracts, it’s even more essential to understand how to structure our tests.

But in order to understand that, I have to first clear up what a unit test is. Cause it is not what I thought it was.

What Even Is a Unit?

For this section, it’s useful to quickly outline a commonly recommended structure for backend systems:

(We organize our HTTP server into layers)

HTTP Request
     |
   v
Handlers → receive HTTP requests and send responses
   |
   v
Services → apply business rules and coordinate work
   |
   v
Repository → reads and writes data in the database
   |
   v
Database

Now back to unit tests.

You might have heard of unit tests before. And if not, you’ll be glad you read this.

Cause see for a long time, when I heard “unit test”, I thought that referred to testing a function.

So if I had three functions, I thought that was three separate units to test.

I was wrong.

In our code, a unit is actually a “cohesive chunk of behavior, with one responsibility, that can be reasoned about independently.”

Which sounds fancy, but it really just means, if I have a service like this:

type Shortener struct {
    store Store // where you store the generated short code and its associated long URL
    gen   Generator // consider this a function that generates the short code
}

func (s *Shortener) Create(url string) (string, error) {
    // generates short code and saves it
}

func (s *Shortener) Resolve(shortCode string) (string, error) {
    // looks up original URL
}

func (s *Shortener) Stats(shortCode string) (int, error) {
    // returns hit count for a short code
}

Then that entire struct — the Shortener plus all its public methods (Create, Resolve , Stats) is ONE unit.

Not three units. One.

So when I write tests for Create, Resolve, and Stats, I’m just testing different behaviors of the same unit.

A way to say it that helped me understand better is:

A unit is the smallest part of your code where you can say: “If this breaks, I know exactly what kind of problem it is. Failures don’t blur together.”

This matters because once I understood this, test organization made way more sense.

So let’s get to it shall we? How to structure our tests.

Good Test Structure

See after writing a bunch of messy tests, I found a structure that works:

Setup → Subject → Assertion

Setup

Here you get everything ready for the test.

Say you want to test the Create function above. Well you need a generator, and a store. The Shortener’s Create function uses the generator to generate short codes, and the store to store the generated codes alongside the original URLs. This is the setup stage. You have to prepare what the element being tested needs, e.g.,:

// Setup
store := NewMockStore()
gen := NewMockGenerator()
service := NewShortener(store, gen)

Subject

The thing you’re actually testing. Call the method, trigger the behavior, e.g.,

// Subject
shortCode, err := service.Create("https://example.com")

Assertion

Verify the contract holds. Check outputs, check errors, check side effects. (Side effects are tricky though. We’ll talk about them in a bit)

Example:

// Assertion
if err != nil {
    t.Fatalf("Expected no error, got %v", err)
}

if shortCode != "abc123" {
    t.Errorf("Expected abc123, got %s", shortCode)
}

And one critical thing to keep in mind (trust me, I made this mistake quite a few times): If setup fails, fail the test immediately. E.g.,:

func TestCreate(t *testing.T) {
    store := NewMockStore()
    if store == nil {
        t.Fatal("Failed to create mock store") // t.Fatal() stops the test here
    }

    service := NewShortener(store, mockGenerator)
    if service == nil {
        t.Fatal("Failed to create service") // stops test here
    }

    // Now do the actual test...
}

There’s a simple reason why, i.e., if setup fails and you keep going, you’ll likely get misleading errors. The test will fail at unexpected points, and it’ll be a nightmare to debug.

And here’s where the bit about what a unit is gets particularly interesting.

See while working on my shortener, I had cases like this:

func TestResolve(t *testing.T) {
    service := NewShortener(mockStore, mockGen)

    // SEE THIS? We need to use Create for setup, cause to resolve (i.e., get the URL a short code points to), we need to have created the short code
    shortCode, err := service.Create("https://example.com")
    if err != nil {
        t.Fatalf("Setup failed: %v", err) // fail fast if setup fails
    }

    // Now test Resolve
    url, err := service.Resolve(shortCode)
    if err != nil {
        t.Errorf("Resolve failed: %v", err)
    }

    if url != "https://example.com" {
        t.Errorf("Expected https://example.com, got %s", url)
    }
}

It’s a case where we have to use the Create function to setup tests for the Resolve function. Because to resolve (i.e., get the URL a short code points to), we need to have created the short code.

Which seems reasonable enough. Until we, for example, go a level up and have a handler that needs another handler for setup. Here’s a snippet:

func TestHandleRedirect_OK(t *testing.T) {
// Setup
    // We would call HandleShorten() here, which creates a short code given the URL to shorten
    handler.HandleShorten(rr, req)

// Subject (the thing we're testing)
// Create a mock request specifically for testing
        req := httptest.NewRequest(http.MethodGet, "/"+shortCode, nil) // NOTICE the short code?
    rr := httptest.NewRecorder()

// Given a short code, HandleRedirect redirects to the URL the short code represents. It internally calls Resolve() to get this done.
    handler.HandleRedirect(rr, req)

    res := rr.Result()
    res.Body.Close()

    if res.StatusCode != http.StatusFound {
        t.Fatalf("expected status 302, got %d", res.StatusCode)
    }
}

Like the test for Resolve() earlier, we need the short code in order to test HandleRedirect. But in order to get the short code, we’d either have to call another handler, HandleShorten , to generate the short code for us…

Or we’d have to reach into the service and call service.Create().

So which approach is better?

Using Other Functions for Test Setup

The key principle I found to deal with these kinds of test setup issues is: A test should not depend on the behavior of another unit.

The key issue here, is unit boundaries. (See? Units came back)

See calling HandleShorten to set up HandleRedirect might feel symmetrical, but it’s actually the wrong move.

This is cause handlers are separate units.

If the HandleRedirect test failed, you wouldn’t be able to easily tell what broke. Was it HandleRedirect or HandleShorten? It messes with failure attribution.

HandleRedirect should not have to depend on HandleShorten to work.

The better approach, would be to reach down, not sideways.

func TestHandleRedirect_OK(t *testing.T) {
    service := NewShortener(mockStore, mockGen)
    handler := NewHandler(service)

    // Setup: reach down to the service
    shortCode, err := service.Create("https://example.com") // We have Create from the service setting up HandleRedirect
    if err != nil {
        t.Fatalf("Setup failed: %v", err)
    }

    // Subject
    req := httptest.NewRequest(http.MethodGet, "/"+shortCode, nil)
    rr := httptest.NewRecorder()

    handler.HandleRedirect(rr, req)

    res := rr.Result()
    res.Body.Close()

    if res.StatusCode != http.StatusFound {
        t.Fatalf("expected status 302, got %d", res.StatusCode)
    }
}

This is better because:

You’re testing one handler
Setup uses lower-level implementation
No handler depends on another handler’s behavior
If the test fails, we know where to look

And if service.Create breaks, the handler test should fail. Because in the real app, the handler relies on the service. It’s honest dependency.

However, it’s pretty important to note that the earlier example of using Create to set up Resolve, is different.

That’s perfectly fine, because

Create and Resolve belong to the same unit.
If Create is broke, Resolve is broken too.

A single service is a single unit. Handlers don’t have that relationship.

A “rule of thumb” of sorts that helps me with this is (and this ties back to the backend layers idea):

A test can depend on lower layers, but it shouldn’t depend on same layer entities unless it’s from the same unit.

However do note that an exception to this is the Service layer. For the service layer, you don’t want to call lower layer functions (from repository/db layer) directly for setup in your service tests, cause you’d be bypassing business rules, which is the core of your application. It’s at a base level, what your application should do.

And now one last thing I mustn’t forget to mention: Side Effects.

Side Effects Are Tricky

I say they’re tricky cause, a side effect is something your function changes internally, that’s not part of its return value.

My service.Resolve function, for example, internally increments a hits counter for a URL every time someone requests that URL.

“hits”, however, is not part of its return values, so you have to observe or test hits separately, for example by creating a separate function for metrics (e.g., my service.Stats function), through database queries, etc.

Key Takeaways

A unit is a cohesive module with a single responsibility, not a single function. (Think, “if something fails, can I tell what kind of problem it is?” - Clear failure attribution)
Tests enforce contracts - Specific inputs → Specific outputs
A reasonable test structure: Setup → Subject → Assertion
Fail fast on setup failures (t.Fatal() ) to avoid misleading errors
Side effects need indirect observation