Forem: Phuong Le

Go Singleflight Melts in Your Code, Not in Your DB

Phuong Le — Tue, 05 Nov 2024 01:00:00 +0000

The original article is posted on the VictoriaMetrics blog: https://victoriametrics.com/blog/go-singleflight/

This post is part of a series about handling concurrency in Go:

Go sync.Mutex: Normal and Starvation Mode
Go sync.WaitGroup and The Alignment Problem
Go sync.Pool and the Mechanics Behind It
Go sync.Cond, the Most Overlooked Sync Mechanism
Go sync.Map: The Right Tool for the Right Job
Go Sync.Once is Simple... Does It Really?
Go Singleflight Melts in Your Code, Not in Your DB (We're here)

Go Singleflight Melts in Your Code, Not in Your DB

So, when you've got multiple requests coming in at the same time asking for the same data, the default behavior is that each of those requests would go to the database individually to get the same information. What that means is that you'd end up executing the same query several times, which, let's be honest, is just inefficient.

Multiple identical requests hitting the database

It ends up putting unnecessary load on your database, which could slow everything down, but there's a way around this.

The idea is that only the first request actually goes to the database. The rest of the requests wait for that first one to finish. Once the data comes back from the initial request, the other ones just get the same result—no extra queries needed.

How singleflight suppresses duplicate requests

So, now you've got a pretty good idea of what this post is about, right?

Singleflight

The singleflight package in Go is built specifically to handle exactly what we just talked about. And just a heads-up, it's not part of the standard library but it's maintained and developed by the Go team.

What singleflight does is ensure that only one of those goroutines actually runs the operation, like getting the data from the database. It allows only one "in-flight" (ongoing) operation for the same piece of data (known as a "key") at any given moment.

So, if other goroutines ask for the same data (same key) while that operation is still going, they'll just wait. Then, when the first one finishes, all the others get the same result without having to run the operation again.

Alright, enough talk, let's dive into a quick demo to see how singleflight works in action:

var callCount atomic.Int32
var wg sync.WaitGroup

// Simulate a function that fetches data from a database
func fetchData() (interface{}, error) {
    callCount.Add(1)
    time.Sleep(100 * time.Millisecond)
    return rand.Intn(100), nil
}

// Wrap the fetchData function with singleflight
func fetchDataWrapper(g *singleflight.Group, id int) error {
    defer wg.Done()

    time.Sleep(time.Duration(id) * 40 * time.Millisecond)
    v, err, shared := g.Do("key-fetch-data", fetchData)
    if err != nil {
        return err
    }

    fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, v, shared)
    return nil
}

func main() {
    var g singleflight.Group

    // 5 goroutines to fetch the same data
    const numGoroutines = 5
    wg.Add(numGoroutines)

    for i := 0; i < numGoroutines; i++ {
        go fetchDataWrapper(&g, i)
    }

    wg.Wait()
    fmt.Printf("Function was called %d times\n", callCount.Load())
}

// Output:
// Goroutine 0: result: 90, shared: true
// Goroutine 2: result: 90, shared: true
// Goroutine 1: result: 90, shared: true
// Goroutine 3: result: 13, shared: true
// Goroutine 4: result: 13, shared: true
// Function was called 2 times

What's going on here:

We're simulating a situation where 5 goroutines try to fetch the same data almost at the same time, spaced 60ms apart. To keep it simple, we're using random numbers to mimic data fetched from a database.

With singleflight.Group, we make sure only the first goroutine actually runs fetchData() and the rest of them wait for the result.

The line v, err, shared := g.Do("key-fetch-data", fetchData) assigns a unique key ("key-fetch-data") to track these requests. So, if another goroutine asks for the same key while the first one is still fetching the data, it waits for the result rather than starting a new call.

Demonstration of singleflight in action

Once the first call finishes, any waiting goroutines get the same result, as we can see in the output. Although we had 5 goroutines asking for the data, fetchData only ran twice, which is a massive boost.

The shared flag confirms that the result was reused across multiple goroutines.

"But why is the shared flag true for the first goroutine? I thought only the waiting ones would have shared == true?"

Yeah, this might feel a bit counterintuitive if you're thinking only the waiting goroutines should have shared == true.

The thing is, the shared variable in g.Do tells you whether the result was shared among multiple callers. It's basically saying, "Hey, this result was used by more than one caller." It's not about who ran the function, it's just a signal that the result was reused across multiple goroutines.

"I have a cache, why do I need singleflight?"

The short answer is: caches and singleflight solve different problems, and they actually work really well together.

In a setup with an external cache (like Redis or Memcached), singleflight adds an extra layer of protection, not just for your database but also for the cache itself.

Singleflight working alongside a cache system

In addition, singleflight helps protect against a cache miss storm (sometimes called a "cache stampede").

Normally, when a request asks for data, if the data is in the cache, great - it's a cache hit. If the data isn't in the cache, it's a cache miss. Suppose 10,000 requests hit the system all at once before the cache is rebuilt, the database could suddenly get slammed with 10,000 identical queries at the same time.

During this peak, singleflight ensures that only one of those 10,000 requests actually hits the database.

But later on, in the internal implementation section, we'll see that singleflight uses a global lock to protect the map of in-flight calls, which can become a single point of contention for every goroutine. This can slow things down, especially if you're dealing with high concurrency.

The model below might work better for machines with multiple CPUs:

Singleflight on cache miss

In this setup, we only use singleflight when a cache miss happens.

Singleflight Operations

To use singleflight, you first create a Group object, which is the core structure that tracks ongoing function calls linked to specific keys.

It has two key methods that help prevent duplicate calls:

group.Do(key, func): Runs your function while suppressing duplicate requests. When you call Do, you pass in a key and a function, if no other execution is happening for that key, the function runs. If there's already an execution in progress for the same key, your call blocks until the first one finishes and returns the same result.
group.DoChan(key, func): Similar to group.Do, but instead of blocking, it gives you a channel (<-chan Result). You'll receive the result once it's ready, making this useful if you prefer handling the result asynchronously or if you're selecting over multiple channels.

We've already seen how to use g.Do() in the demo, let's check out how to use g.DoChan() with a modified wrapper function:

// Wrap the fetchData function with singleflight using DoChan
func fetchDataWrapper(g *singleflight.Group, id int) error {
    defer wg.Done()

    ch := g.DoChan("key-fetch-data", fetchData)

    res := <-ch
    if res.Err != nil {
        return res.Err
    }

    fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, res.Val, res.Shared)
    return nil
}

package singleflight

type Result struct {
    Val    interface{}
    Err    error
    Shared bool
}

To be honest, using DoChan() here doesn't change much compared to Do(), since we're still waiting for the result with a channel receive operation (<-ch), which is basically blocking the same way.

Where DoChan() does shine is when you want to kick off an operation and do other stuff without blocking the goroutine. For instance, you could handle timeouts or cancellations more cleanly using channels:

func fetchDataWrapperWithTimeout(g *singleflight.Group, id int) error {
    defer wg.Done()

    ch := g.DoChan("key-fetch-data", fetchData)
    select {
    case res := <-ch:
        if res.Err != nil {
            return res.Err
        }
        fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, res.Val, res.Shared)
    case <-time.After(50 * time.Millisecond):
        return fmt.Errorf("timeout waiting for result")
    }

  return nil
}

This example also brings up a few issues that you might run into in real-world scenarios:

The first goroutine might take way longer than expected due to things like slow network responses, unresponsive databases, etc. In that case, all the other waiting goroutines are stuck for longer than you'd like. A timeout can help here, but any new requests will still end up waiting behind the first one.
The data you're fetching might change frequently, so by the time the first request finishes, the result could be outdated. That means we need a way to invalidate the key and trigger a new execution.

Yes, singleflight provides a way to handle situations like these with the group.Forget(key) method, which lets you discard an ongoing execution.

The Forget() method removes a key from the internal map that tracks the ongoing function calls. It's sort of like "invalidating" the key, so if you call g.Do() again with that key, it'll execute the function as if it were a fresh request, instead of waiting on the previous execution to finish.

Let's update our example to use Forget() and see how many times the function actually gets called:

func fetchDataWrapperWithForget(g *singleflight.Group, id int, forget bool) error {
    defer wg.Done()

    // Forget the key before fetching
    if forget {
        g.Forget("key-fetch-data")
    }

    v, err, shared := g.Do("key-fetch-data", fetchData)
    if err != nil {
        return err
    }

    fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, v, shared)
    return nil
}

func main() {
    var g singleflight.Group
  wg.Add(3)

    // 2 goroutines fetch the data
    go fetchDataWrapperWithForget(&g, 0, false)
    go fetchDataWrapperWithForget(&g, 1, false)

    // Wait a bit and launch 1 more goroutine
    // Ensures goroutines 0, 1, and 2 overlap
    time.Sleep(10 * time.Millisecond)
    go fetchDataWrapperWithForget(&g, 2, true)

    wg.Wait()
    fmt.Printf("Function was called %d times\n", callCount.Load())
}

// Output:
// Goroutine 0: result: 55, shared: true
// Goroutine 1: result: 55, shared: true
// Goroutine 2: result: 73, shared: false
// Function was called 2 times

Goroutine 0 and Goroutine 1 both call Do() with the same key ("key-fetch-data"), and their requests get combined into one execution and the result is shared between the two goroutines.

Goroutine 2, on the other hand, calls Forget() before running Do(). This clears out any previous result tied to "key-fetch-data", so it triggers a new execution of the function.

To sum up, while singleflight is useful, it can still have some edge cases, for example:

If the first goroutine gets blocked for too long, all the others waiting on it will also be stuck. In such cases, using a timeout context or a select statement with a timeout can be a better option.
If the first request returns an error or panics, that same error or panic will propagate to all the other goroutines waiting for the result.

If you have noticed all the issues we've discussed, let's dive into the next section to discuss how singleflight actually works under the hood.

How Singleflight Works

From using singleflight, you might already have a basic idea of how it works internally, the whole implementation of singleflight is only about 150 lines of code.

Basically, every unique key gets a struct that manages its execution. If a goroutine calls Do() and finds that the key already exists, that call will be blocked until the first execution finishes, and here is the structure:

type Group struct {
    mu sync.Mutex       // protects the map m
    m  map[string]*call // maps keys to calls; lazily initialized
}

type call struct {
    wg    sync.WaitGroup   // waits for the function execution
    val   interface{}      // result of the function call
    err   error            // error from the function call
    dups  int              // number of duplicate callers
    chans []chan<- Result  // channels to receive the result
}

Two sync primitives are used here:

Group mutex (g.mu): This mutex protects the entire map of keys, not one lock per key, it makes sure adding or removing keys is thread-safe.
WaitGroup (g.call.wg): The WaitGroup is used to wait for the first goroutine associated with a specific key to finish its work.

We'll focus on the group.Do() method here since the other method, group.DoChan(), works in a similar way. The group.Forget() method is also simple as it just removes the key from the map.

When you call group.Do(), the first thing it does is lock the entire map of calls (g.mu).

"Isn't that bad for performance?"

Yeah, it might not be ideal for performance in every case (always good to benchmark first) as singleflight locks the entire keys. If you're aiming for better performance or working at a high scale, a good approach is to shard or distribute the keys. Instead of using just one singleflight group, you can spread the load across multiple groups, kind of like doing "multiflight" instead

For reference, check out this repo: shardedsingleflight.

Now, once it has the lock, the group looks that the internal map (g.m), if there's already an ongoing or completed call for the given key. This map keeps track of any in-progress or completed work, with keys mapping to their corresponding tasks.

If the key is found (another goroutine is already running the task), instead of starting a new call, we simply increment a counter (c.dups) to track duplicate requests. The goroutine then releases the lock and waits for the original task to complete by calling call.wg.Wait() on the associated WaitGroup.

When the original task is done, this goroutine grabs the result and avoids running the task again.

func (g *Group) Do(key string, fn func() (interface{}, error)) (v interface{}, err error, shared bool) {
    g.mu.Lock()
    if g.m == nil {
        g.m = make(map[string]*call)
    }

    // If the key is already exist, 
    // increase the duplicate counter and wait for the result
    if c, ok := g.m[key]; ok {
        c.dups++
        g.mu.Unlock()
        c.wg.Wait()

        if e, ok := c.err.(*panicError); ok {
            panic(e)
        } else if c.err == errGoexit {
            runtime.Goexit()
        }
        return c.val, c.err, true
    }

    // Otherwise, create a new call and add it to the map
    c := new(call)
    c.wg.Add(1)
    g.m[key] = c
    g.mu.Unlock()

    // Execute the function
    g.doCall(c, key, fn)
    return c.val, c.err, c.dups > 0
}

If no other goroutine is working on that key, the current goroutine takes responsibility for executing the task.

At this point, we create a new call object, add it to the map, and initialize its WaitGroup. Then, we unlock the mutex and proceed to execute the task ourselves via a helper method g.doCall(c, key, fn). When the task completes, any waiting goroutines are unblocked by the wg.Wait() call.

Nothing too wild here, except for how we handle errors, there are three possible scenarios:

If the function panics, we catch it, wrap it in a panicError, and throw the panic.
If the function returns an errGoexit, we call runtime.Goexit() to properly exit the goroutine.
If it's just a normal error, we set that error on the call.

This is where things start to get a little more clever in the helper method g.doCall().

"Wait, what's runtime.Goexit()?"

Before we dive into the code, let me quickly explain, runtime.Goexit() is used to stop the execution of a goroutine.

When a goroutine calls Goexit(), it stops, and any deferred functions are still run in Last-In-First-Out (LIFO) order, just like normal. It's similar to a panic, but there are a couple of differences:

It doesn't trigger a panic, so you can't catch it with recover().
Only the goroutine that calls Goexit() gets terminated and all the other goroutines keep running just fine.

Now, here's an interesting quirk (not directly related to our topic, but worth mentioning). If you call runtime.Goexit() in the main goroutine (like inside main()), check this out:

func main() {
    go func() {
        println("goroutine called")
    }()

    runtime.Goexit()
    println("main goroutine called")
}

// Output:
// goroutine called
// fatal error: no goroutines (main called runtime.Goexit) - deadlock!

What happens is that Goexit() terminates the main goroutine, but if there are other goroutines still running, the program keeps going because the Go runtime stays alive as long as at least one goroutine is active. However, once no goroutines are left, it crashes with a "no goroutines" error, kind of a fun little corner case.

Now, back to our code, if runtime.Goexit() only terminates the current goroutine and can't be caught by recover(), how do we detect if it's been called?

The key lies in the fact that when runtime.Goexit() is invoked, any code after it doesn't get executed.

func main() {
    normalReturn := false

    defer func() {
        if !normalReturn {
            println("runtime.Goexit() called")
            return
        }

        println("normal return")
    }()

    runtime.Goexit()
    normalReturn = true
}

// Output:
// runtime.Goexit() called
// fatal error: no goroutines (main called runtime.Goexit) - deadlock!

In the above case, the line normalReturn = true never gets executed after calling runtime.Goexit(). So, inside the defer, we can check whether normalReturn is still false to detect that special method was called.

The next step is figuring out if the task is panicking or not. For that, we use recover() as normal return, though the actual code in singleflight is a little more subtle:

// doCall handles the single call for a key.
func (g *Group) doCall(c *call, key string, fn func() (interface{}, error)) {
    normalReturn := false
    recovered := false

    defer func() {
        // The case of calling runtime.Goexit() in the task
        if !normalReturn && !recovered {
            c.err = errGoexit
        }
        ... // handle each cases
    }()

    func() {
        defer func() {
            if !normalReturn {
                if r := recover(); r != nil {
                    c.err = newPanicError(r)
                }
            }
        }()

        c.val, c.err = fn()
        normalReturn = true
    }()

    if !normalReturn {
        recovered = true
    }
}

Instead of setting recovered = true directly inside the recover block, this code gets a little fancy by setting recovered after the recover() block as the last line.

So, why does this work?

When runtime.Goexit() is called, it terminates the entire goroutine, just like a panic(). However, if a panic() is recovered, only the chain of functions between the panic() and the recover() is terminated, not the entire goroutine.

Handling of panic and runtime.Goexit() in singleflight

That's why recovered = true gets set outside the defer containing recover(), it only gets executed in two cases: when the function completes normally or when a panic is recovered, but not when runtime.Goexit() is called.

Moving forward, we'll discuss how each case is handled.

func (g *Group) doCall(c *call, key string, fn func() (interface{}, error)) {
    ...

    defer func() {
        ...

        // Lock and remove the call from the map
        g.mu.Lock()
        defer g.mu.Unlock()
        c.wg.Done()
        if g.m[key] == c {
            delete(g.m, key)
        }

        if e, ok := c.err.(*panicError); ok {
            if len(c.chans) > 0 {
                go panic(e)
                select {} // Keep this goroutine around so that it will appear in the crash dump.
            } else {
                panic(e)
            }
        } else if c.err == errGoexit {
            // Already in the process of goexit, no need to call again
        } else {
            // Normal return
            for _, ch := range c.chans {
                ch <- Result{c.val, c.err, c.dups > 0}
            }
        }
    }()
    ...
}

If the task panics during execution, the panic is caught and saved in c.err as a panicError, which holds both the panic value and the stack trace. singleflight catches the panic to clean up gracefully, but it doesn't swallow it, it rethrows the panic after handling its state.

That means the panic will happen in the goroutine that's executing the task (the first one to kick off the operation), and all the other goroutines waiting for the result will also panic.

Since this panic happens in the developer's code, it's on us to deal with it properly.

Now, there's still a special case we need to consider: when other goroutines are using the group.DoChan() method and waiting on a result via a channel. In this case, singleflight can't panic in those goroutines. Instead, it does what's called an unrecoverable panic (go panic(e)), which makes our application crash.

Finally, if the task called runtime.Goexit(), there's no need to take any further action because the goroutine is already in the process of shutting down, and we just let that happen without interfering.

And that's pretty much it, nothing too complicated except for the special cases we've discussed.

Stay Connected

Hi, I'm Phuong Le, a software engineer at VictoriaMetrics. The writing style above focuses on clarity and simplicity, explaining concepts in a way that's easy to understand, even if it's not always perfectly aligned with academic precision.

If you spot anything that's outdated or if you have questions, don't hesitate to reach out. You can drop me a DM on X(@func25).

Some other posts you might be interested in:

Who We Are

If you want to monitor your services, track metrics, and see how everything performs, you might want to check out VictoriaMetrics. It's a fast, open-source, and cost-saving way to keep an eye on your infrastructure.

And we're Gophers, enthusiasts who love researching, experimenting, and sharing knowledge about Go and its ecosystem.

Go sync.Cond, the Most Overlooked Sync Mechanism

Phuong Le — Tue, 29 Oct 2024 01:00:00 +0000

This is an excerpt of the post; the full post is available here: https://victoriametrics.com/blog/go-sync-cond/

This post is part of a series about handling concurrency in Go:

Go sync.Mutex: Normal and Starvation Mode
Go sync.WaitGroup and The Alignment Problem
Go sync.Pool and the Mechanics Behind It
Go sync.Cond, the Most Overlooked Sync Mechanism (We're here)
Go sync.Map: The Right Tool for the Right Job
Go Singleflight Melts in Your Code, Not in Your DB

In Go, sync.Cond is a synchronization primitive, though it's not as commonly used as its siblings like sync.Mutex or sync.WaitGroup. You'll rarely see it in most projects or even in the standard libraries, where other sync mechanisms tend to take its place.

That said, as a Go engineer, you don't really want to find yourself reading through code that uses sync.Cond and not have a clue what's going on, because it is part of the standard library, after all.

So, this discussion will help you close that gap, and even better, it'll give you a clearer sense of how it actually works in practice.

What is sync.Cond?

So, let's break down what sync.Cond is all about.

When a goroutine needs to wait for something specific to happen, like some shared data changing, it can "block," meaning it just pauses its work until it gets the go-ahead to continue. The most basic way to do this is with a loop, maybe even adding a time.Sleep to prevent the CPU from going crazy with busy-waiting.

Here's what that might look like:

// wait until condition is true
for !condition {  
}

// or 
for !condition {
    time.Sleep(100 * time.Millisecond)
}

Now, this isn't really efficient as that loop is still running in the background, burning through CPU cycles, even when nothing's changed.

That's where sync.Cond steps in, a better way to let goroutines coordinate their work. Technically, it's a "condition variable" if you're coming from a more academic background.

When one goroutine is waiting for something to happen (waiting for a certain condition to become true), it can call Wait().
Another goroutine, once it knows that the condition might be met, can call Signal() or Broadcast() to wake up the waiting goroutine(s) and let them know it's time to move on.

Here's the basic interface sync.Cond provides:

// Suspends the calling goroutine until the condition is met
func (c *Cond) Wait() {}

// Wakes up one waiting goroutine, if there is one
func (c *Cond) Signal() {}

// Wakes up all waiting goroutines
func (c *Cond) Broadcast() {}

Overview of sync.Cond

Alright, let's check out a quick pseudo-example. This time, we've got a Pokémon theme going on, imagine we're waiting for a specific Pokémon, and we want to notify other goroutines when it shows up.

var pokemonList = []string{"Pikachu", "Charmander", "Squirtle", "Bulbasaur", "Jigglypuff"}
var cond = sync.NewCond(&sync.Mutex{})
var pokemon = ""

func main() {
    // Consumer
    go func() {
        cond.L.Lock()
        defer cond.L.Unlock()

        // waits until Pikachu appears
        for pokemon != "Pikachu" {
            cond.Wait()
        }
        println("Caught" + pokemon)
        pokemon = ""
    }()

    // Producer
    go func() {
        // Every 1ms, a random Pokémon appears
        for i := 0; i < 100; i++ {
            time.Sleep(time.Millisecond)

            cond.L.Lock()
            pokemon = pokemonList[rand.Intn(len(pokemonList))]
            cond.L.Unlock()

            cond.Signal()
        }
    }()

    time.Sleep(100 * time.Millisecond) // lazy wait
}

// Output:
// Caught Pikachu

In this example, one goroutine is waiting for Pikachu to show up, while another one (the producer) randomly selects a Pokémon from the list and signals the consumer when a new one appears.

When the producer sends the signal, the consumer wakes up and checks if the right Pokémon has appeared. If it has, we catch the Pokémon, if not, the consumer goes back to sleep and waits for the next one.

The problem is, there's a gap between the producer sending the signal and the consumer actually waking up. In the meantime, the Pokémon could change, because the consumer goroutine might wake up later than 1ms (rarely) or other goroutine modifies the shared pokemon. So sync.Cond is basically saying: 'Hey, something changed! Wake up and check it out, but if you're too late, it might change again.'

If the consumer wakes up late, the Pokémon might run away, and the goroutine will go back to sleep.

"Huh, I could use a channel to send the pokemon name or signal to the other goroutine"

Absolutely. In fact, channels are generally preferred over sync.Cond in Go because they're simpler, more idiomatic, and familiar to most developers.

In the case above, you could easily send the Pokémon name through a channel, or just use an empty struct{} to signal without sending any data. But our issue isn't just about passing messages through channels, it's about dealing with a shared state.

Our example is pretty simple, but if multiple goroutines are accessing the shared pokemon variable, let's look at what happens if we use a channel:

If we use a channel to send the Pokémon name, we'd still need a mutex to protect the shared pokemon variable.
If we use a channel just to signal, a mutex is still necessary to manage access to the shared state.
If we check for Pikachu in the producer and then send it through the channel, we'd also need a mutex. On top of that, we'd violate the separation of concerns principle, where the producer is taking on the logic that really belongs to the consumer.

That said, when multiple goroutines are modifying shared data, a mutex is still necessary to protect it. You'll often see a combination of channels and mutexes in these cases to ensure proper synchronization and data safety.

"Okay, but what about broadcasting signals?"

Good question! You can indeed mimic a broadcast signal to all waiting goroutines using a channel by simply closing it (close(ch)). When you close a channel, all goroutines receiving from that channel get notified. But keep in mind, a closed channel can't be reused, once it's closed, it stays closed.

By the way, there's actually been talk about removing sync.Cond in Go 2: proposal: sync: remove the Cond type.

"So, what's sync.Cond good for, then?"

Well, there are certain scenarios where sync.Cond can be more appropriate than channels.

With a channel, you can either send a signal to one goroutine by sending a value or notify all goroutines by closing the channel, but you can't do both. sync.Cond gives you more fine-grained control. You can call Signal() to wake up a single goroutine or Broadcast() to wake up all of them.
And you can call Broadcast() as many times as you need, which channels can't do once they're closed (closing a closed channel will trigger a panic).
Channels don't provide a built-in way to protect shared data—you'd need to manage that separately with a mutex. sync.Cond, on the other hand, gives you a more integrated approach by combining locking and signaling in one package (and better performance).

"Why is the Lock embedded in sync.Cond?"

In theory, a condition variable like sync.Cond doesn't have to be tied to a lock for its signaling to work.

You could have the users manage their own locks outside of the condition variable, which might sound like it gives more flexibility. It's not really a technical limitation but more about human error.

Managing it manually can easily lead to mistakes because the pattern isn't really intuitive, you have to unlock the mutex before calling Wait(), then lock it again when the goroutine wakes up. This process can feel awkward and is pretty prone to errors, like forgetting to lock or unlock at the right time.

But why does the pattern seem a little off?

Typically, goroutines that call cond.Wait() need to check some shared state in a loop, like this:

for !checkSomeSharedState() {
    cond.Wait()
}

The lock embedded in sync.Cond helps handle the lock/unlock process for us, making the code cleaner and less error-prone, we will discuss the pattern in detail soon.

How to use it?

If you look closely at the previous example, you'll notice a consistent pattern in consumer: we always lock the mutex before waiting (.Wait()) on the condition, and we unlock it after the condition is met.

Plus, we wrap the waiting condition inside a loop, here's a refresher:

// Consumer
go func() {
    cond.L.Lock()
    defer cond.L.Unlock()

    // waits until Pikachu appears
    for pokemon != "Pikachu" {
        cond.Wait()
    }
    println("Caught" + pokemon)
}()

Cond.Wait()

When we call Wait() on a sync.Cond, we're telling the current goroutine to hang tight until some condition is met.

Here's what's happening behind the scenes:

The goroutine gets added to a list of other goroutines that are also waiting on this same condition. All these goroutines are blocked, meaning they can't continue until they're "woken up" by either a Signal() or Broadcast() call.
The key part here is that the mutex must be locked before calling Wait() because Wait() does something important, it automatically releases the lock (calls Unlock()) before putting the goroutine to sleep. This allows other goroutines to grab the lock and do their work while the original goroutine is waiting.
When the waiting goroutine gets woken up (by Signal() or Broadcast()), it doesn't immediately resume work. First, it has to re-acquire the lock (Lock()).

The sync.Cond.Wait() method

Here's a look at how Wait() works under the hood:

func (c *Cond) Wait() {
    // Check if Cond has been copied
    c.checker.check()

    // Get the ticket number
    t := runtime_notifyListAdd(&c.notify)

    // Unlock the mutex     
    c.L.Unlock()

    // Suspend the goroutine until being woken up
    runtime_notifyListWait(&c.notify, t)

    // Re-lock the mutex
    c.L.Lock()
}

Even though it's simple, we can take away 4 main points:

There's a checker to prevent copying the Cond instance, it would be panic if you do so.
Calling cond.Wait() immediately unlocks the mutex, so the mutex must be locked before calling cond.Wait(), otherwise, it will panic.
After being woken up, cond.Wait() re-locks the mutex, which means you'll need to unlock it again after you're done with the shared data.
Most of sync.Cond's functionality is implemented in the Go runtime with an internal data structure called notifyList, which uses a ticket-based system for notifications.

Because of this lock/unlock behavior, there's a typical pattern you'll follow when using sync.Cond.Wait() to avoid common mistakes:

c.L.Lock()
for !condition() {
    c.Wait()
}
// ... make use of condition ...
c.L.Unlock()

The typical pattern for using sync.Cond.Wait()

"Why not just use c.Wait() directly without a loop?"

This is an excerpt of the post; the full post is available here: https://victoriametrics.com/blog/go-sync-cond/

Go sync.WaitGroup and The Alignment Problem

Phuong Le — Tue, 22 Oct 2024 01:00:00 +0000

This post is part of a series about handling concurrency in Go:

Go sync.Mutex: Normal and Starvation Mode
Go sync.WaitGroup and The Alignment Problem (We're here)
Go sync.Pool and the Mechanics Behind It
Go sync.Cond, the Most Overlooked Sync Mechanism
Go sync.Map: The Right Tool for the Right Job
Go Singleflight Melts in Your Code, Not in Your DB

WaitGroup is basically a way to wait for several goroutines to finish their work.

Each of sync primitives has its own set of problems, and this one's no different. We're going to focus on the alignment issues with WaitGroup, which is why its internal structure has changed across different versions.

This article is based on Go 1.23. If anything changes down the line, feel free to let me know through X(@func25).

What is sync.WaitGroup?

If you're already familiar with sync.WaitGroup, feel free to skip ahead.

Let's dive into the problem first, imagine you've got a big job on your hands, so you decide to break it down into smaller tasks that can run simultaneously, without depending on each other.

To handle this, we use goroutines because they let these smaller tasks run concurrently:

func main() {
    for i := 0; i < 10; i++ {
        go func(i int) {
            fmt.Println("Task", i)
        }(i)
    }

    fmt.Println("Done")
}

// Output:
// Done

But here's the thing, there's a good chance that the main goroutine finishes up and exits before the other goroutines are done with their work.

When we're spinning off many goroutines to do their thing, we want to keep track of them so that the main goroutine doesn't just finish up and exit before everyone else is done. That's where the WaitGroup comes in. Each time one of our goroutines wraps up its task, it lets the WaitGroup know.

Once all the goroutines have checked in as ‘done,' the main goroutine knows it's safe to finish, and everything wraps up neatly.

func main() {
    var wg sync.WaitGroup

    wg.Add(10)
    for i := 0; i < 10; i++ {
        go func(i int) {
            defer wg.Done()
            fmt.Println("Task", i)
        }(i)
    }

    wg.Wait()
    fmt.Println("Done")
}

// Output:
// Task 0
// Task 1
// Task 2
// Task 3
// Task 4
// Task 5
// Task 6
// Task 7
// Task 8
// Task 9
// Done

So, here's how it typically goes:

Adding goroutines: Before starting your goroutines, you tell the WaitGroup how many to expect. You do this with WaitGroup.Add(n), where n is the number of goroutines you're planning to run.
Goroutines running: Each goroutine goes off and does its thing. When it's done, it should let the WaitGroup know by calling WaitGroup.Done() to reduce the counter by one.
Waiting for all goroutines: In the main goroutine, the one not doing the heavy lifting, you call WaitGroup.Wait(). This pauses the main goroutine until that counter in the WaitGroup reaches zero. In plain terms, it waits until all the other goroutines have finished and signaled they're done.

Usually, you'll see WaitGroup.Add(1) being used when firing up a goroutine:

for i := 0; i < 10; i++ {   
    wg.Add(1)
    go func() {
        defer wg.Done()
        ...
    }()
}

Both ways are technically fine, but using wg.Add(1) has a small performance hit. Still, it's less error-prone compared to using wg.Add(n).

"Why is wg.Add(n) considered error-prone?"

The point is this, if the logic of the loop changes down the road, like if someone adds a continue statement that skips certain iterations, things can get messy:

wg.Add(10)
for i := 0; i < 10; i++ {
    if someCondition(i) {
        continue  
    }

    go func() {
        defer wg.Done()
        ...
    }()
}

In this example, we're using wg.Add(n) before the loop, assuming the loop will always start exactly n goroutines.

But if that assumption doesn't hold, like if some iterations get skipped, your program might get stuck waiting for goroutines that were never started. And let's be honest, that's the kind of bug that can be a real pain to track down.

In this case, wg.Add(1) is more suitable. It might come with a tiny bit of performance overhead, but it's a lot better than dealing with the human error overhead.

There's also a pretty common mistake people make when using sync.WaitGroup:

for i := 0; i < 10; i++ {
    go func() {
        wg.Add(1)  
        defer wg.Done()
        ...
    }()
}

Here's what it comes down to, wg.Add(1) is being called inside the goroutine. This can be an issue because the goroutine might start running after the main goroutine has already called wg.Wait().

That can cause all sorts of timing problems. Also, if you notice, all the examples above use defer with wg.Done(). It indeed should be used with defer to avoid issues with multiple return paths or panic recovery, making sure that it always gets called and doesn't block the caller indefinitely.

That should cover all the basics.

How sync.WaitGroup Looks Like?

Let's start by checking out the source code of sync.WaitGroup. You'll notice a similar pattern in sync.Mutex.

Again, if you're not familiar with how a mutex works, I strongly suggest you check out this article first: Go Sync Mutex: Normal & Starvation Mode.

type WaitGroup struct {
    noCopy noCopy

    state atomic.Uint64 
    sema  uint32
}

type noCopy struct{}

func (*noCopy) Lock()   {}
func (*noCopy) Unlock() {}

In Go, it's easy to copy a struct by just assigning it to another variable. But some structs, like WaitGroup, really shouldn't be copied.

Copying a WaitGroup can mess things up because the internal state that tracks the goroutines and their synchronization can get out of sync between the copies. If you've read the mutex post, you'll get the idea, imagine what could go wrong if we copied the internal state of a mutex.

The same kind of issues can happen with WaitGroup.

noCopy

The noCopy struct is included in WaitGroup as a way to help prevent copying mistakes, not by throwing errors, but by serving as a warning. It was contributed by Aliaksandr Valialkin, CTO of VictoriaMetrics, and was introduced in change #22015.

The noCopy struct doesn't actually affect how your program runs. Instead, it acts as a marker that tools like go vet can pick up on to detect when a struct has been copied in a way that it shouldn't be.

type noCopy struct{}

func (*noCopy) Lock()   {}
func (*noCopy) Unlock() {}

Its structure is super simple:

It has no fields, so it doesn't take up any meaningful space in memory.
It has two methods, Lock and Unlock, which do nothing (no-op). These methods are there just to work with the -copylocks checker in the go vet tool.

When you run go vet on your code, it checks to see if any structs with a noCopy field, like WaitGroup, have been copied in a way that could cause issues.

It will throw an error to let you know there might be a problem. This gives you a heads-up to fix it before it turns into a bug:

func main() {
    var a sync.WaitGroup
    b := a

    fmt.Println(a, b)
}

// go vet:
// assignment copies lock value to b: sync.WaitGroup contains sync.noCopy
// call of fmt.Println copies lock value: sync.WaitGroup contains sync.noCopy
// call of fmt.Println copies lock value: sync.WaitGroup contains sync.noCopy

In this case, go vet will warn you about 3 different spots where the copying happens. You can try it yourself at: Go Playground.

Note that it's purely a safeguard for when we're writing and testing our code, we can still run it like normal.

Internal State

The state of a WaitGroup is stored in an atomic.Uint64 variable. You might have guessed this if you've read the mutex post, there are several things packed into this single value.

WaitGroup structure

Here's how it breaks down:

Counter (high 32 bits): This part keeps track of the number of goroutines the WaitGroup is waiting for. When you call wg.Add() with a positive value, it bumps up this counter, and when you call wg.Done(), it decreases the counter by one.
Waiter (low 32 bits): This tracks the number of goroutines currently waiting for that counter (the high 32 bits) to hit zero. Every time you call wg.Wait(), it increases this "waiter" count. Once the counter reaches zero, it releases all the goroutines that were waiting.

Then there's the final field, sema uint32, which is an internal semaphore managed by the Go runtime.

when a goroutine calls wg.Wait() and the counter isn't zero, it increases the waiter count and then blocks by calling runtime_Semacquire(&wg.sema). This function call puts the goroutine to sleep until it gets woken up by a corresponding runtime_Semrelease(&wg.sema) call.

We'll dive deeper into this in another article, but for now, I want to focus on the alignment issues.

Alignment Problem

I know, talking about history might seem dull, especially when you just want to get to the point. But trust me, knowing the past is the best way to understand where we are now.

Let's take a quick look at how WaitGroup has evolved over several Go versions:

sync.WaitGroup in different Go versions

I can tell you, the core of WaitGroup (the counter, waiter, and semaphore) hasn't really changed across different Go versions. However, the way these elements are structured has been modified many times.

When we talk about alignment, we're referring to the need for data types to be stored at specific memory addresses to allow for efficient access.

For example, on a 64-bit system, a 64-bit value like uint64 should ideally be stored at a memory address that's a multiple of 8 bytes. The reason is, the CPU can grab aligned data in one go, but if the data isn't aligned, it might take multiple operations to access it.

Alignment issues

Now, here's where things get tricky:

On 32-bit architectures, the compiler doesn't guarantee that 64-bit values will be aligned on an 8-byte boundary. Instead, they might only be aligned on a 4-byte boundary.

This becomes a problem when we use the atomic package to perform operations on the state variable. The atomic package specifically notes:

"On ARM, 386, and 32-bit MIPS, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically via the primitive atomic functions." - atomic package note

What this means is that if we don't align the state uint64 variable to an 8-byte boundary on these 32-bit architectures, it could cause the program to crash.

So, what's the fix? Let's take a look at how this has been handled across different versions.

Go 1.5: state1 [12]byte

I'd recommend taking a moment to guess the underlying logic of this solution as you read the code below, then we'll walk through it together.

type WaitGroup struct {
    state1 [12]byte
    sema   uint32
}

func (wg *WaitGroup) state() *uint64 {
    if uintptr(unsafe.Pointer(&wg.state1))%8 == 0 {
        return (*uint64)(unsafe.Pointer(&wg.state1))
    } else {
        return (*uint64)(unsafe.Pointer(&wg.state1[4]))
    }
}

Instead of directly using a uint64 for state, WaitGroup sets aside 12 bytes in an array (state1 [12]byte). This might seem like more than you'd need, but there's a reason behind it.

WaitGroup in Go 1.5

The purpose of using 12 bytes is to ensure there's enough room to find an 8-byte segment that's properly aligned.

The full post is available here: https://victoriametrics.com/blog/go-sync-waitgroup/

Slices in Go: Grow Big or Go Home

Phuong Le — Sat, 05 Oct 2024 12:32:26 +0000

This is an excerpt of the post; the full post is available here: https://victoriametrics.com/blog/go-slice/

New developers often think slices are pretty simple to get, just a dynamic array that can change size compared to a regular array. But honestly, it's trickier than it seems when it comes to how they change size.

So, let's say we have a slice variable a, and you assign it to another variable b. Now, both a and b are pointing to the same underlying array. If you make any changes to the slice a, you're gonna see those changes reflected in b too.

But that's not always the case.

The link between a and b isn't all that strong, and in Go, you can't count on every change in a showing up in b.

Experienced Go developers think of a slice as a pointer to an array, but here's the catch: that pointer can change without notice, which makes slices tricky if you don't fully understand how they work. In this discussion, we'll cover everything from the basics to how slices grow and how they're allocated in memory.

Before we get into the details, I'd suggest checking out how arrays work first.

How Slice is Structured

Once you declare an array with a specific length, that length is "locked" in as part of its type. For example, an array of [1024]byte is a completely different type from an array of [512]byte.

Now, slices are way more flexible than arrays since they're basically a layer on top of an array. They can resize dynamically, and you can use append() to add more elements.

There are quite a few ways you can create a slice:

// a is a nil slice
var a []byte

// slice literal
b := []byte{1, 2, 3}

// slice from an array
c := b[1:3]

// slice with make
d := make([]byte, 1, 3)

// slice with new
e := *new([]byte)

That last one isn't really common, but it's legit syntax.

Unlike arrays, where len and cap are constants and always equal, slices are different. In arrays, the Go compiler knows the length and capacity ahead of time and even bakes that into the Go assembly code.

Array's length and capacity in Go assembly code

But with slices, len and cap are dynamic, meaning they can change at runtime.

Slices are really just a way to describe a 'slice' of the underlying array.

For example, if you have a slice like [1:3], it starts at index 1 and ends just before index 3, so the length is 3 - 1 = 2.

func main() {
    array := [6]int{0, 1, 2, 3, 4, 5}

    slice := array[1:3]
    fmt.Println(slice, len(slice), cap(slice))
}

// Output:
// [1 2] 2 5

The situation above could be represented as the following diagram.

Slice's length and capacity

The len of a slice is simply how many elements are in it. In this case, we have 2 elements [1, 2]. The cap is basically the number of elements from the start of the slice to the end of the underlying array.

That definition of capacity above is a bit inaccurate, we will talk about it in growing section.

Since a slice points to the underlying array, any changes you make to the slice will also change the underlying array.

"I know the length and capacity of a slice through the len and cap functions, but how do I figure out where the slice actually starts?"

Let me show you 3 ways to find the start of a slice by looking in its internal representation.

Instead of using fmt.Println, you can use println to get the raw values of the slice:

func main() {
    array := [6]byte{0, 1, 2, 3, 4, 5}
    slice := array[1:3]

    println("array:", &array)
    println("slice:", slice, len(slice), cap(slice))
}

// Output:
// array: 0x1400004e6f2
// slice: [2/5]0x1400004e6f3 2 5

From that output, you can see that the address of the slice's underlying array is different from the address of the original array, that's weird, right?

Let's visualize this in the diagram below.

Slice and its underlying array

If you've checked out the earlier post on arrays, you'll get how elements are stored in an array. What's really happening is that the slice is pointing directly to array[1].

The second way to prove it is by getting the pointer to the slice's underlying array using unsafe.SliceData:

func main() {
    array := [6]byte{0, 1, 2, 3, 4, 5}
    slice := array[1:3]

    arrPtr := unsafe.SliceData(slice)
    println("array[1]:", &array[1])
    println("slice.array:", arrPtr)
}

// Output:
// array[1]: 0x1400004e6f3
// slice.array: 0x1400004e6f3

When you pass a slice to unsafe.SliceData, it does a few checks to figure out what to return:

If the slice has a capacity greater than 0, the function returns a pointer to the first element of the slice (which is array[1] in this case).
If the slice is nil, the function just returns nil.
If the slice isn't nil but has zero capacity (an empty slice), the function gives you a pointer, but it's pointing to "unspecified memory address".

You can find all of this documented in the Go documentation.

"What do you mean by 'This pointer is pointing to an unspecified memory address'?"

It's a bit out of context, but let's satisfy our curiosity :)

In Go, you can have types with zero size, like struct{} or [0]int. When the Go runtime allocates memory for these types, instead of giving each one a unique memory address, it just returns the address of a special variable called zerobase.

You're probably getting the idea, right?

The 'unspecified' memory we mentioned earlier is this zerobase address.

func main() {
    var a struct{}
    fmt.Printf("struct{}: %p\n", &a)

    var b [0]int
    fmt.Printf("[0]int: %p\n", &b)

    fmt.Println("unsafe.SliceData([]int{}):", unsafe.SliceData([]int{}))
}

// Output:
// struct{}: 0x104f24900
// [0]int: 0x104f24900
// unsafe.SliceData([]int{}): 0x104f24900

Pretty cool, right? It's like we just uncovered a little mystery that Go was keeping under wraps.

Let's move on to the third way.

Behind the scenes, a slice is just a struct with three fields: array - a pointer to the underlying array, len - the length of the slice, and cap - the capacity of the slice.

type slice struct {
    array unsafe.Pointer
    len   int
    cap   int
}

This is also the setup for our third way to figure out the start of a slice. While we're at it, we'll prove that the struct above is indeed how slices work internally.

type sliceHeader struct {
    array unsafe.Pointer
    len   int
    cap   int
}

func main() {
    array := [6]byte{0, 1, 2, 3, 4, 5}

    slice := array[1:3]
    println("slice", slice)

    header := (*sliceHeader)(unsafe.Pointer(&slice))
    println("sliceHeader:", header.array, header.len, header.cap)
}

// Output:
// slice [2/5]0x1400004e6f3
// sliceHeader: 0x1400004e6f3 2 5

The output is exactly what we expect and we usually refer to this internal structure as the slice header (sliceHeader). There's also a reflect.SliceHeader in the reflect package, but that's deprecated.

Now that we've mastered how slices are structured, it's time to dive into how they actually behave.

How Slice Grows

Earlier, I mentioned that "the cap is basically the length of the underlying array starting from the first element of the slice up to the end of that array." That's not entirely accurate, it's only true in that specific example.

For instance, when you create a new slice using slicing operations, there's an option to specify the capacity of the slice:

func main() {
    array := [6]int{0, 1, 2, 3, 4, 5}
    slice := array[1:3:4]

    println(slice)
}

// Output:
// [2/3]0x1400004e718

By default, if you don't specify the third parameter in the slicing operation, the capacity is taken from the sliced slice or the length of the sliced array.

In this example, the capacity of slice is set to go up to index 4 (exclusive, like the length) of the original array.

Slice's capacity

So, let's redefine what the capacity of a slice really means.

"The capacity of a slice is the maximum number of elements it can hold before it needs to grow."

If you keep adding elements to a slice and it surpasses its current capacity, Go will automatically create a larger array, copy the elements over, and use that new array as the slice's underlying array.

The full post is available here: https://victoriametrics.com/blog/go-slice/

Go sync.Pool and the Mechanics Behind It

Phuong Le — Tue, 17 Sep 2024 09:00:00 +0000

This is an excerpt of the post; the full post is available here: https://victoriametrics.com/blog/go-sync-pool/

This post is part of a series about handling concurrency in Go:

Go sync.Mutex: Normal and Starvation Mode
Go sync.WaitGroup and The Alignment Problem
Go sync.Pool and the Mechanics Behind It (We're here)
Go sync.Cond, the Most Overlooked Sync Mechanism

In the VictoriaMetrics source code, we use sync.Pool a lot, and it's honestly a great fit for how we handle temporary objects, especially byte buffers or slices.

It is commonly used in the standard library. For instance, in the encoding/json package:

package json

var encodeStatePool sync.Pool

// An encodeState encodes JSON into a bytes.Buffer.
type encodeState struct {
    bytes.Buffer // accumulated output

    ptrLevel uint
    ptrSeen  map[any]struct{}
}

In this case, sync.Pool is being used to reuse *encodeState objects, which handle the process of encoding JSON into a bytes.Buffer.

Instead of just throwing these objects after each use, which would only give the garbage collector more work, we stash them in a pool (sync.Pool). The next time we need something similar, we just grab it from the pool instead of making a new one from scratch.

You'll also find multiple sync.Pool instances in the net/http package, that are used to optimize I/O operations:

package http

var (
    bufioReaderPool   sync.Pool
    bufioWriter2kPool sync.Pool
    bufioWriter4kPool sync.Pool
)

When the server reads request bodies or writes responses, it can quickly pull a pre-allocated reader or writer from these pools, skipping extra allocations. Furthermore, the 2 writer pools, *bufioWriter2kPool and *bufioWriter4kPool, are set up to handle different writing needs.

func bufioWriterPool(size int) *sync.Pool {
    switch size {
    case 2 << 10:
        return &bufioWriter2kPool
    case 4 << 10:
        return &bufioWriter4kPool
    }
    return nil
}

Alright, that's enough of the intro.

Today, we're diving into what sync.Pool is all about, the definition, how it's used, what's going on under the hood, and everything else you might want to know.

By the way, if you want something more practical, there's a good article from our Go experts showing how we use sync.Pool in VictoriaMetrics: Performance optimization techniques in time series databases: sync.Pool for CPU-bound operations

What is sync.Pool?

To put it simply, sync.Pool in Go is a place where you can keep temporary objects for later reuse.

But here's the thing, you don't control how many objects stay in the pool, and anything you put in there can be removed at any time, without any warning and you'll know why when reading last section.

The good point is, the pool is built to be thread-safe, so multiple goroutines can tap into it simultaneously. Not a big surprise, considering it's part of the sync package.

"But why do we bother reusing objects?"

When you've got a lot of goroutines running at once, they often need similar objects. Imagine running go f() multiple times concurrently.

If each goroutine creates its own objects, memory usage can quickly increase and this puts a strain on the garbage collector because it has to clean up all those objects once they're no longer needed.

This situation creates a cycle where high concurrency leads to high memory usage, which then slows down the garbage collector. sync.Pool is designed to help break this cycle.

type Object struct {
    Data []byte
}

var pool sync.Pool = sync.Pool{
    New: func() any {
        return &Object{
            Data: make([]byte, 0, 1024),
        }
    },
}

To create a pool, you can provide a New() function that returns a new object when the pool is empty. This function is optional, if you don't provide it, the pool just returns nil if it's empty.

In the snippet above, the goal is to reuse the Object struct instance, specifically the slice inside it.

Reusing the slice helps reduce unnecessary growth.

For instance, if the slice grows to 8192 bytes during use, you can reset its length to zero before putting it back in the pool. The underlying array still has a capacity of 8192, so the next time you need it, those 8192 bytes are ready to be reused.

func (o *Object) Reset() {
    o.Data = o.Data[:0]
}

func main() {
    testObject := pool.Get().(*Object)

    // do something with testObject

    testObject.Reset()
    pool.Put(testObject)
}

The flow is pretty clear: you get an object from the pool, use it, reset it, and then put it back into the pool. Resetting the object can be done either before you put it back or right after you get it from the pool, but it's not mandatory, it's a common practice.

If you're not a fan of using type assertions pool.Get().(*Object), there are a couple of ways to avoid it:

Use a dedicated function to get the object from the pool:

func getObjectFromPool() *Object {
    obj := pool.Get().(*Object)
    return obj
}

Create your own generic version of sync.Pool:

type Pool[T any] struct {
    sync.Pool
}

func (p *Pool[T]) Get() T {
    return p.Pool.Get().(T)
}

func (p *Pool[T]) Put(x T) {
    p.Pool.Put(x)
}

func NewPool[T any](newF func() T) *Pool[T] {
    return &Pool[T]{
        Pool: sync.Pool{
            New: func() interface{} {
                return newF()
            },
        },
    }
}

The generic wrapper gives you a more type-safe way to work with the pool, avoiding type assertions.

Just note that, it adds a tiny bit of overhead due to the extra layer of indirection. In most cases, this overhead is minimal, but if you're in a highly CPU-sensitive environment, it's a good idea to run benchmarks to see if it's worth it.

But wait, there's more to it.

sync.Pool and Allocation Trap

If you've noticed from many previous examples, including those in the standard library, what we store in the pool is typically not the object itself but a pointer to the object.

Let me explain why with an example:

var pool = sync.Pool{
    New: func() any {
        return []byte{}
    },
}

func main() {
    bytes := pool.Get().([]byte)

    // do something with bytes
    _ = bytes

    pool.Put(bytes)
}

We're using a pool of []byte. Generally (though not always), when you pass a value to an interface, it may cause the value to be placed on the heap. This happens here too, not just with slices but with anything you pass to pool.Put() that isn't a pointer.

If you check using escape analysis:

// escape analysis
$ go build -gcflags=-m

bytes escapes to heap

Now, I don't say our variable bytes moves to the heap, I would say "the value of bytes escapes to the heap through the interface".

To really get why this happens, we'd need to dig into how escape analysis works (which we might do in another article). However, if we pass a pointer to pool.Put(), there is no extra allocation:

var pool = sync.Pool{
    New: func() any {
        return new([]byte)
    },
}

func main() {
    bytes := pool.Get().(*[]byte)

    // do something with bytes
    _ = bytes

    pool.Put(bytes)
}

Run the escape analysis again, you'll see it's no longer escapes to the heap. If you want to know more, there is an example in Go source code.

sync.Pool Internals

Before we get into how sync.Pool actually works, it's worth getting a grip on the basics of Go's PMG scheduling model, this is really the backbone of why sync.Pool is so efficient.

There's a good article that breaks down the PMG model with some visuals: PMG models in Go

If you're feeling lazy today and looking for a simplified summary, I've got your back:

PMG stands for P (logical processors), M (machine threads), and G (goroutines). The key point is that each logical processor (P) can only have one machine thread (M) running on it at any time. And for a goroutine (G) to run, it needs to be attached to a thread (M).

PMG model

This boils down to 2 key points:

If you've got n logical processors (P), you can run up to n goroutines in parallel, as long as you've got at least n machine threads (M) available.
At any one time, only one goroutine (G) can run on a single processor (P). So, when a P1 is busy with a G, no other G can run on that P1 until the current G either gets blocked, finishes up, or something else happens to free it up.

But the thing is, a sync.Pool in Go isn't just one big pool, it's actually made up of several 'local' pools, with each one tied to a specific processor context, or P, that Go's runtime is managing at any given time.

Local pools

When a goroutine running on a processor (P) needs an object from the pool, it'll first check its own P-local pool before looking anywhere else.

The full post is available here: https://victoriametrics.com/blog/go-sync-pool/

Go Maps Explained: How Key-Value Pairs Are Actually Stored

Phuong Le — Tue, 10 Sep 2024 01:00:00 +0000

This is an excerpt of the post; the full post is available here: https://victoriametrics.com/blog/go-map/.

If you're new to Go, it can be a bit confusing to figure out how to use maps in Go. And even when you're more experienced, understanding how maps really work can be pretty tough.

Take this example: Have you ever set a 'hint' for a map and wondered why it's called a 'hint' and not something simple like length, like we do with slices?

// hint = 10
m := make(map[string]int, 10)

Or maybe you've noticed that when you use a for-range loop on a map, the order doesn't match the insertion order, and it even changes if you loop over the same map at different times. But weirdly enough, if you loop over it at the exact same time, the order usually stays the same.

This is a long story, so fasten your seat belt and dive in.

Before we move on, just a heads up—the info here is based on Go 1.23. If things have changed and this isn't up to date, feel free to ping me on X(@func25).

Map in Go: Quick Start

So, let's talk about maps in Go, it's a built-in type that acts as a key-value storage. Unlike arrays where you're stuck with keys as increasing indices like 0, 1, 2, and so on, with maps, the key can be any comparable type.

That gives you a lot more flexibility.

m := make(map[string]int)
m["a"] = 1
m["b"] = 2

m // map[a:1 b:2]

In that example, we created an empty map using make(), where the keys are strings and the values are ints.

Map["a": 1, "b": 2]

Now, instead of manually assigning each key, you can save yourself some time by using a map literal. This lets you set up your key-value pairs all at once, right when you create the map:

m := map[string]int{
    "a": 1,
    "b": 2,
}

All you do is list out the keys and their values inside curly braces when you first create the map, simple as that.

And if you realize later that you don't need a certain key-value pair anymore, Go's got you covered. There's a handy delete function that, well, deletes the key you don't want: delete(m, "a").

The zero value of a map is nil, and a nil map is kind of like an empty map in some ways. You can try to look up keys in it, and Go won't freak out and crash your program.

If you search for a key that isn't there, Go just quietly gives you the "zero value" for that map's value type:

var m map[string]int

println(m["a"]) // 0
m["a"] = 1      // panic: assignment to entry in nil map

But here's the thing: you can't add new key-value pairs to a nil map.

In fact, Go handles maps in a way that's pretty similar to how it deals with slices. Both maps and slices start off as nil, and Go doesn't panic if you do something “harmless” with them while they're nil. For example, you can loop over a nil slice without any "drama".

So, what happens if you try to loop over a nil map?

var m map[string]int

for k, v := range m {
    println(k, v)
}

Nothing happens, no errors, no surprises. It just quietly does nothing.

Go's approach is to treat the default value of any type as something useful, not something that causes your program to blow up. The only time Go throws a fit is when you do something that's truly illegal, like trying to add a new key-value pair to a nil map or accessing an out-of-bounds index in a slice.

There are a couple more things you should know about maps in Go:

A for-range loop over a map won't return the keys in any specific order.
Maps aren't thread-safe, the Go runtime will cause a fatal error if you try to read (or iterate with a for-range) and write to the same map simultaneously.
You can check if a key is in a map by doing a simple ok check: _, ok := m[key].
The key type for a map must be comparable.

Let's dive into that last point about map keys. I mentioned earlier that "the key could be any comparable type," but there's a bit more to it than just that.

"So, what exactly is a comparable type? And what isn't?"

It's pretty simple: if you can use == to compare two values of the same type, then that type is considered comparable.

func main() {
    var s map[int]string

    if s == s {
        println("comparable")
    }
}

// compile error: invalid operation: s == s (map can only be compared to nil)

But as you can see, the code above doesn't even compile. The compiler complains: "invalid operation: s == s (map can only be compared to nil)."

This same rule applies to other non-comparable types like slices, functions, or structs that contain slices or maps, etc. So, if you're trying to use any of those types as keys in a map, you're out of luck.

func main() {
  var s map[[]int]string
}

// compile error: invalid map key type []intcompilerIncomparableMapKey

But here's a little secret, interfaces can be both comparable and non-comparable.

What does that mean? You can absolutely define a map with an empty interface as the key without any compile errors. But watch out, there's a good chance you'll run into a runtime error.

func main() {
    m := map[interface{}]int{
        1: 1,
        "a": 2,
    }

    m[[]int{1, 2, 3}] = 3
    m[func() {}] = 4
}

// panic: runtime error: hash of unhashable type []int
// panic: runtime error: hash of unhashable type func()

Everything looks fine until you try to assign an uncomparable type as a map key.

That's when you'll hit a runtime error, which is trickier to deal with than a compile-time error. Because of this, it's usually a good idea to avoid using interface{} as a map key unless you have a solid reason and constraints that prevent misuse.

But that error message: "hash of unhashable type []int" might seem a bit cryptic. What's this about a hash? Well, that's our cue to dig into how Go handles things under the hood.

Map Anatomy

When explaining the internals of something like a map, it's easy to get bogged down in the nitty-gritty details of the Go source code. But we're going to keep it light and simple so even those new to Go can follow along.

What you see as a single map in your Go code is actually an abstraction that hides the complex details of how the data is organized. In reality, a Go map is composed of many smaller units called "buckets."

type hmap struct {
  ...
  buckets unsafe.Pointer
  ...
}

Look at Go source code above, a map contains a pointer that points to the bucket array.

This is why when you assign a map to a variable or pass it to a function, both the variable and the function's argument are sharing the same map pointer.

func changeMap(m2 map[string]int) {
  m2["hello"] = 2
}

func main() {
  m1 := map[string]int{"hello": 1}
  changeMap(m1)
  println(m1["hello"]) // 2
}

But don't get it twisted, maps are pointers to the hmap under the hood, but they aren't reference types, nor are they passed by reference like a ref argument in C#, if you change the whole map m2, it won't reflect on the original map m1 in the caller.

func changeMap(m2 map[string]int) {
  m2 = map[string]int{"hello": 2}
}

func main() {
  m1 := map[string]int{"hello": 1}
  changeMap(m1)
  println(m1["hello"]) // 1
}

In Go, everything is passed by value. What's really happening is a bit different: when you pass the map m1 to the changeMap function, Go makes a copy of the *hmap structure. So, m1 in the main() and m2 in the changeMap() function are technically different pointers point to the same hmap.

Map is passed by value

For more on this topic, there's a great post by Dave Cheney titled There is no pass-by-reference in Go.

Each of these buckets can only hold up to 8 key-value pairs, as you can see in the image below.

Buckets of a map

The map above has 2 buckets, and len(map) is 6.

So, when you add a key-value pair to a map, Go doesn't just drop it in there randomly or sequentially. Instead, it places the pair into one of these buckets based on the key's hash value, which is determined by hash(key, seed).

Let's see the simplest assignment scenario in the image below, when we have an empty map, and assign a key-value pair "hello": 1 to it.

Assign a key-value pair to an empty map

It starts by hashing "hello" to a number, then it takes that number and mods it by the number of buckets.

Since we only have one bucket here, any number mod 1 is 0, so it's going straight into bucket 0 and the same process happens when you add another key-value pair. It'll try to place it in bucket 0, and if the first slot's taken or has a different key, it'll move to the next slot in that bucket.

Take a look at the hash(key, seed), when you use a for-range loop over two maps with the same keys, you might notice that the keys come out in a different order:

func main() {
    a := map[string]int{"a": 1, "b": 2, "c": 3, "d": 4, "e": 5, "f": 6}
    b := map[string]int{"a": 1, "b": 2, "c": 3, "d": 4, "e": 5, "f": 6}

    for i := range a {
        print(i, " ")
    }
    println()

    for i := range b {
        print(i, " ")
    }
}

// Output:
// a b c d e f 
// c d e f a b

How's that possible? Isn't the key "a" in map a and the key "a" in map b hashed the same way?

But here's the deal, while the hash function used for maps in Go is consistent across all maps with the same key type, the seed used by that hash function is different for each map instance. So, when you create a new map, Go generates a random seed just for that map.

In the example above, both a and b use the same hash function because their keys are string types, but each map has its own unique seed.

"Wait, a bucket has only 8 slots? What happens if the bucket gets full? Does it grow like a slice?"

Well, sort of. When the buckets start getting full, or even almost full, depending on the algorithm's definition of "full", the map will trigger a growth, which might double the number of main buckets.

But here's where it gets a bit more interesting.

When I say "main buckets," I'm setting up for another concept: "overflow buckets." These come into play when you've got a situation with high collisions. Imagine you have 4 buckets, but one of them is completely filled with 8 key-value pairs due to high collisions, while the other 3 buckets are sitting empty.

The full post is available here: https://victoriametrics.com/blog/go-map/.

Go Sync Mutex: Normal and Starvation Mode

Phuong Le — Mon, 02 Sep 2024 02:23:33 +0000

This is an excerpt of the post; the full post is available here: Golang Sync Mutex: Normal and Starvation Mode.

Mutex, or MUTual EXclusion, in Go is basically a way to make sure that only one goroutine is messing with a shared resource at a time. This resource can be a piece of code, an integer, a map, a struct, a channel, or pretty much anything.

Now, the explanation above isn't strictly the 'academic' definition, but it's a useful way to understand the concept.

In today's discussion, we're still going from the problem, moving on to the solution, and then diving into how it's actually put together under the hood.

Why we need sync.Mutex?

If you've spent enough time messing around with maps in Go, you might run into a nasty error like this:

fatal error: concurrent map read and map write

This happens because we're not protecting our map from multiple goroutines trying to access and write to it at the same time.

Now, we could use a map with a mutex or a sync.Map, but that's not our focus today. The star of the show here is sync.Mutex, and it's got three main operations: Lock, Unlock, and TryLock (which we won't get into right now).

When a goroutine locks a mutex, it's basically saying, 'Hey, I'm going to use this shared resource for a bit,' and every other goroutine has to wait until the mutex is unlocked. Once it's done, it should unlock the mutex so other goroutines can get their turn.

Simple as that, Let's see how it works with a simple counter example:

var counter = 0
var wg sync.WaitGroup

func incrementCounter() {
    counter++
    wg.Done()
}

func main() {
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go incrementCounter()
    }

    wg.Wait()
    fmt.Println(counter)
}

So, we've got this counter variable that's shared between 1000 goroutines. A newcomer to Go would think the result should be 1000, but it never is. This is because of something called a "race condition."

A race condition happens when multiple goroutines try to access and change shared data at the same time without proper synchronization. In this case, the increment operation (counter++) isn't atomic.

It's made up of multiple steps, below is the Go assembly code for counter++ in ARM64 architecture:

MOVD    main.counter(SB), R0
ADD $1, R0, R0
MOVD    R0, main.counter(SB)

The counter++ is a read-modify-write operation and these steps above aren't atomic, meaning they're not executed as a single, uninterruptible action.

For instance, goroutine G1 reads the value of counter, and before it writes the updated value, goroutine G2 reads the same value. Both then write their updated values back, but since they read the same original value, one increment is practically lost.

Race condition

Using the atomic package is a good way to handle this, but today let's focus on how a mutex solves this problem:

var counter = 0
var wg sync.WaitGroup
var mutex sync.Mutex

func incrementCounter() {
    mutex.Lock()
    counter++
    mutex.Unlock()
    wg.Done()
}

func main() {
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go incrementCounter()
    }

    wg.Wait()
    fmt.Println(counter)
}

Now, the result is 1000, just as we expected. Using a mutex here is super straightforward: wrap the critical section with Lock and Unlock. But watch out, if you call Unlock on an already unlocked mutex, it'll cause a fatal error sync: unlock of unlocked mutex.

It's usually a good idea to use defer mutex.Unlock() to ensure the unlock happens, even if something goes wrong. We've also got an article about Golang Defer: From Basic To Traps.

Also, you could set GOMAXPROCS to 1 by runnning runtime.GOMAXPROCS(1), and the result would still be correct at 1000. This is because our goroutines wouldn't be running in parallel, and the function is simple enough not to be preempted while running.

Mutex Structure: The Anatomy

Before we dive into how the lock and unlock flow works in Go's sync.Mutex, let's break down the structure, or anatomy, of the mutex itself:

package sync

type Mutex struct {
    state int32
    sema  uint32
}

At its core, a mutex in Go has two fields: state and sema. They might look like simple numbers, but there's more to them than meets the eye.

The state field is a 32-bit integer that shows the current state of the mutex. It's actually divided into multiple bits that encode various pieces of information about the mutex.

Mutex structure

Let's make a rundown of state from the image:

Locked (bit 0): Whether the mutex is currently locked. If it's set to 1, the mutex is locked and no other goroutine can grab it.
Woken (bit 1): Set to 1 if any goroutine has been woken up and is trying to acquire the mutex. Other goroutines shouldn't be woken up unnecessarily.
Starvation (bit 2): This bit shows if the mutex is in starvation mode (set to 1). We'll dive into what this mode means in a bit.
Waiter (bit 3-31): The rest of the bits keep track of how many goroutines are waiting for the mutex.

The other field, sema, is a uint32 that acts as a semaphore to manage and signal waiting goroutines. When the mutex is unlocked, one of the waiting goroutines is woken up to acquire the lock.

Unlike the state field, sema doesn't have a specific bit layout and relies on runtime internal code to handle the semaphore logic.

Mutex Lock Flow

In the mutex.Lock function, there are two paths: the fast path for the usual case and the slow path for handling the unusual case.

func (m *Mutex) Lock() {
    // Fast path: grab unlocked mutex.
    if atomic.CompareAndSwapInt32(&m.state, 0, mutexLocked) {
        if race.Enabled {
            race.Acquire(unsafe.Pointer(m))
        }
        return
    }
    // Slow path (outlined so that the fast path can be inlined)
    m.lockSlow()
}

The fast path is designed to be really quick and is expected to handle most lock acquisitions where the mutex isn't already in use. This path is also inlined, meaning it's embedded directly into the calling function:

$ go build -gcflags="-m"

./main.go:13:12: inlining call to sync.(*Mutex).Lock
./main.go:15:14: inlining call to sync.(*Mutex).Unlock

FYI, this inlined fast path is a neat trick that utilizes Go's inline optimization, and it's used a lot in Go's source code.

When the CAS (Compare And Swap) operation in the fast path fails, it means the state field wasn't 0, so the mutex is currently locked.

The real concern here is the slow path m.lockSlow, which does most of the heavy lifting. We won't dive too deep into the source code since it requires a lot of knowledge about Go's internal workings.

I'll discuss the mechanism and maybe a bit of the internal code to keep things clear. In the slow path, the goroutine keeps actively spinning to try to acquire the lock, it doesn't just go straight to the waiting queue.

"What do you mean by spinning?"

Spinning means the goroutine enters a tight loop, repeatedly checking the state of the mutex without giving up the CPU.

In this case, it is not a simple for loop but low-level assembly instructions to perform the spin-wait. Let's take a quick peek at this code on ARM64 architecture:

TEXT runtime·procyield(SB),NOSPLIT,$0-0
    MOVWU   cycles+0(FP), R0
again:
    YIELD
    SUBW    $1, R0
    CBNZ    R0, again
    RET

The assembly code runs a tight loop for 30 cycles (runtime.procyield(30)), repeatedly yielding the CPU and decrementing the spin counter.

After spinning, it tries to acquire the lock again. If it fails, it has three more chances to spin before giving up. So, in total, it tries for up to 120 cycles. If it still can't get the lock, it increases the waiter count, puts itself in the waiting queue, goes to sleep, waits for a signal to wake up and try again.

Why do we need spinning?

The idea behind spinning is to wait a short while in hopes that the mutex will free up soon, letting the goroutine grab the mutex without the overhead of a sleep-wake cycle.

...

Full post is available here: Go Sync Mutex: Normal and Starvation Mode.

How Go Arrays Work and Get Tricky with For-Range

Phuong Le — Tue, 20 Aug 2024 10:18:04 +0000

How Go Arrays Work and Get Tricky with For-Range

This is an excerpt of the post; the full post is available here: How Go Arrays Work and Get Tricky with For-Range.

The classic Golang array and slice are pretty straightforward. Arrays are fixed-size, and slices are dynamic. But I've got to tell you, Go might seem simple on the surface, but it's got a lot going on under the hood.

As always, we'll start with the basics and then dig a bit deeper. Don't worry, arrays get pretty interesting when you look at them from different angles.

We'll cover slices in the next part, I'll drop that here once it's ready.

What is an array?

Arrays in Go are a lot like those in other programming languages. They've got a fixed size and store elements of the same type in contiguous memory locations.

This means Go can access each element quickly since their addresses are calculated based on the starting address of the array and the element's index.

func main() {
    arr := [5]byte{0, 1, 2, 3, 4}
    println("arr", &arr)

    for i := range arr {
        println(i, &arr[i])
    }
}

// Output:
// arr 0x1400005072b
// 0 0x1400005072b
// 1 0x1400005072c
// 2 0x1400005072d
// 3 0x1400005072e
// 4 0x1400005072f

There are a couple of things to notice here:

The address of the array arr is the same as the address of the first element.
The address of each element is 1 byte apart from each other because our element type is byte.

Array [5]byte{0, 1, 2, 3, 4} in memory

Look at the image carefully.

Our stack is growing downwards from a higher to a lower address, right? This picture shows exactly how an array looks in the stack, from arr[4] to arr[0].

So, does that mean we can access any element of an array by knowing the address of the first element (or the array) and the size of the element? Let's try this with an int array and unsafe package:

func main() {
    a := [3]int{99, 100, 101}

    p := unsafe.Pointer(&a[0])

    a1 := unsafe.Pointer(uintptr(p) + 8)
    a2 := unsafe.Pointer(uintptr(p) + 16)

    fmt.Println(*(*int)(p))
    fmt.Println(*(*int)(a1))
    fmt.Println(*(*int)(a2))
}

// Output:
// 99
// 100
// 101

Well, we get the pointer to the first element and then calculate the pointers to the next elements by adding multiples of the size of an int, which is 8 bytes on a 64-bit architecture. Then we use these pointers to access and convert them back to the int values.

Array [3]int{99, 100, 101} in memory

The example is just a play around with the unsafe package to access memory directly for educational purposes. Don't do this in production without understanding the consequences.

Now, an array of type T is not a type by itself, but an array with a specific size and type T, is considered a type. Here's what I mean:

func main() {
    a := [5]byte{}
    b := [4]byte{}

    fmt.Printf("%T\n", a) // [5]uint8
    fmt.Printf("%T\n", b) // [4]uint8

    // cannot use b (variable of type [4]byte) as [5]byte value in assignment
    a = b 
}

Even though both a and b are arrays of bytes, the Go compiler sees them as completely different types, the %T format makes this point clear.

Here is how the Go compiler sees it internally (src/cmd/compile/internal/types2/array.go):

// An Array represents an array type.
type Array struct {
    len  int64
    elem Type
}

// NewArray returns a new array type for the given element type and length.
// A negative length indicates an unknown length.
func NewArray(elem Type, len int64) *Array { return &Array{len: len, elem: elem} }

The length of the array is "encoded" in the type itself, so the compiler knows the length of the array from its type. Trying to assign an array of one size to another, or compare them, will result in a mismatched type error.

Array literals

There are many ways to initialize an array in Go, and some of them might be rarely used in real projects:

var arr1 [10]int // [0 0 0 0 0 0 0 0 0 0]

// With value, infer-length
arr2 := [...]int{1, 2, 3, 4, 5} // [1 2 3 4 5]

// With index, infer-length
arr3 := [...]int{11: 3} // [0 0 0 0 0 0 0 0 0 0 0 3]

// Combined index and value
arr4 := [5]int{1, 4: 5} // [1 0 0 0 5]
arr5 := [5]int{2: 3, 4, 4: 5} // [0 0 3 4 5]

What we're doing above (except for the first one) is both defining and initializing their values, which is called a "composite literal." This term is also used for slices, maps, and structs.

Now, here's an interesting thing: when we create an array with less than 4 elements, Go generates instructions to put the values into the array one by one.

So when we do arr := [3]int{1, 2, 3, 4}, what's actually happening is:

arr := [4]int{}
arr[0] = 1
arr[1] = 2
arr[2] = 3
arr[3] = 4

This strategy is called local-code initialization. This means that the initialization code is generated and executed within the scope of a specific function, rather than being part of the global or static initialization code.

It'll become clearer when you read another initialization strategy below, where the values aren't placed into the array one by one like that.

"What about arrays with more than 4 elements?"

The compiler creates a static representation of the array in the binary, which is known as 'static initialization' strategy.

This means the values of the array elements are stored in a read-only section of the binary. This static data is created at compile time, so the values are directly embedded into the binary. If you're curious how [5]int{1,2,3,4,5} looks like in Go assembly:

main..stmp_1 SRODATA static size=40
    0x0000 01 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00  ................
    0x0010 03 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00  ................
    0x0020 05 00 00 00 00 00 00 00                          ........

It's not easy to see the value of the array, we can still get some key info from this.

Our data is stored in stmp_1, which is read-only static data with a size of 40 bytes (8 bytes for each element), and the address of this data is hardcoded in the binary.

The compiler generates code to reference this static data. When our application runs, it can directly use this pre-initialized data without needing additional code to set up the array.

const readonly = [5]int{1, 2, 3, 4, 5}

arr := readonly

"What about an array with 5 elements but only 3 of them initialized?"

Good question, this literal [5]int{1,2,3} falls into the first category, where Go puts the value into the array one by one.

While talking about defining and initializing arrays, we should mention that not every array is allocated on the stack. If it's too big, it gets moved to the heap.

But how big is "too big," you might ask.

As of Go 1.23, if the size of the variable, not just array, exceeds a constant value MaxStackVarSize, which is currently 10 MB, it will be considered too large for stack allocation and will escape to the heap.

func main() {
    a := [10 * 1024 * 1024]byte{}
    println(&a)

    b := [10*1024*1024 + 1]byte{}
    println(&b)
}

In this scenario, b will move to the heap while a won't.

Array operations

The length of the array is encoded in the type itself. Even though arrays don't have a cap property, we can still get it:

func main() {
    a := [5]int{1, 2, 3}
    println(len(a)) // 5
    println(cap(a)) // 5
}

The capacity equals the length, no doubt, but the most important thing is that we know this at compile time, right?

So len(a) doesn't make sense to the compiler because it's not a runtime property, Go compiler knows the value at compile time.

...

This is an excerpt of the post; the full post is available here: How Go Arrays Work and Get Tricky with For-Range.

Golang Defer: Heap-allocated, Stack-allocated, Open-coded Defer

Phuong Le — Wed, 07 Aug 2024 07:30:16 +0000

This is an excerpt of the post; the full post is available here: Golang Defer: From Basic To Trap.

The defer statement is probably one of the first things we find pretty interesting when we start learning Go, right?

But there's a lot more to it that trips up many people, and there're many fascinating aspects that we often don't touch on when using it.

Heap-allocated, Stack-allocated, Open-coded defer

For example, the defer statement actually has 3 types (as of Go 1.22, though that might change later): open-coded defer, heap-allocated defer, and stack-allocated. Each one has different performance and different scenarios where they're best used, which is good to know if you want to optimize performance.

In this discussion, we're going to cover everything from the basics to the more advanced usage, and we'll even dig a bit, just a little bit, into some of the internal details.

What is defer?

Let's take a quick look at defer before we dive too deep.

In Go, defer is a keyword used to delay the execution of a function until the surrounding function finishes.

func main() {
  defer fmt.Println("hello")
  fmt.Println("world")
}

// Output:
// world
// hello

In this snippet, the defer statement schedules fmt.Println("hello") to be executed at the very end of the main function. So, fmt.Println("world") is called immediately, and "world" is printed first. After that, because we used defer, "hello" is printed as the last step before main finishes.

It's just like setting up a task to run later, right before the function exits. This is really useful for cleanup actions, like closing a database connection, freeing up a mutex, or closing a file:

func doSomething() error {
  f, err := os.Open("phuong-secrets.txt")
  if err != nil {
    return err
  }
  defer f.Close()

  // ...
}

The code above is a good example to show how defer works, but it's also a bad way to use defer. We'll get into that in the next section.

"Okay, good, but why not put the f.Close() at the end?"

There are a couple of good reasons for this:

We put the close action near the open, so it's easier to follow the logic and avoid forgetting to close the file. I don't want to scroll down a function to check if the file is closed or not; it distracts me from the main logic.
The deferred function is called when the function returns, even if a panic (runtime error) happens.

When a panic happens, the stack is unwound and the deferred functions are executed in a specific order, which we'll cover in the next section.

Defers are stacked

When you use multiple defer statements in a function, they are executed in a 'stack' order, meaning the last deferred function is executed first.

func main() {
  defer fmt.Println(1)
  defer fmt.Println(2)
  defer fmt.Println(3)
}

// Output:
// 3
// 2
// 1

Every time you call a defer statement, you're adding that function to the top of the current goroutine's linked list, like this:

Goroutine defer chain

And when the function returns, it goes through the linked list and executes each one in the order shown in the image above.

But remember, it does not execute all the defer in the linked list of goroutine, it's only run the defer in the returned function, because our defer linked list could contain many defers from many different functions.

func B() {
  defer fmt.Println(1)
  defer fmt.Println(2)
  A()
}

func A() {
  defer fmt.Println(3)
  defer fmt.Println(4)
}

So, only the deferred functions in the current function (or current stack frame) are executed.

Goroutine defer chain

But there's one typical case where all the deferred functions in the current goroutine get traced and executed, and that's when a panic happens.

Defer, Panic and Recover

Besides compile-time errors, we have a bunch of runtime errors: divide by zero (integer only), out of bounds, dereferencing a nil pointer, and so on. These errors cause the application to panic.

Panic is a way to stop the execution of the current goroutine, unwind the stack, and execute the deferred functions in the current goroutine, causing our application to crash.

To handle unexpected errors and prevent the application from crashing, you can use the recover function within a deferred function to regain control of a panicking goroutine.

func main() {
  defer func() {
    if r := recover(); r != nil {
      fmt.Println("Recovered:", r)
    }
  }()

  panic("This is a panic")
}

// Output:
// Recovered: This is a panic

Usually, people put an error in the panic and catch that with recover(..), but it could be anything: a string, an int, etc.

In the example above, inside the deferred function is the only place you can use recover. Let me explain this a bit more.

There are a couple of mistakes we could list here. I’ve seen at least three snippets like this in real code.

The first one is, using recover directly as a deferred function:

func main() {
  defer recover()

  panic("This is a panic")
}

The code above still panics, and this is by design of the Go runtime.

The recover function is meant to catch a panic, but it has to be called within a deferred function to work properly.

Behind the scenes, our call to recover is actually the runtime.gorecover, and it checks that the recover call is happening in the right context, specifically from the correct deferred function that was active when the panic occurred.

"Does that mean we can’t use recover in a function inside a deferred function, like this?"

func myRecover() {
  if r := recover(); r != nil {
    fmt.Println("Recovered:", r)
  }
}

func main() {
  defer func() {
    myRecover()
    // ...
  }()

  panic("This is a panic")
}

Exactly, the code above won’t work as you might expect. That’s because recover isn’t called directly from a deferred function but from a nested function.

Now, another mistake is trying to catch a panic from a different goroutine:

func main() {
  defer func() {
    if r := recover(); r != nil {
      fmt.Println("Recovered:", r)
    }
  }()

  go panic("This is a panic")

  time.Sleep(1 * time.Second) // Wait for the goroutine to finish
}

Makes sense, right? We already know that defer chains belong to a specific goroutine. It would be tough if one goroutine could intervene in another to handle the panic since each goroutine has its own stack.

Unfortunately, the only way out in this case is crashing the application if we don’t handle the panic in that goroutine.

Defer arguments, including receiver are immediately evaluated

I've run into this problem before, where old data got pushed to the analytics system, and it was tough to figure out why.

Here’s what I mean:

func pushAnalytic(a int) {
  fmt.Println(a)
}

func main() {
  a := 10
  defer pushAnalytic(a)

  a = 20
}

What do you think the output will be? It's 10, not 20.

That's because when you use the defer statement, it grabs the values right then. This is called "capture by value." So, the value of a that gets sent to pushAnalytic is set to 10 when the defer is scheduled, even though a changes later.

There are two ways to fix this.

...

Full post is available here: Golang Defer: From Basic To Trap.

Vendoring, or go mod vendor: What Is It?

Phuong Le — Wed, 24 Jul 2024 17:21:49 +0000

Vendoring, or go mod vendor (source: victoriametrics.com/blog)

The original post is published on VictoriaMetrics: Vendoring, or go mod vendor: What Is It?

When you're dealing with modules in Go, the compiler usually fetches any needed modules from their online sources or repositories, and stashes them in a local cache. You'll probably find this cache at $GOPATH/pkg/mod.

But if you want to be sure, you can run go env GOMODCACHE to see the exact location.

$ go env GOMODCACHE
/Users/phuong/go/pkg/mod

This cache is just a place on your host machine where Go keeps copies of all the downloaded modules. So, when you build your project using go build, or test it with go test, Go uses these cached copies to find and load the packages it needs.

Now, vendoring is a different strategy as it keeps a copy of all your project's dependencies directly within the project's directory, rather than relying on an external cache.

Here are some examples of well known Go projects that use vendoring:

Kubernetes: You probably know it as the standard for container orchestration.
VictoriaMetrics: An incredibly fast and scalable monitoring solution and time series database.
Moby: Made by Docker to really push forward software containerization.
Slim (Previously DockerSlim): A handy tool for inspecting, slimming down, and debugging your containers.
Delve: A debugger for the Go programming language.

How to Vendor?

Our main player here is the go mod vendor command. It creates a special directory called ./vendor (by default) in your project's main folder.

Vendor Folder (source: victoriametrics.com/blog)

Good to know: your ./vendor directory doesn’t contain any test files (*_test.go), packages used by test files of dependencies, or the vendor folders of other dependencies.

Basically, it focuses on what’s strictly needed for your project's own code and its direct tests.

/my-project
|-- go.mod
|-- go.sum
|-- main.go
|-- vendor/ 
    |-- modules.txt
    |-- github.com/ 
    |   |-- project1/
    |   |   |-- module1/
    |   |   |   |-- file1.go
    |   |   |   |-- file2.go
    |   |   |-- module2/
    |   |       |-- file3.go
    |   |       |-- file4.go
    |-- golang.org/
        |-- x/
            |-- module3/
                |-- file5.go
                |-- file6.go

If you notice, it also generates a file called vendor/modules.txt.

This file lists all the packages copied into the vendor directory and specifies the versions of these modules.

This list acts as a record of what versions of each package were used, which is important for keeping things consistent and making sure the project builds the same way every time:

# cloud.google.com/go v0.112.1
## explicit; go 1.19
cloud.google.com/go/internal/detect
cloud.google.com/go/internal/optional
cloud.google.com/go/internal/pubsub
# cloud.google.com/go/compute v1.25.1
## explicit; go 1.19
cloud.google.com/go/compute/internal
# cloud.google.com/go/compute/metadata v0.2.3
## explicit; go 1.19
cloud.google.com/go/compute/metadata

When vendoring is enabled, vendor/modules.txt becomes a source of info about which versions of modules are being used. Commands like go list -m or go version -m, which report on module versions, rely on this file to provide accurate data.

"How about go.mod? There are two sources of truth?"

Exactly, and they have to be in sync.

When vendoring is enabled, go list -m still prints info about the modules listed in go.mod. It uses vendor/modules.txt to confirm that the versions in the vendor directory match what's declared in go.mod.

So, if there's a mismatch between the versions, Go commands will let you know:

go: inconsistent vendoring in /my-project:
        github.com/Azure/azure-sdk-for-go/sdk/internal@v1.9.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/Azure/azure-sdk-for-go/sdk/internal@v1.9.0: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod

        To ignore the vendor directory, use -mod=readonly or -mod=mod.
        To sync the vendor directory, run:
                go mod vendor

Or if a module is listed in go.mod but not in vendor/modules.txt:

go: inconsistent vendoring in /my-project:
        github.com/user/project@v1.9.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt

        To ignore the vendor directory, use -mod=readonly or -mod=mod.
        To sync the vendor directory, run:
                go mod vendor

In both cases, you can run go mod vendor to update the vendor directory and get it back in sync with your go.mod.

Some commands, like go build and go test, will use packages in the vendor directory. But some commands won't, such as:

go mod download still fetches modules from the internet to the module cache.
go mod tidy still downloads dependencies to the module cache and updates go.mod and go.sum. After that, you might need to run go mod vendor to update the vendor directory.

If you have a vendor directory in the root of your main module and the Go version in go.mod is 1.14 or higher, you don't need to do anything special to enable vendoring. Go will automatically recognize the vendor directory and use it for your project's dependencies.

If you want to disable vendoring and instead use the module cache or the network to fetch dependencies, you have a couple of options with the -mod flag:

-mod=readonly: This uses go.mod, but you can't update it. It'll report an error if something's wrong with go build, go generate, go run, etc., but it doesn't prevent go get or go mod.
-mod=mod: This switches back to the default behavior, ignoring the vendor directory entirely.
-mod=vendor: This forces Go to use the vendor directory, even if there's a go.mod file in the project.

If you don't provide a flag, it'll default to vendor if there's a vendor folder and the Go version is 1.14 or higher, as we discussed earlier.

How Does Vendor Work?

When you run the go mod vendor command, the RunVendor() function in the Go source code is the main engine that makes everything happen. You can check it out if you're curious, but let's break it down now.

Stage 1: Fetching

Vendor - Fetching Stage (source: victoriametrics.com/blog)

Go configures options to load all the packages needed to build and test the main module. It also resolves any missing imports and allows errors if the -e flag is set.

To find the actual source code for the packages, it uses the module cache first. But if the packages haven't been downloaded yet, it fetches them from the module proxy or the source repository.

Next, it figures out where to create the vendor directory.

By default, this is a directory named vendor in the root of your project. But if you use the -o flag, you can specify a different location. Go then clears out any existing content in this directory to start fresh.

That’s why we shouldn’t put or change anything in the vendor directory.

Stage 2: Populating

Vendor - Populating Stage (source: victoriametrics.com/blog)

Go then goes through all the loaded packages and groups them by their containing modules. It also keeps track of which modules are explicitly listed in your go.mod file and gathers info about the Go version directives for each module.

This info is important for creating a detailed modules.txt file later.

# github.com/VictoriaMetrics/easyproto v0.1.4
## explicit; go 1.18
github.com/VictoriaMetrics/easyproto
# github.com/VictoriaMetrics/fastcache v1.12.2
## explicit; go 1.13
github.com/VictoriaMetrics/fastcache
# github.com/VictoriaMetrics/metrics v1.34.1
## explicit; go 1.17
github.com/VictoriaMetrics/metrics

Before continuing, the function sorts the modules to process them in a consistent order, make sure our build is better reproducible.

Now, with the modules and packages organized and sorted, Go starts copying the source files for each package into the vendor directory.

It skips test files and files that are explicitly ignored, ensuring only the necessary files are included. As it copies the files, the function creates a modules.txt file.

So, here's the big picture of what we just discussed, it's also our thumbnail for this post.

Vendor Process (source: victoriametrics.com/blog)

Why Vendoring?

When you've got a ./vendor directory, it holds all the external modules your project uses, right under your nose. If you need to tweak one of these external modules (dependencies), you can do it directly in the vendor directory.

Also, this is definitely handy for CI/CD builds since you don't have to stress about internet availability or external dependencies.

The downside?

Your project doesn't share the same cache with other projects, so you might end up with multiple copies of the same module across different projects. It's a bit inefficient for your local machine.

"Does that mean I can change the code in the vendor directory?"

Yeah, absolutely. But it's not the good idea because your changes will get wiped out when you update the vendor, as we mentioned in How Does Vendor Work.

You can mess around, tweak, and test the code right there in the ./vendor directory until it works perfectly with your project. Once everything is working as expected, you can then send your changes to the original creators of the dependency.

"Is there any way I can make changes but not have them replaced? I still want to update the dependency but keep my changes."

Yes, there’s a simple workaround for that, and this solution does not apply to vendoring.

You can change the modules fetched from the remote repository and fetch the latest version without replacing your changes by using git submodules combined with the 'replace' directive in go.mod.

"So, to sum up, should I use vendoring?"

As an "experienced" writer, I'd say it depends.

If you don't want to worry about the network, don't need to go mod download every time you build on CI/CD, or you want to keep all the dependencies in one place and keep track of which dependency just updated, then vendoring is a good choice.

You can even change the vendor code, which I quite like since I use a lot of my own forked versions.

Of course, if you don't care much about really tiny details, want to reduce repository size, want to avoid duplication of dependencies in multiple projects, you can always rely on the module cache. The default behavior of Go is quite good.

Who We Are

If you want to monitor your services, track metrics, and see how everything performs, you might want to check out VictoriaMetrics. It's a fast, reliable, and cost-saving way to keep an eye on your infrastructure.

And we’re Gophers, enthusiasts who love researching, experimenting, and sharing knowledge about Go and its ecosystem.

6 Cache Strategies to Save Your Database's Performance

Phuong Le — Mon, 09 Oct 2023 18:49:51 +0000

So, you're wondering how to make your application faster and give your database a bit of a break? Using a cache with a suitable strategy is one way to do that.

6 Cache Strategies to Save Your Database's Performance (source: blog.devtrovert.com)

Author

I typically share insights on System Design & Go at Devtrovert. Feel free to check out my LinkedIn Phuong Le for the latest posts.

Let’s look at a social media example.

When a post goes viral, racking up millions of views and reactions, all those database reads and writes happen directly and can really slow things down. Multiply that by numerous viral posts, and you’re talking about billions of operations bogging down your database.

“What exactly does ‘cache population’ mean?”

Cache population strategies typically refer to how your application, database, and cache work together.

Now, it’s worth noting that many databases, like PostgreSQL with its Shared Buffer cache, or MongoDB with its WiredTiger internal cache and filesystem cache, do offer some built-in caching solutions.

Let’s break down these caching strategies into two main categories:

For reading data: cache-aside, read through, refresh ahead.
For writing: write through, write behind, write around.

1. Cache-aside (READ) or Lazy Loading

Also called lazy loading, Cache-aside is the most straightforward caching method you could consider. It works best for apps where most of the operations are reads, although it’s not exactly the top choice for scalability.

How it works?

The mechanics are straightforward:

If a user’s requested data is in the cache, it gets sent back right away.
If not, we fetch it from the database, stash it in the cache, and then return it to the user.

Cache-aside (READ) or Lazy Loading (source: blog.devtrovert.com)

Pros

Only needed data is cached: you only cache what people are asking for, so you know your cache is full of stuff users actually want.
Flexible customization, so you’re not limited to caching a single document but you can cache a bundle of multiple items for a specific query.
Database as backup, If anything goes wrong with the cache, your database is still there as a dependable backup.

Cons

Cache miss impact, a cache miss can slow down the request time, and there’s a risk of something known as a cache poisoning attack (I’ll explain later).
Additional code paths to manage: you’ll have to look after several things like pulling data from the cache, getting it from the database if it’s not in the cache, and then updating the cache, not to mention error handling.
Consistency can be tricky: determining if your cache is up-to-date or outdated can be a complex task.
Concurrent reads during cache update, there are moments when several reads might be hitting the database while you’re still filling the cache with new data

2. Read through (READ)

In this strategy, you let the cache library or service handle the data fetching and cache populating.

This way, your code only needs to call the repository (which is a mix of cache and database), without sweating the small stuff about how it all works.

With write operations, these go straight to the database. Whether or not to populate these changes in the cache is up to you. (I will talk about this when we come to the Write strategy)

How it works?

When your app asks the cache provider for a piece of data A, if it’s there, the service return it immediately.
If A isn’t in the cache yet, the service checks the database.
Once found, it saves A in the cache and hands it back to your app.

Read through (READ) (source: blog.devtrovert.com)

Pros

Simplified code handling, the cache service takes on the burden of managing both the database and the cache, so your app code stays lean.
Better consistency, since all read and write requests pass through the cache, it becomes simpler to keep your data uniform across both cache and database.
Prevents concurrent database reads: all data requests are routed through the cache, reducing the chance of simultaneous database reads with lock.

Cons

Need a learning curve: setting up this system from scratch has its own learning curve. It’s generally easier to rely on an established cache service or library.
Database specificity: if you’ve configured your cache to work with a particular database like MongoDB, ScyllaDB, then pivoting to a different database in the future could be a bit of a headache.
Less room for customization: Using a ready solution could mean you have to adapt to its rules.

For example, the duration data is stored (known as TTL) might not be flexible enough, some data might need to be stored for a month, while some for just a minute.

Limited data model flexibility, you might find yourself restricted to their data model, even when you’d prefer to use a different one for certain queries.

3. Refresh Ahead (READ)

What happens when you’ve got a trending post read by millions, and the cache expiration time for that post is fast approaching?

The Refresh-Ahead strategy aims to solve this problem by automatically renewing the cache’s time-to-live (TTL) for that particular post if it’s still being read close to its expiration time.

Before diving into the mechanics, it’s worth noting there’s a key variable in this strategy called the “Refresh-ahead factor”.

Say you set this factor at 0.5 and assign a 60-second TTL to item A. What happens is, the cache provider springs into action with an asynchronous refresh if item A is accessed when its TTL is already half spent.

How it works (using a Refresh-Ahead factor of 0.5)

The caching mechanism gives data item A a TTL of 60 seconds.
If someone accesses that item when the TTL is still at 31 seconds, no changes are made.
However, if that item is accessed when its TTL drops to 29 seconds, cache will still return that item, but also the cache provider asynchronously reaches out to the database for a TTL refresh.
Once completed, item A’s TTL is reset to a full 60 seconds.

Pros

Avoid cache miss delays, by refreshing popular items before they expire, you cut down on cache misses and the latency they can cause.

Cons

Complex refresh criteria, which items should get their TTL refreshed isn’t straightforward; it requires a thoughtful balance of your specific needs.
Unneeded caching of less popular items, so if an item isn’t truly trending but just receives occasional reads, the cache may still renew its TTL.

4. Write Through (WRITE)

Remember when I discussed how write operations work in the read-through scenario?

In Write-Through, data is first written directly into the cache and then immediately synchronized with the database.

How it works?

The application sends a write command to the cache.
The cache then updates the data based on that write command.
Finally, that updated data is sent over to the database.

Write Through (source: blog.devtrovert.com)

Pros

Consistency is key, Because the cache and database sync in a single operation, your data stays consistent.
Quick subsequent reads, If you’ve just written new data, the next time you read it will be fast because the cache has the freshest data available.
This strategy combines nicely with a read-through approach for caching.

Cons

Slower writes, since writing happens both in the cache and the database at the same time, it can slow down the write operation.
Not atomic, this becomes an issue when you’re updating a single resource with parallel writes.
Inefficient use of cache resources, the cache ends up storing data that’s written more often than it’s actually read, which isn’t super efficient.

5. Write Behind (WRITE) or Write Back

Just like in Write Through, the first step is to write data to the cache. However, we delay a bit when it comes to updating the database. Instead of syncing right away, we take an asynchronous approach and write to the database after a configurable delay.

So imagine, you have a post going viral and reactions are skyrocketing into the millions. Directly logging all those reactions into the database would be overwhelming, right? that’d put too much stress on it.

So we set a limit, let’s say, 1000 reactions before committing the data to the database.

“What happens if a post is just shy of the 1000-reaction mark? Like, it gets stuck at 999?”

Good question.

Our strategy has a workaround for that, we can choose to write the data based on either the number of reactions or after a certain period, say, a second has passed.

We could even use a combination of both, so whichever condition is met first triggers the write.

How it works?

The application sends a write operation to the cache.
The cache writes to the data on the cache, returning the new data immediately.
If the flush conditions are satisfied, the cache will write to the database asynchronously.

Write Behind (source: blog.devtrovert.com)

Pros

Optimized database load: by deferring database writes and using batch updates, you avoid overloading it, this is particularly useful for those viral posts with tons of reactions.
Fast response: since the cache is the first to get updated, your application will receive a quick response.
Flexibility: you can set different triggers for flushing; it could be based on a number count or a time limit.

Cons

Potential data loss: if the cache crashes before an asynchronous write to the database is completed, that data is gone.
Complexity in management: you’re adding another layer to your workflow by having to manage asynchronous writes, and perhaps even a fallback system.
Stale data risk: the database isn’t getting immediate updates, so if you bypass the cache, you risk encountering outdated data.
Requires careful tuning: you’ll have to watch and fine-tune your settings for flush conditions and time delays.
Inefficient use of cache resources like Write-through.

6. Write Around (WRITE)

Write Through and Write Behind have insufficient cache usage, since you’re dealing with data that isn’t frequently accessed but is updated regularly right?

In such a case, you might not want to flood your cache with this type of data, only to push out more frequently accessed items.

How it works?

The application performs a write operation, but it goes directly to the database.
The cache isn’t updated at this point.
When a read request comes in for that data, only then is it loaded into the cache

Write Around (source: blog.devtrovert.com)

Pros:

Efficient cache usage: the cache only stores data that is read frequently, avoiding unnecessary use of cache resources.
Suits infrequently accessed data: Ideal for data that is written often but not read as much.

Cons:

Delay in subsequent reads: the first read following a write operation will be slower since the data has to be pulled into the cache.
Complexity: you’ll still need to manage what gets into the cache and what doesn’t, making cache policies a bit complicated.
Un-optimizing write: since the write goes directly to the database, there’s less complexity in handling asynchronous writes or batching.
Risk of stale data: if you read from the cache right after a write, you may get old data unless the cache is refreshed.

Before you dive into crafting your own cache strategy, first ask yourself if you truly need the performance boost. Sometimes, adding a caching layer can complicate things without delivering any real benefits.

And don’t forget, your database might already have built-in caching features, so look into that as well.

People Addicted to Something Called 'Best Practice'

Phuong Le — Sun, 08 Oct 2023 11:49:45 +0000

Photo by Margarida Afonso on Unsplash

Author

I typically share insights on System Design & Go at Devtrovert. Feel free to check out my LinkedIn Phuong Le for the latest posts.

The following content delves into a controversial topic, please read with an open mind and be kind to different views. Let's talk about it nicely :)

Over the years in programming, I've often been guided by what many call "best practice". But with every project I tackled, a question kept coming up: is sticking to these best practices the only way?

These ‘best practice’ or ‘convention’ guidelines, recommended by experts or built from extensive research, seem to promise optimal results.

However, as I've navigated team dynamics and projects, I've realized that these practices, while valuable, are not always universally applicable.

Understanding Your Team

Teams, much like individuals, have their own identities: their strengths, weaknesses and rhythms.

Each team's distinct blend of personalities, skills, and experiences shapes its unique way of approaching problems and crafting solutions, what works marvelously for one team might be an obstacle for another.

Convention and Its Limitations

One fundamental aspect often overlooked when applying 'best practices' or 'conventions' is the context in which a team operates.

Consider the Golang programming community as an example.

Yes, Go has a widely accepted naming convention, which serves many developers well. But what if a different approach aligns better with your team's unique dynamics and experience?

A new developer walks in, fresh with Go conventions, and throws it into your team's face: “You guys are doing it all wrong! This isn't how Go's naming convention works!”, while they might be right by the book, but reality often tells a different story.

There’re various reasons:

The team might be working with legacy code that was written before certain Go conventions were established or widely adopted.
While deviating from the Go standards, their own convention make more sense to them and facilitate smoother collaboration.
If a Go project is closely integrated with systems written in other languages, it might make sense to adopt naming conventions more consistent with those systems to reduce cognitive load for developers switching between them.
For teams transitioning to Go from other languages, there might be an interim period where they use conventions familiar from their previous experiences to make the learning curve less steep.
The team might value flexibility and pragmatism over strict adherence to conventions. As long as the code is readable, maintainable, and efficient, they might prioritize getting things done over following every convention to the letter.
…

Conventions should adapt to teams, not the other way around, their real worth is gauged not by how popular they are, but by how effectively they serve a team's specific needs.

“But what if that convention is great for my team and I want to change it?”

The bad thing you will do is, change it without notifying EVERY member of the team even if that convention makes you feel like a world class coder for following the standard (I admit I used to think this way).

The solution is to discuss the proposed change with your team and get input from all members. Explain your rationale for wanting to modify the convention, but also listen sincerely to any concerns or counterarguments others may have.

"What if everybody agrees?"

Well, that’s great. But will you change the legacy code to adapt to the new convention, or just apply it to the new code?

The second option means temporarily keeping two co-existing conventions and this takes extra coordination and documentation on when each applies. More critically, it causes cognitive overhead for developers switching between the two conventions.

One more convention = one more confusion

…

“Can I adopt idiomatic or standard conventions for a new project/ codebase?“

Yes, that’s great too. Yet, consider answering over these questions first:

What exactly does your convention involve? Is it purely about coding aspects like naming, or does it also change models or HTTP Responses?
Does this new project have any connections or shared aspects with older ones?
Are there any systems that will engage with both projects and could be impacted by these changes?

If a system interacts with both projects, and they employ different conventions, it might sow seeds of confusion. In such cases, I'd advise caution and perhaps retaining consistency.

Similarly, the broader concept of 'best practices' arises.

The Nuances of ‘Best Practice’

Best practices are often hailed as tried-and-true methods that promise consistency and success. Yet, just as with conventions, they can't be applied universally without considering the team's unique dynamics.

Let's look at Scrum's sprint cycle as an example, a small team in a startup might need to change things often based on what their users say. So, a two-week sprint might not be the best for them.

But some people, after learning about Scrum, might say, “We have to do two-week sprints because that's the Agile way”, this misses the main idea of Agile, which is to be flexible and fit the team's needs.

Every team is different.

They have different sizes, types of work, challenges, and ways they talk to each other. Using Agile without thinking about these differences can cause problems.

Don't just be professional, be an essential part of the team.