Leveraging the Layer-cake design in Go

Adele Reed — Fri, 17 May 2019 02:31:19 +0000

It's no secret that I love go's multi-threading support. It's a one-liner for a new thread:

go func(){}()

And a three liner (minimum) to talk to that thread:

messenger := make(chan int, 3)
go func(){ 
    for message := <-messenger {
        fmt.Println(<-messenger) 
    }
}()
messenger <- 5

Threads and channels are a match made in heaven that let you build a reaction-oriented program, divide-and-conquer recursion, and many, many more things I'm bound to eventually talk about here on dev.to.

But let's take a look at that first one. It caused me to build my own personal little design philosophy that I find myself taking into every project. It may not be the most accurate way to describe it, but the idea gets across in the name. Everything's a layer, and it's all caked together.

In my opinion, this methodology encourages me to build my applications, servers, etc. in a way that encourages an extensible, modular, and understandable style. As such, new services, interfaces, etc. can be tacked on without a huge amount of hassle of integrating them. That being said, I haven't followed it perfectly to tune but, hey, that's why I'm writing this article. I want to stay to exactly this structure in the future rather than the messy way I've implemented it the last few times.

Furthermore, since Go disallows recursive imports, this encouraged the creation of what I'd like to refer to as the "substrate". This is the core for communication within your application. Your substrate defines everything that can be (internally) commanded, your command format, your parameters, etc.

Initially, you need to iron a couple of lists of things down:

What interfaces can your application react to?
What services does your application provide?

Generally, my interfaces comes out as

CLI
RPC (not used in my clipboard manager)
TCP based protocol

Services will vary, but in the case of my clipboard manager it was

Monitor the clipboard
(soon) client-side encryption pipe

So let's enumerate those. There are a couple of primary enumeration structures in Go (to which I've defaulted most to the first, and after working with AzCopy I see why that may be faulty):

The type-const enumeration (What I'll typically use if there's only one enum in the package)

type Location uint8

const (
    CLI Location = iota
    TCP
    MONITOR
)

//Why might you want to use this?
/*
- Output as the const name in debug logs
- Easy to reference
*/

//Why might you not want to use this?
/*
- Multiple enums in one package is confusing
- May accidentally collide with function names, etc.
*/

-The type-func enumeration (what AzCopy uses)

type Location uint8
const ELocation = Location(0)

func (l Location) CommandLine() Location { return Location(1) }
func (l Location) TcpClient() Location { return Location(2) }
func (l Location) Monitor() Location { return Location(3) }

func (l location) String() string { return ... }

//Why might you want to use this?
/*
- Isolated by variable type, not by package
- Can't collide with other functions, enums, etc.
*/

//Why might you not want to use this?
/*
- Requires a Location variable to actually get the enumeration values
- No easy way to print a string of it for debug purposes
- Enumerated by functions, not by any concrete value
*/

-The double-map enumeration

var ELocation = map[string]uint8{...}
var LocationToString = map[uint8]string{...}

//Why might you want to use this?
/*
- Isolated by variable type, not by package
- Can't collide with other functions, enums, etc.
*/

//Why might you not want to use this?
/*
- Not immutable
- Requires more memory
- Not forced to a single type
- No easy way to print a string of it for debug purposes
*/

I'm not a huge fan of the lack of an enum structure in go, but, you make do with what you have. All of these have their individual problems. All of them can be mixed & matched to create the "perfect" enum implementation.

At this point, it's time to build your substrate. Your command structure is almost always the most important thing. You can always add to it, but it gets harder and harder to remove from it as time goes on. To keep it simple, I generally include a command ID, a list of strings as parameters, and the origin location.

type Command struct { //Could have multiple command structures for each layer
    CommandID  uint8 //Could be a enum
    Parameters []string
    Origin     Location //This can be a string or an enumeration. Your choice.
}

So you've got your command structure and now it's time to set up your channels. Set up one channel for each of your layers.

var (
    CLIChannel = make(chan Command, 50) //Reasonable buffer size.
    ClientChannel
    MonitorChannel
)

Now, your substrate is ready. I typically put this under the "common" package (or a package literally named "substrate"). It's time to create your layers. Make a package for each of your layers, and create a baseline goroutine for them. Maybe they'll spawn their own in the future. Let's take a sample TCP Client layer for me.

package client

type extMessage struct {
    CommandID  uint8
    Parameters []string
}

func TcpClient() {
    ExternalMessages := make(chan extMessage, 50)

    go func(){
        //TCP listener in here, pipes to ExternalMessages
    }()

    for {
        select {
        case extMsg := <- ExternalMessages:
             //Handle external message
        case intMsg := <- common.TcpChannel:
             //Handle internal message
        }
    }
}

After you've got your skeleton going, plop those suckers down in main.go because you're good to go.

package main

import (
    "sync"
)

func main() {
    var wg sync.WaitGroup
    wg.Add(3)

    go client.TcpClient()
    go commandline.CLI()
    go monitor.Monitor()

    wg.Wait()
}

You can of course, kill the program in different ways. Perhaps have a suicide command for each of your layers and have them call back to the wait group with wg.Done() to terminate the program safely. If the currently active data doesn't matter, a os.Exit(0) will do you just fine.

The result of all of this work is that you now have a nice layer-cake and you can understand:

Where every command comes from
Who interacts with what and how

Atop this, you reap the benefits of being modular, maintainable, and multi-threaded.

Forem: Adele Reed

Leveraging the Layer-cake design in Go