Forem: Jacob Kim

Working with Time Data in Go

Jacob Kim — Tue, 07 Feb 2023 05:20:14 +0000

On many occasions, you will have to write code that has to deal with time. You could write a clock program or measure the time difference between two points in your code. Either way, it's important to know how to manipulate time data in Go. It's pretty simple, and knowing some nifty functions can help you save time. Working with time data in Go requires you to import the time package from the Go standard library. This package has a lot of methods and types that you can use, but I took the most used ones and described them here in this post. If you would like to see more info on the subject, check out the docs page for the time package.

Tell the time

This is probably the most commonly used method in the package. How do we tell the current time? Like this:

t := time.Now()
fmt.Println(t)

2023-02-06 23:01:48.9983151 -0500 EST m=+0.000030901

That's the current time. It's pretty cumbersome, so I will break this down into components.

2023-02-06: Year, month, day.
23:01:48.9983151: Hour, minutes, seconds
-0500: Time difference from GMT.
EST: The current time zone you are in.
m=+0.000030901: Monotonic clock reading.

We will get to monotonic clock later in this post. We can move on for now.

Is there a better way to format this?

You bet.

t := time.Now()
fmt.Println(t.Year())
fmt.Println(t.Month())
fmt.Println(t.Day())
fmt.Println(t.Date())
fmt.Println(t.Hour())
fmt.Println(t.Minute())
fmt.Println(t.Second())

2023
February
6
2023 February 6
23
26
6

Here's how you can extract each element of the time. Pretty straightforward, right?

How can we print this in a prettier format?

fmt.Printf("%d %d %d\n", t.Year(), t.Month(), t.Day())

2023 2 6

You can see how we can use the fmt.Printf function to format the time to our liking.

But what if we want to show months by their names, like February instead of 2? What if we want to show the time on a 12-hour scale instead of 24? You can see how it gets complex very quickly.

There is a better way to format time

Fortunately, we have the time.Format function to help us. Let's see how it works.

fmt.Println(t.Format("Mon Jan 2 15:04:05 2006 MST"))
fmt.Println(t.Format("Mon Jan 2 15:04:05"))
fmt.Println(t.Format("2006/01/02"))
fmt.Println(t.Format("3:04PM"))
fmt.Println(t.Format("15:04PM"))

Mon Feb 6 23:47:43 2023 EST
Mon Feb 6 23:47:43
2023/02/06
11:47PM
23:47PM

I think this is the part that tripped me up the most when I first started learning about this topic. And it's also the part where you can see how cheeky the language designers were.

We can see that time.Format accepts a string that denotes the format we want our time to be in. This is the weird part, because Go is very, extremely particular about the format string's format.

Mon Jan 2 15:04:05 2006 MST

The format string has to be a variation of this string, or else the code prints out weird times. Interestingly enough, if you exclude the Mon, each element of the format string represents an integer. Jan is 1, 2 is 2, 15 is 3, etc. Nerdy people, these developers.

1 2 3 4 5 6 -7

From the code above, though, you can see how we can format our time the way we want it to. And we don't have to write extra functions to convert hours into 12 or 24-hour scales, or to map each integer to a month.

You could also use a predefined format as well, like so:

fmt.Println(time.UnixDate)
fmt.Println(time.RFC3339)

Mon Jan _2 15:04:05 MST 2006
2006-01-02T15:04:05Z07:00

Check the docs for time package for more formats.

What about different time zones?

Time zones are automatically detected, as shown above. However, there may be cases where you need to show time in different time zones.

nt := time.Now()
lt := time.Now().Local()
ut := time.Now().UTC()
fmt.Println(nt)
fmt.Println(lt)
fmt.Println(ut)

2023-02-07 00:09:08.7052349 -0500 EST m=+0.000121601
2023-02-07 00:09:08.705235 -0500 EST
2023-02-07 05:09:08.705235 +0000 UTC

Local() gets the local time zone, which is the same as what time.Now() would automatically detect. Calling UTC() will convert the time zone to UTC.

But what if we need something more robust?

l, _ := time.LoadLocation("UTC")
fmt.Printf("%s\n", nt.In(l))

l, _ = time.LoadLocation("Europe/London")
fmt.Printf("%s\n", nt.In(l))

l, _ = time.LoadLocation("America/Los_Angeles")
fmt.Printf("%s\n", nt.In(l))

l, _ = time.LoadLocation("Asia/Seoul")
fmt.Printf("%s\n", nt.In(l))

2023-02-07 05:12:10.4423244 +0000 UTC
2023-02-07 05:12:10.4423244 +0000 GMT
2023-02-06 21:12:10.4423244 -0800 PST
2023-02-07 14:12:10.4423244 +0900 KST

time.LoadLocation will load a locale of your choice. You can use this result to convert your time by passing in to yourTime.In.

You can also read time from strings

In many cases, you will have to read in a string. Often, these will be timestamps. Wouldn't it be great to use the time library's functions on these timestamp strings?

By default, the time library works on time.Time types. However, we can parse these timestamp strings by using time.Parse.

timestamp := "2023-02-06 23:49:01"
ts, err := time.Parse("2006-01-02 15:04:05", timestamp)
if err != nil {
    fmt.Println(err)
}
fmt.Println(ts)

2023-02-06 23:49:01 +0000 UTC

You can also format ts using the Format method described above.

Wait, what is a monotonic clock?

Let's come back to this topic. It sounds a lot scarier than it actually is.

Your computer has a clock that measures time. Chances are that this time is rather volatile. Ever had the experience where your clock gets off by a couple of hours after traveling to another country? Ever had to set your clock again to match it with your watch? This clock is called the wall clock, and it is prone to change.

While the wall clock is good at telling the time, it's not good for measuring time. For instance, look at this piece of code.

t1 := time.Now()
fmt.Println(t1)
time.Sleep(time.Second())
t2 := time.Now()
fmt.Println(time.Now())
fmt.Println(t2.Sub(t1))

2023-02-06 23:23:32.2520709 -0500 EST m=+0.000031501
2023-02-06 23:23:33.2525152 -0500 EST m=+1.000475801
1.0004442s

The above code measures the time elapsed between t1 and t2. This seems obvious because we waited one second before declaring t2. But what if we somehow managed to switch our OS time settings in that span? If we add 4 hours to our system's clock, does that mean the time elapsed between t1 and t2 would be 4 hours and 1 second? That's ridiculous!

This is why Go uses a monotonic clock to measure time instead. You can see from the m values in the printed times that the time difference is about one second. Monotonic clocks allow for accurate measurement of time.

Conclusion

We are so familiar with the concept of time that we tend to take our understanding of it for granted. However, time tends to be one of the more frustrating things to represent in a computer. Fortunately, Go developers have abstracted away most of the raw conversions from us, so that we can use the simple-to-understand functions of the time package. This post covers a lot of the necessary functions, but if you need the nitty-gritty details, you can always refer to the official docs.

Thank you for reading! You can also view this post on Medium.

Structs, Methods, and Receivers in Go

Jacob Kim — Mon, 30 Jan 2023 14:03:11 +0000

When I was first learning how to use Go, the idea of Go being a non-OOP language was interesting. I was used to using Python, which was an OOP language, and thus had a class. I thought it was an odd design choice because I couldn't really imagine a world where I couldn't use a class. But after extensive programming in Go, I really enjoyed the Go way of writing code. Go does not have a class, but has structs. Structs can have methods attached to them, just like classes.

What are structs?

Structs are complex data types that hold other types. There are some things that we would like to represent as a group of attributes. For example, say that we have a laptop. A laptop isn't just one thing. Rather, it's a combination of many parts. So we could represent a laptop as a struct, like so.

type laptop struct {
    cpu string
    ram int
    storage int
    manufacturer string
}

Let's say that we want to represent a MacBook Air. We can do this:

mba := laptop{"M2", 16, 256, "Apple"}

This is cool because we can group multiple types into logically coherent objects.

How are structs different from classes?

People who have used another programming language, most likely Python or Javascript, may notice that Go does not have a class, only structs. Some might wonder how structs differ from classes. It turns out that they are actually pretty similar! For our purposes, at least.

Both are used to group together different data into a single logically coherent object.
Both have fields that can be private and public. Do keep in mind that there are no private and public keywords in Go. To make a field (or a struct itself) public, you just need to capitalize the first letter of its name. For example, Laptop instead of laptop and Cpu instead of cpu.
Both have access to a set of defined methods. Methods can be private and public as well, using the method (haha) above.

And honestly, that's all we need to know about. We could go into much more depth regarding the similarities and differences, but these three points drive the point home: for our intents and purposes, they basically accomplish the same thing.

What are methods?

Methods are like functions, but they are bound to a certain type or object, which are called receivers. I honestly don't know why they are called methods, and it's probably because engineers are bad at naming things.

Receivers do NOT have to be structs. They could be other types such as int, string, or even interface.

Let's write a method for our laptop struct defined above. Say that we want to upgrade our storage. We can write a method like this:

func (l laptop) upgradeStorage(size int) {
    l.storage += size
}

Seems pretty reasonable, right? It looks exactly the same as a normal function, but it has one key difference. Look at the part between func and upgradeStorage. The part (l laptop) is where we define this method's receiver. An object l that is of the type laptop can call upgradeStorage. We can access l's fields by using a dot. Here, we are editing its storage by accessing it through l.storage.

So what happens when we run this?

func main() {
    thinkpad := laptop{"i5-1240p", 16, 256, "Lenovo"}
    fmt.Println(thinkpad.storage)
    thinkpad.upgradeStorage(100)
    fmt.Println(thinkpag.storage)
}

256
256

Interesting. The storage didn't change at all! Why would this be the case?

The astute readers may have already noticed, but this has to do with the concept of pass-by-value and pass-by-reference. Pass-by-value means that the method is working on a separate copy of the receiver. Pass-by-reference means that the method is working on a reference to the receiver.

In this context, this means that when we call thinkpad.upgradeStorage(), Go is actually creating an identical copy of thinkpad, then applying changes to that copy instead of the original. Why does it do it? It's for safety reasons, mostly. If you didn't want to change a field value in a struct, but have the power to do so, there is a chance that you'd be overwriting an important piece of data. Go only supports pass-by-value by design. When you know you want to change the field value, you must explicitly state it. How? By using pointer receivers.

func (l *laptop) upgradeStorage(size int) {
    l.storage += size
}

Do you notice the change? The receiver is now of type *laptop, which is a pointer to the struct laptop. This way, the method can access the memory location of l and actually change the original object. We aren't playing around with copies anymore. Doing it this way has the added benefit of making the code run faster because Go doesn't have to spend time making an identical copy of l.

Now let's try running the code again.

256
356

Nice.

What about getters and setters?

Short answer: if you need it, do it.

For those of you who may be unfamiliar with the concept, it's common in many other languages to define a getter and a setter method.

A getter method will get the value of a field.
A setter method will set the value of a field.

For instance, if we were to write this in Go, we would do something like this:

func (l *laptop) getCpu() string {
    return l.cpu
}

func (l *laptop) setCpu(newCpu string) {
    l.cpu = newCpu
}

getCpu returns the cpu value of l, and setCpu updates l.cpu to whatever CPU we pass as newCpu.

Why do this? Well, people wanted to limit access to some fields, and encapsulate them so that people can only access the fields via getters and setters.

Go doesn't provide this out of the box, and if you wish to use one, you must write one yourself. There are many reasons why you would want an accessor like this, but always make sure that you aren't overengineering anything. Sometimes, just accessing the struct field is ok. There's a whole debate on this topic in the Go community, as well, which you can check out if you are curious.

One more thing: if you do choose to write a getter, don't include the word get. For example, instead of getCpu(), use Cpu().

Conclusion

Thank you for tuning in again this week! This is a simple topic, but I realized that I haven't touched on it specifically, and because Go is a rather nuanced language, some people may not be used to the Go way of doing things. Hopefully, this post cleared up some questions you had in mind.

You can also read this post on Medium.

College Coding Assignment Tips

Jacob Kim — Tue, 24 Jan 2023 01:56:48 +0000

I genuinely enjoy coding assignments. In my school, a lot of programming courses have projects that the students must complete. The projects can be rather intimidating to people who aren't experienced in writing long programs. There are also test cases that the project needs to take into account as well. I helped many people get better grades in their coding assignments and projects, and no, I didn't just give out my code. I thought a lot of people can benefit from these tips, so I decided to write a post on it. Enjoy!

Read the specifications carefully

Your assignment will have some sort of specification that describes what you are building. Make sure to read this carefully. If you received it in a PDF, try printing out first and then reading it. You can highlight some important parts in the spec, and physical paper tends to be much easier to read for me. Specs are your instructions, and you want to make sure that you understand them perfectly before you begin.

Draft first, code later

After reading the specs, you would have some idea of what is expected of you. It is very tempting for many students to jump into their IDEs or text editors to start coding. I want to emphasize one thing.

Don't.

If there's anything you take away from this post, it is this point. You shouldn't jump to coding immediately. Take out a piece of paper and draft your program. If you have an iPad or a digital note-taking device, create a new note and scribble down your plans first.

The reason why you want to do this is that coding isn't really about the code. It's rather an art of system design. A programming project is like an architectural project. You don't want to build a building without a blueprint.

Too many times I saw people mindlessly coding away, only to get stuck in the middle because something breaks. It's like trying to build a skyscraper with wood, only to realize later that there's a limit to how tall you can go.

Here are some things that I consider when planning out my code:

What data structures am I going to use? Which one would do the job the best?
How would function A work? B? C? What algorithm would it use?
What is the desired flow of data?
What input am I accepting and what output am I returning?
Do I wrap the variables and functions in a class? Or is it unnecessary?
How do I deal with memory allocation? Where do I create a new object in the heap?
Do I pass by value or by reference?

Trust me when I say this, you want to plan first before taking action. You will see a noticeable improvement in your development experience.

Write modular code

I mentioned above that coding is mostly the art of system design. I still think this is true, but that doesn't mean that you are excused from writing terrible code. You are only doing yourself a disservice when you write ugly, unreadable, and unmaintainable code.

Style is a personal choice, and I'm not the one to bash people for their taste. However, there are some practices that you should adhere to help you help yourself.

One of these is to write modular code. This means abstracting away as much code as possible. This also connects to the DRY (don't repeat yourself) principle. If you see yourself writing lots of repeated code, you may want to consider abstracting that away into a function that you can reuse later. This way, your code becomes much easier to read, and it becomes much easier to debug.

Make sure you know how to use a debugger

This will save you lots of time. I know a lot of people default to print debugging, and I am guilty of this too. If you are unfamiliar with it, print debugging is a technique where you would insert print statements in areas of the code where you think an error might occur, or would print out variables at certain points in the code. Print debugging is useful when you need to do some quick sanity checks, but they aren't very efficient. You could get away with it for most of your student career, but I'd strongly suggest you learn how to use a debugger. Here are some reasons why:

Your code gets really, really messy, really, really quickly.
If you have a code that outputs to the command line or a file, print debugging will mess your output up, making it time-consuming to read and check. Scrolling through the terminal isn't the best way to use your precious time.
If you want to check items inside a container via print debugging, you would need to loop over the container and print out each item. This is ugly.
Sometimes you need to check memory addresses and stack, and the debugger will do this for you.
Sometimes you just wish you could track a variable during its lifetime. A debugger helps you do this.

If you are using Visual Studio Code, I'd recommend you learn the built-in visual debugger. It's a very easy-to-use debugger that saves you so much time. I saw a noticeable improvement in my ability to fix bugs after learning how to use a debugger.

Explicitly list out possible vectors for errors

Also another debugging tip. When you are writing your code, you want to stop and think about what possible errors might occur. For example, if your code inserts items into an array, you want to acknowledge that a very possible error could be something to do with the index. If you are working with pointers, you always want to make sure that it is deleted, and make sure you aren't trying to double-free the memory in the heap or to use a dangling pointer. I will make a tutorial on pointers soon, so don't worry if you don't understand these just yet.

The point is, if you wait until the very end to check for bugs, you need to scroll through a couple of hundreds of lines to find where it could've occurred. If you list out possible error spots, then you have a much easier time identifying them.

This leads me to another great practice: testing each function individually. You want to make sure that each function works as expected on its own. I see many assignments which ask students to write test cases to test their code. However, these generally test the entire program rather than individual functions and methods. Write unit tests if possible! I know writing tests take a long time, but debugging takes longer in my experience. You can check out my post on testing your code in Go here!

Conclusion

This is a bit of a detour from my usual programming language tutorials, but I wanted to take the time to address some challenges incoming developers and CS students might face. I faced a lot of challenges as well, and I didn't have these resources. I've accrued specific knowledge from doing lots of coding and failing lots. This blog is meant to help people start their developer journey, and I think this post holds to that ideal. For the more experienced developers reading this, please share your two cents on tips and tricks!

Thank you for reading! You can also read this post on Medium.

Intro to Generics in Go

Jacob Kim — Sun, 15 Jan 2023 23:36:15 +0000

Generics is a topic that many people have opinions about. Some people say that the lack of generics is why they can't write in Go, while some say that the lack of generics makes the code simple and elegant. Whichever way you think, I think it is important that you know how to use it before having any opinions about it. It doesn't make a lot of sense to me to blindly follow some zealous users' opinions on the r/Golang subreddit, although I understand how fun it is to jump on the bandwagon. That said, let's learn about one of the newer, more substantial additions to the Go programming language.

What is generic programming?

Generic programming is a paradigm in which developers try to use a generic function that encompasses many different types. In a strongly typed language, it is common to see that a function enforces a specific parameter type. For example, take a look at the code below:

func addInts(a, b int) int {
    return a+b
}

addInts is a function that takes two int values a and b. If you try to pass something other than an int, your code will throw an error. We need to write another function to handle different types in this case.

func addInts(a, b int) int {
    return a+b
}

func addFloats(a, b float64) float64 {
    return a+b
}

addFloats is a function that takes two float64 values a and b. With these two functions, we can add integers and floats.

Although the above code works, it doesn't spark joy in many developers' hearts. Do we really need to write two different functions with the exact same logic just because the type is slightly different? It's more understandable if we want to handle completely different types such as an int and a string, but int and float64? Really? Surely there is a more elegant way to handle this and not repeat ourselves.

And this is where generic functions come into play.

How do we use it?

Let's create a generic version of the above functions such that we can add an int or a float64.
The function doesn't look very different but has some notable changes.

func genericAddNums[N int | float64](n1, n2 N) N {
    return n1+n2
}

Don't panic! I know it looks weird, and it most definitely looked weird to me at first too. But trust me, it all makes sense. Let's take this function apart.

genericAddNums is the name of the function.
The function takes n1 and n2 as parameters, and returns the sum of the two.

Now for the whacky part.

[N int | float64]

This portion of the function is called the "type parameter". What it does is that it creates an arbitrary type N that can either be an int or a float64 but not simultaneously. The point of a generic function is to write more elegant code by reducing rewrites, thus keeping the code DRY (aka don't repeat yourself). We do this by allowing our function to take different types, represented by N.

(n1, n2 N) N {...}

This means that our parameters n1 and n2 are of type N, and the function returns a value of type N.

Let's see how this works in the context of the whole code.

package main

import "fmt"

func genericAddNums[N int | float64](n1, n2 N) N {
    return n1+n2
}

func main() {
    fmt.Println(genericAddNums(1, 2))
    fmt.Println(genericAddNums(1.1, 2.2))
}

3
3.3

Pretty straightforward, right? We can see that genericAddNums can be used to add two int's or two float64's. But what if we tried to add an int and a float64?

fmt.Println(genericAddNums(1, 2.2))

default type float64 of 2.1 does not match inferred type int for N

We get the above error. When we pass 1 to the genericAddNums first, the code infers N to be an int. Even though N can be either an int or a float64, it cannot be both simultaneously. Since Go is a statically typed language, cross-type operations are not allowed. N is like a ? box in the Super Mario series. It can be any item, but if you roll a mushroom, it can't be any other item simultaneously.

A couple of tips

It can be tedious to write out individual types to cover. For example, let's say that I want to make a generic function that takes in all comparable types.

Quick disclaimer: in Go, there are certain types that are deemed "comparable". This means that they are allowed to be compared via comparison operators such as == or !=. You can read more about this in The Go Programming Language Specification.

TLDR, this is a list of comparable types:

booleans
numbers
strings
pointers
channels
arrays of comparable types
structs whose fields are all comparable types

How do we define this function? We could do something like this:

func thisIsBad[V bool | int | float32 | float64 | string](datum V) V {
    fmt.Println(datum)
    return datum
}

But this looks ridiculous. Fortunately, Go has a special built-in type called comparable that helps us deal with this mess. It's basically an interface that all comparable types implement. Read more about this here.

func thisIsGood[V comparable](datum V) V {
    fmt.Println(datum)
    return datum
}

Much cleaner, right? But sometimes, you might want a very specific subset of types that you want to accept, but don't want to write it out all the time. How do we do this? Well, remember how comparable is just an interface? It turns out that we can take a page from this book and make our own interface.

type MyTypes interface {
    int | float64 | string
}

func thisSparksJoy[V MyTypes](datum V) V {
    fmt.Println(datum)
    return datum
}

Instead of typing out [V int | float64 | string], we can just type [V MyTypes]. It's just a nifty way to keep things in manageable bits.

Conclusion

I hope this guide was easier to digest than the official Go documentation of generics. It's a pretty cool way to write software, and I'm currently trying to use it in my own projects too.

We learned about why we want to use generic functions, and how to write them in Go. We also learned how to type less and define our custom types. Generics is something that many Gophers love or hate. Lovers praise its flexibility and that it is easier for developers to use other languages to transition to Go. Haters worry that people will overuse generic functions in places where it's not needed, slowing down the program and making code harder to read. I tend to be a purist in many areas of life, and I love the idea of using pre-existing tools to do my job efficiently. I also love the Go philosophy of keeping code simple and easy to read, even at a cost of more typing and repetition. However, I think generics will provide new ways to do things. Developers are creative, and we will most definitely run into situations where using generic functions is the easiest way to solve a problem. All in all, I am in favor of generics, as long as I don't fall down the rabbit hole of viewing every problem as a nail to hammer in with generics. Say no to drugs ;)

Also, thank you all for waiting! I'm back in school now, and my initial plan was to keep writing during school. However, I bit off more than I could chew, and markedly failed at accomplishing this. I made some changes to the schedule so that I can spend some time writing every day, and this will hopefully help me produce more content. I don't want to let my readers down again, and I genuinely want to produce helpful content for newbies and refreshers for seasoned veterans. That being said, I should've updated my status, and I apologize to my readers for not doing so.

See you next time :)

You can also read this post on Medium.

I'm back!

Jacob Kim — Sun, 08 Jan 2023 19:18:44 +0000

Sorry for leaving without saying anything. I started university again, and was a bit overwhelmed with the amount of work I had to do. I thought I could continue writing while going to school, but I have grossly overestimated my abilities. As work started piling up, I found myself focusing more on schoolwork than writing.

After the term ended, I had a chance to reflect about the past four months of school. What are my priorities? What are my goals? Why did I even start writing? Hours turned into days, and days turned into weeks. I realized that I must keep writing, because this gives me a sense of purpose and mission. I want to become a positive influence in the developer community, no matter how big or small I get. I genuinely enjoy helping others out, and I have helped many of my university friends with homework and projects. It's a commitment for sure, but I am willing to set aside time to develop my personal brand and provide valuable help that can make someone's day. I felt a sense of void when I ghosted away from this platform and the developer community. I want to keep helping new developers learn, and I want to get feedback from more advanced developers so that I can improve my craft.

The blog has mostly been exclusive to the Go programming language, but I decided to open up the range of topics. I still love Go, and it is still my bread and butter language, but I think to help many new developers, writing about other topics such as C++ or Python would be very helpful as well.

If there are any topics that you'd want me to write about, please comment below!

Thank you so much for your patience.

Jacob

Mutexes in Go

Jacob Kim — Mon, 19 Sep 2022 15:36:03 +0000

Welcome back to Intro to Concurrency in Go! Today we will be looking at how to protect shared resources from being overwritten by other goroutines. If you have played Minecraft before, chances are that you couldn't build something because your friends took that one item in the chest before you could use it. Wouldn't it be nice to figure out a way to make sure that only one person can interact with the chest at a given time? Well, it turns out that gamers aren't the only ones who thought of this.

Mutexes

In Go, cases like these are called "data races." This happens when more than one goroutine tries to access the same data at the same time. As developers, we want to prevent data races as much as possible. If you go around the Go developer community, you may have heard the term "concurrency-safe" be thrown around. This basically means that there are no data races.

To prevent a data race, we can use something called a "mutex." A mutex acts like a switch that we can toggle on or off. When it's on, we are not allowed to touch the variable at hand. When it's off, we can do whatever we want to that variable.

A mutex usually lives inside a struct alongside the variable we would like to modify. So the variable becomes a bit clunkier but receives protection from other goroutines.

Example code

A code tends to speak a thousand words. Let's see how we can use one.

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    type chest struct {
        mu        sync.Mutex
        container []string
    }

    newChest := chest{container: []string{}}
    newChest.container = append(newChest.container, "Oak Plank", "Spruce Plank", "Birch Plank", "Jungle Plank")
    var wg sync.WaitGroup
    wg.Add(2)

    go func() {
        defer wg.Done()
        newChest.mu.Lock()
        newChest.container = newChest.container[0:2]
        newChest.mu.Unlock()
    }()

    go func() {
        defer wg.Done()
        newChest.mu.Lock()
        if len(newChest.container) == 4 {
            fmt.Println("we can make a crafting table!")
            time.Sleep(time.Second)
            fmt.Println("currently have: ", newChest.container)
            fmt.Println("making a crafting table...")
            fmt.Println("done!")
        }

        newChest.mu.Unlock()
    }()

    wg.Wait()

    fmt.Println(newChest.container)
}

we can make a crafting table!
currently have:  [Oak Plank Spruce Plank Birch Plank Jungle Plank]
making a crafting table...
done!
[Oak Plank Spruce Plank]

Here, we have two goroutines trying to access the chest. The first goroutine wants to take the plank, and the second wants to create a crafting table, which requires four planks. For this run, the second goroutine fired first, which made the first goroutine wait until the second goroutine access the chest first.

What happens if we take out the mutex?

func main() {
    type chest struct {
        mu        sync.Mutex
        container []string
    }

    newChest := chest{container: []string{}}
    newChest.container = append(newChest.container, "Oak Plank", "Spruce Plank", "Birch Plank", "Jungle Plank")
    var wg sync.WaitGroup
    wg.Add(2)

    go func() {
        defer wg.Done()
        // newChest.mu.Lock()
        newChest.container = newChest.container[0:2]
        // newChest.mu.Unlock()
    }()

    go func() {
        defer wg.Done()
        // newChest.mu.Lock()
        if len(newChest.container) == 4 {
            fmt.Println("we can make a crafting table!")
            time.Sleep(time.Second)
            fmt.Println("currently have: ", newChest.container)
            fmt.Println("making a crafting table...")
            fmt.Println("done!")
        }

        // newChest.mu.Unlock()
    }()

    wg.Wait()

    fmt.Println(newChest.container)
}

we can make a crafting table!
currently have:  [Oak Plank Spruce Plank]
making a crafting table...
done!
[Oak Plank Spruce Plank]

What happens here doesn't make sense. We let the first goroutine take the planks before we can even make a crafting table, but our code still said that we can make a crafting table. This is an unexpected behavior caused by data race.

Now you see why mutexes are important in designing concurrent programs. You don't know when what goroutine will modify a certain value, and you do not want any discrepancies happening due to data races. Imagine your user logging out before you can deliver a resource to him. You certainly want the resources delivery goroutine to lock the user authentication status until it's done, right?

There are different types of mutexes

sync.Mutex isn't the only mutex there is. Another frequently used mutex is called the sync.RWMutex. RWMutex, which is short for a read-write mutex, is a special type of mutex that allows many goroutines to read from a shared resource at the same time, but only allows one to write to it. This comes in handy when you are going to do a lot of read operations on your resources. A normal mutex will only have the ability to lock out every other goroutine aside from the one currently accessing it, making reading operations very slow. RWMutexes have a special locking method called RLock that allows simultaneous read operations. Use whatever floats your boat - I have seen both used widely. Usage is very similar to normal mutexes, so I didn't bother adding a code snippet just for RLock. Use it for functions that read from a resource, and make sure to RUnlock after you are done. For functions that write to a resource, use Lock and Unlock.

Conclusion

Thanks for reading! With this post, the Intro to Concurrency in Go series is finally over! We covered most topics on concurrency in Go. I will come back with a new topic next week, so stay tuned!

This post can be viewed on Medium and my personal site.

Listening to Multiple Channels in Go

Jacob Kim — Mon, 05 Sep 2022 22:55:08 +0000

Welcome back to the series! Today we will look at ways to listen to multiple channels at once. The guides before helped you get started with concurrency in Go. Although simple methods are often the best ones, you may have been stuck trying to implement more complex behaviors. After reading this guide, you will be able to make your concurrent code more flexible.

The select keyword

We can use the select keyword to listen to multiple goroutines at once.

package main

import (
    "fmt"
    "time"
)

func main() {
    c1 := make(chan string)
    c2 := make(chan string)

    go func() {
        time.Sleep(1 * time.Second)
        c1 <- time.Now().String()
    }()

    go func() {
        time.Sleep(2 * time.Second)
        c2 <- time.Now().String()
    }()

    for i := 0; i < 2; i++ {
        select {
        case res1 := <-c1:
            fmt.Println("from c1:", res1)
        case res2 := <-c2:
            fmt.Println("from c2:", res2)
        }
    }
}

from c1: 2022-09-04 14:30:39.4469184 -0400 EDT m=+1.000172801
from c2: 2022-09-04 14:30:40.4472699 -0400 EDT m=+2.000524401

The code above shows how the select keyword works.

We first create two channels c1 and c2 to listen to.
Then we spawn two goroutines that each sends the current time to c1 and c2.
Within the for loop, we create a select statement and define two cases: the first one for when we can receive from c1 and the second one for when we can receive from c2.

You can see that the select statement is very similar in design to the switch statement. Both define different cases and run the respective code when a certain case is met. Also, we can see that the select statement is blocking. That is, it will wait until one of the cases is met.

We iterate twice for the loop because there are only two goroutines to listen to. More exactly, each goroutine is a fire-and-forget goroutine, meaning that they only send to a channel once before returning. Therefore, there are two messages maximum at all times in this code, and we only need to select twice.

What if we don't know when jobs will end?

Sometimes we don't know how many jobs there are. In this case, put the select statement inside a while loop.

package main

import (
    "fmt"
    "math/rand"
    "time"
)

func main() {
    c1 := make(chan string)
    rand.Seed(time.Now().UnixNano())

    for i := 0; i < rand.Intn(10); i++ {
        go func() {
            time.Sleep(1 * time.Second)
            c1 <- time.Now().String()
        }()
    }

    for {
        select {
        case res1 := <-c1:
            fmt.Println("from c1:", res1)
        }
    }
}

Because we let a random number of goroutines run, we don't know how many jobs there are. Thankfully, the for loop at the bottom encasing the select statement will capture every output. Let's see what happens if we run this code.

from c1: 2022-09-04 14:48:47.5145341 -0400 EDT m=+1.000257801
from c1: 2022-09-04 14:48:47.5146126 -0400 EDT m=+1.000336201
from c1: 2022-09-04 14:48:47.5146364 -0400 EDT m=+1.000359901
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan receive]:
main.main()
        /home/jacob/blog/testing/listening-to-multiple-channels-in-go/main.go:22 +0x128
exit status 2

Hmm, the select statement received three times as expected, but the program errored out due to a deadlock. Why would this be the case?

Remember that without any condition, a for loop in Go will run forever. This means that the select statement will try to receive forever. However, there is a finite number of jobs to run. Even though there are no more jobs, the select statement will still try to receive.

Remember back in the first post of the series where I said that your program will run into a deadlock if you try to receive from an unbuffered channel when the sender is not ready? This is exactly the case in our example.

So how do we solve this? We can use a combination of the concepts covered in previous posts: exit channels and WaitGroups.

package main

import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)

func main() {
    c1 := make(chan string)
    exit := make(chan struct{})
    rand.Seed(time.Now().UnixNano())
    var wg sync.WaitGroup

    go func() {
        numJob := rand.Intn(10)
        fmt.Println("number of jobs:", numJob)
        for i := 0; i < numJob; i++ {
            wg.Add(1)
            go func() {
                defer wg.Done()
                time.Sleep(1 * time.Second)
                c1 <- time.Now().String()
            }()
        }
        wg.Wait()
        close(exit)
    }()

    for {
        select {
        case res1 := <-c1:
            fmt.Println("from c1:", res1)
        case <-exit:
            return
        }
    }
}

3
from c1: 2022-09-04 15:09:08.6936976 -0400 EDT m=+1.000287801
from c1: 2022-09-04 15:09:08.6937788 -0400 EDT m=+1.000369101
from c1: 2022-09-04 15:09:08.6937949 -0400 EDT m=+1.000385101

We create an exit channel and a WaitGroup.
The number of jobs is random for every run. For numJobs amount of times, we fire off goroutines. To wait for the jobs to finish, we add them to wg. When a job is done, we subtract one from wg.
Once all jobs are complete, we close the exit channel.
We wrap the above section in a goroutine because we want all of it to be non-blocking. If we were to not wrap it in a goroutine, the wg.Wait() will wait until the jobs are done. This will block the code and won't let the for-select statement at the bottom run.
Furthermore, because c1 is an unbuffered channel, waiting for all the goroutines to send the message to c1 will result in many messages being sent to c1 without the for-select statement to receive them. This results in a deadlock because the receiver is not ready when the sender is.

How to make select non-blocking

The select statement is blocking by default. How do we make this non-blocking? It's simple - we just add a default case.

package main

import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)

func main() {
    ashleyMsg := make(chan string)
    brianMsg := make(chan string)
    exit := make(chan struct{})
    rand.Seed(time.Now().UnixNano())
    var wg sync.WaitGroup

    go func() {
        numJob := rand.Intn(10)
        fmt.Println("number of jobs:", numJob)
        for i := 0; i < numJob; i++ {
            wg.Add(2)
            go func() {
                defer wg.Done()
                time.Sleep(time.Duration(rand.Intn(10)) * time.Millisecond)
                ashleyMsg <- "hi"
            }()
            go func() {
                defer wg.Done()
                time.Sleep(time.Duration(rand.Intn(10)) * time.Millisecond)
                brianMsg <- "what's up"
            }()
        }
        wg.Wait()
        close(exit)
    }()

    for {
        select {
        case res1 := <-ashleyMsg:
            fmt.Println("ashley:", res1)
        case res2 := <-brianMsg:
            fmt.Println("brian:", res2)
        case <-exit:
            fmt.Println("chat ended")
            return
        default:
            fmt.Println("...")
            time.Sleep(time.Millisecond)
        }
    }
}

...
number of jobs: 4
brian: what's up
...
ashley: hi
...
...
brian: what's up
ashley: hi
ashley: hi
brian: what's up
...
...
ashley: hi
...
brian: what's up
...
chat ended

Aside from the lame conversation, we can see how a default case works. Instead of waiting for chats to arrive, we can do something when there are no channels to receive from. In this example, we just printed out ellipses, but you could do anything you'd like.

Conclusion

That's it for this post! Now you can listen to multiple channels simultaneously, which can be a huge asset when you are developing your personal project. Thanks for reading, and I'll see you guys next time.

You can also read this post on Medium and my personal site.

WaitGroups in Go

Jacob Kim — Mon, 29 Aug 2022 14:07:00 +0000

Welcome back to Introduction to Concurrency in Go! Last time, we had our first taste of concurrency in Go. We examined what goroutines and channels are, the differences between buffered and unbuffered channels, and a simple way to wait for goroutines to finish. This time, we are going to talk about WaitGroups, which is a different way to sync your goroutines.

What are WaitGroups?

WaitGroups are an effective way to sync your goroutines. Remember how we used channels to wait for other goroutines to finish? WaitGroups can help you achieve the same goal. Imagine a car trip with your family. Your dad stops by a strip mall or a fast food restaurant to get some food and go to the bathroom. You ideally want to wait for everyone to come back before driving off to the horizon again. WaitGroups help you do this.

WaitGroups are defined by calling the sync package in the Go standard library.

var wg sync.WaitGroup

So what is a WaitGroup even? A WaitGroup is a struct that holds certain information about how many goroutines the program needs to wait on. It's a group that contains the number of goroutines that you need to wait on.

WaitGroups have three most important methods: Add, Done, and Wait.

Add adds to the total amount of goroutines you need to wait on.
Done subtracts one from the total amount of goroutines you need to wait on.
Wait blocks the code from continuing until there are no more goroutines to wait on.

How to use WaitGroups

Let's see an example code snippet.

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    var wg sync.WaitGroup
    wg.Add(1)

    go func() {
        defer wg.Done()

        fmt.Println(time.Now(), "start")
        time.Sleep(time.Second)
        fmt.Println(time.Now(), "done")
    }()

    wg.Wait()
    fmt.Println(time.Now(), "exiting...")
}

2022-08-21 17:01:54.184744229 +0900 KST m=+0.000021800 start
2022-08-21 17:01:55.184932851 +0900 KST m=+1.000210473 done
2022-08-21 17:01:55.18507731 +0900 KST m=+1.000354912 exiting...

We first initialize an instance of WaitGroup wg.
Then we add 1 to wg because we want to wait for one goroutine to finish.
We then run the goroutine. Inside the goroutine, we make a defer call to wg.Done() to make sure that we decrement the number of goroutines to wait on. If we don't do this, then the code will wait forever for the goroutine to finish and will result in a deadlock.
After the goroutine call, we make sure to block the code until the WaitGroup is empty. We do this by calling wg.Wait().

Why use WaitGroups over channels?

Now that we know how to use WaitGroups, a natural flow of thought leads us to this question: why use WaitGroups over channels?

Based on my experience, there are a few reasons.

WaitGroups tend to be more intuitive. When you read a snippet of code, when you see a WaitGroup, you immediately know what the code is doing. The method names are explicit and to the point. However, it's sometimes not that clear with channels. Using channels is clever, but it can be a hassle to understand when you are reading a complicated piece of code.
There are times when you don't need to use a channel. For example, let's look at this code:

  var wg sync.WaitGroup

  for i := 0; i < 5; i++ {
      wg.Add(1)
      go func() {
          defer wg.Done()

          fmt.Println(time.Now(), "start")
          time.Sleep(time.Second)
          fmt.Println(time.Now(), "done")
      }()
  }

  wg.Wait()
  fmt.Println(time.Now(), "exiting...")

You can see that the goroutine isn't communicating data to other goroutines. If your goroutines are one-off jobs where you don't need to know the results, using a WaitGroup is desirable. Now take a look at this code:

  ch := make(chan int)

  for i := 0; i < 5; i++ {
      go func() {
          randomInt := rand.Intn(10)
          ch <- randomInt
      }()
  }

  for i := 0; i < 5; i++ {
      fmt.Println(<-ch)
  }

Here, the goroutine is sending data back to the channel. In these cases, we don't need to use a WaitGroup because it would be redundant. Why wait for goroutines to finish if receiving already does enough blocking?

WaitGroups are specialized for dealing with waiting for goroutines. I feel like a channel's main purpose is to communicate data. You can't use a WaitGroup to send and receive data, but you can use a channel to sync your goroutines.

In the end, there is no right answer. I know that this can be annoying, but it is up to you and the team you are working with. Whatever works is best, and no answer is wrong. My personal preference is to use WaitGroups for syncing, but your mileage may vary. Pick something that is the most intuitive to you.

One thing to watch out for

Sometimes, you might need to pass the WaitGroup instance to the goroutine. There may be several WaitGroups that handle different goroutines, or maybe it's a design choice. Whatever the reason may be, make sure to pass the pointer to the WaitGroup, like so:

var wg sync.WaitGroup

for i := 0; i < 5; i++ {
    wg.Add(1)
    go func(wg *sync.WaitGroup) {
        defer wg.Done()

        fmt.Println(time.Now(), "start")
        time.Sleep(time.Second)
        fmt.Println(time.Now(), "done")
    }(&wg)
}

wg.Wait()
fmt.Println(time.Now(), "exiting...")

The reason is that Go is a pass-by-value language. What this means is that whenever you pass an argument to a function, Go will make a copy of the argument and pass that instead of the original object. What happens in this context is that the entire WaitGroup object will be copied, meaning that the goroutine would be dealing with a completely different WaitGroup. The wg.Done() won't subtract from the original wg, but a copy of it which only lives inside the goroutine.

Conclusion

Sorry for the later-than-usual upload. I was busy preparing for school, and had to do some initial setup after arriving in the States. I should've said something about it but I forgot to do so. I'll make sure to keep you guys updated from now on!

That being said, I hope this guide helped you design your concurrent code easily! WaitGroups are easy to understand and implement. Next time, we will talk about how to listen to multiple channels at the same time. See you next week!

You can also read this post on Medium and my personal site.

Intro to Concurrency in Go

Jacob Kim — Mon, 15 Aug 2022 06:54:00 +0000

Concurrency is a cool topic that can be a huge asset once you get the hang of it. To be honest, I was scared to write this post at first because I myself wasn't too comfortable with concurrency until recently. I got the basics down, so I wanted to help other beginners learn concurrency in Go. This is the first of many concurrency tutorials, so stay tuned for more!

What is concurrency and why does it matter?

Concurrency is the ability to run multiple things at the same time. Your computer has a CPU. A CPU has several threads. Each thread usually runs one program at a time. When we normally write code, that code runs sequentially, meaning that each job is run back to back. In concurrent code, those jobs are run simultaneously by the thread.

A good analogy is the one to a home cook. I still remember the first time I tried to cook pasta. I followed the recipe step-by-step. I chopped the vegetables, made the sauce, then cooked the spaghetti, then mixed the two. Here, every step was done sequentially, so the next job had to wait until the current job was done.

Fast forward to now, where I became more experienced at cooking pasta. I now start the spaghetti first, and then work on the sauce in the meantime. Cooking time was reduced to almost half because cooking the spaghetti and the sauce happened concurrently.

Concurrency vs. Parallelism

Concurrency is a bit different from parallelism. Parallelism is similar to concurrency in that multiple jobs are happening at once. However, in parallelism, multiple threads are working different jobs each, whereas, in concurrency, one thread is juggling between different jobs.

So concurrency and parallelism are two different concepts. A program can run both concurrently and parallel. Your code can be written either sequentially or concurrently. That code can be run on a single-core machine or a multi-core machine. Think of concurrency as a characteristic of your code, while parallelism as a characteristic of the execution.

Goroutines, the worker Mortys

Go makes it very simple to write concurrent code. Each concurrent job is represented by a goroutine. You can start a goroutine by using the go keyword before a function call. Ever watched Rick and Morty? Imagine your main function as a Rick who delegates tasks to goroutine Mortys.

Let's start with a sequential code.

package main

import (
    "fmt"
    "time"
)

func main() {
    simple()
}

func simple() {
    fmt.Println(time.Now(), "0")
    time.Sleep(time.Second)

    fmt.Println(time.Now(), "1")
    time.Sleep(time.Second)

    fmt.Println(time.Now(), "2")
    time.Sleep(time.Second)

    fmt.Println("done")
}

2022-08-14 16:22:46.782569233 +0900 KST m=+0.000033220 0
2022-08-14 16:22:47.782728963 +0900 KST m=+1.000193014 1
2022-08-14 16:22:48.782996361 +0900 KST m=+2.000460404 2
done

The code above prints out the current time along with a string. Each print statement takes one second to run. In total, the code took around three seconds to complete.

Now let's compare that to a concurrent code.

func main() {
    simpleConc()
}

func simpleConc() {
    for i := 0; i < 3; i++ {
        go func(index int) {
            fmt.Println(time.Now(), index)
        }(i)
    }

    time.Sleep(time.Second)
    fmt.Println("done")
}

2022-08-14 16:25:14.379416226 +0900 KST m=+0.000049175 2
2022-08-14 16:25:14.379446063 +0900 KST m=+0.000079012 0
2022-08-14 16:25:14.379450313 +0900 KST m=+0.000083272 1
done

The code above fires off three goroutines that each print the current time and i. The program waits for a second, and exits. This code took around one second to complete. That's about three times faster than the sequential version.

"Hold on," I hear you ask. "Why wait a whole second? Couldn't we remove that line to make the program run as fast as possible?" Good question! Let's see what happens.

func main() {
    simpleConcFail()
}

func simpleConcFail() {
    for i := 0; i < 3; i++ {
        go func(index int) {
            fmt.Println(time.Now(), index)
        }(i)
    }

    fmt.Println("done")
}

done

Hmm... The program did exit without any panics, but we are missing the output from the goroutines. Why were they skipped?

This is because by default, Go does not wait for goroutines to finish. Did you know that main is also run inside a goroutine? The main goroutine fires off the worker goroutines by calling simpleConcFail, but it exits before the workers can even finish their job.

Let's go back to the cooking analogy. Imagine you have three chefs each responsible for cooking the sauce, the spaghetti, and the meatball. Now, imagine if Gordon Ramsey orders the chefs to cook a plate of spaghetti & meatballs. The three chefs will work hard to cook the sauce, the spaghetti, and the meatballs. But before the chefs are even done, Gordon rings the bell and orders the waiter to serve the food. Obviously, the food isn't ready, and the customers will only get an empty plate.

This is why we wait for a second before exiting the program. We aren't always sure that every job will finish in one second. There is a better way to wait for jobs to be done, but we first need to learn another concept.

To summarize, we learned these things:

Jobs are delegated to goroutines.
Using concurrency can boost your performance.
The main goroutine does not wait for worker goroutines to finish by default.
We need a way to wait for each goroutine to finish.

Channels, the green portal

How do goroutines communicate with each other? Through channels, of course. Channels act like portals. You can send and receive data through channels. Here's how you make a channel in Go.

ch := make(chan int)

Every channel is strongly typed, and will only allow data of that type to pass through. Let's see how we can use this.

func main() {
    unbufferedCh()
}

func unbufferedCh() {
    ch := make(chan int)

    go func() {
        ch <- 1
    }()

    res := <-ch
    fmt.Println(res)
}

Simple, right? We make a channel named ch. We have a goroutine that sends 1 to ch, and we receive that data and save it to res.

Why do we need a goroutine here, you ask? Because not doing so will result in a deadlock.

func main() {
    unbufferedChFail()
}

func unbufferedChFail() {
    ch := make(chan int)
    ch <- 1
    res := <-ch
    fmt.Println(res)
}

fatal error: all goroutines are asleep - deadlock!

We run into a new word. What is a deadlock? A deadlock is where your program is stuck. Why would the above code get stuck in a deadlock?

To understand this, we need to know an important characteristic of channels. We created an unbuffered channel, meaning that nothing can be stored within it at a given time. This means that both the sender and the receiver must be ready simultaneously before data can be transferred across the channel.

In the failed example, the send and receive actions happen sequentially. We send 1 to ch, but at that time there is nobody to receive the data. The receiving happens at a later line, meaning that 1 cannot be sent until the receiving line is run. Sadly, 1 can't be sent first because ch is unbuffered and has no space to hold any data.

In the working example, the send and receive actions happen simultaneously. The main function fires off the goroutine and tries to receive from ch. At that time, the goroutine is sending 1 to ch. Therefore this code can run without deadlock.

One other way to receive from a channel without deadlocks is to close the channel first.

func main() {
    unbufferedCh()
}

func unbufferedCh() {
    ch2 := make(chan int)
    close(ch2)
    res2 := <-ch2
    fmt.Println(res2)
}

Closing the channel means that no more data can be sent to it. We can still receive it from the channel. For unbuffered channels, receiving from a closed channel will return a zero value of the channel's type.

To summarize, we learned these things:

Channels are the way goroutines communicate with each other.
You can send and receive data through channels.
Channels are strongly typed.
Unbuffered channels have no space to store data, so sending and receiving must happen simultaneously. Otherwise, your code will be stuck in a deadlock.
A closed channel will not accept any data.
Receiving from a closed unbuffered channel will return a zero value.

Wouldn't it be nice for channels to hold data for some time? Here's where buffered channels come to play.

Buffered channels, the portal that is somehow cylindrical?

Buffered channels are channels with buffers. Data can be stored in these, so sending and receiving don't have to happen simultaneously.

func main() {
    bufferedCh()
}

func bufferedCh() {
    ch := make(chan int, 1)
    ch <- 1
    res := <-ch
    fmt.Println(res)
}

Here, 1 is stored inside ch until we receive it.

Obviously, we can't send more to a full buffered channel. You need to have space in the buffer before you can send more.

func main() {
    bufferedChFail()
}

func bufferedChFail() {
    ch := make(chan int, 1)
    ch <- 1
    ch <- 2
    res := <-ch
    fmt.Println(res)
}

fatal error: all goroutines are asleep - deadlock!

You also cannot receive from an empty buffered channel.

func main() {
    bufferedChFail2()
}

func bufferedChFail2() {
    ch := make(chan int, 1)
    ch <- 1
    res := <-ch
    res2 := <-ch
    fmt.Println(res, res2)
}

fatal error: all goroutines are asleep - deadlock!

If a channel is full, the send operation will wait until there is space available. This is demonstrated here in this code.

func main() {
    bufferedCh2()
}

func bufferedCh2() {
    ch := make(chan int, 1)
    ch <- 1
    go func() {
        ch <- 2
    }()
    res := <-ch
    fmt.Println(res)
}

We receive once to take out the 1 so that the goroutine can send 2 to the channel. We didn't receive from ch twice, so only 1 will be received.

We can also receive from closed buffered channels. In this case, we can range over the closed channel to iterate over the remaining items inside it.

func main() {
    bufferedChRange()
}

func bufferedChRange() {
    ch := make(chan int, 3)
    ch <- 1
    ch <- 2
    ch <- 3
    close(ch)
    for res := range ch {
        fmt.Println(res)
    }
    // you could also do this
    // fmt.Println(<-ch)
    // fmt.Println(<-ch)
    // fmt.Println(<-ch)
}

1
2
3

Ranging over an open channel will never stop. This means that at some point, the channel will be empty and the range loop will try to receive from an empty channel, resulting in a deadlock.

To summarize,

Buffered channels are channels with space to hold items.
Sending and receiving does not have to happen at the same time, unlike unbuffered channels.
Sending to a full channel and receiving from an empty channel will result in a deadlock.
You can iterate over a closed channel to receive the remaining values inside the buffer.

Waiting for Godot... I mean, goroutines to finish, using channels

Channels can be used to sync goroutines. Remember how I told you that the sender and receiver must both be ready before you can transfer data through an unbuffered channel? This means that the receiver will wait until the sender is ready. We can say that receiving is blocking, meaning that the receiver will block the rest of the code from running until it has received something. Let's use this nifty trick to sync our goroutines.

func main() {
    basicSyncing()
}

func basicSyncing() {
    done := make(chan struct{})

    go func() {
        for i := 0; i < 5; i++ {
            fmt.Printf("%s worker %d start\n", fmt.Sprint(time.Now()), i)
            time.Sleep(time.Duration(rand.Intn(5)) * time.Second)
        }
        close(done)
    }()

    <-done
    fmt.Println("exiting...")
}

We make a done channel that is responsible for blocking the code until the goroutine is done. done can be of any type, but struct{} is used often for these types of channels. Its purpose is not to transfer structs, so its type doesn't matter.

The worker goroutine will close done once its job is done. At this point, we can receive from done, which will be an empty struct. The receiving action unblocks the code, allowing it to exit.

This is how we wait for goroutines to finish using channels.

Conclusion

Concurrency can seem like a daunting topic. I certainly believed that to be the case. However, after understanding the basics, I think the implementation is really beautiful. Hopefully, you guys can get something out of this tutorial! We've merely scratched the surface, and there is much more that Go has to offer us. I'll see you next time with more concurrency tutorials. Bye!

You can also read this post on Medium and my personal site.

Reading and Writing Different Files in Go

Jacob Kim — Mon, 08 Aug 2022 02:14:00 +0000

Reading and writing files is an important feature of your program. Not all data is stored in the same memory space as your program, and sometimes you will need to share data with other programs or view it later with a different program. Storing your data in a file is a good way to achieve these goals. Today, we will look at how you can read from and write to commonly used file types.

io.Reader, io.Writer

Have you ever wondered how Go can read and write so many different things? It's all thanks to these powerful interfaces. io.Reader describes anything that Read's, and io.Writer describes anything that Write's. Because this behavior can be replicated easily, anything that implements the io.Reader and io.Writer interfaces can be used for I/O operations. This means that you can plug and play different inputs and outputs. You could read from a CSV file and output it to JSON, or you could read from a stdin and write to CSV.

CSV

A CSV file consists of rows of data, where each value is separated by a comma (hence the name comma separated values). CSV isn't the fastest format to read and write out there, but it is very versatile and can be understood by many other tools.

Reading

package main

import (
    "encoding/csv"
    "fmt"
    "os"
)

func main() {
    f, err := os.Open("see-es-vee.csv")
    if err != nil {
        fmt.Println(err)
    }
    defer f.Close()

    r := csv.NewReader(f)

    records, err := r.ReadAll()
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(records)
}

[[id fruit color taste] [0 apple red sweet] [1 banana yellow sweet] [2 lemon yellow sour] [3 grapefruit red sour]]

Here is a very simple way to read data from a CSV file.

We first open our file see-es-vee.csv and save that instance as f.
We want to close f once we are done. This is a good way to save memory. Although this is a short program and closing isn't all that necessary, it is good to get in the habit of doing so.
f is of type *os.File, which implements the io.Reader interface. Therefore, we can pass f to csv.NewReader.
This returns a csv.Reader object r. csv.Reader is a type of io.Reader that specializes in reading CSV files.
Each line of a CSV file is called a record. So we can think of a CSV file as a slice of records. r.ReadAll returns this slice of records.
If we print records, we can see a 2D slice of rows.

This is great, but what if we want to apply some operations to each record as we read? Fortunately, we can take a more granular approach.

func main() {
    f, err := os.Open("see-es-vee.csv")
    if err != nil {
        fmt.Println(err)
    }
    defer f.Close()

    r := csv.NewReader(f)

    for {
        record, err := r.Read()
        if err != io.EOF {
            break
        }
        if err != nil {
            fmt.Println(err)
        }

        fmt.Println(record)
    }
}

Looks similar to the above, right? The part until creating r is the same. Let's see what happens next.

We enter an infinite for loop, because we want to read the file line by line, and Go has no idea how long the file is.
We read each record using r.Read.
How do we know if we reached the end of the file? r.Read returns a special error called io.EOF. EOF stands for end of file. We catch this error first and tell our program to break out of the for loop once we reach the end.
For any other errors, you can just handle them the way you would normally do.
After we handle errors, we can do whatever we want with the record that has been extracted. Some ideas that I can think of are capitalization, rounding, comparing it to an arbitrary value, etc.

Now let's look at how to write to a CSV file.

package main

import (
    "encoding/csv"
    "os"
)

func main() {
    f, err := os.Create("output.csv")
    if err != nil {
        panic(err)
    }
    defer f.Close()

    w := csv.NewWriter(f)

    records := [][]string{
        {"id", "fruit", "color", "taste"},
        {"0", "apple", "red", "sweet"},
        {"1", "banana", "yellow", "sweet"},
        {"2", "lemon", "yellow", "sour"},
        {"3", "grapefruit", "red", "sour"},
    }

    w.WriteAll(records)
}

id,fruit,color,taste
0,apple,red,sweet
1,banana,yellow,sweet
2,lemon,yellow,sour
3,grapefruit,red,sour

Very simple and straightforward.

We first need to create a file which we can dump our data into. We are going to call this output.csv. We just call os.Create, and save an instance of it as f.
Remember how we created a csv.Reader before? Here we are creating a csv.Writer object w. csv.NewWriter accepts the io.Writer interface, which f implements.
We prepare our data as a 2D slice of strings. We will call this records.
Finally, we just use w.WriteAll to write records to output.csv.

JSON

Because Go is used extensively in web services, it has robust support for JSON. I wrote an entire post about reading and writing JSON files, so check that out for a more in-depth guide!

Reading

package main

import (
    "encoding/json"
    "fmt"
    "os"
)

type fruit struct {
    Id    int    `json:"id"`
    Fruit string `json:"fruit"`
    Color string `json:"color"`
    Taste string `json:"taste"`
}

func main() {
    f, err := os.Open("jay-son.json")
    if err != nil {
        fmt.Println(err)
    }
    defer f.Close()

    dec := json.NewDecoder(f)

    // read opening bracket
    _, err = dec.Token()
    if err != nil {
        fmt.Println(err)
    }

    for dec.More() {
        var fr fruit
        err := dec.Decode(&fr)
        if err != nil {
            fmt.Println(err)
        }

        fmt.Println(fr)
    }

    // read closing bracket
    _, err = dec.Token()
    if err != nil {
        fmt.Println(err)
    }
}

{0 apple red sweet}
{1 banana yellow sweet}
{2 lemon yellow sour}
{3 grapefruit red sour}

// jay-son.json
[
    {"id": 0, "fruit": "apple", "color": "red", "taste": "sweet"},
    {"id": 1, "fruit": "banana", "color": "yellow", "taste": "sweet"},
    {"id": 2, "fruit": "lemon", "color": "yellow", "taste": "sour"},
    {"id": 3, "fruit": "grapefruit", "color": "red", "taste": "sour"}
]

Here's a simple way to read JSON files. Normally, JSONs come in streams. That is, they come in a list of objects. Go handles streams via a decoder.

Let's take a look at this first:

type fruit struct {
    Id    int    `json:"id"`
    Fruit string `json:"fruit"`
    Color string `json:"color"`
    Taste string `json:"taste"`
}

This struct acts as a model. Because JSON can come in various shapes and sizes, you ideally want to have a model that mirrors the JSON's structure. The struct fields need to be public and should have a tag to the right which denotes what key it refers to. If you don't know what the JSON is going to look like in advance, Go will just use a map[string]interface{}.

First, we open the file and save its instance to f. Don't forget to defer a call to close f later!
We create our decoder object dec using json.NewDecoder. Just like csv.NewReader, it accepts an io.Reader. You can start to see the power of interfaces - the details of reading are abstracted, and the workflow for reading is consistent across so many different file types.
Once dec is created, we can read our JSON. Just one problem, though. We need to make sure that we catch the opening and closing brackets using dec.Token. Not doing so is like trying to eat a subway sandwich with the wrapper still on. Bleh.
Just like how we read the CSV file line by line, we read the JSON stream object by object. We loop over dec.More, which returns true as long as there are objects still left to read.
We create an instance of fruit to store our object's data. Use dec.Decode to dump the object data to f. You can do what you want with this now.
After you are done reading, make sure to catch the closing bracket.

Writing

Writing to a JSON file is simple as well. We call this encoding.

package main

import (
    "encoding/json"
    "fmt"
    "os"
)

type Fruit struct {
    Id    int    `json:"id"`
    Fruit string `json:"fruit"`
    Color string `json:"color"`
    Taste string `json:"taste"`
}

func main() {
    f, err := os.Create("output.json")
    if err != nil {
        panic(err)
    }
    defer f.Close()

    enc := json.NewEncoder(f)

    apple := Fruit{Id: 0, Fruit: "apple", Color: "red", Taste: "sweet"}
    banana := Fruit{Id: 1, Fruit: "banana", Color: "yellow", Taste: "sweet"}
    lemon := Fruit{Id: 2, Fruit: "lemon", Color: "yellow", Taste: "sour"}
    grapefruit := Fruit{Id: 3, Fruit: "grapefruit", Color: "red", Taste: "sour"}

    fruits := []Fruit{apple, banana, lemon, grapefruit}

    err = enc.Encode(fruits)
    if err != nil {
        fmt.Println(err)
    }
}

[{"id":0,"fruit":"apple","color":"red","taste":"sweet"},{"id":1,"fruit":"banana","color":"yellow","taste":"sweet"},{"id":2,"fruit":"lemon","color":"yellow","taste":"sour"},{"id":3,"fruit":"grapefruit","color":"red","taste":"sour"}]

This one's also very straightforward.

We create a file named output.json to dump our data into.
We create a new encoder enc by using json.NewEncoder.
We prep our data, which is a slice of Fruit objects.
Finally, we encode this slice by using enc.Encode.

Excel

Go does not support reading and writing Excel files by default. However, there is a popular library called qax-os/excelize that helps you do this. If you take apart the source code, you can see that the package uses *os.File extensively, which is also an io.Reader and io.Writer. I think this shows the beauty of the io.Reader and io.Writer interfaces, because with a bit of tweaking, you can make a custom reader and writer that implements those interfaces, allowing you to support more file types.

Conclusion

Hopefully, this post served as a quick intro to reading and writing files in Go, and how powerful the io.Reader and io.Writer interfaces are. I think this is one of Go's beauty - interfaces allow for a very flexible and reusable code. There are certainly more files that we haven't covered in this post, but the general gist is the same: open the file, create a reader or a writer, and read/write from it. Thank you!

You can also read this post on Medium and my personal site.

Working in a Team

Jacob Kim — Mon, 01 Aug 2022 02:45:00 +0000

I spend a lot of time developing solo, working on my personal projects. However, I have worked in big project teams in the past, and have had some experience working with other developers and non-developers. Here are some tips that I would like to share with you. Please understand that this is a beginner’s viewpoint and can be subjective. If there are more tips that you would like to share, please comment below!

During my first year at university, I joined a Mars rover project team. Our team was tasked with creating a web-based simulator to test the rover’s autonomous navigation capabilities. I learned a lot of things and gained a lot from the experience. There were a couple of roadblocks that I have faced, and I think many new developers will face these issues as well.

Reading other people’s code

Reading other people’s code can be really confusing, especially in a large project. This was one of the areas that I struggled with the most. Heck, I can’t even read my own code the next day, how am I supposed to read others’ code?

First, get a good grasp of the tools used in the project.

Figure out what stack your team is using. The language, framework, tooling, anything. If you don’t understand the tools used, you will have a rough time figuring out what a piece of code is doing. If your team uses different languages for different parts of the code, you can usually pick one and choose to work on that part. For me, we were using Vue for the frontend and Python for the backend. I wasn’t very keen on learning frontend tools, so I decided to stick to the Python backend.

Second, make full use of your editor/IDE.

I use Visual Studio Code, and it comes with a plethora of tools to make my life easier. The peek feature was a lifesaver in many occasions, because it helped me figure out what a function does. This feature is also useful when you are working with a complicated struct or classes. You can also use a split view, where you can open two tabs side by side. This is useful when debugging a function that refers to another one.

Fireship.io has a very good video on VSCode features, and I highly recommend you watch it.

Third, reading code from top to bottom might not be the best way.

One of the biggest mistakes I made was to clone the repository, open any file, and start reading from top to bottom. As you may have expected, this left me even more confused. I wasted too much time reading unimportant code, and couldn’t get to the important ones in time.

The method that I found useful was reading the code alongside the overall system design. Find someone who understands how the system works and ask a lot of questions. Figure out how data is inputted. Understand how that input is processed, and then outputted. For each step, find the code responsible for it and read it.

A function usually calls other functions and creates instances of data structures defined in another file. These create rabbit holes that you need to follow and can get confusing quickly. I started to pin these on the Paint app. If function funcA() called funcB(), then I would screenshot funcA() and funcB(), paste those inside the Paint app, and draw an arrow pointing from funcA() to funcB(). It looks very similar to a UML diagram that describes how classes are laid out.

Fourth, you don’t need to understand everything from the get-go.

As an analytical thinker, I find it very uncomfortable to skip over information that I do not understand. I tend to try and understand everything before moving on. This process of fitting knowledge into my logical realm works when I can understand the topic, but it holds me back when topics get more complicated. Sometimes, moving on is the smarter option.

Instead, use the black box method. This term is derived from black box testing, a type of software testing that tests the code without having to look at its inner workings. The key point here is that the code itself is abstracted. We can take a page from this book. If you can only understand one part of the code, then consider the rest of the code as a black box for now. You can go back to this later when the time calls. Just keep a note to yourself and don't forget to ask about these other team members.

You don't always have to chase the fancy roles

A project consists of many different parts, some of them being cooler than the other ones. Some people in our team had a chance to work on the autonomous navigation code. Many people wanted to work on the algorithm because it was the meat of the project. However, I decided to stick with the simulator, writing backend code and libraries. I sometimes thought about chasing the clout. I thought that too many people were riding on the hype train.

It turned out that my work was crucial because it made everyone's life easier. Having a robust backend that communicated with the autonomous navigation code and the frontend was key to testing new features. Writing libraries allowed everyone in the team to quickly scaffold a prototype without having to re-implement every feature on their own.

Your team may be focused on delivering a novel, fancy feature, but you can still be a huge asset to the team even if you don't chase the hype. Find yourself a blue ocean and become a master of that sea.

Communication

You don’t need to worry a lot about communication when working solo, but it is crucial when working with a team. I was never really an eloquent salesman, but I knew how to form coherent thoughts. If you aren't good at speaking fast like me, you should take your time to collect your thoughts first. Let others talk and listen to what they have to say. Then you can say what you want, and people will tend to listen to you more because A: you have a very well-thought-out idea, and B: it's impolite to cut a shy speaker in the middle.

When there is a bug, not only should you point out the issue but also think of a fix. Anyone can say that module A is not working as expected, but not everyone understands why the bug happens and how one could fix it. Also, pointing out issues makes you sound rather pessimistic. No one wants to work with a pessimistic, cynical person, even if you are right.

Finally, always write documentation as you go. When I first joined the team, I didn’t have any documentation to refer to, so I had a rather slow start. Documenting code is tedious, but it is very important when it comes to kickstarting newbies like me. A well-documented code should highlight what the snippet of code does.

Conclusion

Thank you for reading the post. I hope that new developers can use this post to kickstart their team development journey. For seasoned veterans, I hope this post was a quick and fun read about a junior developer's experiences.

This post is also available on Medium and my personal site.

Journal: The Story of My Imposter Syndrome

Jacob Kim — Mon, 25 Jul 2022 10:33:55 +0000

Let’s take a slight detour from the usual programming guides. Today, I want to address the bane of all developers. The one that plagues juniors and seniors alike.

func (s *self) ImposterSyndrome() {
    for {
        doubt()
        fear()
        panic()
    }
}

s.ImposterSyndrome()

“Maybe I’m not smart enough to be a developer.”

“He’s so much better at this than I am.”

“How come she gets it and I don’t, when we started the same?”

“What if people realize that I am not well-versed in CS principles?”

“I only got this position because of luck.”

“People will realize all my successes were just sheer luck.”

“Why even start this project when there are other people who are way better than me already doing this?”

Imposter syndrome. I think this, along with burnout, is a major reason why developers get stuck in a rut. I had it too. The thoughts still haunt me today, but I got a lot better at dealing with them. I’d like to share with you how I struggled through my imposter syndrome, and how I overcame it for the most part.

My story

I am a university student, so I am always surrounded by smart people. I’ve heard people say that doing well in high school doesn’t guarantee your success in university, because you are one among many smart people who were the best performers in their high schools. At first, I was excited to start university. I got accepted to the CS department, and I was ready to butt my head against other talented and passionate students. Fast forward a couple of months, and I am starting to doubt myself. How did I even get into this program in the first place? I am slow in literally every way compared to my peers. They seem to be “getting” it as soon as they listen to the professor speak. Whenever I ask someone about the parts I’m struggling with, they would look at me as if I didn’t know how to add 1 to 1. Am I a fraud? I am a fraud, aren’t I? I shouldn’t have made it this far into the program…

I started to get a bit fearful, a bit resentful, and a bit bitter. Every time I went to a lecture, the professor would be speaking gibberish, and somehow my friend sitting next to me would nod and rub his chin as if he had connected with the professor on a deep level. I felt a sense of crushing defeat and guilt whenever anything happened in the class. Oh, she’s asking these high-level questions that I can’t even think of. Oh, he’s helping other people with their homework when I can’t even get my work done even if there were 48 hours in a day. Oh, he yawned in class - this material must be so boring for him. Oh, she didn’t come to lecture today - she probably knows everything anyways.

I always felt like a burden. I felt as if I was not meant to pursue a CS major. I felt like people would stare right through my thin-as-paper mask and into my pathetic, lying-ass self.

Why imposter syndrome is a thing

I feel like imposter syndrome affects many people, especially developers. Why would this be?

First, I think it has to do with how many things you need to know to be a developer. I’d say that being a developer requires you to learn a lot of things, from language to syntax to frameworks to design patterns to testing, and on. This is a lot to swallow for many people. Some people will fare better, and this drives an infinite loop of comparing oneself to others. It’s as if you are running self.Doubt() on a while loop that never ends.

Second, there are many different fields in the realm of programming. One could become a web developer, while others could pursue a goal of embedded systems developer. Someone could develop an Android app, while others could develop an iOS app. There are so many different areas that you simply cannot be an expert in all of them. Yet, people tend to try and “catch ‘em all.” It’s only natural - your brain doesn’t like to leave puzzles incomplete, and not learning a certain field makes you feel like you are missing out on a lot of subjects.

Third, jargon. This was one of my gripes about this field. Whenever I was in a library trying to finish that “Intro to C++” homework, I would get stuck and consult other people for help. They would spew out everything but English. It’s not very helpful to speak in devTalk when the other person is only fluent in English and Korean. What do you mean by “don’t use the namespace std because it’s bad practice?” What about “allocating a couple of bytes in the heap with the new() function to make it persist even when the function is removed from the call stack?” What the heck even is a “Hash map”? Do you mean the hash brown we had for breakfast today? Wait, hash has another meaning?

Fourth, the ever-expanding library of new technology. Every 60 seconds on Earth, a new Javascript framework is created. Sometimes, I feel as if technology is evolving faster than the universe is expanding. For many people, learning a new technology takes a bit. It took me a good while to get used to the C++ syntax, only to realize that some of the things that I learned are now considered bad practices and have been deprecated. So what to do instead? There are six different ways you can do to accomplish the same thing. Whenever I hear about new technology, I sometimes can’t help but roll my eyes instead of getting all excited. What’s even more ridiculous is that someone’s already claiming to be an expert in that field, and at least ten thousand people are making tutorials for millions to watch. I’ve only just heard of this framework!

Finally, recursion. Finally, recursion. I don’t understand how this works I don’t understand how this works I don’t understand how this works I don’t understand how this works I don’t understand how this works

Let’s try that again.

Segmentation fault (core dumped)

F**k.

What to do about it

Realize that we tend to not see the big picture. It’s like social media. We see these stories and pictures of them having a great time, but they never show you what happens behind the scene. We usually only see each other's highlights. But he’s probably struggling with his issues. You tend to feel a bit better if you realize that other people are struggling just as much as you are. As Sam Smith once sang, “I know I’m not the only one.”

And finally, git gud. I feel like this doesn’t get discussed very often because our society today encourages us to stay where we are.

“No, you are doing great! Keep it up!”

“Don’t let others get to you. You are perfect the way you are.”

These comments are like drugs. They feel soothing at first, but they will eat you from the inside out. These words make you complacent. Some people say this because they really mean it, but you know deep inside that you have a lot to work on. That's why you are struggling with feelings of inadequacy.

Be confident? Yeah right, there’s nothing that I am proud of, what am I supposed to be confident about? Be yourself? Yeah, I guess I will stay pathetic because this is who I am and I can't get any better.

Instead, focus on getting better. Prove to yourself that you aren’t a fluke. Grab a book, read the documentation, write some code, stumble upon a roadblock, tumble down, punch the ground, learn something new, and get better. Show yourself that you got what it takes. I will admit that the first step is tough. Your shadow will do whatever it takes to convince you that you aren’t worthy of taking your first step to change. But once you take that step, the rest will only become easier.

I like to think of imposter syndrome as a mask that really wants to cling onto you, sucking you dry of motivation. It's a parasite that feeds off your feelings of uncomfort. To keep doing that, it injects you with a dose of self-help and laziness. If you become too addicted to this substance, you will be okay with the way you are currently and refuse to take that mask off. This starts a vicious cycle of not improving and constantly feeling inadequate. Don't let this happen to you! I let it happen to me, and I passed on so many opportunities to improve.

Imposter syndrome is a tough boss to beat. It holds you like a vice, and it’s probably going to be very painful to get yourself out of its grip. But that pain is temporary. It can take you a month, or even years to escape, but you will escape. However, if you stay the way you are, you will rot away, trapped inside the face-hugger that is imposter syndrome.

Conclusion

Thank you for reading! I hope this post helped convince you to make the leap of faith. Let me tell you, it’s a very refreshing experience, to fight your demons. I’m not claiming to be perfect - I still have a lot ahead of me. But I am at least ready to take them on. I’m not perfect, but I know that I took the first step on the right path. I hope I can help you step your first foot into the dungeon as well. Imposter syndrome happens to all of us - it's up to us to decide what to do with it.

You can read this post on Medium and my personal site as well.