Forem: Julian-Chu

Go 1.19 beta1

Julian-Chu — Thu, 16 Jun 2022 18:46:11 +0000

Compared to 1.18, 1.19 is not a big release, but something still catches my eye.

Memory Model

The Go memory model has been revised to align Go with the memory model used by C, C++, Java, JavaScript, Rust, and Swift. Go only provides sequentially consistent atomics, not any of the more relaxed forms found in other languages

Doc Comments

support for links, lists, and clearer headings in doc comments

Go runtime

file descriptor

automatically increase the open file limit (RLIMIT_NOFILE) to the maximum allowed value

new env: GOMEMLIMIT

The runtime now includes support for a soft memory limit. This memory limit includes the Go heap and all other memory managed by the runtime, and excludes external memory sources such as mappings of the binary itself, memory managed in other languages, and memory held by the operating system on behalf of the Go program

it's possible to avoid/reduce OOM kill of golang app by env. Looking forward to reading the GC guide

self-adaptive initial stack size( before is fixed 2KB):

allocate initial goroutine stacks based on the historic average stack usage of goroutines

Compiler

jump table for large integer and string switch statements. Performance improvements for the switch statement vary but can be on the order of 20% faster. (GOARCH=amd64 and GOARCH=arm64 only)

modern CPUs do branch prediction for performance, but instead when the prediction fails too often, the performance becomes worse, introduce branchless(jump table) to avoid branch prediction to get better performance.

IMHO go memory model and the GC guide worth to spend some time to read.

eBPF learning material(updating)

Julian-Chu — Mon, 13 Jun 2022 22:16:55 +0000

preparing for next learning...

Background and design

A thorough introduction to eBPF
An introduction to the BPF Compiler Collection
Early packet drop — and more — with BPF
XDP

Good to start

ebpf-beginners

eBPF tool

c

https://github.com/iovisor/bcc

go

libbpfgo
ebpf

Book

BPF Performance Tools

website

https://www.brendangregg.com/ebpf.html

supported eBPF program

What kind of eBPF program can be triggered by event, depends on the kernel version.

can be found in the bpf.h
https://elixir.bootlin.com/linux/v5.18/source/include/uapi/linux/bpf.h#L922

or use bpftool to find them

//todo

eBPF command

https://elixir.bootlin.com/linux/v5.18/source/include/uapi/linux/bpf.h#L840

[Go] unusual: move "if err !=nil" into struct methods

Julian-Chu — Sat, 09 Jan 2021 18:52:50 +0000

"if err != nil" is a pain in golang error handling, you may need to write the following code many times.

if err != nil{
     return err
}

Common case

For example, if we'd to execute some actions, the error check comes after each function.

func DoActions() error {
    err := Action1()
    if err != nil {
        return err
    }
    err = Action2()
    if err != nil {
        return err
    }
    err = Action3()
    if err != nil {
        return nil
    }
    return nil
}

With error handling function"

The idea to handle error explicitly is good, but it also makes the code longer and noises in readability. One way very usually used to reduce lines of code and keep readability is to have a error handle function in some cases.

func DoActions()  {
    err := Action1()
    handleError(err)
    err = Action2()
    handleError(err)
    err = Action3()
    handleError(err)
}

func handleError(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

Error check in bufio.Scanner

Here is another way to reduce code of error check by method. The idea is to use error field inside struct for methods. One example you can find in the bufio.Scanner

package main

import (
    "bufio"
    "fmt"
    "os"
)

func main() {
    scanner := bufio.NewScanner(os.Stdin)
    for scanner.Scan() {
        fmt.Println(scanner.Text())
    }
    if err := scanner.Err(); err != nil {
        fmt.Fprintln(os.Stderr, "reading standard input:", err)
    }
}

The Scan method doesn't return any error, the internal error of struct is checked after for loop. If going to the Scan() method, we can see the Scan() only set internal error and use internal error as flag.

Rewrite first example with struct error field

let's take the struct error field idea to refactor first example.

func DoActions() error {
    ag:=&ActionsGroup{}
    ag.Action1()
    ag.Action2()
    ag.Action3()
    if ag.Err() != nil{
        return ag.Err()
    }
    return nil
}

type ActionsGroup struct {
    err error
}

func (ag ActionsGroup) Err() error {
    return ag.err
}

func (ag ActionsGroup) Action1() {
    if ag.Err != nil {
        return
    }
    // if something wrong, set ag.err
}

func (ag ActionsGroup) Action2() {
    if ag.Err != nil {
        return
    }
    // if something wrong, set ag.err
}

func (ag ActionsGroup) Action3() {
    if ag.Err() != nil {
        return
    }
    // if something wrong, set ag.err
}

Obviously it doesn't reduce the lines of error check, instead, just encapsulate error check into each method, and make code block for high level business logic clearer and less code for error handling.
No silver bullet. This can't be applied to all cases, but if you need error check between methods of one struct or can group functions with high cohesion into struct, it could be an alternative way for you.

[Go] Shutdown services in Go - single service

Julian-Chu — Thu, 11 Jun 2020 15:18:42 +0000

Shut down gracefully is always an important topic. Many things should be considered like how to release resource, how to close active connection properly and so on.

let's step by step go from a basic go http service shutdown to shutdown multiple go services gracefully

Start a http server

func main() {
    srv := &http.Server{
        Addr: ":8080",
    }
    srv.ListenAndServe()
}

The code just starts a http server, but problem is that when an error happens, we get no error message, it's terrible for debug and troubleshooting.

if we look at the ListenAndServe(), we could find the method returns server error.

func (srv *Server) ListenAndServe() error

so we can do something to add log for server error.

Print err for http server

func main() {
    srv := &http.Server{
        Addr: ":8080",
    }
    if err := srv.ListenAndServe(); err != nil {
        log.Fatalf("server err: %s", err)
    }
}

Looks better, now we have error message for debug and troubleshooting.

Signal from OS

Life is not easy, generally we want go application also handle signal from system, like ctrl-c or kill process command
go signal example

And after we receive signal from system, what can we do to shut down the http server?
Go to http package, there's a method call Shutdown: link

// Shutdown gracefully shuts down the server without interrupting any
// active connections. Shutdown works by first closing all open
// listeners, then closing all idle connections, and then waiting
// indefinitely for connections to return to idle and then shut down.
// If the provided context expires before the shutdown is complete,
// Shutdown returns the context's error, otherwise it returns any
// error returned from closing the Server's underlying Listener(s).
//
// When Shutdown is called, Serve, ListenAndServe, and
// ListenAndServeTLS immediately return ErrServerClosed. Make sure the
// program doesn't exit and waits instead for Shutdown to return.
//
// Shutdown does not attempt to close nor wait for hijacked
// connections such as WebSockets. The caller of Shutdown should
// separately notify such long-lived connections of shutdown and wait
// for them to close, if desired. See RegisterOnShutdown for a way to
// register shutdown notification functions.
//
// Once Shutdown has been called on a server, it may not be reused;
// future calls to methods such as Serve will return ErrServerClosed.
func (srv *Server) Shutdown(ctx context.Context) error {

points to pay attention to:

this method can close http server gracefully
context can be used to control the shutdown timeout
after calling this method, ListenAndServe will return ErrServerClosed, but it's not kind of error we could like to catch from server

here's implementation combined with os.signal and shutdown method in http package

func main() {
    shutdown := make(chan os.Signal, 1)
    signal.Notify(shutdown, syscall.SIGINT, syscall.SIGTERM)
    srv := &http.Server{
        Addr: ":8080",
    }
    go func() {
        if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatalf("Server Err: %s", err)
        }
    }()

    <-shutdown

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    if err := srv.Shutdown(ctx); err != nil {
        log.Fatalf("Server Shutdown Failed:%+v", err)
    }
    log.Print("Shutdown properly")
}

hints:

any os.signal you want to handle can be added in Singal.Notify(channel, signals to handle.....) like variadic args, if no assigned value, it will handle all signals.
http server should run on its goroutine, and not catch the http.ErrServerClosed

[Go] testing tips part 2: write flexible mock object for test cases without library

Julian-Chu — Mon, 30 Mar 2020 20:23:09 +0000

In last previous article, we refactored our simple AuthService with dependency injection, and added first unit test with mock object.

The code, we write so far.

Our business logic

type AuthService struct {
    pwd IPassword
}

func NewAuthService(IPassword IPassword) *AuthService {
    return &AuthService{pwd: IPassword}
}

func (s AuthService) Login(username, pwd string) (bool, error) {
    pwdIsValid, err := s.pwd.verify(username, pwd)
    if err != nil {
        return false, err
    }
    if !pwdIsValid {
        return false, nil
    }
    return true, nil
}

type IPassword interface {
    verify(string, string) (bool, error)
}

Our test code

func TestAuthService_LoginSucceed(t *testing.T) {
    s := NewAuthService(&mockPasswordService{})
    ok, err := s.Login("testuser", "testpwd")
    if err != nil {
        t.Error(err)
    }

    if !ok {
        t.Error("login failed")
    }
}

type mockPasswordService struct {
}

func (m mockPasswordService) verify(username, password string) (bool, error) {
    return true, nil
}

It looks like everything is fine, so now we are gonna to add a new test case for LoginFailed.

func TestAuthService_LoginFailed(t *testing.T) {
    s := NewAuthService(&mockPasswordService{})
    ok, err := s.Login("testuser", "wrongPwd")
    if err != nil {
        t.Error(err)
    }

    if ok {
        t.Error("login succeed!")
    }
}

When we execute the tests, the new test case is failed because of the hardcoded mockPasswordService. Maybe you consider we could add another mock like mockPasswordService2 in the following code:

func TestAuthService_LoginFailed(t *testing.T) {
    s := NewAuthService(&mockPasswordService2{})
    ok, err := s.Login("testuser", "wrongPwd")
    if err != nil {
        t.Error(err)
    }

    if ok {
        t.Error("login succeed!")
    }
}

type mockPasswordService2 struct {
}

func (m mockPasswordService2) verify(string, string) (bool, error) {
    return false, nil
}

Of course, it works well, but the code looks like duplicates, and not flexible for different scenarios. For example, when a new test case asks verify function to return an error, we have to add something similar again, then leads to more duplicates.

How can we write flexible mock objects? Let's try to do it without library, then do it again with library.

Doing without library

Remember what we want to mock? The return of verify function,
so here we will use one very nice feature in Go, "functions are first class".

First of all, we add a public func member, which has the same return type as verify function, in mockPasswordService struct, and second, we call the func member in verify funcion. Finally we assign a function to this member in test case.

// Before
type mockPasswordService struct {
}

func (m mockPasswordService) verify(username, password string) (bool, error) {
    return true, nil
}

//After
type mockPasswordService struct {
    VerifyFunc func() (bool,error)
}

func newMockPasswordService() *mockPasswordService {
    return &mockPasswordService{}
}

func (m mockPasswordService) verify(username, password string) (bool, error) {
    return m.VerifyFunc()
}

Of course you can use private member, then inject the VerifyFunc from constructor.

The benefit is that we can change the behavior of mock object in our test case for different scenarios.

func TestAuthService_LoginSucceed(t *testing.T) {
    mockPasswordService := newMockPasswordService()
    mockPasswordService.VerifyFunc = func() (bool, error) {
        return true, nil
    }
    s := NewAuthService(mockPasswordService)
    ok, err := s.Login("testuser", "testpwd")
    if err != nil {
        t.Error(err)
    }

    if !ok {
        t.Error("login failed")
    }
}

func TestAuthService_LoginFailed(t *testing.T) {
    mockPasswordService := newMockPasswordService()
    mockPasswordService.VerifyFunc = func() (bool, error) {
        return false, nil
    }
    s := NewAuthService(mockPasswordService)
    ok, err := s.Login("testuser", "wrongPwd")
    if err != nil {
        t.Error(err)
    }

    if ok {
        t.Error("login succeed!")
    }
}

If we are gonna to add 3rd test case like LoginFailedWithError, we don't have to touch the mock type anymore, and easy to assign the behavior of mock object in the test case.

Conclusion

We can use "function as first class" to make flexible mock objects, it works much better than hard-coded mock object. But for complex scenarios or not so small project, using the mock library is the best choice to reduce our work. We will do it in the next article.

[k8s] Basic Resource Management

Julian-Chu — Wed, 19 Feb 2020 22:32:19 +0000

kubernetes provides 3 objects to manage the resource in container, pod and namespace level

Resource management in container by pod spec

In pod spec, we can add the fields "resource:requests" and "resource:limits" for CPU and memory in container level.

CPU time unit, m means Milli, 500m = 0.5 CPU time
memory unit, Mi means MB

apiVersion: v1                               
kind: Pod                                    
metadata:                                    
  creationTimestamp: null                    
  name: pod-a                                
spec:                                        
  containers:                                
  - image: busybox                           
    command: ["/bin/sh", "-ec","sleep 1000"] 
    name: main                               
    resources:                               
      limits:                                
        cpu: 500m                            
        memory: 50Mi                         
      requests:                              
        cpu: 200m                            
        memory: 20Mi

different behavior between requests and limits

Requests guarantee the assigned resources, which are never less than the setting values. By contrast, the limits allow the resource usage more than the setting values, but when the total resource usage over the capability of node, Kubernetes will do something based on the limits. By CPU usage, the limits will work to limit the CPU time shared by different containers. By memory usage, the limits will affect which pod to be OOMKilled(out of memory).

LimitRange: for each pod or container in the same namespace

LimitRange is a namespace-scope object to set limit and requests range for pod, container, and persistentVolumeClaim.

apiVersion: v1
kind: LimitRange
metadata:
  name: limit-range
spec:
  limits:
  - type: Container  
    max:
      cpu: "800m"
      memory: "1Gi"
    min:
      cpu: "100m"
      memory: "50Mi"
    default:
      cpu: "600m"
      memory: "90Mi"
    defaultRequest:
      cpu: "110m"
      memory: "60Mi"

  - type: Pod  
    max:
      cpu: "2"
      memory: "2Gi"
    min:
      cpu: "500m"
      memory: "2Gi"

  - type: PersistentVolumeClaim
    max:
      storage: 2Gi
    min:
      storage: 1Gi

With container type, LimitRange set value range for container resource spec, and apply default values to pods which don't assign any resource requests and limits in spec. The default and defaultRequest make the management of container easier, we don't have to set spec for each containers, only focus on specific containers, which need separate settings.

With pod type, the value range means the total resource usage limits and requests of containers in the same pod.

With pvc type, it provides the range of storage we can set.

Resource Quota: total resource usage of pods in the same namespace.

Administrator can use this object to plan resource configurations for different purpose namespace, for example, production namespace needs more resource, and test namespace needs less. ResouceQuota can not only apply to resouces like CPU, memory, and storage, which you have already seen in LimitRange, but also provide the capability to limit storage class and the number of different objects, like services/pod/job.

apiVersion: v1                                                                           
kind: ResourceQuota                                                                      
metadata:                                                                                
        name: resource-quota                                                             
spec:                                                                                    
      hard:                                                                              
       # requests of cpu and memory                                               
        requests.cpu: 300m                                                         
        requests.memory: 100Mi                                                     

        # limits of cpu and memory                                                 
        limits.cpu: 1                                                              
        limits.memory: 1Gi                                                         

        # storage and total number of pvc                                                                  
        requests.storage: 300Gi        
        persistentvolumeclaims: 3                             

        # storage class                                                            
        dev-storage-class.storageclass.storage.k8s.io/requests.storage: 500Gi  
        dev-storage-class.storageclass.storage.k8s.io/persistentvolumeclaims: 3

        # Object Count
        pods: 2
        configmaps: 10
        services: 1

If you change your current LimitRange and ResourceQuota, the change won't apply to running/existing objects, only affect when you create new object which consumers more resource than limit and quota. By the way, ResouceQuota need each pod has request and limit, we can use LimitRange to apply default value rather than writing value in different pods.

The following diagram shows scopes of these 3 objects

[Go] testing tips part 1: isolate your unit tests from external dependency with DI

Julian-Chu — Sat, 15 Feb 2020 11:23:13 +0000

Series about testing tips in go

I'd like to write a series of articles about testing tips in go, but not from scratch. DI, mock, subtests are planned to cover in this series. If any topic you're interested, just write down your comment.

let's start with a simple scenario:

There is a AuthService in your system to handle user login, you'd like to write unit tests for CI.


///login.go
package auth

type AuthService struct {
}

func NewAuthService() *AuthService {
    return &AuthService{}
}

func (s AuthService) Login(username, pwd string) (bool, error) {
    pwdIsValid, err := s.verifyPassword(username, pwd)
    if err != nil {
        return false, err
    }

    if pwdIsValid {
        return false, nil
    }

    return true, nil
}

// External dependency
func (s AuthService) verifyPassword(username, pwd string) (bool, error) {
    // connect to Database to verify username and password
    panic("not implemented")
}

/// login_test.go
func TestAuthService_LoginSucceed(t *testing.T) {
    s := NewAuthService()
    ok, err := s.Login("testuser", "testpwd")
    if err != nil {
        t.Error(err)
    }

    if !ok {
        t.Error("login failed")
    }
}

After writing first test case, you find the function verifyPassword(..) has external dependency to database. The issue is you can't control what database returns, or you need to make more effort to write integration test to setup and clean database. In order to write unit test here, we have to simulate the return from database. The external dependency is hard coded inside, how can we simulate it?

Dependency injection

The external dependency is hard coded inside and difficult to simulate, so the dependency must be easy to exchange, then we could control it easily.
That's what we need DI(dependency injection). The selected dependency are injected when the AuthService instance is created.

Refactoring code to DI

At the next step, we have to add an interface IPassword, which is embedded in our AuthService, to replace verifyPassword(...).

///login.go
type AuthService struct {
    pwd IPassword
}

func NewAuthService(IPassword IPassword) *AuthService {
    return &AuthService{pwd: IPassword}
}

func (s AuthService) Login(username, pwd string) (bool, error) {
    pwdIsValid, err := s.pwd.verify(username, pwd)
    if err != nil {
        return false, err
    }
    if !pwdIsValid {
        return false, nil
    }
    return true, nil
}

type IPassword interface {
    verify(string, string) (bool, error)
}

Then we could write first mock for our dependency

/// login_test.go
func TestAuthService_LoginSucceed(t *testing.T) {
        // the mock dependency is injected when creating AuthService 
    s := NewAuthService(&mockPasswordService{})
    ok, err := s.Login("testuser", "testpwd")
    if err != nil {
        t.Error(err)
    }

    if !ok {
        t.Error("login failed")
    }
}

type mockPasswordService struct {
}

func (m mockPasswordService) verify(string, string) (bool, error) {
    return true, nil
}

Now our first test case should be passed.

Problem!

You could find in the mockPasswordService, the returned value of verify(...) is hard coded, so when we have to add more test cases, like verify() returns error or false, should we write more mock structs for each scenario?
Of course you should not do this. In next article, I will show to write mock effectively w/o DI framework.

[Go] internal vs external testing

Julian-Chu — Thu, 06 Feb 2020 21:56:24 +0000

What is internal and external testing?

Let's show a example

package operator

func Add(a, b int) int {
    return a + b
}

func subtract(a, b int) int {
    return a - b
}

Internal test is that the test cases belongs to the package we want to test

package operator // same package

import (
    "testing"
)

func Test_Add(t *testing.T) {
    if got := Add(1, 1); got != 2 {
        t.Errorf("Add(%v, %v) = %v, want %v", 1, 2, got, 2)
    }
}

func Test_subtract(t *testing.T) {
    if got := subtract(1, 1); got != 2 {
        t.Errorf("Add(%v, %v) = %v, want %v", 1, 2, got, 2)
    }
}

External test is that the test cases locate in different package. The functions to test need to be exported,


package operator_test //different package

import (
    "awesomeProject/operator"   // package needs to import here
    "testing"
)

func Test_Add(t *testing.T) {
    if got := operator.Add(1, 1); got != 2 {
        t.Errorf("Add(%v, %v) = %v, want %v", 1, 2, got, 2)
    }
}



func Test_subtract(t *testing.T) {
// get error here, because operator.subtract can't be exported
    if got := operator.subtract(1, 1); got != 2 {
        t.Errorf("Add(%v, %v) = %v, want %v", 1, 2, got, 2)
    }
}

By internal testing, you can directly access all thing in your package, including unexported var,func,struct.

By external testing, you can only test exported things.

Internal testing is better, because we could test everything in the package?

The answer is No, if you come from java/c# or other OOP language, you may have seen some discussion like "should I test private method?","how can I test private method?". The conclusion is that you should test public methods only, because test is to simulate how other users use your method or how external project calls your package.

Unexported things are not under testing?

Come back to golang, right, exported method/function/struct is the important thing you should test, how other gophers/projects use your package.
Then you may ask how about unexported things, they're not under testing. The answer is also No. Actually all unexported things should be covered by exported things, otherwise the unexported things are redundant code.(because no one call them)

Conclusion

Just see some discussion, so write down this article for new gophers have no idea which to use. Maybe there're some cases that internal testing is necessary, but external testing is to choose in the general case.

[k8s] Life cycle of volumes

Julian-Chu — Fri, 31 Jan 2020 11:45:56 +0000

So many types of volumes supported in k8s, when learning the volume concept of k8s, they made me confused, how to choose correct one.

Types of Volumes in k8s

Let's try to make it easier by understanding the life cycle of different volumes.

We can sort different types into 3 kinds of life cycle

life cycle follows Pod
life cycle follows Node
Independent life cycle

Life cycle follows Pod

This kind of volume will be created when Pod is up. Wenn pod destroyed, of course the volume is removed. One example is emptyDir. Some use cases are to share files or information between container.

Life cycle follows Node

hostPath is the typical example. It mounts a file or directory from host node into pod. When pods are destroyed, it will remain. But if node is unhealth, the volume is down as well. Use cases like using daemonset to load log files from other pods or sharing configuration files cross pods on the same node.

Independent Life cycle

For the last kind of volume, we need to consider which infrastructure we use, because the actual storage is independent from k8s cluster. It means, when node or cluster is done, the data is still persisted in actual storage. If you have any data to persist, please choose this based on your infrastructure, for example, database.

pv and pvc

There're two ways to use this. One is refer to persistent volume(pv) directly, and another is to use persistent volume claim(pvc) to bind with persistent volume.

Why do we need pvc? Because pvc provides loose couple between persistent volume and pod. The developer and architect don't need to consider the concrete infrastructure too early. For more details, please see the document of k8s.

How to choose

persistent data?
share data cross containers, pods, or nodes?
which infrastructure your product used ex: AWS/Azure/GCP/on-premise

Final

Hope this article could help someone who is also confused with k8s volume. If any mistake or discussion, please feel free to leave your comment.

[Go] use custom type to improve your code

Julian-Chu — Tue, 28 Jan 2020 18:01:34 +0000

I will not discuss struct type here, what I would like to focus is custom type for basic type. What is a custom type for basic type? You could think it likes a alias name of basic type, for example:

type Username string
type ErrorCode int

Readability

Why we should use custom type instead of basic type?
One of my opinion is readability, like the above example, if you only give an int for error code, it's not so easy to understand.

compare the following 2 functions, assign specific type to arguments make the function easier to understand

func Print(s string) {
    fmt.Println(s)
}

func Print(u Username)  {
    fmt.Println(u)
}

Cohesion

Another benefit is, it can have superpower to use method like struct.
Then the code has high cohesion, and because of explicit/specific type, we could have more intellisense/auto-complete support from editor/IDE.

func Print(s string) {
    fmt.Println(s)
}

func Print(u Username)  {
    fmt.Println(u)
}

func (u Username) Print() {
    fmt.Println(u)
}

Let's write another example:
scenario: we would like to extract origin, productId, prodcution year, batch number from a product serial number,

first version: only use basic type

func main() {
    p := "XX-0010-2020-0015"
    fmt.Println(GetOrigin(p))
    fmt.Println(GetProductId(p))
    fmt.Println(GetProductionYear(p))
    fmt.Println(GetBatchNumber(p))
}

const (
    origin = iota
    productId
    productionYear
    batchNumber
)

func GetOrigin(s string) string {
    return getInformation(s, origin)
}

func GetProductId(s string) string {
    return getInformation(s, productId)
}
func GetProductionYear(s string) string {
    return getInformation(s, productionYear)
}

func GetBatchNumber(s string) string {
    return getInformation(s, batchNumber)
}

func getInformation(s string, field int) string {
    return strings.Split(s, "-")[field]
}

second version: use custom type of string

func main() {
    p:= ProductSerialNumber("XX-0010-2020-0015")
    fmt.Println(p.GetOrigin())
    fmt.Println(p.GetProductId())
    fmt.Println(p.GetProductionYear())
    fmt.Println(p.GetBatchNumber())
}
type ProductSerialNumber string

const (
    origin = iota
    productId
    productionYear
    batchNumber
)

func (p ProductSerialNumber) GetOrigin() string {
    return  p.getInformation(origin)
}

func (p ProductSerialNumber) GetProductId() string {
    return p.getInformation(productId)
}

func (p ProductSerialNumber) GetProductionYear() string {
    return p.getInformation(productionYear)
}

func (p ProductSerialNumber) GetBatchNumber() string {
    return p.getInformation(batchNumber)
}

func (p ProductSerialNumber) getInformation(field int) string {
    return strings.Split(string(p), "-")[field]
}

Which one you prefer, if you may change or reorganize code 1 year later.

Example in standard library

Let's look one example in encoding/json.
json.Number is essentially a string which decode a number from json into string format, but with custom type, we can easily convert to string/float/int
as we need.

json.Number

// A Number represents a JSON number literal.
type Number string

// String returns the literal text of the number.
func (n Number) String() string { return string(n) }

// Float64 returns the number as a float64.
func (n Number) Float64() (float64, error) {
    return strconv.ParseFloat(string(n), 64)
}

// Int64 returns the number as an int64.
func (n Number) Int64() (int64, error) {
    return strconv.ParseInt(string(n), 10, 64)
}

Conclusion

Yes, using custom type may need to write more code(but not so much), but it provides good readability and cohesion. From my point of view, it's worthwhile.

[Go] for-select with timer

Julian-Chu — Mon, 20 Jan 2020 22:07:21 +0000

"for-select" is very important in golang for non-blocking operation. "Timer" is a good partner with for-select to handle timeout and performance issue.

In this post, I show some use cases, how for-select with timer work together.

Timeout

Listen to 2 channels, if no receive from any channel in 1s, it goes to the timeout case

    for {
        select {
        case a := <-ch1:
            fmt.Println(a)
        case b := <-ch2:
            fmt.Println(b)
        case <-time.After(time.Second):
                // timeout, do something
        }
      }

Interval

Sometimes we can see new gophers write the following code:

    for {
        select {
        case a := <-ch1:
            fmt.Println(a)
        case b := <-ch2:
            fmt.Println(b)
        default:
                // always non-block here, 
        }
             // no interval, run this for-loop intensively
      }

Then most of usage CPU maybe occupied because of intensive non-blocking infinite loop. We could use timeout case to avoid it, or add interval to entire for loop like this.

    for {
        select {
        case a := <-ch1:
            fmt.Println(a)
        case b := <-ch2:
            fmt.Println(b)
        default:                     
        }
        <-time.After(time.Second) // or time.sleep(time.Second)
      }

The for loop now has 1s interval.

Timer

Sometimes we would like to set timer to break for-loop.
You may implement like following code:

    t := time.NewTimer(time.Second * 5)
    loop:
        for {
            select {
            case a := <-ch1:
                fmt.Println(a)
            case b := <-ch2:
                fmt.Println(b)
            case <-time.After(time.Millisecond *500):
            case <-t.C:
                break loop
            }
        }

The code works, but there's a potential issue in this implementation. Be careful, "select" statement chooses any non-blocked case randomly, for example, even if the timer is fired, select can pick up another cases when they're non-blocked.
So we can improve it by adding another select statement.

    t := time.NewTimer(time.Second * 5)
    loop:
        for {                        
            select {
            case <-t.C:  // if timer is fired, break the loop 
                break loop
            default:    // timer is not fired, non-blocked 
            }

            select {
            case a := <-ch1:
                fmt.Println(a)
            case b := <-ch2:
                fmt.Println(b)
            case <-time.After(time.Millisecond *500):           
            }

    }

I think these are common use cases for for-select with Timer.
If any mistake and question, please feel free to write comments.

[Go] Ensuring struct has implemented interface

Julian-Chu — Wed, 04 Dec 2019 22:31:43 +0000

Structural typing vs Class-based interface

In other OOP languages, the interface is a contract by implementing class. By Contrast, Golang checks the interface implementation for struct when you call struct with interface, not by implementing struct.

Let's see some code

interface IMessageService{
  void Send()
}
// compile time error to show the class doesn't implement the interface
public class MyService: IMessageService{

}

func main() {
        // compile time error happens when using it
    var service IMessageService = &MyService{}
    service.Send()
}


type IMessageService interface {
    Send()  
}

type MyService struct {

}

Can we check the interface implementation before calling it?

Of course, we can. There're 2 options to do this.

Constructor returns interface:

func NewMyMessageService() IMessageService {
    return &MyService{}
}

Using underscore to do check

var _ IMessageService = &MyService{}

In case you have multiple interfaces to implement:

Embedding multiple interfaces into one, and return it

type IMyService interface {
    IMessageService
    IEmailService
}

type IMessageService interface {
    Send()
}

type IEmailService interface {
    SendEmail()
}

type MyService struct {

}

func NewMyService() *IMyService {
    return &MyService{}
}

Using underscore to do multiple checks

var _ IMessageService = &MyService{}
var _ IEmailService = &MyService{}

Underscore is more useful

"_" is also useful to check whether the 3rd party struct has implement client side interface.

import(
   "thirdparty"
)

var _ IMessageService = &thirdparty.NotMyService{}