Hasslefree version management of Golang

#go #backenddevelopment #webdev #microservices

📌 Introduction

Hey folks , it's me 😊. Well , this time I have come with an awesome as well as debatable topic which roam around developer's minds 🧠. Developers who are passionate about building scalable backends with Golang , often suffer from keeping its multiple versions alive in a system 😵‍💫.Golang's installation doesn't discuss this issue elaborately. So there are scattered opinions and methods to arrange the versions together. Some refers to use an external package called - gvm , but believe me , it has its own faults. gvm is only suitable to download older go versions but its not suitable to install new ones. In this blog 🖋️ , I am going to discuss my steps and key takeaways to manage golang's different version in my system, without using any external packages or libraries. I have done this in my WSL ubuntu , so it will surely work in your native LINUX (can't say more about WINDOWS , sorry!! 😼). But Before that Let's discuss what is golang and how it became so popular

📌 A little bit about Golang

Go is a general-purpose programming language that was created by Google in 2007. Google created Go to serve as a contemporary substitute for C. Well , it couldn't do that but got its own position. Nowadays ,it is mostly used to create reliable backends 💿 of various applications that require characteristics like concurrency for scalability. Go is a compiled language so it doesn't need the overhead of any virtual machine . The standard go toolchain builds the filename.go code into bytecode . Does Golang follow any precise architecture like any framework ? Yes . The official docs says that a golang project must have 3 folders :

bin/ 📂 (for storing the binary executables of your go files)
pkg/ 📂 (the packages which you install from any server like GitHub , golang , bitbucket)
src/ 📂 (the folder where you create your go files). But do you know what's the most interesting part of it ? Your Golang source project is itself built by this architecture, means the whole Golang installation creates a folder where you will see these 3 folders inside it and if you look into src/ folder you will go files, wow , so self-dependent language it is.

But this same solution creates another problem which I am going to tell later on in the what went wrong section. Now, be comfortable with some terminologies around golang -

Toolchain ⛓️‍💥 - A set of essential tools provided by the Go programming language for developing, building, testing, and managing Go code. It's the foundation you rely on to work with Go effectively. When you install a go version in Linux , its corresponding toolchain is downloaded inside the source project folder (specifically in the pkg/mod/golang.org)
Based on this primary version and toolchain , you can only download other golang toolchains and use them.
GO ENV 🌳 - A set of environment variables that go borrow and set inside your Linux system . Few important ones among them are -
A. GOROOT - The entire Go installation lives here
B. GOPATH - The workspace where Go will download its packages
C. GOBIN - The bin/ folder inside GOPATH
D. GOTOOLCHAIN - it decides which other toolchains of golang is eligible to install
E. GO111MODULE - Helps to download other golang toolchains
F. GOPROXY - configure to use a module proxy server. Instead of directly downloading dependencies from their version control system repositories (like GitHub, Bitbucket), Go will fetch them from the specified proxy.
G. GOSUMDB - Helps to check the integrity and authenticity of downloaded modules by verifying the checksum of each package and a database stores this checksums

Now till now , if I have created some interests to learn golang within you , you can follow my tutorial in GitHub - Tutorial. No doubt , you will code 'Hello World' in Golang if you follow my tuts 😄.

Now without any further story-telling , let's hop into the main chapter

📌 What went wrong

If you have never installed golang into your system , means you are a newborn 🧒 to golang's world . No worries my friend , you are also welcomed . You will directly get the nutritious foods 🍲without tasting other foods.

But if you've installed golang into your system from the official website , at least once , then you are my type of guy 🫂 . In the previous section , I have told you about golang's folder architecture . Now the problem occurs when go claims to create its own environment. Peeps who are coming from python , developed some stuffs there , knows about Virtual Environment. This venv is an isolated area inside the project's physical location where you do install some libraries , write some code , get your job done , without actually cluttering the memory with these libraries permanently .
But that doesn't happen for Golang , golang tell us to follow a specific architechture which is also being followed by itself. So when you make a golang based project and follows the architecture (bin-pkg-src) , you are passively recreating the go source project :)

Sounds Crazy, right? Let's dissect a real life scenario -

Let's say , you have downloaded go1.22.x version in your linux to learn golang . You have unzipped the tar file in /usr/local location and that's your GOROOT which you have appended in the PATH environment variable . For GOPATH , you just gave the location of your go-programs folder because the golang's docs doesn't really say anything about it. Your go-programs folder reside in the ~/ folder , mind it. Your go-programs folder follows bin-pkg-src architecture and yeah! you think you have done correct , because what will be the location other than your folder where you will download necessary go packages inside pkg/ folder .
It has been 2 months and you have done some coding in golang (created some go files in src/ folder , downloaded some packages in your own folder) . Now you have decided to install another version of golang by using go install golang.org/dl/go1.23.x.... & go1.23.x.. download (as per the docs). Maybe you succeed . You have also kept in mind that this is just another toolchain which is being installed using the base source project of version 1.22.x . So you know when you will do go version it will throw 1.22.x but when you run go1.23.x version you EXPECT to see 1.23.x but NO!! That doesn't happen. Instead you see , command not found from bash.
While inspecting in ~/ folder 📂 , you see a sdk/ 📂 folder. This folder contains only bin/ pkg/ and inside pkg/mod/golang.org there are 2 toolchains. You became shocked , how can it be possible 🫢, even after installation of new toolchain you can't use it !!
So you decided to remove all go source projects from /usr/local to sdk/ to just install the new version , because you thought that as you have hardcoded the environment variables GOROOT in .bashrc or .zshrc , that's became the problem and you have to install the new go 1.23.x from source. You do it , you changes the GOROOT but don't change GOPATH , because it targets your go-programs where your packages will be installed . Seems simple . Well that's the actual reason of failure . After installation when you will run go version you will expect to see 1.23.x but instead , it automatically starts to download go 1.22.x . That go1.22.x is now a toolchain inside go1.23.x/pkg/mod/golang.org but its the default and if you have to use your go1.23.x , you have to create a go.mod file manually without using go mod init.....

At this point , you are pretty much messed up 😖 .You are having terrible thoughts 😡. But wait! DON'T BREAK YOUR PC or LAPTOP :), there is definitely a way to get rid of this problem. You can also think about how unorganized your go source project is , inside your Linux system , it's terrible! and t*he go docs doesn't really address this problem.* That's why I have come here to show you the right way , the perfect way to manage your go-programs and go-source project .
Let's go for the solution

📌 Solution

A short disclaimer to you : I have decided to keep both go source project and my own go programs both in ~/ directory but you can keep them different locations. But remember , other than ~/ directory , all directories need strict permission and its can be accessed by sudo only , you will have to change the permission and that's harmful for your system because for you tiny go source code that entire system directory became prone to malicious attacks. So its safe to do everything inside ~/ directory

If you have installed and used golang for quite a while . Remove all your go folders (source and own programs) by sudo rm -rf.... Then run this two commands in your Linux system ,

find . -name go.mod

This command will find if there is anything with go.mod inside every folder of your Linux system . Will quickly execute and return nothing if there is no go.mod . Why you need to do this ? because go.mod is the requirements.txt for golang , except it also stores the snapshot of the golang version which forces to download that exact version of toolchain in the previous section . So your system should not have any of this go.mod

env | grep GOROOT

This command will show if there is any variable named GOROOT inside your .bashrc or .zshrc . If it gives some output , just open the .bashrc or .zshrc and remove all GOROOT,GOPATH,PATH (that one which connects the GOROOT to PATH , not other PATH), GOxx variables and do source ~/.bashrc. It will be better , if you restart the system.

Now your system is ready to begin . Let's make folder 📂 by naming something like go-toolchains/ or go-source/ where your entire go source project will reside . Inside that folder , create 2 folders source/ and packages/. This source/ will be your new GOROOT and packages/ will be your new GOPATH. NO , this GOPATH will have no connection to your go-programs ,will come up to that later. Inside packages/ make 2 folders - bin/, pkg/ (because here you won't write any go files so you don't need any src/ folder)

You came inside the source/ folder again , downloaded the tar.gz file here and you need just one more command to run ,

tar --strip-components=1 -xzf go1.24.2.linux-amd64.tar.gz

The argument --strip-component doesn't make another folder but makes the existing one as the source folder by flooding every files and folders inside it , similar like . in python where the source project is flooded inside the existing directory.

Now open then .bashrc or .zshrc in your favourite editor with sudo command (precisely ~/.bashrc or ~/.zshrc) . You have to paste these environment variables whose explanations I have given long before in this blog. You should not change the values of the variables

  export GOROOT="$HOME/go-toolchains/source"   
  export GOPATH="$HOME/go-toolchains/packages"
  export GOBIN="$GOPATH/bin"                
  export PATH="$GOROOT/bin:$GOBIN:$PATH"    
  export GOTOOLCHAIN='auto'       # Check before paste
  export GO111MODULE='on'         # Check before paste
  export GOPROXY="https://proxy.golang.org,direct"    # Check before paste
  export GOSUMDB="sum.golang.org"   # Check before paste

NOTE : Do go env and see if GOTOOLCHAIN , GO111MODULE , GOPROXY, GOSUMDB is previously set to the values mentioned in the snippet given above , if not then change it to these values.

Save it inside the file , do exit from the editor and run source ~/.bashrc or source ~/.bashrc. Its preferred to do a restart again.

Now when you will write go version, you will see go1.23.x . Not only that , if you install any other toolchains using go install golang.org/dl/go.... & go...download , you will see it has been installed inside go-toolchains/packages/pkg/mod/golang.org. Also if you do go... version, you will definitely see the new toolchain and it's version whereas if you do go version, you will see the base version.

📌 Conclusion

That's it. Now you can download your go-programs which may have some other versions and no worries , your base-version will help to install that toolchain . You can see how this solution not only solves the problem of version conflict but also keeps things in a single location and restricts to mix up with your own go-program folder. With that said , now it's time to bid goodbye 👋 . will meet in some other beautiful day.