Forem: Francisco Javier Sánchez Fuentes

How to make Coverage in Golang?

Francisco Javier Sánchez Fuentes — Sun, 15 Aug 2021 06:28:08 +0000

Hi Everyone!

In this post we'll to check about coverage in Golang.

Alternatively, you can download the Source code and follow the instructions in the README.md.

TL;DR

# Run the tests and save the coverage profile in "coverage.out"
go test -coverprofile=coverage.out ./...

# View the coverage profile in your browser
go tool cover -html=coverage.out

Coverage in Golang

For this example, we have to create an user entity and store it in memory. This is the structure of the project.

.
├── README.md
├── cmd
│   └── main.go
├── go.mod
├── go.sum
└── internal
    └── user
        ├── storage.go
        ├── storage_test.go
        └── user.go

we need to write the program

First of all, we need to declare the user entity, the filename for this is 'user.go' (in the "internal" folder in called user).

// Package user describe user entity attributes and manage user storage
package user

type User struct {
    ID        uint64
    FirstName string
    LastName  string
    Age       uint16
}

And now, we have to create the user storage. In the same package, the storage.go was created with the following structure and methods.

package user

type UserStorage struct {
    DB []User
}

// AddUser Add user information to the UserStorage database
func (us *UserStorage) AddUser(user User) {
    us.DB = append(us.DB, user)
}

// FindUserByID find the user by ID
func (us *UserStorage) FindUserByID(id uint64) User {
    for _, user := range us.DB {
        if user.ID == id {
            return user
        }
    }
    return User{}
}

// Count total users in the database
func (us *UserStorage) Count() int {
    return len(us.DB)
}

We need to test the program

For now, we have three methods of adding, searching, and counting your own users in storage. But we have to test that, so we add the storage_test.go file with its test. The following code has three tests that make 100% coverage for UserStorage.

package user_test

import (
    "testing"

    user "github.com/fransafu/coverage-golang-example/internal/user"
    "github.com/stretchr/testify/assert"
)

func TestUserStorage_SaveUser(t *testing.T) {
    var userStorage user.UserStorage

    user1 := user.User{}
    user1.ID = 1
    user1.FirstName = "Francisco"
    user1.LastName = "Sanchez"
    user1.Age = 99

    userStorage.AddUser(user1)

    assert.Equal(t, 1, userStorage.Count())
}

func TestUserStorage_SearchUser(t *testing.T) {
    var userStorage user.UserStorage

    user1 := user.User{}
    user1.ID = 1
    user1.FirstName = "Francisco"
    user1.LastName = "Sanchez"
    user1.Age = 99

    userStorage.AddUser(user1)

    assert.Equal(t, user1, userStorage.FindUserByID(user1.ID))
}

func TestUserStorage_EmptySearchUser(t *testing.T) {
    var userStorage user.UserStorage

    assert.Equal(t, 0, userStorage.Count())
    assert.Equal(t, user.User{}, userStorage.FindUserByID(1))
}

If you thinking about improving the tests with "Test Suite" or "Test Group", yes, this is the way, but for now, these examples are simple and atomic for this post.

Get the coverage

To get the coverage in a Golang project you can use the "./..." wildcard and the Golang CLI will search for all tests declared in a project.

# Run the tests and save the coverage profile in "coverage.out"
go test -coverprofile=coverage.out ./...

Ok, we have the file coverage.out that contains the coverage information but we need to view the results, the following command line shows the coverage program in the browser.

# View the coverage profile in your browser
go tool cover -html=coverage.out

Now you can check all files that contain declared tests. If you don't declare a test file, you won't have coverage for that module.

It's all for now, thanks for reading this post.

:)!

A brief introduction to Observability Driven-Development

Francisco Javier Sánchez Fuentes — Sun, 27 Sep 2020 19:34:04 +0000

Hi Everyone!

Let's talk about the Observability Driven-Development and how it works.

Why software development needs a "methodologies driven development"?

First of all, I took advantage of this instance to share answers from another user to discuss with a short explanation of why software development "Driven-development" or "Driven-design" are needed.

how come we don't have -Driven Development in other engineering disciplines, only in software? It's because other branches of engineering are faced with much lower levels of uncertainty. When engineering physical objects/systems. it is much cleared what are we building and what are the actual physical constraints (for example, when building a bridge). discuss source here

Ok, you just keep in mind that "the software development process has uncertainty" because we have the human factor in a transversal way throughout the process.

Observability Driven-Development

Now, Observability Driven-Development applies when you monitor your process before, during, and after the development process. The goal is to detect user experience issues or bugs that have not yet been reported. So at best, you can run into problems through observability.

What the mean "Observability"?

The concept of "Observability" doesn't have its origin in computer science and is applied to software engineering, it originated in systems theory issues where it was about "understanding a system through observation". So, this methodology aims to understand and improve a system by observing at startup or when it is mainly operational (production). The reason for observing a system in those processes is because when a system is running it is exposed to other unexpected actions by users.

What about it with other Driven development methodologies?

The authors say that the methodology doesn't conflict with other methodologies such as TDD or BDD. Then, it is a complementary methodology in which DevOps, SRE, QA, Developers, and Product Owner participate where a monitoring system is used to understand where they can debug to find a problem or where they can improve a system.

Conclusions

ODD doesn't say where the problems are, but it's useful to find them by observation.

Thanks for reading :)

I hope helpful this brief introduction.

More info about it in infoq.com

Detect and track your errors in production and fix it!

Francisco Javier Sánchez Fuentes — Wed, 06 May 2020 01:40:24 +0000

Hi Everyone!

Today, I talk about track errors in production.

Detecting errors is very important for Dev and DevOps team, It's much better when you detect de error and fix it before the client sends a notification, especially if your error is in production, for that you have many options :).

First, you should put the correct format in the logs of your app, ok... if you don't have logs, maybe you should install the log module.

Recommend format: [DATE][ENVIRONMENT][APP][PROCESS_ID/THREAD_ID] message:

Why I put the "process id" or "thread id"? If you run the cluster mode or multicore (with thread) you need to know if the error causes a domino effect in another process/thread (anyway, this is optional).

Ok, you have logs... your module of logs constantly block the I/O thread? is it efficient when printing your app logs? No? Ok... change your log module.

Note: Don't forget the format of your logs and especially use an efficient logger.

This is the way for improving your logs for seeing the errors in production... but, what about if I want to see the real-time error? (What??)

R: Bugsnag or Sentry

Yes, you have to register, then get the client token. When you have the token, incorporate the Sentry/Bugsnag module into your application (use the token) and your app must send the error through the client. That is all, you will see all error (nearly real-time) and if you configure logs by environment and client... much better :)

Finally, I want to share the log module for Node.js I most use: Winston. If you want to read more about logs, click here

Note: This is applicable for other languages, runtime and more.

Good luck guys!

I hope you like this post :)

The first experience with k3s (lightweight Kubernetes). Deploy your first app!

Francisco Javier Sánchez Fuentes — Mon, 04 May 2020 04:17:02 +0000

Hi everyone!

Today, I talk about k3s. It's a lightweight version of Kubernetes (k8s) and you can use for the Internet of Things (Can you imagine your own IoT cluster?) or Edge computing (distributed computing with the cluster?) ok, that is a little crazy but real.

TL;DR

# Install k3s
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s -

# Check if response the node with the current version
k3s kubectl get node

# Create the namespace (only dev)
kubectl create namespace retail-project-dev

# Download the manifest
curl -X GET -L https://gist.githubusercontent.com/fransafu/4075cdcaf2283ca5650e71c7fd8335cb/raw/19d7cfa0f82f1b66af6e39389073bcb0108c494c/simple-rest-golang.yaml > simple-rest-golang.yaml

# Create ingress, service, and deployment
k3s kubectl apply -f simple-rest-golang.yaml

# Check ingress (its show your IP), service and pods
k3s kubectl get ingress,svc,pods -n retail-project-dev

# Test cluster and app
curl -X GET http://YOUR_IP

# That is the output
{"data":"RESTful with two Endpoint /users and /notes - v1.0.0"}

Introduction

K3s an initiative of Rancher so that you may create a k8s cluster but in lower requirements. You will be able to install k3s in a Raspberry and run your programs or microservices in High-Availability (HA). However, you have to be careful when you start your setup in case you if want HA.

Key concepts: Before continuing, you need to familiarize yourself with the following concepts

High-Availability (HA)

When you need reliability, a minimum amount of down-time, redundant instances of your programs or computers... you are looking for High-Availability. It's the main characteristic of this concept, so HA is the ability to provide service for your users although your cluster fails.

Kubeconfig

It's a file used to configure access to Kubernetes, in this case specifically used by kubectl (it's a CLI for k8s). Moreover, used by CI/CD Pipeline for access to the cluster.

Artifacts

An Artifact is a common concept of Software engineering to describe the component of software as part of code, description/documentation, and more. An Artifact is all "things" generated in process design, develop, implement, and maintain the software. Please, visit the following link for more details.

Get started

Disclaimer: This configuration is for development environment (non-production!!).

Install k3s in your laptop

Run the next command for install and change permissions of Kubeconfig. By default, k3s start their own services.

curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s -

Note: This command allow access to kubeconfig (default path: /etc/rancher/k3s/k3s.yaml). For example, when used Kubectl.

Check installation

First, check your nodes

k3s kubectl get node

You should see the one node with the current version of k3s. Now, the following step maybe is unnecessary, but I want to show you the cluster API, k8s have good interoperability with good design of API.

Note: The API of k3s is a little bit least than API k8s in terms of interoperability

Secondly, you should get the credentials. In the next step, you view the kubeconfig and you can see how k3s configure that file. The main concepts you need know is:

Cluster: Basic configs fo certificates and server IP.
Context: Is a group access parameters, for example, the namespace access by default.
kind: This word define what kind is the config file (you will see it in many files).
users: Yes! Here are the credentials for users.

Important: The common format for files of k8s and k3s is YAML.

cat /etc/rancher/k3s/k3s.yaml

Now, copy & paste server IP in your browser (https://127.0.0.1:6443), click enter and put the credentials for access, then It's showing you all paths of your k3s. You probably won't use them but you need to know they exist

Deploy your application

If you don't have any application, you can use my sample app

This project is deployed under the name of 'retail-project'.

First, you must create a namespace, the namespace is a logical division in your cluster, is used for encapsulate groups of application and configuration, if the namespace has a problem, this problem doesn't impact another namespace. A good practice is to create al least two namespaces by the project (prod: production and dev: development).

For this example only create the development environment.

kubectl create namespace retail-project-dev

Perfect! now, deploy your apps. but before that, I should explain about configuration the manifest for creating artifacts (resources into the cluster).

Previously, I talked about the main format of config files is YAML and each config file has a Kind label followed a description of the artifact (resources). Ok, when you specified the "kind" of artifact and run the command apply over k3s, It creates a group(s) of Pods, ReplicaSet, Deployment, or another artifact.

Concepts:

Pod: A Pod is a group of one or more containers (such as Docker containers).
ReplicaSet: A ReplicaSet’s purpose is to maintain a stable set of replica Pods running at any given time.
Deployment: A Deployment's purpose is to declare how many instances of Pod need and create a ReplicaSet for that, and declare what image of app use for creating instances in a Pod.

Note: All these definitions are from official k8s documentation, except Deployment.

Ok, let's deploy the app. I hope these concepts help your knowledge about artifacts of k8s and k3s.

Prepare the template

Well, this template has 3 sections, the first section is app definition (simple-rest-golang-deployment), also the Kind is a Deployment, that is important because a Deployment generates the number of replicas (in this case is two) through the ReplicaSet so that It generates the Pods. Therefore, ReplicaSet is responsible of Pods, and Deployment is responsible of ReplicaSet. It's important to understand the hierarchy.

Deployment uses the name "simple-rest-golang-deployment" in the namespace "retail-project-dev" and It generates two replicas of simple-rest-golang from Docker registry repository with the name "fransafu/simple-rest-golang:1.0.0".

*Note: * If you want to read more about resources of containers Click here

Download the file

curl -X GET -L https://gist.githubusercontent.com/fransafu/4075cdcaf2283ca5650e71c7fd8335cb/raw/19d7cfa0f82f1b66af6e39389073bcb0108c494c/simple-rest-golang.yaml > simple-rest-golang.yaml

Or you can copy & paste the template in a file: 'simple-rest-golang.yaml'

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: simple-rest-golang-deployment
  namespace: retail-project-dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: simple-rest-golang
  template:
    metadata:
      labels:
        app: simple-rest-golang
    spec:
      containers:
        - name: simple-rest-golang
          image: fransafu/simple-rest-golang:1.0.0
          resources:
            requests:
              memory: "64Mi"
              cpu: "100m"
            limits:
              memory: "128Mi"
              cpu: "500m"
          ports:
          - containerPort: 8080
          imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: simple-rest-golang-service
  namespace: retail-project-dev
spec:
  ports:
  - port: 80
    targetPort: 8080
    name: tcp
  selector:
    app: simple-rest-golang
--------
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: simple-rest-golang-ingress
  namespace: retail-project-dev
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: simple-rest-golang-service
          servicePort: 80

Apply the template

Now, create the ingress, service, and pods from manifest

k3s kubectl apply -f simple-rest-golang.yaml

Output

deployment.apps/simple-rest-golang-deployment created
service/simple-rest-golang-service created
ingress.networking.k8s.io/simple-rest-golang-ingress created

Check your cluster and apps

First, you need to see your resources in k3s cluster

k3s kubectl get ingress,svc,pods -n retail-project-dev

Take your IP from the field ADDRESS. Then replace below "YOUR_IP" with your IP.

Perfect, run the next command.

curl -X GET http://YOUR_IP

If you get the following output, congratulation! Your cluster work fine :)

That is the output:

{"data":"RESTful with two Endpoint /users and /notes - v1.0.0"}

Congratulations! You have k3s cluster in your computer :)

I hope this tutorial is helpful!

Docker for all #1: Introduction

Francisco Javier Sánchez Fuentes — Sat, 11 Apr 2020 03:56:33 +0000

Hi Everyone!

I wrote an introduction about how does use with docker (is very simple), I hope you like :)!!

What is Docker?

"Docker is an open platform for developing, shipping, and running applications" -- Docker docs

Is technology for encapsulating application. All applications when running with docker have the own environment and isolate another app running over the same operating system.

How's use docker?

You can develop an application, they create Dockerfile with all dependencies and step for the run, then you should test docker in localhost and validate for deploy.

List of validations before deploy:

The behavior of the app is correct?
Request and Response it's fine (maybe unit testing)?
The image is secure for production (validate with the author or check with your own skills)?
Just have only necessary dependencies (production mode)?

When all validations pass, you must push all code to Git repository and run the Pipeline for deploy.

How to deploy Docker?

Actually have many options for deploy Docker, but principally, we can split this into two categories:

Pipeline CI/CD
Docker Cluster

If you want to make the build, test and deploy with your own hands... but I don't recommend this method (Shell Scripting)

Pipeline

It is a toolchain of jobs for build, test, upload to docker registry and deploy to cluster.

The main options are:

I prefer the Gitlab.

Docker Cluster

The cluster is a manager of Docker applications and its responsible for stay running, deploy all services, and more.

Actually existing many clusters for manage applications with Docker, the more famous is Kubernetes, also for the Internet of. Things (IoT) exists k3s (is the best option for IoT) and finally Swarm.

Kubernetes have many implementations on the cloud GKE, AKS, EKS

The main docker cluster for developing (non-production) is Minikube

With that, you know de main concepts and stages for work with docker.

Remember:

Develop
Make Dockerfile
Validate
Push to Git (never forget)
Run Pipeline
Deploy to docker cluster

Perfect :)! your app is running in high availability, with the best practices for deploy. Don't forget to do implement the 'Rollback' stage useful when implementation fails (but surely it doesn't happen to you, however, implement the stage).

Thanks for your time and I waiting for your feedback :)!

Hi I'm Francisco

Francisco Javier Sánchez Fuentes — Sat, 11 Apr 2020 02:48:46 +0000

Hi Everyone!

My name is Francisco Sánchez, I living and work in Santiago of Chile.

I have been coding for 5 years professionally for companies.

I'm a Software Engineer with a specialist in Backend. The main interesting topics are Architecture, Design, and improve performance of Software, also check the cost of operations for IaaS, PaaS, and SaaS in AWS and GCP.

I mostly program in theses languages: Javascript and Python.

You can find me on Github as fransafu

I hope to be a contribution to this community.

Nice to meet you :)