Forem: Raul Paes Silva

DevPill 10 - Fault tolerance: adding retries to your Go code

Raul Paes Silva — Fri, 12 Dec 2025 01:21:44 +0000

Adding retries to your API is a must to make your system more resilient. You can add them in database operations, communication with external apis and every other operation that might depend on a response of a third party.

Here's an example of a retry implementation for database transient errors:

1. First, let's code the retry function

The function receives the number of attempts, delay (interval between attempts) and the function which represents the operation you want to do.

func Retry(attempts int, delay time.Duration, fn func() error) error {
    err := fn()
    if err == nil {
        return nil
    }

    if !IsTransientError(err) {
        // log.Println("--- retry not needed")
        return err
    }

    if attempts--; attempts > 0 {
        time.Sleep(delay)
        // log.Println("--- trying again...")
        return Retry(attempts, delay*2, fn) // exponential backoff
    }

    return err
}

2. IsTransientError function

In this example I created a function that check if the operation is worth trying again based on the error.
Let's say the repository returns "sql.ErrNoRows", which is returned when the query indicates no rows were found for the select command. This error definitely does not need to be attempted again. That's why we're looking for errors that are transient, that might happen for a few milliseconds/seconds for network reasons for example.

func IsTransientError(err error) bool {
    if err == nil {
        return false
    }

    // example : deadlock postgres
    if strings.Contains(err.Error(), "deadlock detected") {
        return true
    }

    // closed connection
    if strings.Contains(err.Error(), "connection reset by peer") {
        return true
    }

    // network/postgres timeout
    if strings.Contains(err.Error(), "timeout") {
        return true
    }

    // Max connections exceeded
    if strings.Contains(err.Error(), "too many connections") {
        return true
    }

    return false
}

3. Using the function

Here's an example of retrying a database operation called in a login method in service layer:

func (s *UserService) Login(ctx context.Context, email, password string) (*UserOutput, error) {

    var user *domain.User
    err := retry.Retry(3, 200*time.Millisecond, func() error {
        var repoError error
        user, repoError = s.repo.GetUserByEmail(ctx, email)
        if repoError != nil {
            if repoError == sql.ErrNoRows {
                return ErrUserNotFound
            }
            return repoError
        }
        return nil
    })

    if err != nil {
        return nil, err
    }
    //... ... ...

DevPill 9 - IP Based Rate Limiter Middleware for native Go REST API

Raul Paes Silva — Thu, 11 Dec 2025 20:54:37 +0000

1. Function to get the right remote address ip

Usually we use the remoteAddr() function inside the type http.Request to get the remote address. But in cloud solutions it might not return the correct reference.
Use the function below to catch the right address:

func GetClientIP(r *http.Request) string {
    // if app in on Kubernetes ingress or other cloud services r.RemoteAddr might not work
    //get the client ip using the header X-Forwarded-For
    forwarded := r.Header.Get("X-Forwarded-For")
    if forwarded != "" {
        parts := strings.Split(forwarded, ",")
        return strings.TrimSpace(parts[0])
    }

    // Fallback to RemoteAddr
    ip, _, _ := net.SplitHostPort(r.RemoteAddr)
    return ip
}

2. Getting the specific limiter for the received IP

Use a map of visitors to store every new IP as a key to a pointer of rate.Limiter (package "golang.org/x/time/rate").
We'll also use sync.Mutex to avoid race conditions.
The function below check if there's a limiter for the received ip, if not, it creates a new limiter allowing maximum 1 request per second and burst of 3 (meaning it will allow 3 requests at once for the "first" interaction ).

var visitors = make(map[string]*rate.Limiter)
var mu sync.Mutex

func getLimiter(ip string) *rate.Limiter {
    mu.Lock()
    defer mu.Unlock()

    limiter, exists := visitors[ip]
    if !exists {
        limiter = rate.NewLimiter(1, 3) // 1 req/s, 3 burst
        visitors[ip] = limiter
    }

    return limiter
}

3. Middleware function

Get the client ip, request for its limiter, check if new request is allowed, if not returns "429 - Too many requests".

func IPBasedRateLimiter(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ip := GetClientIP(r)

        limiter := getLimiter(ip)

        if !limiter.Allow() {
            http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
            return
        }

        next.ServeHTTP(w, r)
    })

}

DevPill 8 - How to set up jwt authentication for your Go REST API

Raul Paes Silva — Thu, 11 Dec 2025 02:44:24 +0000

1. Create your login handler to check user credential and generate a new JSON Web Token.

Example:

func (h *AuthHandler) Login(w http.ResponseWriter, r *http.Request) {

    var req dto.LoginRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "invalid body", http.StatusBadRequest)
        return
    }

    // Validate user from database
    user, err := h.Service.Login(r.Context(), req.Email, req.Password)
    if err != nil {
        http.Error(w, "invalid credentials", http.StatusUnauthorized)
        return
    }

    // Create JWT token
    token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
        "sub": user.ID,
        "exp": time.Now().Add(h.JWTExpiry).Unix(),
    })

    tokenString, err := token.SignedString([]byte(h.JWTSecret))
    if err != nil {
        http.Error(w, "could not create token", http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(map[string]string{
        "token": tokenString,
    })
}

The important part here is jwt.NewWithClaims(), where the JWT is generated with the user payload and predefined expiration time.

    // Create JWT token
    token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
        "sub": user.ID,
        "exp": time.Now().Add(h.JWTExpiry).Unix(),
    })

    tokenString, err := token.SignedString([]byte(h.JWTSecret))
    if err != nil {
        http.Error(w, "could not create token", http.StatusInternalServerError)
        return
    }

2. Create a new middleware to check Authorization header

Here, you'll have to get the header "Authorization" to look for the Bearer, which will inform a valid Json Web Token.
Example:

func JWTAuth(secret string) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

            authHeader := r.Header.Get("Authorization")
            if authHeader == "" || !strings.HasPrefix(authHeader, "Bearer ") {
                http.Error(w, "unauthorized", http.StatusUnauthorized)
                return
            }

            tokenStr := strings.TrimPrefix(authHeader, "Bearer ")

            token, err := jwt.Parse(tokenStr, func(t *jwt.Token) (interface{}, error) {
                if _, ok := t.Method.(*jwt.SigningMethodHMAC); !ok {
                    return nil, fmt.Errorf("invalid signing method")
                }
                return []byte(secret), nil
            })

            if err != nil || !token.Valid {
                http.Error(w, "invalid token", http.StatusUnauthorized)
                return
            }

            claims := token.Claims.(jwt.MapClaims)

            ctx := context.WithValue(r.Context(), "userID", claims["sub"])

            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }
}

Additionally, it will check if it is valid according to the configured secret.

3. Make sure you use the middleware

Add the middleware handler to all the endpoints you want to protect with JWT authentication.
Example:

    publicMux := http.NewServeMux()
    publicMux.HandleFunc("POST /users/signup", userHandler.CreateUser)
    publicMux.HandleFunc("POST /users/login", authHandler.Login)

    var public http.Handler = publicMux
    public = middleware.Logger(public)
    public = middleware.RequestID(public)

    protectedMux := http.NewServeMux()
    protectedMux.HandleFunc("POST /devices", devHandler.CreateDevice)
    protectedMux.HandleFunc("PUT /devices/{id}", devHandler.UpdateDevice)

    var protected http.Handler = protectedMux
    protected = middleware.JWTAuth(cfg.JWTSecret)(protected)
    protected = middleware.Logger(protected)
    protected = middleware.RequestID(protected)

    root := http.NewServeMux()
    root.Handle("/users/signup", public)
    root.Handle("/users/login", public)

    root.Handle("/devices/", protected)
    root.Handle("/devices", protected)

    server := http.Server{
        Addr:    fmt.Sprintf(":%d", cfg.WebServerPort),
        Handler: root,
    }

DevPill 7 - Cloud SQL Access for your containers on Google Kubernetes Engine

Raul Paes Silva — Sun, 07 Dec 2025 16:05:31 +0000

Pre-requirements:

CloudSQL instance running
a database created

1. Create a new service account

name: "cloud-sql-gke-sa"
role: "roles/cloudsql.client"

2. Get the JSON key from the service account

After accessing the service account, go to "Keys" > Create New Key > type JSON > Create.

3. Create a new Kubernetes Secret

kubectl create secret generic cloud-sql-sa-key \
  --from-file=credentials.json=/path/to/service-account.json

4. Alter your deployment file to add Cloud SQL Auth Proxy as a sidecar in the same pod.

Example:

containers:
  - name: stock-service
    image: us-central1-docker.pkg.dev/{project_id}/{repo_name}/{database_name}
    env:
      - name: DB_HOST
        value: "127.0.0.1"
      - name: DB_PORT
        value: "5432"
      - name: DB_USER
        valueFrom:
          secretKeyRef:
            name: app-secrets
            key: DB_USER
      - name: DB_PASSWORD
        valueFrom:
          secretKeyRef:
            name: app-secrets
            key: DB_PASSWORD
      - name: DB_NAME
        valueFrom:
          configMapKeyRef:
            name: app-config
            key: DB_NAME

  - name: cloud-sql-proxy
    image: gcr.io/cloudsql-docker/gce-proxy:1.33.2
    command:
      - "/cloud_sql_proxy"
      - "-instances={project_id}:{region}:stock-db-instance=tcp:5432"
      - "-credential_file=/secrets/credentials.json"
    volumeMounts:
      - name: cloudsql-creds
        mountPath: /secrets
        readOnly: true

volumes:
  - name: cloudsql-creds
    secret:
      secretName: cloud-sql-sa-key

From this moment on your application will connect to a local proxy(host=127.0.0.1 port=5432) to access the CloudSQL database.

DevPill 6 - SQLC: how to scaffold your database code

Raul Paes Silva — Sun, 07 Dec 2025 01:35:25 +0000

1. Install

go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest

2. Create sqlc.yaml file

Example:

version: "2"
sql:
  - schema: "db/schema/schema.sql"
    queries: "db/queries"
    engine: "postgresql"
    gen:
      go:
        package: "sqlc"
        out: "internal/infra/db/sqlc"

3. Edit query.sql to add all the queries you will use on the application

Example:

-- name: ListProducts :many
SELECT * FROM products;

-- name: GetProduct :one
SELECT * FROM products WHERE id = $1;

-- name: CreateProduct :one
INSERT INTO products (name, created_at, updated_at)
VALUES ($1, $2, $3)
RETURNING id, name, created_at, updated_at;

-- name: UpdateProduct :exec
UPDATE products
SET name = $1,
    updated_at = $2
WHERE id = $3;

-- name: DeleteProduct :exec
DELETE FROM products WHERE id = $1;

4. Last step:

sqlc generate

And you're ready to use the Go code created by SQLC!

DevPill 5 - Basic steps to document your REST API with Swagger

Raul Paes Silva — Sun, 07 Dec 2025 01:20:14 +0000

Install

go install github.com/swaggo/swag/cmd/swag@latest
go get github.com/swaggo/http-swagger
go get github.com/swaggo/swag
go get -u github.com/swaggo/http-swagger

Add the comments on the following sections of your code

(reference):

main function on main.go
webserver handlers
dtos

Tip: you can send your code to A.I. and ask to create the Swagger comments based on it.

Add the following "imports" in main.go:

import(
    _ "github.com/raulsilva-tech/devices-api/internal/docs" 
    httpSwagger "github.com/swaggo/http-swagger"
)

Add a new handler to serve the documentation on the API

Example:

mux.Handle("/swagger/", httpSwagger.WrapHandler)

The following command create de json docs in the path "internal/docs":

swag init -g cmd/api/main.go -o internal/docs

Devpill 4 - Database migrations with Migrate

Raul Paes Silva — Sun, 07 Dec 2025 00:47:42 +0000

1. Install:


    # download it
    curl -L https://github.com/golang-migrate/migrate/releases/download/v4.17.0/migrate.linux-amd64.tar.gz | tar xvz

    # copy it to go path bin
    cp -r migrate $GOPATH/bin/migrate

    # if migrate command does not work
    # add the folliwng lines to ~./bashrc
    export GOPATH=$HOME/go
    export PATH=$PATH:$GOPATH/bin

2. Command to create base files

This command will create 2 files:
init.up.sql = insert all sql commands to impact the database
init.down.sql = insert all sql commands to be executed to revert what was done in the up file


    # creating the init down and up files
    migrate create -ext=sql -dir=db/migrations -seq init

3. Execute Migration:

    # executing init.up.sql
    migrate -path=db/migrate -database "postgres://myuser:mypassword@localhost:5432/databaseName?sslmode=disable" -verbose up

    # to revert
    # execute down file to revert what was done in the up file
    migrate -path=db/migrate -database "postgres://myuser:mypassword@localhost:5432/devices-api?sslmode=disable" -verbose down

DevPill 3: How to deploy a container to your Kubernetes cluster (GKE)

Raul Paes Silva — Sat, 06 Dec 2025 22:22:08 +0000

1. Specify the deployment.yaml file.

Example:

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: api-gateway
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: api-gateway
    template:
      metadata:
        labels:
          app: api-gateway
      spec:
        containers:
          - name: api-gateway
            image: us-central1-docker.pkg.dev/<project-id>/<repository-name>/api-gateway
            ports:
              - containerPort: 8081
            resources:
              requests:
                memory: "128Mi"
                cpu: "125m"
              limits:
                memory: "128Mi"
                cpu: "125m"
            env:
              # Accessing an env variable from the app-config config map
              - name: GATEWAY_HTTP_ADDR
                valueFrom:
                  configMapKeyRef:
                    key: GATEWAY_HTTP_ADDR
                    name: app-config
              - name: JAEGER_ENDPOINT
                valueFrom:
                  configMapKeyRef:
                    key: JAEGER_ENDPOINT
                    name: app-config


  apiVersion: v1
  kind: Service
  metadata:
    name: api-gateway
  spec:
    type: LoadBalancer
    ports:
      - port: 8081
        targetPort: 8081
    selector:
      app: api-gateway

2. Access Google Cloud Console and create your GKE cluster

(this might take a while)

3. Execute the following command:

gcloud container clusters get-credentials <cluster-id> --region us-central-1 --project <project-id>

4. Upload the manifests.

Example:

kubectl apply -f infra/production/k8s/app-config.yaml
kubectl apply -f infra/production/k8s/secrets.yaml

kubectl apply -f infra/production/k8s/api-gateway-deployment.yaml

If you want to check the status of the deployments try the following commands:

kubectl get deployments
kubectl get pods

DevPill 2: How to build and push a docker image to google cloud Artifact Registry

Raul Paes Silva — Sat, 06 Dec 2025 21:02:08 +0000

1. Specify Dockerfile.

Example:

FROM golang:1.25 AS builder
WORKDIR /app
COPY . .
WORKDIR /app/services/api-gateway/cmd
RUN CGO_ENABLED=0 GOOS=linux go build -o api-gateway

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/services/api-gateway/cmd/api-gateway .
CMD ["./api-gateway"]

2. Docker "build" to create the image locally. Example:

docker build -t us-central1-docker.pkg.dev/<project-id>/<repository-name>/api-gateway:latest --platform linux/amd64 -f infra/production/docker/api-gateway.Dockerfile .

3. Push image to Google Cloud Artifact Registry:

docker push us-central1-docker.pkg.dev/<project-id>/<repository-name>/api-gateway:latest

Now your image will be available to any GCP service you would like to use.

DevPill 1: How to set your own GCP Account locally

Raul Paes Silva — Sat, 06 Dec 2025 18:35:26 +0000

Make sure you have gcloud command installed locally. Then type the following command in the terminal:

gcloud init

Select option "[2] Create a new configuration"

Select option "[3] Sign in with a new Google Account"

Click on the link provided by the command and sign in using your google account credentials

Go back to terminal and select the project you'll work on.