Changing the middleware behavior with mux

JuanPablo — Sat, 20 Mar 2021 00:20:49 +0000

TL;DR: You can change the behavior of a middleware exchanging it for another previously stored.

Probably you have used gorilla/mux.

Something that I like about mux is the possibility to add middlewares to it. You can find the description and some examples of that in the repository.

Here a middleware example based on the documentation.

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "time"

    "github.com/gorilla/mux"
)

func loggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        log.Println(r.RequestURI)

        defer func(startedAt time.Time) {
            log.Println(r.RequestURI, time.Since(startedAt))
        }(time.Now())

        next.ServeHTTP(w, r)
    })
}

func home(w http.ResponseWriter, r *http.Request) {
    json.NewEncoder(w).Encode(map[string]bool{"ok": true})
}

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", home)
    r.Use(loggingMiddleware)
    log.Fatal(http.ListenAndServe(":8080", r))
}

In that example, every request will pass through the middleware and it will show the request's URI and the request's time duration in the logs.

$ curl 0.0.0.0:8080?bar=1

2021/03/19 19:55:17 /?bar=1
2021/03/19 19:55:17 /?bar=1 17.395µs

Thanks to the middleware loggingMiddleware, you can know which requests are calling your service.

Sometimes I have needed to change the behavior to middlewares similar to the previous one. For example, do more or fewer actions. To try to explain that idea I will show another simpler middleware.

func noVerboseMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        log.Println(r.RequestURI)
        next.ServeHTTP(w, r)
    })
}

The middleware noVerboseMiddleware shows less information, it shows the URI only once, at the beginning of the request, without the second log line with the request's time duration.

$ curl 0.0.0.0:8080?bar=1

2021/03/19 20:39:12 /?bar=1

Similar to the previous one but less verbose.

I would like to exchange the behavior of them, without rebuilding the code or restart the service. For that, I will create a struct to store a map of middleware. It will give us the possibility to exchange them.

type statefulMiddleware struct {
    current     func(http.Handler) http.Handler
    middlewares map[string]func(http.Handler) http.Handler
}

func (s *statefulMiddleware) update(nextName string) error {
    next, ok := s.middlewares[nextName]
    if !ok {
        return errors.New(invalidName)
    }

    s.current = next

    return nil
}

func (s *statefulMiddleware) main(next http.Handler) http.Handler {
    return s.current(next)
}

func newStatefulMiddleware(first string, middlewares map[string]func(http.Handler) http.Handler) (*statefulMiddleware, error) {
    current, ok := middlewares[first]
    if !ok {
        return nil, errors.New(invalidName)
    }

    stateful := statefulMiddleware{current: current, middlewares: middlewares}
    return &stateful, nil
}

The function update exchanges the middlewares based on their name.

The function main will be used by mux as a normal middleware.

r.Use(s.middleware.main)

The function newStatefulMiddleware creates the middleware with a map of middlewares with their names.

    stateful, _ := newStatefulMiddleware(
        "no_verbose",
        map[string]func(http.Handler) http.Handler{
            "no_verbose": noVerboseMiddleware,
            "verbose":    loggingMiddleware})

Finally, a new handler in the server to choose which middleware I prefer. It will change the middleware base on a JSON message.

func (s *service) config(w http.ResponseWriter, r *http.Request) {
    options := map[string]string{}

    err := json.NewDecoder(r.Body).Decode(&options)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    option, ok := options["option"]
    if !ok {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    err = s.middleware.update(option)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    json.NewEncoder(w).Encode(map[string]bool{"changed": true})
}

With all that code, I can exchange the middleware in real-time.

First a request

$ curl 0.0.0.0:8080?bar=1

To get a verbose log, it was generated by loggingMiddleware.

2021/03/19 20:49:41 /?bar=1
2021/03/19 20:49:41 /?bar=1 12.016µs

Changing the middleware

$ curl -d '{"option": "no_verbose"}' 0.0.0.0:8080/config

A new request

$ curl 0.0.0.0:8080?bar=1

To get a less verbose log, it was generated by noVerboseMiddleware.

2021/03/19 20:51:43 /?bar=1

All the code put together

package main

import (
    "encoding/json"
    "errors"
    "log"
    "net/http"
    "time"

    "github.com/gorilla/mux"
)

const invalidName = "invalid middleware name"

type statefulMiddleware struct {
    current     func(http.Handler) http.Handler
    middlewares map[string]func(http.Handler) http.Handler
}

func (s *statefulMiddleware) update(nextName string) error {
    next, ok := s.middlewares[nextName]
    if !ok {
        return errors.New(invalidName)
    }

    s.current = next

    return nil
}

func (s *statefulMiddleware) main(next http.Handler) http.Handler {
    return s.current(next)
}

func newStatefulMiddleware(first string, middlewares map[string]func(http.Handler) http.Handler) (*statefulMiddleware, error) {
    current, ok := middlewares[first]
    if !ok {
        return nil, errors.New(invalidName)
    }

    stateful := statefulMiddleware{current: current, middlewares: middlewares}
    return &stateful, nil
}

func noVerboseMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        log.Println(r.RequestURI)
        next.ServeHTTP(w, r)
    })
}

// loggingMiddleware example from https://github.com/gorilla/mux#examples
func loggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        log.Println(r.RequestURI)

        defer func(startedAt time.Time) {
            log.Println(r.RequestURI, time.Since(startedAt))
        }(time.Now())

        next.ServeHTTP(w, r)
    })
}

type service struct {
    middleware *statefulMiddleware
}

func (s *service) home(w http.ResponseWriter, r *http.Request) {
    json.NewEncoder(w).Encode(map[string]bool{"ok": true})
}

func (s *service) config(w http.ResponseWriter, r *http.Request) {
    options := map[string]string{}

    err := json.NewDecoder(r.Body).Decode(&options)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    option, ok := options["option"]
    if !ok {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    err = s.middleware.update(option)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    json.NewEncoder(w).Encode(map[string]bool{"changed": true})
}

func main() {
    stateful, _ := newStatefulMiddleware(
        "no_verbose",
        map[string]func(http.Handler) http.Handler{
            "no_verbose": noVerboseMiddleware,
            "verbose":    loggingMiddleware})

    s := service{stateful}

    r := mux.NewRouter()
    r.HandleFunc("/config", s.config)
    r.HandleFunc("/", s.home)
    r.Use(s.middleware.main)

    log.Fatal(http.ListenAndServe(":8080", r))
}

The source is available in this repository https://github.com/juanpabloaj/stateful_middleware_example

Some disclaimers:

I didn't add some things like a mutex to protect the map, because I tried to reduce the number of lines of the example.
It example could be made with different levels of the log but it was the simplest example that I could create to explain the idea.

What do you think of this approach?

Do you have another way to change the behavior of middleware?

Any suggestions or commentaries are welcome.

Recovering OSX disk space

JuanPablo — Sat, 23 Jan 2021 14:15:34 +0000

Warning: Be careful about these commands, before applying these, remember, you are deleting files and data. Be aware of what are you deleting. You could lose data forever.

I wrote this post because when you are developing you depend on tools, dependencies, docker images, etc. And in some moment your disk is full.

Using the next commands you can recover some disk space used.

Recover some disk space used by:

Brew

brew cleanup

Docker

If you are using docker, check docker's directory

du --max-depth=1 -h ~/Library/Containers/com.docker.docker/Data

If it is using too much space you could apply a prune. Remember this command will delete docker's images and volumes.

If you have important data, back up it before.

docker system prune -a --volumes

Cache directories

Check your Cache directory and delete some of the directories that use more disk space (in my case those were Spotify, pip, and Cypress).

du --max-depth=1 -h ~/Library/Caches/ | sort -hk1

I hope that these commands could be useful for recovering some space on your disk.

Do you have other tips to recover disk space on OSX? please add those in the commentaries.

Ordering engineering-blogs entries by votes

JuanPablo — Sat, 03 Aug 2019 16:05:31 +0000

TL;DR

A web to order the RSS entries contained in the engineering blogs list

https://github.com/kilimchoi/engineering-blogs

this is the web

https://entries-by-votes.now.sh/

Source code

https://github.com/juanpabloaj/entries-by-votes

Short description

If you are like me, probably as part of your daily routine you check some engineering blogs. A few months ago I found this repo with an amazing list of blogs.

https://github.com/kilimchoi/engineering-blogs

It is great, but how to identify the most interesting (or popular posts)? well, I remembered this article related to how Hacker News ordered his posts.

https://medium.com/hacking-and-gonzo/how-hacker-news-ranking-algorithm-works-1d9b0cf2c08d

Score = (P-1) / (T+2)^G

where,
P = points of an item (and -1 is to negate submitters vote)
T = time since submission (in hours)
G = Gravity, defaults to 1.8 in news.arc

With that info, a few lines of python/tornado to get votes from Hacker News and Reddit, and the help of our friends of zeit.co/now (as serverless deployments service), the dinner has been served.

Peace.