Forem: baize

Web Development JWT Practices

baize — Thu, 17 Nov 2022 08:42:15 +0000

Foreword

The previous post briefly introduced a high-performance Go HTTP framework - Hertz. This article is based on a demo from the Hertz open-source repository, which describes how to use Hertz to complete the JWT authentication and authorization process.

It should be noted here that hertz-jwt is one of many external extensions to Hertz, it is forked from gin-jwt and adapted to Hertz.

Hertz's rich extension ecosystem is a great convenience for developers and worth exploring beyond this article.

Introduction

Use hz to generate code
Use JWT to complete login and authentication
Use Gorm and MySQL

Download

git clone https://github.com/cloudwego/hertz-examples.git
cd bizdemo/hertz_jwt

Architecture

hertz_jwt
├── Makefile # Generate hertz scaffolding code using the hz command line tool
├── biz
│   ├── dal
│   │   ├── init.go 
│   │   └── mysql
│   │       ├── init.go # Initialising database connections
│   │       └── user.go # Database operations
│   ├── handler
│   │   ├── ping.go
│   │   └── register.go # Register handler
│   ├── model
│   │   ├── sql
│   │   │   └── user.sql
│   │   └── user.go # Defining the database model
│   ├── mw
│   │   └── jwt.go # Initialising the hertz-jwt middleware
│   ├── router
│   │   └── register.go
│   └── utils
│       └── md5.go # MD5
├── docker-compose.yml # MySQL container
├── go.mod
├── go.sum
├── main.go # Hertz startup functions
├── readme.md
├── router.go # Routing Registration
└── router_gen.go

Analysis

The list of interfaces for this demo is as follows.

// customizeRegister registers customize routers.
func customizedRegister(r *server.Hertz) {
    r.POST("/register", handler.Register)
    r.POST("/login", mw.JwtMiddleware.LoginHandler)
    auth := r.Group("/auth", mw.JwtMiddleware.MiddlewareFunc())
    auth.GET("/ping", handler.Ping)
}

User Registration

The user data of the current demo is persisted through mysql by gorm, so before logging in, the user needs to be registered. The registration process is as follows.

get the username, password and email address
determine if the user exists
create the user

User log-in (Authentication)

The server needs to validate the user account and password and sign a jwt token when the user logs in for the first time.

JwtMiddleware, err = jwt.New(&jwt.HertzJWTMiddleware{
    Key:           []byte("secret key"),
    Timeout:       time.Hour,
    MaxRefresh:    time.Hour,
    Authenticator: func(ctx context.Context, c *app.RequestContext) (interface{}, error) {
        var loginStruct struct {
            Account  string `form:"account" json:"account" query:"account" vd:"(len($) > 0 && len($) < 30); msg:'Illegal format'"`
            Password string `form:"password" json:"password" query:"password" vd:"(len($) > 0 && len($) < 30); msg:'Illegal format'"`
        }
        if err := c.BindAndValidate(&loginStruct); err != nil {
            return nil, err
        }
        users, err := mysql.CheckUser(loginStruct.Account, utils2.MD5(loginStruct.Password))
        if err != nil {
            return nil, err
        }
        if len(users) == 0 {
            return nil, errors.New("user already exists or wrong password")
        }

        return users[0], nil
    },
    PayloadFunc: func(data interface{}) jwt.MapClaims {
        if v, ok := data.(*model.User); ok {
            return jwt.MapClaims{
                jwt.IdentityKey: v,
            }
        }
        return jwt.MapClaims{}
    },
})

Authenticator: A function that sets up the user information to be authenticated when logging in. The demo defines a loginStruct structure that receives the user login information and authenticates the validity. The return value of this function, users[0], will provide the payload data source for the subsequent generation of the jwt token.
PayloadFunc: Its input is the return value of Authenticator, which is responsible for parsing users[0] and injecting the username into the payload part of the token.
Key: specifies the key used to encrypt the jwt token as "secret key".
Timeout: specifies that the token is valid for one hour.
MaxRefresh: sets the maximum token refresh time, allowing the client to refresh the token within TokenTime + MaxRefresh, appending a Timeout duration.

Return of Token

JwtMiddleware, err = jwt.New(&jwt.HertzJWTMiddleware{
    LoginResponse: func(ctx context.Context, c *app.RequestContext, code int, token string, expire time.Time) {
        c.JSON(http.StatusOK, utils.H{
            "code":    code,
            "token":   token,
            "expire":  expire.Format(time.RFC3339),
            "message": "success",
        })
    },
})

LoginResponse: After a successful login, jwt token information is returned with the response, you can customize the content of this part, but be careful not to change the function signature as it is strongly bound to LoginHandler.

Token Validation

When a client requests a route with a jwt middleware configured, the server verifies the jwt token it carries.

JwtMiddleware, err = jwt.New(&jwt.HertzJWTMiddleware{
    TokenLookup:   "header: Authorization, query: token, cookie: jwt",
    TokenHeadName: "Bearer",
    HTTPStatusMessageFunc: func(e error, ctx context.Context, c *app.RequestContext) string {
        hlog.CtxErrorf(ctx, "jwt biz err = %+v", e.Error())
        return e.Error()
    },
    Unauthorized: func(ctx context.Context, c *app.RequestContext, code int, message string) {
        c.JSON(http.StatusOK, utils.H{
            "code":    code,
            "message": message,
        })
    },
})

TokenLookup: This is used to set the source of the token, you can choose header, query, cookie, or param, the default is header:Authorization, the first one read from the left side takes precedence. The current demo will use header as the data source, so when accessing the /ping interface, you will need to store the token information in the HTTP Header.
TokenHeadName: This is used to set the prefix used to retrieve the token from the header, the default is "Bearer".
HTTPStatusMessageFunc: This is used to set the error message that will be included in the response when an error occurs in the jwt validation process, you can wrap these yourself.
Unauthorized: used to set the response function for a failed jwt validation process, the current demo returns the error code and error message.

Extracting User Information

JwtMiddleware, err = jwt.New(&jwt.HertzJWTMiddleware{
    IdentityKey: IdentityKey,
    IdentityHandler: func(ctx context.Context, c *app.RequestContext) interface{} {
        claims := jwt.ExtractClaims(ctx, c)
        return &model.User{
            UserName: claims[IdentityKey].(string),
        }
    },
})

// Ping .
func Ping(ctx context.Context, c *app.RequestContext) {
    user, _ := c.Get(mw.IdentityKey)
    c.JSON(200, utils.H{
        "message": fmt.Sprintf("username:%v", user.(*model.User).UserName),
    })
}

IdentityHandler: The function used to set the identity information to be retrieved. In the demo, here the payload of the token is extracted and the username is stored in the context information with the IdentityKey.
IdentityKey: sets the key used to retrieve the identity, the default is "identity".
Ping: Constructs the response. Retrieves the username information from the context information and returns it.

Other Components

Code Generation

Most of the code above is scaffolded code generated through the hz command line tool, so developers don't need to spend a lot of time building a good code structure and just focus on writing the business.

hz new -mod github.com/cloudwego/hertz-examples/bizdemo/hertz_jwt

Sample Code（Sourced from hz Official Document）：

// idl/hello.thrift
namespace go hello.example

struct HelloReq {
    1: string Name (api.query="name"); // api annotations to facilitate parameter binding
}

struct HelloResp {
    1: string RespBody;
}

service HelloService {
    HelloResp HelloMethod(1: HelloReq request) (api.get="/hello");
}

// execute under $GOPATH
hz new -idl idl/hello.thrift

Parameter Binding

Hertz uses the open-source library go-tagexpr for parameter binding and validation, which the current demo uses for user registration and login.

// register
var registerStruct struct {
    // Binding and validation of parameters by declaring tags
    Username string `form:"username" json:"username" query:"username" vd:"(len($) > 0 && len($) < 128); msg:'Illegal format'"`
    Email    string `form:"email" json:"email" query:"email" vd:"(len($) > 0 && len($) < 128) && email($); msg:'Illegal format'"`
    Password string `form:"password" json:"password" query:"password" vd:"(len($) > 0 && len($) < 128); msg:'Illegal format'"`
}

// login
var loginStruct struct {
    // Binding and validation of parameters by declaring tags
    Account  string `form:"account" json:"account" query:"account" vd:"(len($) > 0 && len($) < 30); msg:'Illegal format'"`
    Password string `form:"password" json:"password" query:"password" vd:"(len($) > 0 && len($) < 30); msg:'Illegal format'"`
}
if err := c.BindAndValidate(&loginStruct); err != nil {
    return nil, err
}

vd: used to validate the data format, e.g. string length checks (128), email type checks (email)
form: binds the body content of the request. content-type -> multipart/form-data or application/x-www-form-urlencoded, binds the key-value of the form
json: bind the body content of the request content-type -> application/json, bind the json parameters
query: binds the query parameter of the request

The parameter bindings need to be in accordance with the priority.

path > form > query > cookie > header > json > raw_body

Further usage can be found in Documentation

Gorm

For more information on Gorm's operation of MySQL, please refer to Gorm

Run

Running the mysql container

cd bizdemo/hertz_jwt && docker-compose up

Creating a mysql database

After connecting to mysql, execute user.sql

run the demo

cd bizdemo/hertz_jwt && go run main.go

API Requests

Register

# request
curl --location --request POST 'localhost:8888/register' \
--header 'Content-Type: application/json' \
--data-raw '{
    "Username": "admin",
    "Email": "admin@test.com",
    "Password": "admin"
}'
# response
{
    "code": 200,
    "message": "success"
}

Login

# request
curl --location --request POST 'localhost:8888/login' \
--header 'Content-Type: application/json' \
--data-raw '{
    "Account": "admin",
    "Password": "admin"
}'
# response
{
    "code": 200,
    "expire": "2022-11-16T11:05:24+08:00",
    "message": "success",
    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Njg1Njc5MjQsImlkIjoyLCJvcmlnX2lhdCI6MTY2ODU2NDMyNH0.qzbDJLQv4se6dOHN51p21Rp3DjV1Lf131l_5k4cK6Wk"
}

Routing Access

# request
curl --location --request GET 'localhost:8888/auth/ping' \
--header 'Authorization: Bearer ${token}'
# response
{
    "message": "username:admin"
}

References

Source Code Analysis for Go HTTP Framework Hertz

baize — Sun, 30 Oct 2022 04:16:40 +0000

CloudWeGo-Hertz

Hertz [həːts] is a high-performance, high-usability, extensible HTTP framework for Go. It’s designed to simplify building microservices for developers.

Hertz was inspired by other open-source frameworks like fasthttp, gin, and echo, in combination with unique challenges faced by ByteDance, Hertz has become production ready and powered ByteDance’s internal services over the years.

Nowadays, as Go gain popularity in microservice development, Hertz will be the right choice if you are looking for a customizable, high-performance framework to support a variety of use case.

This article will explain the structure of the Hertz source code and analyze the startup of Hertz.

Next, you need to open the official documentation for Hertz and clone the repository locally, let's get started.

Architecture

Above is an architectural design diagram from the official Hertz documentation. The components in the picture correspond to the package folders within the hertz source code package.

Getting Started

Following the instructions in the documentation, we could initialize a minimal hertz project via the hertz cmd tool.

# Install the command line tool for hertz, used to generate the initial hertz code
go install github.com/cloudwego/hertz/cmd/hz@latest
# Generate code via the hz tool. If the project created is not in the GOPATH/src path, "-module" needs to be declared
hz new -module hertz-study

Run the project then you can access the default HTTP service by /ping.

curl http://127.0.0.1:8888/ping
# response
{"message":"pong"}%

Source Code Analysis

server

Look at the main.go function, which is the startup entry for the hertz service. It initializes a default hertz service, does some registration work, and runs the hertz service (HTTP service).

package main

import (
    "github.com/cloudwego/hertz/pkg/app/server"
)

func main() {
    h := server.Default()

    register(h)
    h.Spin()
}

Thinking back to the interface about http://127.0.0.1:8888/ping which was declared by server.Default().

Conversely, if you want to specify custom configurations for HTTP service, you need to pass parameters to the server.Default(), Or using the server.New() method.

server.Default()

// Default creates a hertz instance with default middlewares.
func Default(opts ...config.Option) *Hertz {
   h := New(opts...)
   h.Use(recovery.Recovery())

   return h
}

// New creates a hertz instance without any default config.
func New(opts ...config.Option) *Hertz {
    options := config.NewOptions(opts)
    h := &Hertz{
        Engine: route.NewEngine(options),
    }
    return h
}

Looking at the Default() method, we find it calls server.New() internally, which accepts an Option array of indeterminate length as a parameter.

// Option is the only struct that can be used to set Options.
type Option struct {
    F func(o *Options)
}

// New creates a hertz instance without any default config.
func New(opts ...config.Option) *Hertz {
    options := config.NewOptions(opts)
    h := &Hertz{
        Engine: route.NewEngine(options),
    }
    return h
}

Then we go to the config.NewOptions method to see how this Option slice will apply our custom content to the Hertz configuration.

func NewOptions(opts []Option) *Options {
    options := &Options{
        KeepAliveTimeout: defaultKeepAliveTimeout,
        ReadTimeout: defaultReadTimeout,
        IdleTimeout: defaultReadTimeout,
        RedirectTrailingSlash: true,
        RedirectFixedPath: false,
        HandleMethodNotAllowed: false,
        UseRawPath: false,
        RemoveExtraSlash: false,
        UnescapePathValues: true,
        DisablePreParseMultipartForm: false,
        Network: defaultNetwork,
        Addr: defaultAddr,
        MaxRequestBodySize: defaultMaxRequestBodySize,
        MaxKeepBodySize: defaultMaxRequestBodySize,
        GetOnly: false,
        DisableKeepalive: false,
        StreamRequestBody: false,
        NoDefaultServerHeader: false,
        ExitWaitTimeout: defaultWaitExitTimeout,
        TLS: nil,
        ReadBufferSize: defaultReadBufferSize,
        ALPN: false,
        H2C: false,
        Tracers: []interface{}{},
        TraceLevel: new(interface{}),
        Registry: registry.NoopRegistry,
    }
    // apply a custom []Option to Options
    options.Apply(opts)
    return options
}

func (o *Options) Apply(opts []Option) {
    for _, op := range opts {
        op.F(o)
    }
}

By looking at the config.NewOptions source code, it first initializes an Options structure that holds various initialization information for the Hertz service. The Options properties are fixed by default until the options.Apply(options) method is called to apply the custom configuration.

It passes a pointer to the Options structure created by default as an argument to the F method of each Option you declared. It assigns values to the Options structure via the F method call, which, because it is a pointer, will naturally apply all the assignments to the same Options.

And how the specific Option's F method is defined can be implemented flexibly, which is one of the reasons why Hertz has good scalability.

// Default creates a hertz instance with default middlewares.
func Default(opts ...config.Option) *Hertz {
    // "Hertz" is the core structure of the framework
    h := New(opts...)
    h.Use(recovery.Recovery())

    return h
}

Notice that there is also a h.Use(recovery.Recovery()) method.

// Recovery returns a middleware that recovers from any panic.
func Recovery(opts ...Option) app.HandlerFunc {
    cfg := newOptions(opts...)

    return func(c context.Context, ctx *app.RequestContext) {
        defer func() {
            if err := recover(); err != nil {
                stack := stack(3)

                cfg.recoveryHandler(c, ctx, err, stack)
            }
        }()
        ctx.Next(c)
    }
}

This is a middleware for recovering from panic. By default, it will print the time, content, and stack information of the error and write a 500. Overriding the Config configuration, you can customize the error printing logic.

By the way, if you Initialize a Hertz HTTP service by server.New(), which won't hold the Recovery() method, and you need to inject it yourself.

package main

import (
    "context"

    "github.com/cloudwego/hertz/pkg/app"
    "github.com/cloudwego/hertz/pkg/app/middlewares/server/recovery"
    "github.com/cloudwego/hertz/pkg/app/server"
    "github.com/cloudwego/hertz/pkg/common/hlog"
    "github.com/cloudwego/hertz/pkg/protocol/consts"
)

func main() {
    h := server.New()

    h.Use(recovery.Recovery(
        recovery.WithRecoveryHandler(func(c context.Context, ctx *app.RequestContext, err interface{}, stack []byte) {
            hlog.SystemLogger().CtxErrorf(c, "[Recovery] err=%v\nstack=%s", err, stack)
            ctx.AbortWithStatus(consts.StatusInternalServerError)
        })))
    register(h)
    h.Spin()
}

register()

package main

import (
    "github.com/cloudwego/hertz/pkg/app/server"
)

func main() {
    h := server.Default()

    register(h)
    h.Spin()
}

After the above analysis, we know that server.Default() completes the declaration of the Hertz structure. Let's pay more attention to the content of the register(h).

// register registers all routers.
func register(r *server.Hertz) {

    router.GeneratedRegister(r)

    customizedRegister(r)
}

// GeneratedRegister registers routers generated by IDL.
func GeneratedRegister(r *server.Hertz) {
    //INSERT_POINT: DO NOT DELETE THIS LINE!
}

// customizeRegister registers customize routers.
func customizedRegister(r *server.Hertz) {
    r.GET("/ping", handler.Ping)

    // your code ...
}

register(h) does two types of route registration internally. The annotation of GeneratedRegister() indicates that this part of the route is generated by the IDL.

customizedRegister() is used to register custom routes which initializes a /ping by default, using it in a very similar way to gin.

Spin()

Finally, let's analyse the third part of the main() method, the h.Spin().

// Spin runs the server until catching os.Signal or error returned by h.Run().
func (h *Hertz) Spin() {
    errCh := make(chan error)
    h.initOnRunHooks(errCh)
    go func() {
        // core
        errCh <- h.Run()
    }()

    signalWaiter := waitSignal
    if h.signalWaiter != nil {
        signalWaiter = h.signalWaiter
    }

    if err := signalWaiter(errCh); err != nil {
        hlog.Errorf("HERTZ: Receive close signal: error=%v", err)
        if err := h.Engine.Close(); err != nil {
            hlog.Errorf("HERTZ: Close error=%v", err)
        }
        return
    }

    hlog.Infof("HERTZ: Begin graceful shutdown, wait at most num=%d seconds...", h.GetOptions().ExitWaitTimeout/time.Second)

    ctx, cancel := context.WithTimeout(context.Background(), h.GetOptions().ExitWaitTimeout)
    defer cancel()

    if err := h.Shutdown(ctx); err != nil {
        hlog.Errorf("HERTZ: Shutdown error=%v", err)
    }
}

After a series of initialization and declaration operations, Spin() is responsible for triggering the run of Hertz and handling any errors during the runtime. The most important step is errCh <- h.Run().

func (engine *Engine) Run() (err error) {
    if err = engine.Init(); err != nil {
        return err
    }

    if !atomic.CompareAndSwapUint32(&engine.status, statusInitialized, statusRunning) {
        return errAlreadyRunning
    }
    defer atomic.StoreUint32(&engine.status, statusClosed)

    // trigger hooks if any
    ctx := context.Background()
    for i := range engine.OnRun {
        if err = engine.OnRun[i](ctx); err != nil {
            return err
        }
    }

    return engine.listenAndServe()
}

Then you see the engine.listenAndServe() method at the end, which is declared in an interface. Looking at its implementation classes, you find that it can be traced back to the standard and netpoll packages.

As an HTTP framework, the most important thing is to provide network communication capabilities. hertz uses the pluggable network library netpoll to handle network communication and further optimize performance.

The service is now running, and you can send requests via the console:

curl http://127.0.0.1:8888/ping
{"message":"pong"}%

Summary

After generating the simplest Hertz code using the hz tool, this article roughly analyses the contents of the main method, dividing it into three parts, the service configuration declaration Default(), the route registration register() and the HTTP service start Spin().

If you want to know more about Hertz, you could visit the Reference.

Forem: baize

Web Development JWT Practices

Foreword

Introduction

Download

Architecture

Analysis

User Registration

User log-in (Authentication)

Return of Token

Token Validation

Extracting User Information

Other Components

Code Generation

Parameter Binding

Gorm

Run

API Requests

Register

Login

Routing Access

References

Source Code Analysis for Go HTTP Framework Hertz

CloudWeGo-Hertz

Architecture

Getting Started

Source Code Analysis

server

server.Default()

register()

Spin()

Summary

Reference List