Forem: Rizky Darmawan

Basic Concept & Benefit of API Gateway

Rizky Darmawan — Thu, 27 Mar 2025 04:52:06 +0000

Photo by Javier Allegue Barros on Unsplash

Api gateways are an important component in modern software architecture, especially in systems that implement microservices. Api Gateway acts as the main gateway to receive all incoming API requests. With this role, API Gateway simplifies API management and improves overall system performance and security.

Using API Gateway helps build systems that are scalable and easy to maintain. As system complexity increases, API Gateway plays a critical role in managing effective integration and communication between various backend services and clients.

In this article, we will learn basic concepts and benefits from using API Gateway. The hope is that this article can provide new understanding or strengthen the insights you already have.

What is API Gateway ?

An API Gateway is an API management tool that sits between a client and a collection of backend services. It acts as an intermediary, routing client requests to the appropriate backend service required to fulfill them and then returning the corresponding response Additionally, an API Gateway provides several important features such as load balancing, circuit breaking, logging, authentication, and caching, making it a crucial component in modern API architecture.

In today's fast-paced world of digital development, efficiency is everything, which is why API Gateways have become essential in modern software projects. Acting as a central point of control for API across microservices architecture handling thousands of concurrent API calls efficiently.

How does API Gateway work ?

Api Gateway work with receive request from internal and external, called "API Calls". API Gateway as software layer that manage traffic for routes them ti appropriate API and then delver the responses to the particular user or devic thath made the request.

This the explain API Gateway Workflow from image above:

A client (browser, mobile app, or another service) sends an API request.
The API Gateway processes the request with routes the request to the correct backend service.
The backend service processes the request and sends a response.
The API Gateway modifies or enhances the response.
Finally, the response is returned to the client.

Benefits of API Gateway

Using an API Gateway can bring several benefits such as enhance performance, improve security, easier maintenance of multiple backend services. Here are some key benefits of using an API Gateway:

1. Enhanced security

An API Gateway enforce authentication and authorization policies, ensuring that only authorized users can access backend services. Additionally, it can help mitigate security threats through:

Rate limiting to prevent API abuse and DDoS attacks.
IP whitelisting and blacklisting to restrict access.
TLS encryption to secure communication between clients and services.

2. Better monitoring and visibility

An API gateway can collect metrics and other data about the requests and responses, providing valuable insights into the performance and behavior of the system. This can help to identify and diagnose problems, and improve the overall reliability and resilience of the system.

3. Data Format Transformation

An API Gateway can convert request and response or data data format (e.g., JSON to XML). Making it easier to manage data formats when needed.

4. API Versioning and Backward Compatibility

An API Gateway can manage multiple versions of an API, allowing developers to introduce new features or make changes without breaking existing clients. This enables a smoother transition for clients and reduces the risk of service disruptions.

5. Enhanced Error Handling

An API Gateway can provide a consistent way to handle errors and generate error responses, improving the user experience and making it easier to diagnose and fix issues.

Drawbacks of Impelementation API Gateway

In addition to having many benefits that we get when we implement API Gateway, on the other hand we also have disadvantages in using it, here are some points:

1. Single Point of Failure

Since the API Gateway acts as a central access point for all API requests, it can become a single point of failure if not properly managed. If the gateway experiences downtime, it could disrupt the entire API ecosystem.

2. Additional Complexity

Integrating an API Gateway introduces an additional layer of complexity to your architecture. It requires careful configuration, monitoring, and maintenance. If not managed efficiently, it can slow down development and increase operational overhead.

3. Increased Latency

An API Gateway processes incoming requests before forwarding them to the appropriate services, which can introduce latency. This delay may impact overall system performance, especially if multiple processing steps (such as authentication, logging, and rate limiting) are involved.

4. Cost

Operating an API Gateway, particularly in high-traffic environments, can add significant costs to your infrastructure. Expenses may include hosting, licensing fees, or managed API Gateway services from cloud providers.

Simple Example of an API Gateway

To implement an API Gateway, you can use a popular API Gateway such as Nginx, Kong, KrakenD, and many more popular choices. However, our goal for the topic is to understand how API Gateway works. So, we can understand by simply building a simple API Gateway using golang. Let's begin.

For the complete code you can see following repository: Click me

The structure Folder

We can create a simply the API Gateway following the structure folder:

├── Makefile
├── api_gateway
│   ├── go.mod
│   ├── go.sum
│   └── main.go
├── service_cart
│   ├── go.mod
│   ├── go.sum
│   └── main.go
├── service_order
│   ├── go.mod
│   ├── go.sum
│   └── main.go
└── service_product
    ├── go.mod
    ├── go.sum
    └── main.go

The rate limitter function code

First, we can create a file with the name main.go in the api_gateway folder which contains:

package main

import (
    "time"

    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/fiber/v2/middleware/limiter"
)

func rateLimiter() fiber.Handler {
    return limiter.New(limiter.Config{
        Expiration: time.Second * 60,
        Max:        5,
    })

}

The rateLimiter() function is used to limit the number of requests per client. Where this rate limiter is useful for Preventing abuse & DDoS, Controlling API traffic, Protecting backend services from overload.

With the configuration Expiration: time.Second, Max: 5, each client can only make 5 requests per second. If this limit is reached, the client will receive error HTTP 429 Too Many Requests. After 1 second, the limit will be reset and the client can send requests again.

The proxy function code

Second, we also create a function with the name reverseProxy:


package main

import (
    // ... other import libraries
        "log"
        "github.com/gofiber/fiber/v2/middleware/proxy"
)

// ... rate limitter function

func reverseProxy(target string) fiber.Handler {
    return func(c *fiber.Ctx) error {
        url := target + c.OriginalURL()
        return proxy.Do(c, url)
    }
}

The proxy(target string) function is used to forward requests (proxy requests) from clients to other backend servers specified in the target. This function returns a middleware handler (fiber.Handler) that can be used on Fiber.

We also see c.OriginalURL() returns the path and query parameters of the original request sent by the client. For example, if the client accesses http://localhost:3000/api/orders?id=10, then:

/api/orders?id=10

And last we return proxy.Do(c, url) is used to forward requests to the destination url. This url is a combination of target and c.OriginalURL(), so all incoming requests will be directed to another server with the same path. In this way, requests that enter the API Gateway will be automatically directed to the appropriate backend service without the client knowing.

Then main function code

And latest file we create main function, is a entrypoint of this application. In this case, it acts as an API Gateway. The code is like this:

package main

import (
    // ... other import libraries
        "github.com/gofiber/fiber/v2/middleware/logger"
)


func main() {
    app := fiber.New()

    app.Use(logger.New())

    // Implement rate limitter
    app.Use(rateLimiter())

    // Implement proxy
    // Redirect request to service product
    app.Use("/product", reverseProxy("http://localhost:5001"))

    // Redirect request to service order
    app.Use("/order", reverseProxy("http://localhost:5002"))

    // Redirect request to service cart
    app.Use("/cart", reverseProxy("http://localhost:5003"))

    port := ":3000"
    log.Printf("Starting API Gateway in port %s", port)
    log.Fatal(app.Listen(port))
}

We can focus on the code part:

    // Implement rate limitter
    app.Use(rateLimiter())

This code applies rate limiting to the entire application with Fiber's limiter middleware. This middleware is attached with app.Use(rateLimiter()), which means it will apply to all routes. The function rateLimiter() returns the rate limiter middleware we created earlier with limiter.New().

And also for some of these code:

    // Implement proxy
    // Redirect request to service product
    app.Use("/product", reverseProxy("http://localhost:5001"))

    // Redirect request to service order
    app.Use("/order", reverseProxy("http://localhost:5002"))

    // Redirect request to service cart
    app.Use("/cart", reverseProxy("http://localhost:5003"))

This code uses the reverseProxy function we created earlier, which will forward the request to the appropriate service.

As we know in the The structure Folder section, we also have several service folders that will be directed according to incoming requests from clients.

For the complete code you can see following repository: Click me

Let's run it

To run it there are several ways, if Makefile is installed on your computer you only need to run the command below in the terminal on root folder project:

make run

If not you can run it one by one by going into all folders and starting to run with the command:

cd service_product && go run main.go

cd service_order && go run main.go

cd service_cart && go run main.go

cd api_gateway && go run main.go

If successful, you can see display like this:

From the image above you can see several services running on their respective ports.

Service API Gateway running on port 3000

Service product running on port 5001

Service order running on port 5002

Service cart running on port 5003

Here is the advantage of using API Gateway, imagine you have more than 3 services and on the client we have several features that must be run simultaneously. Do we have to define the url with each specific port? of course it is very troublesome. And later one of the services must change its url address, of course it is even more troublesome to have to refactor here and there on the client side.

But with API Gateway you only need to know the url on the API Gateway and define the appropriate url, let's try to access several services via the API Gateway url only. Run this command:

curl http://localhost:3000/product/1
curl http://localhost:3000/order
curl http://localhost:3000/cart

And the result is you can access multiple services with just one URL.

And in the log you can see that the api gateway actually forwards the request to the service that corresponds to the address being called.

And I give a challenging task, try to make more requests than the maximum rate limiter that we have set in the rateLimiter function until you get the error HTTP 429 Too Many Requests.

Conclusion

API Gateway acts as the primary gateway to manage request traffic in a microservice system. With rate limiting, APIs are more secure from abuse such as DDoS. Reverse proxies allow requests to be directed to the appropriate backend service without changing the client side, increasing flexibility and ease of management. In addition, API Gateway also supports caching, logging, monitoring, and other security features such as authentication & authorization. With proper implementation, API Gateway can improve the security, performance, and scalability of the overall system.

However, implementing API Gateway also has some drawbacks that we must be aware of such as a single point of failure, additional complexity, increased latency, and higher operational costs. Despite these drawbacks, a well-implemented API Gateway remains essential for scalable and secure microservice architectures.

Hopefully this article can help to improve understanding or recall what is already known. If you have additions or corrections to the discussion above, let's discuss it in the comments column. Hopefully it helps 👋.

Reading References

Connection Pool in Backend Development: Basic Concept, Benefits, and Implementation

Rizky Darmawan — Thu, 06 Feb 2025 02:43:38 +0000

Photo by Christine Tutunjian on Unsplash

Connection Pooling is a mechanism that creates and manages a pool of database connections that can be used by applications. This concept is important in managing connections to the database with the aim of optimizing resource use and improving the performance of applications that frequently interact with the database.

Instead of creating a new connection every time it is needed (which is expensive in terms of time and resources), connection pools allow applications to borrow/use existing connections and return them to the pool when they are finished using them. That's why it's called connection pool.

Why connection pool is important ?

The important question is why is connection polling important? why not just make 1 connection to be used alternately. Let's discuss 1 by 1 why connection pooling is important.

Better performance

Opening a new connection to the database can take time because it goes through several processes such as authentication, network settings and so on. With a connection pool, applications can avoid this process because connections are already available if needed.

Resource savings

Without a pool, if the application gets 100 requests, the application will create 100 new connections. This can burden the database server which is always busy carrying out the initialization process. The connection pool can limit the number of connections used simultaneously, this keeps the database server load stable.

Scalability

Connection pooling helps applications manage load well when the application scales up. With good connection pooling settings, we can handle a large number of requests coming at the same time and ensure the application remains responsive.

Load configuration

The connection pool can be configured with a maximum number of connections according to needs. So it can prevent applications from flooding connections to the database.

How to Works ?

Now that we understand why connection pooling is important to use, we will now discuss how connection polling works.

Pool Initialization

The first time an application is run, it makes a certain number of connections to the database (for example, 10 connections). This number is called Pool size. This connection will be idling waiting to be used.

Connection usage

When the application needs a connection to the database it no longer creates a new connection, instead it requests a connection from the pool. If a connection is available in the pool, it will be assigned to the application. If no connections are available, the application may have to wait until a connection is released and returns to the pool after being used by another process.

Restore connection

Once an application has finished using a connection, it will not close the connection. Instead, the connection is returned to the pool to be used again by the next request.

The common main parameter in connection pool

The following are some of the main parameters that are commonly used when creating a connection pool. I will try to explain in detail followed by analogies that can help us understand each context better.

Pool Size

We can determine the number of connections provided in Pool Size. If all connections are used then when there is a request to use the connection it will queue up waiting for an available connection or it will fail if the queue is full.

For example, if we set the max pool size to 10, only 10 active connections can be used by the application. If an 11th request comes, this request will wait for a connection from the pool that is already in use. The analogy is like this:

Imagine there is a parking lot, if all the slots are full, a new car that wants to enter (connection request) has to wait until another car comes out (idle connection).
If we want to accommodate more cars, we have to build more parking slots (increase MaxPoolSize) so that we can accommodate more cars. But there are increased maintenance costs (server resources).

Minimum Connections

Minimum connection is the minimum number of connections that will be created the first time the application is run. This is a connection that will always be on standby waiting to be used, even when there are no requests. For example, if we set minimum pool size 5, the pool will ensure there are always 5 idle connections. Even if there are no active requests, the pool will still make sure these 5 connections are open to reduce the connection opening time when a request comes in.

This is useful for anticipating erratic loads, for example suddenly our application gets a lot of requests and connections are available ready to use without having to open a new connection. The analogy is like this:

Imagine that a restaurant has 10 chairs that are deliberately not arranged in the dining room, but are stored in the warehouse. But when there are a lot of visitors, we need extra chairs, we don't need long to buy them at the shop. But just need to go to the warehouse and pick up the 10 chairs needed for use.

Idle Connection

Idle connections are the number of connections that are not being used but remain open in the pool, which can also be called "idle connections". This parameter is usually set by:

Maximum Idle Time: How long an idle connection is maintained before being closed.
Maximum Idle Connections: The number of idle connections allowed to remain in the pool.

For example, if we set MaxIdleConnections = 5, the pool will maintain a maximum of 5 idle connections. If there are more idle connections it will be closed. If there are no new requests within a certain time (for example, 10 minutes), these idle connections may be removed to save resources. The analogy is like this:

In the supermarket there are 5 cashiers, 3 cashiers are serving buyers (active connection) and 2 cashiers are on standby in their positions without a queue of buyers (idle connection) and are ready whenever a buyer comes.
Supermarkets have a rule, "If the cashier is on standby for 30 minutes without a customer, then they can go home." These 30 minutes are idle time.

Timeout

We can set the maximum time the application waits for a connection from the pool before it fails (error). This is done to prevent applications from waiting too long for available connections from the pool, if the pool is busy or full. This is useful to prevent the application from waiting too long for a connection which will cause the application to "freeze" where it is better to give an error to the user, so that it can provide information to the user to try again rather than resulting in a freeze request.

For example, if we set a timeout of 5 seconds, if a request does not get a connection, after waiting for 5 seconds it will receive an error. The point to remember is that a timeout that is too short can cause the request to fail too quickly and a timeout that is too long means the request will hang for too long. The analogy is like this:

We are queuing to buy at a restaurant (request), but we can't wait for the queue (timeout limit). And we decided to leave the restaurant (return error).

Max lifetime

Lifetime determines how long a connection can live before being reset or closed by the pool, even if the connection is active. Usually used to avoid old connection problems or update configurations to ensure connections remain fresh and reliable.

For example, if we set max lifetime 20 minutes, any connection that has been used for 20 minutes will be closed and removed from the pool and will be replaced by a new connection that will be created.

This is important because sometimes some database servers have a time limit for old connections which can prevent stale connections** problems. The analogy is like this:

An office has 3 security officers per shift. Each officer works a maximum of 8 hours (Max Lifetime)
After 8 hours, the officers must be replaced with new officers, even though the previous 5 officers are still in good condition, there are no problems and work is running smoothly. But it prevents officers from getting tired and losing concentration when checking incoming visitors (requests).

Implementation

Now we will try to implement a connection pool using Golang. Note the following code:

package main

import (
    "database/sql"
    "fmt"
    "log"
    "time"

    _ "github.com/go-sql-driver/mysql"
)

func main() {
    dsn := "username:password@tcp(127.0.0.1:3306)/dbname?parseTime=true"

    db, err := sql.Open("mysql", dsn)
    if err != nil {
        log.Fatalf("Failed to open connection to database: %v", err)
    }
    defer db.Close()

    db.SetMaxOpenConns(10)
    db.SetMaxIdleConns(5)
    db.SetConnMaxLifetime(time.Minute * 10)
    db.SetConnMaxIdleTime(time.Minute * 5)

    err = db.Ping()
    if err != nil {
        log.Fatalf("Failed to ping database: %v", err)
    }
    fmt.Println("Successfully connected to database!")

    rows, err := db.Query("SELECT id, name FROM users")
    if err != nil {
        log.Fatalf("Failed to run query: %v", err)
    }
    defer rows.Close()

    for rows.Next() {
        var id int
        var name string
        if err := rows.Scan(&id, &name); err != nil {
            log.Fatalf("Failed to read query results: %v", err)
        }
        fmt.Printf("ID: %d, Name: %s\n", id, name)
    }

    if err := rows.Err(); err != nil {
        log.Fatalf("Error after iteration: %v", err)
    }
}

We can focus on the following code snippet:

db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)
db.SetConnMaxLifetime(time.Minute * 10)
db.SetConnMaxIdleTime(time.Minute * 5)

Here's the explanation:

SetMaxOpenConns(10): Determines the maximum number of connections that can be open simultaneously.
SetMaxIdleConns(5): Determines the maximum number of idle connections maintained in the pool.
SetConnMaxLifetime(time.Minute * 10): Determines how long a connection can live before being reset or closed.
SetConnMaxIdleTime(time.Minute * 5): Determines how long an idle connection can be maintained before being closed.

Conclusion

The use of connection pooling is very important, as this can anticipate the application running smoothly when handling an erratic load of incoming requests. But the right settings must also be implemented, so that the use of connection pooling can be more optimal.

If you have additions or corrections to the discussion above, let's discuss it in the comments column. Hope it helps 👋.

Mastering ENUMs in Go

Rizky Darmawan — Tue, 24 Dec 2024 02:41:21 +0000

Image by Hans-Peter Gauster from Unsplash

Let's say we are building an E-commerce API that will receive several orders, each order process has several statuses such as Pending, Processed, Shipped, Delivered, Cancelled. And our application receives input strings which will be stored in the database, for example the status is Processed, received Process, Processing or something else that causes data inconsistencies. Here Enum has an important role.

In Golang enums unlike other languages such as Java or C# which offer built-in support for enums, Go takes a different approach. In Go, enums are not a native language feature, but developers have several techniques that can be used to achieve similar functionality.

Understanding Enum

In Golang, enums (short for enumerations) provide a way to represent a set of named constants. Although Go does not have built-in enum types like some other languages, developers can emulate enum-like behavior using constants or custom types. Let's discuss the purpose and syntax of enums in Go:

Objective

Readability and Maintainability: Enums make code more readable and easy to understand by giving meaningful names to specific values. This increases the maintainability of the code because the purpose of each constant becomes easier to understand.
Type Safety: Enums help enforce type safety by restricting variables to a predefined set of values. This reduces the possibility of runtime errors caused by using incorrect values.

Create Enums

Here we will discuss step by step how to create an enum in Golang, so that it is easy to understand at each stage we will explain the meaning of the code we write.

Create a New Type

The first thing we will do is create a new type for the enum we need. The method is quite easy, we only need to use the keyword type and followed by the name of the type here with the name StatusOrder and the type here we define the type unsigned integer like this:

type StatusOrder uint

Well, just make it easy.

Define constant ENUM

With the new type that we have created, now is the time for us to define some of the order statuses that we have with constants. Where we define the type StatusOrder which we create as the type, like this:

const (
    Pending StatusOrder = iota
    Processed
    Shipped
    Delivered
    Cancelled
)

Maybe you ask, what is the keyword iota? This keyword makes GO assign a value of 0 to the first constant and then increase the value by 1 sequentially for each subsequent constant. This makes it easier for us rather than defining the values manually 1 by 1. About iota you can read here.

Function Strings

The next step we will take is to create a String function that is used to represent each string value from the StatusOrder enum.

func (s StatusOrder) String() string {
    switch s {
    case Pending:
        return "Pending"
    case Processed:
        return "Processed"
    case Shipped:
        return "Shipped"
    case Delivered:
        return "Delivered"
    case Cancelled:
        return "Cancelled"
    default:
        return "Unknown"
    }
}

Does the function name have to be String? We'll discuss it at the end.

Testing

Now we will carry out testing, the last code we will write is, function main and in it we print the results using the help of the fmt package. yes.

func main() {

    processed := Processed
    fmt.Printf("Order Status: %s (%d)\n", processed, processed)

    pending := Pending
    fmt.Printf("Order Status: %s (%d)\n", pending, pending)

}

Now the complete code can be seen here.

Of course, we carry out the last step and we will see the results like this:

Fmt Stringer

The question may arise, why is the String function also called when we call the enum constant?. The answer is because we use the fmt package. The fmt package explicitly uses the fmt.Stringer interface to process types that implement the String() method. So, if you don't use fmt, the String() method will not be called automatically. To explain more, you can explore in more detail here.

Conclusion

Although Golang does not offer native enum types, the techniques we learn here are often used in building applications with Golang. And for the type itself, we can freely use other types, not just integer. By utilizing this technique, readability, ease of maintenance and security can be significantly improved.

Maybe there are some points explained above that you feel are lacking, we can discuss them in the comments column below. Hope it helps 👋.

Reading References

Defensive Programming as a Backend Developer: Building Robust and Secure Systems

Rizky Darmawan — Mon, 11 Nov 2024 03:10:17 +0000

In backend application development, ensuring application security and stability is an absolute must. The backend is the backbone of the application, which is responsible for handling business logic, storing data, and interacting with external systems. Writing strong and reliable code is essential for all software developers. However, no matter how careful we are, bugs and unexpected situations can still occur. This is where Defensive programming comes into play.

Defensive programming is a coding practice aimed at ensuring that software functions correctly even when unexpected events or invalid input occur. For a backend developer defensive programming is an important approach, which allows us to design applications that can survive bad input, system errors, and external attacks.

In this article, we will discuss several points on how defensive programming can be applied to increase the resilience of backend applications and provide examples in the Golang language to illustrate how to implement it, although its implementation is not tied to just one language.

Validate Input Data from External Sources

External sources such as third-party API users, or databases can be entry points for various attacks, especially if the input provided is invalid or malicious. Attacks such as SQL injection, cross-site scripting (XSS), and command injection often occur due to input that is not properly filtered. We can apply the following points to overcome this:

1. Input Sanitation

Any input from outside sources must be sanitized. Input sanitization is the process of ensuring that data received from users or external sources is cleaned of malicious characters before it is used in an application, especially when the data is used for SQL queries, system commands, or output to the browser. For example, if you receive string input from a form, avoid entering that input directly into a SQL query without sanitization. What you can do is use prepared statements to prevent SQL Injection. With prepared statements, user input is not allowed to be part of the SQL command, but rather as a safe parameter. For example, we have input to check the user's username and password for login and our backend system, rather than directly running the following query:

// direct query with parameters
query := fmt.Sprintf("SELECT * FROM users WHERE username = '%s' AND password = '%s'", "john_doe", "12345")
rows, err := db.Query(query)

It is better to use prepared statements, where we ensure that user input is never executed as part of a SQL query. Input is considered a value, not part of a SQL command as follows:

// prepared statement
stmt, err := db.Prepare("SELECT * FROM users WHERE username = ? AND password = ?")
if err != nil {
    log.Printf("Error while prepared statement: %v", err)
    return
}
defer stmt.Close()

// excecution prepared statement
username := "john_doe"
password := "12345"
rows, err := stmt.Query(username, password)
if err != nil {
    log.Printf("Error while run query: %v", err)
    return
}
defer rows.Close()

2. Data Type Validation

Data type validation is the process of checking and ensuring that the type of data received is as expected before further processing. This is important to prevent errors in data processing and also reduce security risks, such as SQL injection and XSS. For example, if an API expects an email format, validate that the input is indeed an email format. The following is an example of carrying out the data type validation process in Golang:

func main() {
    input := "hello@world.com"

    // Use regex to validate email format
    re := regexp.MustCompile(`^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,} $`)
    if !re.MatchString(input) {
        fmt.Println("Invalid email format.")
        return
    }

    fmt.Println("Valid email:", input)
}

Here, we use a regular expression to validate whether the input corresponds to the correct email format.

3. Whitelist & Blacklist

Whitelist and blacklist are two approaches used to validate and manage user input in applications. Both have the goal of improving security and preventing exploits, but they work and are applied differently.

- whitelist

Whitelisting is an approach where only certain data or characters are allowed to be processed by the application. In the context of user input, a whitelist determines what is considered "safe" and allows only input that meets those criteria. It is often used to validate very specific data. Example of use:


func isValidUsername(username string) bool {
    // using regex to check allowed characters 
    re := regexp.MustCompile(`^[a-zA-Z0-9_-]+$`)
    return re.MatchString(username)
}

func main() {
    input := "user_name-123"
    if isValidUsername(input) {
        fmt.Println("Valid username:", input)
    } else {
        fmt.Println("Invalid username.")
    }
}

In the example above, the use of regex ^[a-zA-Z0-9_-]+$ is used to limit the characters allowed in the username. Only letters, numbers, _ (underscore), and - (dash) are accepted.

- Blacklist

Blacklisting is an approach where you specify certain data or characters that are not allowed. In this case, all other inputs are considered safe, except those on the blacklist. This approach is often used when it is difficult to predict all possible safe characters. Example of use:

func isValidInput(input string) bool {
    // List not allowed characters
    blacklist := []string{"<", ">", "&"}

    for _, char := range blacklist {
        if strings.Contains(input, char) {
            return false
        }
    }
    return true
}

func main() {
    input := "hello & welcome"

    if isValidInput(input) {
        fmt.Println("Valid input:", input)
    } else {
        fmt.Println("Invalid input.")
    }
}

Here, we use a blacklist to check whether the input contains unwanted characters. If present, the input is considered invalid.

4. Input Limit

Input Limit is the practice of limiting the amount of data or size of input that can be accepted from users in an application. The goal is to prevent excessive use of resources, improve application performance, and protect against potential attacks, such as Denial of Service (DoS) or SQL injection. The following is an example of its implementation:

func validateUsername(username string) bool {
    if len(username) > 20 {
        return false // Username is too long username is too long
    }
    return true // username is valid
}

func main() {
    input := "ThisIsAReallyLongUsername"

    if validateUsername(input) {
        fmt.Println("Valid username:", input)
    } else {
        fmt.Println("Invalid username: username cannot exceed 20 characters.")
    }
}

Of course, some of the points above can be applied to any programming language and can be used using libraries that are widely available so that the data validation process is easier than carrying out a manual validation process like some of the examples above.

Data Security

Sensitive data such as personal information or credentials must always be protected, both when stored and when transferred. We can apply the following points to overcome this:

1. Data encryption

Sensitive data such as passwords or credit card information should always be encrypted. Encryption involves the process of converting readable data (plaintext) into an unintelligible form (ciphertext) using an encryption algorithm and encryption key. The main purpose of encryption is to ensure that even if data falls into the wrong hands, it cannot be read without the right key to decrypt it.

Use modern encryption algorithms such as AES and avoid outdated algorithms such as MD5 or SHA1.

2. Use HTTPS

Use HTTPS (Hypertext Transfer Protocol Secure) to ensure that all data transferred between the client and server is properly encrypted. HTTPS (Hypertext Transfer Protocol Secure) is a secure version of HTTP, which is a protocol used for data transfer between a browser (client) and a server.

HTTPS works by adding a layer of security by using TLS (Transport Layer Security) or SSL (Secure Sockets Layer) to encrypt data sent over the network. This is especially important in backend development because it prevents third parties (such as hackers) from reading or modifying data that is being sent between the client and the server.

3. Tokenization & Masking

In certain cases, tokenization or masking can be used to reduce the risk of a data breach. Tokenization and Masking are two data security techniques frequently used in backend applications to protect sensitive data.

Tokenization is the process of replacing sensitive data (such as credit card numbers) with meaningless values called “tokens.” The original data is stored separately and can only be linked back to the token through a system that has access to the system that manages the token. This is important because if a data leak occurs, but the leaked data is only a token and not real data, the risk of attack is greatly reduced because the token has no value outside the system context. For example, in online payments, credit card numbers can be replaced with tokens so that card data does not actually need to be stored on the server.

Data masking is the process of hiding part of sensitive data so that only some parts are visible. This is often used in user interfaces or reports to prevent complete information from being exposed. How does this work? data masking protects sensitive information when displayed to users who do not need to know the full data. Even if the data displayed in the UI or logs is stolen, the masked data only shows partial information, so the risk is lower. An example that we may often see is hiding part of a credit card number in a display like this: **** **** **** 1234.

Error Management & Logging

In building a system errors will always occur, but how we handle them determines whether our application can survive without causing damage or loss of data. Good logging helps track problems in production systems and detect suspicious patterns. Here are some points that we do:

1. Handle Errors Appropriately

Don't just catch errors without taking action or providing information. It's best to log errors and provide a good fallback to avoid a total crash. For example, if the connection to the database fails, try retrying or switching to read-only mode temporarily.

2. Granular Logging

Log critical events such as API requests, authentication failures, or database connection problems with a sufficient level of detail. However, remember to avoid recording sensitive information such as passwords in logs.

3. Log Splitting

Use different log levels such as INFO, WARNING, and ERROR to separate different types of events. More granular logs make it easier for developers to diagnose problems.

For a more complete discussion and examples of Error Management & Logging I have written an article about this here: Observability - Why logging is important

Avoid mistakes elegantly

Crashing is a response that occurs when the system encounters an unexpected error. However, errors can be avoided or minimized through better handling. The strategies below can help us handle errors elegantly to make the system more resilient.

1. Timeouts

When backend applications access external services or wait for resources such as databases, it is important to use timeouts. Timeouts prevent code from being "stuck" in an indefinite waiting state, thereby maintaining application performance and avoiding deadlocks. The following is an example of implementing a timeout when accessing an external service:

client := http.Client{
    Timeout: 5 * time.Second,  // Set timeout 5 detik
}

resp, err := client.Get("https://example.com/api")
if err != nil {
    fmt.Println("Request failed:", err)
    return
}
defer resp.Body.Close()

fmt.Println("Response received:", resp.Status)

In this example, we set a 5 second timeout for HTTP requests. If the service doesn't respond within 5 seconds, the app won't hang, and we can handle the error by announcing the failure.

2. Resource Existence Checks

Before accessing other resources such as databases, files or APIs, always verify their existence and availability. This prevents applications from making requests to resources that may not exist, which saves time and avoids unnecessary errors. Here's an example of how we check:

func main() {
    filePath := "./data.txt"
    _, err := os.Stat(filePath) // check status file

    if os.IsNotExist(err) {
        fmt.Printf("File %s not found.\n", filePath)
    } else {
        fmt.Printf("File %s is available.\n", filePath)
    }
}

3. Fallback

When the system carries out certain processes, we might get an error. For example, our system sends an OTP during the authentication process. When the OTP sending process fails or gets an error, we can use a fallback mechanism or backup method, such as diverting OTP sending to our backup provider.

By implementing the strategies above, we can reduce the frequency of errors that cause crashes, maintain application performance, and improve the overall user experience.

Secure and Solid API Design

We can see that APIs function as the backbone for applications or web that require dynamic data resources. However, because they are considered such a critical backbone, APIs are often the main target for attacks, so it is important to ensure APIs that are secure and difficult to exploit.

1. Authentication & Authorization

Use strong authentication (such as OAuth or JWT) to ensure that only authorized users or systems can access our APIs. Implement granular access control to limit the access rights of clients who access the API that we have. This can prevent clients with malicious intent from accessing the resources we have.

2. Rate Limiting and Throttling

Rate Limiting and Throttling are critical to controlling API usage and preventing misuse or overloading. The application of rate limiting is also to prevent attacks such as DDoS or excessive exploitation of the API by users or bots, and also to maintain the stability and availability of the API for all users. Rate Limiting works by regulating the volume of requests from API clients within set intervals, ensuring fair and equal access to the resources we have.

Efficient Use of Resources

Backend systems often have to manage various resources such as database connections, files, or memory. Efficient resource management is critical to maintaining application stability and performance. By handling resources wisely, we can avoid problems such as "resource leaks" and system failures. The following are several strategies that can be used for efficient use of resources:

1. Connection Pooling

Connection Pooling is a technique where a number of connections to a service such as a database are recycled rather than creating a new connection every time an application needs access. With connection pooling, applications do not need to create expensive new connections every time there is a request, but instead use existing connections in the "pool".

Why is connection pooling important?, Every connection to a database or other service has overhead in terms of time and memory. Continuously opening and closing connections can drain system resources. By recycling existing connections, applications can respond more quickly and reduce the load on external services such as databases.

2. File & Memory Management

When dealing with files or memory allocation, it is important to ensure that resources are released after they have been used. If not, it could cause resource leaks which result in the application running out of memory or file handles.

For example, in Golang we can use the built-in function defer to ensure files, connections, or other resources are always released after they are finished using them. Then also check for errors (error handling) when working with resources, so that if a problem occurs, the resource can still be released correctly.

Here's a little code snippet of how we use defer to turn off an operation and close it after a function has finished carrying out a process:

func main() {
    // open file process
    file, err := os.Open("example.txt")
    if err != nil {
        fmt.Println("Error opening file:", err)
        return
    }

    // Make sure the file is closed when finishedinished
    defer file.Close()

    // Process file...
}

By using defer, we ensure that the file will be closed as soon as the function exits, whether it is successful or an error occurs. This helps prevent file handle leaks that can occur if you forget to close a file.

3. Asynchronous Operations

In situations where the application performs heavy tasks or slow I/O operations (such as accessing large files, or calling external APIs), running it Asynchronously is an efficient way to avoid blocking the application. Where applications that perform synchronous operations can be hampered while waiting for heavy tasks to complete, which will affect overall performance. So the use of asynchronous processes is sometimes important because it allows applications to continue other tasks while waiting for I/O or network tasks to complete.

Let's say we have a case where we clear the Redis cache after data input. The cache deletion process can be run asynchronously so as not to slow down the main process. This means the application does not need to wait for Redis to respond before continuing to execute the next code. As a result, the application remains responsive and efficient.

Conclusion

As a backend developer, defensive programming is an essential approach to ensuring that the applications we build can handle errors, invalid input, and external threats properly. By practicing the techniques above, we can ensure that the backend applications we build are more secure, stable and resilient in facing various unexpected situations. Defensive programming helps us create systems that not only function well, but are also able to survive the worst conditions.

Of course, there may be many other important points that are not discussed above, you can add or provide suggestions in the comments column above.

Hopefully it's useful 👋.

Reference

How to Handle Time Zones and Sync Your Software on the server side using Go

Rizky Darmawan — Tue, 01 Oct 2024 02:08:22 +0000

When your application starts on a large scale, the increase in users will increase. What is very likely to happen is that the user's location is not only in the same area, it could be in another area that has a different time zone. So as a Backend developer, things related to handling time zone differences are very important to think about.

I recently came across an issue involving time zones. Let's be honest, dealing with dates and times is one of the most complicated areas a human being has to deal with. And this was an opportunity for me to learn how to properly handle dates and times on the server side.

In this article, I will share my experience on how I handle time zone differences on the server side as a Backend developer. Maybe if someone is willing to correct and provide additional input that would be valuable for me.

About Time Zone

Time zones are a standard time division system used throughout the world to organize and standardize the measurement of time. This concept emerged as a response to the need for global time coordination, especially along with developments in communications and transportation technology.

The basic principle of time zones is the division of the earth into 24 zones. Each time zone is generally one hour different from the zone next to it. The main reference for time zones is Greenwich Mean Time (GMT) or Coordinated Universal Time (UTC), which is at the zero degrees longitude line that passes through Greenwich, England.

Illustration by Hellerick from Wikimedia Commons

A small example is when the clock shows 12:00 noon in Jakarta, in New York the time shows 00:00 or midnight. This means that while Jakartans are enjoying lunch, New Yorkers are just starting their new day. From here you can certainly imagine the importance of handling time zones correctly in building an application.

Timezone handling on the server side

After we have seen the explanation above, now we will go into the points that can be done when our server application receives a request from a client that accesses our API to handle its time zone.

In this article, I will discuss several approaches to handling time zones on the server side. Here I will use code examples in the Golang (Go) language. Golang has a time package for working with time-related data which is considered quite complete. Here are some points that we will discuss:

How to save date to DB
Conversion of user's local time
Testing and Validation

How to save date to DB

The first thing we will discuss is which time we will save in the database, for example we have an ecommerce application that does flash sales, where our application is already on an international scale.

When a user processes a purchase transaction in America or if the user is in Indonesia, the user will send their different local time to the server. The question is, will our database store time data according to the user's local time? If the answer is yes, it is likely a complicated problem when we want to retrieve the data or for example the admin wants to do data processing, which user makes the earliest transaction.

To overcome this, the best practice is to store transaction times in the UTC (Coordinated Universal Time) time zone which is the primary time standard for clocks and time settings. Here is the application of the time to UTC.

package main

import (
    "fmt"
    "time"
)

func main() {

    now := time.Now()
    fmt.Printf("Local time: %s\n", now)

    nowInUTC := now.UTC()
    fmt.Printf("UTC time: %s\n", nowInUTC)

}

Let's see the meaning of the code above.

First, in the now := time.Now() code line, this line uses the Now() function from the time package to get the current time based on the system's local time zone. The result is stored in the current variable.

Then, in the nowInUTC := now.UTC() line, here we convert the local time (now) to UTC time using the UTC() method. The results are stored in a new variable nowInUTC and this time can be stored on the server, where it is hoped that developers can avoid ambiguity and errors that may arise due to time zone differences between servers and users in various regions with different time zones.

Here are the results if we run the code above:

But this is not always the best solution that you should use. There may be some points you can keep in mind in certain use cases, one of which is is it true that our users come from different time zones? If it's not possible, perhaps storing the time in UTC will add complexity to your code.

Change time to user locale

At the discussion point above, we have agreed to store user time data in one location, namely UTC. Now how users can see accurate time according to their location. An example of the discussion above is a flash sale on an e-commerce application that we have, where users also want to know information about which user made the first transaction. So at this point, converting the time we store in the database to the user's local time is another important thing that we should not ignore.

The approach I take is that the server side always asks the client to send the timezone on the user side. This can be done on the request side where the client sends a header with the key timezone and has the user's timezone value. For example, Indonesia has 3 time zone divisions, namely WIT(+9), WITA(+8), WIB(+7). Where each zone has a difference of 1 hour. If previously on our server we stored UTC time at 00.00, then on the WIT side it was at 09.00, then on the WITA side it was at 08.00 and WIB at 07.00.

Here's an example code to handle the above case:

package main

import (
    "fmt"
    "time"
)

func ParseTimezoneFromString(tz string) *time.Location {

    if len(tz) > 0 {

        t, err := time.Parse("2006 -07:00", fmt.Sprintf("2007 %s", tz))

        if err != nil {

            panic(err)

        } else {

            return t.Location()
        }
    }

    return time.Now().Location()
}

func main() {

    timeServerInUTC := "2024-08-04 00:00:00"
    nowInUTC, err := time.Parse("2006-01-02 15:04:05", timeServerInUTC)
    if err != nil {
        panic(err)
    }

    fmt.Printf("UTC time: %s\n", nowInUTC)

    witLocation := ParseTimezoneFromString("+09:00")

    nowInWIT := nowInUTC.In(witLocation)
    fmt.Printf("WIT time: %s\n", nowInWIT)

    witaLocation := ParseTimezoneFromString("+08:00")

    nowInWITA := nowInUTC.In(witaLocation)
    fmt.Printf("WITA time: %s\n", nowInWITA)

    wibLocation := ParseTimezoneFromString("+07:00")

    nowInWIB := nowInUTC.In(wibLocation)
    fmt.Printf("WIB time: %s\n", nowInWIB)

}

credit to dikac for create this function ParseTimezoneFromString

Let's understand the meaning of the code above:

First, we create a function ParseTimezoneFromString, where this function is used to find the time location based on the argument tz or timezone of the given user location. If the string value tz is valid, we will convert the string's timezone using the time.Parse function to convert the string to a time object, then extract the location (timezone) from that object. And we also handle if the string is empty or parsing fails, the function returns the local time zone of the system.

func ParseTimezoneFromString(tz string) *time.Location {

    if len(tz) > 0 {

        t, err := time.Parse("2006 -07:00", fmt.Sprintf("2007 %s", tz))

        if err != nil {

            panic(err)

        } else {

            return t.Location()
        }
    }

    return time.Now().Location()
}

Next we also define time data in the following string format:

timeServerInUTC := "2024-08-04 00:00:00"

nowInUTC, err := time.Parse("2006-01-02 15:04:05", timeServerInUTC)
if err != nil {
    panic(err)
}

You can think of this as the timing data that we get from the server. And parse it into a time object.

Next, we try to find the user's accurate location based on the ParseTimezoneFromString function that we previously created based on the string argument that we defined. What needs to be paid attention to is that this string argument is what is meant by the value of the timezone header sent by the client via the incoming request.

We can use the location we get from the ParseTimezoneFromString function to convert or shift the time we get from the server to the user's local time. We can do this using the In function which is also included in the time package which we can see in the following code snippet:

nowInWIT := nowInUTC.In(witLocation)
nowInWITA := nowInUTC.In(witaLocation)
nowInWIB := nowInUTC.In(wibLocation)

If we run it, we will get the time that corresponds to the timezone location that we defined.

Testing and Validation

The final point is no less important, namely testing and validation. When the development process often causes developers to make unexpected mistakes, testing and validation are always important.

In the discussion of point 2 above, the ParseTimezoneFromString function has been important in handling our time zones. Repeated testing and validation are important to make our applications get results that meet our expectations.

For testing, it is recommended to use unit tests, where testing will be carried out on the smallest unit with several scenarios that can be added. The more scenarios there are, the less likely it is to handle these time differences.

Conclusion

Handling time zones can indeed be tricky for backend developers. However, it's important to remember that every challenging task we overcome contributes to our growth and skill improvement. Properly managing time zones is not just a technical necessity, it ensures accuracy in scheduling and provides a smooth experience for our application users across different regions.

The points shared in this article about storing times in UTC, converting to user local times, and implementing robust conversion functions are starting points in tackling this complex issue. However, I acknowledge that there may be shortcomings or areas for improvement in the approaches discussed. This is why additional input and suggestions from the developer community are invaluable.

I sincerely hope that the insights and code examples provided in this article will be helpful to you in the future when you encounter time zone-related challenges in your projects. Remember, the goal is to create applications that work seamlessly for users, regardless of their geographical location.

Let's continue this discussion in the comments section below. I'd love to hear about your experiences with handling time zones, any challenges you've faced, or alternative approaches you've found effective. Your insights could be extremely valuable to me and other readers facing similar issues.

Thank you for reading, and I hope this article proves useful in your development journey. Let's keep learning and improving together! 👋

Reading References

Observability - Why logging its important

Rizky Darmawan — Fri, 28 Jun 2024 09:39:28 +0000

In an increasingly complex digital era, observability is the main key in managing modern software systems. One of the most important pillars of observability is logging. Let's explore why logging is so important and how to make optimal use of it.

What is Logging?

Logging is the process of recording activities and events in a system. This includes a variety of information, from error messages, user activity, to system performance. Think of logging as an airplane 'black box' for your system - always recording what's happening, ready to provide insights when needed.

Why is Logging So Important?

Here are some points that can be considered why logs are important:

Faster Problem Solving
With good logs, development teams can identify root causes without guesswork. It's like having a treasure map when looking for bugs!
Security Improvements
Logs can be your 'spy' in detecting suspicious activity. Security teams can respond to threats more quickly, such as having a fire department always on standby.
Performance Analysis
Through logs, you can identify bottlenecks in the system. It's like having a personal doctor for your app's health.
Understanding User Behavior
User activity logs provide valuable insight into how the product is used. It's like having a personal assistant constantly observing and reporting customer preferences.

Best Practices in Logging

To maximize the benefits of logging, below are some of the best practices that can be carried out:

Determine the Appropriate Log Level

Using these appropriate log levels can help you filter information quickly, such as sorting logs by urgency.

The following is an example of displaying logs using the Golang language with various levels. Here we use the Logrus.

package main

import (
    "github.com/sirupsen/logrus"
)

func main() {
    log := logrus.New()
    log.SetLevel(logrus.DebugLevel)

    log.Debug("Starting app..")
    log.Info("User has successfully logged in")
    log.Warn("CPU usage exceeds 80%")
    log.Error("Failed to save data to database")
    log.Fatal("A critical error occurs, the application will stop")
}

The following is an explanation for the several log levels above:

DEBUG: Detailed information for debugging, usually only enabled during development.
INFO: General information about the normal flow of the application.
WARNING: For situations that have the potential to become problematic in the future, but do not stop the application.
ERROR: An error that causes a specific function to fail, but the application is still running.
FATAL: Serious error that may cause the application to stop.

Include relevant contextual information

Each log entry should provide enough context to understand what happened. This could include:

Timestamp.
Transaction or session ID.
User ID (if relevant).
Function or module name.
Relevant input data (be careful with sensitive data).
Stack trace for errors

This is an example of code when printing a log, including context information that will help us trace.

package main

import (
    "github.com/sirupsen/logrus"
    "time"
)

type UserAction struct {
    UserID    int
    Action    string
    Timestamp time.Time
}

func main() {
    log := logrus.New()
    log.SetLevel(logrus.DebugLevel)

    // Use format json
    log.SetFormatter(&logrus.JSONFormatter{})

    // Dummy data
    action := UserAction{
        UserID:    12345,
        Action:    "checkout",
        Timestamp: time.Now(),
    }

    // Print log
    log.WithFields(logrus.Fields{
        "user_id":    action.UserID,
        "action":     action.Action,
        "timestamp":  time.Now().Format(time.RFC3339),
        "session_id": generateSessionID(),
        "module":     "payment_processor",
        "ip_address": "192.168.1.100",
    }).Error("Payment failed")

}

func generateSessionID() string {
    return "sess_abc123"
}

We can see that we have included several elements of context information that can make it easier for us to carry out tracing in the future. What are the conveniences in question, namely that we can search logs based on level, for example the error level in the code example above, and also based on time and others based on the information we enter.

Use consistent formatting

A consistent log format makes parsing and analysis easier, especially if using automated tools (regarding tools, will be discussed below). Formatting also makes it easier for us to search logs based on criteria, for example log level, message, or time. Example format:

[TIMESTAMP] [LEVEL] [MODULE] [MESSAGE]

Or JSON format for easy parsing like the results in the code example above:

{
    "action": "checkout",
    "ip_address": "192.168.1.100",
    "level": "error",
    "module": "payment_processor",
    "msg": "Payment failed",
    "session_id": "sess_abc123",
    "time": "2024-06-26T20:59:02+07:00",
    "timestamp": "2024-06-26T20:59:02+07:00",
    "user_id": 12345
}

Implement log rotation to manage file size

Log rotation prevents log files from becoming too large and difficult to manage. This involves:

Limits the size of log files.
Create new log files periodically (e.g. daily or weekly).
Archive or delete old log files.
Using tools such as logrotate on Linux or a logging framework that supports rotation.

Consider privacy and security in logged information

Security and privacy are very important in logging:

Do not log sensitive data such as passwords or credit card information.
Mask or encrypt personal data if necessary.
Ensure access to log files is restricted to authorized personnel only.
Implement a retention policy to delete old logs according to company policies and regulations.

Tools for Monitoring and Analyzing Logs

As system complexity increases, the need for sophisticated tools to monitor and analyze logs also becomes increasingly important. Here are some popular tools that can help with observability and log analysis:

Grafana
Grafana is an open-source platform for visualizing our log data. These tools can be integrated into various data sources including logs. Enables the creation of customized and interactive dashboards. Suitable for real-time visualization of metrics and logs.
New Relic
New Relic is an all-in-one observability platform
Provides log analysis, tracing, and metrics in one place. There are also AI features to detect anomalies and correlate problems.
Suitable for monitoring large-scale applications and infrastructure.
Loki
Loki is a lightweight and cost-effective log aggregation system. Loki is designed to work well with Grafana
Uses label-based indexes, similar to Prometheus
Ideal for organizations already using Prometheus and Grafana.
AWS CloudWatch Logs Insights
This integrated log analysis service from AWS enables querying and analysis of logs from various AWS services.
Feature to detect slow queries in RDS and other database services
Easy integration with other AWS services.

Conclusion

Logging is not just an additional feature, but a vital component in building a reliable system. With proper implementation, logging can become your supersensor - providing full visibility into system operations, helping prevent problems before they occur, and speeding resolution when problems arise.

So, start investing in good logging practices today. Remember, in the world of complex technology, good logs can be a guiding light in the midst of a storm!

If you have additional information, please enter it in the comments column below.

Reading References

Overcoming the Thundering Herd Problem in Go Applications with Circuit Breaker Pattern

Rizky Darmawan — Tue, 11 Jun 2024 03:07:00 +0000

Imagine you are on your way to the office during the morning rush hour. The highways were filled with slow-moving vehicles due to heavy traffic jams. When the traffic light turns green, thousands of vehicles try to drive at once, creating even worse traffic jams. This situation is similar to what is called the "thundering herd problem" in the world of computing.

What is the Thundering Herd Problem

Thundering Herd is a term in distributed computing and large-scale systems that refers to a situation where multiple processes or threads try to obtain the same resource simultaneously after a certain delay or event.
This issue often occurs in scenarios where a large number of processes are waiting for a limited resource, such as a database, shared cache, network service, or remote service.

When this resource finally becomes available, all waiting processes will try to access it simultaneously, causing a spike in activity that can overload or even disrupt the service.

The effects of Thundering Herd can cause performance degradation, delays, or even service failure. To overcome this problem, mechanisms such as scheduling, throttling, or using design patterns such as Circuit Breaker are needed to limit the number of requests at once to limited resources.

Case Example

Suppose you have a web application that fetches user data from a MySQL database and stores it in the Redis cache to improve access speed. When user data is not found in the cache, many requests will try to retrieve data from the database simultaneously, causing excessive load on the database and reducing overall application performance.

Look at the code below:

package main

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "github.com/go-redis/redis"
    _ "github.com/go-sql-driver/mysql"
    "log"
    "math/rand"
    "time"
)

type User struct {
    Id       int
    Username string
    Email    string
}

type Config struct {
    redisClient *redis.Client
    dbClient    *sql.DB
}

func NewUser(dbClient *sql.DB, redisClient *redis.Client) *Config {
    return &Config{
        dbClient:    dbClient,
        redisClient: redisClient,
    }
}

func (e *Config) getDataFromMysql(username string) (*User, error) {
    row := e.dbClient.QueryRow("SELECT * FROM users WHERE username = ?", username)

    user := &User{}
    err := row.Scan(&user.Id, &user.Username, &user.Email)
    if err != nil {
        return nil, err
    }

    err = e.saveToRedis(username, user)
    if err != nil {
        return nil, err
    }

    return user, nil

}

func (e *Config) getDataFromRedis(username string) (*User, error) {
    val, err := e.redisClient.Get(username).Result()
    if err != nil {
        log.Printf("failed to get redis with key [%s], err: %v", username, err)

        log.Println("get data from mysql")
        user, err := e.getDataFromMysql(username)
        if err != nil {
            return nil, err
        }

        return user, nil

    }

    user := &User{}
    err = json.Unmarshal([]byte(val), &user)
    if err != nil {
        return nil, err
    }

    return user, nil
}

func (e *Config) saveToRedis(key string, data *User) error {
    jsonData, err := json.Marshal(data)
    if err != nil {
        return err
    }

    ttl := 10 * time.Second
    err = e.redisClient.Set(key, jsonData, ttl).Err()
    if err != nil {
        return err
    }

    return nil

}

func main() {
    mysqlConn, err := sql.Open("mysql", "root:root@tcp(localhost:3306)/employees")
    if err != nil {
        panic(err)
    }

    redisConn := redis.NewClient(&redis.Options{
        Addr:     "127.0.0.1:6379",
        Password: "",
        DB:       0,
    })

    client := NewUser(mysqlConn, redisConn)

    username := "jhon_doe"

    // Simulate get cache miss as asyncronous
    for i := 0; i < 100; i++ {
        go func() {
            time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond)
            val, err := client.getDataFromRedis(username)
            if err != nil {
                fmt.Println(err)

            } else {
                fmt.Printf("Got value: %v\n", val)

            }
        }()
    }

    select {}
}

Now let's try running the code above and in the middle of the process we will try to turn off the MySQL database that we have.

In the above process we can see, when data is not found in the Redis cache, all requests will try to retrieve data from the MySQL database simultaneously. Imagine that when you try to retrieve data from MySQL there are lots of requests. This can cause a thundering herd problem and overload the database with excessive requests and also when the database recovers it immediately gets a lot of requests that continue to be sent.

Circuit Breaker Pattern

There are many solutions that can be used to deal with this thundering herd problem. One way is to use Circuit Breaker Pattern. Before we go into the solution to the problem in the code above, let's first find out what a Circuit Breaker Pattern is. In the world of software, the Circuit Breaker Pattern is a design pattern used to prevent system failure due to unresponsive or failed external components. This pattern is similar to how an electrical Circuit Breaker works at home.

In everyday life, we often use electrical appliances at home, such as irons, toasters or washing machines. If too many appliances are used at the same time, this can place an excessive load on your home's electrical system. To prevent damage or fire, an electrical safety device called a "Circuit Breaker" is installed.

Likewise, in a distributed system, components such as databases, web services, or other external systems can fail or become unresponsive for various reasons, such as excessive load, network problems, or internal errors. If your application continually tries to access unresponsive components, this can lead to excessive resource usage, a backlog of requests, and ultimately a complete system failure.

Circuit Breaker Pattern addresses this problem by monitoring the success and failure of requests to external components. If too many failures occur within a certain time period, the Circuit Breaker will "open" and temporarily prevent new requests to external components. As long as the Circuit Breaker is open, new requests will be rejected or redirected to a safe fallback mechanism, such as returning cached data or a default response.
After a certain time delay, the Circuit Breaker will try to close itself again and allow new requests to external components. If the request is successful, the Circuit Breaker will remain closed.

However, if the failure occurs again, the Circuit Breaker will open again and the cycle will repeat. By using the Circuit Breaker Pattern, your application becomes more resilient to external component failures, preventing excessive resource usage, and giving external components time to recover before accepting new requests. This improves overall system stability, reliability and robustness.

Implementation of solutions with Circuit Breaker Pattern

Now we will use the Circuit Breaker Pattern to overcome the thundering herd problem above. Circuit Breaker acts as a safeguard that limits access to a resource (in this case, a MySQL database) when a series of failures occurs. If too many failures occur within a certain time period, the Circuit Breaker will "open" and prevent new requests from the failed resource. Here's the complete code:

package main

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "github.com/go-redis/redis"
    _ "github.com/go-sql-driver/mysql"
    "log"
    "math/rand"
    "time"
)

type User struct {
    Id       int
    Username string
    Email    string
}

type Config struct {
    redisClient *redis.Client
    dbClient    *sql.DB
    circuit     *CircuitBreaker
}

type CircuitBreaker struct {
    failureThreshold   int
    consecutiveFailure int
    open               bool
    openedAt           time.Time
}

const circuitBreakerResetTimeout = 2 * time.Second

func NewCircuitBreaker(failureThreshold int) *CircuitBreaker {
    return &CircuitBreaker{
        failureThreshold:   failureThreshold,
        consecutiveFailure: 0,
        open:               false,
    }
}

func NewUser(dbClient *sql.DB, redisClient *redis.Client, failureThreshold int) *Config {
    return &Config{
        dbClient:    dbClient,
        redisClient: redisClient,
        circuit:     NewCircuitBreaker(failureThreshold),
    }
}

func (cb *CircuitBreaker) IsOpen() bool {
    if cb.open {
        // Check last time opened
        if time.Since(cb.openedAt) >= circuitBreakerResetTimeout {
            cb.open = false
            cb.consecutiveFailure = 0
            log.Println("Circuit Breaker closed")
        } else {
            return true
        }
    }
    return false
}

func (cb *CircuitBreaker) IncrementConsecutiveFailure() {
    cb.consecutiveFailure++
    if cb.consecutiveFailure >= cb.failureThreshold {
        cb.open = true
        cb.openedAt = time.Now()
        log.Println("Circuit Breaker opened")
    }
}

func (e *Config) getDataFromMysql(username string) (*User, error) {
    row := e.dbClient.QueryRow("SELECT * FROM users WHERE username = ?", username)

    user := &User{}
    err := row.Scan(&user.Id, &user.Username, &user.Email)
    if err != nil {
        return nil, err
    }

    err = e.saveToRedis(username, user)
    if err != nil {
        return nil, err
    }

    return user, nil

}

func (e *Config) getDataFromRedis(username string) (*User, error) {
    if e.circuit.IsOpen() {
        return nil, fmt.Errorf("circuit breaker is open")
    }

    val, err := e.redisClient.Get(username).Result()
    if err != nil {
        log.Printf("failed to get redis with key [%s], err: %v", username, err)

        log.Println("get data from mysql")
        user, err := e.getDataFromMysql(username)
        if err != nil {
            e.circuit.IncrementConsecutiveFailure()
            return nil, err
        }

        return user, nil

    }

    user := &User{}
    err = json.Unmarshal([]byte(val), &user)
    if err != nil {
        return nil, err
    }

    return user, nil
}

func (e *Config) saveToRedis(key string, data *User) error {
    jsonData, err := json.Marshal(data)
    if err != nil {
        return err
    }

    ttl := 2 * time.Millisecond
    err = e.redisClient.Set(key, jsonData, ttl).Err()
    if err != nil {
        return err
    }

    return nil

}

func main() {
    mysqlConn, err := sql.Open("mysql", "root:mysqlsecret@tcp(localhost:3306)/employees")
    if err != nil {
        panic(err)
    }

    redisConn := redis.NewClient(&redis.Options{
        Addr:     "127.0.0.1:6379",
        Password: "",
        DB:       0,
    })

    failureThreshold := 3
    client := NewUser(mysqlConn, redisConn, failureThreshold)

    username := "user20001"

    // Simulate get cache miss as asyncronous
    for i := 0; i < 100; i++ {
        go func() {
            time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond)
            val, err := client.getDataFromRedis(username)
            if err != nil {
                fmt.Println(err)

            } else {
                fmt.Printf("Got value: %v\n", val)

            }
        }()
    }

    select {}
}

Now we will see any changes to the code above. The first thing we do is modify the truct in the Config struct by adding a circuit field that holds a CircuitBreaker object:

type Config struct {
    redisClient *redis.Client
    dbClient    *sql.DB
    circuit     *CircuitBreaker
}

With the addition of this circuit field, the Config object now also has a CircuitBreaker object that will be used to apply the Circuit Breaker pattern. Next, there is the addition of a new CircuitBreaker struct and a NewCircuitBreaker function:

type CircuitBreaker struct {
    failureThreshold   int
    consecutiveFailure int
    open               bool
    openedAt           time.Time
}

The CircuitBreaker struct has three fields:

failureThreshold: The maximum number of failures allowed before the Circuit Breaker opens.
ConsecutiveFailure: The number of consecutive failures that have occurred.
open: Status whether the Circuit Breaker is currently open or not.
openedAt: Records when the Circuit Breaker opened.

func NewCircuitBreaker(failureThreshold int) *CircuitBreaker {
    return &CircuitBreaker{
        failureThreshold:   failureThreshold,
        consecutiveFailure: 0,
        open:               false,
    }
}

The NewCircuitBreaker function is a constructor to create a new CircuitBreaker object with the given failureThreshold, consecutiveFailure set to 0, and open set to false (the Circuit Breaker is in the closed state initially).
Finally, there is a modification to the NewUser function to register the circuit breaker so that it can be used in the function that initializes the Config object:

func NewUser(dbClient *sql.DB, redisClient *redis.Client, failureThreshold int) *Config {
    return &Config{
        dbClient:    dbClient,
        redisClient: redisClient,
        circuit:     NewCircuitBreaker(failureThreshold),
    }
}

In the code above we also add the Is Open function:

func (cb *CircuitBreaker) IsOpen() bool {
    if cb.open {
        // Check last time opened
        if time.Since(cb.openedAt) >= circuitBreakerResetTimeout {
            cb.open = false
            cb.consecutiveFailure = 0
            log.Println("Circuit Breaker closed")
        } else {
            return true
        }
    }

    return false
}

The IsOpen function will first check whether the Circuit Breaker is open (cb.open). If open, the function will check whether the time that has passed since the Circuit Breaker opened (time.Since(cb.openedAt)) has reached or exceeded the circuitBreakerResetTimeout.
If the elapsed time has reached the waiting time limit, the Circuit Breaker will be closed again (cb.open = false), and consecutiveFailure is set back to 0.

The function will also record a log message "Circuit Breaker closed" to notify you that the Circuit Breaker has closed again. If the elapsed time has not reached the waiting time limit, the function will return true to indicate that the Circuit Breaker is still open.
If the Circuit Breaker is not open, the function will return false.

And don't forget to add the constant circuitBreakerResetTimeout to provide an automatic checking time for the circuit breaker to reopen in the IsOpen function above.

const circuitBreakerResetTimeout = 2 * time.Second

And then the IncrementConsecutiveFailure function is also a method of the CircuitBreaker struct. This function is responsible for counting the number of consecutive failures that occurred when trying to access an external resource.

func (cb *CircuitBreaker) IncrementConsecutiveFailure() {
    cb.consecutiveFailure++
    if cb.consecutiveFailure >= cb.failureThreshold {
        cb.open = true
        cb.openedAt = time.Now()
        log.Println("Circuit Breaker opened")
    }
}

First, this function will increase the consecutiveFailure value (the number of consecutive failures) by adding it by 1. Then, the function will check whether the consecutiveFailure has reached or exceeded the failureThreshold (the specified maximum failure limit).

If the consecutiveFailure reaches or exceeds the failureThreshold, the Circuit Breaker will be changed to an open state (open = true) and record the time when the circuit breaker was opened for comparison in the IsOpen function above. When the Circuit Breaker opens, all new requests to external resources will be rejected or redirected to a secure fallback mechanism. In addition, the function will also record a log message "Circuit Breaker opened" to notify that the Circuit Breaker has opened.

After all the necessary configurations and functions have been created, now we will carry out the implementation to handle the thundering herd problem. First, in the main function, we will determine the initial value of failureThreshold, which in this case is the limit of 3 times for failure.

func main() {
    // Others code ...

    failureThreshold := 3
    client := NewUser(mysqlConn, redisConn, failureThreshold)

    // Others code ...
}

The last thing we do in implementing the circuit breaker is the getDataFromRedis function, let's discuss it.

func (e *Config) getDataFromRedis(username string) (*User, error) {
    if e.circuit.IsOpen() {
        return nil, fmt.Errorf("circuit breaker is open")
    }

    // others code ...

First, in the getDataFromRedis function we add checking whether the Circuit Breaker is open or not by calling e.circuit.IsOpen(). If the Circuit Breaker is open, the function will immediately return the error "circuit breaker is open" and will not continue the process of retrieving data from Redis or MySQL. This is done to prevent new requests to external resources while the Circuit Breaker is open.

If Circuit Breaker is not open, the function will try to retrieve data from Redis using e.redisClient.Get(username).Result(). If an error occurs while retrieving data from Redis, the function logs the error and tries to retrieve data from MySQL by calling e.getDataFromMysql(username).

// others code ...

val, err := e.redisClient.Get(username).Result()
    if err != nil {
        log.Printf("failed to get redis with key [%s], err: %v", username, err)

        log.Println("get data from mysql")
        user, err := e.getDataFromMysql(username)
        if err != nil {
            e.circuit.IncrementConsecutiveFailure() // <-- call function IncrementConsecutiveFailure 
            return nil, err
        }

        return user, nil
    }

// others code ...

If an error occurs while retrieving data from MySQL, the function will call e.circuit.IncrementConsecutiveFailure() to increase the number of consecutive failures in the Circuit Breaker. If the number of consecutive failures reaches a certain limit (failureThreshold), then the Circuit Breaker will open. After that, the function will return error and nil as user values. As in the example below when we run the code above and our MySQL database dies.

After several consecutive failures to retrieve data from MySQL, the Circuit Breaker will open and there will be no more attempts to retrieve data from MySQL. At this stage all incoming requests will be rejected or we will provide an appropriate fallback for the user to obtain. And the Circuit Breaker will close again until the time limit we specified in the constant circuitBreakerResetTimeout.

With the implementation of the Circuit Breaker Pattern, the getDataFromRedis function will prevent new requests to external resources (MySQL) while the Circuit Breaker is open. In addition, the function will also monitor failures when retrieving data from MySQL and open the Circuit Breaker if a certain failure limit is reached. This helps prevent thundering herd problems and gives external resources time to recover before accepting new requests again.

Conclusion

Thundering herd problem is a common problem in distributed systems and can cause significant performance degradation. By using design patterns like Circuit Breaker, you can overcome these problems and ensure your applications run smoothly and efficiently. Always consider the potential thundering herd problem when designing systems that depend on limited resources and implement appropriate solutions to prevent it.

Reading References

Optimizing Relational Databases for Best Performance in MySQL

Rizky Darmawan — Tue, 07 May 2024 09:08:14 +0000

Introduction

In today's data-driven world, relational databases are the backbone of countless applications. They store and manage critical information, but their performance can significantly impact user experience and overall system efficiency.

This blog post dives into key strategies for optimizing relational databases and ensuring they run at peak performance.

🔍 Common Query Mistake That Lead Bottleneck

Before optimizing, in this writing the discussion only covers Unnecessary Full Table Scans, Inefficient Queries, Denormalization, Insufficient Hardware Resources. It's crucial to identify performance bottlenecks. Here are some common causes:

1. Unnecessary Full Table Scans

For example, I have table products with a total 4000000 rows of data. If your query doesn't have proper indexing, the database may be forced to scan the entire table for each lookup. For example how we can know a table caught the table scan:

Here's, I can try to select the products with name condition



SELECT * from products WHERE name = "Produk 300665";

And this result I have a total time 0.628s, it's not a long time.

But the surprising thing is when you run explain to see what happens:



explain SELECT * from products WHERE name = "Produk 300665";

See you have scanned a total of 3052749 rows just to find 1 row of data. And look at the type column here it has the value ALL, meaning you are scanning row by row looking for data. This will be a nightmare when the total data keeps getting bigger.

2. Inefficient Queries

Poorly written SQL queries can take longer to execute than necessary. Here's an example of a poorly written SQL query:



SELECT * FROM orders WHERE YEAR(order_date) = 2024;

This query retrieves all orders from 2024. However, using the YEAR() function in the WHERE clause makes it inefficient because it prevents the query from using an index on the Order Date column.

Instead, this forces the database to apply the YEAR() function to every row in the Orders table, potentially resulting in a full table scan. For example, the above is a query snippet from the orders table, now let's look at the table.

My orders has an index on the order_date column. Run this query to see a list of available indexes:



show indexes from order_table;

And see I do have an index for the order_date column.

Now, when you run explain to see what happens:



explain SELECT * FROM orders WHERE YEAR(order_date) = 2024;

It can be seen in the results below, using functions such as YEAR(), can cause the available indexes to not be used.

3. Denormalization (Careful Approach)

In some situations, denormalizing your database schema (introducing controlled redundancy) can improve query performance by reducing the need for complex joins. Here's a simple example to illustrate denormalization in MySQL:

Let's consider a hypothetical scenario where you have two tables: users and orders. Below are the Original Normalized Tables:

users table with structure:



user_id (Primary Key)
username
email

orders table with structure:



order_id (Primary Key)
user_id (Foreign Key referencing user_id in the users table)
order_date
total_amount

Now, let's say you frequently need to retrieve a user's orders along with their username and email. The most common way is to always join the orders table to the users table. Here's an example query:



SELECT
    orders.order_id,
    users.name,
    users.email,
    orders.order_date,
    orders.total_amount
FROM
    orders
    JOIN users ON orders.user_id = users.user_id;

Joining these tables every time you need this information could become a performance bottleneck, especially if the tables are large.

We will discuss how to denormalize it in the solutions chapter below.

4. Insufficient Hardware Resources

Database performance is highly dependent on factors such as CPU, RAM, and storage capacity. This is very difficult to see without the help of supporting software. We will discuss several solutions below.

🎯 Solution

Once you've identified the bottlenecks, you can employ various techniques to streamline your database:

1. Unnecessary Full Table Scans Solution

Indexes act like shortcuts for the database, allowing it to quickly locate specific data. Strategically creating and maintaining indexes on frequently used columns can significantly improve query speed.

Diving Deeper into Indexing Types

In the field of database management and optimization, indexing plays an important role in improving query performance. Understanding the different types of indexing is critical to efficient data retrieval and manipulation.

Let's dive deeper into two fundamental types of indexing: single index and compound index, and explore their function with illustrative examples.

Single Index:

A single index is a straightforward mechanism in which an index is created on a single column of a database table. It speeds up the search process by organizing data in indexed columns in a structured format, often a B-tree or hash table, allowing for faster retrieval of specific records.

In the example above, we are looking for a name in the products table with a large amount of data in the user table. Remember, to search for 1 row of data, we are required to scan each row. Now let's try adding a single index to the name column with the following query:



CREATE INDEX product_name_idx ON products (name);

To see if an index has been created, run this query:



show indexes from users;

Yeah, index has created:

Now, try again to select the user with name condition:



SELECT * from products WHERE name = "Produk 300665";

And it's amazing how quickly you can get data when you only take 2 second to do it, compared to the previous time of 0.628s seconds.

And the important thing is that we noticed many rows were scanned. Please rerun this query again.



explain SELECT * from products WHERE name = "Produk 300665";

There is only 1 row scanned and you can also observe that we are using an index. Small notes, if you want to remove index can run query:



DROP INDEX product_name_idx ON product;

Composite Index:

Unlike single index, a composite index involves indexing multiple columns within a database table. This type of indexing is particularly advantageous when queries involve conditions on multiple columns or require sorting based on a combination of fields.

Expanding on the previous example, let's say we frequently run a query to retrieve products based on their name, category, and price.

In such a scenario, creating a combined index on the name, category, and price columns will optimize the search process by storing and sorting the data based on these combined criteria.

For example, if we want to search for users with a specific name, category (gadget), and price greater than 400.000, the query would look like this:



SELECT *  from products WHERE name = "Produk 300665" AND category = "gadget" and price > 400;

As a result we get 25 data.

See the single index that we previously had has no effect on queries with multiple conditions.

If we explain with the single index we previously created, we scan 28 rows:



explain SELECT *  from products WHERE name = "Produk 300665" AND category = "gadget" and price > 400;

Obviously 25 lines are obtained but by scanning only 28 lines it is not a big problem. But what we need to remember is what happens if we have a large amount of data. Certainly the number of rows we scan will be much larger than the data we get. And now we try to create a composite index, by combining several columns in 1 index, a query like this:



CREATE INDEX product_name_category_price_idx ON products (name, category, price);

Now looks, a composite index has created:

To differentiate between single index and composite index, refer to the column key_name. A composite index is an index with multiple column_name values that share the same name as another index.

And now, we trying run explain query in above for see different result:

Voila, we only scanned 25 rows for 25 results using the composite index. And the composite index that we have is also included in the list of indexes that we will use.

2. Inefficient Queries Solution

Writing clean and optimized SQL queries is very important. Techniques such as avoiding complex functions in the WHERE clause and using appropriate JOINs can make a big difference. Or as in the example above, use the YEAR function to sort the years.

A more efficient version of this query would directly compare the order_date column to a date range in 2024:



SELECT * FROM Orders WHERE order_date >= '2024-01-01' AND order_date < '2025-01-01';

See nothing happens and nothing seems strange. But let's try running an explain query:



EXPLAIN SELECT * FROM Orders WHERE order_date >= '2024-01-01' AND order_date < '2025-01-01';

Now the index is used, and this is very important when the data we are scanning is very large.

3. Denormalization Solution

To optimize this scenario using denormalization, you might add redundant columns from the users table into the orders table:

Denormalized Orders Table:

orders table:



order_id (Primary Key)
user_id (Foreign Key referencing user_id in the users table)
order_date
total_amount
name (Redundant column from the users table)
email (Redundant column from the users table)

By adding name and email columns to the orders table, you eliminate the need for frequent joins when querying user-specific order data. This can significantly improve the performance of queries that retrieve orders along with user information.

Now the query we have only deals with the orders table, without joining the users table and has the same results.



SELECT
    orders.order_id,
    orders.name,
    orders.email,
    orders.order_date,
    orders.total_amount
FROM
    orders;

However, it's important to note that denormalization comes with trade-offs.

It can lead to data redundancy, increased storage requirements, and potential data inconsistency if updates are not properly managed. Therefore, denormalization should be used judiciously and in situations where the performance benefits outweigh the drawbacks.

Additionally, denormalized data should be kept consistent through proper data management practices such as triggers or application logic.

4. Insufficient Hardware Resources

To monitor Insufficient Hardware Resources performance, consider using a third-party monitoring tool designed specifically for MySQL performance monitoring.

Tools such as MySQL Enterprise Monitor, Percona Monitoring and Management (PMM), New Relic, or open source solutions such as Prometheus with MySQL Exporter can provide comprehensive monitoring capabilities.

📋 Conclusion

Optimizing relational databases is critical to ensuring optimal performance and providing a smooth user experience. By overcoming common bottlenecks such as unnecessary full table scans, inefficient queries, denormalization trade-offs, and insufficient hardware resources, you can significantly improve the speed and efficiency of your MySQL database.

Key strategies include intelligent indexing techniques such as single and composite indexes, writing clean and optimal SQL queries, and judicious application of denormalization when the performance benefits outweigh the disadvantages. Additionally, investing in adequate hardware resources and utilizing monitoring tools can help identify and resolve performance issues proactively.

By implementing these optimization strategies, you can unlock the full potential of your relational database, ensuring lightning-fast data retrieval, efficient data manipulation, and a responsive application experience for your users.

Thank you, hope it helps. 👋