Forem: Idris Akintobi

Understanding Go's context Package: A Guide to Proper Usage

Idris Akintobi — Thu, 13 Nov 2025 20:21:53 +0000

A comprehensive guide to understanding and correctly implementing Go's context package for robust concurrent applications

In the world of concurrent programming in Go, the context package stands as a fundamental tool for managing the lifecycle of goroutines, handling cancellations, and carrying request-scoped data. However, a misunderstanding of its proper application can lead to subtle bugs, resource leaks, and unexpected application behavior. This article will delve into the correct use of context in Golang, ensuring that you, as a Golang engineer, can leverage its power effectively and avoid common pitfalls.

Why Context Matters

At its core, the context package provides a mechanism to control and coordinate concurrent operations. Its primary purposes include:

🚫 Cancellation

It allows for the graceful termination of long-running operations when they are no longer needed, preventing unnecessary work and resource consumption.

⏱️ Timeouts and Deadlines

It enables setting time limits on operations, which is crucial for maintaining system responsiveness and preventing indefinite hangs.

📦 Request-Scoped Data

It provides a way to carry request-specific information, such as user IDs or trace IDs, through the layers of an application without cluttering function signatures with extra parameters.

A context.Context is an immutable interface that carries these signals and values across API boundaries. The convention in Go is to pass the context as the first argument to a function.

Understanding the Context Package

The context package provides several functions to create and derive contexts:

`context.Background()`

Returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used at the highest level of an application (in main or init functions) as the starting point for all other contexts.

`context.TODO()`

Similar to Background(), this also returns an empty Context. It should be used as a placeholder when you are unsure which Context to use or when a function should accept a Context but hasn't been updated yet. It serves as a signal that the code needs to be revisited.

`context.WithCancel(parent Context)`

Returns a copy of the parent context along with a cancel function. Calling the cancel function signals that the work associated with this context should be stopped.

`context.WithDeadline(parent Context, d time.Time)`

Returns a copy of the parent context that will be canceled when the specified deadline is reached.

`context.WithTimeout(parent Context, timeout time.Duration)`

A convenience function that calls WithDeadline with a timeout relative to the current time.

`context.WithValue(parent Context, key, val interface{})`

Allows you to associate a key-value pair with a context. This should be used sparingly for request-scoped data, not for passing optional parameters.

⚠️ Common Context Pitfalls

Understanding the theory is one thing, but applying it correctly in real-world scenarios is what truly matters. Let's explore some common situations where a misunderstanding of context can lead to problems.

Scenario 1: The Database Operation Trap

A frequent pattern in web services is to perform a database operation in response to an incoming HTTP request. The net/http package provides a Context() method on the *http.Request object, which is canceled when the client disconnects.

❌ The Problem

Consider an operation that needs to complete regardless of whether the client stays connected, such as processing a payment or creating a new user. If you pass the request's context to this database operation, and the client disconnects, the context will be canceled. This cancellation will propagate to the database driver (if it supports contexts), potentially causing the database transaction to be rolled back.

✅ The Solution

For operations that should outlive the incoming request, you should not use the request's context. Instead, create a new context, typically derived from context.Background().

Here's a code example using SQLite to illustrate the wrong and right way:

Incorrect Usage:

package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"
    "net/http"
    "time"

    _ "github.com/mattn/go-sqlite3"
)

func setupDatabase() *sql.DB {
    db, err := sql.Open("sqlite3", "./test.db")
    if err != nil {
        log.Fatal(err)
    }
    sqlStmt := `
    CREATE TABLE IF NOT EXISTS users (id INTEGER NOT NULL PRIMARY KEY, name TEXT, created_at DATETIME);
    `
    _, err = db.Exec(sqlStmt)
    if err != nil {
        log.Fatalf("%q: %s\n", err, sqlStmt)
    }
    return db
}

func handleRequest(w http.ResponseWriter, r *http.Request, db *sql.DB) {
    reqCtx := r.Context()  // Initialize the request context
    // ❌ Incorrectly passing the request context to a critical db operation.
    // If the client disconnects, the context is cancelled, and the DB operation will fail.
    err := criticalDBOperation(reqCtx, db)
    if err != nil {
        log.Printf("Handler error: %v", err)
        http.Error(w, "Database operation failed or was cancelled", http.StatusInternalServerError)
        return
    }
    fmt.Fprintln(w, "Request processed and data saved.")
}

func criticalDBOperation(ctx context.Context, db *sql.DB) error {
    // Simulate a delay. If a client disconnects during this time, the context is cancelled.
    time.Sleep(5 * time.Second)
    // db.ExecContext respects the context. If ctx.Done() is closed, it will return an error.
    _, err := db.ExecContext(ctx, "INSERT INTO users (name, created_at) VALUES (?, ?)", "critical_user", time.Now())
    if err != nil {
        log.Println("Database operation cancelled or failed:", err)
        return err
    }
    log.Println("Critical database operation completed successfully")
    return nil
}

func main() {
    db := setupDatabase()
    defer db.Close()
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        handleRequest(w, r, db)
    })
    log.Println("Server starting on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

If you run this server, make a request with curl, and cancel curl before 5 seconds, you will see the "Database operation cancelled or failed: context canceled" message in the server logs.

Correct Usage:

func handleRequest(w http.ResponseWriter, _ *http.Request, db *sql.DB) {
    // Initialize db context. We can use timeout context here to prevent long running db operations but let keep it simple.
    dbCtx := context.Background()
    // ✅ Correctly create a new context for the DB operation that must complete.
    // If the client disconnects, the DB operation will succeed.
    // This operation should be run in a new goroutine so the HTTP handler can return immediately but let keep it as it is.
    err := criticalDBOperation(dbCtx, db)
    if err != nil {
        log.Printf("Handler error: %v", err)
        http.Error(w, "Database operation failed or was cancelled", http.StatusInternalServerError)
        return
    }
    fmt.Fprintln(w, "Request processed and data saved.")
}

In this corrected version, the database operation runs with an independent context, ensuring it completes even if the client disconnects.

Scenario 2: Background Task Independence

Another common use case is spinning up a goroutine to perform a non-critical or "forgivable" task, like sending a login attempt email or logging analytics. It might seem convenient to pass the same context used by the main business logic to this background task.

❌ The Problem

If the context passed to the background goroutine is canceled for any reason (e.g., a timeout in the main logic), the background task will also be canceled. This might be undesirable.

✅ The Solution

For independent, non-critical background tasks, it's often best to create a new, detached context. This ensures that the lifecycle of the background task is not tied to the lifecycle of the main operation.

Incorrect Usage:

package main

import (
    "context"
    "fmt"
    "time"
)

func main() {
    // Simulating login authentication logic cancellable context
    // We are using short-lived context here. This might happen due to client-side timeout, network issuesm server overload, or slow db queries.
    ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond)
    defer cancel()

    fmt.Println("Starting user authentication...")
    // ❌ Start a background task with the same context
    go sendLoginAttemptEmail(ctx)

    // Wait to observe the goroutine's fate
    time.Sleep(200 * time.Millisecond)
}

func sendLoginAttemptEmail(ctx context.Context) {
    // Assuming context is needed for email sending logic
    // In real usecase, we might as well want an internal timeout for the email itself, so it doesn't hang forever.
    select {
    case <-time.After(100 * time.Millisecond):
        fmt.Println("Login attempt email sent successfully")
    case <-ctx.Done():
        fmt.Println("Failed to send login attempt email: context cancelled")
    }
}

Correct Usage:

func main() {
    // Login authentication logic with a 2-second timeout
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()

    fmt.Println("Starting user authentication...")

    // ✅ Start a background task with a new, independent context
    go sendLoginAttemptEmail(context.Background())

    // Simulate authentication process
    select {
    case <-time.After(3 * time.Second):
        fmt.Println("User authentication completed successfully")
    case <-ctx.Done():
        fmt.Println("Authentication timed out, but email task is unaffected.")
    }

    time.Sleep(2 * time.Second) // Wait to observe the goroutine's fate
}

Here, sendLoginAttemptEmail is called with context.Background(), making it independent of the authentication logic's context. The login attempt email will be sent even if the authentication times out.

🎯 Best Practices for Context Usage

To ensure you're using the context package correctly, keep these best practices in mind:

1. Pass Context Explicitly

Always pass the context as the first argument to functions. Do not store it in a struct.

// ✅ Good
func ProcessData(ctx context.Context, data []byte) error {
    // ...
}

// ❌ Bad
type Worker struct {
    ctx context.Context
}

2. Never Pass nil Context

If you're unsure what Context to use, pass context.TODO().

// ✅ Good
ProcessData(context.TODO(), data)

// ❌ Bad
ProcessData(nil, data)

3. Use WithValue Sparingly

It should only be used for request-scoped data that must be passed through API boundaries, not for optional parameters. Always use user-defined types for keys to avoid collisions.

// ✅ Good - request-scoped data with user-defined key types
type contextKey string

const (
    UserIDKey contextKey = "userID"
    TraceIDKey contextKey = "traceID"
)

ctx = context.WithValue(ctx, UserIDKey, userID)
ctx = context.WithValue(ctx, TraceIDKey, traceID)

// Helper functions for type-safe retrieval
func GetUserID(ctx context.Context) (string, bool) {
    userID, ok := ctx.Value(UserIDKey).(string)
    return userID, ok
}

func GetTraceID(ctx context.Context) (string, bool) {
    traceID, ok := ctx.Value(TraceIDKey).(string)
    return traceID, ok
}

// ❌ Bad - using string keys directly (collision risk)
ctx = context.WithValue(ctx, "userID", userID)
ctx = context.WithValue(ctx, "traceID", traceID)

// ❌ Bad - configuration data
ctx = context.WithValue(ctx, "maxRetries", 3)

4. Always Call Cancel

When you create a context with WithCancel, WithTimeout, or WithDeadline, you must call the returned cancel function to release the associated resources.

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel() // ✅ Always defer the cancel call

5. Check ctx.Done() in Long Operations

In your goroutines, you should select on the ctx.Done() channel to gracefully exit when the context is canceled.

func longRunningOperation(ctx context.Context) error {
    for {
        select {
        case <-ctx.Done():
            return ctx.Err() // Graceful exit
        default:
            // Do work
            time.Sleep(100 * time.Millisecond)
        }
    }
}

Key Takeaways

The context package is a powerful tool for managing concurrent operations in Go, but it requires careful consideration of when and how to use it:

Use request contexts for request-scoped operations that should be canceled when the client disconnects
Create independent contexts for critical operations that must complete regardless of client state
Isolate background tasks with their own contexts to prevent unwanted cancellation propagation
Understand propagation behavior: cancellation, deadlines, and values automatically propagate downward from parent to child contexts once derived (e.g., via context.WithCancel, context.WithTimeout, or context.WithValue).
Always call cancel functions to prevent resource leaks
Check ctx.Done() regularly in long-running operations for graceful shutdowns

By adhering to these principles and understanding the nuances of context propagation, you can write more robust, efficient, and maintainable concurrent Go applications.

Have you encountered context-related bugs in your Go applications? Share your experiences and solutions in the comments below! 👇

How to Retrieve User Country from IP Address

Idris Akintobi — Wed, 26 Mar 2025 15:06:49 +0000

While working on a small project that needed to return content based on the user's country, I found a convenient, free approach to get the user's country from the IP address in the request object without relying on API calls. However, this approach has its caveats, as you must regularly update your database.

The approach I will discuss involves using a database provided by MaxMind to query IP address information. While there are various types of information you can query, we will focus solely on retrieving the country ISO and city in this article. There are packages that can be used to achieve this, such as geoip-lite or fast-geoip.

We will explore two approaches: one that requires setting up an account with MaxMind to download the latest data, and another that uses a GitHub repository with a frequently updated downloadable database.

If you plan to use this service at an enterprise level, please purchase a MaxMind license to access a more accurate and up-to-date database.

Using Database from Your MaxMind Account

Register for a MaxMind account.
Create a license key (Optional). This is only needed for updating the database.
Go to My Account and select Download Database. Download the database in the format of your choice. We will use the binary (.mmdb) format in this article. Download the zip file and extract it on your server. The CSV format is ideal if you want to dump the data into a database like PostgreSQL or MySQL. You can find the guide to do that here.

Code Implementation for Country ISO

import { Reader } from "@maxmind/geoip2-node";
import { readFileSync } from "fs";

// Synchronous database opening
const dbBuffer = readFileSync("GeoLite2-Country.mmdb");
// This reader object should be reused across lookups as creation of it is expensive.
const reader = Reader.openBuffer(dbBuffer);

const ipAddr = "154.113.170.145";

const IP2Country = (ip) => {
  try {
    const { country } = reader.country(ip);
    return country
      ? { countryCode: country.isoCode, country: country.names["en"] }
      : null;
  } catch (error) {
    console.log(error); // Handle the error as needed. You can make an API call here.
    return null;
  }
};

console.log(IP2Country(ipAddr));
// { countryCode: "NG", country: "Nigeria" }

You can also download the database with city records, which allows you to retrieve city information. Note that the city data may not be available for some IP addresses, so you will need to handle such cases. You should consider purchasing a license to access a more accurate database.

Code Implementation for City Information

import { Reader } from "@maxmind/geoip2-node";
import { readFileSync } from "fs";

// Synchronous database opening
const dbBuffer = readFileSync("GeoLite2-City.mmdb");
// This reader object should be reused across lookups as creation of it is expensive.
const reader = Reader.openBuffer(dbBuffer);

const ipAddr = "154.113.170.145";

const IP2City = (ip) => {
  try {
    const { country, city, continent } = reader.city(ip);
    return {
      city: city?.names["en"],
      country: country?.names["en"],
      countryCode: country?.isoCode,
      continent: continent?.names["en"],
    };
  } catch (error) {
    console.log(error); // Handle the error as needed. You can make an API call here.
    return null;
  }
};

console.log(IP2City(ipAddr));
// { city: "Lagos", country: "Nigeria", countryCode: "NG", continent: "Africa" }

The city database is quite large and can be resource-intensive to load into memory, depending on your server resources. If city data is necessary for your service, I recommend downloading the CSV database and importing it into your database to query. The guide can be found here.

Using Database from Sapic IP-Location-DB GitHub Repository

This approach does not require setting up a MaxMind account, as the database files are already available in the repository and are frequently updated.

Go to the repository and select geolite2-country-mmdb to download the database. We will be using the IPv4 binary database, but you can also download the IPv6 database and implement the function if needed.

This minimal database is resource-efficient, as the data loaded into memory is smaller.

Code Implementation for Country ISO (Using Sapic Repository)

import { readFileSync } from "fs";
import { Reader } from "maxmind";

// Synchronous database opening
const dbBuffer = readFileSync("geolite2-country-ipv4.mmdb");
// This reader object should be reused across lookups as creation of it is expensive.
const reader = new Reader(dbBuffer);

const ipAddr = "154.113.170.145";

const IP2Country = (ip) => {
  try {
    return { countryCode: reader.get(ip).country_code };
  } catch (error) {
    console.log(error); Handle the error as needed. You can make an API call here.
  }
};

console.log(IP2Country(ipAddr));
// { countryCode: "NG" }

Code Implementation for City Information (Using Sapic Repository)

import { readFileSync } from "fs";
import { Reader } from "maxmind";

// Synchronous database opening
const dbBuffer = readFileSync("geolite2-city-ipv4.mmdb");
// This reader object should be reused across lookups as creation of it is expensive.
const reader = new Reader(dbBuffer);

const ipAddr = "154.113.170.145";

const IP2City = (ip) => {
  try {
    const { city, country_code } = reader.get(ip);
    return { city, countryCode: country_code };
  } catch (error) {
    console.log(error); // Handle the error as needed. You can make an API call here.  }
};

console.log(IP2City(ipAddr));
// { city: "Lagos", countryCode: "NG" }

Conclusion

In this article, we explored two ways to retrieve country and city data from an IP address without relying on API requests. Both methods involve using a local database that needs to be updated regularly to ensure new IP addresses are included. It is essential to automate the process of downloading the latest database and updating your local copy, as MaxMind updates its GeoLite2 database every Tuesday and Friday. You should set up a job that runs to update your database and keep in mind that the database size will grow as it is being updated. Here is a solution created by MaxMind.

Ensure you abide by MaxMind's terms of service when using the data, and consider purchasing a license for more accurate and up-to-date information.

A Step-by-Step Guide to Google OAuth2 Authentication with JavaScript and Bun

Idris Akintobi — Tue, 28 Jan 2025 23:02:02 +0000

When it comes to authenticating users with Google, OAuth2 provides a secure and standardized approach. While there are libraries that abstract much of the process, this article walks you through a foundational implementation using JavaScript and an Express server. By the end, you'll understand the underlying concepts and have a running example of Google OAuth2.

The project can be run with any JavaScript runtime engine like Node or Deno, but we are using Bun (a fast, modern JavaScript runtime) to run our project.

Setting Up Google OAuth2

Before diving into the code, configure your project in the Google Cloud Console.

Step 1: Create a Google Cloud Project

Visit Google Cloud Console.
Create a new project if you do not have an existing one.

Step 2: Configure OAuth Consent Screen

Navigate to API & Services > OAuth Consent Screen to configure your application.
Fill in the necessary details like app name and support email.
Customize branding (optional) to provide more details about your app’s consent screen.

Step 3: Set Up Scopes

Under the Scopes section, add:
- email: Access the user’s email address.
- openid: OpenID Connect scope for identity verification.
- profile: Access basic profile information like name and picture.

Step 4: Create OAuth Client ID

It might take a few minutes before you can create a client ID.

Go to Credentials > Create Credentials > OAuth Client ID.
Select Web Application as the application type.
Set the Authorized JavaScript Origin to http://localhost:3000.
Add a Redirect URI: http://localhost:3000. Replace these with your hosted URI when deploying to production. Google will redirect users to the Redirect URI after authentication, appending an authorization code and state parameters. The redirect URI can be another page or the backend service. For simplicity, we are using the same page to handle the authenticated user code.

Step 5: Add Test Users

In test mode, you must explicitly add the email addresses of users you want to test the flow with. You should publish your app once you're satisfied with the test.

Coding the Google OAuth2 Implementation

Project Setup

Install Bun:

If you don’t have Bun installed, follow the instructions at bun.sh. We are using Bun version 1.2.
Initialize the Project:

Create a new directory for your project and initialize it with Bun:

   mkdir google-oauth-express
   cd google-oauth-express
   bun init

Folder Structure

Organize your project as follows:

project/
├── public/
│   └── index.html
├── src/
│   └── index.js
├── .env
└── bun.lock

Install Dependencies: Install the required dependencies:

   bun add express google-auth-library

Set Environment Variables: Create a .env file and add:

   GOOGLE_CLIENT_ID=your_google_client_id
   GOOGLE_CLIENT_SECRET=your_google_client_secret
   GOOGLE_REDIRECT_URI=http://localhost:3000

`index.html`

The HTML file provides a simple interface for initiating Google Sign-In and handling the OAuth redirect.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Google OAuth Test App</title>
    <script>
    //   Function to capture URL parameters and handle the OAuth redirect
    // If you're using React, you can have this run on page load effect
      function handleOAuthCallback() {
        const urlParams = new URLSearchParams(window.location.search);
        const code = urlParams.get("code");
        const state = urlParams.get("state");
        const error = urlParams.get("error");

        if (code && state) {
          // Check the CSRF token
          const storedState = localStorage.getItem("oauth_state");
          if (state === storedState) {
            // Send the code to the server for token exchange
            // For security reason you should have some logic on the backend to ensure requests only come from your apps. Such as using CORS for web apps.
            fetch("/auth/google/login", {
              method: "POST",
              headers: {
                "Content-Type": "application/json",
              },
              body: JSON.stringify({ code }),
            })
              .then((response) => response.json())
              .then((data) => {
                console.log("Success:", data);
              })
              .catch((error) => {
                console.error("Error:", error);
              });
          } else if (error) {
            console.error("Authentication failed", error);
          } else {
            console.error("State mismatch");
          }
        }
      }

      // Call the function when the page loads
      handleOAuthCallback();
    </script>
  </head>
  <body>
    <h1>Google OAuth Test App</h1>
    <button id="google-signin-button">Sign in with Google</button>

    <script>
      const GOOGLE_OAUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth";
      const GOOGLE_CLIENT_ID = "your_google_client_id"; // Replace with your client ID
      const GOOGLE_CALLBACK_URL = "http://localhost:3000";
      const GOOGLE_OAUTH_SCOPES = [
        "https://www.googleapis.com/auth/userinfo.email",
        "https://www.googleapis.com/auth/userinfo.profile",
      ];

      // Function to generate the OAuth URL and redirect the user
      function initiateGoogleOAuth() {
        const state = crypto.randomUUID(); // Generate a CSRF token
        localStorage.setItem("oauth_state", state);

        const params = new URLSearchParams({
          client_id: GOOGLE_CLIENT_ID,
          redirect_uri: GOOGLE_CALLBACK_URL,
          access_type: "offline",
          response_type: "code",
          state: state,
          scope: GOOGLE_OAUTH_SCOPES.join(" "),
        });

        const GOOGLE_OAUTH_CONSENT_SCREEN_URL = `${GOOGLE_OAUTH_URL}?${params.toString()}`;
        window.location.href = GOOGLE_OAUTH_CONSENT_SCREEN_URL;
      }

      // Attach the function to the button click event
      document
        .getElementById("google-signin-button")
        .addEventListener("click", initiateGoogleOAuth);
    </script>
  </body>
</html>

`index.js`

This file handles the server-side implementation of Google OAuth2.

import express from "express";
import { OAuth2Client } from "google-auth-library";

const app = express();

app.use(express.static("public"));
app.use(express.json());

const GOOGLE_CLIENT_ID = process.env.GOOGLE_CLIENT_ID;
const GOOGLE_CLIENT_SECRET = process.env.GOOGLE_CLIENT_SECRET;
const GOOGLE_REDIRECT_URI = process.env.GOOGLE_REDIRECT_URI;

if (!GOOGLE_CLIENT_ID || !GOOGLE_CLIENT_SECRET || !GOOGLE_REDIRECT_URI) {
  throw new Error("Missing required environment variables for Google OAuth.");
}

const googleClient = new OAuth2Client(GOOGLE_CLIENT_ID);

// Middleware to check if the user is authenticated
async function isAuthenticated(req, res, next) {
  const token = req.headers.authorization?.split(" ")[1]; // Extract token from "Bearer <token>"

  if (!token) {
    return res.status(401).json({ message: "Unauthorized: No token provided" });
  }

  try {
    // This is to verify the google id_token.
    const ticket = await googleClient.verifyIdToken({
      idToken: token,
      audience: GOOGLE_CLIENT_ID,
    });

    const payload = ticket.getPayload();
    req.user = payload;
    next();
  } catch (error) {
    console.error("Token verification failed:", error);
    res.status(401).json({ message: "Unauthorized: Invalid token" });
  }
}

// Route to handle Google OAuth login
app.post("/auth/google/login", async (req, res) => {
  const { code } = req.body;

  if (!code) {
    return res.status(401).json({ message: "Unauthorized" });
  }

  try {
    const response = await fetch("https://oauth2.googleapis.com/token", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        code,
        client_id: GOOGLE_CLIENT_ID,
        client_secret: GOOGLE_CLIENT_SECRET,
        redirect_uri: GOOGLE_REDIRECT_URI,
        grant_type: "authorization_code",
      }),
    }).then((res) => res.json());

    // The `response` object contains id_token which is a jwt token with the payload of the user information. If you have existing application authorization logic
    // you can use it here by checking if the email address in the jwt payload exists in your database and return the authorization token to the user.
    // In this article we are using the id_token provided by google as the authorization token.
    res.json({ token: response.id_token });
  } catch (error) {
    console.error("Error during token exchange:", error);
    res.status(500).json({ message: "Failed to authenticate with Google" });
  }
});

// Protected route example
app.get("/protected", isAuthenticated, (req, res) => {
  res.json({ message: "You are authenticated!", user: req.user });
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server is listening on port ${PORT}`);
});

Running the Project

Run the Server: Execute the following command:

   bun run src/index.js

Test the Flow:
- Open http://localhost:3000 in your browser.
- Click Sign in with Google.
- Authenticate with your Google account.
- Check the console for the token.
- Use the bearer token to access the /protected route.

Conclusion

This implementation provides a solid understanding of Google OAuth2 and can be ported to other frameworks or production-ready applications. While tools like passport or next-auth simplify OAuth2, building it manually ensures you understand how authentication works under the hood.

Setting Up Paystack for Subscription-Based Billing in NestJS

Idris Akintobi — Mon, 23 Dec 2024 18:51:52 +0000

Subscriptions are integral to many applications, enabling recurring billing for customers. This tutorial walks you through integrating Paystack for subscription-based billing using NestJS, providing insights into key steps, potential pitfalls, and best practices.

1. Understanding Paystack’s Subscription Model

Before diving into the code, let’s clarify some basics:

Account Setup: Create a Paystack account. Note that, as at the time of writing this article, Paystack accepts account creation per currency and supports only a few countries. Confirm if your country and currency are supported.
Currency Handling: A business operating in multiple currencies needs separate accounts for each. Payments in foreign currencies update your account balance in your default currency unless specified otherwise. Reach out to Paystack support for clarification.
Testing Mode: Always test in Test Mode before switching to Live Mode.

2. Bootstrapping the NestJS Application

Let’s start by creating a new NestJS project using the Nest CLI:

nest new paystack-nestjs --strict

Move into the project directory:

cd paystack-nestjs

For simplicity, we’ll leave the default files intact and focus on implementing our features.

3. Managing Environment Variables

We’ll use the @nestjs/config package to manage environment variables:

npm i --save @nestjs/config

Import the ConfigModule into the application module to make environment variables accessible.

Setting Up Environment Variables

Create a .env file:

touch .env && echo "\n.env" >> .gitignore

Add the following variables:

PAYSTACK_SECRET=your_paystack_secret_key
PAYSTACK_BASE_URL=https://api.paystack.co
PAYSTACK_SUCCESS_URL=https://example.com #Page the customer will be redirected to after a successful charge.

Retrieve the secret key from the API & Webhooks tab on the Paystack.

4. Webhook Configuration

Paystack uses webhooks to notify your system for events like successful charges. You can use Webhook.site to view and confirm that the payloads were received.

Generate a webhook URL and paste it into the Paystack Webhook field.
Save your changes and keep the Webhook.site page open to monitor events.

5. Creating a Subscription Plan

To enable subscriptions, create a plan in Paystack:

Set the plan interval and maximum charges or leave it untouched for continuous billing.

6. Implementing Paystack Client

Generate a PaystackClient class to handle API calls:

nest generate class paystack/PaystackClient --no-spec --flat

Install Axios to make HTTP requests:

npm install --save axios

PaystackClient Methods

The client will include:

createCustomer: Registers a customer on Paystack.
findCustomerByEmail: Searches for a customer using their email.
listSubscriptions: Lists active customer subscriptions.
initializeTransaction: Creates a payment link for customer charge authorization. Various payment methods is available. This is enough if you only want a one time payment because you can set the amount you want to charge the customer in the request.
initializeSubscription: Creates a payment link for customer charge authorization. This is to charge and subscribe a customer a plan. We are passing the planId here which the plan cost will override any amount set in the request. Only card payment is available with this.
getSubscription: Fetches subscription details by ID.
createSubscription: Subscribes a customer to a plan.
cancelSubscription: Cancels an active subscription.

import { BadRequestException, Injectable, InternalServerErrorException, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import axios, { AxiosInstance } from 'axios';
import type { Customer } from 'src/types/dtos';
import type {
    CustomerResponseDTO,
    PaystackCustomer,
    PaystackPaymentSessionResponse,
    PaystackPaymentSessionResult,
    PlanResponse,
    SubscriptionData,
    SubscriptionResponse,
} from 'src/types/paystack-api-types';

@Injectable()
export class PaystackClient {
    private readonly httpInstance: AxiosInstance;

    private readonly checkoutSuccessUrl: string;

    private readonly logger: Logger;

    public constructor(private readonly configService: ConfigService) {
        this.httpInstance = axios.create({
            baseURL: this.configService.get('PAYSTACK_BASE_URL'),
            headers: {
                Authorization: `Bearer ${this.configService.get('PAYSTACK_SECRET')}`,
                'Content-Type': 'application/json',
            },
        });
        this.logger = new Logger('PAYSTACK CLIENT');
        this.checkoutSuccessUrl = this.configService.get('PAYSTACK_SUCCESS_URL');
    }

    async createCustomer(customerData: Customer): Promise<PaystackCustomer> {
        try {
            const { data } = await this.httpInstance.post<CustomerResponseDTO>('/customer', {
                first_name: customerData.firstName,
                last_name: customerData.lastName,
                email: customerData.email,
                phone: customerData.phoneNumber,
            });

            return data.data;
        } catch (e) {
            this.handleError(e);
        }
    }

    async findCustomerByEmail(email: string): Promise<PaystackCustomer> {
        try {
            const { data } = await this.httpInstance.get<CustomerResponseDTO>(`/customer/${email}`);
            return data.data;
        } catch (e) {
            this.handleError(e);
        }
    }

    async listSubscriptions(email: string): Promise<SubscriptionData[] | null> {
        const { subscriptions } = await this.findCustomerByEmail(email);

        return subscriptions.length ? subscriptions : null;
    }

    /**
     * This method is to initialize a charge transaction. We need to pass the amount that we want the customer to be charged.
     * Various payment method is available.
     * We concatenate the planId and transactionId as the value to the reference because we will need the planId to subscribe the user to the plan when the webhook event is received.
     */
    async initializeTransaction(
        customerEmail: string,
        planId: string,
        transactionId: string,
    ): Promise<PaystackPaymentSessionResult> {
        try {
            const planCost = await this.getPlanCost(planId);
            const { data } = await this.httpInstance.post<PaystackPaymentSessionResponse>('/transaction/initialize', {
                email: customerEmail,
                amount: planCost,
                reference: planId + '__' + transactionId,
                callback_url: this.checkoutSuccessUrl + '?' + transactionId, // This is to add the transactionId as a query parameter to the checkoutSuccessUrl
            });

            return {
                authorizationUrl: data.data.authorization_url,
                accessCode: data.data.access_code,
                reference: data.data.reference,
            };
        } catch (e) {
            this.handleError(e);
        }
    }

    /**
     * This method is to charge a customer and subscribe them to a plan.
     * Only card payment is allowed. Other payment methods can not be charged automatically
     */
    async initializeSubscription(
        customerEmail: string,
        planId: string,
        transactionId: string,
    ): Promise<PaystackPaymentSessionResult> {
        try {
            const { data } = await this.httpInstance.post<PaystackPaymentSessionResponse>('/transaction/initialize', {
                email: customerEmail,
                plan: planId,
                reference: transactionId,
                callback_url: this.checkoutSuccessUrl + '?' + transactionId,
            });

            return {
                authorizationUrl: data.data.authorization_url,
                accessCode: data.data.access_code,
                reference: data.data.reference,
            };
        } catch (e) {
            this.handleError(e);
        }
    }

    async getSubscription(id: string): Promise<SubscriptionData> {
        try {
            const { data } = await this.httpInstance.get<SubscriptionResponse>(`/subscription/${id}`);
            return data.data;
        } catch (e) {
            this.handleError(e);
        }
    }

    async createSubscription(customerId: string, planId: string): Promise<void> {
        try {
            await this.httpInstance.post<SubscriptionResponse>('/subscription/', {
                customer: customerId,
                plan: planId,
            });
        } catch (e) {
            this.handleError(e);
        }
    }

    async cancelSubscription(subCode: string, emailToken: string): Promise<void> {
        try {
            await this.httpInstance.post(`/subscription/disable`, {
                code: subCode,
                token: emailToken,
            });
        } catch (e) {
            this.handleError(e);
        }
    }

    /**
     * This is a function to get the cost of a plan.
     * For simplicity a map can be used but we want a single source of truth.
     */
    private async getPlanCost(id: string): Promise<string> {
        try {
            const { data } = await this.httpInstance.get<PlanResponse>(`/plan/${id}`);
            return data.data.amount.toString();
        } catch (e) {
            if (axios.isAxiosError(e)) {
                this.handleError(e);
            }
        }
    }

    private handleError(e: Error) {
        if (axios.isAxiosError(e)) {
            const response = e.response?.data as Record<string, unknown>;
            this.logger.warn('Paystack API error', JSON.stringify(response));
            throw new BadRequestException(response.message || 'Paystack API error');
        }
        throw new InternalServerErrorException((e as Error).message);
    }
}

We will be using the initializeTransaction in our implementation because we want to be able to show various payment methods to our customers and manually subscribe them to the plan when we receive the charge.success event. We had to concatenate the planId and unique transactionId with the separator __ because the charge.success event payload will not contain any plan details if initializeTransaction method was used.
With this implementation, auto renewal will only be applicable to users who has the card authorization; user who paid with their cards.

7. Building the Paystack Service

Generate the Paystack service:

nest generate service paystack/Paystack --no-spec --flat

PaystackService Methods

This service will include:

createPaymentSession: Generates an authorization URL for the customer.
cancelSubscription: Cancels an active subscription.

import { BadRequestException, Inject, Injectable, Logger } from '@nestjs/common';
import { PaystackEvents } from 'src/types/enums';
import { PaystackPaymentSessionResult, PaystackWebhookPayload } from 'src/types/paystack-api-types';
import { PaystackPaymentSessionDTO } from '../types/dtos';
import { PaystackClient } from './paystack-client';

@Injectable()
export class PaystackService {
    private readonly logger: Logger;

    constructor(@Inject(PaystackClient) private readonly client: PaystackClient) {
        this.logger = new Logger('PAYSTACK SERVICE');
    }

    public async createPaymentSession(
        paymentSession: PaystackPaymentSessionDTO,
    ): Promise<PaystackPaymentSessionResult> {
        this.logger.localInstance;
        if (!paymentSession.planId) {
            throw new BadRequestException('Paystack Plan ID is required');
        }

        if (!paymentSession.transactionId) {
            throw new BadRequestException('transaction ID is required');
        }

        const customer = await this.client.createCustomer(paymentSession.customer);

        const activeSubscriptionResult = await this.client.listSubscriptions(customer.email);

        if (activeSubscriptionResult) {
            throw new BadRequestException('customer already has an active subscription');
        }

        const subscriptionResponse = await this.client.initializeTransaction(
            paymentSession.customer.email,
            paymentSession.planId,
            paymentSession.transactionId,
        );

        return subscriptionResponse;
    }

    // In our implementation, the customer can only be subscribed to a plan at a time.
    //There will only be one active subscription for a customer and they can cancel it if they want to subscribe to another plan.
    public async cancelSubscription(email: string): Promise<void> {
        const customerSubscriptions = await this.client.listSubscriptions(email);
        if (!customerSubscriptions) {
            throw new BadRequestException('User have no active subscription');
        }

        const activeSubscriptions = customerSubscriptions[0];
        await this.client.cancelSubscription(activeSubscriptions.subscription_code, activeSubscriptions.email_token);
    }

8. Creating the Controller

Generate a controller for the service:

nest generate controller paystack/Paystack --no-spec --flat

The controller will include /create endpoint to handle payment session creation requests.

import { Body, Controller, Inject, Post } from '@nestjs/common';
import { PaystackPaymentSessionDTO } from 'src/types/dtos';
import { PaystackService } from './paystack.service';

@Controller('paystack')
export class PaystackController {
    constructor(@Inject(PaystackService) private readonly paystackService: PaystackService) {}

    @Post('/create')
    async createPaymentSession(@Body() body: PaystackPaymentSessionDTO) {
        return await this.paystackService.createPaymentSession(body);
    }
}

9. Handling Webhooks

Webhook Payload Handling

When a charge succeeds, We need to subscribe the user to the plan. We can get the planId to subscribe the customer to from the reference by separating it with __. We have created the customer record and it is updated with their payment authorization after the charge.

Event Handling: We will handle events charge.success and subscription.create events.

Update the Paystack service with a handleWebhookEvents method to process these events and update the controller with the route to handle webhook events.

Add webhook-event route in the controller

Securing Webhooks

Use x-paystack-signature in there webhook event request to verify payload authenticity. Add the rawBody option to your application to ensure you can get the raw request body:

Generate a guard for signature validation:

nest generate guard paystack/PaystackWebhook --no-spec --flat

import { CanActivate, ExecutionContext, Injectable, UnauthorizedException } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import * as crypto from 'crypto';

@Injectable()
export class PaystackWebhookGuard implements CanActivate {
    constructor(private readonly config: ConfigService) {}

    async canActivate(context: ExecutionContext): Promise<boolean> {
        const http = context.switchToHttp();
        const request = http.getRequest();
        const secret = this.config.get<string>('PAYSTACK_SECRET');

        const signature = request.headers['x-paystack-signature'];
        if (!signature) {
            throw new UnauthorizedException('No signature provided');
        }

        const hash = crypto.createHmac('sha512', secret).update(JSON.stringify(request.body)).digest('hex');

        const signatureBuffer = Buffer.from(signature);
        const hashBuffer = Buffer.from(hash);

        try {
            const isValid =
                signatureBuffer.length === hashBuffer.length && crypto.timingSafeEqual(signatureBuffer, hashBuffer);

            if (!isValid) {
                throw new UnauthorizedException('Invalid signature');
            }

            return true;
        } catch (error) {
            throw new UnauthorizedException('Invalid signature');
        }
    }
}

Add the guard to the webhook-event route

10. Testing and Deployment

You can host the service publicly or use VS Code port forwarding tunnels for testing.
Update the webhook URL on Paystack with your public endpoint.

Final Checks

Simulate a payment, monitor webhook logs, and confirm subscription creation on the Paystack dashboard.

Conclusion

This guide covered integrating Paystack for subscription billing in a NestJS application. We explored creating subscriptions, handling webhooks, and securing webhook endpoints. While we focused on the basics, additional steps like proper logging, error handling, and input validation are crucial for production environments.

You can find the complete code for this implementation on GitHub: https://github.com/IdrisAkintobi/paystack-subscription-nestjs.

Happy coding! 🚀

Setting up a SOCKS Proxy Server with Dante on AWS

Idris Akintobi — Thu, 11 Jan 2024 23:23:10 +0000

A SOCKS proxy server serves various purposes. In this guide, we'll explore how to set up one using Dante on Amazon Web Services (AWS), particularly useful in scenarios such as running applications on a serverless solution where external services request your service's static public IP for whitelisting. As serverless solutions limit direct IP provision, employing a SOCKS proxy offers a versatile solution by routing all requests to the external service through the SOCKS server. It's advisable to assign a public static IP for the SOCKS server in this context. Additionally, we'll address another common use case: enhancing privacy and security by configuring our SOCKS server to route all requests through the dynamic public IP address assigned to the server instance during creation. For a guide on how to host this on Google Cloud, you can click here.

Requirements

To set up a SOCKS proxy server on AWS, you'll need:

An AWS account
A security group rule
An EC2 instance (virtual machine)

Security Group Rule Setup

Go to the AWS Management Console and navigate to the EC2 Dashboard > Security Groups.
Click on "Create Security Group." Name the security group (e.g., 'allow-sock-1047') and create a security group description.
Create security group Inbound Rules: Add a rule to allow incoming traffic on port 22 for ssh and your desired SOCKS port (e.g., port 1047). Apply additional security rules, such as IP restrictions, as needed.

EC2 Instance Setup

Navigate to the EC2 Dashboard > Instances.
Click on "Launch Instance."
Name your instance (e.g., 'sock-server') and choose an Amazon Linux 2023 (AMI). If you opt for a different image, make sure to update the commands in the script to be compatible with the selected machine image.
Choose an appropriate instance type (e.g., t2.micro, which is free-tier eligible).
Select an existing key pair or create a new one to access your instance securely.
Click 'Edit' in network settings and enable 'Auto-assign public IP.'
Click on 'Select existing security group' and choose the 'allow-sock-1047' rule.
Click 'Advanced options' and paste the script into the 'User Data' text area field.
Review your configurations and click on "Launch."

script

#!/bin/bash

# Set terminal type
export TERM=xterm-256color

# Variables
SOCK_USERNAME="socksproxy"
SOKSPROXY_PASS="socksproxy"
SOCK_PORT=1047

# Function to print stage
print_stage() {
    local TEXT="$1"
    echo -e "\n\e[32m\e[1m $TEXT \e[0m\e[32m"
}

# Update system and install dante-server dependencies
print_stage "Updating System and install development tools"
# use yum to install if you've selected machine image earlier than 2023
sudo dnf update -y
sudo dnf groupinstall "Development Tools" -y
sudo dnf install libevent-devel -y

# Download and install dante-server
print_stage "Downloading and installing dante-server"
curl -O https://www.inet.no/dante/files/dante-1.4.3.tar.gz
tar xvzf dante-1.4.3.tar.gz
cd dante-1.4.3
./configure
make && make install

# Create dante configuration file
print_stage "Setting up dante-server"
cat <<EOF | tee /etc/sockd.conf
logoutput: /var/log/sockd.log
internal: 0.0.0.0 port = ${SOCK_PORT}
external: enX0
clientmethod: none
socksmethod: username
user.privileged: root
user.notprivileged: nobody
client pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    log: error connect disconnect
}
socks pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    log: error connect disconnect
}
EOF

# Add socksproxy user
print_stage "Adding ${SOCK_USERNAME} user"
sudo useradd -r -s /bin/false ${SOCK_USERNAME}
echo "${SOCK_USERNAME}:${SOKSPROXY_PASS}" | sudo chpasswd

# Start sockd
sudo /usr/local/sbin/sockd -D

print_stage "Installation Completed"

# Get public IP address (metadata version 1)
# PUBLIC_IPV4=$(curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address 2>/dev/null || echo "N/A")

# Get public IP address (metadata version 2 - token required)
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600" 2>/dev/null)
PUBLIC_IPV4=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/public-ipv4 2>/dev/null || echo "N/A")

# Display configuration details and IP addresses
print_stage "Configuration Details:"
echo -e "\033[1mIP:\033[0m ${PUBLIC_IPV4}"
echo -e "\033[1mUsername:\033[0m ${SOCK_USERNAME}"
echo -e "\033[1mPassword:\033[0m ${SOKSPROXY_PASS}"
echo -e "\033[1mPort:\033[0m ${SOCK_PORT}"

Script Explanation

For brevity, we'll focus on the Dante configuration in the script. For more configuration options you can visit here. The script uses Dante version 1.4.3 which is the latest at the time of writing this article. Version 1.5 is currently in testing phase and it is set to roll out first quarter of this year.

logoutput: /var/log/sockd.log: Specifies the path for the log file. In this case, it will be stored at /var/log/sockd.log.
internal: 0.0.0.0 port = ${SOCK_PORT}: Configures the internal IP address and port on which the SOCKS server will listen for incoming connections. The variable ${SOCK_PORT} is used to set the specified port (in this case, 1047).
external: ens4: Defines the external network interface to bind to. In this script, it uses the interface ens4.
clientmethod: none: Specifies that clients connecting to the SOCKS server do not need to authenticate.
socksmethod: username: Specifies that the SOCKS server will use username/password authentication for connections.
user.privileged: root: Sets the privileged user for the SOCKS server to root.
user.notprivileged: nobody: Sets the non-privileged user for the SOCKS server to nobody.
client pass { … }: Defines a rule allowing connections from any IP (0.0.0.0/0) to any destination. It logs errors related to connection and disconnection.
socks pass { … }: Similar to the client pass rule, this allows SOCKS connections from any IP to any destination and logs related errors.

Once the instance is running, SSH into it to view the script log using cat /var/log/cloud-init-output.log. Please note that the script might take a while to complete due to the compilation of Dante and the use of a small machine type.
You can also rerun the startup script from the SSH terminal by using sudo bash /var/lib/cloud/instance/user-data.txt. This could be useful when you edit the instance startup script and need to rerun it manually.

Testing the SOCKS Proxy

Check if the configured port is open using:
```
nc -zv ${IP} ${PORT}
```
Verify the SOCKS server functionality by curling Google's home page with the command:

socks5://${USERNAME}:${PASS}@${IP}:${PORT} https://www.google.com

SOCKS Usage

In browser
Install the Proxy Toggle Firefox extension and configure it with the SOCKS settings to route browser traffic through the SOCKS server.
In code
Use Axios in JavaScript with the socks-proxy-agent module for making requests through the SOCKS server.
```
import axios from 'axios';
import { SocksProxyAgent } from 'socks-proxy-agent';
```

// Get socks config from the environment. - Assuming the variables are available
const { SOCKS_HOST, SOCKS_PORT, SOCKS_USERNAME, SOCKS_PASSWORD } = process.env;

// Setup proxy agent
const socksV5Agent = new SocksProxyAgent(socks5://${SOCKS_USERNAME}:${SOCKS_PASSWORD}@${SOCKS_HOST}:${SOCKS_PORT});

// Create socks axios instance
const socksClient = axios.create({
httpsAgent: socksV5Agent,
httpAgent: socksV5Agent,
});

// Use the instance
(async () => {
try {
const { data } = await socksClient.get('https://api.ipify.org');
console.log(Our socks IP is ${data});
} catch (error) {
console.error('Error fetching IP:', error.message);
// Handle the error as needed
}
})();


## Conclusion

In conclusion, this guide has equipped you with the knowledge and step-by-step instructions to seamlessly set up a SOCKS proxy server on AWS using Dante. By following the outlined procedures, you can establish a robust SOCKS proxy server ready to meet your specific needs.

Setting up a SOCKS Proxy Server with Dante on Google Cloud

Idris Akintobi — Mon, 08 Jan 2024 22:16:18 +0000

A SOCKS proxy server can serve various purposes, and in this guide, we'll explore how to set up one using Dante on Google Cloud. One common use case arises when you're running your applications on a serverless solution, and external services request your service's static public IP for whitelisting. Since we know that serverless solutions limit direct IP provision, employing a SOCKS proxy can be a versatile solution. All requests to the external service can be routed through the SOCKS server. It is advisable to assign a public static IP for the SOCKS server in this kind of solution. Another common use case we will consider in this article is enhancing privacy and security. We will be setting up our sock server to route all requests through the dynamic public IP address assigned to the server instance when created. For a guide on how to host this on AWS, you can click here.

Requirements

To set up a SOCKS proxy server, you'll need:

A Google Cloud account
A firewall rule
A virtual machine (VM) instance

Firewall Rule Setup

Go to the GCloud console, navigate to VPC Network > Firewall.
Click on "Create Firewall Rule."
Name the rule (e.g., 'allow-sock-1047') and create a target tag that will be used to tag the VM. In this case, we are using the rule name as the target tag.
Set the rule to allow traffic on your desired port. For this guide, we will be using port 1047. You can apply additional security rules, such as IP restrictions, as needed.

VM Instance Setup

Navigate to Compute Engine > VM Instances.
Click on "Create Instance."
Name your instance (e.g., 'sock-server') and set the location to your preference.
Choose an appropriate machine type (e.g., e2-micro to reduce costs).
Under Advanced Options and Networking, add the 'allow-sock-1047' tag (firewall rule tag).
In the Management section, paste the provided script into the automation text field, then click create.

script

#!/bin/bash

# Set terminal type
export TERM=xterm-256color

# Variables - update these
SOCK_USERNAME="socksproxy"
SOKSPROXY_PASS="socksproxy"
SOCK_PORT=1047

# Function to print stage
print_stage() {
    local TEXT="$1"
    echo -e "\n\e[32m\e[1m $TEXT \e[0m\e[32m"
}

print_stage "Updating System and installing dante-server"
# Install gcc, make, and curl
apt-get update
apt-get install -y gcc make curl

# Download and install dante-server
curl -O https://www.inet.no/dante/files/dante-1.4.3.tar.gz
tar xvzf dante-1.4.3.tar.gz
cd dante-1.4.3
./configure
make && make install

# Create danted configuration file
cat <<EOF | tee /etc/sockd.conf
logoutput: /var/log/sockd.log
internal: 0.0.0.0 port = ${SOCK_PORT}
external: ens4
clientmethod: none
socksmethod: username
user.privileged: root
user.notprivileged: nobody
client pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    log: error connect disconnect
}
socks pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    log: error connect disconnect
}
EOF


# Add socksproxy user
print_stage "Adding ${SOCK_USERNAME} user"
useradd -r -s /bin/false ${SOCK_USERNAME}
echo "${SOCK_USERNAME}:${SOKSPROXY_PASS}" | chpasswd

# Start sockd
/usr/local/sbin/sockd -D

print_stage "Installation Completed"

# Get public IP address
PUBLIC_IPV4=$(curl -s -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip)

# Display configuration details
print_stage "Configuration Details:"
echo -e "\033[1mIP:\033[0m ${PUBLIC_IPV4}"
echo -e "\033[1mUsername:\033[0m ${SOCK_USERNAME}"
echo -e "\033[1mPassword:\033[0m ${SOKSPROXY_PASS}"
echo -e "\033[1mPort:\033[0m ${SOCK_PORT}"

Script Explanation
For brevity, we'll focus on the Dante configuration in the script. For more configuration options you can visit here. The script uses Dante version 1.4.3 which is the latest at the time of writing this article. Version 1.5 is currently in testing phase and it is set to roll out first quarter of this year.

logoutput: /var/log/sockd.log: Specifies the path for the log file. In this case, it will be stored at /var/log/sockd.log.
internal: 0.0.0.0 port = ${SOCK_PORT}: Configures the internal IP address and port on which the SOCKS server will listen for incoming connections. The variable ${SOCK_PORT} is used to set the specified port (in this case, 1047).
external: ens4: Defines the external network interface to bind to. In this script, it uses the interface ens4.
clientmethod: none: Specifies that clients connecting to the SOCKS server do not need to authenticate.
socksmethod: username: Specifies that the SOCKS server will use username/password authentication for connections.
user.privileged: root: Sets the privileged user for the SOCKS server to root.
user.notprivileged: nobody: Sets the non-privileged user for the SOCKS server to nobody.
client pass { … }: Defines a rule allowing connections from any IP (0.0.0.0/0) to any destination. It logs errors related to connection and disconnection.
socks pass { … }: Similar to the client pass rule, this allows SOCKS connections from any IP to any destination and logs related errors.

Once the instance is running, SSH into it to view the script log using sudo journalctl -u google-startup-scripts.service. Please note that the script might take a while to complete due to the compilation of Dante and the use of a small machine type.
You can also rerun the startup script from the SSH terminal by using sudo google_metadata_script_runner startup. This could be useful when you edit the instance startup script and need to rerun it manually.

Testing the SOCKS Proxy

Check if the configured port is open using:

nc -zv ${IP} ${PORT}

Verify the SOCKS server functionality by curling Google's home page with the command:

curl -v -x socks5://${USERNAME}:${PASS}@${IP}:${PORT} https://www.google.com

SOCKS Usage

In browser
Install the Proxy Toggle Firefox extension and configure it with the SOCKS settings to route browser traffic through the SOCKS server.
In code
Use Axios in JavaScript with the socks-proxy-agent module for making requests through the SOCKS server.

import axios from 'axios';
import { SocksProxyAgent } from 'socks-proxy-agent';

// Get socks config from the environment. - Assuming the variables are available
const { SOCKS_HOST, SOCKS_PORT, SOCKS_USERNAME, SOCKS_PASSWORD } = process.env;

// Setup proxy agent
const socksV5Agent = new SocksProxyAgent(`socks5://${SOCKS_USERNAME}:${SOCKS_PASSWORD}@${SOCKS_HOST}:${SOCKS_PORT}`);

// Create socks axios instance
const socksClient = axios.create({
    httpsAgent: socksV5Agent,
    httpAgent: socksV5Agent,
});

// Use the instance 
(async () => {
    try {
        const { data } = await socksClient.get('https://api.ipify.org');
        console.log(`Our socks IP is ${data}`);
    } catch (error) {
        console.error('Error fetching IP:', error.message);
        // Handle the error as needed
    }
})();

Conclusion

In conclusion, this guide has equipped you with the knowledge and step-by-step instructions to seamlessly set up a SOCKS proxy server on Google Cloud using Dante. By following the outlined procedures, from firewall rule setup to the execution of the installation script, you can establish a robust SOCKS proxy server ready to meet your specific needs.

Caveat of using javascript Date object constructor

Idris Akintobi — Sun, 24 Dec 2023 13:42:27 +0000

JavaScript's Date object provides powerful tools for working with dates, but it's important to be mindful of the timezone implications, especially in distributed systems.

Example Scenarios
1. Converting "YYYYMMDDHHMMSS" to Epoch Milliseconds
Imagine a scenario where an external service provides dates in the format "YYYYMMDDHHMMSS," and the task is to convert them to Unix timestamps (epoch milliseconds). The following function accomplishes this task:

function convertYMDHMSToEpoch(dateString) {
    const date = new Date(
        +dateString.substring(0, 4),
        +dateString.substring(4, 6) - 1,
        +dateString.substring(6, 8),
        +dateString.substring(8, 10),
        +dateString.substring(10, 12),
        +dateString.substring(12, 14)
    );
    return date.getTime().toString();
};

However, it's crucial to note that this function may return different values depending on the timezone of the environment where the code is executed.

2. Converting ISO Date String to User-Friendly Format
Suppose you receive a date in ISO format, and the goal is to display it in a user-friendly format. The following code accomplishes this using the Date object constructor:

const isoString = '2023-01-01T23:00:00Z';
new Date(isoString).toString().substring(0, 24);

Again, the issue here is that the output may vary based on the timezone of the runtime environment.

Testing with Different Timezones
To illustrate the impact of timezones, let's take the second example and test it with different timezones:

const isoString = '2023-01-01T23:00:00Z';

process.env.TZ = 'UTC-8';
new Date(isoString).toString().substring(0, 24); // Mon Jan 02 2023 07:00:00

process.env.TZ = 'UTC';
new Date(isoString).toString().substring(0, 24); // Sun Jan 01 2023 23:00:00

The output will vary based on the runtime environment's timezone.

Best Practices

1. Default to UTC
Defaulting backend services to UTC is a widely adopted best practice. You can set the environment variable TZ to 'UTC' to enforce this.

export TZ = UTC

//setting the TZ environment variable to UTC
process.env.TZ = 'UTC'

We can refactor the function in out first example to enforce UTC timezone with the Date constructor.

// Convert "YYYYMMDDHHMMSS" to milliseconds in UTC
mapToTimestamp(dateString) {
    const date = new Date(
        Date.UTC(
            +dateString.substring(0, 4),
            +dateString.substring(4, 6) - 1,
            +dateString.substring(6, 8),
            +dateString.substring(8, 10),
            +dateString.substring(10, 12),
            +dateString.substring(12, 14)
        )
    );
    return date.getTime().toString();
}

Now, this function will return the same value in different timezones.

We can then convert to local time on the client side using the toLocaleString method:

const date = new Date('2023-01-01T12:00:00Z');

// Display in the system's timezone
const formattedDate = date.toLocaleString('en-US');

// Display in a specific timezone passed in the options (e.g., UTC)
const formattedDateUTC = date.toLocaleString('en-US', { timeZone: 'UTC' });

2. Explore Date Manipulation Libraries
Consider using third-party date manipulation libraries like day.js or date-fns. These libraries offer more control over formatting and timezone handling, allowing you to create consistent and customized date displays across different environments.

// Example using day.js
import dayjs from 'dayjs';
import utc from 'dayjs/plugin/utc.js';

dayjs.extend(utc);
process.env.TZ = 'America/New_York';

console.log(dayjs().utc().format()); // Output in UTC
console.log(dayjs().format()); // Output in America/New_York

Conclusion
Considering time zones is essential for robust and reliable JavaScript applications. This combination of defaulting backend services to UTC and converting to local time on the client side promotes consistency in date and time handling throughout your application, contributing to a smoother user experience and mitigating potential issues related to timezone disparities. Date manipulation libraries can also be leveraged to ensure consistency.

SendGrid Mail Service in NestJS

Idris Akintobi — Wed, 20 Dec 2023 15:36:38 +0000

This article illustrates how to integrate SendGrid mail service into your NestJS application.

Step 1: Install NestJS CLI
Skip to step 3 if you already have an existing NestJS application. Install the NestJS CLI globally using npm:

npm install -g @nestjs/cli

Step 2: Create a New NestJS Project
Use the NestJS CLI to create a new project. Name it whatever you like; for this article, we'll use "sendgrid-in-nestjs."

nest new sendgrid-in-nestjs
cd sendgrid-in-nestjs

Step 3: Install SendGrid Package
Install the @sendgrid/mail package:

npm install @sendgrid/mail

Step 4: Set up SendGrid API Key
Obtain your SendGrid API key from the SendGrid dashboard and store it in your environment variables or use a configuration service.

SENDGRID_API_KEY=your-sendgrid-api-key

Step 5: Create Email Module
Create an email module that will contain our implementations.

nest generate module email

Step 6: Create SendGrid Client class
Create SendGridClient in sendgrid-client.ts inside the email folder, which will use the API key from the configuration to interact with the SendGrid API using the @sendgrid/mail package:

nest generate class email/sendgrid-client --flat --no-spec

Step 7: Update SendGrid Client
Update sendgrid-client.ts to use the API key from the configuration:

// src/email/sendgrid-client.ts

import { Injectable, Logger } from '@nestjs/common';
import { MailDataRequired, default as SendGrid } from '@sendgrid/mail';

@Injectable()
export class SendGridClient {
  private logger: Logger;
  constructor() {
    //Initialize the logger. This is done for simplicity. You can use a logger service instead
    this.logger = new Logger(SendGridClient.name);
    //Get the API key from config service or environment variable
    SendGrid.setApiKey(process.env.SENDGRID_API_KEY);
  }

  async send(mail: MailDataRequired): Promise<void> {
    try {
      await SendGrid.send(mail);
      this.logger.log(`Email successfully dispatched to ${mail.to as string}`);
    } catch (error) {
      //You can do more with the error
      this.logger.error('Error while sending email', error);
      throw error;
    }
  }
}

//NOTE You have to set "esModuleInterop" to true in your tsconfig file to be able to use the default key in import.

Step 8: Create Email Service
Create email.service.ts file. It will abstract the SendGridClient, so that we can have different method for specific email use case.

nest generate service email/email --flat --no-spec

Step 9: Update Email Service
Our email service class will consist of 2 methods: "sendTestEmail" and "sendEmailWithTemplate".

// src/email/email.service.ts

import { Injectable } from '@nestjs/common';
import { MailDataRequired } from '@sendgrid/mail';
import { SendGridClient } from './sendgrid-client';

@Injectable()
export class EmailService {
  constructor(private readonly sendGridClient: SendGridClient) {}

  async sendTestEmail(recipient: string, body = 'This is a test mail'): Promise<void> {
    const mail: MailDataRequired = {
      to: recipient,
      from: 'noreply@domain.com', //Approved sender ID in Sendgrid
      subject: 'Test email',
      content: [{ type: 'text/plain', value: body }],
    };
    await this.sendGridClient.send(mail);
  }

  async sendEmailWithTemplate(recipient: string, body: string): Promise<void> {
    const mail: MailDataRequired = {
      to: recipient,
      cc: 'example@mail.com', //Assuming you want to send a copy to this email
      from: 'noreply@domain.com', //Approved sender ID in Sendgrid
      templateId: 'Sendgrid_template_ID', //Retrieve from config service or environment variable
      dynamicTemplateData: { body, subject: 'Send Email with template' }, //The data to be used in the template
    };
    await this.sendGridClient.send(mail);
  }
}

Step 10: Update Email Module
Update the email module by adding the SendGridClient class as a provider and exporting the email service.

// src/email/email.module.ts

import { Module } from '@nestjs/common';
import { EmailService } from './email.service';
import { SendGridClient } from './sendgrid-client';

@Module({
  providers: [EmailService, SendGridClient],
  exports: [EmailService],
})
export class EmailModule {}

Step 11: Use Email Service in Controller
Import the email service and make it available in the appropriate controller.

import { Body, Controller, Get, Post } from '@nestjs/common';
import { AppService } from './app.service';
import { EmailService } from './email/email.service';

@Controller()
export class AppController {
  constructor(
    private readonly appService: AppService,
    private readonly emailService: EmailService,
  ) {}

  @Get()
  getHello(): string {
    return this.appService.getHello();
  }

  @Post('send-test-email')
  async sendEmail(
    @Body() sendEmailDTO: { recipient: string; body: string },
  ): Promise<void> {
    await this.emailService.sendTestEmail(
      sendEmailDTO.recipient,
      sendEmailDTO.body,
    );
  }
}

Step 12: Run your NestJS application
Run your NestJS application:

npm run start:dev

Make a POST request to http://localhost:3000/send-test-email using an API testing tool such as Postman. Include the recipient email and optional body parameters in the request body as JSON payload.

Remember to replace placeholders like 'Sendgrid_template_ID', 'SENDGRID_API_KEY', etc., with your actual values.

Conclusion
In summary, this guide has demonstrated the straightforward integration of SendGrid mail service into your NestJS application. By following the provided steps, you've set up the project, configured the SendGrid API key, and created essential modules and classes. With a modular structure and clear instructions, you now have a solid foundation for incorporating efficient email functionality into your NestJS projects. For any questions or issues, don't hesitate to ask. Happy coding!

Using Redis Client in NestJS

Idris Akintobi — Wed, 27 Sep 2023 21:48:45 +0000

In this article, we will explore how to set up a Redis client in a NestJS application for caching and storing short-lived data. We will provide code snippets and explanations for each step of the process.

Setting Up the Redis Client

First, we need to install the Nest CLI by running the following command in your terminal:

npm install -g @nestjs/cli

Next, create a new NestJS project using the following command, replacing <your-project-name> with your desired project name:

nest new <your-project-name>

This project will focus on two primary uses of Redis: caching and storing short-lived data. We'll use Node.js version 18 and the ioredis library to connect to the Redis service.

Setting Up the Redis Client Factory

To interact with Redis, we need to set up a Redis client factory that will create a Redis connection instance used throughout the code. Here's an example of the Redis client factory:

import { FactoryProvider } from '@nestjs/common';
import { Redis } from 'ioredis';

export const redisClientFactory: FactoryProvider<Redis> = {
    provide: 'RedisClient',
    useFactory: () => {
        const redisInstance = new Redis({
            host: process.env.REDIS_HOST,
            port: +process.env.REDIS_PORT,
        });

        redisInstance.on('error', e => {
            throw new Error(`Redis connection failed: ${e}`);
        });

        return redisInstance;
    },
    inject: [],
};

Please note that in this example, we are not implementing any connection retry mechanism. An error will be thrown if there's an issue with the Redis connection, which may not be the best practice but suits our use case here.

Setting Up the Redis Repository

We will set up a Redis repository to abstract our Redis client and provide methods for interacting with the Redis store in the application layer. Here's an example of the Redis repository:

import { Inject, Injectable, OnModuleDestroy } from '@nestjs/common';
import { Redis } from 'ioredis';
import { RedisRepositoryInterface } from '../../../domain/interface/redis.repository.interface';

@Injectable()
export class RedisRepository implements OnModuleDestroy, RedisRepositoryInterface {
    constructor(@Inject('RedisClient') private readonly redisClient: Redis) {}

    onModuleDestroy(): void {
        this.redisClient.disconnect();
    }

    async get(prefix: string, key: string): Promise<string | null> {
        return this.redisClient.get(`${prefix}:${key}`);
    }

    async set(prefix: string, key: string, value: string): Promise<void> {
        await this.redisClient.set(`${prefix}:${key}`, value);
    }

    async delete(prefix: string, key: string): Promise<void> {
        await this.redisClient.del(`${prefix}:${key}`);
    }

    async setWithExpiry(prefix: string, key: string, value: string, expiry: number): Promise<void> {
        await this.redisClient.set(`${prefix}:${key}`, value, 'EX', expiry);
    }
}

We use the NestJS onModuleDestroy lifecycle event method to close the Redis connection when the application is torn down. This event is triggered when app.close() is called or when the process receives a special system signal, such as SIGTERM, if enableShutdownHooks is called during application bootstrap.

As you can see, we use a prefix for all keys stored in Redis. This allows us to create a folder-like structure when accessing the Redis store with applications like RedisInsight, using the colon (':') as the delimiter. The prefix is defined as an enum value containing all the data stores (collections/tables) used in the project.

Setting Up the Redis Service

Finally, we create a Redis service to be used in the application layer. The Redis service encapsulates the Redis repository and provides the methods needed in the application layer. Here's an example of the Redis service:

import { Inject, Injectable } from '@nestjs/common';
import { RedisPrefixEnum } from '../domain/enum/redis-prefix-enum';
import { ProductInterface } from '../domain/interface/product.interface';
import { RedisRepository } from '../infrastructure/redis/repository/redis.repository';

const oneDayInSeconds = 60 * 60 * 24;
const tenMinutesInSeconds = 60 * 10;

@Injectable()
export class RedisService {
    constructor(@Inject(RedisRepository) private readonly redisRepository: RedisRepository) {}

    async saveProduct(productId: string, productData: ProductInterface): Promise<void> {
        // Expiry is set to 1 day
        await this.redisRepository.setWithExpiry(
            RedisPrefixEnum.PRODUCT,
            productId,
            JSON.stringify(productData),
            oneDayInSeconds,
        );
    }

    async getProduct(productId: string): Promise<ProductInterface | null> {
        const product = await this.redisRepository.get(RedisPrefixEnum.PRODUCT, productId);
        return JSON.parse(product);
    }

    async saveResetToken(userId: string, token: string): Promise<void> {
        // Expiry is set to 10 minutes
        await this.redisRepository.setWithExpiry(
            RedisPrefixEnum.RESET_TOKEN,
            token,
            userId,
            tenMinutesInSeconds,
        );
    }

    async getResetToken(token: string): Promise<string> {
        return await this.redisRepository.get(RedisPrefixEnum.RESET_TOKEN, token);
    }
}

In this service, we use the RedisPrefixEnum as the collection/table name. We have implemented four methods: saveProduct to cache a product for a day, saveResetToken to store a reset token for ten minutes, and getProduct and getResetToken to retrieve stored product and token values, respectively.

Using Redis Service in Other Services

Now, let's see how we can use the Redis service in our password reset service and product service.

Password Reset Service

Here's an example of the password reset service using the Redis service:

import { Inject, Injectable } from '@nestjs/common';
import { ResetTokenInterface } from '../domain/interface/reset.token.interface';
import { RedisService } from './redis.service';

@Injectable()
export class PasswordResetService {
    constructor(@Inject(RedisService) private readonly redisService: RedisService) {}

    async generateResetToken(userId: string): Promise<ResetTokenInterface> {
        // Check if the user exists in the database
        // Generate a random number token with a length of 6
        const token = Math.floor(100000 + Math.random() * 900000).toString();
        await this.redisService.saveResetToken(userId, token);
        return { token };
    }

    async getTokenUserId(token: string): Promise<string | null> {
        return await this.redisService.getResetToken(token);
    }
}

Product Service

Here's an example of the product service using the Redis service:

import { Inject, Injectable, InternalServerErrorException } from '@nestjs/common';
import { ProductInterface } from '../domain/interface/product.interface';
import { RedisService } from './redis.service';

// We will be using a dummy JSON product API to fetch our product data
// This could be any API call or database operation
const productURL = 'https://dummyjson.com/products/';

@Injectable()
export class ProductService {
    constructor(@Inject(RedisService) private readonly

 redisService: RedisService) {}

    async getProduct(productId: string): Promise<any> {
        // Check if the product exists in Redis
        const product = await this.redisService.getProduct(productId);
        if (product) {
            console.log('Cache hit! Product found in Redis');
            return { data: product };
        }

        const res = await fetch(`${productURL}${productId}`);

        if (res.ok) {
            const product: ProductInterface = await res.json();
            // Cache the data in Redis
            await this.redisService.saveProduct(`${product.id}`, product);
            console.log('Cache miss! Product not found in Redis');
            return product;
        } else {
            throw new InternalServerErrorException('Something went wrong');
        }
    }
}

These services can be injected into other services or controllers. To keep this article concise, we have not included the controller code, but you can find it in the code repository.

Conclusion

In this article, we explored how to set up a Redis client in a NestJS application for caching and storing short-lived data. We created a Redis client factory, a Redis repository, and a Redis service to abstract the Redis interactions in the application layer. We also demonstrated how to use the Redis service in password reset and product services to enhance data retrieval and caching. By implementing the techniques and structures outlined in this article, you can seamlessly incorporate Redis into your own application, optimizing its performance and enhancing data management.