Forem: Nelson Odo

Using MinIO Server for Local Development: A Smarter Alternative to S3

Nelson Odo — Tue, 31 Dec 2024 21:12:55 +0000

File uploads are a staple of many applications, enabling users to share images, videos, documents, and more. While S3 is a popular choice for handling file storage, it’s not always ideal for local development and testing. The need to make HTTP calls via the AWS SDK introduces latency, and relying on S3 during development can be slow and cumbersome. Mocking or spying during end-to-end testing might work, but it often feels like a compromise.

That’s where MinIO Server comes in—a high-performance, S3-compatible object storage solution that’s perfect for local development. MinIO offers the same S3 APIs, making it a seamless alternative to integrate into your existing workflows. In this article, we’ll demonstrate how to set up and use MinIO with a Node.js application, while also highlighting its unique benefits.

Why MinIO?

Lightning-fast local operations instead of internet-dependent API calls
Identical S3 API compatibility – no code changes needed
A sleek web interface for manual file management
Cost savings on S3 requests during development
Perfect isolation for testing environments
Zero internet dependency for file operations

Setting Up MinIO with Docker

Getting started is straightforward. While you can download MinIO directly, Docker makes it even easier. Here’s how to configure MinIO using Docker Compose.

Install Docker: If you don’t already have Docker installed, download it here.
Create a docker-compose.yml File: Add the following configuration:

version: "3"
services:
  file-storage:
    restart: always
    image: minio/minio:RELEASE.2022-06-11T19-55-32Z
    container_name: s3-file-storage
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: Password1234
    command: server --console-address ":9001" /file-storage-volume
    volumes:
      - file-storage-volume:/file-storage-volume
    ports:
      - 9000:9000 # Files served via this port
      - 9001:9001 # Admin console accessible via this port

volumes:
  file-storage-volume:

Run Docker Compose:

docker-compose up -d

Once running, you can access:

API endpoint at http://localhost:9000
Web console at http://localhost:9001 (login with admin/Password1234)

Integrating MinIO with Your Node.js Application

Here's where the magic happens. MinIO seamlessly replaces S3 in your development environment while keeping production untouched. Here's how to set it up:

Install Dependencies

Make sure you have the AWS SDK installed:

npm install @aws-sdk/client-s3

Configure MinIO and S3

Use the following configuration to create a storage instance:

import { once } from "lodash";
import { S3 } from "@aws-sdk/client-s3";

const config = {
  awsRegion: "eu-west-2",
  minioBaseUrl: "http://localhost:9000",
  minioUsername: "admin",
  minioPassword: "Password1234",
};

const isDevelopmentOrTest = process.env.NODE_ENV !== "production";

const createOrGetStorageInstance = once(() => {
  if (isDevelopmentOrTest) {
    return new S3({
      endpoint: config.minioBaseUrl,
      forcePathStyle: true,
      region: config.awsRegion,
      credentials: {
        accessKeyId: config.minioUsername,
        secretAccessKey: config.minioPassword,
      },
    });
  }

  return new S3({ region: config.awsRegion });
});

export const s3 = createOrGetStorageInstance();

This setup uses MinIO during development and seamlessly falls back to S3 in production.

Putting It All Together: File Upload Example

Here's a practical example using Express and express-fileupload:

Install Dependencies

npm install express express-fileupload

Implement File Upload

Here’s a simple Express route for uploading files:

import express from "express";
import fileUpload from "express-fileupload";

const app = express();
app.use(fileUpload());

app.post("/upload", async (req, res) => {
  if (!req.files||!req.files.files) {
      return res.status(400).send("No files were uploaded.");
  }

  const uploadedFiles = Array.isArray(req.files.files)
    ? req.files.files
    : [req.files.files];

  const uploadedUrls = await Promise.all(
    uploadedFiles.map(async (file) => {
      const key = `uploads/${file.name}`;
      const uploadParams = {
        Bucket: "my-bucket",
        Key: key,
        Body: file.data,
        ContentType: file.mimetype,
      };

      const uploadResult = await s3.upload(uploadParams).promise();
      return uploadResult.Location;
    })
  );

  res.json({ message: "Files uploaded successfully", urls: uploadedUrls });
});

app.listen(3000, () => console.log("Server running on port 3000"));

### Postman Example:
Below is an example of how to test this route using Postman:

Pro Tips for MinIO Usage

Web Console Features: Take advantage of MinIO's web interface to:
- Browse uploaded files
- Create and manage buckets
- Set up access policies
- Monitor usage statistics
Development Workflow:
- Create separate buckets for different file types (images, documents, etc.)
- Use the web console to verify uploads during development
- Use environment variables to switch between MinIO and S3

Conclusion

MinIO Server is a game-changer for local development. It provides the perfect balance of functionality and simplicity, making your development process faster and more reliable. With identical S3 API compatibility and a user-friendly web interface, it's the ideal solution for local development and testing environments.

Remember: your production environment still uses real S3, but your development environment gets all the benefits of local, lightning-fast file operations. It's the best of both worlds!

Try it out in your next project, and you'll wonder how you ever developed without it.

Source Code 📂

GitHub Repository: https://github.com/Nedum84/minio-s3-server

Let's Connect 🌐

Twitter: @thenelson_o
LinkedIn: Nelson Odo

What Do You Think? 💭

Share your thoughts! Did this post help you?
Have ideas or feedback to share? Let’s continue the conversation in the comments below! 😊

Advanced CSRF Protection with RSA

Nelson Odo — Mon, 23 Dec 2024 00:33:18 +0000

A Modern Take on Enhancing API Security Beyond Authentication 🔒🚀

In the evolving world of web security, authentication and authorization are the backbone of most secure systems. However, relying solely on these mechanisms without addressing CSRF (Cross-Site Request Forgery) or brute-force risks can leave your APIs exposed. A malicious actor can inspect network calls, mimic headers, and execute requests outside your browser—using tools like Postman, CURL, or custom scripts.

Sounds alarming, right? Even top platforms like X (formerly Twitter), Facebook, TikTok, and Instagram are not immune to this issue. So, how do we fight back? Introducing Advanced CSRF Protection using RSA encryption—a lightweight yet effective way to prevent unauthorized API usage, brute-force attempts, and stale requests.

This technique adds an extra layer of security on top of your existing authentication system. By encrypting request metadata like the method, URL, and timestamp using RSA, we ensure each API request is uniquely validated and cannot be replayed or reused.

How Does It Work? 🛠️

We leverage RSA asymmetric encryption to validate the integrity of API requests. Here's the flow:

Generate an RSA key pair (public and private keys).You can use tools like cryptotools.net/rsagen.
The public key is stored on the client (ideally in environment variables).
Before making an API request, we encrypt a payload containing:
- The request method
- The request URL
- A current timestamp
This encrypted payload is sent as a custom csrf header.
On the backend:
- The payload is decrypted using the private RSA key.
- Validation ensures:
  - The method and URL match.
  - The timestamp isn’t stale (e.g., older than 15 seconds).
If validation passes, the request proceeds; otherwise, it’s rejected.

This approach ensures that every API request is tightly bound to specific parameters, preventing replay attacks, stale requests, and brute-force abuse.

Client-Side Implementation (React + Axios) ⚛️

To streamline CSRF handling & ensure that it'll be available for all our request, we’ll use Axios interceptors. Here’s the setup in TypeScript:

import axios, { AxiosError } from "axios";
import appConfig from "config"; // Import your app config
import { rsaCrypt } from "Utils/rsaCrypt"; // RSA utility

const axiosClient = axios.create({
  baseURL: appConfig.BASE_URL,
  headers: {
    "Content-Type": "application/json",
  },
});

// Helper function to create the CSRF payload
const getCsrfPayload = (method: string, url: string) => {
  return {
    url,
    method,
    timestamp: Date.now(),
  };
};

// Axios request interceptor
axiosClient.interceptors.request.use(
  async (config) => {
    if (config && config.headers) {
      // Generate CSRF payload
      const csrfPayload = getCsrfPayload(config.method!, config.url!);
      // Encrypt the payload
      const csrfToken = await rsaCrypt.encrypt(JSON.stringify(csrfPayload));
      if (csrfToken) {
        config.headers["csrf"] = csrfToken; // Attach the encrypted token to headers
      }
    }
    return config;
  },
  (error) => {
    return Promise.reject(error);
  }
);

// Axios response interceptor
axiosClient.interceptors.response.use(
  function (response) {
    return response;
  },
  async function (error: AxiosError<{ code: string; message: string }>) {
    return Promise.reject(error);
  }
);

export default axiosClient;

Backend Implementation (Node.js + Express) 🛡️

On the server, we’ll validate the incoming CSRF token using the RSA private key and ensure the request integrity.

Middleware for CSRF Validation

import { NextFunction, Request, Response } from "express";
import { BadRequestError } from "../utils/custom.error";
import { rsaEncrypt } from "../utils/rsa.crypt";

type EncPayload = {
  url: string;
  method: string;
  timestamp: number;
};

export async function csrfValidation(req: Request, res: Response, next: NextFunction) {
  const urlPath = decodeURIComponent(req.originalUrl.split("?")[0]?.trim() || "");
  const method = req.method.toLowerCase();

  const csrf = req.headers["csrf"];

  if (!csrf || Array.isArray(csrf)) {
    throw new BadRequestError("CSRF token is missing or malformed.", "CSRF_TOKEN_MISSING");
  }

  const now = Date.now();
  const payload = decryptPayload(csrf);

  if (!payload) {
    throw new BadRequestError("Failed to decrypt or parse CSRF token.", "CSRF_TOKEN_INVALID");
  }

  const payloadUrlPath = decodeURIComponent(payload.url.split("?")[0]?.trim() || "");
  if (urlPath !== payloadUrlPath || payload.method.toLowerCase() !== method) {
    throw new BadRequestError(
      "CSRF token validation failed: URL or method mismatch.",
      "CSRF_VALIDATION_FAILED"
    );
  }

  const queryExpiry = 15 * 1000; // 15 seconds
  if (now - payload.timestamp > queryExpiry) {
    throw new BadRequestError("CSRF token has expired.", "CSRF_TOKEN_EXPIRED");
  }

  next();
}

function decryptPayload<T = EncPayload>(payload: string): T | undefined {
  try {
    return JSON.parse(rsaEncrypt.decrypt(payload)) as T;
  } catch (error) {
    console.error("Error decrypting payload:", error);
    return undefined;
  }
}

Key Features 🎯

Asymmetric Encryption: Only the server knows the private key, ensuring tokens cannot be forged.
Request Freshness: Timestamps limit the token's validity to 15 seconds, preventing replay attacks.
Method & URL Validation: Ensures tokens are tied to specific actions, blocking brute-force scripts.

I implemented this approach in one of my projects, Winzy Social, to enhance the security of API interactions and prevent unauthorized or stale requests. Feel free to check it out!

Demo & Source Code 📂

GitHub Repository: https://github.com/Nedum84/csrf-with-rsa
Live Demo: https://nedum84.github.io/csrf-with-rsa

Conclusion 📝

This approach enhances your API security in addition to existing authentication and authorization mechanisms. By introducing encrypted, timestamped CSRF tokens, you can prevent malicious actors from abusing your APIs outside the browser.

Give this a shot and level up your application security! 🚀

Let's Connect 🌐

Twitter: @thenelson_o
LinkedIn: Nelson Odo

What Do You Think? 💭

Share your thoughts! You can improve the code to serve your use case. Did this post help you?
Have ideas or feedback to share? Let’s continue the conversation in the comments below! 😊

Simplified State Management with useRQState: A Better Alternative to useState

Nelson Odo — Mon, 23 Dec 2024 00:26:29 +0000

Imagine this: you're building a React app, and suddenly, state management spirals out of control. You find yourself passing props down through a labyrinth of components, a process affectionately known as "prop drilling." It's like playing a game of telephone, but instead of whispers, you're passing data through multiple layers of components, and by the time it reaches the intended recipient, we all know how that game ends – with a big mess!

But fear not, dear React developers! There's a new kid on the block, and its name is useRQState (React Query State). This nifty hook is built on top of React Query and is designed to make local state management a breeze.

This powerful hook elegantly sidesteps the prop-drilling menace by allowing you to access and update state across your entire app using a single, magical query key.

How useRQState Works:

Define Your State: Use useRQState to initialize your state, just like you would with useState.
Assign a Query Key: Give your state a unique identifier, the "query key."
Access and Update: From any component, access and update the state using the same query key. It's like having a secret handshake that only components with the same key understand.

What's the Big Deal About useRQState?

Glad you asked! Here are just a few reasons why useRQState is about to become your new best friend:

Prop Drilling? Pshaw! Access and update state anywhere in your app without the dreaded prop-drilling.
State Persistence Across Pages: Your state remains steadfast even as you navigate through your app, like a loyal companion.
Simplicity Itself: useRQState mimics the familiar useState API, making it easy to adopt without a steep learning curve.

useRQState Implementation

import { useQuery, UseQueryResult } from  "@tanstack/react-query";
import { queryClient } from  "Utils/reactquery";

export function useRQState<T>(queryKey: string, initialData?: T | ((...args: any[]) => T)): 
[ T, (update: Partial<T> | ((prevState: T) => Partial<T>)) => void, Omit<UseQueryResult<T>, "data"> ] 
{
  const { data, ...others } = useQuery({
    queryKey: [queryKey],
    initialData: () => {
      if (initialData === undefined) {
        const storedData = getQueryCache<T>(queryKey, true);
        return storedData;
      }

      if (typeof initialData === 'function') {
        const val = (initialData as CallableFunction)();
        return val;
      }

      return initialData;
    },
    refetchInterval: false,
    refetchOnReconnect: false,
    refetchIntervalInBackground: false,
  });

  function setData(update: Partial<T> | ((prevState: T) => Partial<T>)) {
    queryClient.setQueryData([queryKey], (prevState: T | undefined) => {
      if (typeof update === 'function') {
        return (update as (prevState: T) => Partial<T>)(prevState as T);
      }
      return update;
    });
  }


  return [data, setData, { ...others }];
}

getQueryCache

const getQueryCache = <T>(queryKey: string, exact = false) => {
  const [itemCache] = queryClient.getQueriesData<T>({
    exact,
    queryKey: [queryKey],
  });

  const [_queryKeys, data] = itemCache ?? [];

  return data;
};

Utils/reactquery.ts

import { QueryClient } from "@tanstack/react-query";

export const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      staleTime: Infinity,
      refetchOnMount: false,
      refetchOnWindowFocus: false,
    },
  },
});

Let's See It in Action!

import { useRQState } from "Hooks/useRQState"; 

function MyComponent() {
  const [counter, setCounter] = useRQState("counter", 0); 

  return (
    <button onClick={() => setCounter(count + 1)}>
      Increment (with a touch of magic!)
    </button>
  );
}

In this example, we're using the query key "counter" to manage our count state. Any component that uses this key can access and update the count, regardless of its position in the component tree.

Updating State with Previous Values

Like useState, useRQState supports updating state based on the previous value, making it perfect for scenarios where you need to increment counters or manage complex state updates:

<button
  onClick={() => setCounter((prev) => prev + 1)}
  className="px-4 py-2 rounded-md bg-green-600 text-white flex-1"
>
  Increment by 1
</button>

Ready to Ditch Prop Drilling?

With useRQState, managing state in your React app becomes a breeze. Say goodbye to spaghetti code and hello to a clean, maintainable, and scalable state management solution.

Want to see it in action? Check out our live demo or head over to the GitHub repo for the source code.

Demo & Source Code 📂

GitHub Repository: https://github.com/Nedum84/use-rq-state
Live Demo: https://nedum84.github.io/use-rq-state

Let's Connect 🌐

Twitter: @thenelson_o
LinkedIn: Nelson Odo

What Do You Think? 💭

Share your thoughts! You can improve the code to serve your use case. Did this post help you?
Have ideas or feedback to share? Let’s continue the conversation in the comments below! 😊

useAxios - React custom http client

Nelson Odo — Thu, 10 Feb 2022 08:20:52 +0000

Often times we are faced with difficulty in tracking the request status while making api calls on our react app. Request status here could be the data response, loading status, error encountered, etc.
The work around is usually to create individual useState to manage all the status that we want (like error, data, etc).

This hook is built on top of javascript axios package

First, let’s create the useAxios custom hooks

//useAxios.js

import { useState, useEffect } from "react";
import axios from "axios";

const useAxios = props => {
  const [axiosData, setAxiosData] = useState({
    url: null,
    method: "GET",
    body: null,
    headers: null
  });
  const [data, setData] = useState(null);
  const [error, setError] = useState(null);
  const [loading, setLoading] = useState(false);
  const [status, setStatus] = useState(null);

  const sourceToken = axios.CancelToken.source();
  const fetchApi = async () => {
    setLoading(true);
    setData(null);
    setError(null);
    setStatus(null);
    try {
      const data = axiosData?.body ? { data: axiosData?.body } : {};
      const headers = axiosData?.headers ? { headers: axiosData?.headers } : {};
      const response = await axiosClient({
        url: axiosData.url,
        method: axiosData.method ?? "GET",
        cancelToken: sourceToken.token,
        ...data,
        ...headers,
      });
      setLoading(false);
      setStatus(response?.status);
      setData(response?.data);
      if (props.onSuccess) props.onSuccess(response?.data);
    } catch (error) {
      const errorMsg = error?.message;
      setLoading(false);
      setError(String(errorMsg));
      setStatus(error?.response?.status);
      if (props.onError) props.onError(errorMsg);
    }
  };

  useEffect(() => {
    if (axiosData.url != null) {
      fetchApi();
    }
    // abort the fetch if screen is dead/inactive
    return () => sourceToken.cancel();
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [axiosData]);

  // return [setPostData, data, error, loading, status];
  return { axios: setAxiosData, data, error, loading, status };
};

export default useAxios;

Initialize request with useAxios

To initialize your request

//index.js

import useAxios from "customHooks/useAxios";
const { axios, data, error, loading, status } = useAxios();

axios: custum axios function to make the api request data: reponse data from the endpoint error: contains error message if an error occures status: returns status code of the request

To trigger the api request, for instance when a button is clicked

const onClick = ()=>{
  axios({
      url: "https://example.com"
    });
}

This makes a get request to https://example.com
The axios function can accept multiple parameters like method, body, and headers

What about POST/PATCH/DELETE requests

To make a request with a custom methods, you could do

const onClick = ()=>{
    const data = {
      name:"Sample name",
      category:"sampleCat",
    }
    axios({
      url: "https://example.com",
      method:"POST",
      body:data,
      headers:{...}
    });
}

You can also also listen for success or error response while initializing the useAxios. Here is how you can do this,

 const { axios, data, error, loading, status } = useAxios({
    onSuccess: (data) => {
      console.log(data);
    },
    onError: (error) => {
      console.log(error);
    }
  });

onSuccess is fired when the data is returned from the endpoint
onError is fired when an error occurs.

You can as well enhance the useAxios hook to include base url, token, etc...

Thanks for reading!