Converting Excel File to JSON in JS /React.js /Next.js from an URL.

Shafiul Azim — Tue, 09 Jan 2024 07:47:18 +0000

In web development, dealing with data in various formats is a common task. One frequent scenario is the need to convert Excel files into a more flexible and accessible format, such as JSON. In this blog post, we’ll explore how to achieve this using JavaScript, with a focus on React.js and Next.js.You can use this code in Vanila.js project also.

Getting Started
To get started, we’ll use the excellent xlsx library, which provides functionality for reading and writing Excel files.

Installing package
Open your code editior & open project folder.Now open terminal on this folder & write this script to install xlsx package.

npm install xlsx
or
yarn add xlsx

The code we’ll be working with is encapsulated in a module named excelToJson.js.

// excelToJson.js

import * as XLSX from "xlsx";

This module contains two essential functions: downloadExcelAsArrayBuffer and convertExcelToJson. Let's break down each step of the process.

Downloading Excel File
The first step is to download the Excel file as an ArrayBuffer. This function uses the fetch API to retrieve the file from a given URL.

// excelToJson.js

import * as XLSX from "xlsx";
//The first step is to download the Excel file as an ArrayBuffer. This function uses the fetch API to retrieve the file from a given URL. 

const downloadExcelAsArrayBuffer = async (excelUrl) => {
  try {
    // Fetch the Excel file
    const response = await fetch(excelUrl);

    if (!response.ok) {
      throw new Error(`Failed to download Excel file. Status: ${response.status}`);
    }
    // Convert the response to ArrayBuffer
    const arrayBuffer = await response.arrayBuffer();
    return arrayBuffer;
  } catch (error) {
    console.error("Error downloading Excel file:", error);
    throw error;
  }
};

Converting Excel to JSON
The core functionality lies in the convertExcelToJson function. It performs the conversion from Excel to JSON using the xlsx library.

const convertExcelToJson = async (excelUrl) => {

  try {
    // Download the Excel file as ArrayBuffer
    const arrayBuffer = await downloadExcelAsArrayBuffer(excelUrl);

    // Convert the Excel buffer to a workbook
    const workbook = XLSX.read(arrayBuffer, { type: "array" });

    // Get the first sheet
    const sheetName = workbook.SheetNames[0];
    const sheet = workbook.Sheets[sheetName];

    // Convert the sheet data to JSON
    const jsonData = XLSX.utils.sheet_to_json(sheet, { header: 1 });

    // Map the array to an array of objects
    const resultArray = jsonData.map((row) => {
      const obj = {};
      for (let i = 0; i < jsonData[0].length; i++) {
        obj[jsonData[0][i]] = row[i];
      }
      return obj;
    });

    return resultArray.slice(1);
  } catch (error) {
    console.error("Error converting Excel to JSON:", error);
    throw error;
  }
};

Implementing in React.js/Next.js
To use this functionality in a React.js or Next.js project, simply import the convertExcelToJson function and use it as needed.

import convertExcelToJson from "./path/to/excelToJson";

const YourComponent = async () => {
  try {
    const jsonData = await convertExcelToJson("your_excel_file_url.xlsx");
    console.log(jsonData);
    // Now you can use jsonData in your React component
  } catch (error) {
    // Handle errors
  }
};

With this script, you can easily convert downloadable Excel files to JSON in your JavaScript projects. This process is especially handy when dealing with data imports and exports in web applications.

Happy coding!

External or remote image size compression / optimization in Next.js

Shafiul Azim — Tue, 09 Jan 2024 07:37:37 +0000

Sometimes we need to reduce the size of the image that are getting from external API.As it’s just an url not the file & we use custom loader we don’t get the Next js Image optimization feature.Sometimes we had to implement prefix in the url and also prefix before static assests (“example/_next/static/….) we don’t get default next js image optimization feature if we use custom loader in the Image component.

First create a imageLoader.js file in top level of your project directory and paste the following code,

const imageLoader = ({ src, width, quality }) => {

  if (!src?.includes("https")) {
    return `${src}?w=${width}&q=${quality || 75}`; //for static images
  } else {
    src = src.split("https://").join(""); //will remove "https://" from the url to avoid CORS error
  }

  let secretUrl = encodeURIComponent(src); //encode image source url.

  const apiEndpoint = "/api/imageloader" //Write your internal(next js) api here

  return `${
    window?.location?.origin
  }${apiEndpoint}?url=${secretUrl}&w=${width}&q=${quality || 75}`;
};

export default imageLoader;

Now create an api for image optimization under api folder “api/imageloader.js” and paste the following code,we use redis for caching and reduce server pressure.If you want you can ignore redis.

import axios from "axios";
import sharp from "sharp";
import redis from "../../../lib/redis";


export default async function imageLoader(req, res) {
  let url = decodeURIComponent(req?.query?.url); //decoding the url
  url = `https://${url}`; //adding https:// that we were removed in the previos file

  try {

    if (!url) {
      return res.status(400).send("400 Bad request. url is missing");
    }

    const cachedUrl = await redis.getBuffer(`${url}`); //if already cached ther return cached one & reduce server load

    if (!cachedUrl) {
      const width = req.query.w ?? "384"; //default width
      const quality = req.query.q ?? "86"; //default quality
      // get the image information
      const response = await axios.get(url, {
        responseType: "arraybuffer",
      });

      // use sharp lib to resize the image based on the parameters
      const optimized = await sharp(response.data)
        .resize({
          withoutEnlargement: true,
          width: parseInt(width),
        })
        .webp({ quality: parseInt(quality) }) //transform image to webp format
        .toBuffer();

      // set public cache to 10  min
      res.setHeader("Cache-Control", "public, max-age=600, must-revalidate");

      // set content type to webp.
      res.setHeader("content-type", "image/webp");

      await redis.set(`${url}`, optimized);

      // send buffered image
      return res.status(200).send(optimized);
    }

    res.setHeader("content-type", "image/webp");
    return res.status(200).send(cachedUrl);

  } catch (e) {
    return res.status(e?.response?.status || 401).json({ message: e.message });
  }
}

Now we are ready to use our optimization api and loader.Pass the loader to the Image component.

<Image
    loader={imageLoader}
    src={imageUrl}
    alt="external image"
   />

We are good to go.Now enjoy your custom image optimization for next js external image.

Please feel free to share if this article helps you.

Forem: Shafiul Azim

Converting Excel File to JSON in JS /React.js /Next.js from an URL.

External or remote image size compression / optimization in Next.js