Forem: Leonard Ginters

Smaller, Free Database Providers I've Found

Leonard Ginters — Mon, 19 Aug 2024 19:23:07 +0000

A list of smaller and free-to-use database providers.

AstroDB
- Currently in closed beta
- Up to 1B reads and 1M writes per month
TursoDB
- SQLite database
- Unlimited replicas
- Up to 9GB storage, 1B read rows, 25M written rows

The JavaScript event loop execution order, summarized in a simple manner

Leonard Ginters — Thu, 18 Nov 2021 19:50:08 +0000

The simplified order

Everything in sync
Microtask queue
Task queue

Real-world examples

Tasks

setTimeout, setInterval, setImmediate, ...

Microtasks

Promises, queueMicrotask, MutationObserver, ...

Everything combined

// Schedules a new microtask
queueMicrotask(() => console.log("microtask"));

// Schedules a new task
setTimeout(() => console.log("task"), 0);

console.log("main");
// Output order: main, microtask, task

Please note that this describes a really, really simplified perfect-world case. If you would like to find out more, I can highly recommend Jake Archibald's blog post on this topic.

Send multiple fetch requests simultaneously

Leonard Ginters — Sun, 05 Jul 2020 13:24:29 +0000

First of all, sending multiple fetch requests simultaneously is not always the right solution and has some downsides. One downside is for example that you could end up being blocked if you send too many requests at the same time, to one server. I personally write a solution for sending requests simultaneously if I need to fetch a big list of URLs with unknown response times (> 100 URLs).

Okay, now let's get started. First of all we need to create a async fetchSimultaneously function and create the constants queue and maxSimultaneouslyRequests inside it. The queue constant exists just so we can keep control of which URLs have been fetched and which not. We also need to initialize a start index (i) and a variable for storing the value of the current open requests (currentRequests).

async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;
}

To now send the requests simultaneously and handle when we want to stop the loop, setInterval is the best option.
To return the result array more easily, we are wrapping everything in a new Promise and call the resolve function when all requests have been finished.

async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;

    return await new Promise(resolve => {
        const result = [];

        const fetcher = setInterval(async () => {
        }, 100);
    });
}

The first thing we need to check inside the setInterval function is if all results have been fetched and if so, we resolve the promise and clear the interval. The easiest way to find out if a queue array entry has already been fetched is to set the value of the entry to undefined. This allows us to filter for array entries with a value different than undefined to find out if any array entries haven't been fetched yet.

if (queue.filter(url => url).length === 0) {
    clearInterval(fetcher);
    resolve(result);
}

The next thing we need to check is if the ongoing requests are exceeding the maxSimultaneouslyRequests.
Since we need to wait for all responses to arrive, we also need to check if the value of i is exceeding the length of our queue. (At least the last request will take longer to finish than the value of i to reach the length of the queue array.)

if (currentRequests >= maxSimultaneouslyRequests || i > queue.length - 1) {
    return;
}

Now we can finally fetch the data and set it to the correct index of the result array.

async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;

    return await new Promise(resolve => {
        const result = [];

        const fetcher = setInterval(async () => {
            if (queue.filter(url => url).length === 0) {
                clearInterval(fetcher);
                resolve(result);
            }

            if (currentRequests >= maxSimultaneouslyRequests || i > queue.length - 1) {
                return;
            }

            // Get current index and increase i
            const index = i++;
            const url = queue[index];

            currentRequests++;
            // Keep same index as of the passed urls array
            result[index] = await (await fetch(url)).text();
            currentRequests--;

            // Set value of index to empty (undefined)
            delete queue[index];
        }, 100);
    });
}

Get files recursive with the Node.js File System (FS)

Leonard Ginters — Sat, 21 Mar 2020 11:25:47 +0000

The File System Promises API

In the following code snippets, I'll be using the fs Promises API.
It's available to you if you're rocking at least Node.js v10.

const { promises: fs } = require("fs");

Separating the directory entries

To be able to separate the entries, we have to explicitly ask for all information about the file type.

const entries = await fs.readdir(path, { withFileTypes: true });

If we now want to separate the entries, we can just do this by calling the isDirectory method.

const folders = entries.filter(entry => entry.isDirectory());
const files = entries.filter(entry => !entry.isDirectory());

Getting the files recursive

If we now combine the previously mentioned ways to separate the files, put everything in a function and call this function recursively for every subdirectory, we are able to get all files within the current directory and all subdirectories.

async function getFiles(path = "./") {
    const entries = await fs.readdir(path, { withFileTypes: true });

    // Get files within the current directory and add a path key to the file objects
    const files = entries
        .filter(entry => !entry.isDirectory())
        .map(file => ({ ...file, path: path + file.name }));

    // Get folders within the current directory
    const folders = entries.filter(entry => entry.isDirectory());

    for (const folder of folders)
        /*
          Add the found files within the subdirectory to the files array by calling the
          current function itself
        */
        files.push(...await getFiles(`${path}${folder.name}/`));

    return files;
}