Forem: Manish Kumar Sahu

Infinite scrolling using Intersection Observer API

Manish Kumar Sahu — Tue, 06 Jul 2021 06:17:19 +0000

Introduction

Infinite scrolling is a way to organize the contents of the pages in a website. This allows the user to browse all the contents in a single webpage by scrolling down. When the user reaches the bottom of the page, more contents load automatically.

In this article, we will discuss infinite scrolling and how to implement it using Intersection Observer API.

Usage

Let's say a website has 100+ posts to show, but it can't just load all the posts altogether otherwise it will consume that much data and will take much time to load all the posts which ruins the user experience.

So the website will just load the first 10 posts (which will consume very little data and time to load). Then, whenever the user almost reaches the bottom of the page, the next 10 posts will load. In this way, the posts will load and make the user experience better.

Mostly social media websites like Facebook and Twitter implemented infinite scrolling in their user feed pages.

Long and endless pages are good for time-killing activities because users arein the mindset for smooth exploration and discovery, which results in higher engagement of users in the website.

But, not every website should implement this, specially the goal oriented website like e-commerce sites, because when user wants to revisit some product page, it is difficult to find it, in this case pagination is really helpful.

Implementaion

We will implement this by using an array of 20 images. First only 5 images will be loaded, then as the user scrolls down and reaches the bottom of the page, more 5 photos will load, this will continue till all the images are loaded.

Setup

HTML

 <body>
    <h1>Oberserver Intersection API</h1>
    <div id="root"></div>
    <div id="loading">Loading...</div>
    <script src="script.js"></script>
  </body>

All the images will be appended in the div element having id="root.
Whenever user scrolls to the bottom of the page, div element having id="loading" will be shown.

Javascript
We will be using fetchImage() which when called fetches 5 images from the array of 20 images. Here, fetchImages() can be any server API call.

function fetchImage() {
  if (imgCount !== 20) {
    setTimeout(() => {
      for (let i = imgCount; i < imgCount + 5; i++) {
        const image = document.createElement("img");
        image.src = data[i].src;
        rootElement.append(image);
      }
      imgCount += 5;
    }, 500)
  }
}

Now we can setup the Intersection Observer API

document.addEventListener("DOMContentLoaded", () => {
  let options = {
    root: null,
    rootMargin: "0px",
    threshold: 0.25
  };

  function handleIntersect(entries, observer) {
    entries.forEach((entry) => {
      if (entry.isIntersecting) {
        if (imgCount !== 20) {
          fetchImage()
        }
      }
    });
  }

let observer = new IntersectionObserver(handleIntersect, 
 options);
observer.observe(loader);
})

Lets analyse what is happening in the above code.

We are calling InterectionObserver() which takes two parameters, first one is a callback which would decide what will happen when intersection happens, second parameter is a set of options.
In options, root will be the element with which intersection occurs, by defining it as null means it will take window by default. rootMargin is just the CSS property for the root element. And threshold can be an array of ratios which when passed can decide on what intersection ratio, the callback should be executed.
In handleIntersect(), we are checking if the loader is intersected, then we are calling fetchImage() to fetch 5 more images.
Atlast, we are calling the observer to observe loader. That means we are targeting the loader, whenever it appears in the viewport, we will load 5 more images.

You can view the full code setup here.

Conclusion

Infinite scrolling can be done in other ways, in the on scroll approach, every time a user scrolls, but we have to keep calculating scroll positions and can debounce for delays.
While using Intersection Observer API, the callback will be fired only when the target element comes to the viewport which prevents the javascript engine to run unnecessary calls.

You can explore more about Intersection Observer API here

Hope you enjoyed it ✌

Cover Image

Throttling and Debouncing in ReactJS

Manish Kumar Sahu — Thu, 01 Jul 2021 07:18:26 +0000

Introduction

This article talks about how these two techniques can be applied to optimize website performance and covers how to make a optimized searchbar.

Suppose there is a program in which the programmer gives users to run any function, users can run them by firing events such as click, change, resize, scroll etc. So it's upto the user that how many times they want to run that function. The user can spam any number of times which is not good for the website, because those functions maybe going through heavy computations which may slow the website.

Let's consider an example in which whenever user types in searchbar, function is called which returns the total number of letter changes.

Now imagine, if there is a program in which for each change there is an function call which have heavy computations, this will slow down the website.

To prevent this kind of scenario, programmers use Throttling and Debouncing to limit the number of function calls so that even if the user spams, the performance will not be affected.

Implementation

These two techniques are not something which is provided by javascript itself.
These are the two techniques which uses web APIs i.e. setTimeout() and clearTimeout().

setTimeout() function takes a function and a time delay as parameters, call that function after the given delay and returns us a timerId which we can store.
clearTimeout() function takes that timerId as a parameter and clears the timeout set by the setTimeout() function.

Throttling

Throttling is a technique in which no matter how many times user fires the event, the function will run once in a given interval of time.

Suppose there is an API which fetches some data from the server.
Whenever the user searches something in the searchbar, then the function searches the letters typed by the user and returns the data accordingly. By using throttling here, the function will be called once in every 1 second.

Let's examine what is happening in the code.

Whenever user types something, handleChange() function is called which contains a search() function.
The search() function returns the data according to user typed letters, but it is not yet called.
The search() function is passed to throttlingFunc() as a parameter with delay of 1 second.
Inside throttlingFunc(), it checks if there is some value in timerId which is a global variable, if true it don't call setTimeout().
If there is some value in timerId, it calls the setTimeout() which calls the search() and sets previous value of timerId as undefined.
In this way the search() function will be called only after 1 seconds when user types something.

Debouncing

Debouncing is a technique in which no matter how many times user fires an event, the function will run only after a certain time after user stops firing events.

We will use the same API to fetch some data from the server and the same function which searches the data according to the letters typed by the user.But here, we will use debouncing and the function will be called after 1 second after user stops typing.

Let's examine what is happening in the code.

Whenever user types something handleChange() function is called which contains a search() function.
The search() function returns the data according to user typed letters, but it is not yet called.
The search() function is passed to debouncingFunc() as a parameter with delay of 1 second.
Inside debouncingFunc(), clearTimeout() clears the previous timeout by taking previous timerId value.
Then calls setTimeout() which calls search() and gives new timerId.
In this way the search() function will be called only after 1 seconds when user stops typing something.

Conclusion

In this article we understood how debouncing and throttling technique works while implementing it on a search function.

Hope you find it helpful 😄 .

References
https://www.telerik.com/blogs/debouncing-and-throttling-in-javascript

Cover image

Authentication and Authorization using AccessTokens - RefreshTokens | Part 2

Manish Kumar Sahu — Mon, 31 May 2021 07:03:17 +0000

Authorization

Session based authorization

Earlier developers were using session based authorization.
After the user logs in, a session Id is sent to client in form of a cookie and when there is a user request, the server checks for that cookie and gives access and response.

Drawbacks

The problem is sessions are stored in the server, and when the app scalability increases, user requests also increase which eventually put loads on the server.
It doesn't support multiple servers. For example, suppose user logs in to the 'Amazon' website then the session id is passed to the client, but if the same user wants to open 'Amazon Prime', the user has to log in once again to create a session id.

JWT based authorization

In this process, when the user logs in, the user information is signed with a secret (which is stored in the server) and a JSON web token is generated and it is sent to the client. Generally, it is stored in browser localStorage, whenever a user sends a request to the server in which the authorization token is present in the headers of the request , the server just verifies that JWT token with the secret it has by unsigning that JWT token and gives users access.

Why JWT?

All the user state is stored on the client-side, so there is no load on the server.
It supports multiple servers by just passing the secret to multiple servers for verification.

Implementation using AccessToken and RefreshToken

RefreshToken is used to generate new AccessToken when it expires. AccessToken contains the user state or information and RefreshToken.
When user logs in, server creates AccessToken and RefreshToken stores the userId or something which when unsigns by someone, the person would not understand the context, and it returns response to the client. Usually AccessToken expires after a short time and RefreshToken after a long time.

router.post('/signin', async (req, res) => {
  try {
    const user = await req.body;
    const { email, password } = user;
    const userEmail = email;

    await User.findOne({ email: userEmail })
      .exec((err, user) => {
        if (err || user === null) {
          return res.status(400).json({
            message: "user does not exists!",
          });
        } else if (!user.authenticate(password)) {
          return res.status(401).json({
            message: "please enter the correct password!",
          });
        }
        const accessToken = jwt.sign(
          { user },
          process.env.ACCESS_TOKEN_SECRET,
          {
            expiresIn: "15m",
          }
        );
        const refreshToken = jwt.sign(
          { userId: user._id },
          process.env.REFRESH_TOKEN_SECRET,
          {
            expiresIn: "7d",
          }
        );
        res.json({ user, accessToken, refreshToken });
      });
  } catch (error) {
    res.status(400).json({
      message: error.message,
    });
  }
})

Then we can make a middleware for the verification of the token and put it in any route for authorization.

exports.authorizeToken = async (req, res, next) => {
  if (
    !req.headers["authorization"] &&
    typeof req.headers["authorization"] !== "string"
  ) {
    return res.status(401).json({
      message: "No tokens found",
    });
  }

  try {
    const accessToken = req.headers["authorization"].split(" ")[1];
    const { user } = jwt.verify(accessToken, process.env.ACCESS_TOKEN_SECRET);
    req.user = user;
    return next();
  } catch (error) {
    res.status(401).json({
      message: "token cannot be verified! please check it again.",
    });
  }
};

Before accessToken gets expire, we generate another accessToken in server from the refreshToken we got from the client's request. Then we send both the tokens to the client.

router.post('/token/access', (req, res) => {
  if (
    !req.headers["refresh-token"] &&
    typeof req.headers["refresh-token"] !== "string"
  ) {
    return res.status(401).json({
      message: "No refresh tokens found",
    });
  }

  try {
    const oldRefreshToken = req.headers["refresh-token"].split(" ")[1];
    const { userId } = jwt.verify(
      oldRefreshToken,
      process.env.REFRESH_TOKEN_SECRET
    );
    const refreshToken = jwt.sign(
      { userId: userId },
      process.env.REFRESH_TOKEN_SECRET,
      {
        expiresIn: "7d",
      }
    );
    const accessToken = jwt.sign(
      { user },
      process.env.ACCESS_TOKEN_SECRET,
      {
        expiresIn: "15m",
      }
    );
    res.json({ accessToken, refreshToken });
  } catch (error) {
    res.status(401).json({
      message: "refresh token cannot be verified! please check it again.",
    });
  }
})

The refreshToken is generally stored in the localStorage of the browser and the accessToken is stored in the headers of the API requests.

Voila! your app is now secured with the authentication and authorization.
Hope you find this useful.

references
cover image : https://jwt.io/
youtube: https://www.youtube.com/watch?v=7Q17ubqLfaM&t=332s

Authentication and Authorization using AccessTokens - RefreshTokens | Part 1

Manish Kumar Sahu — Mon, 31 May 2021 06:24:44 +0000

Authentication VS Authorization

Many people mismatch these terms, but they are two different things.

Authentication

This is a process of validating the user that sends the request to the server is the same as the one that logged in. This is generally done by checking the password, OTP, biometrics, etc.

Authorization

This is a process of giving permission and access to a specific resource or function. This is generally done by using tokens sent from auth server to the client.

I will be using MERN (MongoDB, Express, React, NodeJS) stack to show the implementation of authentication and authorization

Authentication

So, the first step of authentication is to encrypt the the user credentials or resources which are very much private and important, like passwords.

Let's say we have a userSchema using mongoose.
We cannot just store plain passwords in the database, so for hashing, we can use crypto. It is an inbuilt module that comes with nodeJS.

First of all, we made two methods
1. To authenticate or check if the password is correct or not.
2. To hash the password.
Then we made a virtual schema to take the plain password and encrypt it using the securePassword method we made and store to store it in database.
After that, we can import the authenticate method to /signin route to check for the password that user passed.

const mongoose = require("mongoose");
const { createHmac } = require("crypto");

const userSchema = mongoose.Schema(
  {
    name: {
      type: String,
      required: true,
    },
    email: {
      type: String,
      required: true,
      unique: true,
    },
    encrypted_password: {
      type: String,
      required: true,
    },
  },
  { timeStamps: true }
);

userSchema
  .virtual("password")
  .set(function (password) {
    this._password = password;
    this.encrypted_password = this.securePassword(password);
  })
  .get(function () {
    return _password;
  });

userSchema.methods = {
  securePassword: function (plainPassword) {
    if (!plainPassword) {
      return;
    }
    try {
      const secret = "anySecretYouWant";
      return createHmac("sha256", secret).update(plainPassword).digest("hex");
    } catch (error) {
      return;
    }
  },
  authenticate: function (plainPassword) {
    return this.encrypted_password === this.securePassword(plainPassword);
  },
};

module.exports = mongoose.model("User", userSchema);

So basically all the passwords of the user will be stored in a hashed or encrypted format so that none of us can read it and use it directly. It is done by passing a salt (any random long string ) to mix the characters in the password.
To decrypt those passwords, we need only that particular salt string.
As salt strings can decrypt the passwords, hence it is stored in Node environment files (.env files).

Now the user is authenticated, next move on to authorization part.

cover image : https://www.progress.com/

Monorepo and its setup for MERN stack

Manish Kumar Sahu — Sat, 15 May 2021 11:47:34 +0000

What is monorepo?

Monorepo is a single repository that will store all the code base and assets. For example, If you're making a full-stack app, you can store both frontend and backend parts in a single repository.

Why monorepo?

Dependency management

You can manage all of your code dependencies from a single source, you don't have to change the directory to add or remove any dependency

Server starting

You can start all the servers from a single command. For example, if you want to start the servers of both ReactJS and Express, you can do it with a single command.

Symlinks

Any package can be a dependency to another without going through npm or any package manager.

Collaboration

A single repository is very much helpful in sharing and collaborating because developers can access all codes and assets from a single source.
That's why event tech giants like Google is also using monorepos.

Speed

Monorepo will increase the speed of the development as you do not have to write different commands in different directories. Also, one can change across multiple files together.

Setup

I am using Yarn workspace here to explain monorepo setup for a MERN stack project.

Structure :
root-folder >
- api-folder
- app-folder

1.In the root folder, make a folder name packages and inside it, make your client and api folders.

2.In the package.json file of both client and api, change the name to the @[rootfolder_name]/client and @[rootfolder_name]/api respectively. For reference, this is my package.json of both folders, here my root folder name is videolibrary :

client>package.json

{
  "name": "@videolibrary/client",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^5.11.4",
    "@testing-library/react": "^11.1.0",
    "@testing-library/user-event": "^12.1.10",
    "react": "^17.0.2",
    "react-dom": "^17.0.2",
    "react-router-dom": "^5.2.0",
    "react-scripts": "4.0.3",
    "web-vitals": "^1.0.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

api>package.json

{
  "name": "@videolibrary/api",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "body-parser": "^1.19.0",
    "cors": "^2.8.5",
    "dotenv": "^9.0.1",
    "express": "^4.17.1",
    "mongoose": "^5.12.5",
    "nodemon": "^2.0.7"
  },
  "scripts": {
    "serve": "nodemon index.js",
    "start": "node index.js"
  }
}

3.In the root folder, install a package called concurrently.
Concurrently is a package that can run multiple npm scripts simultaneously. Here, we are using it to run React and Node servers simultaneously.

4.In the package.json of the root folder,

Add "workspaces" : ["packages/*"]
Add

"scripts": 
{
    "app": "yarn workspace @[rootfolder_name]/client start",
    "server": "yarn workspace @[rootfolder_name]/api serve",
    "build-app": "yarn workspace @[rootfolder_name]/client build",
    "build-server": "yarn workspace @[rootfolder_name]/api build",
    "start": "concurrently --kill-others-on-fail \"yarn client\"  \"yarn server\"",
    "start:server" : "yarn workspace @[rootfolder_name]/api start"
}

Add "engines": { "node": "14.15.0" }

For reference, you can see my root package.json file :

{
  "name": "videolibrary",
  "version": "1.0.0",
  "license": "MIT",
  "private": true,
  "workspaces": [
    "packages/*"
  ],
  "scripts": {
    "app": "yarn workspace @videolibrary/app start",
    "server": "yarn workspace @videolibrary/api serve",
    "build-app": "yarn workspace @videolibrary/app build",
    "build-server": "yarn workspace @videolibrary/api build",
    "start": "concurrently --kill-others-on-fail \"yarn app\"  \"yarn server\"",
    "start:server" : "yarn workspace @videolibrary/api start"
  },
  "engines": {
    "node": "14.15.0"
  },
  "dependencies": {
    "concurrently": "^6.0.2"
  }
}

That's it, you are good to go now! Your monorepo is ready to use.