Forem: Hamza Rahman

Understanding !! in JavaScript - Simple Guide

Hamza Rahman — Thu, 14 Nov 2024 20:51:03 +0000

Ever seen !! in JavaScript code and wondered what's going on? Let's break it down in simple terms.

What Does !! Do?

The double bang (!!) is a quick way to convert any value to a boolean (true or false). It's that simple!

Think of it like asking JavaScript: "Hey, does this value exist and is it meaningful?"

If yes, you get true
If no, you get false

See It In Action

Let's look at some quick examples:

// These become true
!!42          // true (any number except 0)
!!"Hello"     // true (any non-empty string)
!!{}          // true (any object)
!![]          // true (any array)

// These become false
!!0           // false
!!""          // false (empty string)
!!null        // false
!!undefined   // false

Real-World Examples

1. Checking if a User Provided Their Name

function validateName(name) {
    if (!name) {  // same as if (!!name === false)
        return "Please enter your name";
    }
    return `Thanks, ${name}!`;
}

validateName("")        // "Please enter your name"
validateName("Sarah")   // "Thanks, Sarah!"

2. Checking if an API Response Has Data

function handleResponse(data) {
    const hasData = !!data?.items?.length;

    if (hasData) {
        return "Found some results!";
    }
    return "No results found";
}

handleResponse({items: []})      // "No results found"
handleResponse({items: [1,2,3]}) // "Found some results!"

When Should You Use !!?

!! is particularly useful when:

You need a true boolean instead of a "truthy" or "falsy" value
You're checking if a value exists and is meaningful
You want to make your code's intention very clear

What Values Become False?

These are all the values that become false when using !!:

0
"" (empty string)
null
undefined
NaN
false

Everything else becomes true!

Pro Tips

1. The If Statement Shortcut

In if statements, JavaScript automatically converts values to booleans, so this:

if (!!username) {
    // do something
}

Is the same as:

if (username) {
    // do something
}

2. Using Boolean() Instead

Some developers prefer using Boolean() because it's more obvious what's happening:

Boolean("hello")   // true
Boolean("")        // false
!!("hello")        // true
!!("")            // false

Both work exactly the same way - use whichever makes more sense to you!

Quick Recap

!! converts values to true/false
It's great for checking if values exist
Use it when you specifically need a boolean result
In if statements, you usually don't need it

Remember: The best code is code that your team can easily understand. Whether you use !! or Boolean(), just be consistent!

Nextjs link onClick: Routing and Form Submission Examples

Hamza Rahman — Wed, 09 Oct 2024 12:44:14 +0000

In this tutorial, we'll explore how to leverage the power of Nextjs link onClick functionality for programmatic routing. We'll cover various scenarios including click-based navigation, form submissions, and optimizing page transitions. The examples are provided in both JavaScript and TypeScript to cater to different developer preferences.

Using Next.js Link Component
Simple Click-based Routing
Form Submission and Routing
TypeScript Implementations
Best Practices and Optimization

Using Next.js Link Component

Next.js provides a powerful Link component from the next/link module that allows you to create interactive links in your application. This component enables navigation between pages without triggering a full page reload, which is crucial for maintaining a smooth user experience in a single-page application.

The Link component can be combined with onClick events to create dynamic and interactive navigation experiences. Here's a basic example of how you might use the Link component with an onClick event:

import Link from 'next/link';
import { useRouter } from 'next/router';

export default function NavigationExample() {
  const router = useRouter();

  const handleClick = (e) => {
    e.preventDefault();
    // Perform some action here
    console.log('Link clicked!');
    // Then navigate programmatically
    router.push('/about');
  };

  return (
    <Link href="/about">
      <a onClick={handleClick}>About</a>
    </Link>
  );
}

In this example, we're using both the Link component and the useRouter hook. The Link component provides the client-side routing capabilities, while the useRouter hook allows us to programmatically navigate after performing some custom logic in our handleClick function.

This approach gives you the flexibility to execute custom code before navigation occurs, which can be useful for various scenarios such as form validation, data fetching, or state updates.

Now, let's dive deeper into more specific use cases and advanced techniques for handling routing with onClick events in Next.js.

Nextjs onClick redirect

The following example demonstrates how to use the useRouter hook to handle click events for routing:

import { useRouter } from 'next/router'

function ClickExample({ link }) {
  const router = useRouter()

  const handleClick = event => {
    event.preventDefault()
    router.push(link)
  }

  return (
    <a href={link} onClick={handleClick}>
      Handle Click
    </a>
  )
}

export default ClickExample

In this example, we use router.push(link) to navigate to the specified link. This method adds the new route to the browser's history stack. If you don't want to save the URL in history, you can use router.replace(link) instead.

Nextjs onClick redirect (TypeScript)

import { FC } from 'react'
import { useRouter } from 'next/router'

interface ClickExampleProps {
  link: string
}

const ClickExample: FC<ClickExampleProps> = ({ link }) => {
  const router = useRouter()

  const handleClick = (event: React.MouseEvent<HTMLAnchorElement>) => {
    event.preventDefault()
    router.push(link)
  }

  return (
    <a href={link} onClick={handleClick}>
      Handle Click
    </a>
  )
}

export default ClickExample

We can utilize useRouter hook to handle click events routing.
Here router.push(link) pushes the link to router history.
If you do not want the link URL to be saved in history,
then router.replace(link) can be used.

Nextjs Login Form example redirect and prefetch

import { useCallback, useEffect, useState } from 'react'
import { useRouter } from 'next/router'
import axios from 'axios'

export default function Login() {
  const router = useRouter()
  const [username, setUsername] = useState('')
  const [password, setPassword] = useState('')

  const handleSubmit = async e => {
    e.preventDefault()

    console.log(username, password)

    if (username && password) {
      const options = {
        method: 'post',
        url: 'http://localhost:3000/login',
        headers: {
          'Content-Type': 'application/json; charset=utf-8',
        },
        data: {
          username,
          password,
        },
      }
      const response = await axios(options)
      if (response.status == '200') {
        router.push('/home')
      }
    }
  }

  useEffect(() => {
    // Prefetch the home page for faster transition
    router.prefetch('/home')
  }, [])

  return (
    <form onSubmit={handleSubmit}>
      <input
        type='text'
        name='username'
        onChange={e => {
          setUsername(e.target.value)
        }}
      />
      <input
        type='password'
        name='password'
        onChange={e => {
          setPassword(e.target.value)
        }}
      />
      <button type='submit'>Login</button>
    </form>
  )
}

Nextjs Login Form example redirect and prefetch (TypeScript)

import { useState, useEffect, FormEvent, ChangeEvent } from 'react'
import { useRouter } from 'next/router'
import axios from 'axios'

interface LoginResponse {
  status: number
  data: {
    token: string
  }
}

const Login = () => {
  const router = useRouter()
  const [username, setUsername] = useState<string>('')
  const [password, setPassword] = useState<string>('')

  const handleSubmit = async (e: FormEvent<HTMLFormElement>) => {
    e.preventDefault()

    if (username && password) {
      const options = {
        method: 'post',
        url: 'http://localhost:3000/login',
        headers: {
          'Content-Type': 'application/json; charset=utf-8',
        },
        data: {
          username,
          password,
        },
      }

      try {
        const response = await axios(options)
        if (response.status === 200) {
          router.push('/home')
        }
      } catch (error) {
        console.error('Login failed:', error)
      }
    }
  }

  useEffect(() => {
    router.prefetch('/home')
  }, [router])

  const handleUsernameChange = (e: ChangeEvent<HTMLInputElement>) => {
    setUsername(e.target.value)
  }

  const handlePasswordChange = (e: ChangeEvent<HTMLInputElement>) => {
    setPassword(e.target.value)
  }

  return (
    <form onSubmit={handleSubmit}>
      <input type='text' name='username' onChange={handleUsernameChange} />
      <input type='password' name='password' onChange={handlePasswordChange} />
      <button type='submit'>Login</button>
    </form>
  )
}

export default Login

In this simple login form example, we can see how to use Nextjs link onClick to redirect to the home page after a successful login API call.
router.push('/home') will redirect to the homepage, and similarly, on failure, we could redirect to an error page.
Here, router.prefetch('/home') prefetches the home page for a faster transition.
One thing to note is that as useRouter is a hook, it can only be called in a functional component.
The Nextjs link onClick functionality is demonstrated through the use of the router.push() method, which allows for programmatic navigation based on user interactions or form submissions.

Best Practices and Optimization

When working with Next.js link onClick functionality, consider the following best practices:

Use router.push() for most cases: This method adds the new route to the browser's history stack, allowing users to navigate back.
Use router.replace() for login/logout: This replaces the current history entry, preventing users from navigating back to a logged-out state.
**Leverage router.prefetch(): Prefetching can significantly improve perceived performance by loading the target page in the background.
Handle errors gracefully: Always include error handling in your routing logic, especially when dealing with asynchronous operations like API calls.
Use TypeScript for better type safety: TypeScript can help catch potential errors early and improve code maintainability.

Here's an example incorporating these best practices:

import { useRouter } from 'next/router'
import { useState } from 'react'

const OptimizedNavigation = () => {
  const router = useRouter()
  const [isLoading, setIsLoading] = useState(false)

  const handleNavigation = async (event: React.MouseEvent<HTMLAnchorElement>) => {
    event.preventDefault()
    setIsLoading(true)

    try {
      // Perform any necessary async operations here
      await someAsyncOperation()

      // Use replace for login/logout scenarios
      await router.replace('/dashboard')
    } catch (error) {
      console.error('Navigation failed:', error)
      // Handle error (e.g., show error message to user)
    } finally {
      setIsLoading(false)
    }
  }

  // Prefetch the dashboard page
  useEffect(() => {
    router.prefetch('/dashboard')
  }, [router])

  return (
    <a href="/dashboard" onClick={handleNavigation} aria-disabled={isLoading}>
      {isLoading ? 'Loading...' : 'Go to Dashboard'}
    </a>
  )
}

export default OptimizedNavigation

By following these best practices, you can create more robust and performant applications using Next.js link onClick functionality.

Scraping Jobs listed on Hacker News

Hamza Rahman — Mon, 19 Aug 2024 13:07:21 +0000

Introduction

Web scraping is a way to get data from websites, the data can be text, pictures, or any other downloadable files, etc.
In this tutorial, we will scrape a static website that is not CSR i-e client-side rendered with JavaScript (React/ Angular sites).

In this tutorial will be using web scraping to get relevant jobs that are listed on Hacker News. We will be saving JavaScript, react, node, etc. jobs to Excel whenever our scraper code is executed.
Also, we will be making the code configurable to easily change filter parameters.

We will be web scraping with node.js, to do this we will get the site's HTML with Axios and then with the help of cheerio find the data we need to save.
Further, we will see how to save the scraped data to a CSV file in node.js and handle pagination.

Project setup

Let's start by creating a new directory for our project, we can name the folder web-scraping-tutorial

$ mkdir web-scraping-tutorial

go into the new folder

$ cd web-scraping-tutorial

Now assuming Node.js and npm is installed, we can initiate our project with:

$ npm init -y

And install the dependencies:

$ npm i axios cheerio csv-write-stream dotenv moment

Code

Requesting website HTML

We will use Axios to visit the website URL and get the HTML.
Let's create a new file request.js and use this code to visit Hacker News

const requestURL = async () => {
  const axios = require('axios')

  let options = {
    method: 'get',
    url: 'https://news.ycombinator.com/',
  }

  try {
    const response = await axios(options)
    console.log(response.data)
  } catch (error) {
    console.error(error)
  }
}

requestURL()

Here we configure the options for Axios and set the URL we want to visit and set method to GET, passing this options object to Axios, we get the response object after the Axios promise is resolved.
Running request.js file with node

$ node request

We see in response.data the site's HTML, now we can move to the next step and find the data we need in the HTML response.
Let's make the function more dynamic and export it, so we can pass it a URL and receive the site HTML back.

const axios = require('axios')

const requestURL = async URL => {
  let options = {
    method: 'get',
    url: URL,
  }

  try {
    const response = await axios(options)
    return response.data
  } catch (error) {
    console.error(error)
  }
}

module.exports = requestURL

env file

With an env file, we can pass configs to our handler function. Notice here FILTER_VALUES is an array but env file variables are string, so we will need to convert it to an array when using it.

URL="https://news.ycombinator.com/jobs"
FILTER_VALUES=["javascript", "node", "react", "full stack", "full-stack"]
MAX_DAYS_BACK=30

URL variable is the site we will be visiting to scrape jobs.

FILTER_VALUES variable are the job types based on which we will be filtering data. e.g if we just keep 'javascript' here then only jobs that mention javascript will be saved.

MAX_DAYS_BACK variable will allow us to configure how old jobs we want to scrape.

Handler function

Let's make a handler.js file and make a function to control the app flow.

require('dotenv').config()
const request = require('./request')

//env file variables
let { URL, FILTER_VALUES: filterValues, MAX_DAYS_BACK: maxDaysBack } = process.env
filterValues = JSON.parse(filterValues) //convert string to array

const scrapePage = async URL => {
  //get site HTML
  let siteHTML = await request(URL)
  console.log(siteHTML)

  //process HTML and get relevant data
  //save to CSV file
  //scrape next pages
}

scrapePage(URL)

Now that we have accessed the site's HTML with Axios we can pass this HTML to cheerio.

Cheerio

Cheerio can parse the HTML and provide us with an API with which we can access and manipulate data as we require.
Cheerio uses a subset of jQuery and therefore makes it easy for us to use CSS selectors to target DOM elements and access data.

Let's make a processHTML.js file where cheerio can run its magic.

const cheerio = require('cheerio')

const processHTML = HTML => {
  const $ = cheerio.load(HTML)
  let data = []
  let nextPage = ''
  //css selectors
}

module.exports = processHTML

CSS selectors

From the HTML we require 4 things, the next page URL, the title of the job, the Link of the job, and the time job was posted.
Let's go for the CSS selector of these, one by one:

Next Page URL

At the bottom of the page, you can find a more button that takes us to the next page. In our browser, if we right-click the more button and inspect the element, we see the site's HTML and the selected element focused.

With the help of this inspect element tool, it's easy for us to see the element's CSS class. In this case, it's 'morelink'
In our code below the//css selectors line, let's use the nextPage variable to save the next page URL.

nextPage = 'https://news.ycombinator.com/' + $('.morelink').attr('href')

This way with the help of cheerio we get the element with class morelink and access its attribute href. Now, combining this URL with the site's base URL gives us the complete URL for the next page.

Job Title

As the other 3 remaining fields are multiple, unlike nextPage, we will have to loop through the elements.
If we inspect the job title, we see something like this:

Instead of targeting the class titlelink directly, we will loop on a parent class of title athing and then find the element titlelink, doing so will ease the process of targeting the time posted selector as it's the next element to athing
In our code below the nextPage selector we can do this:

$('.athing').each((index, element) => {
  let title = $(element)
    .find('.titlelink')
    .text()
})

Here we are using .each, so we can access all the elements on the page with class athing, after doing so we can work on each element in the arrow function as we have access to the index and element.

Inside the arrow function, we can get the title of each element with the help of the .find() function, where we pass the CSS selector of an element we want to access. In our case, we pass .titlelink, so it can find it based on its class. And finally, .text() allows us to keep the text content of an element.

Job Link URL

Just like the title, link will use the same selector but instead of element text, we need the href.

let link = $(element)
  .find('.titlelink')
  .attr('href')

.attr() function allows us to select an elements attribute, In our case, we use it to get href.

Time Posted

Time of job posted is not a child element of athing, but it's the child element of the next sibling element.

let timeAgo = $(element)
  .next()
  .find('.age')
  .text()

Here .next() allows us to move to the next sibling element in which we can find the element with class age and save its text content.

Filtering Data

To the processHTML functions let's pass the additional fields from the env file according to which we can filter out our data.

const processHTML = (HTML, filterValues, maxDaysBack) => {
  //...
}

Below the timeAgo selector, we will add this code to check whether the title includes any of the job titles we are interested in.

let titleLowerCase = title.toLowerCase()

let hasFilterItem = filterValues.some(item => titleLowerCase.includes(item))

The first step for filtering is to convert the title to lower case so we don't miss any jobs that may have upperCase words. In our enc file filterValues are also all lower case.

In the second step, we use the javascript .some method to find whether any of the filterValues are present in the job title. If one or more are present hasFilterItem becomes true otherwise false.

The other thing we want to keep in mind is to only keep freshly posted jobs, for that we have the MAX_DAYS_BACK variable in the env file. In our code below hasFilterItem let's add
this code:

//time of last job posted on current page
let timeAgoSimplified = timeAgo.includes('days') ? timeAgo.split(' ')[0] : 1
timeAgoSimplified = parseInt(timeAgoSimplified)

Here timeAgoSimplified takes the timeAgo string we extracted earlier and if it is in days we take only the number value from it and convert the string to an integer using the parseInt method. And in case the timeAgo isn't in days it would be in hours/minutes, so in that case, we save it as 1 day.

Saving Data

Now that we have the filters ready we can push each job that fulfills our requirement to the data array.

//check if time posted was less than maxDaysBack variable
if (timeAgoSimplified < maxDaysBack) {
  data.push({
    link,
    title,
    timeAgo,
    hasFilterItem,
  })
} else {
  //if current page data is already older we don't move to the next page
  nextPage = null
}

With timeAgoSimplified we can check whether the job posted fulfills the maxDaysBack requirement and save only new jobs. In this code, I am saving all jobs to CSV whether they pass or fail the hasFilterItem check, but I will be adding a boolean column that will be true only if hasFilterItem is true.

If we wanted to keep only the jobs that have hasFilterItem true we can use:

//check if time posted was less than maxDaysBack AND filter values are also present
if (timeAgoSimplified < maxDaysBack && hasFilterItem) {
  //...
}

Outside the .each loop we return the data handler file:

return { data, nextPage }

With this our complete processHTML.js code looks like this:

const cheerio = require('cheerio')

const processHTML = (HTML, filterValues, maxDaysBack) => {
  const $ = cheerio.load(HTML)
  let data = []
  let nextPage = ''

  nextPage = 'https://news.ycombinator.com/' + $('.morelink').attr('href')

  //.each to loop over all class athing elements
  $('.athing').each(async (index, element) => {
    let title = $(element)
      .find('.titlelink')
      .text()
    let link = $(element)
      .find('a')
      .attr('href')
    let timeAgo = $(element)
      .next()
      .find('.age')
      .text()

    let titleLowerCase = title.toLowerCase()
    let hasFilterItem = filterValues.some(item => titleLowerCase.includes(item))

    //time of last job posted on current page
    let timeAgoSimplified = timeAgo.includes('days') ? timeAgo.split(' ')[0] : 1
    timeAgoSimplified = parseInt(timeAgoSimplified)

    //check if time posted was less than 30 days maxDaysBack
    if (timeAgoSimplified < maxDaysBack) {
      data.push({
        link,
        title,
        timeAgo,
        hasFilterItem,
      })
    } else {
      nextPage = null
    }
  })

  return { data, nextPage }
}

module.exports = processHTML

Now that our processHTML.js function is set up, from our handler.js file we can call it and pass the HTML and env variables required to like so:

//process HTML and get relevant data
let { data, nextPage } = processHTML(siteHTML, filterValues, maxDaysBack)
console.log(data, nextPage)

Now running handler.js gives us an output like this, which is all the data we require from the page!

Saving data to CSV file

Now that we have the data accessed and well-formatted we can save it to a CSV file.
To do so we will be using the csv-write-stream library.

In handler.js let's import csv-write-stream, moment and fs and use them to create a CSV file.

//New code added below previous imports
const moment = require('moment')
const fs = require('fs')
const csvWriter = require('csv-write-stream')

//set csv file headers
const writer = csvWriter({
  headers: ['Link', 'Title', 'timeAgo', 'hasRequiredJob'],
})

//create a file in which data will be appended
const fileName = moment().format('YY-MM-DD-HH:mm:ss') + '-hackerNews-jobs' + '.csv'
writer.pipe(fs.createWriteStream(fileName, { flags: 'a' }))

After importing the new libraries we set the CSV file headers. We initialize the filename and keep it unique every time. If the filename is not unique running the app will append data and headers to the old file. To create a unique filename every time I am using moment to add the date and time to the filename.

We are using { flags: 'a' } in createWriteStream to append to the file created so writer will write new data at the bottom of all previous data(rows).

Now to write the incoming data to this file, add new code below the processHTML method call:

//process HTML and get relevant data
let { data, nextPage } = processHTML(siteHTML, filterValues, maxDaysBack)

// save to CSV file
for (let rowData of data) {
  let { link, title, timeAgo, hasFilterItem } = rowData

  writer.write([link, title, timeAgo, hasFilterItem])
}

Now running the handler.js file creates a CSV file and saves the data rows.

Handling site Pagination

Now that we have saved data for a single page successfully, saving data for the next pages is simple enough.

We need to visit nextPage from processHTML and repeat the same process for saving data.

// scrape next pages

if (nextPage) {
  console.log(URL)
  scrapePage(nextPage)
} else {
  console.log('Finished!')
}

If processHTML method returns nextPage we visit it in a recursive loop till nextPage is null(In case data doesn't satisfy maxDaysBack check)

Our complete code for handler.js

require('dotenv').config()
const request = require('./request')
const processHTML = require('./processHTML')
const moment = require('moment')
const fs = require('fs')
const csvWriter = require('csv-write-stream')

//set csv file headers
const writer = csvWriter({
  headers: ['Link', 'Title', 'timeAgo', 'hasRequiredJob'],
})

//create a file in which data will be appended
const fileName = moment().format('YY-MM-DD-HH:mm:ss') + '-hackerNews-jobs' + '.csv'
writer.pipe(fs.createWriteStream(fileName, { flags: 'a' }))

//env file variables
let { URL, FILTER_VALUES: filterValues, MAX_DAYS_BACK: maxDaysBack } = process.env
filterValues = JSON.parse(filterValues) //convert string to array

const scrapePage = async URL => {
  //get site HTML
  let siteHTML = await request(URL)

  //process HTML and get relevant data
  let { data, nextPage } = processHTML(siteHTML, filterValues, maxDaysBack)

  // save to CSV file
  for (let rowData of data) {
    let { link, title, timeAgo, hasFilterItem } = rowData
    writer.write([link, title, timeAgo, hasFilterItem])
  }

  console.log(data.length, ' Jobs saved!')

  // scrape next pages
  if (nextPage) {
    console.log(URL)
    scrapePage(nextPage)
  } else {
    console.log('Finished!')
  }
}

scrapePage(URL)

Conclusion

Web scraping is like a jigsaw puzzle but in reverse. You get the complete site's HTML(Complete puzzle) and then you have to somehow programmatically find only the data pieces you need(individual pieces 🧩).
This process is made easier with cheerio which allows us to select the HTML element we need with easy-to-use Jquery API.

In a future article, we will learn how to scrape with puppeteer.js to handle JS-generated sites and also perform actions on websites.

Complete Code

You can find the complete code at this repo:

Github