Forem: Faisal

JavaScript Coercion : Beyond Basics

Faisal — Sat, 02 Dec 2023 15:03:14 +0000

In JavaScript, we often see implicit type conversion in our code which occurs due to abstract operation. In JS, we use the term coercion for what's commonly known as type conversion. When we consider conversion and coercion, it's best to see them as interchangeable, especially in the context of JavaScript.

Coercion is a weird topic in javascript, that is why many tends to ignore this topic. However, we can't ignore something that behaves counterintuitively. We will start by exploring how abstract operations and coercion occur implicitly. Following that, we'll discuss why they are important.

Here one thing to note that, abstract operations are not like regular functions that you explicitly call in your code. Instead, they represent conceptual operations that help describe the behavior of certain language features.

While there may be internal methods or mechanisms within the JavaScript engine that handle the tasks associated with abstract operations, the key idea is that these operations are more about defining behavior and functionality at a higher level of abstraction.

ToPrimitive Operation

The abstract operation that we will talk about first is called ToPrimitive or primitive coercion process. The ToPrimitive operation converts a value to a primitive value, either a string, number, or a default value. It is often invoked implicitly by JavaScript when an object is used in a context where a primitive value is expected.

Let's say we are performing an operation where it requires a value to be primitive. Now, if we don't have a primitive in there, we need to turn that into a primitive.

If we have a non-primitive entity that needs to be transformed into a primitive one, conceptually, what it must follow is a series of algorithmic steps. This set of steps is referred to as ToPrimitive, almost as if it were a function that could be called.

The ToPrimitive operation takes an optional type hint. The hint helps to figure out what it needs to be converted to. Let's say if we are doing a numeric operation and if there needs to run the ToPrimitive operation, the hint will be something like this - 'I would like it to be a number'. It's just an implicit process where JavaScript understands what type it needs to be. If we do something string-based, it sends a hint as a string. These are the two basic hints. And if it can't figure out the hint, it returns whatever the best primitive value it can return in that context.

One important thing is that -

JavaScript algorithms inherently follow a recursive pattern. For example, if what returns from ToPrimitive isn't a primitive type value but instead another non-primitive thing, ToPrimitive will be called again. It will keep getting invoked until we obtain an actual primitive. If it can't return a primitive value, it eventually results in an error.

The ToPrimitive operation relies on two methods - valueOf and toString. These can be available on any non-primitive value in JavaScript, be it an object, function, array, or any other non-primitive type. These two methods/functions play a crucial role in the process of converting non-primitive values to primitive ones.

Remember, ToPrimitive operation takes an optional hint.

Now if the hint is number, then ToPrimitive invokes the valueOf method first and see what it returns. If valueOf method gives a primitive, then we're done. However, if it doesn't give a primitive or if the method itself doesn't exist, then it tries the second method - toString.

If toString method also fails to give a primitive, it usually leads to an error.

JavaScript calls the toString method to convert an non primitive to a primitive value. We rarely need to invoke the toString method ourselves; JavaScript automatically invokes it when encountering an object where a primitive value is expected. MDN

If the hint is string, still both valueOf and toString method occurs, but in reverse order. In terms of string hint, if it ToPrimitive operation tries to make a non primitive into a primitive, toString method invoke first. Then, if it gives us a primitive like a string, we will just use that.

So, no matter what the hint is, if we are trying to use something that is not primitive where it needs to be primitive, like in some math or concatenation, then it goes through ToPrimitive algorithm and it ends up either invoking the valueOf method or toString method.

Diving deep into valueOf method

Let's take an object - const obj = {value: 4}.

Now, obj + 3 returns us '[object Object]3' as output. But, shouldn't it be 7? Since the object's value is giving a number hint, it should already be turned into a number, right? Then, why is it behaving this way?

To figure out that answer, we need to understand that, under the hood, JavaScript is still running the ToPrimitive operation on our written obj to convert it to work in primitive operations - like adding value. And, since it's taking a number as a hint - as we know, it is running the valueOf method first. But, the problem is, the default valueOf method of JavaScript, which gets inherited from Object.prototype, is basically useless in this case because the valueOf method inherited from Object.prototype returns the object itself (this).

And, since the ToPrimitive algorithm is recursive, it runs again, and sees that the valueOf method is not working, so, it falls back into the toString method. And when the toString method runs on any object type, it returns - "[object Object]"

So, our object basically is turning into a string first which is "[object Object]", Then, when we use the + operator between two string types, it just concatenates them.

obj + 3

"[object Object]" + 3

"[object Object]3"

Many built-in objects override the valueOf method to return an appropriate primitive value. When you create a custom object, you can override valueOf to call a custom method so that your custom object can be converted to a primitive value when needed.

For example -

const obj = {
    value: 4,
}

obj.valueOf() // {value: 4}

It is returning the object itself(this). But, you can create a function to be called in place of the default valueOf method. Your function should take no arguments, since it won't be passed any when called during type conversion.

const obj = {
    value: 4,
    valueOf: function() {
        return this.value;
    }
};

obj.valueOf() // 4

In this case, it is just returning the value. And, if we try to execute something where our written obj will need to be a primitive type, we will see that it will behave as expected -

obj + 3 will return 7.

Why? Because, since the adding operation is invoking the ToPrimitive operation with a number hint, so, the valueOf method is running first. But, now, our obj has its own custom valueOf method, so it's not executing the inherited valueOf method from Object.prototype which returns this object itself. Instead, for our custom valueOf method, it's giving us the actual number form of the value, since we are getting a primitive type already, it's not falling into the toString method.

ToString Operation

Here are some values and their corresponding string values after performing the ToString operation on them:

null => "null"
undefined => "undefined"
true => "true"
false => "false"
3.14 => "3.14"
0 => "0"
-0 => "0"

The ToString operation is executed during abstract operations involving implicit type conversion.

When ToString is applied to object types like arrays, it returns values as demonstrated in the following examples:

[] => ""
[1, 2, 3] => "1,2,3"
[null, undefined] => ","
[ [ [ ], [ ] ], [ ] ] => ",,"
[,,,] => ",,,"

For objects:

{} => "[object Object]"
{a: 2} => "[object Object]"

We can modify the default behavior of toString by implementing a custom toString method in our object. It will be called when needed:

{toString() { return "A"; }} => "A"

ToNumber Operation

There are a lot of corner cases involved in terms of ToNumber.

If we need to do something numeric and we don't have a number, JS invokes the ToNumber abstract operation. Some conversions are pretty straightforward, while others are counterintuitive.

"" => 0
- An empty string returns 0. Empty string should be NaN, right? Empty means there is nothing. How can nothing be 0? That's weird. It's not only a weird case in JavaScript; it creates a lot of other problems in terms of [[Coercion]] in JS.
" 003 " => 3
- If there are any empty spaces, JS removes those and returns the corresponding number.

Others are pretty straightforward:

"0" => 0
"-0" => -0
"3.14" => 3.14
"0.0" => 0
"." => NaN
- "." turns into NaN, which is as it should.
"0xaf" => 175
- Hexadecimal string turns into their corresponding numbers.
false => 0
true => 1
- Within the context of programming, maybe it's a sensible thing to turn false and true into 0 and 1, as that's how computers treat them. But, it creates an issue. false and true should turn into NaN. We will see the reason later when we discuss the corner cases.
null => 0
undefined => NaN
- Null and undefined both should be NaN, but one turns into 0, and another turns into NaN. That's interesting. For those who think that converting null uses the valueOf method of Object because null is an object, one interesting thing is that null doesn't inherit methods from Object.prototype.

ToNumber Operation on Non-Primitive

When we run ToNumber on a non-primitive or object, it evokes the ToPrimitive operation with the number hint. In ToPrimitive operation, it first runs the valueOf method, which is inherited from the Object.prototype by default, and it returns the object itself (this). JS ignores that, and it falls into the toString method.

Since it falls into tpString, no matter if our hint is a number or not, the value turns into a string. So, we can think of the numberification of an object as the stringification of it.

There are cases where we want something to be a primitive number, and we can end up with a primitive string for the internal process of valueOf. We need to be aware of that.

If it's empty like [] or {}, then the same things happen. It evokes the ToPrimitive, and the valueOf returns the empty parenthesis - [] or {}, then it falls into toString as usual.

Some more ToNumber operations in terms of arrays:

[""] => 0
["0"] => 0
["-0"] => -0
[null] => 0
[undefined] => 0
[1,2,3] => NaN
[[[]]] => 0

Few important things to notice here:

[""] falls into ToString, and then it returns an empty string. If ToNumber tries to convert an empty string, it converts it to 0, which is our old problem as we saw earlier.
Similar things happen for [[[]]] - first, it turns into "" by toString, then, it converts into a number, which is 0. Emptiness shouldn't be 0 in this case.
We see that [null] and [undefined] return 0 in the ToNumber operation, which doesn't make sense. We have seen different behavior in their case when we have tried to convert primitive null and undefined. But, in this case, they are returning an identical value, which is 0. Why? In this case, this is a non-primitive thing in both cases, and as it's non-primitive, ToPrimitive evokes, and the valueOf method runs first. It returns this, then it falls into the toString method, and remember what toString does on a non-primitive with null or undefined? It turns into an empty string. And then, that empty string turns into 0. Our root problem again.
{..} => NaN
- If it's an object {}, then the same thing happens. ToPrimitive evokes, then it falls into toString. ToString returns "[object Object]", which is definitely not a representation of a number, so it returns NaN.
- But, if we add our custom valueOf method, it doesn't fall into toString, and it returns what we want to be returned.

ToBoolean Operation

This is pretty straightforward. We just need to remember all the falsy values, which will always return false if we try to run ToBoolean operation on them. Everything else returns true.

Falsy values: "", 0, -0, null, NaN, false, undefined Truthy values: Everything else.

One thing is important to notice here:

When we try to convert anything to Boolean, it doesn't evoke any other operation under the hood, like ToNumber. It just checks its truthiness or falsiness.

That's why if we convert [ ] to a boolean, it returns true, and if we convert "" to a boolean, it returns false. Because an empty string is a falsy value, [ ] is not.

If it would evoke the toString method under the hood, that would convert [] to an empty string, and then we would get false because of that empty string. But, in this case, that is not happening.

Corner Cases

Every language has type conversion. We can't just avoid coercion in JavaScript for its corner cases. No language is free from various kinds of corner cases. We just need to be aware of those corner cases and learn how to work around them. Here are some of the corner cases we need to be aware of -

Number("") // 0  (!!!)
Number("   \t\n") // 0  (!!!)
Number(null) // 0  (!!!) should be NaN!
Number(undefined) // 0 
Number([]) // 0
Number([1,2,3]) // NaN


Number([null]) // 0  -> !!!
Number([undefined]) // 0  -> !!!

Number({})  // NaN


String(-0)  // "0"   -> what! should be "-0"!
String(null)  // "null"  -> good!
String(undefined)  // "undefined"  -> good!
String([null])  // ""  -> !!!
String([undefined])  // ""  -> !!!

Boolean(new Boolean(false)); // true  -> what!!!

One of the root causes of corner cases occurring in coercion in JS is the conversion of "" to 0. We have already seen what kind of problems it can cause. Many coersions would just be gone if an empty string wouldn't turn into 0 in number conversion; if it would be NaN. Whatever, we can't do anything about it now because of backward compatibility.

Another important corner case is true becoming 1, and false becoming 0 in terms of converting to number operation. Number(true) -> 1 Number(false) -> 0

Though it is an intuitive thing to us. Let's see how it creates a problem. 1 < 2 -> true; Okay. 2 < 3 -> true; It's also okay. 1 < 2 < 3 -> true. Okay. You are thinking - it's exactly working as it should be.

But that is not the case; what is happening under the hood is it first returns true for the first part. Then, true converts into 1, since we are performing a numeric operation. Then, 1 < 3 obviously returns true.

(1 < 2) < 3
(true) < 3
1 < 3

We see how this can create a problem.

7 > 5 // true

5 > 3 // true

7 > 5 > 3 // false --> !!!

Let's see, which is actually happening.

(7 > 5) > 3
(true) > 3
1 > 3 // false

Since 1 is not greater than 3, it returns false. Now, we understand how converting true and false to 1 and 0 creates problems. It would be solved if it were just NaN.

Some of us hate coercion because of it's weird corner cases. But, we can't ignore coercion. We use coercion all the time.

Practical Use Cases of Coercion

Coercion is a weird topic in JavaScript. But, we can't avoid it either. It turns out, we all deal with coercion all the time! Let's see some examples.

var numPeople = 19;

console.log(`There are ${numPeople} people out there.`);

// There are 19 people out there.

var firstPart = "There are ";
var numPeople = 19;
var secondPart = " people out there.";

console.log(firstPart + numPeople + secondPart);

// There are 19 people out there.

It turns out that if we use the + operator and any of the values is a string, it converts the entire expression into a string. This is known as Operator Overloading. Our number is implicitly converting into a string. Some may want to make it explicit:

var numPeople = 19;

console.log(`There are ${numPeople + ""} people out there.`);

// There are 19 people out there.

var numPeople = 19;

console.log(`There are ${[numPeople].join("")} people out there.`);

// There are 19 people out there.

var numPeople = 19;

console.log(`There are ${numPeople.toString()} people out there.`);

// There are 19 people out there.

var numPeople = 19;

console.log(`There are ${String(numPeople)} people out there.`);

// There are 19 people out there.

Writing explicit code is not a good decision always.

Some may ask, what about converting a string to a number? We do that too. What about form data? We all need to work with form data as JavaScript developer.

function addOnePeople(numPeople) {
    return numPeople + 1;
}

addOnePeople(peopleInputElement.value); // get people number from form

// "191" - WHAT!!!!

So, again, we need to convert it into a number first! There are two ways. First one -

function addOnePeople(numPeople) {
    return numPeople + 1;
}

addOnePeople(+peopleInputElement.value);

// 20

or we can use the Number function, which is the more preferred way.

function addOnePeople(numPeople) {
    return numPeople + 1;
}

addOnePeople(Number(peopleInputElement.value));

// 20

How about reducing the input value?

function ignoreOnePeople(numPeople) {
    return numPeople - 1;
}

ignoreOnePeople(peopleInputElement.value);

// 18

Since numPeople is a string and we are trying to perform an operation where a number is needed, the ToPrimitive process is triggered, and numPeople converts into a number automatically. That's why we don't need to explicitly mention that. We even need to check truthy and falsy values all the time.

if (peopleInputElement.value) {
    numPeople = Number(peopleInputElement.value);
}

Again, if we aren't aware of the corner case of Boolean, we may face a bug here. What if the input element contains a bunch of white spaces? That won't be a falsy value!

Boxing

We have seen that non-primitive values don't have methods in them. If that's the case, how can we access methods in a string like somestring.length?

It's called Boxing. It's a form of implicit coercion, although it doesn't occur in the same process as abstract operations do.

It's like when JavaScript sees that you are trying to run a method on a non-primitive value, JavaScript does you a favor. It thinks, "Okay, this person is trying to run an operation as if it's a primitive value (object); let's turn it into that and make their life easier!" And then, it converts it into the corresponding object representation of the string.

Maybe from this, the falsy notion comes: In JavaScript, everything is an object. No, just because it is converting into an object-like structure under the hood doesn't mean it's an object. Things can behave as similar, but that doesn't make them similar.

Two things are pretty different.

A significant part of my motivation to delve deeply into JavaScript was influenced by Kyle Simpson. I conclude here by sharing some of his philosophy on "coercion."

You don't deal with corner cases by avoiding coercions.
You have to adopt a coding style that makes value types plain and obvious.
A quality JS program embraces coercions, making sure the types involved in every operation are clear. Thus, corner cases are safely managed.
JavaScript's dynamic typing is not a weakness, it's one of it's strong qualities.
Implicit doesn't mean magic. It means abstraction. So, that it can hide unnecessary details, re-focusing the reader and increasing clarity.
- We can simiply depends on implicit type coercion rather than always describing type convertion explicitely every time, which will decrease readability.

It's "Useful", when the reader is focused on what's important. It's "Dangerous", when the reader can't tell what will happen. It's "Better", when the reader understands the code.

It's "Irresponsible" to knowingly avoid usage of a feature that can improve code readability.

And it's always better to learn how things work under the hood!

Happy Learning!

If you enjoyed reading this, you can connect with me on Twitter or check out my other articles.

Additional Resources

JavaScript Data Structures - MDN

valueOf - MDN

valueOf in JavaScript - ECMA

Abstract Operations: To Primitive - ECMA

You Don't Know JS by Kyle Simpson

7 Reasons Why You Need to Master the Fundamentals (A Visual Guide)

Faisal — Wed, 22 Feb 2023 02:29:12 +0000

Fundamentals are boring.

Admit it; we've all been there before: wanting to skip the dry and boring fundamentals and jump right into the exciting stuff. I know I have. But trust me when I say this: taking time to learn the basics can make all the difference in the long run. I had to learn this the hard way. But you’ll not have to make the same mistake I did.

Here are my 7 takeaways on why you should be patient enough to master the fundamentals first.

1. Builds a strong foundation

Fundamentals are the building blocks of the more advanced concepts you'll learn in the future. They help to establish a strong foundation upon which you can build more advanced knowledge or expertise. Your understanding of the subject will be solid and complete with a strong foundation.

2. Enables innovation

Creating anything truly impressive or innovative can only be easy with a solid understanding of the basics. Because innovation requires a mental understanding of certain underlying frameworks, this understanding will allow you to devise innovative solutions that can change the game. Innovation often comes from building on existing knowledge and improving it. It is only possible if you have a strong grasp of the basics.

3. Effortless learning

Once you master the fundamentals, it becomes easier to learn more advanced concepts. It may take time to build the foundation. But when you build the foundation, you can easily jump from one advanced topic to another.

4. Makes you efficient

You can increase your efficiency and productivity in the long run by mastering the basics. If you skip the basics and rush to advanced concepts, you may struggle to understand the material. It will slow down your learning process. Also, with a strong foundation, you will be able to finish tasks fast and are less likely to make mistakes.

5. Makes you a better problem-solver

Fundamentals help you develop critical thinking and problem-solving skills. You can better analyze and break big problems into small pieces and develop more effective solutions.

6. Builds confidence

Mastering the underlying principles lets you see how new and complex information fits together. It builds confidence and helps you feel more comfortable with the material. You can approach more advanced concepts easily and confidently with a solid foundation. Also, you will know what is happening and why. It also helps to reduce your imposter syndrome.

7. Provides versatility

Understanding the fundamentals of a subject can make you more versatile and adaptable. You can apply that knowledge in various situations by deeply understanding the basics. Also, you can connect knowledge among multiple disciplines if you master the fundamentals.

So, Even though it may be tempting to skip over the basics, stick with it and give the fundamentals the attention they deserve.

Recap

Fundamentals are important because it-
builds a strong foundation
enables innovation
makes learning effortless
increases efficiency and productivity
develops problem-solving skills
builds confidence and reduces imposter syndrome
provides versatility and adaptability

That's it! If you enjoyed reading this, you can connect with me on Twitter or check out my other articles. Additionally, you can subscribe to my non-tech newsletter, where I share my thoughts on life, productivity, and more, as well as the best content I’ve come across each month.

A Deep Dive into Node.js: Understanding its history, threading, and event-driven architecture

Faisal — Sat, 18 Feb 2023 04:58:11 +0000

What is Node.js?

According to Node’s official documentation, Node.js is a JavaScript runtime built on Google’s open-source V8 JavaScript engine. Node.js can execute JavaScript code on the server side, making it possible to create fast, scalable, high-performance network applications.

History of Node.js

Node.js was introduced in 2009 by Ryan Dahl. Despite initial skepticism, the concept gained popularity in 2011 when Dahl presented Node.js at a PHP meetup in San Fransisco on 22 Feb 2011. The video went viral, drawing attention to using JavaScript on the server side. In that video, Ryan Dahl explained Node.js in front of a bunch of programmers. He explained how he developed a new JavaScript runtime with the help of the v8 engine of Google Chrome and how we can use JavaScript on the server side. Before that, JavaScript was used only on the client and browser sides. After listening to Ryan’s explanation, most programmers made fun of him because they couldn’t accept JavaScript as a server-side language. At that time, some hot server-side languages like Ruby on Rails existed. But Ryan Dahl didn’t stop. Dahl saw the potential for using JavaScript on the server side and developing and improving Node.js as a solution.

Reasons Behind the Popularity of Node.js

There was some critical incident that helped the Node gain its popularity.

In 2010, Isaac Z. Schlueter created NPM, short for Node Package Manager. At the time, Node.js was a relatively new platform and lacked a standard package manager like those used in other programming languages. It made it easier for developers to share their code with other developers and reuse existing packages in their projects.
In 2007, Dwight Merriman and Eliot Horowitz created MongoDB. It uses a JSON-style binary data structure to store data which is easy to use with JavaScript. As expected, developer communities did not accept a NoSQL database like MongoDB because the SQL relational database was popular then. As social media companies like Twitter and Facebook started to publish APIs in JSON format, JSON-style data structures increased as developers found it more challenging to work with relational databases and JSON-format data. As a result, solutions like MongoDB and Node.js became more widely adopted.
Node.js has made it easier for developers to use JavaScript in both the front-end and back-end for their web applications. This capability has been a significant factor in developers’ popularity of Node.js. By using the same language on both the client and server sides, Node.js eliminates the need to switch between multiple programming languages, leading to more streamlined and efficient development. Furthermore, since JavaScript is a widely-used language, many developers already have experience with it, making the transition to using Node.js for server-side development a natural choice. These factors have contributed to Node.js becoming a go-to solution for building fast, scalable, and highly performant network applications.

The Architecture of Node.js and Understanding the V8 Engine

Let’s learn about the node architectures regarding the Node’s dependencies.

Node.js has several dependencies to work/ function properly. The most important ones are the v8 engine and the libuv. We saw earlier that Node.js was built on Chrome’s v8 engine. First, we must know the role of the v8 engine here.

Many of us write JavaScript for the browser. When we write code, machines don’t understand our code directly. The machine must convert it into machine code to understand what we command. There are engines in the browser to understand our code as machine code. The different browsers have different engines to understand/convert JavaScript code into machine code. Here are some examples:

Google Chrome: V8
Mozilla Firefox: Spider Monkey
Apple Safari: JavaScriptCore
Microsoft Edge: Chakra
Opera: Blink (which is based on the same engine as Google Chrome, V8)

The most powerful one is Chrome’s v8 engine.

On the server side, there has to be something that converts JavaScript code to machine code. For this, Ryan Dahl used a super-speed V8 engine in Node.js. Now, by this, Node.js can convert JavaScript code into machine code. That’s the use case of the V8 engine in Node.

libuv Library

V8 engine is not alone enough to run a server-side framework like Node.js. Here comes another critical library called “libuv.” It’s an open-source library that strongly focuses on asynchronous IO(Input/output).

By libuv, Node.js can access the underlying computer operating system, file system, stuff related to networking, and more.

“libuv” implements two critical features of Node.js —

Event Loop, which handles easy tasks like a callback or network io
Thread Pool, to handle heavy work like accessing files or file compression.

I’ll discuss these two featured in detail later.

Is Node.js Entirely Written in JavaScript?

We intuitively think there is only JavaScript code in Node.js behind the scenes, as Node.js is a JavaScript runtime. But that’s not the case. As I said, libuv is an essential part of Node, and libuv is entirely written in C++. On the other hand, V8 is also written in JS & C++. So, Node is not only written in JavaScript but also C++. But there are some abstraction layers so we can access all functions and everything in the Node by pure JavaScript functions. We don’t have to mess up with C++ or any underlying code. For example- for file reading, we write a pure js function to read from the file system, which functionality is written in libuv in C++. Pretty interesting.

An important thing to note is that Node.js depends not only on the V8 engine and libuv. There are other libraries as well, like — HTTP parser (for parsing HTTP), c-ares (for handling DNS request stuff), OpenSSL (for cryptography), zlib (for file compression), etc.

Now let’s see what thread and thread pool are.

Thread & Thread Pool

A node process is initiated on our computer whenever we use Node.js. This process represents a program that is currently in execution.

Within that process, the Node runs in a single thread. A thread is like a mini-program within the bigger program, a sequence of instructions that can perform multiple tasks simultaneously. It’s like a box where our code is executed.

It’s essential to understand the concepts of multithreading and single threading clearly. Multithreading enables the execution of multiple operations simultaneously, where several tasks can run in the background at the same time. On the other hand, single threading only allows one operation to be performed at a time. In other words, it operates in a linear sequence and cannot execute multiple operations simultaneously.

It’s important to understand that Node runs on a single thread, no matter we have how many users. This means that if the single thread gets blocked, the entire application can be affected. So, we need to be extra mindful of actions that could potentially block the thread.

Let’s see what happens when the Node runs in a single thread.

When a Node application is run, the program goes through several stages. Node.js initializes the program, executes all the top-level code, requires necessary modules, and then event callbacks are registered.

Once all these steps are completed, the event loop starts running. Event loop where the majority of the work of the application gets done. It’s the heart of the application. But, some tasks can be resource-intensive and heavy. They can potentially block the single thread if executed within the event loop. Handling these heavy tasks outside the event loop is important to avoid this.

Here, the thread pool comes to the rescue, which the libuv library provides. The event loop offloads those heavy tasks to the thread pool. It happens behind the scene. We don’t have to worry about which tasks will go to the thread pool and which will not.

The thread pool has four additional threads separate from the single main thread. We can configure a thread pool of up to 128 threads. But, usually, we don’t need to. Four threads are enough.

Here are some task examples that are offloaded to the thread pool from Event Loop -

Tasks that use file system API: When we perform operations like reading or writing files, these tasks can take a significant amount of time. They can block a single thread and cause performance issues.
Cryptography-related task: Cryptography involves complex mathematical calculations. And performing these operations in the Event Loop can slow down our application.
Compression tasks: When we need to compress large data like images or videos, these tasks can take a lot of time and can also slow down the Event Loop. Therefore, these tasks are offloaded to the thread pool to prevent this.
DNS lookup: When we need to resolve a domain name into an IP address, it’s done through a DNS lookup process. This process can also take time and block the Event Loop, which may cause performance issues. Therefore, to prevent this, these tasks are offloaded to the thread pool, where they can run concurrently without blocking the main thread.

Event Loop and Its Implementation in Node.js

Now, we are in the Node.js process. This process contains a thread where the event loop runs. The Event Loop is an essential component of the Node.js process. It is responsible for executing all of the code within callback functions that make up a Node.js application.

As we learned earlier, some tasks may also be offloaded to the thread pool for efficient processing. But, the event loop is the central mechanism of the Node.js runtime. Node.js is based on a callback-driven architecture. It means that functions are called as soon as a certain event has finished or been emitted. This happens because of the event-driven architecture of Node.js.

Event-Driven Architecture

Event-driven architecture is a system where events are emitted, picked up, and processed by the Event Loop. Then the Event Loop executes the associated callback functions to that event.

For example, events such as a new HTTP request, expiration of a timer, or a finished file reading or writing into a file operation will emit events, which the Event Loop will then pick up. It then processes those events by executing the associated callback functions. The Event Loop observes events and performs the necessary orchestration behind the scenes.

Let’s take a closer look at how the event loop actually operates behind the scene:

The event loop starts running as soon as the Node.js application starts. The Event Loop has several phases. Each phase has its own callback queue. Some might say the Event Loop has only one callback or event queue. But it has multiple phases, each with its own callback queue.

The four most important phases of the Event Loop are as follows:

Handling expired timer callbacks (setTimeout()): The first phase involves executing callbacks from expired timers. If a timer expires during any of the other phases, its associated callback function will be executed as soon as the current phase finishes, and the event loop returns to its first phase.
I/O polling and execution of I/O callbacks: Node IO means networking or file system stuff like file reading, writing into new files, etc., and Polling means — looking for new IO events that are ready to be processed and place them in the callback queue. This phase is where 99% of the application code is executed.
Executing SetImmediate callbacks: This is a special type of timer that allows code to be executed immediately after the execution of the I/O callbacks.
Executing close callbacks: The Event Loop processes all close events in this phase. This includes events such as closing a web server or a web socket.

In addition to these four phases, two other queues are important to be aware of:

The process.nextTick() queue: This queue executes its callbacks immediately after the current phase finishes. It is similar to the SetImmediate callbacks. The only difference is that SetImmediate callbacks only execute immediately right after the execution of the I/O callbacks, but process.nextTick() executes right after any of the phases.

The microtask queue for resolving promises: This queue executes its callbacks immediately after the current phase finishes, just like the process.nextTick() queue. If there are any callbacks in either of these two queues, they will be executed immediately after the current phase is completed rather than waiting for the entire event loop to finish its four phases. For example, if a promise resolves and returns data from an API call while the callback of an expired timer is running, its callback will be executed immediately after the timer’s callback has finished.

After each of the first four phases, the event loop checks if there are any callbacks in these two queues. If there are, they will be executed right away. This completes one tick of the event loop. A tick is defined as one cycle of the loop. After one cycle, the Node.js runtime checks if there are any pending timers or I/O tasks, or any callback of any phases. If there are, the loop runs again. Otherwise, the application exits.

Asynchronous programming is made possible in Node.js due to the Event Loop, which makes the Event Loop the most important feature of Node.js. It allows Node.js to run in a single thread, making it lightweight and fast but also presenting some potential risks. Since Node.js is a single-threaded program, it’s possible to block our application accidentally. So, it’s important to take extra care while writing code to avoid accidentally blocking the application.

Steps to Avoid Blocking the Event Loop

Here are some tips to avoid blocking the code in Node.js:

Avoid using synchronous versions of functions; write any necessary synchronous code outside of any callback functions. This way, the code will be executed before the event loop starts.
Avoid complex calculations such as nested loops in Node.js, as they can cause the application to become blocked.
Be cautious when dealing with large JSON objects, as parsing or stringifying them can become time-consuming.
Avoid using complex regular expressions, as they require a lot of processing power that can be resource-intensive and slow down the event loop. If we use a complex regular expression, it can take significant time to execute. That’s why it is also advisable to break down large regular expressions into smaller, simpler ones that can be executed more efficiently. This will ensure that the event loop continues to run smoothly and that the application remains responsive.
Do not perform any CPU-intensive tasks within the event loop. It can cause the application to become blocked. Here are some CPU-intensive tasks example-
Heavy computation, such as mathematical calculations and scientific simulations
Image and video processing, such as resizing, cropping, and filtering images and videos
Cryptographic operations, such as encryption and decryption of data
Data compression and decompression
It’s important to be careful if we want to do these tasks in Node.js. We will have to take extra steps to minimize their impact on the event loop. This can be done by manually offloading these tasks to a separate process or thread or using an external library optimized for performing these types of operations.

In addition, it’s important to structure the application’s code in a way that doesn’t put pressure on the event loop, such as by avoiding complex algorithms, minimizing blocking operations, and using asynchronous programming techniques wherever possible. This will help to ensure that our Node.js application remains fast, efficient, and responsive even under heavy load.

That’s it for today!

Congratulations on finishing this article! You now have a deeper understanding of Node.js. Whether a beginner or an experienced developer, this information is valuable in helping you build efficient, scalable, and high-performance applications.

If you enjoyed reading this, you can connect with me on Twitter or check out my other articles. Additionally, you can subscribe to my non-tech newsletter, where I share my thoughts on life, productivity, and more, as well as the best content I’ve come across each month.

Happy Learning!

How the Web Works: Understanding the Architecture of the Web

Faisal — Wed, 15 Feb 2023 03:18:08 +0000

It’s easy to take for granted the intricacies of the web as we seamlessly browse through various websites daily. We enter a URL into our browser, and within seconds, a whole website appears. But have you ever stopped to wonder about the technology and processes that make this possible? This article delves into the various components and technologies that make the seemingly effortless experience of accessing the web a reality.

Also, You don’t need to know how the web works to write code. But it will help you to understand the whole system.

What happens when we try to access a website

We know our website’s code is not stored on our computer. We need to get the code into our computer or browser from where it is kept to visit the website. How does this happen?

The process is quite simple. When we enter a URL into our browser, the browser sends a request for the necessary data to the server. The server then responds by sending back the code and data for the website. The browser interprets the codes and displays the website for us to view. This process is known as the request-response model or client-server architecture.

Now let’s look at what the **client **and **server **are.

**Clients **are those devices connected to the internet, like our phones or computers connected to the mobile network or wi-fi. The client is where all the user interaction happens. In the web context, a client is typically a web-accessing software like a browser, such as Chrome, Firefox, or Safari, that requests web pages and other resources from a server. The client (web browser) receives the code for the website and then renders it for the user to view.

Though we access the website from the browser, we can treat the whole device as a client of the client-server architecture.

On the other end of the spectrum are Servers, specialized computers designed to store and manage data, websites, and web applications. These servers are called such because they serve up code or data in response to requests made by clients. A server waits for requests to come in from clients, processes the request, and then sends back the requested information. There are various types of servers, such as web servers, file servers, and database servers, each with its specific function. In this article, we will primarily focus on web servers.

What is URL, and how does it get resolved

A URL (Uniform Resource Locator) is a string of characters that specifies the location of a resource on the internet. URLs are used to identify and locate web pages, images, videos, and other resources on the World Wide Web. URLs typically contain several parts, including a protocol, domain name, and path.

The protocol is the method used to transfer data over the internet. The most common protocol used for the web is HTTP (Hypertext Transfer Protocol) or HTTPS (HTTP Secure), a more secure version of HTTP that encrypts data for added security.
The domain name is the unique name that identifies a website or web server on the internet. To access a website, the name is typed into the browser’s address bar. For example, “github.com” is the domain name of Github’s website.
The path indicates the location of the resource within the server. This URL part usually includes the file name or folder containing the resource, such as “about” or “contact.” For example, “github.com/about” is the path for the about page of the website “github.com.”
Optionally, the URL may also include a query string, a set of characters added to the end of the URL that contains additional information or parameters for the resource.

It’s interesting to note that the domain name we enter into the browser is not the actual physical address of the website. Every website and device connected to the internet has a unique IP address, a numerical label that is often difficult to remember. To make it easier to remember, we use domain names. When an URL is entered, the browser first sends a request to the DNS (Domain Name System) server, which matches the domain name to the corresponding IP address of the website. This is called DNS Lookup. This process is facilitated by your internet service provider and returns the IP address, including the port number of the server being accessed.

Establishing protocols

When we enter a web address and get the actual IP address of the website in the browser, a connection known as a TCP/IP socket is established between the browser and the server. This connection remains active while files are being transferred from the server to the client.

TCP (Transmission Control Protocol) and IP (Internet Protocol) are two of the main protocols that make up the internet protocol (IP) suite. Together, they provide the foundation for communication on the internet.

TCP (Transmission Control Protocol) is one of the core protocols of the internet protocol suite (TCP/IP), which is used to establish and maintain connections between devices on a network. The primary function of TCP is to ensure the reliable delivery of data between devices.

TCP breaks down data into small chunks called packets before they are sent over the network. Each packet is labelled with a TCP header, which includes the source and destination port numbers, and an IP header which consists of the source and destination IP addresses, to identify it. Each packet also contains a sequence number that allows the receiving device to reassemble the packets in the correct order. If a packet is lost or corrupted during transmission, TCP will automatically retransmit the packet to ensure that all packets are received correctly.

This protocol also provides flow control and congestion control. Flow control ensures that the sender does not overwhelm the receiver by sending too much data at once, and congestion control ensures that the network does not become overloaded by too much traffic.

It is a connection-oriented protocol that establishes a virtual connection between the sender and receiver before exchanging any data. Once a connection is established, both parties can exchange data in a reliable and orderly manner.

TCP is widely used in many applications, including web browsing, email, file transfer, and online gaming. It is a reliable and efficient protocol that ensures data is delivered correctly and in the right order, making it an essential part of the internet infrastructure.

On the other side, IP addresses and routes packets between devices on a network. It assigns a unique IP address to each device connected to the internet. When data is sent from one device to another, the IP protocol determines the destination address and the best route for the data to take. Its job is to send and route all the packets through the internet.

TCP and IP work together to provide reliable, efficient communication on the internet. IP addresses and routes data packets, while TCP ensures that the data is delivered correctly and in the proper order. These are communication protocols or the internet’s fundamentals control system that defines and sets the rules for how data travels across the web. Communication Protocol sets the rules for how two or more parties communicate within them.

Once the connection is established, the communication process begins with an HTTP request sent from the browser. HTTP, or Hypertext Transfer Protocol, is the standard protocol for transmitting data over the internet and is the backbone of the World Wide Web. It enables the browser to send requests to a web server and receive the server’s response.

HTTP is based on a request-response model we discussed before, in which a client (such as a web browser) sends a request to a server, and the server sends a response.

HTTP Request has a few parts, which include the start line(HTTP method + request target + HTTP version), HTTP request header, and request Body.

The most common HTTP methods are GET, which requests a resource from the server, and POST, which sends data to the server to be processed. There are also PUT and Patch methods that are used to modify data.

HTTP sits on top of the TCP/IP protocol stack. It’s usually used with the SSL/TLS (Secure Sockets Layer/Transport Layer Security) protocols to provide secure and encrypted communication over the internet.

When the server receives the request, it processes it and sends back an HTTP Response. The HTTP Response is composed of several parts:

The start line includes the HTTP version, status code, and message. The HTTP version indicates which version of the HTTP protocol is being used. The status code is a three-digit numerical code that indicates the outcome of the request. The status message is a brief text description of the status code.
The HTTP response header contains additional information about the response, such as the type of content in the response body, the date and time the response was sent, and the server’s name.
The response body contains the actual data or HTML file that was requested.

The backend developer is responsible for specifying the data included in the response header. This data can include things like the content type, date and time, and server name. In the response body, we get the data or HTML file requested in the original request. If the server cannot locate the requested page, it will send an HTTP 404 error message indicating that the page could not be found.

What happens next after our browser receives the response?

When a web browser receives an HTML (Hypertext Markup Language) document from a web server, it parses the document to convert it into a visual representation that can be displayed to the user. The process of parsing an HTML document involves the following steps:

The browser begins by reading the HTML code from top to bottom, line by line.
As it reads through the code, the browser looks for tags, which are used to define the structure and layout of the document. For example, the tag indicates the start of an HTML document, and the
and tags indicate the beginning of the head and body sections of the document, respectively. The browser understands how to interpret the code by an HTML parsing algorithm.
As the browser encounters tags, it creates corresponding elements in the Document Object Model (DOM), a tree-like representation of the HTML document. The browser uses it to understand how the page should be rendered and displayed. Each element in the DOM corresponds to a single tag in the HTML code, and we can manipulate those elements by scripting language like JavaScript.

The browser scans the document for additional assets to render the page correctly. These assets can include CSS files, JavaScript files, images, and other multimedia files. When the browser encounters a reference to an asset in the HTML document, it will send an additional request to the server to retrieve that asset. The browser will repeat this process for each asset that it encounters. For example, if the HTML document links.
For a CSS file, the browser will request the server to retrieve that file. The CSS file will then be parsed and used to apply the visual styles to the elements defined in the HTML. Similarly, if there are images on the page, the browser will send a request to the server for each image and render them on the page. It’s important to note that the browser will only download and parse the assets necessary for the current viewport or the visible part of the webpage. This is called Lazy Loading. As the user scrolls down the page, the browser will download and parse additional assets as they are needed. This downloading and parsing of additional assets can impact the web page’s performance, as it can take time to retrieve and process these files. Therefore, developers need to optimize the loading of these assets to ensure that the page loads quickly and efficiently.
It’s important to understand that HTML doesn’t provide information on how a website should appear visually. It only defines the structure and tells the browser the different parts of the content, like headings, images, and paragraphs.
CSS (Cascading Style Sheets) provides visual styling and presentation information for a web page. CSS allows developers to specify the font, colour, size, and position of elements on a web page, as well as other visual properties. CSS can be linked to an HTML document and will be used to apply the styles to the elements defined in the HTML.
The browser also uses JavaScript to add dynamic functionality to web pages, like form validation and interactive elements.
Once the parsing process is complete, the browser renders the visual representation of the HTML document, which the user can interact with and view in the browser window.

The parsing process is complex, but it is done quickly and efficiently by modern web browsers, allowing users to interact seamlessly with web pages. After parsing everything, the connection gets closed.

This is it for now! However, many other complex processes and technologies make the internet function. This article should give you an overview of the essential parts of the web, but keep in mind that there is much more to learn and explore in the field of the web.

If you enjoyed reading this, feel free to connect with me on Twitter or check out my other articles. Additionally, you can subscribe to my non-tech newsletter, where I share my thoughts on life, productivity, and more, as well as the best content I’ve come across each month.

Happy Learning.