Forem: TK

The Evolution of React Rendering Architectures

TK — Mon, 03 Jul 2023 11:00:00 +0000

This post originally published at iamtk.co

In this post, we'll see how different rendering architectures work. We'll talk about Client-Side Rendering (CSR), Server-Side Rendering (SSR), and Streaming SSR in React and how they impact performance on the web.

First of all, how do we measure performance on the web? The current standard is the core web vitals (take a look at the web performance research for more info). These metrics are:

LCP: Largest Contentful Paint — measures the time it takes to render the main content
FID: First Input Delay — measures the latency of the first interaction a user has made with the page
CLS: Cumulative Layout Shift — measures the stability of the page

But we'll also talk about other complementary performance metrics in this article. They are:

TTFB: Time To First Byte — measures the connection setup time and server responsiveness
FCP: First Contentful Paint — measures the time it takes to render the first content
INP: Interaction to Next Paint — measures the latency of all interactions a user has made with the page
TTI: Time To Interactive - measures the time at which a page becomes interactive (events wired up, etc).

With that context in mind, we dive deep into the rendering architectures and how each impacts the performance of a web app.

CSR — Client Side Rendering

As the name suggests, CSR renders the content on the client. It means JavaScript will handle how the page is rendered, but how does it work?

A common approach, perhaps popularized by create-react-app, is the “single-page application”. It has a very thin layer of HTML, a file that has only the head, a body with a div (where the content is going to be placed), and the script tag, which will be responsible to make the browser download, parse, and execute the JavaScript bundle. While running JavaScript, React.js can render all related DOM nodes on the browser.

This is what it usually looks like:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Undefined</title>
  </head>
  <body>
    <div id="root"></div>
    <script src="/app.js" type="text/javascript"></script>
  </body>
</html>

This is the backbone of a Client-Side Rendering architecture. A small HTML file with the basic stuff and ready to make the browser uses JavaScript to render the actual content.

This is how it works under the hood:

These are the steps of a CSR architecture:

The client (browser) sends a request
Connection happens
A minimal HTML/CSS is rendered
Scripts tags are executed: JavaScript files downloaded, parsed, and executed
Hydration: making the website interactive
Fetching data: it can be requested to any API — user API to get user data, content API (e.g. itinerary data if it's a hotel app, products data if it's an e-commerce kind of app)
Rendering happens and the content is rendered on the page (LCP will be measured only here, probably)

Here we can see Client-Side Rendering has issues in some of these steps and because of them, it can translate into bad performance.

The most problematic part is the need for browsers to download, parse, and execute the generated bundle and only then be able to fetch the data and finally render the content on the page.

The browser on the client handles the execution of JavaScript
The browser on the client handles data fetching
Rendering on the client happens only after JavaScript is executed and data fetching is done

The time it takes to handle this whole process is relied on the user's device. If it's more powerful and the internet connection is good, browsers can handle it well with a certain limit. It can be a huge performance bottleneck if the device is less powerful or the connection is not good (3g or worse).

It's challenging to optimize LCP on a CSR application as it needs to have all these steps one after another to finally render the LCP element.

The time between the start of JavaScript download to rendering the LCP can be huge, in some cases, it can even reach 16s on mobile 4g. Even if you optimize the application with code-splitting and browser caching, it's challenging to get better Core Web Vitals (CWV). The problem increases the bigger your final bundle is. The more your application's bundle size increases, the more you hurt the final users’ experience.

“CSR cannot get good performance as measured by Core Web Vitals (CWV). SSR / SSG can, but it's not a guarantee. If good performance is important in your use-case then you will need to move beyond CSR.” — Dan Shappir

Some of the benefits of Client-Side Rendering is that it doesn't really need a server to handle the application. You can easily setup the app and serve it on a CDN. Depending on how the app is structured and implemented, it has soft navigations which provide a better UX. But if performance is a bottleneck and is hurting the final user, we should think of a better architecture.

SSR — Server-Side Rendering

Different than Client-Side Rendering, SSR handles rendering on the server, so everything (data fetching, generated HTML, etc) is done on the server and not on the client anymore. This is how it works under the hood:

These are the SSR steps:

On the server
- Fetch data for the entire app: user data, content data
- Render the entire app to HTML and send it in the response
On the client
- Load the JavaScript code for the entire app
- Hydration: Connect the JavaScript logic to the server-generated HTML for the entire app

This part is important: each step has to finish before the next step can start. There’s no concurrency/parallelism here. This means it begins with data fetching, then generate the HTML, renders the HTML on the client, loads the JavaScript bundle, and then hydrates the app. In this order, no parallelism.

The performance gain is in these particular parts:

Data fetching happens on the server. With a more powerful machine, it handles this process faster than when it's done on the client, which relies on the user's device
SSR: generates HTML from react components on the server and sends that HTML to the client
As the page is already generated, the user can see the content before loading the JavaScript bundle.

The server handles data fetching and the generation of static HTML and responds to the client's request pretty fast compared to what's done in Client-Side Rendering.

An interesting piece of info about data fetching is:

Server-side data fetching can be faster than client-side data fetching, particularly for large datasets or complex queries. This is because servers often have more resources available than client devices, such as more processing power, memory, and faster internet connections.
- After the request is sent, we have the TTFB or the time it needs to wait for the server response and the content download (resource)
- TTFB: 1 roundtrip of latency and the time the server took to prepare the response. It relies on the server (in this case, SAPI) to respond to the website
- Content download: the total amount of time reading the response body. It relies on the machine (servers vs. browsers/user devices)

Because of these facts, SSR can hugely improve the performance metrics of your application.

To back it up, we have interesting case studies on the CSR → SSR migration and how it impacted performance:

Optimizing Core Web Vitals on a Next.js app — reported 73% and 51% improvements for FCP and LCP, respectively when migrating the app to Next.js SSR.
Dan Shappir - The Challenges and Pitfalls of Server Side Rendering — reported 60% improvements on Time to visible (it’s similar to Start render and FCP metrics)
Improving The Performance Of Wix Websites (Case Study) — Smashing Magazine — reported an improvement from 17.55s to 2.55s in rendering the complete page. It doesn’t rely only on SSR though.

But, as we saw, SSR also has some performance bottlenecks. It needs to wait for the entire app to fetch the data to be able to render the HTML. And it needs to do a whole DOM hydration. Some would say Hydration is a Pure Overhead and for others SSR was never the problem. The problem was, and still is, the client-side hydration.

This can be a big problem for interactivity metrics like FID (First Input Delay) and INP (Interaction to Next Paint).

What if we could be more granular about which parts of HTML we can flush rather than flush the entire HTML just once? We would flush smaller, granular chunks over time.

Static parts are rendered from the server without the need to wait for the data to be fetched (first flush)
Parts of the UI that are still waiting for the data to be fetched can render a loading indicator (spinner, skeleton)
and when the fetch is done, the server streams again (second flush) the missing (dynamic) parts.

That’s called Streaming SSR and it's what we are going to see next.

Streaming SSR

Streaming is all about parallelizing operations. The idea is to send HTML in chunks and progressively render it as it's received. So, we can parallelize work on the server and send small HTML chunks over time to the client.

With that in mind, we can solve, or at least improve the two major problems of SSR:

Rendering on the server is blocked by data: it needs to wait for data fetching to be finished to respond to the request and render on the client, which offers a slower TTFB
Hydration is a bottleneck: making the website interactive depends on JavaScript, so the browser needs to download, parse, and execute JS before hydrating the entire page, making TTI (and other interactivity metrics like FID and INP) worse

Streaming SSR tackles these two problems. Now that SSR can rely on Streams, we can send granular, smaller chunks of HTML to the client. Before it was blocked by data fetching, now we can render the “static” parts first while the server handles data fetching. So, rendering the first HTML chunk is pretty fast, which improves TTFB and FCP for the app.

Then, when SSR finishes the data fetching, it can stream the second HTML chunk to the client. We now clearly draw the line between static HTML and HTML that relies on data fetching. But we can make it even more granular. If we have parts of the page that relies on different data, we can divide them into N pieces of HTML chunks and stream them back to the client whenever they're ready to be rendered.

We all know hydration is a big performance bottleneck when it comes to SSR and Streaming SSR tries to fix/improve this problem by using something called Selective Hydration.

SSR can stream HTML in chunks over time, so each HTML chunk delivered and rendered on the client can be hydrated right away. It doesn't need to wait for the second and the following chunks to be rendered.

Imagine we took a strategy of dividing the HTML into 3 chunks:

static chunk
dynamic chunk relied on fetching an API (fetch #1)
dynamic chunk relied on another API (fetch #2)

While fetch #1 and fetch #2 are running, the static chunk can be delivered to the client. When the browser receives it, it gets rendered and it starts hydrating this selected part (selective hydration!). Let's say fetch #1 finishes first. It gets rendered and hydrated on the client without the need to wait for fetch #2 to be finished. And then, fetch #2 gets rendered and hydrated after its related data is fetched.

You can see that everything is now parallelized. But keep in mind that it has a caveat here. When each chunk gets to the browser, for it to be rendered and hydrated, the main thread needs to be free to handle the work. If it's busy, there's no way it gets worked on, unless it gets prioritized.

Parallelizing hydration (with Selective Hydration), we get better TTI (and FID and INP) compared to “pure” SSR.

So, in general, with Streaming SSR, we get:

Parallelized server rendering HTML in smaller chunks
Selective hydration for each streamed HTML chunk
Better TTFB and FCP because of the smaller HTML chunk streamed
Better TTI, FID, and INP because of selective hydration of the smaller chunks

In terms of implementation, we use a nice abstraction called “React Suspense”.

Let's think about an example to illustrate how this would work. Imagine an e-commerce.

An e-commerce has a logo, a search bar, an avatar, filters, and a list of products
We can organize the components into “static” and “dynamic”
- Dynamic components are components that rely on data fetching like the products list and the avatar
- And the Static components are the ones that don't rely on data fetching and can just render the HTML right away, like all components besides the list of products and the avatar

This is how it would look like in a “pure” SSR (or even a CSR) implementation:

<Layout>
  <Header>
    <Logo />
    <SearchBar />
    <Avatar />
  <Header>
  <Filters />
  <Products />
</Layout>

It's an “all or nothing” approach. It needs to fetch the User API for the data to the <Avatar /> component and the Products API for the data to the <Products /> component before rendering and hydrating everything.

The Streaming SSR with the Suspense boundary would look like this:

<Layout>
  <Header>
    <Logo />
    <SearchBar />
    <Suspense fallback={<AvatarSkeleton />}>
      <Avatar />
    </Suspense>
  <Header>
  <Filters />
  <Suspense fallback={<ProductsSkeleton />}>
    <Products />
  </Suspense>
</Layout>

While the data fetching for the user and the products are running on the server, everything besides the <Avatar /> and the <Products /> components will be rendered. For <Avatar /> and <Products /> the server will fall back to the placeholder. In this case, they are skeleton loaders.

And then the client can already work on the hydration of the components rendered on the browser while the data fetching is still running on the server. It doesn't need to wait for the data fetching to be finished.

And finally, when the data fetch is done, the products and the avatar are rendered and hydrated on the client.

With this approach, we can now render the static parts of the page running the data requests in parallel. It can also use Selective Hydration to handle the hydration of these static parts. It doesn't need to wait for anything else.

When the data is ready, the server can stream back the HTML and it will be rendered and hydrated on the client.

All this with a simple abstraction called Suspense.

It's important to notice that Suspense is not a feature exclusive to Streaming SSR. Suspense is a mechanism to show a fallback for a certain part of the application while it's waiting for the data. When the data is ready, the fallback UI (spinners, skeletons) will be updated with the actual content. So it can also be used in Client-Side Rendering.

Performance impact on the web

There's a collection of interesting performance case studies, and here I list ones that talk about rendering architectures and how they optimized performance for their app.

Optimizing Core Web Vitals on a Next.js app — reported 73% and 51% improvements for FCP and LCP, respectively when migrating the app to Next.js SSR.
Dan Shappir - The Challenges and Pitfalls of Server Side Rendering — reported 60% improvements on Time to visible (it’s similar to Start render and FCP metrics)
Improving The Performance Of Wix Websites (Case Study) — Smashing Magazine — reported an improvement from 17.55s to 2.55 in rendering the complete page. It doesn’t rely only on SSR though.
How QuintoAndar increased conversion rates and pages per session by improving page performance — 39% improvement on LCP & 39% improvement on TTI

This post originally published at iamtk.co

Resources

Rendering Architecture

React

Web Performance

Frontend System Design — Spotify Playlist Architecture

TK — Sun, 05 Mar 2023 13:14:05 +0000

The original post was published at iamtk.co.

This post is part of the series Crafting Frontend, the Frontend Architecture version.

These are my notes from a frontend architecture interview I had in the past. Here we will talk about the challenge (a Spotify Playlist), and think together about the whole architecture: API request/payload, Components, State Management, and Performance Optimizations we can do.

Let's do it!

Requirements & Clarifications

The challenge is the following: we should design a music playlist, similar to Spotify. It needs to list a list of songs, play a song, go to the next one, go to the previous one, and stop the song.

It starts very generically and we need to build the foundation by asking clarifying questions.

These are good starting points:

How many songs per playlist are possible to store?
- if infinite, you can start thinking about pagination (API architecture), image optimization, and list virtualization (web performance)
- If at most 10/20, maybe it's not necessary to think about all that listed above
Is it possible to add or remove songs from the playlist? Should I care about it for this current design?
- This an interesting question, because if the answer is “yes”, we should be careful with how we structure our state data structure. e.g. if removing a song, the previous song's should “point” to the next song when the user clicks “next”. The same for the “previous”.
- Also, we should probably design a new API endpoint to handle the songs list update: adding or removing a song from a playlist doesn't only update on the frontend, but also in the database
Do I need to handle authentication? How will auth for this application be?
- It can open the scope to handle unlogged users: redirect page if you are not the owner of the playlist (maybe think about 401 — unauthorized), only able to see “public” playlists
- It can have a simple auth token and you just need to send it to the backend through API requests, so you probably don't need to focus on it that much
Should we think about responsive design (desktop and mobile)?
- In this case, it doesn't matter that much if we focus on one or the other because the UI and how we handle user behavior will be pretty much the same for both.
- But this question is interesting because mobile devices are less powerful, and you can focus on the performance optimizations for this specific device
- Also, it's always good to show that you care about responsive design (as mobile is eating the world)

In general, think about questions that

will increase or close the scope of the architecture
- e.g. for an increase: adding a new feature, think about mobile and desktop
- e.g. for closing the scope: asking about features and the answer is “no need for this scope”
- e.g. should handle authenticated user
will change how think about the architecture
- e.g. the size of the list
- e.g. focus on mobile too — maybe need to tackle performance problems (should be asked)

From UI to components

This is probably one of the easiest parts of the design. I guess.

My process is to get every little part of the UI and think of it as a UI component or a part of a bigger component. Then I break down the components into a “tree” so I can understand the relationship between components.

Having this tree, it's even better and easier to visualize how we can design the data structure and how we should colocate it / be used by each component.

The first part is pretty obvious. We separate the playlist from the player. So now we know that we would have a Playlist and Player components.

And then we can break down each component into smaller components that combined will build up the whole UI component.

The <Playlist /> first: we can separate it into 4 smaller components. <AlbumPhoto />, <Title />, <Artist />, and <Album />.

The album photo is just the image rendered for the song in the list and the title, artist, and album are strings or information about the song.

The <Player /> is pretty similar: also 4 components. <PreviousButton />, <PlayButton />, <PauseButton />, and <NextButton />.

All 4 components are just buttons and will respond to user interactions.

Of course, we can break them down in different ways but this is how I first thought about it, and to be honest, I feel pretty good about it.

API Requests & Payload Interface

Now we should design how the client will interact with the server and the type interface of the data coming from the backend.

When designing this, the first thought that came to my mind is the idea of the client just doing a simple API request to the backend and the server responding a JSON data (that we can define its type interface later on).

One question you could ask is if we need to handle authentication. When I asked this question, the interviewer just answered that I could think I have a auth token and this is what I need to request the endpoint and get my data as a user.

A simple API endpoint would be something like this: /api/playlist/:id.

That way, you can just request this URL, pass the playlist id, and the auth token. It would return the playlist or a list of songs.

The :id could be just an integer number like /api/playlist/123 or a “UUID”/”ULID” like /api/playlist/01ARZ3NDEKTSV4RRFFQ69G5FAV. But I guess this is just an implementation detail.

To start the playlist interface, we first think about the song data we need to render all the necessary information in the UI. It would look like this:

interface Song {
  id: string;
  title: string;
  artist: string;
  album: string;
  albumPhoto: string;
  songUrl: string;
}

And the playlist interface is just a list of songs:

type Playlist = Song[];

The playlist can have a huge list of songs, so it's better if we paginate it, right? The data interface for pagination is simple as we commonly use a “pattern” for it:

interface Data {
  playlist: Playlist;
  page: number;
  perPage: number;
  pageCount: number;
  totalCount: number;
  previous: string | null;
  next: string | null;
}

page: the current page
perPage: how many songs per page
pageCount: the number of pages
totalCount: the number of songs in total
previous: the URL for the previous page (it can be null if the current page is the first one)
next: the URL for the next page (it can be null if the current page is the last one)

Other two interesting ideas to discuss here are how we would handle the loading experience and the error handling.

Every time we request data from an API, we need to show a loader for the user while we are fetching the data. And if the request fails, we should handle this error.

The loader can be a loading spinner or a skeleton for example.

And an example of error handling is showing a text for different status codes

404: couldn't find the playlist
401: you're not authorized to see this playlist. Re-authenticate and try again
403: you're not allowed to see the playlist
500: our server encountered a problem and we're fixing it

For each error code, we can show a different message to the user.

This is how we can think about the API request, the data type interface, and how we handle the loading and the error experience for the user.

State Management: handling client data

Let's recap what we should handle to enable the features for the users:

It should play the current song
It should be able to go to the next song
It should be able to go to the previous song
It should be able to highlight the current song in the playlist

We need a way to hold the data for the list and the current song. We can do it in a normalized way using objects or using arrays that can be more performant.

Just to recap, we have the song and the playlist interfaces:

interface Song {
  id: string;
  title: string;
  artist: string;
  album: string;
  albumPhoto: string;
  songUrl: string;
}

type Playlist = Song[];

Let's see how the normalized way would work first.

type PlaylistById = Record<string, Song>;

interface State {
  currentSongId: number;
  playlist: {
    byId: PlaylistById;
    allSongIds: number[];
  };
}

Here we have the state interface. We have the currentSongId to hold the, well, the current song id. And the playlist. The playlist will contain the list of songs by id and all the song ids.

With the currentSongId, we can contemplate one of the features: getting the current song (to play and highlight it on the list).

But we also need to contemplate the other features as well: get the next and the previous songs based on the current song.

This is why we have the allSongIds in the playlist object. With that, we maintain the ids in an ordered way so we know what's the previous and the next ids. The problem is: every time we need to access the previous or the next id, we need to traverse this list, so we would have the time complexity of O(N), N being the number of songs in the list.

Another approach is to use arrays to handle that. At first, you'll think that it's counterintuitive to use arrays over hashmaps to improve performance, right? But in this case, we can use “almost” like hashmaps and keep the O(1) time complexity. Let's see how it would work.

interface State {
  currentSongIndex: number;
  playlist: Playlist;
}

So now, instead of the currentSongId, we have the currentSongIndex. The current index refers to the index in the playlist list and not the id anymore. With that, we can access the list with the index in O(1) time complexity.

To get the previous song index, we just need to access it by using the currentSongIndex - 1 and the next one by using currentSongIndex + 1. And access the previous and next songs using the new index.

This is how we keep the time complexity of O(1).

So, from one side, we have more normalized data and on the other, we have a more performant approach. We just need to keep in mind that the main feature is playing the current song, and the user will sporadically click “next” or “previous”. Another important piece of information is the average number of songs in the playlist. Let's suppose it's 20 per playlist. So it would not be a big deal.

Performance Optimizations

When thinking about performance optimizations, there is a whole new world of possibilities (with research). We can discuss loading time, rendering time, images, audio optimizations, and so on.

So let's discuss them by topics.

When thinking about loading time, part of the “performance optimization” we deal with while working on the loading experience with skeletons. It's called perceived performance, or the users' notion they are having a faster application even though it's just an intermediate state.

But we can also think about code splitting the application to deliver the bundle the user really needs.

And maybe one of the most impactful decisions is the rendering architecture. Should it be an SPA, SSR, ISP, etc?

A Single Page Application gives a lot of flexibility and improves interactivity for the user, but with that comes a lot of issues. We can talk about render-blocking scripts, all the JavaScript has to be downloaded before it can be run, and because of that, it increases, in ms/seconds, the FCP and LCP metrics. Also, it could also have CLS issues as SPAs are kinda unfairly penalized

For Server Side Rendering apps, if the LCP element (text or image) is attached to the HTML, it will have a big impact on the FCP and LCP scores as it doesn't need to download and execute the whole JavaScript (that can be a huge bundle), the HTML is generated in the server and returned to the browser, so it gets rendered very fast.

One of the SSR's bottlenecks is the cost of hydration. Some would state that hydration is pure overhead and we should think about different ways of using browsers to render pages in an optimal way: Progressive Hydration, Islands Architecture, and React Server Components come to mind.

There are a lot of discussions around this, new frameworks were created, and a lot of buzz on different types of rendering architectures.

Talking about rendering, what we definitely can think about is virtualizing the list of songs. With List virtualization, only a small subset of the list will be rendered in the DOM. When the user scrolls down, the list virtualization will remove and recycle the first items and replace them with newer and subsequent items in the list.

Using this technique can hugely impact user experience, not only the loading time but mostly user interaction metrics like First Input Delay (FID) and Interaction to Next Paint (INP). Having a faster rendering mechanism helps reduce the work of the browser's main thread and the browser being less busy, responds to the user more quickly.

Another technique that could help render data in the UI faster is the usage of the stale-while-revalidate cache. The simpler definition of this mechanism is: we load the cached content right away while updating the cached content to make it fresh for the future. It only updates the cache if it's stale, this is why “stale” (get the cached content that's “stale”) while “revalidate” (update the stale cache to make it fresh).

We can use it in our API request when getting the playlist data. So if the user goes to another page and comes back to the playlist, it will get the content from the cache (and maybe revalidate it if it's stale), making the experience so much faster and smoother.

Another topic we can think about when it comes to performance and making a better user experience is “Image Optimization”.

The first tiny idea is to opt for SVGs over an image (using the image tag) on the “play”, “stop”, “next”, and “previous” buttons. Image requests don't block the main thread but it's one more network request the application needs to handle. In this case, it would be 4 “unnecessary” network requests if you can replace them with SVGs.

Another tiny image performance pattern is to make sure your images are hosted in a CDN so we can get them faster and also from the cache. Also, prefer self-hosting your images:

A less-optimised image served from the same origin is always faster than a more-optimised image from a third party. Self-host your static assets! — Self-Host Your Static Assets

We can also generate different image formats like WebP and AviF for better performance.

The size of images is also important. We need to make sure we won't load a 1mb image for a mobile phone (or even for desktop), or in the words, make our images responsive. To do that, we can use the help of srcset. We provide a bunch of different sizes so the browser can pick the right one. If we don't use an image tool like Cloudinary, we probably need to host the images and generate all possible sizes we expect that can be rendered on the page.

Another image perf technique to improve perceived performance is to request a low-quality image with blur, download a high-quality image in the background and smoothly transition from the low-quality to the high-quality image with the ease-in/ease-out transition.

To do image caching, we can rely on browser caching or setup a service worker to handle the image cache if we need more fine-grained control over the network request. Service workers are interesting because they also provide other benefits like offline experience and more memory and storage space for our origin

Another type of file we can optimize in this particular case is audio files. One way of doing this is to store them in the “file system” if it has access, or in a “client database” like IndexedDB. The audio files would be stored in the browser and if the user has the possibility to download them, we need to think about security and probably implement file encryption which I think starts to get more and more complex for a simple file cache.

Another approach is to use service workers the same way we did for images so we don't need to handle what is stored in the browser.

Final words

Designing new frontend architectures is always a challenge and we should take everything into consideration to choose the best trade-offs.

Clarifying questions is super important because you need to understand the user/system requirements, what you should focus on first, and what you can postpone to a "next version".

Then some patterns start to emerge like building the component tree from the UI mockup, designing the data structures to handle the client state, thinking about the API request and the required payload, and performance optimizations we can do to improve the application.

I hope you could learn about some ideas from this post and maybe use them in your next architectural design or even in future interviews.

Crafting Frontend is a series of posts and experiments I'm doing to craft the art of frontend engineering. To see all the experiments I've been doing, follow the Crafting Frontend github repo.

Peace! ✨

Optimizing the performance of a React App for a better User Experience — A Case Study

TK — Thu, 22 Sep 2022 21:37:57 +0000

This post was originally published at iamtk.com

A couple of weeks ago I was developing a new authentication feature for the FindHotel website and after finishing the implementation, I started testing the changes.

One thing that I noticed was how laggy the scrolling experience was and how much I should wait until I was able to click elements of the page and receive feedback. The page's interactivity was slow and jank. So I started investigating this problem. And I knew it was a "performance" problem.

To have a good comprehension of this problem, let's start with how the browsers work.

Browsers & the Rendering Process

It won't be a complete, exhaustive description of how browsers work but my thought is to give an idea about the rendering process and talk a bit about some moving parts of this process.

The first two steps are about building the DOM and the CSSOM. The HTML document is downloaded from the server and parsed into the DOM (Document Object Model).

And the CSS sources are also downloaded and parsed into the CSSOM (CSS Object Model).

Together, they are combined into a rendering tree. This tree contains only the nodes required to render the page.

The Layout is a big part of this process. It computes the size and position of each object. Layout is also called Reflow in some browsers.

And finally, the render tree is ready to be “painted". It renders the pixels to the screen and the result is displayed in the browser.

Gecko, the engine used by Mozilla Firefox has a pretty interesting video about how Reflow works

Reflows are extremely computationally expensive in terms of performance and can cause render speeds to slow down significantly. This is why DOM manipulation (inserting, deleting, updating the DOM) has a high cost as it causes the browser to reflow.

A quick crash course on profiling with DevTools

From this foundation, let's now start going in-depth into the problem. In my experience, I could see the UI was sluggish and the pixels were slow to render and there was a white block (when the pixels were not rendered yet) when scrolling the page.

It could be a variety set of reasons the page had this problem. But without measuring the performance, I couldn't really know. Let's start profiling the page.

With the DevTools, I was able to have an overview of the UI using the Performance Monitor.

Here I have access to a set of different attributes:

The CPU usage
JS heap size
DOM nodes
JS event listeners

And other attributes I didn't add to the performance monitor.

Interacting with the page and seeing these attributes change was the first thing I did to have a better understanding of what was happening there.

The DOM nodes were big but the attribute that caught my attention was the CPU usage. When scrolling the page down and up, the % of the CPU usage was always 100% or around it. Now I needed to understand why the CPU was always busy.

The Chrome devtools is a nice tool to investigate possible performance issues.

We can use CPU throttling to simulate how a user in a slower device would experience the website. In the video, I set this up to slowdown 4x.

The Performance tab is powerful and overwhelming at the same time because it has a lot of functionality and it's easy to lose yourself in so much information in just one tab.

For now, we will focus on two things: the frames and the Main Thread.

In the video, you can see I was hovering the frames and it shows the actions and interactions that were recorded. In the Main Thread, you can see the flame graph, a bunch of tasks that were executed. With both information, we can match the frames with the tasks that took more time to be computed.

With that in mind, a new concept comes up: Long Tasks.

A Long Task is “any uninterrupted period where the main UI thread is busy for 50 ms or longer". As we all know, JavaScript is single threaded and every time the main thread is busy, we are blocking any user interactions leading to a very bad user experience.

We can go deep into each Long Task and see all related tasks it’s computing. With the flame chart, it's easier to understand parts of the application that are causing the long tasks or in many cases, what are the group of actions and components that are causing the performance issue.

Another interesting piece of information from the DevTools is the Bottom-Up tab.

We can click the Long Task and see a bunch of information about the activities in the task. It's a bit different from the flame chart because you can sort the activities based on the cost (Total Time) and also filter the activities related to your application code because it shows other information like cost of garbage collector, time to compile code and evaluate the script, and so on.

It's powerful to investigate the tasks using the flame chart together with the information in the Bottom-Up tab. If you are using React, it's easier to find things like performSyncWorkOnRoot and performUnitOfWork, that's probably related to how React renders and re-renders its components. If you are using Redux, you'll probably see things like dispatch and it requesting React to update some components.

Profiling the Search page with the DevTools Performance tab

Profiling the page's performance I was working on, I found the long tasks and tried to match these tasks with the frames (components that were rendering/re-rendering).

In the Main Thread, I could see a lot of Long Tasks. Not only the number of Long Tasks was a problem, but also the cost of each task. An important part of the whole process of profiling is to find the portion of your codebase in the long task flame chart. At first sight, I could see things like onOffersReceived, onComplete, and onHotelReceived which are callback actions after fetching data through an API.

I dug deeper into one of the Long Tasks and I could see Redux dispatching an action and React "performing units of work" (aka re-rendering the components that use the Redux state).

The whole workflow of this page looks like this:

The page performs an API request: requesting offers, hotels, and other data
With the API response, we dispatch a Redux action to update the state in the store
All components that use this updated state will be re-rendered

Dispatching actions and re-rendering components cause the most expensive Long Tasks.

And it's not a single component we're talking about. As this page has a list of hotels and offers, whenever the state changes, A LOT of components will be re-rendered. In the flame chart is possible to see the most expensive components when it comes to re-rendering.

List virtualization: runtime optimizations

Now that we know that reflow, repaint, and DOM manipulation like adding and updating elements in the DOM have a meaningful cost, some solutions to this problem come to mind: reduce the components re-rendering and reduce the elements in the DOM.

One common pattern for lists is to use List Virtualization.

With List virtualization, only a small subset of the list will be rendered in the DOM. When the user scrolls down, the list virtualization will remove and recycle the first items and replace them with newer and subsequent items in the list.

In the video, you can see the DOM being updated. It renders only 3 elements in the DOM and when the user starts scrolling, it will recycle and replace the first items with the subsequent items in the list. We can see the data-index changing. It starts with a 0 index and renders only 0, 1, and 2 at first. Then 0 becomes 1, 1 becomes 2, and finally, 3 is the new element rendered in the DOM.

We have a set of nice windowing libraries that do this job very well for us. I considered react-window, but in the end, I chose react-virtuoso to test and analyze the results.

Recording both the current approach we use to render the list and the solution with list virtualization, I could see improvements in the latter.

localhost #2 — current search page without list virtualization: more long tasks, long tasks costing more, more CPU usage, and laggy scrolling experience

localhost #1 — with list virtualization: fewer long tasks, long tasks costing less, less CPU usage, and a more smooth scrolling experience

Examples of long tasks on this page are the onOffersReceived and onComplete. They cost around 400ms and 300ms respectively in the current version. With list virtualization, the cost decreased to around 70ms and 120ms respectively.

In summary, with list virtualization, we rendered fewer elements in the DOM, less components are re-rendered, this reduces the number of long tasks and the cost of each task, which provides a silky smooth rendering experience.

Other benefits include faster user interactivity (Time To Interactive), fewer images downloaded in the first render, and scalability when it comes to the list pagination.

Faster user interactivity: fewer long tasks / lower long task costs make the Main UI Thread less busy and faster to respond to user interactions.
Fewer images downloaded: as we only render 3 elements in the dom per "window", we also only need to download 3 images per window (each image for each hotel card).
List pagination scalability: when the user starts clicking "load more" items, the current approach appends more items to the DOM and the Main Thread needs to handle more elements, increasing the cost of components updates and re-renders.

One thing to mention and consider in this analysis is that we are downloading a new library in the final bundle. This is the cost of the download time of react-virtuoso: 347ms (slow 3g) / 20ms (emerging 4g).

Improvements and Results

After 3 weeks running this performance optimizations as an A/B test experiment, I came across interesting results.

The best two results are the conversion and engagement metrics.

All users that used the search page and booked a hotel offer is considered a conversion for us. This metric increased by 2.4% comparing the users who used the list virtualization and the users who didn't.
Our engagement metric measures different ways our users interact with the page, and we consider it the best proxy to predict the likelihood of future bookings. This metric improved by 2.2%.

I also wanted to understand which performance metrics drove these conversion improvements. Before launching the A/B test, my assumption was that the loading performance like FCP and LCP wouldn't change and user interactivity metrics like FID, TTI, and TBT would be improved.

It's pretty much what happened. FCP and LCP stayed still, no much difference. And the user interactivity metrics improved.

If we see the FID for the p75 before the A/B test, it was around 140ms. With the A/B test, it went down to 50ms. And now that we went 100% with the list virtualization solution, it went down, even more, to 25ms-30ms.

Digging more deep into the data, I wanted to understand how the device type influenced FID. If you think about, as mobile devices are less powerful than desktop and tablet devices, this makes them more "sensitive" to any performance improvement. Let's see this in practice.

The p75 for desktop (the above picture in the left) was around 30ms and went down to 12ms-17ms. The p75 for mobile (the above picture in the right) was around 200ms and went down to 30ms.

Mobile devices were more sensitive to this improvement, it reduced by 85%. Huge gain and now it's running within the threshold (a good FID score is between 0ms and 100ms for p75).

The p95 also followed the same pattern.

The desktop is in the left and the right is the mobile.

As we all know that performance is not a single number/score, it's a range of experiences, it becomes easier to read performance data showing a range of scores. A great way to read performance metrics like that are histograms.

On the left, we can see the FID scores from 0s to 1s. Most of the scores are within the p75, that's great.

On the right, we can see a more focus vision of the same data but from 0ms to 100ms, the threshold for a good FID score. The p75 is 48ms, that's great too.

But if you look close to the 0s-1s histogram, we can still see people experiencing bad interactivity as the FID score go up to 1s. Datadog still doesn't allow us to group by network connection (3g, 4g, etc) so I grouped by country and continent as a proxy to understand how network connection influences the interactivity metric.

I think it's still not the best way to understand how it influences this metric but I found this info from the HTTP Archive's Almanac:

Regions in parts of Asia and Europe continued to have higher performance. This may be due to higher network speeds, wealthier populations with faster devices, and closer edge-caching locations. — link

In the "FID geomap", I see the poor experiences are from South America and more especifically from Africa. And in the FID p75 grouped by continent and device type, all continents had the same pattern. But I could see Africa on mobile was more sensitive to this improvement. It went from 450ms to only 35ms. The others had the same pattern but less sensitve. Still significant improvements though.

I am still trying to figure out why some people are experiencing this FID score. My guess is still "poor network connection" and less powerful devices as the main factor for bad FID scores.

As people still have these bad experiences, there're more things to optimize.

Future optimizations and experiments

This post is just the first optimization I did to improve the performance and thus the user experience. There are other experiments and investigations I want to do and share in the next blog posts.

Things on my To Do list:

Investigate the most expensive components and hooks: re-architect, memorize, reduce the re-renders
Getting only attributes from object references to reduce re-renders
Code split parts of the page like login dialog, banners, and footer to reduce the size of JavaScript in the final bundle: less JS, less cost for the main thread
Measure runtime performance with automated tests like Playwright
And many other investigations and experiments I want to do

Resources

There are many resources about web performance out there. If you are interested in this topic, you should definitely follow the Web Performance Research repository.

There I'll be updating with new resources, doing more research, and writing my findings.

Writing a Memoization Function from Scratch

TK — Mon, 25 Jul 2022 11:53:05 +0000

This post was originally published at TK's website.

Today we'll learn how to write a memoization function and speed up the performance of our functions.

Let's start with how we would use it. The API of the memoization function should be like this:

const memoizedFn = memoize(fn);

The function receives a function as an input and returns the memoized function.

In the first function call, it's still "cold", so every function call will be executed and then the output value will be cached for the subsequent calls.

In any subsequent function call, the values are cached, so rather than executing the function body, it'll just return the cached value.

Let's go to the implementation first and then do some performance comparison.

The first thing is that the memoize function should receive a function as an input and return the memoized function. Simple steps:

function memoize(fn) {
  return fn;
}

So now we can just call the memoize function passing a new function and it will return the same input function.

const fn = (a, b) => a * b;
const memoizedFn = memoize(fn);

memoizedFn(1, 1); // 1 (first call: cold)
memoizedFn(1, 1); // 1 (not memoized: it'll execute the function again)
memoizedFn(1, 2); // 2 (first call: cold)

You see here that when we call the memoizedFn function for the second time, we expect that it just returns the cached value but as our implementation is not finished, it calls the fn function again for the same input values.

To memoize the function, I will use the Map as a cache object. So every time the function is called, we can access the cache object using the arguments as a key and get the output.

The cache object will look like this:

const cache = new Map();

cache.set('1', 1);
cache.get('1'); // 1

cache.set('2', 2);
cache.get('2', 2);

The key of the cache object will be the arguments of the function and the value will be the output of the function call. So every time we call the memoized function again, we can access the cache object before calling the function.

The algorithm is pretty simple:

get all the arguments and "stringify" them to build the key of the cache object
verify if the key exists in the cache.
- If it does
- return the output value
- if it doesn't
- call the function
- store the output value in the cache
- and return the output value

Now implementing each part:

get the arguments: a simple function we should return, and get the arguments using the spread operator

(...args) => {};

stringify the arguments to build the key of the cache object by using the JSON.stringify API

const key = JSON.stringify(args);

verify if the key exists in the cache. If it does, return the output value

if (cache.has(key)) {
  return cache.get(key);
}

if it doesn't: call the function, store the output value in the cache

const result = fn(...args);
cache.set(key, result);

And this is the final version:

function memoize(fn) {
  const cache = new Map();

  return (...args) => {
    const key = JSON.stringify(args);

    if (cache.has(key)) {
      return cache.get(key);
    }

    const result = fn(...args);
    cache.set(key, result);

    return result;
  };
}

To do the performance comparison and make sure our memoization function works, let's see the example of sum and factorial functions.

The sum is a pretty simple function and doesn't cost much, so we wouldn't see any significant improvements after caching the function calls.

function sum(a, b) {
  return a + b;
}

And now calling it:

const memoizedSum = memoize(sum);

memoizedSum(1, 1); // 2 (not cached)
memoizedSum(1, 1); // 2 (cached)

To do a better comparison and make sure the memoization speed up the function calls, I created two simple helper functions to build the testing case.

getNumbers is a generator of all the numbers we want to test as inputs for the memoization function.

function getNumbers(limit = 1000000) {
  let numbers = [];

  for (let i = 0; i < limit; i++) {
    numbers.push(i);
  }

  return numbers;
}

testSum is a function to test the execution time of a given callback function.

function testSum(label, numbers, callback) {
  console.time(label);
  for (let number of numbers) {
    callback(number, number);
  }
  console.timeEnd(label);
}

So let's test it: Calling the getNumbers, we get an array of 1.000.000 numbers.

const numbers = getNumbers();

Calling the testSum passing the memoized function:

cold call
cached call

testSum('cold', numbers, memoizedSum);
testSum('cached', numbers, memoizedSum);

Running in my machine (MacBook Pro (13-inch, M1, 2020), Chip Apple M1, Memory 16GB), I got this comparison.

------- sum --------
cold: 495.026ms
cached: 371.011ms
------- // --------

The memoized version is 1.33% faster than the pure version.

As I mentioned earlier, the sum function is a simple function therefore its execution doesn't cost that much, so we won't see a lot of performance improvements in the cached version.

But now, let's see the factorial function being compared to its memoized version. As it's a somewhat more complex function, we'll probably see the memoized version sped up.

function factorial(number) {
  if (number < 0) return -1;
  if (number === 0) return 1;
  return number * factorial(number - 1);
}

Without a caching mechanism, we can implement the factorial function using the recursion technique.

if the number is smaller than zero: returns -1
if the number is zero: returns 1
otherwise, return the number times the factorial of the number - 1

For the test, this is our memoized version of the factorial:

const memoizedFactorial = memoize(factorial);

Let's call it and compare the cold and the cached versions.

Similar to the testSum function, I created a testFactorial function to handle the testing.

function testFactorial(label, numbers, callback) {
  console.time(label);
  for (let number of numbers) {
    callback(number);
  }
  console.timeEnd(label);
}

It's very similar to the testSum function, the only difference is that the callback only receives one parameter.

Running these two times:

testFactorial('cold', numbers, memoizedFactorial);
testFactorial('cached', numbers, memoizedFactorial);

We get this performance comparison (running on a MacBook Pro (13-inch, M1, 2020), Chip Apple M1, Memory 16GB machine):

------ factorial ------
cold: 572.349ms
cached: 1.919ms
--------- // ---------

The memoized version is 298% faster than the pure version.

Closing thoughts

Memoization and other performance optimization approaches are always interesting. And one of the best ways to deeply understand anything is by building from scratch.

We have implemented the memoization algorithm step by step, we tested the function with the sum function, and saw some performance improvements but not that much, as the sum function is pretty simple.

We also tested with a more complex function, the factorial function. In this performance test, we could see a lot of improvements and how this concept is powerful.

Linked List Data Structure in JavaScript

TK — Wed, 16 Mar 2022 13:53:43 +0000

This post was originally published at TK's blog.

A linked list is a collection of nodes that form a linear sequence. The difference between an array and a linked list is that the array has indexed elements, so we can get an element by constant time by just searching by its index. In the linked list, we need to go through the nodes to get the searched element and that takes linear time.

The advantage is that the linked lists can insert and remove items in constant time.

A Linked List is a sequence of nodes and each node has two attributes: the value it stores and the reference to the next node of the sequence.

The first and last nodes are called head and tail of the list, respectively. So to get to the tail of the last, we traverse the linked list by moving from one node to another using each node's next reference.

The Linked List having the head and the tail as attributes helps add new nodes to the start and the end of the list. But we can implement it with or without the tail attribute. We will dive into this implementation.

We can separate the linked list from its elements. Each element is a node and we can implement this representation with a Node class.

class Node {
  constructor(value, next = null) {
    this.value = value;
    this.next = next;
  }
}

Basically, it has a value and the reference to the next node. We add a default value (null) to the next parameter to make it more flexible to use when creating new nodes.

The simplest way to use it is:

new_node = new Node(1);
new_node.value; // 1
new_node.next; // null

Instantiate the new node.
We can access the value and the next attributes.

But with the flexibility of the next parameter, we can also use it by passing the next node reference.

const nextNode = new Node(2);
const newNode = new Node(1);

newNode.next = nextNode;
newNode.value; // 1
newNode.next.value; // 2

Have the next node.
Instantiate the new node by passing the value and then assigning the reference to the next node (nextNode in our case).
We can access the value and the next value.

For the linked list, the first step is to create a class representing it. For now, we just want a head attribute when creating an empty list.

class LinkedList {
  constructor() {
    this.head = null;
  }
}

Simple as that. Just a class and initialize the head attribute with null for an empty list.

Let's implement the easier method: is_empty. How do we know when a list is empty? If the head is null, we didn't add any node to this list. This is the logic behind the is_empty method.

isEmpty() {
  return this.head === null;
}

Pretty simple, huh?

Now the pushFront method. We basically need to create a new node, points the next attribute from this new node to the head, and assign this new node to be the new linked list head.

Remember we have the next parameter when creating a new node? We can use it to assign the previous head when creating the new node. Something like this:

new Node(value, previousHead);

In the context of the linked list, we will have the self.head. So:

new Node(value, this.head);

The last step is to assign this new node to the head and we will prepend it.

this.head = new Node(value, this.head);

Create new node
Assign the next attribute to the previous head
And assign the new node to the head

The complete method will be like this:

pushFront(value) {
  this.head = new Node(value, this.head);
}

Just one line. Pretty good!

For the pushBack, it's a bit different, because, instead of adding a new node to the head of the list, we need to add to the tail. So basically we need to iterate through the list to be in the last node and point it's next attribute to the newly created node.

The question is: How do we iterate through the list?

The difference between the tail node and the rest is the next attribute. The tail has no next. It points to null. The rest always point to a different node.

To iterate through the list to get the last node, we get the next node until the node has no next attribute. Start with the first node: the head.

let currentNode = this.head;

And then iterate.

while (currentNode && currentNode.next) {
  currentNode = currentNode.next;
}

We divide this code into two parts:

looping while the node is not null and the node's next attribute is also not null
update the current node by assigning the next node

When the while loop breaks, we have the last node, so we just need to update the last node next attribute.

currentNode.next = new Node(value);

The complete code:

pushBack(value) {
  let currentNode = this.head;

  while (currentNode && currentNode.next) {
    currentNode = currentNode.next;
  }

  currentNode.next = new Node(value);
}

The size method implementation is straightforward. We basically need to iterate through the whole list and count each node.

To iterate is pretty simple. We just need to loop while the current node is not null.

while (currentNode) {
  currentNode = currentNode.next;
}

And for each iteration, we need to increase our counter.

size() {
  let count = 0;
  let currentNode = this.head;

  while (currentNode) {
    count += 1;
    currentNode = currentNode.next;
  }

  return count;
}

Initialize the count with 0.
Get the current node: the head.
Iterate through the list.
For each iteration, increase the counter.
Returns the count.

For the search algorithm, we need to receive a value and return true or false if this value is in the linked list.

So we basically need to iterate through the linked list searching for this value.

The iteration is simple:

while (currentNode) {
  currentNode = currentNode.next;
}

Now, for each node, we see if the current node value is the same as the searched value.

while (currentNode) {
  if (currentNode.value === value) {
    return true;
  }

  currentNode = currentNode.next;
}

We can do this way to return true if the searched value is found. Or we can do this verification only after the loop stops. So we would need to stop the loop if we find the value.

while (currentNode && currentNode.value !== value) {
  currentNode = currentNode.next;
}

We will iterate while we didn't find the value and it is not the last node
Basically, the loop will stop when finding the searched value or finish the entire linked list

To return the value, we can use the Boolean function.

return Boolean(currentNode && currentNode.value === value);

With this, we cover all possibilities:

When currentNode is null: Boolean transform null into false
When currentNode is not null and the value is equal to the searched value

To simplify, we could also write the statement like this:

return Boolean(currentNode);

Because if we have the currentNode, it's because we found the searched value. If it doesn't have the currentNode (node is null), it's because we didn't find the searched value.

search(value) {
  let currentNode = this.head;

  while (currentNode && currentNode.value !== value) {
    currentNode = currentNode.next;
  }

  return Boolean(currentNode);
}

The last method to be implemented is the remove method. We can think about this method in separated cases:

when the list is empty.
when we want to remove the head node.
when we want to remove a node from the middle or the last one.

For the empty case is pretty simple. We just check the list with our isEmpty method.

if (this.isEmpty()) {
  return;
}

We can also throw an error exception or just print "The list is empty", for example.

For the case when we want to remove the head node, we check it first and then remove it.

if (this.head.value === value) {
  this.head = this.head.next;
  return;
}

To remove it, we just need to point the head to the its next node.

The last case is when we want to remove a node in the middle or the last one. Let's draw it!

For this algorithm, what we want is to get the previous node of the node to be removed and point to the next node of the node to be removed. So we need to have the previous node in each iteration. This is the fundamental part of our algorithm.

let currentNode = this.head;

while (currentNode.next) {
  if (currentNode.next.value === value) {
    currentNode.next = currentNode.next.next;
  }

  currentNode = currentNode.next;
}

This is the algorithm.

We will iterate through the list while the current node's next is not a null value. Why? Because we want to compare the next node value. Not the current one.

currentNode.next.value === value;

This the logic we are searching for. Does the current node's next value is the value we want to remove?

If it is true, we basically remove the current node's next node by pointing the next to the next.next, and returning the function.

If it is false, we keep iterating until we find the value we want or when we finish the entire list.

Joining all the parts, we have:

remove(value) {
  if (this.isEmpty()) {
    return;
  }

  if (this.head.value === value) {
    this.head = this.head.next;
    return;
  }

  let currentNode = this.head;

  while (currentNode.next) {
    if (currentNode.next.value === value) {
      currentNode.next = currentNode.next.next;
    }

    currentNode = currentNode.next;
  }
}

The Linked List class

Joining all the parts we talked about and implemented, we have:

class Node {
  constructor(value, next = null) {
    this.value = value;
    this.next = next;
  }
}

class LinkedList {
  constructor() {
    this.head = null;
  }

  pushFront(value) {
    this.head = new Node(value, this.head);
  }

  pushBack(value) {
    let currentNode = this.head;

    while (currentNode && currentNode.next) {
      currentNode = currentNode.next;
    }

    currentNode.next = new Node(value);
  }

  size() {
    let count = 0;
    let currentNode = this.head;

    while (currentNode) {
      count += 1;
      currentNode = currentNode.next;
    }

    return count;
  }

  search(value) {
    let currentNode = this.head;

    while (currentNode && currentNode.value !== value) {
      currentNode = currentNode.next;
    }

    return Boolean(currentNode);
  }

  remove(value) {
    if (this.isEmpty()) {
      return;
    }

    if (this.head.value === value) {
      this.head = this.head.next;
      return;
    }

    let currentNode = this.head;

    while (currentNode.next) {
      if (currentNode.next.value === value) {
        currentNode.next = currentNode.next.next;
        return;
      }

      currentNode = currentNode.next;
    }
  }

  isEmpty() {
    return this.head === null;
  }
}

Let's test it!

const linkedList = new LinkedList();
linkedList.isEmpty(); // true
linkedList.size(); // 0

linkedList.pushFront(1);
linkedList.isEmpty(); // false
linkedList.size(); // 1
linkedList.head; // new Node(1)

linkedList.pushBack(2);
linkedList.pushBack(3);
linkedList.pushBack(4);
linkedList.size(); // 4

linkedList.pushFront(0);
linkedList.size(); // 5

linkedList.search(0); // true
linkedList.search(1); // true
linkedList.search(2); // true
linkedList.search(3); // true
linkedList.search(4); // true
linkedList.search(5); // false

linkedList.remove(5);
linkedList.size(); // 5

linkedList.remove(0);
linkedList.size(); // 4

linkedList.remove(4);
linkedList.size(); // 3

linkedList.remove(2);
linkedList.size(); // 2

linkedList.remove(1);
linkedList.size(); // 1

linkedList.remove(3);
linkedList.size(); // 0
linkedList.isEmpty(); // true

What do we do here?

Create the linked list
Verify if it is empty
Verify the size of the list
Push a new item to the front
Now it's not empty anymore, have size of 1, and the head is the node with value 1
Push new values to the end of the list: 2, 3, 4. And now the size of the list is 4
Push a new value to the beginning of the list: 0. Size: 5
Search for 0 to 4: all return true, we found the value
Search for 5: it returns false as we don't have this value in the list
Remove 5 and the list keeps the size of 5
Remove values 4 to 0, the list is empty, and with size 0

Resources

Queue Data Structure in JavaScript

TK — Wed, 16 Mar 2022 13:50:07 +0000

This post was originally published at TK's blog.

The queue data structure is a collection of items that follow the first-in first out principle. The first added element will be the first element to be removed from the queue. So, elements are added in the back and removed from the front.

An analogy would be a simple line of people waiting for the next train. In the software engineering context, an example is a web server receiving and responding requests.

The main API methods are enqueue (add) and dequeue (remove). But we can also add other methods as part of the API implementation: size, front, back, and isEmpty.

Queue Implementation

We can create a Queue class as a wrapper and use the Python list to store the queue data. This class will have the implementation of the enqueue, dequeue, size, front, back, and isEmpty methods.

The first step is to create a class definition and how we are gone store our items.

class Queue {
  constructor() {
    this.items = [];
  }
}

This is basically what we need for now. Just a class and its constructor. When the instance is created, it will have the items list to store the queue items.

For the enqueue method, we just need to use the list append method to add new items. The new items will be placed in the last index of this itemslist. So the front item from the queue will always be the first item.

enqueue(item) {
  this.items.push(item);
}

It receives the new item and appends it to the list.

The size method only counts the number of the queue items by using the length attribute.

size() {
  return this.items.length;
}

The idea of the isEmpty method is to verify if the list has or not items in it. If it has, returns false. Otherwise, true. To count the number of items in the queue, we can simply use the size method already implemented.

isEmpty() {
  return this.size() === 0;
}

The shift method from the list data structure can also be used to dequeue the item from the queue. It dequeues the first element as it is expected for the queue. The first added item.

dequeue() {
  this.items.shift();
}

For the front method, we can just access the first element in the items list.

front() {
  return this.items[0];
}

If it has at least one item, we get the front, the first added item in the queue.

For the back method, I used the at method to access the last element by passing a -1:

back() {
  return this.items.at(-1);
}

This feature (.at()) is only available for NodeJS v17 or later. If using old versions, we can use the good-old this.items[this.items.length - 1].

If it has at least one item, we get the back item, the last added item in the queue.

Queue usage

The usage would be something like:

const queue = new Queue();

queue.isEmpty(); // true
queue.size(); // 0

queue.enqueue(1); // [1]
queue.enqueue(2); // [1, 2]
queue.enqueue(3); // [1, 2, 3]
queue.enqueue(4); // [1, 2, 3, 4]
queue.enqueue(5); // [1, 2, 3, 4, 5]

queue.isEmpty(); // false
queue.size(); // 5;
queue.front(); // 1;
queue.back(); // 5;

queue.items; // [1, 2, 3, 4, 5];

queue.dequeue(); // [2, 3, 4, 5];
queue.dequeue(); // [3, 4, 5];
queue.dequeue(); // [4, 5];
queue.dequeue(); // [5];
queue.isEmpty(); // false
queue.dequeue(); // []
queue.isEmpty(); // true
queue.size(); // 0;

We first instantiate a new queue from the Queue class.

So now we can verify its emptiness: yes, it is!
Verify size: 0.
Enqueue 5 new items to the queue: [1, 2, 3, 4, 5].
Verify emptiness again: not anymore!
Verify size: 5.
Get the front element: 1 because it was the first added item.
Get the back element: 5 because it was the last added item.
Dequeue 4 items: 1, 2, 3, and 4.
Verify emptiness: it is not empty yet!
The size is 1 and the back and front are the same number: 5
Dequeue the remaining item.
Verify emptiness: it is empty now!
Size is back to 0.

The whole implementation

class Queue {
  constructor() {
    this.items = [];
  }

  enqueue(item) {
    this.items.push(item);
  }

  dequeue() {
    this.items.shift();
  }

  isEmpty() {
    return this.size() === 0;
  }

  front() {
    return this.items[0];
  }

  back() {
    return this.items.at(-1);
  }

  size() {
    return this.items.length;
  }
}

Runtime and Space complexities

Now about space and runtime complexities for each method implemented.

The space is pretty simple. It's a list, so it's O(n) where n is the current number of items in the stack.

The runtime for each method is O(1), constant time.

Resources

Stack Data Structure in JavaScript

TK — Wed, 16 Mar 2022 13:43:35 +0000

This post was originally published at TK's blog.

The stack is a collection of items that follows the last-in, first-out concept.

For the addition of new items, the stack only allows it to push the new item to the top. When it comes to removing items, it only allows us to remove the last added item, or commonly known as the top item.

The main API methods are push (add) and pop (remove). But we can also add other methods as part of the API implementation: size, top, and isEmpty.

Stack Implementation

We can create a Stack class as a wrapper and use the JavaScript list to store the stack data. This class will have the implementation of the push, pop, size, top, and isEmpty methods.

The first step is to create a class definition and how we are gone store our items.

class Stack {
  constructor() {
    this.items = [];
  }
}

This is basically what we need for now. Just a class and its constructor. When the instance is created, it will have the items list to store the stack items.

For the push method, we can just push the new item to the end of the list. The new item will be placed in the last index of this items list. So the top item from the stack will always be the last item.

push(item) {
  this.items.push(item);
}

It receives the new item and push it to the list.

The size method only counts the number of the stack items by using the .length attribute.

size() {
  return this.items.length;
}

The idea of the isEmpty method is to verify if the list has or not items in it. If it has, returns false. Otherwise, true. To count the number of items in the stack, we can simply use the size method already implemented.

isEmpty() {
  return this.size() === 0;
}

The pop method from the list data structure can also be used to pop the item from the stack. It pops the last element as it is expected for the stack. The last added item.

pop() {
  return this.items.pop();
}

For the top method, we just need to get the last element of the list:

top() {
  return this.items[this.items.length - 1];
}

If it has at least one item, we get the top, the last added item in the stack. Otherwise, it returns an undefined value.

Stack usage

The usage would be something like:

const stack = new Stack();

stack.isEmpty(); // true

stack.push(1);
stack.items; // [1]
stack.push(2);
stack.items; // [1, 2]
stack.push(3);
stack.items; // [1, 2, 3]
stack.push(4);
stack.items; // [1, 2, 3, 4]
stack.push(5);
stack.items; // [1, 2, 3, 4, 5]

stack.isEmpty(); // false
stack.top(); // 5

stack.pop();
stack.items; // [1, 2, 3, 4]
stack.pop();
stack.items; // [1, 2, 3]
stack.pop();
stack.items; // [1, 2]
stack.pop();
stack.items; // [1]

stack.isEmpty(); // false

stack.pop();
stack.items; // []

stack.isEmpty(); // true
stack.top(); // undefined

We first instantiate a new stack from the Stack class.

Verify emptiness: yes, it is!
Add 5 new items to the stack: [1, 2, 3, 4, 5].
Verify emptiness: not anymore!
Get the top element: 5 because it was the last added item.
Remove 4 items: 5, 4, 3, and 2.
Verify emptiness: not empty yet!
Remove the remaining item.
Verify emptiness: it is empty now!

The whole implementation

class Stack {
  constructor() {
    this.items = [];
  }

  push(item) {
    this.items.push(item);
  }

  pop() {
    return this.items.pop();
  }

  top() {
    return this.items[this.items.length - 1];
  }

  isEmpty() {
    return this.size() === 0;
  }

  size() {
    return this.items.length;
  }
}

Runtime and Space complexities

Now about space and runtime complexities for each method implemented.

The space is pretty simple. It's a list, so it's O(n) where n is the current number of items in the stack. The runtime for each method is O(1), constant time.

Reversing a list

We can use the stack data structure for a diverse number of algorithms. An example is to reverse the items from a list.

We want to reverse a list of books, a bookshelf.

function reverse(list) {
  const stack = new Stack();

  for (item of list) {
    stack.push(item);
  }

  const reversedList = [];

  while (!stack.isEmpty()) {
    reversedList.push(stack.pop());
  }

  return reversedList;
}

Create a stack instance
Push each list item to the stack
Create an empty reversed list
Pop each item until the stack is empty
For each popped item, append it to the reversed list
Return the reversed list

The idea is to make the last list item the first to be popped from the stack.

The function usage would be something like:

const reversedBooks = reverse([
  'Harry Potter',
  'Atomic Habits',
  'Leonardo da Vinci',
  'Sapiens',
  'Peak',
]);

reversedBooks;
// [
//   'Peak',
//   'Sapiens',
//   'Leonardo da Vinci',
//   'Atomic Habits',
//   'Harry Potter'
// ]

Other examples

We can also implement the stack concept in a undo command. Imagine our text editor. For each document change, we store the new document in the stack. If we want to undo the change, we just need to pop the last change and stay with the previous state of the document.

Web Browsers can also use stacks to store the visited website. When the user visits a new website, it pushes the new URL to the stack. When the user goes back, using the "back" button, it pops the last visited website and uses the previous URL.

Resources

Frontend Challenges: Front-end Engineer Assignment

TK — Mon, 17 Jan 2022 12:23:35 +0000

The original post was published at iamtk.co

This is part of the Frontend Challenges Series.

Today I finished a frontend challenge and I had a blast working on it. There were so many interesting concepts I debated with myself while designing and implementing the feature, so I was eager to document and share everything I learned through this entire journey.

Let's talk about the challenge:

Technology

The project is setup using React
They prefer using TypeScript (or Flow)
They use EmotionJS as a CSS tool

UI

The idea is to create a Guest and Room Overlay component. The user can open it, add different rooms, select any limited number of adults and children, and select the children's ages.

Input rules

The component should be able to pass a string as the default data. These are the rules:

Rooms are separated by pipe |
Adults and children are separated by a colon :
Children ages are separated by a comma ,

Examples:

"1:4,6|3" → Two rooms, one with one adult and two children ages four and six and the other with three adults and no children
"3" → One room with three adults and no children
"2:4" → One room with two adults and one child aged four
"1:0,13,16" → One room with one adult and three children (aged zero, thirteen, and sixteen)

Functional Requirements

Up to eight rooms can be added
Each room has at least one adult and up to a maximum of five
Each room has zero or more children up to a maximum of three
Each child needs to have their age supplied, so we know what kind of bed or cot to provide and what to charge for the room
Each room has a maximum occupancy of five. This is, adults plus children per room
The Guest and Room selector should always yield a valid room occupancy, use button disablement to avoid invalid configurations
A user can either click Search to commit the output to the URL or click the x on top to reset the chosen room selection and revert the UI back to the original state.

Now, giving this introduction to the challenge, I want to share the topics I will cover in this piece. The main topics are:

Data Structure & State Management: in this part, we'll discuss how to design the UI’s state data structure and manage it throughout the entire component.
UI & Style: creating reusable components, handling responsive design with media queries and react-device-detect, and handling animation.
Unit & Integration tests: the implementation of the tests to make us confident about the feature. Unit tests will be handled by react-testing-library and Integration tests by Cypress.

Data Structure & State Management

I came up with a data structure to represent this UI and it looks like this:

{
  rooms: [
    {
      adultsCount: <number>,
      children: [
        {
          age: <number>,
        },
      ],
    },
  ],
}

A TypeScript implementation looks like this:

type Child = {
  age: number;
};

type Room = {
  adultsCount: number;
  children: Child[];
};

type GuestRooms = {
  rooms: Room[];
};

And an example would look like this:

const GuestRooms = {
  rooms: [
    {
      adultsCount: 2,
      children: [
        {
          age: 4,
        },
      ],
    },
    {
      adultsCount: 2,
      children: [
        {
          age: 4,
        },
        {
          age: 10,
        },
      ],
    },
  ],
};

Now that we have defined the data structure and have a simple implementation of it, we can move to the next part that's how we use this data and which API we should provide to update this state in different parts of the component.

Listing all the behaviors makes it easier to understand how we should handle the data and what are the APIs we need to provide for each component to update our data structure.

I sketched a little drawing with all the behaviors:

Let's list them here:

Update adults count for room X: updateAdultsCount, which should receive the room index and the new count. The function's type contract should be:

  (roomIndex: number, count: number) => void

Update children count for room X: addChild, which should receive the room index and it should add a new child to the children list with the age's value as 8 (the default age's value). The function's type contract should be:

  (roomIndex: number) => void

Remove a child from room X: removeChild, which should receive the room index and the child index. The function's type contract should be:

  (roomIndex: number, childIndex: number) => void

Remove room X: removeRoom, which should receive the room index. The function's type contract should be:

  (roomIndex: number) => void

Select a child's age from room X: updateChild, which should receive the room index, the child index, and the new child age.

  (roomIndex: number, childIndex: number, childAge: number) => void

Add new room section: addRoom, which should just add a new room to the rooms list.

  () => void

Search with the selected rooms & guests: this function won't update our data structure, it'll only receive the data structure, transform it into a string representation and append the result as a query param in the url.

Nice, we have all the necessary APIs to handle the component's state. Now let's start implementing them all.

Context Provider

When I started implementing this solution, I didn't want to use any library or framework to handle the state. I wanted to keep it very simple. I start with a useState hook. But soon some interesting (and common) problems started to arise.

If we have centralized data, to be able to access it using a useState hook, we need to the state via props to all components. And prop drilling can be a big problem in terms of runtime performance and impacting user experience. Updating the state had the same issue. I needed to pass this update function as a prop for all components.

The second solution that I came across, as I still didn't want to use a library for it, was to use the Context API and provide the state's data structure and function APIs to every component that is wrapped by the context provider. This is the fundamental part of my solution to handle the state.

The provider will be pretty simple. It should just be a component that wraps our component and provide a value for it.

export const GuestRoomsContext = createContext<GuestRoomsValues>(undefined);

const GUEST_ROOMS_DEFAULT = {
  rooms: [
    {
      adultsCount: 2,
      children: [],
    },
  ],
};

type GuestRoomsProviderPropTypes = {
  guestRoomsString?: string;
};

export const GuestRoomsProvider: FC<GuestRoomsProviderPropTypes> = ({
  children,
  guestRoomsString,
}) => {
  const defaultGuestRooms = guestRoomsString
    ? toGuestRooms(guestRoomsString)
    : GUEST_ROOMS_DEFAULT;

  const [guestRooms, setGuestRooms] = useState<GuestRooms>(defaultGuestRooms);

  // ...

  return (
    <GuestRoomsContext.Provider value={providerValue}>
      {children}
    </GuestRoomsContext.Provider>
  );
};

So it will receive children and a guestRoomsString. Receiving the guestRoomsString enables us to pass a string as the initial state of our data structure. You can see in the GuestRoomsProviderPropTypes, that this prop is optional, so if we don't pass any string for the provider, it should use the default value GUEST_ROOMS_DEFAULT.

We also use a simple useState and it should be the source of truth of our data. guestRooms is the state and setGuestRooms is the function API to update the state.

The GuestRoomsContext is created and exported. We will use this context in our components to access the data and function APIs. And we also use it to create the provider. children is wrapped by this provider and we also see a providerValue that will talk about it soon.

Before moving to the other implementations, I want to talk a bit about the toGuestRooms function. This is just a transformer, specifically transforming the string format into a GuestRooms data structure.

Why did I decide to do this? My approach is to have an internal data structure for the component instead of using string as the state type. I think designing a better data structure to represent the state of this UI would help a lot when managing the state. This is how is looks like:

const ROOM_SEPARATOR = '|';
const ADULT_CHILDREN_SEPARATOR = ':';
const CHILDREN_SEPARATOR = ',';

function parseChildren(children: string) {
  return children
    .split(CHILDREN_SEPARATOR)
    .map((age: string) => ({ age: Number(age) }));
}

function parseGuestRooms(guestRooms: GuestRooms, room: string) {
  const [adultsCount, childrenString] = room.split(ADULT_CHILDREN_SEPARATOR);
  const children = childrenString ? parseChildren(childrenString) : [];

  guestRooms.rooms.push({
    adultsCount: Number(adultsCount),
    children,
  });

  return guestRooms;
}

export function toGuestRooms(guestRooms: string) {
  const rooms = guestRooms.split(ROOM_SEPARATOR);
  const guestRoomsInitialValue = { rooms: [] };

  return rooms.reduce<GuestRooms>(parseGuestRooms, guestRoomsInitialValue);
}

Using the separators to get each meaningful data and returning the GuestRooms data structure.

As a pure function, we can easily test it.

describe('toGuestRooms', () => {
  it('generates GuestRooms based on "1:4,6|3"', () => {
    expect(toGuestRooms('1:4,6|3')).toEqual({
      rooms: [
        {
          adultsCount: 1,
          children: [
            {
              age: 4,
            },
            {
              age: 6,
            },
          ],
        },
        {
          adultsCount: 3,
          children: [],
        },
      ],
    });
  });

  it('generates GuestRooms based on "3"', () => {
    expect(toGuestRooms('3')).toEqual({
      rooms: [
        {
          adultsCount: 3,
          children: [],
        },
      ],
    });
  });

  it('generates GuestRooms based on "2:4"', () => {
    expect(toGuestRooms('2:4')).toEqual({
      rooms: [
        {
          adultsCount: 2,
          children: [
            {
              age: 4,
            },
          ],
        },
      ],
    });
  });

  it('generates GuestRooms based on "1:0,13,16"', () => {
    expect(toGuestRooms('1:0,13,16')).toEqual({
      rooms: [
        {
          adultsCount: 1,
          children: [
            {
              age: 0,
            },
            {
              age: 13,
            },
            {
              age: 16,
            },
          ],
        },
      ],
    });
  });
});

...to make sure it works and also gain confidence in the implementation.

Number input

Let's now create the NumberInput component as it will be the building block for the adults count input and children count input.

This component is a very simple one. It should only handle the UI and be able to receive data and functions to trigger when necessary.

The type contract (or prop types) should be like:

type NumberInputPropTypes = {
  value: number;
  increaseValue: () => void;
  decreaseValue: () => void;
  minValue: number;
  maxValue: number;
};

value: the value we want to show to the user.
increaseValue: the function to increase the value (meaning adults or children count)
decreaseValue: the function to decrease the value (meaning adults or children count)
minValue: the minimum value accepted by the component. It will be useful to disable the decrease button
maxValue: the maximum value accepted by the component. It will be useful to disable the increase button

And that's it.

There's a simple logic that I wanted to do to disable (or not) the increase and the decrease buttons.

const isAbleToDecreaseValue = value > minValue;
const isAbleToIncreaseValue = value < maxValue;

const isDecreaseDisabled = value === minValue;
const isIncreaseDisabled = value === maxValue;

const decreaseNumber = () => isAbleToDecreaseValue && decreaseValue();
const increaseNumber = () => isAbleToIncreaseValue && increaseValue();

const decreaseButtonVariant = isDecreaseDisabled ? 'disabled' : 'secondary';
const increaseButtonVariant = isIncreaseDisabled ? 'disabled' : 'secondary';

I not only wanted to add a disabled variant for the buttons and change the UI but also disable the state update because it's possible for the user to just disable it via devtools and be able to click the button. Having this second constraint is nice to block this behavior.

And here's the UI:

<div>
  <Button
    disabled={isDecreaseDisabled}
    onClick={decreaseNumber}
    variant={decreaseButtonVariant}
  >
    <MinusIcon />
  </Button>
  <span>{value}</span>
  <Button
    disabled={isIncreaseDisabled}
    onClick={increaseNumber}
    variant={increaseButtonVariant}
  >
    <PlusIcon />
  </Button>
</div>

Adults count input

Now that we have this base component, we can build the AdultsCountInput and the ChildrenCountInput on top of that.

It should be very simple actually.

type AdultsCountInputPropTypes = {
  roomIndex: number;
};

export const AdultsCountInput: FC<AdultsCountInputPropTypes> = ({
  roomIndex,
}) => {
  const { guestRooms, updateAdultsCount } = useContext(GuestRoomsContext);
  const adultsCount = getAdultsCount(guestRooms, roomIndex);

  const increaseValue = () => updateAdultsCount(roomIndex, adultsCount + 1);
  const decreaseValue = () => updateAdultsCount(roomIndex, adultsCount - 1);

  return (
    <NumberInput
      value={adultsCount}
      increaseValue={increaseValue}
      decreaseValue={decreaseValue}
      minValue={1}
      maxValue={5}
    />
  );
};

The AdultsCountInput component can receive the roomIndex as we need this value to be able to update the correct adults count for a given room.

We use the useContext passing the GuestRoomsContext to get the guestRooms and the updateAdultsCount (that will be implemented in a sec).

But I want to focus on the getAdultsCount first. The idea is to just implement a “getter” to, well, get the adults’ count.

export function getAdultsCount(guestRooms: GuestRooms, roomIndex: number) {
  return guestRooms.rooms[roomIndex].adultsCount;
}

It's pretty straightforward. It receives the guestRooms and the roomIndex and it should get the adultsCount from a specific room.

With that, we can use this value to pass to the NumberInput.

We can also see the minValue and maxValue:

minValue={1}
maxValue={5}

These are parts of the business rules. For adults, it should have this interval.

Now let's talk about updateAdultsCount. As we mentioned earlier, it should have this type definition:

updateAdultsCount: (roomIndex: number, count: number) => void;

Within the provider, we can access the guestRooms state and the setGuestRooms function to update the state. Receiving the roomIndex and the new adults’ count should be enough to update the state.

function updateAdultsCount(roomIndex: number, count: number) {
  guestRooms.rooms[roomIndex] = {
    ...guestRooms.rooms[roomIndex],
    adultsCount: count,
  };

  setGuestRooms({
    rooms: guestRooms.rooms,
  });
}

And that's it. We use the spread operator to just update the adultsCount and keep the children value. Pass the updated value to the setGuestRooms and it should update properly.

Back to the component, we can use this new function:

const increaseValue = () => updateAdultsCount(roomIndex, adultsCount + 1);
const decreaseValue = () => updateAdultsCount(roomIndex, adultsCount - 1);

The increaseValue should add +1 to the adultsCount and the decreaseValue should add -1 to the adultsCount.

Children count input

The ChildrenCountInput has a similar behavior but the data structure is a bit different. For the adults, the data representation is a number. For children, it's a list of objects.

type ChildrenCountInputPropTypes = {
  roomIndex: number;
};

export const ChildrenCountInput: FC<ChildrenCountInputPropTypes> = ({
  roomIndex,
}) => {
  const { guestRooms, addChild, removeChild } = useContext(GuestRoomsContext);
  const childrenCount = getChildrenCount(guestRooms, roomIndex);

  const increaseValue = () => addChild(roomIndex);
  const decreaseValue = () => removeChild(roomIndex);

  return (
    <NumberInput
      value={childrenCount}
      increaseValue={increaseValue}
      decreaseValue={decreaseValue}
      minValue={0}
      maxValue={3}
    />
  );
};

The ChildrenCountInput also has a roomIndex prop. It should receive a minValue and a maxValue. As the functional requirements say, the minimum should be 0 and the maximum children's count should be 3.

The getChildrenCount is also very similar.

export function getChildrenCount(guestRooms: GuestRooms, roomIndex: number) {
  return guestRooms.rooms[roomIndex].children.length;
}

Get the children's length from a specific room.

To increase and decrease the children's count, we should add a new child or remove the child from the children's list. Let's implement the addChild and the removeChild functions.

function addChild(roomIndex: number) {
  const children = guestRooms.rooms[roomIndex].children;

  children.push({
    ...children,
    age: 8,
  });

  setGuestRooms({
    rooms: guestRooms.rooms,
  });
}

It receives the roomIndex, gets the children’s list, and pushes a new child with the age of 8 (the default age). Then we should just update the guestRooms state.

The removeChild should work in a similar way but remove a specific child.

function removeChild(roomIndex: number, childIndex: number = -1) {
  const children = guestRooms.rooms[roomIndex].children;

  children.splice(childIndex, 1);

  setGuestRooms({
    rooms: guestRooms.rooms,
  });
}

We use splice to remove the child by its index and then update the guestRooms state.

It receives a childIndex because, in the future, we should use it in our favor to remove a specific child. In this case, we just want to remove the last one. This is why we add a default value as -1, so when calling splice, it removes the last one.

Child select

The next part is about the ChildSelect. It should show all possible age options and handle the select on change.

About the options, I just created a ageOptions with a simple array.

const ageOptions = [...Array(18)];

And we use it to create all options for the select. The entire the ChildSelect component will look like this:

type ChildSelectPropTypes = {
  child: Child;
  roomIndex: number;
  index: number;
};

export const ChildSelect: FC<ChildSelectPropTypes> = ({
  child,
  roomIndex,
  index,
}) => {
  const { updateChild } = useContext(GuestRoomsContext);

  const childAgeOnChange =
    (childIndex: number) => (event: ChangeEvent<HTMLSelectElement>) => {
      const childAge = Number(event.target.value);
      updateChild(roomIndex, childIndex, childAge);
    };

  return (
    <select onChange={childAgeOnChange(index)} value={child.age}>
      {ageOptions.map((_, age) => (
        <option
          value={age}
          key={`${roomIndex}-child-${index}-age-option-${age}`}
        >
          {age ? age : '<1'}
        </option>
      ))}
    </select>
  );
};

The component receives the child (to get the current age), the roomIndex (to be able to find and update the child in a specific room), and the index (the child's index to update its age).

Now we need to implement the updateChild in the provider. This is the type definition:

updateChild: (
  roomIndex: number,
  childIndex: number,
  childAge: number
) => void;

And the implementation looks like this:

function updateChild(roomIndex: number, childIndex: number, childAge: number) {
  const children = guestRooms.rooms[roomIndex].children;

  children[childIndex] = {
    age: childAge,
  };

  guestRooms.rooms[roomIndex] = {
    ...guestRooms.rooms[roomIndex],
    children,
  };

  setGuestRooms({
    rooms: guestRooms.rooms,
  });
}

The idea here is to get a specific child from a given room, update this child's age, and update the guestRooms state.

This component is used by the ChildrenSelect, where we get all children from a room and iterate through it:

export const ChildrenSelect = ({ roomIndex }: ChildrenSelectPropTypes) => {
  const { guestRooms } = useContext(GuestRoomsContext);
  const chidren = getChildren(guestRooms, roomIndex);

  return (
    <div className={childrenSelectWrapper}>
      {chidren.map((child, index) => (
        <div
          className={childAgeSelectWrapper}
          key={`${roomIndex}-child-${index}`}
        >
          <span>Child {index + 1} age</span>
          <div className={selectWrapperStyle}>
            <ChildSelect child={child} roomIndex={roomIndex} index={index} />
            <CloseButton roomIndex={roomIndex} index={index} />
          </div>
        </div>
      ))}
    </div>
  );
};

Here are just iterating through the children. To get the children, we need to implement a simple getter.

export function getChildren(guestRooms: GuestRooms, roomIndex: number) {
  return guestRooms.rooms[roomIndex].children;
}

Remove child

Now that we can add a new child and update its age, we need to be able to remove it with the close button.

type CloseButtonPropTypes = {
  roomIndex: number;
  index: number;
};

export const CloseButton: FC<CloseButtonPropTypes> = ({ roomIndex, index }) => {
  const { removeChild } = useContext(GuestRoomsContext);

  const removeOnClick = (childIndex: number) => () => {
    removeChild(roomIndex, childIndex);
  };

  return (
    <Button variant="danger" onClick={removeOnClick(index)}>
      <CloseIcon />
    </Button>
  );
};

It's actually a very simple implementation. We need a button and a way to handle the button's onClick event. Remember when I said that we could use the removeChild in other places too? This is the case for this component. To remove it, we'll use the removeChild function we already implemented but now passing the childIndex for it, so we can remove a specific child from a room.

And that's it!

Add room

Adding a new room is also very simple. We need a button and the addRoom function that will update the rooms list by pushing a new room with the default value to it.

<Button variant="secondary" onClick={addRoom} fullWidth>
  + Add room
</Button>

And the addRoom implementation looks like this:

function addRoom() {
  setGuestRooms({
    rooms: [
      ...guestRooms.rooms,
      {
        adultsCount: 2,
        children: [],
      },
    ],
  });
}

We keep the current rooms and add a new room two adults and no children.

Remove room

To remove a room, we need a button and the room's index.

const { removeRoom } = useContext(GuestRoomsContext);
const removeRoomOnClick = (roomIndex: number) => () => {
  removeRoom(roomIndex);
};

<Button variant="danger" onClick={removeRoomOnClick(index)}>
  Remove room
</Button>;

We have the button and the removeRoomOnClick. Now we should implement the removeRoom function:

function removeRoom(roomIndex: number) {
  guestRooms.rooms.splice(roomIndex, 1);

  setGuestRooms({
    rooms: guestRooms.rooms,
  });
}

Here we use the same concept we used to remove children from the children's list. Using splice with a specific roomIndex and then updating the guestRooms state.

Search button

To handle the search button, I needed to enable users (engineers) to pass a callback function to the main component and pass it down to the search button component to call it when the user clicks the button. This way, we enable engineers to do whatever they want with the current state.

In the case of this challenge, we are required to get the state data structure, transform it into the string format and attach it to the url.

To do this transformation, we can create a simple function to handle this part:

const ROOM_SEPARATOR = '|';
const ADULT_CHILDREN_SEPARATOR = ':';
const CHILDREN_SEPARATOR = ',';

function toChildrenAgesString(children: Child[]) {
  return children.map(({ age }) => age).join(CHILDREN_SEPARATOR);
}

function toAdultsAndChildrenAgesString({ adultsCount, children }: Room) {
  const childrenAges = toChildrenAgesString(children);

  return childrenAges
    ? adultsCount + ADULT_CHILDREN_SEPARATOR + childrenAges
    : adultsCount;
}

export function toGuestRoomsString(guestRooms: GuestRooms) {
  return guestRooms.rooms
    .map(toAdultsAndChildrenAgesString)
    .join(ROOM_SEPARATOR);
}

A toGuestRoomsString to transform the GuestRooms data structure into a string. We use the separators to construct it. To “prove” it works, we can add some tests and gain more confidence.

describe('toGuestRoomsString', () => {
  it('generates "1:4,6|3"', () => {
    expect(
      toGuestRoomsString({
        rooms: [
          {
            adultsCount: 1,
            children: [
              {
                age: 4,
              },
              {
                age: 6,
              },
            ],
          },
          {
            adultsCount: 3,
            children: [],
          },
        ],
      }),
    ).toEqual('1:4,6|3');
  });

  it('generates "3"', () => {
    expect(
      toGuestRoomsString({
        rooms: [
          {
            adultsCount: 3,
            children: [],
          },
        ],
      }),
    ).toEqual('3');
  });

  it('generates "2:4"', () => {
    expect(
      toGuestRoomsString({
        rooms: [
          {
            adultsCount: 2,
            children: [
              {
                age: 4,
              },
            ],
          },
        ],
      }),
    ).toEqual('2:4');
  });

  it('generates "1:0,13,16"', () => {
    expect(
      toGuestRoomsString({
        rooms: [
          {
            adultsCount: 1,
            children: [
              {
                age: 0,
              },
              {
                age: 13,
              },
              {
                age: 16,
              },
            ],
          },
        ],
      }),
    ).toEqual('1:0,13,16');
  });
});

That's it! Now we are able to transform it into the string format before attaching it to the URL. To the function and call the callback with the result of it, I created a search function:

function search(guestRooms: GuestRooms, callback: OnSearchFunction) {
  const guestRoomsString = toGuestRoomsString(guestRooms);

  return () =>
    callback(
      { guestRooms: guestRoomsString },
      `?guestRooms=${guestRoomsString}`,
    );
}

This way, we just need to implement a possible callback for it. As I'm not using any library or framework for this, we can use the History API.

type State = any;
type Url = string | null;

export type PushStateSignature = (state: State, url?: Url) => void;

export const pushState: PushStateSignature = (state, url) => {
  window.history.pushState(state, '', url);
};

It expects the state and the URL. Passing the pushState as a callback for the search function and we are able to append the guest rooms string as a query param.

UI & Style

Building a lot of different React apps, ones with pure React others with NextJS, made me experience different approaches to CSS styles. Even though it's simple to use inline CSS on React components, I didn't really like the experience as it lacks a lot of “features” like pseudo-classes, selectors.

So in this challenge, I was open to learning and applying a new CSS tool. I’ve heard about emotion-js before but never really tried. It seemed very simple, just CSS styles that you could attach to components. It's like the old days when you just write pure CSS but now with the power of making it modular.

I didn't want to use styled components, so I just installed @emotion/css.

npm i @emotion/css

Button

The first component I wanted to focus on was the <Button>. I wanted to make a component that I could reuse for the entire application. With a “type”, I could change the entire style of a component, so what I built was a variant and this it looks like:

type ButtonVariants = 'primary' | 'secondary' | 'disabled' | 'danger' | 'close';

And now we are able to use this as the prop type:

type ButtonPropTypes = {
  variant?: ButtonVariants;
};

If the users (engineers using this component) are also using TypeScript, it requires they use one of these variants in compile time. This is a beautiful use of TypeScript with React.

With this variant, we can style anything. I used the idea of an object that matches a variant with its style. The first is the cursor:

const Cursor = {
  primary: 'pointer',
  secondary: 'pointer',
  disabled: 'not-allowed',
  danger: 'pointer',
  close: 'pointer',
};

And the usage is simple:

cursor: ${Cursor[variant]};

We would do the same thing for all other styles:

const Colors = {
  primary: 'white',
  secondary: '#0071f3',
  disabled: '#6a7886',
  danger: '#d83b3b',
  close: '#6a7886',
};

const BackgroundColors = {
  primary: '#0071f3',
  secondary: '#f7fbff',
  disabled: '#eff2F6',
  danger: 'rgba(255, 255, 255, 0)',
  close: 'rgba(255, 255, 255, 0)',
};

const BackgroundColorsHover = {
  primary: '#0064d8',
  secondary: '#e4f0fe',
  disabled: '#eff2F6',
  danger: 'rgba(255, 255, 255, 0)',
  close: 'rgba(255, 255, 255, 0)',
};

const BoxShadow = {
  primary: 'none',
  secondary: '#bfdaf9 0px 0px 0px 1px inset',
  disabled: 'none',
  danger: 'none',
  close: 'none',
};

The usage is similar to the cursor:

color: ${Colors[variant]};
background-color: ${BackgroundColors[variant]};
box-shadow: ${BoxShadow[variant]};

&:hover {
  background-color: ${BackgroundColorsHover[variant]};
}

In this component, I also enable it to receive these props: disabled, onClick, dataTestid, and children.

<button
  disabled={disabled}
  onClick={onClick}
  data-testid={dataTestid}
    ...
>
    {children}
</button>

I also saw the need to be able to customize the styles if the user wants. For example, the component has a default padding spacing. But the user could need a different spacing for padding, so we can add a className prop as well and add it to the css like this:

className={css`
    ...
    ${className}
`}

It's actually a lot of power we give to the user. The option was to have an object for the padding and margin, and any other property we want to match with the button's variant.

The last part of this component is the prop fullWidth. The name says everything. If enabling this prop, the button will have a full width, if not it will have an auto width.

width: ${fullWidth ? '100%' : 'auto'};

The prop types look like this:

type ButtonVariants = 'primary' | 'secondary' | 'disabled' | 'danger' | 'close';

type ButtonPropTypes = {
  disabled?: boolean;
  onClick: () => void;
  variant?: ButtonVariants;
  className?: string;
  fullWidth?: boolean;
  dataTestid?: string;
};

And the whole component has these props, types, and styles.

export const Button: FC<ButtonPropTypes> = ({
  children,
  disabled = false,
  onClick,
  variant = 'primary',
  className,
  fullWidth = false,
  dataTestid,
}) => (
  <button
    disabled={disabled}
    onClick={onClick}
    data-testid={dataTestid}
    className={css`
      display: inline-flex;
      border: 0px;
      border-radius: 6px;
      margin: 0px;
      cursor: ${Cursor[variant]};
      align-items: center;
      justify-content: center;
      text-align: center;
      vertical-align: middle;
      position: relative;
      text-decoration: none;
      font-size: 16px;
      font-weight: 600;
      padding: 16px 32px;
      color: ${Colors[variant]};
      background-color: ${BackgroundColors[variant]};
      box-shadow: ${BoxShadow[variant]};

      width: ${fullWidth ? '100%' : 'auto'};

      &:hover {
        background-color: ${BackgroundColorsHover[variant]};
      }

      ${className}
    `}
  >
    {children}
  </button>
);

Animation

To make sure we have an effect when opening the overlay component in the mobile view, we will use keyframes and animation.

The code looks very simple for this transition.

The keyframes is imported from the emotion library, we create an animation name to go from top 100% to top 0, and set the duration for this transition.

import { css, keyframes } from '@emotion/css';

const overlayFade = keyframes`
  from {
    top: 100%;
  }
  to {
    top: 0;
  }
`;

const modelStyle = css`
  // ...
  animation-name: ${overlayFade};
  animation-duration: 0.3s;
  // ...
`;

Simple as that.

Responsive design

To handle responsive design, I focused on mobile-first, and additional adjustments for desktop.

To be able to change a style for specific screen size, we can use media queries. Using emotion-js looks like this:

const style = css`
  border-radius: 0;

  @media (min-width: 576px) {
    border-radius: 6px;
  }
`;

For mobile view, it won't have a border-radius, but all screens with a minimum size of 576px will have a border-radius of 6px.

To make it more consistent across all components and remove the need to write the correct media query, I created a mediaQuery object with all the possibilities.

type Breakpoints = 'xs' | 'sm' | 'md' | 'lg' | 'xl';
type MediaQuery = Record<Breakpoints, string>;

export const mediaQuery: MediaQuery = {
  xs: '@media (max-width: 576px)',
  sm: '@media (min-width: 576px)',
  md: '@media (min-width: 768px)',
  lg: '@media (min-width: 992px)',
  xl: '@media (min-width: 1200px)',
};

Now we can use our object without the need to understand the implementation details about each query. Refactoring the style CSS code above, we get something like this:

const style = css`
  border-radius: 0;

  ${mediaQuery['sm']} {
    border-radius: 6px;
  }
`;

Now we can reuse this mediaQuery code in all components that need to handle different views.

I also created a simple unit test for it:

describe('mediaQuery', () => {
  it('returns the correct media query for each breakpoint', () => {
    expect(mediaQuery['xs']).toEqual('@media (max-width: 576px)');
    expect(mediaQuery['sm']).toEqual('@media (min-width: 576px)');
    expect(mediaQuery['md']).toEqual('@media (min-width: 768px)');
    expect(mediaQuery['lg']).toEqual('@media (min-width: 992px)');
    expect(mediaQuery['xl']).toEqual('@media (min-width: 1200px)');
  });
});

I also needed to handle different HTML elements and styles for desktop vs. mobile views. This is why I used a library called react-device-detect.

In this case, our desktop modal should not only have a modal component but also a backdrop-overlay side by side. If the user clicks the overlay, it should close the modal.

In the mobile view, it doesn't have this overlay component. It should just open a dialog.

The desktop dialog:

export const DialogBrowserView: FC<DialogBrowserViewPropTypes> = ({
  guestRoomsString,
  onClose,
  onSearch,
}) => (
  <BrowserView>
    <div className={dialogStyle}>
      <div onClick={onClose} className={backdropStyle} />
      <Dialog
        guestRoomsString={guestRoomsString}
        onClose={onClose}
        onSearch={onSearch}
      />
    </div>
  </BrowserView>
);

And the mobile dialog:

export const DialogMobileView: FC<DialogMobileViewPropTypes> = ({
  guestRoomsString,
  onClose,
  onSearch,
}) => (
  <MobileView>
    <Dialog
      guestRoomsString={guestRoomsString}
      onClose={onClose}
      onSearch={onSearch}
    />
  </MobileView>
);

And use them:

<DialogBrowserView
  guestRoomsString={guestRoomsString}
  onClose={onClose}
  onSearch={onSearch}
/>
<DialogMobileView
  guestRoomsString={guestRoomsString}
  onClose={onClose}
  onSearch={onSearch}
/>

We could also replace react-device-detect with media queries.

Code splitting

An additional thing that I did was to code split the dialog. Create a chunk for the mobile dialog and another chunk for the desktop dialog.

That way the application won't need to load the mobile dialog code in the desktop view and vice versa.

To do this, I used a library called Loadable Components.

I just needed to do this for the mobile dialog

import loadable from '@loadable/component';

export default loadable(
  () => import(/* webpackChunkName: "DialogMobileView" */ './DialogMobileView'),
);

And this for the desktop view

import loadable from '@loadable/component';

export default loadable(
  () =>
    import(/* webpackChunkName: "DialogBrowserView" */ './DialogBrowserView'),
);

And now the application doesn't need to load unnecessary JavaScript code for each screen size.

Unit & Integration Tests

Unit tests

To verify that all variants are working with the correct styles, I created a test for each variant.

describe('Button', () => {
  describe('primary variant', () => {
    it('verifies correct styles for primary button', () => {
      render(<Button onClick={noop}>{text}</Button>);

      const buttonText = screen.getByText(/Text/i);

      expect(buttonText).toBeInTheDocument();
      expect(buttonText).toHaveStyle('cursor: pointer');
      expect(buttonText).toHaveStyle('color: white');
      expect(buttonText).toHaveStyle('background-color: #0071f3');
      expect(buttonText).toHaveStyle('box-shadow: none');
    });
  });
});

We can use the toHaveStyle API to verify each CSS property. I wanted to test if the button is rendered and these four properties: cursor, color, background-color, and box-shadow.

And I also have a similar test for all other variants: secondary, disabled, danger, and close.

For the header, I added a very simple unit test to verify the header text and if the close button triggers everything correctly.

const noop = jest.fn();

describe('Header', () => {
  it('renders the header text', () => {
    render(<Header onClose={noop} />);

    const headerText = screen.getByText(/Rooms & Guests/i);
    expect(headerText).toBeInTheDocument();
  });

  it('triggers the onClose after clicking the close button', () => {
    render(<Header onClose={noop} />);

    const onCloseButton = screen.getByRole('button');
    userEvent.click(onCloseButton);
    expect(noop).toBeCalled();
  });
});

For the header text, it's a nice test, but mocking the onClose function is not ideal. I will test it properly in an integration test where we simulate how the user interacts with the dialog and close it.

The test for the AdultsCountInput is very interesting as we can test it the way an user would use it.

describe('AdultsCountInput', () => {
  it('increases and decreases count by clicking buttons', () => {
    render(
      <GuestRoomsProvider>
        <AdultsCountInput roomIndex={0} />
      </GuestRoomsProvider>,
    );

    const count = screen.getByText('2');
    expect(count).toBeInTheDocument();

    const minusButton = screen.getAllByRole('button')[0];
    userEvent.click(minusButton);
    const decreasedCount = screen.getByText('1');
    expect(decreasedCount).toBeInTheDocument();

    const plusButton = screen.getAllByRole('button')[1];
    userEvent.click(plusButton);
    userEvent.click(plusButton);
    const increasedCount = screen.getByText('3');
    expect(increasedCount).toBeInTheDocument();
  });
});

We start with rendering the component
Verify if the current count's value is correct
Click the button to decrease the count and verify if it actually decreases it
Click the button to increase the count twice and verify the current count's value

We can have a lot of confidence in this test as it simulates how a user would use it.

The test for the ChildrenCountInput works the same way:

describe('ChildrenCountInput', () => {
  it('increases and decreases count by clicking buttons', () => {
    render(
      <GuestRoomsProvider>
        <ChildrenCountInput roomIndex={0} />
      </GuestRoomsProvider>,
    );

    const count = screen.getByText('0');
    expect(count).toBeInTheDocument();

    const plusButton = screen.getAllByRole('button')[1];
    userEvent.click(plusButton);
    userEvent.click(plusButton);
    const increasedCount = screen.getByText('2');

    expect(increasedCount).toBeInTheDocument();
    const minusButton = screen.getAllByRole('button')[0];
    userEvent.click(minusButton);
    const decreasedCount = screen.getByText('1');
    expect(decreasedCount).toBeInTheDocument();
  });
});

The select component is also very interesting. The experience using userEvent is smooth and does what it intends to do.

But first, let's just add a test to verify the ChildrenSelect doesn't render any select because the current state doesn't have any children.

describe('ChildrenSelect', () => {
  it("does not render a child selector when there's no child", () => {
    render(
      <GuestRoomsProvider>
        <ChildrenSelect roomIndex={0} />
      </GuestRoomsProvider>,
    );

    const selectLabel = screen.queryByText('Child 1 age');
    expect(selectLabel).not.toBeInTheDocument();
  });
});

Now we can create a test to interact with the select and choose different age options.

First I created a helper function to get the first option from the select element.

function getFirstOption(name: string) {
  return screen.getAllByRole('option', {
    name,
  })[0] as HTMLOptionElement;
}

And now I can use it to verify the rendered selects and interact with each one of them.

describe('ChildrenSelect', () => {
  it('selects new option and verify selected item', () => {
    render(
      <GuestRoomsProvider guestRoomsString="1:4,6">
        <ChildrenSelect roomIndex={0} />
      </GuestRoomsProvider>,
    );

    const selectLabel1 = screen.getByText('Child 1 age');
    expect(selectLabel1).toBeInTheDocument();

    const selectLabel2 = screen.getByText('Child 2 age');
    expect(selectLabel2).toBeInTheDocument();

    const selectLabel3 = screen.queryByText('Child 3 age');
    expect(selectLabel3).not.toBeInTheDocument();

    const select = screen.getAllByRole('combobox')[0];
    const selectedOption = getFirstOption('4');
    expect(selectedOption.selected).toBeTruthy();

    const newSelectedOption = getFirstOption('3');
    userEvent.selectOptions(select, newSelectedOption);
    expect(selectedOption.selected).toBeFalsy();
    expect(newSelectedOption.selected).toBeTruthy();
  });
});

Context: “1:4,6” means

1 adult
2 children: one with age 4 and the other with age 6.

We test a lot of things here:

Verify that child 1 and child 2 are rendered
Make sure that child 3 is not rendered
Verify if the selected option is the age 4
Select a new option (age 3)
Verify that the option age 4 is no more the selected one and now the selected option is age 3

For the NumberInput component, the test is pretty straightforward. Just render it and make sure the correct number is rendered.

describe('NumberInput', () => {
  it('renders the value between buttons', () => {
    const noop = () => {};

    render(
      <GuestRoomsProvider>
        <NumberInput
          value={3}
          increaseValue={noop}
          decreaseValue={noop}
          minValue={1}
          maxValue={5}
        />
      </GuestRoomsProvider>,
    );

    expect(screen.getByText('3')).toBeInTheDocument();
  });
});

The test for the SearchButton is also similar to the test above as we just want to make sure we are rendering the correct component with the right value.

describe('SearchButton', () => {
  it('renders the button', () => {
    render(
      <GuestRoomsProvider>
        <SearchButton onSearch={() => {}} />
      </GuestRoomsProvider>,
    );

    const button = screen.getByRole('button', {
      name: /Search 1 room • 2 guests/i,
    });

    expect(button).toBeInTheDocument();
  });
});

I also created a test for the GuestRooms component but it is very similar to the integration test I created later on. I will cover this test in the next section.

Integration tests

To gain even more confidence in the feature, I created an integration test using Cypress.

First, created a function to test the query params in the URL:

function verifyQueryParams(queryParams) {
  cy.location().should((location) => {
    expect(location.search).to.eq(queryParams);
  });
}

Also created a function to click the search button and provide the verifyQueryParams:

function clickSearchButtonWithText(text) {
  cy.get('button').contains(text).click();

  return {
    andVerifyQueryParams: verifyQueryParams,
  };
}

This way we can use it like this:

clickSearchButtonWithText('Search 1 room • 2 guests').andVerifyQueryParams(
  '?guestRooms=2',
);

Then I created a function to handle the test for the adults’ count selection:

function selectAdultsCount() {
  const adultsBlock = 'div[data-testid="adults-count-input-block"]';

  cy.get(adultsBlock).within(() => {
    cy.contains('2').should('exist');

    const adultsMinusButton = cy.get('button[data-testid="minus-button"]');

    adultsMinusButton.click();
    adultsMinusButton.should('be.disabled');
    cy.contains('1').should('exist');

    const adultsPlusButton = cy
      .get('button[data-testid="plus-button"]')
      .first();

    adultsPlusButton.click();
    adultsPlusButton.click();
    adultsPlusButton.click();
    cy.contains('4').should('exist');
  });
}

Verify the count is 2
Click the decrease button and verify the button is now disabled because it's the minimum number of adults and verify that 1 is rendered as the count
Then click the increase button 3 times and verify the current's count is 4

Now we need to create a function to test the children's count selection and their ages.

function selectChildrenCountAndAges() {
  const childrenBlock = 'div[data-testid="children-count-input-block"]';

  cy.get(childrenBlock).within(() => {
    cy.contains('0').should('exist');

    const childrenMinusButton = cy.get('button[data-testid="minus-button"]');

    childrenMinusButton.should('be.disabled');
    cy.contains('0').should('exist');

    const childrenPlusButton = cy
      .get('button[data-testid="plus-button"]')
      .first();

    childrenPlusButton.click();
    childrenPlusButton.click();
    childrenPlusButton.click();
    cy.contains('3').should('exist');

    cy.contains('Child 1 age');
    cy.contains('Child 2 age');
    cy.contains('Child 3 age');

    cy.get('button[data-testid="close-button-1"]').click();
    cy.contains('Child 3 age').should('not.exist');

    cy.get('select').first().select('3');
  });
}

Verify it starts with the count 0 and the decrease button should be disabled
Click the increase button 3 times and it should add three age selects for each child's age
Click the close button for the third child and verify if it is not there anymore
Select the age for the first child

Now that we have all the building blocks, we can use them to create a complete test for the dialog.

function verifyGuestRoomsBehavior() {
  const openDialogButton = cy.get('button');
  openDialogButton.click();

  clickSearchButtonWithText('Search 1 room • 2 guests').andVerifyQueryParams(
    '?guestRooms=2',
  );

  const firstRoom = 'div[data-testid="room-key-0"]';

  cy.get(firstRoom).within(() => {
    selectAdultsCount();
    selectChildrenCountAndAges();
  });

  clickSearchButtonWithText('Search 1 room • 6 guests').andVerifyQueryParams(
    '?guestRooms=4:3,8',
  );

  cy.contains('Room 2').should('not.exist');
  cy.get('button').contains('+ Add room').click();
  cy.contains('Room 2').should('exist');

  const secondRoom = 'div[data-testid="room-key-1"]';

  cy.get(secondRoom).within(() => {
    selectAdultsCount();
    selectChildrenCountAndAges();
  });

  clickSearchButtonWithText('Search 2 rooms • 12 guests').andVerifyQueryParams(
    '?guestRooms=4:3,8|4:3,8',
  );

  cy.get('button').contains('Remove room').click();
  cy.contains('Room 2').should('not.exist');

  clickSearchButtonWithText('Search 1 room • 6 guests').andVerifyQueryParams(
    '?guestRooms=4:3,8',
  );
}

Click the button to open the dialog
Click the search button and verify the expected query param in the URL
In the first room, select adults count and the children's count and ages
Click the search button again and verify the correct query param
Add a second room and add adults and children to it. Verify the query param again
Remove the second room, verify it is not there anymore, click the search button and verify the expected query param

I also created a function to handle the dialog close button and verify its behavior.

function verifyCloseButtonBehavior() {
  cy.contains('Rooms & Guests').should('exist');
  cy.get('button[data-testid="dialog-close-button"]').click();
  cy.contains('Rooms & Guests').should('not.exist');
}

Everything together looks like this:

it('verifies guest rooms dialog behavior', () => {
  verifyGuestRoomsBehavior();
  verifyCloseButtonBehavior();
});

But this is a test for the desktop. I also wanted to test it works fine for the mobile view. The idea is very similar, but add a different viewport and then test it.

describe('on iPhone X', () => {
  it('verifies guest rooms dialog behavior', () => {
    cy.viewport('iphone-x');
    verifyGuestRoomsBehavior();
    verifyCloseButtonBehavior();
  });
});

Everything together looks like this:

describe('GuestRoomsDialog', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  describe('on iPhone X', () => {
    it('verifies guest rooms dialog behavior', () => {
      cy.viewport('iphone-x');
      verifyGuestRoomsBehavior();
      verifyCloseButtonBehavior();
    });
  });

  describe('on desktop', () => {
    it('verifies guest rooms dialog behavior', () => {
      verifyGuestRoomsBehavior();
      verifyCloseButtonBehavior();
    });
  });
});

Let's see a preview of the integration test in action?

That's it!

This is the first article from this series I'm working on: Frontend Challenges. I want to challenge myself in different challenges in the frontend domain and see what I can learn from it. Every challenge will be documented and shared with you all.

I hope you liked this post and feel free to steal some ideas for the projects and products you are building.

See ya!

Resources

Revisiting Conditionals in JavaScript and TypeScript

TK — Mon, 10 Jan 2022 12:53:41 +0000

The original post was published at iamtk.co.

This post is part of the Mastering JavaScript Series.

We are going to see different ways we can handle conditions in JavaScript and how TypeScript can help us make better use of code.

Imagine we have a boolean value and based on this boolean, we want to assign a value to a new variable.

const isActive = true;

With this boolean, we want:

if active (isActive = true): assign a value on to the variable toggle.
if inactive (isActive = false): assign a value off to the variable toggle.

let toggle;

if (isActive) {
    toggle = 'on';
} else {
    toggle = 'off';
}

To do this, we usually use a var or a let statement. Create a toggle with undefined value and then assign the correct value based on the isActive value.

This works.

But we can't use const in this case. When defining a const, we need to add a value attached to it. Doing something like this will throw an error:

> Uncaught SyntaxError: Missing initializer in const declaration

We also can't use const inside the if-else condition.

If we do this:

if (isActive) {
    const toggle = 'on';
} else {
    const toggle = 'off';
}

And then verify the toggle value, it throws an error because this constant is not in the scope.

$ toggle
> Uncaught ReferenceError: toggle is not defined

Another way to handle this type of condition is by using the ternary operator.

const toggle = isActive ? 'on' : 'off';

That's nice and beautiful. Capture everything in a very short and readable way.

Now imagine handling multiple conditions. We can't really use the ternary operator. The first thought is to come back to the if-else statement, but now with multiple possible conditions:

let label;

if (status === 'finished') {
    label = 'Finished task';
} else if (status === 'inactive') {
    label = 'Task inactive';
} else if (status === 'ongoing') {
    label = 'Ongoing task';
}

Another possibility that comes to mind is using a switch case.

let label;

switch (status) {
    case 'finished':
        label = 'Finished task';
        break;
    case 'inactive':
        label = 'Task inactive';
        break;
    case 'ongoing':
        label = 'Ongoing task';
        break;
}

But what if we also want to assign a value to another variable? A tag variable in this case. The tag's value follows this logic:

finished: Finished
inactive: Inactive
ongoing: Ongoing

Let's build it!

let label;
let tag;

switch (status) {
    case 'finished':
        label = 'Finished task';
        tag = 'Finished';
        break;
    case 'inactive':
        label = 'Task inactive';
        tag = 'Inactive';
        break;
    case 'ongoing':
        label = 'Ongoing task';
        tag = 'Ongoing';
        break;
}

Now we also want a button's variant for each status. The logic follows:

finished: secondary
inactive: disabled
ongoing: primary

Let's add this variable to the switch case.

let label;
let tag;
let variant;

switch (status) {
  case 'finished':
    label = 'Finished task';
    tag = 'Finished';
    variant = 'secondary';
    break;
  case 'inactive':
    label = 'Task inactive';
    tag = 'Inactive';
    variant = 'disabled';
    break;
  case 'ongoing':
    label = 'Ongoing task';
    tag = 'Ongoing';
    variant = 'primary';
    break;
}

The lesson here is that the switch case starts to get bigger and more complex. To abstract this complexity, we can use object to map the status to an object that represents the status.

const statusMap = {
  finished: {
    label: 'Finished task',
    tag: 'Finished',
    variant: 'secondary',
  },
  inactive: {
    label: 'Task inactive',
    tag: 'Inactive',
    variant: 'disabled',
  },
  ongoing: {
    label: 'Ongoing task',
    tag: 'Ongoing',
    variant: 'primary',
  },
};

const { label, tag, variant } = statusMap['finished'];
label; // => Finished tag
tag; // => Finished
variant; // => secondary

And if you are using a type system like TypeScript, we can do even better things.

We can type the statusMap’s key and value and require to use the existing keys.

type Statuses = 'finished' | 'inactive' | 'ongoing';
type StatusObject = {
    label: string;
    tag: string;
    variant: string;
};

type StatusMap = Record<Statuses, StatusObject>;

And we used in the map:

const statusMap: StatusMap = {
    finished: {
        label: 'Finished task',
        tag: 'Finished',
        variant: 'secondary'
    },
    inactive: {
        label: 'Task inactive',
        tag: 'Inactive',
        variant: 'disabled'
    },
    ongoing: {
        label: 'Ongoing task',
        tag: 'Ongoing',
        variant: 'primary'
    },
};

When you use it (and if your editor is configured to make the IntelliSense works), it will show all the possibilities for you.

It will also get errors in compile-time if you use a different key to access the object.

Great! Now we have a solution abstracting the complexity and getting errors in compile-time. In the future, it will also be possible to use pattern matching in JavaScript and we can come up with more solutions to handle conditions.

Resources

How I received 4 salary raises in 2 years as a Software Engineer

TK — Mon, 27 Dec 2021 21:32:23 +0000

Or a better title would be: "how to be a Rebel at Work and keep improving the craft".

The original post was published at iamtk.co.

Disclaimer:

these ideas and insights come from a privileged person working in a very privileged industry.
my intent is not to show off, but to share my experience and I hope you can steal some ideas.
it's ok to see money as a valuable capital in your life, but I'll focus on how I improved my craft in this post.

Before telling my experience

In this essay, I'll talk about skills, experiences, opportunities, and mindset. Basically how I became better at my own craft.

My focus was always on the journey and I didn't intend to grow at the company. This was not my goal. I actually didn't have a real goal, just wanted to have fun while working, search for autonomy and control, and have a space I could learn a lot.

With that in mind, my experience may not help you "climb" the Career Ladder. But I hope you can steal some ideas and use them in your favor for better working life.

Also, if your goal is to grow at the company, climb the career ladder, and get raises, maybe I'll write about it in the future, but here are two (shallow) advice:

Understand the career ladder (if your company has one): to be able to get promotions, you need to understand where you are at and what are the expectations for the next level.
Work with your manager: use One-on-One meetings to sync the career ladder expectations, receive feedback, ask for projects you could work on. Everything that can help you go to the next level.

There are many other things you can do to get raises, but, for me, these are the fundamental pieces that will help you take the first step. And as I don't want this essay to focus on promotions, I'll probably write a separate article for this topic. But you can find awesome content on the internet that is probably better than my future one.

My Experience

External Factors

To start, I have to say that I had important External Factors. QuintoAndar is one of the fastest-growing startups in Brazil. Today it's valued at $5.1 billion. It's an innovative company in the real estate industry, and it has a business model that works well in Brazil.

The fact that it's a company that's making money, was invested a huge amount of capital, and keeps growing, it's a very important factor when it comes to getting raises and promotions.

Besides this, it's a company that attracts real talented people, not only in engineering but in the product and other organizations too.

IMHO, the culture is great for engineers as I always felt that it was a safe place to expose and discuss my and new ideas, to share my honest opinion on what was working or not, and was always incentivized to challenge the status quo.

So these are the important external factors that had nothing to do with my skills and work:

QuintoAndar is a fast-growing company
It kept growing financially over the years
Real talented people
Great culture (but not only) for engineers

Achieving my team's goals

Before anything else, my focus was always on my team and our quarter goals. It was truly important to me that I understand the team's goals, the business metrics we should look at, and why we were doing this list of features.

To have everybody on board, we had a Notion page with everything business and product-related to our team. I also created a dashboard with all the important pages we had in the Amplitude platform to be easier to access key metrics.

Understanding why is just the first step. As engineers, one of our responsibilities is to help and come up with solutions for the how.

Collaborating with the product manager, the designer, and the other engineers here is key to succeeding as a team. I think this part not many people talk about and it was always a bit fuzzy for me when I first started working in software engineering. Collaborating means a lot of different things here, but I'll list some ideas I executed in the past:

Ideas can come from different places: your manager, the PM, the designer, or any other person outside your team. After understanding the idea (and most importantly, the problem), you help shape and polish the solution with your engineering background and knowledge from the current system you're working on.
- You probably understand the pain points from the system you're working on.
- With your background, you can think of different solutions: each with its own effort and time you and your team will be needed to spend on.
- Negotiate to have more time to polish the chosen solution after the implementation that needed to be deployed faster to the real user but lacked a bit of the engineering best practices.
- Or better, negotiate to have time to deliver the feature or product with best practices already built-in.
Ideas can come from you: a feature, a product, or an improvement that you saw that could improve the user experience and business metrics. Later, we'll talk about my experience leading the web performance project and being a product-minded software engineer.
Create a safe space to collaborate: it's easy when the company's culture already creates this space that you feel safe to discuss ideas, come up with your own, or disagree with others. Having a more and more diverse team was important to enrich the discussions we had in the team, and having a safe space is the building block that enables people to discuss without fear of judgment.
- Creating space by asking everybody in the team to ask questions, discuss, and share their opinions.
- Actively listen to people, care about what they are saying, and document the insights to use to polish the ideas discussed.
- Having better processes for developing products: a space to discuss the strategy, a space to discuss quarter's goals and metrics, a space to discuss the engineering challenges and architectures/solutions.

The thing is, as you mature as an engineer, you not only code to deliver products but you collaborate in all aspects of software engineering. You understand that making the team, the processes, the delivery, and the dev experience better are part of your "responsibility".

Challenge the status quo: everything can be improved

I need to emphasize that the QuintoAndar's culture not only expected me to challenge the status quo but I was always incentivized to improve – or create awareness – about everything that was suboptimal. So it was always an environment where I could question everything and experiment with ideas and then improve things that were not working or could be better.

When thinking about improving stuff in a company, the goal is to make the organization better. I could think of processes, technologies, the team's culture, the developer experience, and so on.

My idea was always to do things that don't scale first: focus on my team, try experiments, learn with this process, and improve it. Do this cycle until the idea gets matured. When I felt it was ready to scale, I could share it in more broad scope, like a tribe (a set of squads/teams) or for the whole organization.

A good solution always comes up with a well-defined problem (or a well-defined thing that could be improved). This is why understanding the problem and the context is so important. Living and breathing the engineering problems on a daily basis gave me a lot of understanding of how things worked and enabled me to think of solutions to make it better.

As I was working in a big tech company, the technologies, patterns, and conventions are extremely important to make us move fast but always with quality in mind.

My first initiative was to create guidelines about how we used React, optimizations, immutability, and testing in JavaScript and React. Guideline documents are a nice initiative because first, we can normalize the conventions and patterns cross-company, and second, they gave awareness about these topics and as engineers, we could discuss the conventions we wanted to adopt and formalize. I liked the whole process, mainly because it was always nice to receive feedback, different views, and insights from other engineers and we could come up with great solutions for frequently problems we face on a daily basis.

With this first initiative, I gained confidence that I could improve more and more things in the engineering scope. For every problem that I faced, I documented the solutions and patterns that I came up with and shared them with other engineers.

I remember when we didn't have the opportunity to use Hooks in our PWAs. So I made some experiments (React Hooks, Context API, and Pokemons) and shared them.

Regarding JavaScript patterns, I also experimented with some ideas on Closures and Currying, Internationalization abstraction in React, and Functional Programming.

As our codebase was getting more and more complex, it started to get difficult to reason about it, mostly the state management part, which we were heavily using Redux.

"Typing" our state management gave us more confidence and make it more clear how data was structured in the app.

Choosing TypeScript to handle this made me study a bit more and create a Mental Model to think in TypeScript. As we were using React in almost all codebases, it was important to have examples of how React and TypeScript can work together.

Testing applications was a topic people were always interested in the company. We had mini guilds to discuss these ideas and create patterns and conventions we wanted to use. I could share my two experiments: TDD, JavaScript, and React and Basic Recipes for React Testing Library. I think TDD is a more personal process, but using the testing library was our default choice to test our applications.

Technology is always changing and improving. We saw many libraries coming up and two gained our attention: react-query and swr. I could make experiments with both and my colleague and I came up with an Architecture Decision Record (ADR) for react-query. We saw the potential to use this pattern and replace redux-pack and redux-saga, which were bottlenecks in terms of performance for us. Also, react-query brings a more intuitive approach and focuses on the server cache challenge.

Besides technology, we can think of developer experience. Engineering time is getting more and more important.

We recently found that the new 2021 M1 MacBooks cut our Android build times in half.

So for a team of 9, $32k of laptops will actually save $100k in productivity over 2022. The break-even point happens at 3 months.

TL;DR Engineering hours are much more expensive than laptops!
— Jameson (@softwarejameson) November 3, 2021

But in my opinion, it's not only important but can be crucial for many companies that are scaling their business. You don't need a separate team or a platform squad to improve the engineer's productivity, we as engineers can always design better processes and solve productivity bottlenecks. It's up to us to talk to our managers, allocate more time for this type of work, or even create a new team from scratch.

I did work on developer experience and I need to say that I had a blast. I wrote a whole piece about this experience if you want to take a look at it: DX & Software Maintainability in Frontend Engineering.

But in summary, I worked on the engineers' pain points at that time:

Monitoring: instrument the new system, create dashboard links, document how to use monitoring systems on a daily basis.
Tests: experiment with new testing toolings, share the insights from these experiences, write guidelines and examples on how to use them, show different patterns.
CI/CD: not only optimized the CI builds to tune developers' productivity but also automated the release process. Before it was GitHub tags, now they just need to merge the PR and it automatically deploys the new change.
Code Formatting: now using prettier together with ESLint's rules. I was able to format the whole codebase with prettier in less than a week. And with confidence as we had quality automated tests and our monitoring system backing us. Also was able to implement a pre-commit hook to run prettier and ESLint for every commit made on the repository.
Web Performance: I had the opportunity to work on a web performance project before, but in this case, I could work on instrumenting performance tooling to collect real user metrics (Core Web Vitals).

Monitoring and Tests were the building blocks to do more experiments, refactoring, and change code with confidence. Formatting code automatically was a huge gain in letting engineers discuss business rules and architecture instead of debating if you need to add a semicolon or not. Web performance tooling was the building block to measure any initiative related to improving the performance of that software. Making CI faster improved the experience for every engineer that worked on that project. Building a Continuous Delivery also makes the developer experience smoother.

And the last piece in this Challenge the status quo part is Knowledge Sharing.

For me, everything that I learn, research, or experiment, I like to document and share. The document format can vary. It can be an article, a guideline, a Notion document, slides to use for tech talks, or simple notes on my Moleskine to be used to create content based on this draft.

I've been doing this for 8 years now and everything is documented here: writings.

It's a way to share my learnings with my teammates, but also share ideas with the tech communities. Having this process helps me craft my thoughts and learn more about each topic I experimented with before.

First writing, second any kind of content I want to share. Writing down and organizing the ideas and experiments is the foundation to create any other content. With the written article, I basically detailed everything I wanted to share and it could turn into guidelines (the example here was the testing recipes), engineering critics, or slides for a tech talk, as I did for Web Performance Improvements.

But the thing that I liked the most was turning my writings into topics to discuss with the team. We had a group meeting called Guildinha, or Little Guild, and every engineer could plan the meeting with a topic in mind. It was always an informal meeting to chat, discuss, and learn from each other.

With more knowledge and experience I could also mentor informally and formally my colleagues throughout the 3 years I was working there. It was always nice to talk to other people, help them grow in their careers, and impact their work somehow.

Improving my Craft

Among all topics I'm covering in this article, this is what I find the most interesting: Improving my Craft.

I partially wrote about this before in two essays:

I know I'm not a machine, but I'm constantly looking for ways to learn more and gain more knowledge about things I care about. Every day. Every time. Forever.

I've been rethinking the way I learn in my free time and usually, I have three ways

Fundamentals
On-demand
Local Maxima vs Global Maxima

When I think about fundamentals, I think of First Principles. What are the building blocks of software engineering? What are we absolutely sure is true? And reasoning from this principle, we can understand and solve more difficult and complex problems.

In software engineering, we have an ocean of things we can learn. When I was a backend engineer, my focus was on understanding APIs, system architecture, automated tests, and databases. For each topic, I could go deep and understand the building block of each one of them. With a better understanding, I could link this knowledge (or "connect the dots") and solve more difficult problems in backend engineering.

This worked the same way for frontend engineering. I started with HTML and CSS and then learned more about JavaScript. Now we have other things we need a better understanding of to do the work (or at least more complex work) like build systems (the compilers and bundlers), automated tests (for components and integration), browsers engines, and so on.

Knowledge is infinity and I won't learn everything but reasoning from first principles helps me not only solve complex problems but also understand what are the knowledge gaps that I'm missing.

On-demand way of learning is basically when I have a new challenge at work or in my side-project and I need this specific kind of knowledge to solve the problem. It happened when I needed to understand monitoring, testing, and web performance at work and it's also happening on my current side-project which I need to have a better understanding of CMS and text editors.

And Local Maxima vs Global Maxima is one of the things that I learned these days. It was very counterintuitive for me but as I'm doing more and more it got clear to me how important it's to diversify your knowledge portfolio.

If you're a JavaScript engineer, the obvious choice is to keep learning about JavaScript but soon you can get trapped in the local maxima. What I did was to first get one step away from my comfort zone and learn TypeScript and how to get the best out of type systems. My next step was to learn more about browsers, algorithms & data structures. Now I'm investing my time and effort in learning about compilers, how bundlers work, and Rust. Everything I've been learning has a direct or indirect impact on my work and how I think about engineering.

At work, it was no different. My approach was to actively look for feedback, from my managers and peers, and figure out how to be a better software/product engineer.

Actively looking for feedback is a habit that I wanted to start earlier in my career. In the early days, my thoughts were "feedback is great, I'm always open to receiving feedback". But it was not enough. I did at my last job was to have a weekly meeting with my manager and always ask about her perspective about my work on that week:

The things I was doing well and should keep doing
The things I was doing ok and could improve or could do more
The things I was doing were not that great and could improve

I think these bullet points are a good starting point to discuss things I could learn, do, and build habits.

The other thing was to provide feedback and ideas to make the project better. I had a weekly log about the things that could be better and I shared it with my manager to discuss how to implement them in our processes and project's codebases.

Another effective way to receive feedback is to work on your IDP (Individual Development Plan). I wrote about that before but the idea is to develop a plan for your (career) growth. With this shared document, I could always debate with my manager about the path I was following, what I was learning, what were things I was interested in, and receive feedback and advice on how to improve my actions and habits. It's also a nice way to become more self-aware and share with your manager the things you like the most and possibly receive projects that have the intersections of your skills and interests. I really liked working on my IDP and I came up with a public version of it to share with my teammates. Everyone knew what I was learning and it was the starting point to discuss and share more knowledge within the team.

But It's ok if you don't want to create a plan to grow in your career. There are always other ways to incentivize receiving (and giving) feedback in your team. To receive feedback from my peers I first gave feedback (informal and formal) for each one of them and kindly shared that if they had time, I would be happy to receive feedback too.

Regarding figuring out how to be a better product/software engineer, there are very interesting articles about product-minded engineers out there but I'll give my two cents on this topic through my lens and experience.

For me, the first step is to understand the product you are working on.

Is it a product for final users?

who are the clients?
can I understand their behavior through data?
talk to designers who interviewed them, and have documented insights.
how does the business work?

Is it a product for internal stakeholders?

what are the goals of these stakeholders when using this product?
what are the missing functionalities?
what are the general complaints?

Is it a product for engineers?

what's their current workflow?
what are the things they are happy about?
what are the things they are angry about?

But for all three (and any other) groups, we can always think about UX (what are the UX pain points?) and find ways to help them achieve their goal.

Figuring out which product you're working on is a good starting point. Asking a lot of questions about the business, the users, and how it is used are also tools to gain more knowledge about the product.

The second thing I usually do is to understand the next steps, the OKR, the goals of the team regarding the product. Understanding why we are creating this feature X, the related metrics we want to achieve, and actively participating in discussions to debate what and how to do things.

Having open doors to engineers in these meetings is awesome because we can not only help thinking about product ideas but also give insights into the software underneath the product. As we have the engineering and the software background, it's very useful when creating strategies and debating engineering X product tradeoffs.

Don't miss the opportunity to learn more about the product and actively participate in strategy and planning meetings if it's possible.

Challenge yourself with interesting projects

Interesting projects can be the building block for your learning experience at work. While doing a challenging project, you'll probably learn a lot. Things you don't know and need to search and learn. Things you know and could do the best work. After finishing this kind of project, it's nice to do a retrospective on things that went well and things that you could do better. It's always a learning experience.

I like challenging projects because, first, I like challenges. Second, because I always have fun at work when I have the opportunity to learn new things. And finally, because I like to look at my past self and be proud of the work I did.

Things I did:

Built products and business from scratch:
- Built a whole new app for photographers
- Built the for sale business in the affiliates product
Web Performance optimizations for the real estate's owners product
Developer Experience in frontend engineering
CI/CD optimizations, monitoring systems, and automated tests

In all these projects I could always learn a lot. Building the photographers app was the first time I worked with serverless functions and React. Building the affiliate product was the first time I worked with Clojure and the payment system. Improving web performance was the first time I worked with performance in the web and build tooling (mostly Webpack) in depth. Thinking about DX gave me a macro perspective about the engineer's workflow and I can apply this knowledge in other places as well.

Challenging projects are always great tools for self-improvement as they are opportunities to keep learning and improving your craft. Seek interesting projects and keep learning. The feeling to look to your past self and be proud of your decisions and development is amazing.

Keep a Journal

I have a habit to document everything. From things I'm learning to book's notes. From articles ideas to daily journals. Journaling is part of my day and it helps me think and opens space to reflect on the things I did in the past.

These are a series of ideas I came across and you can use in your favor as well:

Document all the things I accomplished

What: What was the project I was working on, the problem I was solving, or features I implemented.
How: How were the solution and the architecture decision, which PRs I worked on, what were the tradeoffs, how the team worked together, and what was my role in this whole project.
Learnings: What are things I would do differently – it can be architecture, solution, process, delegation, communication, prioritization, anything that could be improved and you can use it as a learning experience for another project.

Understand the impact of your work

The first things that should be 100% clear to me are: what's the problem we are solving, why (this usually come from the PM, designer, or business people), and how we should solve this.
With that in mind, I keep my eye on the important "metrics":
- business/ux: I work together with PMs and designers to understand the business metrics and OKRs we'll be looking at.
- engineering: performance, build time, error logs, monitoring systems.
- dx: feedback from my teammates about what are the pain points and create initiatives to solve these problems.

Weekly documents

As I told you, I document everything. It's a habit. At the end of the week, I tend to compile all my daily documents in a weekly document.
It's cool to see what I've been working on, what I've learned, and the challenges I had to overcome.

Quarter document

It is also great to document in this way because I can just compound all notes I wrote at the end of the quarter, and it works like a report document for everyone at the company that wants to know what I've been doing.
This document also serves as a "brag document" to talk to your manager.
It's easy for your manager to understand your accomplishments.
It's easy for your manager to use this document to work on your promotion (if you're interested in this).

This is why I think documentation and journals are powerful. Making it a tiny habit has a huge impact after compounding all your work. As engineers, we tend to focus on the little things in our daily lives, but having this macro vision about our work is rewarding. It makes me proud and aware of my self-development.

Final words

As I wrote at the beginning of this essay, even though the title is a true story and money is a valuable and important capital in our lives, I wanted to talk about skills, experiences, opportunities, and mindset. Basically how I became better at my own craft. And I hope this piece can help you in some way in your career. Feel free to steal some ideas you find interesting and apply them in your life.

Finally, I need to say Thanks! to QuintoAndar for its great culture and the amazing people I worked with: my peers, my managers and leaders, and stakeholders that enabled me to be the best version of myself. I only have gratitude.

Resources

Writings

Series

Learning

Interesting resources

Frontend: Developer Experience & Software Maintainability

TK — Mon, 05 Apr 2021 12:12:18 +0000

This article was originally published at TK's blog.

In the last quarter, I was assigned a big project to work on. Our main goal was to understand the biggest problems and technical challenges we have in our current frontend application and build a thorough vision of how we can move forward.

I've been doing a lot of work on frontend engineering and backend for frontend (BFF) applications and I had an accumulated experience of what problems I could work on. But before start executing (like crazy!), I wanted to make the goal clear and set principles for that project.

Goals

Improve and enable better user and developer experience

This role gave me a lot of autonomy to execute the project in my own way. I liked the idea of balancing execution (improve) and exploration (enable).

DX and UX are one of the core principles I followed to act on the most impactful challenges. Improving DX makes our engineers move faster, increase their productivity to work on business projects, and ship products without (much) friction. Improving DX also can enable better UX as engineers are moving fast to ship product features, find bugs and easily fix them and focus more on the business part of coding.

Build a vision and how we can move forward

For this specific project I worked "alone", I didn't have a squad, but a group of people I could use their support. Thinking about that, it would be impossible to organize my time to explore the problems and technical opportunities and execute everything.

Big challenges require time and effort. As people in the tech industry usually say: "Software development is a team sport". My goal was not to get some time to solve all problems but to show possible solutions, understand the effort needed to execute these solutions, and build a vision to show how we can move forward.

The vision can be built in a documentation format. But part of my goal was also to create space to discuss, build the culture, and enforce the ideas in our actions while building software.

Principles

"Engineering principles realize our values in concrete concepts and guide everyone in a fair and structured way." - Ilya Kozlov

Root for simplicity
Work on what matters
Share knowledge & Transparency

"Simplicity is prerequisite for reliability." - Edsger W. Dijkstra

All these principles are interconnected to the work I did in this project. To reduce the system's complexity, I needed to always think of the simplest way to (re)build the software. When striving for simplicity, we ease our understanding of the software, making it easier to maintain, because it's simpler to change, debug, and refactor.

So, simpler systems have a real impact on the developer experience and productivity. And this is really connected to the "work on what matters". I could start refactoring all code I wanted, but it could have little or no impact on the developer's productivity. Focusing on the developer experience that enables better user experience was my goal and a great variable when thinking in prioritization of what I should work on.

As a "one-person team", I understood that if I really wanted to have a real impact in the organization, I needed a better way to scale the knowledge and the vision. From day 0, I had a Notion page representing the project with everything I was doing documented: backlog, meeting notes, goal & principles, weekly report. Transparency and accessibility were part of the vision I wanted to build with this project together with incremental knowledge sharing throughout the entire quarter.

Starting up

Before start working on this project, I worked on other 4 different products at QuintoAndar. They were all different in terms of business contexts and clients, but very similar when it comes to tech stack and technical challenges.

Over time, I noticed the similarities and started to document the same technical debts, reimagining future architectures, building abstractions that could be reused across the organization, proposing new ways to handle data contracts and consistent state management, build tools to improve DX, etc.

They were all frontend engineering challenges I encountered on my way by building different products. It was a good starting point to have these initial ideas in my backlog to start exploring. But it was very limited.

I also start sketching the engineer's workflow:

From getting the Jira ticket to deploy and taking a look at the monitoring system, we had a lot of opportunities to improve and make the dev experience awesome. It's great to have the big picture's vision, but I also wanted to focus on another powerful tool: listening!

"Listening through questions is a form of active listening with the goal of understanding the rest of the room's perspectives" - @lethain

To improve the developer experience, I needed to understand the real problems the engineers were facing, so I scheduled a brainstorming meeting. In this meeting, I explained my project and shared a Metro Retro board to write down all issues our application had at that time, what was the bad DX we had, and doubts related to this application (e.g. how do we handle data fetching?; how do we handle error handling?; how do we know if our application is health?).

With all these ideas - actually, "issues" - I could organize, improve, and prioritize my backlog.

Besides those issues, one thing that came to my mind was that I (and the team) needed to be able to refactor a lot of code to improve the software quality and pay tech debts.

To enable refactoring, I added 2 "tasks" to my backlog:

Frontend Testing: how should we test frontend & add tests to make our team more confident when deploying new features.
App Monitoring: as an organization, we have many different tooling that could help us monitor our application. I wanted an easy way to answer three questions:
- Is the application health?
- How to use tools to help us debug bugs?
- I deployed a new feature, where should I look at?

Tests

"Quality must be enforced, otherwise it won't happen. We programmers must be required to write tests, otherwise we won't do it." - Yegor Bugayenko

Together with the Quality Assurance team, we add a lot of integration tests using Cypress to the most important workflows in the PWA. Running these integration tests in our CI for each Pull Request helped us a lot in having the confidence to ship features, refactorings, and dependencies upgrades.

I was also working on some studies related to component tests with the testing-library. Our codebase was heavily using Enzyme for most of the tests we have for React components. As we wanted to test more user behavior instead of the component structure, we started to replace Enzyme and add testing-library kinds of tests for new features and components.

As we didn't have much experience nor knowledge of the best way of how to use it, I started learning it, writing tests as examples, document all my learning, and improving our test setup.

I started writing simple recipes to use the testing-library and about testing-driven development in React. Then I wrote tests for different use cases: query elements, expecting content, user behavior (click, focus, etc), custom hooks, better setup for components using redux.

The idea behind this was to make the tests very easy to implement and improve all the pain points. This study's final result was a set of tests as examples and a wiki in our Github project to guide How to test the frontend that covers integration tests, custom hooks, component tests, using the redux store, etc.

Monitoring & Observability

"Originally the feedback loop was you would break stuff, people would yell at you, and then they would praise you when you fixed it, but then the Internet became a thing and our systems got more complicated.”

Our old way to deploy was creating Github release tags. With all these tests I mentioned earlier, we had the confidence to change the release tag to deploy on PR merge. Now we are running full CI/CD.

For the monitoring systems, we had a lot of different possibilities, but we are using mostly Sentry to make the most out of the error tracings to find bugs, debug, and fix them. We are also using Instana to keep up with the API endpoints the PWA uses. Together with the product manager, we consistently take a look at the Amplitude trackings to make sure our users can use our product

Disclaimer: Amplitude is definitely not a tracing tool, but it's nice to keep up with the user's behavior and see some weekly patterns to find issues.

One of our users' worst experiences in our application is when they are using the product and a drawer (our error boundary) shows that they had an error.

To make it distinct in the Sentry logs, we added the "fatal" level for these errors that occurs and trigger the Error Boundary (drawer). When shipping a new feature, we can look at this kind of error by filtering by level fatal.

Now we have toolings to help us verify the health of our systems. But this project aimed to improve the developer experience and I wanted to reduce the cognitive load even more, so I created a Github wiki with all the important links for the Sentry, Instana, and Amplitude dashboards.

Before, the engineers would need to open each dashboard and make their own queries, but now it's easy to access the most important metrics we want to see: open the Github wiki and they are all there.

Foundation & Tooling

"We want everything we offer to be easy to adopt. The simpler a tool or workflow is to adopt and use.. so that our users have a great out-of-box experience." - Lei Zhang

Formatting

The codebase lacked formatting consistency. We were using only eslint to lint the code and break the build process if it finds any lint error. But each engineer has their own code style, so the codebase starts to become very inconsistent.

Now we are using a code formatter called Prettier, it's an opinionated formatter, and all our codebase and PRs are consistent. We can focus on the business part of development and code reviews instead of being distracted by the format issues.

We also had a problem that we pushed code to the github remote and the eslint break the build. To avoid breaking only in the build process, we break it as fast as possible to have no need to wait for the CI build.

Now we are using husky to run eslint in each commit's changed files and format with prettier in the pre-commit. It helped us be more productive and fix things faster and before pushing to github.

I had one learning experience while adopting Prettier for legacy software to run prettier. My first idea was to scale the prettier use for each PR, but it was not a good experience because sometimes the engineer only needs to fix a line of code, but prettier would format the entire file and make it really difficult to code review.

PR Template

The Pull Request template was a bit outdated so I did a revamp to have only the necessary things to create a new PR. It's not required, but we have a description section, type of change, a checklist to help engineers be aware of all technical details we need to take a look at before shipping a new feature, and screenshots if needed.

Performance tooling

My latest project was very related to web performance (I also wrote about this experience: Optimizing the Performance of a React Progressive Web App). But I was only using lab metrics. I wanted to start collecting metrics related to real users as well. This would enable us to see if we have any performance issues for our users and tackle that problem.

We have an internal tool to handle the RUM (Real User Metrics), so I set up the tooling our PWA to start collecting these metrics. And also started to measure the navigation between pages. One of our main users flows is house registration, so adding navigation metrics would have a huge impact if we find any issue and fix it.

Dependencies

"We strive to maintain minimalism and clarity to drive development to completion." - Suckless Philosophy

Minimizing software bloat and rooting for simplicity and minimalism in software was my way to improve the dev experience. In the JavaScript ecosystem, it is very common to have a dozen different libraries that do the same things and other dozens that are borning daily. It's also usual to have many packages in a JavaScript project, even if it isn't really used or replaced with a smaller library or implemented in-house.

I started to explore all the dependencies, which ones I should take a closer look at, which ones I could upgrade and would enable new possibilities for us, and which ones I could just remove.

I could remove a ton of libraries that were not actually being used and it also pointed me to some old and dead code, they were basically features that were not being used by users or dead components that were there in the codebase. Overall, I could remove 10,200 lines of code.

Some other big packages like webpack, babel, and immutable were in my backlog to plan how I could upgrade everything. At that time, we had a team working close to the immutable removal (we're striving to not use any library or immer if an engineer prefers), so I let this library to their team to work on. And we also had a team experimenting with NextJS in some projects, so bumping webpack and babel could not be worthy, so I make this task less of a priority at that time.

Other libraries that upgraded would improve our dev experience and enable us to use interesting APIs like TypeScript and react-redux.

Bumping react-redux enabled us to use hooks together with redux, removing all the mapStateToProps and mapDispatchToProps boilerplates. Even though we are moving away from Redux (testing out react-query for server cache and hooks for client state), the codebase is heavily using it to handle state. Using hooks together with Redux became simpler to handle state and reduced the need for a lot of boilerplate code.

We don't use TypeScript heavily in our codebase yet, but upgrading it to 3.9 was easy and enabled us to use any new features from it mainly in the interface between our frontend and our backend for frontend (BFF).

Work on what matters

"To build scalable and maintainable frontend systems, we need a strategy for managing and organizing the complexity that exists in the user interface." - Safia Abdalla

Listening to the team that was heavily working on this codebase, I could understand some parts that had a lot of complexity that I could work on to make simpler and improve the developer productivity and experience.

House Registration and Edition complexity

In this codebase, we have the house registration flow for landlords, but the code modules were reused for the edition flow as well. In the beginning, it started with good intentions to reuse the same abstraction, but over time it became more and more complex and coupled.

The number of if statements were the first thing that pointed me to this problem. The second was related to the consistency of bugs that was happening in this flow. With the increase of complexity, the tests (manually and automated) didn't cover everything, it was very easy to add features that could break any part of this flow in the product.

"With great complexity comes great bugs and even greater maintenance burdens." - Safia Abdalla

Separating the modules and reusing only the components and some auxiliary helpers would reduce the complexity of each module. I created a migration strategy document organized in "Problems", "Architecture", and the "Plan" to execute it.

Bringing clarity to the team about the vision and how we could solve this problem was the first step to execute the migration. Making everyone understand the problems and the steps to refactor help scale the migration. Every person that would touch that code, could look at it with new eyes and refactor it if possible.

We are still in the process to finish the refactoring, but now we have an optimistic vision that we'll incrementally solve this problem and improve the developer experience.

DDD Architecture

In frontend applications, it's not that common to follow Domain-Driven Design, but in our context, we started to rethink our applications and how we could make them more organized and easy to reason about.

The current state of our PWAs is using components and containers folders to organize shared components and pages. Some helper functions were in big utils files or folders, which became difficult to find and reuse.

The first step was to understand what were domains

This was the first draft of the domain I designed looking at the product and the codebase. For each domain, I added a simple description to make it clear to everyone what was each one.

The same way I did for the registration and edition modules split, I did for this DDD Architecture: create a document to illustrate the problem I was solving, the vision, and the strategy to make it happen.

To make it very clear, I mapped each file/folder in the codebase to the new architecture in the document. It was not an exhaustive mapping, but it brought a lot of clarity to start refactoring the codebase.

Among all explorations I did, this is the one that we could really do incrementally over time without the need to stop everything and focus on that for 2 weeks. This is why I started to refactored it to bring some examples on how to it and scale the knowledge to every engineer in the team.

Interface between backend and frontend

One of the goals for the future is to have a layer between the PWA and the OwnerappAPI to make the data consistent using TypeScript.

We are experimenting with TypeScript in our Backend for Frontend (BFF) to have better and explicit types for each payload. And also in the frontend. I've been studying TypeScript and got to understand the real benefits of applying it to the state part of the frontend application, but also in the UI part by replacing the PropTypes with "compile-time" types.

It's an initial thought yet, but an idea is to have a common repository of types to reuse it between the PWA and the BFF. With this package of types, we can make the interface really consistent. But at the same time, we can add a level of bureaucracy that makes us slower in shipping features. It's a tradeoff that we need to think about before implementing it. But this is just an idea for the future.

Owner Landing Page DX Issue

To give context, we have a different kind of development when talking about our landing pages. We use React to develop them, but we have tooling that removes the react code and library in build time.

Every time we want to test if the landing page is correct - in terms of UI and functionality - we need to run the build process or push the new feature to the test environment, which takes about 18 minutes. Even though we have a "static" landing page without React, the developer experience was suboptimal.

This was actually an issue brought by all engineers that worked in this codebase last quarter. We knew the pain that was to build new features for landing pages.

It started as an exploration to think about which solutions I could come up with. We could work with a static site generator like 11ty, but we only have a design system for React application. This would increase the complexity to rebuild the entire landing page and make space to design inconsistencies.

Another approach was to use NextJS in the project as the default framework and serve the landing pages with SSR. The last approach was to split the landing page from the main application and use NextJS from scratch. This last approach was very interesting for us because the landing page is a living thing that many teams can work on and it shouldn't be coupled with the rest of the application. This strategy would also decrease the build time for each PR and production deployment as we didn't need to run the tests, lint, and build tool for the landing page in the application's CI build.

In this exploration, I also created a document with all possible solutions, the effort and time we needed to rebuild this landing page, the tradeoffs, and the steps for each solution.

Error Handling & Data Fetching

Most of the errors we handle for each request are by using a high order component called withDialogError to provide and open an error dialog when the request returns an error.

Using this approach made it very coupled to the container and Redux because of the way we need to pass data down to the high order component.

When we don't handle errors for a given request, we don't have the data, and it can get us the famous:

Uncaught TypeError: Cannot read property 'a' of undefined

With our ErrorBoundary, it gets this exception and opens a drawer kind of page showing a friendly (but generic) message about the current error. Looking at the fatal errors in Sentry, I understood the correlation (and sometimes causation) with the missing error handling.

I started to redesign the way we handle the errors to remove all the Redux boilerplate and how the error dialog was coupled to the Redux container. Instead of using a high order component, it would be easier to reason about if it was just a declarative component that we add to the page and it receives the correct props to open and show the necessary content and action buttons.

To make the error dialog works properly, I needed to be able to always provide if it is open or not (basically an isError from a request-response), the content (specific for each request/page), and the possibility to refetch/re-request with an action button.

This discovery made me rethink how we handle data fetching - today using Redux - and I started a new exploration looking for new ways to do data fetch stuff, state management, and improve the developer experience of handling errors.

I started by designing the best experience when it comes to data fetching and then I thought of an abstraction to be able to use it in all codebase. But instead of implementing the best abstraction, I started to search for a solution. Well, data fetching is a common challenge in all kinds of frontend applications and we actually have nice alternatives in the community (react-query, swr).

I created some Proofs-of-Concept (PoC) to handle use cases using react-query. The result is pretty interesting. The way react-query handles cache and revalidation and separates client state and server cache is pretty interesting and it also enables us to rethink state management in our frontends. It could be the way we handle data fetching in our PWAs and I started to share this exploration across the organization.

Scale & Culture

"If You Want to Go Fast, Go Alone. If You Want to Go Far, Go Together."

When my manager talked to me about this project, I realized it was a technical leadership role. I was given the opportunity and a lot of autonomy to explore different technical and product issues, understand the pain points that made engineering processes slower and build a vision to make the team move forward.

As I wrote in the beginning, sharing knowledge and transparency were principles I wanted to follow in the entire project. I have a Notion page with everything about the project: roadmap, backlog, goals & principles, documented discoveries, meeting notes, etc. Everything there, open, and easy to access and find information.

Have an open page to give visibility to the entire organization was the first step, but I also wanted to be very intentional about the project. I had the opportunity to talk about what I was working on 3 times a week in the team's daily meeting, I had weekly engineering syncs with the engineering leadership.

For each exploration I did, it was all documented and shared with the team. With the draft idea, I could call a meeting with engineers to explain the exploration, the problem I was working on, and open for discussions and questions.

"The act of asking good questions with good intent opens up a conversation, creating space and safety for others to ask their own questions." - Will Larson

With these meetings, I could make three things happen:

Give visibility of the problem, possible solutions, and a vision.
Give space for them to ask or share opinions and be part of the vision.
And refine the solutions and vision.

As Kevan Lee said: "Transparency starts as a mindset change". I wanted to be intentional in terms of transparency and give them space and accessible documents were the ways I found to make them part of the whole process.

Final words & Resources

I learned a lot throughout this project. It was for sure one of the most challenging projects I did in my career and I had a lot of fun studying, learning, applying my knowledge, thinking in strategy, communicating as a leader, and enjoying the opportunity to work on developer experience and software maintainability.

Throughout the project, I used some resources that I want to share with you all. It's not a prerequisite to manage a project like this, but these resources helped me a lot.

Software Maintainability

Developer Experience

Engineering Leadership

React PWA Performance Case Study

TK — Sat, 23 Jan 2021 14:45:48 +0000

This article was originally published at TK's blog.

In the last quarter, I started working in a new team with different business goals. It was a great team, really fun to work with, and we did a lot of amazing work that I'm proud of.

One of the projects we managed in the last 4 months was web performance improvements for the application we were working on. This post intends to share the improvements we made and the things we learned throughout this process.

Context

Before we start a conversation about web performance, it's important to show the context behind this work.

The first thing I want to mention is the fact that the application (PWA) we started to work on had (actually it still has) most parts of the codebase written 2 years ago. It's a React PWA using Webpack 3, Babel 6, react-redux 5, and so on. A small number of hooks. Most classes components.

The project didn't have real ownership of a team or responsible engineer to take care of the codebase. Different teams need to add features here and there in the PWA but don't actually own the codebase. Effect: the codebase grows in features, bugs, and tech debt, but it is not improved nor refactored.

With this context, we already had a lot of space to improve the codebase. This project was our focus and started to be our own responsibility as well.

My colleague and I became "Service Owners" for this project. The idea of the "Service Owners" is someone (or two people in this case) to be the focal point to clear doubts, manage the tech debts, issues, bugs, etc. Basically, someone that takes care of a project.

Our team was focused on providing the best experience for house owners (landlords): to ease their understanding of the product, register new houses or apartments, and manage the rental and sale of theirs houses.

Together with the product manager and designer, we created a roadmap of features we wanted to ship that quarter. At the same time, performance is a critical piece of the puzzle to provide a good experience for users. We can start with the basic two "metrics": page-load and time to interactivity. There's a correlation (and sometimes causality) between these metrics and user experience.

We also wanted to ship A/B tests and make sure that performance was not a variable that could affect the results of these tests. Basically, we wanted to prevent performance regressions to not influence the tests (but we needed metrics - we'll talk about it soon!).

Our team was not a performance expert team. But the company has a team called Core UX, mainly focused on Web Performance. A team that had experience with frontend performance in the first 3 quarters of 2020.

The Process

Our first idea was to understand the metrics we wanted to track and take care of and do "discovery" tasks to understand potential issues and how we could improve the user experience and app performance. Along with that, we also tried to have a channel with the web performance team to discuss, learn from them, ask questions, and try to find performance issues and fix them.

So we opened a new Slack channel to ease this whole idea and have a bi-weekly meeting with them to show what we were working on, what they were working on, discuss possibilities to improve performance, and have time to ask questions and open discussions.

With this open relationship, we could learn faster and prioritize low-hanging fruit kind of tasks to have faster results with little or no effort at all. We'll discuss this in-depth later in the Performance Improvements section.

The whole process was documented: the meetings, our learning, our discoveries, and the performance fixes.

Metrics & Measure

We had the first discussion about the metrics we wanted to track and my team started to learn more about them. For us that didn't have much familiarity, at first, it was a bunch of acronyms we didn't truly understand. FCP, LCP, FID? What's that?

To understand these terms, I like to first understand the user experience metrics, because it's all linked together.

So, for user experience metrics, I like this User-centric performance metrics article by Google where it defines these metrics:

Perceived load speed: how quickly a page can load and render all of its visual elements to the screen.
Load & Runtime responsiveness: how quickly a page can load and execute any required JavaScript code in order for components to respond quickly to user interaction
Visual stability: do elements on the page shift in ways that users don't expect and potentially interfere with their interactions?
Smoothness: do transitions and animations render at a consistent frame rate and flow fluidly from one state to the next?

I like this because it's very relatable. As website users, we can understand these metrics (and possible frustration when using some "bad-performance" kind of websites).

This is also interesting because we can map the user-centric metrics to the performance metrics we commonly see.

First contentful paint (FCP): measures the time from when the page starts loading to when any part of the page's content is rendered on the screen.
Largest contentful paint (LCP): measures the time from when the page starts loading to when the largest text block or image element is rendered on the screen.
First input delay (FID): measures the time from when a user first interacts with your site (i.e. when they click a link, tap a button, or use a custom, JavaScript-powered control) to the time when the browser is actually able to respond to that interaction.
Time to Interactive (TTI): measures the time from when the page starts loading to when it's visually rendered, its initial scripts (if any) have loaded, and it's capable of reliably responding to user input quickly.
Total blocking time (TBT): measures the total amount of time between FCP and TTI where the main thread was blocked for long enough to prevent input responsiveness.
Cumulative layout shift (CLS): measures the cumulative score of all unexpected layout shifts that occur between when the page starts loading and when its lifecycle state changes to hidden.

I built a simple table to map the performance metrics to the user-centric metrics to be easier to understand each acronym.

As I said earlier, this relation is very interesting and makes us focus not only on bits and bytes but also on the user experience as a whole.

Tooling, Audit & Knowledge Sharing

After having a better understanding of user experience and performance metrics, we wanted to start tracking them. There is a difference between Lab and Field metrics. According to Google:

Lab metrics: using tools to simulate a page load in a consistent, controlled environment.
Field metrics: on real users actually loading and interacting with the page.

Lab Metrics

For the lab metrics, we set up the Lighthouse in our CI using Lighthouse CI. So, for every Pull Request (PR) opened, we run the Lighthouse to gather performance-related data and lock PRs until we fix the performance issue.

With this tool, we can validate various aspects of the PWA (accessibility, SEO, best practices, and performance), but also add assertions to break PRs when it surpasses a budget threshold we set.

For example, we can add assertions related to JavaScript and images sizes (in bytes):

assertions: {
  'resource-summary:script:size': [
    'error',
    { maxNumericValue: 1000000 },
  ],
  'resource-summary:image:size': [
    'error',
    { maxNumericValue: 100000 },
  ],
  'resource-summary:total:size': [
    'error',
    { maxNumericValue: 1000000 },
  ],
},

This JavaScript object is part of the configuration we can use to gather different info about performance. To better understand the configuration for Lighthouse CI, take a look at this docs: Lighthouse CI Configuration.

Another very cool tool we are using for lab metrics is Speed Curve. It's super simple to set up and start gathering data. This tool works better for "unlogged pages" because we add the URL of the website, and based on the website load and interaction, it will collect performance metrics.

The Speed Curve dashboard is very flexible to show (or hide) the metrics we want to focus on. In our case, we wanted to see the evolution of the JavaScript total size, First Contentful Paint, Largest Contentful Paint, Cumulative Layout Shift, JS Total Blocking Time, Backend (TTFB) Time, and Lighthouse Performance Score.

This is working very cool for our landing and home pages.

The last tool we set up is an in-house tool the performance team built. This is a tool to analyze the app bundles and it has 3 main features now:

Bundle Analyze Report: collects and saves the bundle analyzer HTML results.
Bundle Budgets: sets up a budget configuration to add a threshold for the bundle sizes. It breaks the PR if the size of a bundle surpasses the threshold.
Bundle Changes: shows the bundle size changes between the PR and the master (or main) branch. It helps us easily answer "did it increase/decrease the bundle size for X?"

This tool is run in our CI pipeline for every PR and the result is shown in the Github PR (it uses Danger behind it).

These tools are very interesting because

it helps us prevent performance regressions
it also creates awareness about web performance, its metrics, and share knowledge

Field Metrics

For now, we are using Instana to collect Real User performance-related data.

The next step for Real User Monitoring (RUM) is to track more user behavior in our application to gather web vitals metrics in the PWA flow.

Performance Improvements

In this section, I want to detail the process behind each discovery and fixes we did to improve performance and user experience in our application.

Landing Page

We started with our landing page. The first action was to analyze the JavaScript bundle size using Webpack Bundle Analyzer.

Side note: 2 years ago, the team responsible for the landing page decided to use a tool to develop the landing page with react but in the build time, we remove the react from the application to reduce the bundle size served in the landing page.

And this is what we got:

We can analyze a lot of things here, but one that got our attention was the React library in our landing page bundle. As I wrote above, in the side note, React is not being used in production, and we are unintentionally serving it in production making our users download the library without the need to do this.

// components/XYZ.js
import React from 'react';

export const A_CONSTANT = 'A_CONSTANT';

We had a constant inside a React component file. And we were importing that constant in the landing page.

// landing.js
import { A_CONSTANT } from 'components/XYZ';

So importing this constant, we were also importing React.

A possible simple fix was to separate this constant outside the React component file and import it from this new file.

// components/XYZ/constants.js
export const A_CONSTANT = 'A_CONSTANT';

And import the constant from the new file:

// landing.js
import { A_CONSTANT } from 'components/XYZ/constants';

Let's see the bundle size impact after this change:

We reduced 95KB! It's interesting to think we can have a huge impact on a small change after carefully analyzing our bundles. This will be the process behind each improvement we did for the rest of this article:

Analyze the bundles
Fix the performance issue
Gather results & keep track of the metrics

We run again the bundle analyzer and we got this:

The first things that got our attention were the appboy.min.js and the transit.js libraries. The appboy is the Braze, a library we use for communication, and the transit is a library to transform JSON format data into our app state.

The Braze library was very similar to the React library. It was an import statement in a file that the landing page was using but not really using the Braze.

// braze.js
import braze from 'appboy-web-sdk';

export const aFunction = (brazeInstance = braze) => {
  // do something with braze
};

It was importing Braze in the file and using the instance as a default value for a function. The simple solution was to remove the import statement and enforce that every place that was using the aFunction function passes the braze instance. So we don't need to import Braze and add a default value to the parameter:

// braze.js
export const aFunction = (brazeInstance) => {
  // do something with braze
};

Running the bundle analyzer again, we got an astonishing result.

The AnnounceYourHouse landing page was reduced to 90 KB. We could remove almost 50% of the main landing bundle.

We also improved a lot the bundle size of the PriceSuggestion landing page. From 115 KB to 4 KB was an amazing result.

For the transit library, we did a temporary workaround solution. It was importing the library to transform the string JSON saved in the local storage to get info from a single attribute from this object.

The temporary solution was to verify if the string included the info we wanted and remove the need to use the transit library.

We could improved a lot the bundle size of the main landing. Removing almost 50% of the bundle.

As we saw in the metrics section, we had set up the Speed Curve to track the performance of some pages in all this journey. So, for every improvement we made in our application, we keep track of the metrics in these tools.

The total size of the landing page reduced drastically: -2.16 MB.

The Lighthouse Performance score was from 73 to 97:

The Largest Contentful Paint was improved in 1s:

Vendor Chunk

When running npm run bundle:analyzer, we also notice a big dependency in our vendor chunk.

In the vendor chunk, we noticed all the icons from Material UI. Every time a user enters the website, and if the chunk is not cached in the browser, it would need to download the whole chunk. If it's a big chunk to download, it has an impact on the performance and consequently on the user experience.

This is a common problem when importing a Material UI icon in a React component.

One of our components was using an internal component library that used the "named import" style to import the Material UI icon. This, without a proper babel plugin, also adds the rest of the unused icons to the vendor chunk.

We came up with two solutions:

Fix the import from this internal component library we had stopping the use of named import.
Add the babel plugin and configure the app to not add unused modules.

As this internal component library was the first and the deprecated version of our design system, we didn't want to keep maintaining it. The best approach was to not use this library anymore and move all the codebase to use the new design system library (and we're working on it!).

This performance project was not our main project in the quarter so we had less time to focus on it in the sprint. The babel plugin was a more straightforward and simple solution for us at that moment.

We basically needed to add this new babel plugin babel-plugin-transform-imports and configure the babelrc:

[
  "babel-plugin-transform-imports",
  {
    "@material-ui/core": {
      "transform": "@material-ui/core/esm/${member}",
      "preventFullImport": true
    },
    "@material-ui/icons": {
      "transform": "@material-ui/icons/esm/${member}",
      "preventFullImport": true
    }
  }
]

And with it, we prevent the full import of the library in the vendor chunk.

The vendor became way smaller. We also had some impact in the main chunk (the next chunk will talk soon).

With this simple analysis and configuration, we could reduce the vendor chunk by more than 50% (it's still 2.83 MB and could be improved. We will see later!) and the main chunk by 28%.

A huge improvement for the whole app as these chunks were downloaded on each page, if not cached in the browser.

Main Chunk

The main chunk has some common modules among all parts of the application. But after running the bundle analyzer, we got this:

The main chunk is the bottom-left block in the bundle. One thing that got our attention was some containers and components in the bundle. Why are there some components that are specific to only one page but we are making our users download the whole main chunk?

The issue was simple: our code splitting was not working properly.

Our initial idea was to make sure all routes had dynamic import for our components to code split in each router entry point. And this was the problem: not all routes entry points had loadable components, so they were joined in the main chunk instead of creates their own chunk for that specific route and page.

In this application we were using, at that time, react-loadable, so the idea was to simply create these loadables:

import Loadable from 'react-loadable';

export default Loadable({
  loader: () => import(/* webpackChunkName: "component" */'./Component'),
  loading: () => null,
});

And use them for each route entry point.

Running bundle analyzer, we got this:

The main chunk is way smaller and Webpack created more page-specific chunks as well.

The result was huge. The main chunk got more than 50% smaller and the vendor chunk also decreased by 29%.

Caching Biggest Dependencies

Reading this article, you probably saw some big dependencies in our bundle like firebase, braze, immutable, and so on.

Every time we do a new product release, our build system generate a new bundle with the chunks. If anything related to the vendor chunk changes, Webpack will generate a new hash for the chunk. So the browser will not have a cached version for this chunk and it will make the user download it again.

But sometimes, or most of the time, we don't really change these biggest dependencies (only when the dependency is upgraded), and we are making our users paying for that huge chunk.

Our idea was to split these biggest dependencies in its own chunk and make sure the browser has a cached version of this chunk and the user doesn't need to download it again until it is needed.

As we were using Webpack 3 at that time, we needed to use the CommonsChunkPlugin to split these dependencies in its own chunk.

We created a list of all biggest dependencies:

It was mapped as a list data structure in our Webpack config as well:

const VENDOR_LIBRARIES = [
  {
    name: '@firebase',
    label: 'firebase',
  },
  {
    name: '@braze',
    label: 'braze',
  },
  {
    name: 'transit-js',
    label: 'transit-js',
  },
  {
    name: '@material-ui(?!\\/icons)',
    label: 'material-ui',
  },
  {
    name: 'react-dom',
    label: 'react-dom',
  },
  {
    name: 'amplitude-js',
    label: 'amplitude-js',
  },
  {
    name: 'immutable',
    label: 'immutable',
  },
  {
    name: 'raven-js',
    label: 'raven-js',
  },
];

Along with CommonsChunkPlugin, we just needed to iterate through this list to create each chunk.

We can see that the vendor chunk got way smaller and some new chunks were created.

Running the application, we can also test the download of each separate chunk.

And we got a really cool result:

The user still needs to download the dependencies, but after download the first time, the browser will cache them and they won't need to be downloaded again until we bump their version. If we change the vendor chunk, Webpack only generates a new hash for the vendor and doesn't change the other dependencies.

We saw some nice improvements in the Speed Curve dashboard:

As expected, we saw a huge improvement in the JavaScript size: -1.43 MB

Decreasing the JavaScript size also had an impact on the total time the user is blocked to interact with the page: -1.2s

The speed index is a metric to show how quickly the contents of a page are visibly populated. We improved the page to load 2.2s faster.

And the largest contentful paint went from 6s to 3.75s.

Recap

To recap what we saw in this article, let's see the list of things we did in this journey:

Measure: metrics as the foundation of performance improvements.
Lock: prevent regressions & scale the performance knowledge.
Analyze: with data and metrics, analyze the possible problems.
Improvements: code.
Impact: measure the before and the later picture.

I would also recommend talking to more experienced people in this performance domain if it is possible.

Next steps

We have more things to do, but we didn't have time to focus on those things in the last quarter. This is a list of things that come to my mind now:

More metrics: rum for logged pages, ux metrics (engagement, bounce rate), business metrics (conversion).
Manage requests: server requests caching.
More Analysis: backend, chunks, prefetching, etc.
Removable Dependencies: analyze big dependencies that can be removable or replaced
Webpack Upgrade: bump to v5 - cache, optimization, code-splitting, tree shaking.
Webpack Optimization: the need to build faster.
Keep studying: learn more to discover more opportunities.

Resources

I have some resources I used along the way while doing this project. I hope it can be helpful to you too: Web Performance Studies.