Forem: Hotjar

Temporal Dead Zone - the less obvious part and how to avoid it

Eryk Napierała — Thu, 02 Mar 2023 10:33:06 +0000

let and const bindings in JavaScript have a property called Temporal Dead Zone (TDZ). It’s a deadly trap indeed, and often not easy to spot. I learned this the hard way some time ago when working on JavaScript bundle optimization at Hotjar.

What is a Temporal Dead Zone?

According to MDN:

A let or const variable is said to be in a ‘Temporal Dead Zone’ from the start of the block until code execution reaches the line where the variable is declared and initialized.

{ // TDZ starts at beginning of scope
  console.log(bar); // undefined
  console.log(foo); // ReferenceError
  var bar = 1;
  let foo = 2; // End of TDZ (for foo)
}

Okay, this is obvious—you cannot access a constant before you declare it. But nobody writes code like this!, you say. Moreover, ESLint will yell at you if you try. What’s the big deal, then?

Consider a more complex example.

const recurse = (() => {
  if (recurse) { // throws an error, it's a TDZ for recurse
    recurse();
  }
})();

Expressions in JavaScript are evaluated in a certain order. Assigning a value is an expression evaluated from right to left. That’s why const a = 1 + 2 equals a === 3. So, in the example above, first, the IIFE is evaluated, then the recurse name is initialized. That makes the usage of recurse inside the IIFE forbidden. Even if the name can be seen earlier in the code, from the program evaluation flow, it’s no go. But, again, nobody writes code like this, right? And ESLint is smart enough to warn you if you try. What’s your problem, dude?

When ESLint can’t help you

ESLint analyzes the code statically. It’s smart, but because it doesn’t understand all the nuances of the runtime flow, a code like the one below is perfectly valid for the checker.

const callCallback = (cb) => {
  cb();
  return () => {
    console.log('Returned function called!');
  };
};

const returned = callCallback(() => {
  returned();  // throws an error, it's a TDZ for returned
});

A real-life example

It all seems made-up: do we use patterns like this in our codebase? It turns out they’re rare, but they’re there. And they look completely innocent.

Take a look at what I found in Hotjar’s codebase:

const unsubscribe = useStore.subscribe((state: Store) => {
  // ...
  if (unsubscribe) unsubscribe();
  // ...
});

This code may feel illogical, but the subscribe method is a bit more complex than the callCallback you could see above. It calls the callback asynchronously most of the time, but in certain cases, it does it synchronously. And this leads to a runtime error about an uninitialized variable when you try to unsubscribe inside the callback.

But… I’ve never encountered such an error

How could a code like this even slip into the main branch? It’s broken, and loading it in the browser or running a test would detect it immediately, right? It would—if we actually used const and let bindings. But for a long time, this wasn’t the case at Hotjar, and we didn’t realize it!

Despite this syntax having been the norm here for years now, with several such bindings present in our codebase, so far, all of them have been transpiled to var before they reached the browser. For these, hoisting mechanism kicks in and silently evaluates the uninitialized binding to undefined. If you’ve only read ‘JavaScript: The Good Parts’, you may even forget this mechanism exists.

(image source: https://blog.klipse.tech/javascript/2016/09/21/valueOf-js.html)

At Hotjar, the transpilation step was there because we were supporting IE 11 and other ‘odd, old browsers’ that didn’t understand the const and let syntax. It was replaced with var on CI (via Babel), in a way engineers working in product squads weren’t noticing. Noone could see errors like the one above when testing the application.

Things changed, though. Last year we officially dropped support for IE 11 in our product, which opened possibilities for dropping some compilation steps. It not only made our CI and development environment faster, but also had benefits for customers. ES6+ code is more terse, thus takes less time to load. The side-effect is that consts and lets are directly executed in the browser. So now, Hotjar engineers can, and certainly will, spot these errors occasionally. They’ve always been there, but hidden!

How to detect TDZ access

In the example above, failing E2E (we use Cypress) tests for some areas of our codebase uncovered the issue. This was great, but also strange (and a bit concerning) at the same time: no other check—linter, type check, or unit tests—failed. This proves that some mistakes can be only spotted in runtime. Different test types are complementary and cannot replace each other.

How to prevent TDZ access

Use let, whenever there is a risk of accessing a binding before its initialization ends.

// first, initialize empty let binding with null
let unsubscribe: ReturnType<typeof useStore.subscribe> | null = null;
// then, assign it 
unsubscribe = useStore.subscribe((state: Store) => {
  // ...
  if (unsubscribe) unsubscribe();
  // ...
});

The code is now even clearer and more explicit. Looking at the original version, you might not understand why we need to check if the unsubscribe isn’t empty before calling it. After all, its type, according to TypeScript, would always be () => void, no? But as we’ve proven, this wasn’t true. After switching to let, it’s obvious that, in some particular moment, this name could contain null.

Takeaway

Watch out for Temporal Dead Zones even if ESLint doesn’t warn you! Write tests, including E2E ones. And make sure you’re

(image source: https://giphy.com)

Some funky parts of Typescript

Conrad Holtzhausen — Tue, 21 Feb 2023 09:24:11 +0000

And by funky I don’t mean like the smell of chunky milk. I mean funky like the sweet music 💃.

At Hotjar, we have bi-weekly Typescript pair programming sessions where we try and solve a few of the type challenges together. In one session after solving the challenges we set out to, we chose a random challenge as a last hurrah and boy was it a doozy.

So why are you reading this? Well let me tell you what you can expect if you take the red pill with this TL;DR:

First I will take you down the rabbit hole we went on together in the session until we ran out of time.
Secondly, I will take you deeper down the same rabbit hole I went on alone the day after.
I will then finally share the solution that I shamelessly looked at after being thoroughly defeated by the challenge
The main reason I share all this with you is that it contains some interesting nuggets on how the type system works underneath.

The challenge

As I mentioned already, we chose to do the Promise.all challenge. Basically this:

Type the function PromiseAll that accepts an array of PromiseLike objects, the returning value should be Promise<T> where T is the resolved result array.

so:

// when passing this array
PromiseAll([Promise.resolve(3), 42]);

// PromiseAll's return type should be this
Promise<[number, 42]>

Recursion

Our first thought was to recursively go through the passed array and return the awaited value of each for the result. Something like this:

declare function PromiseAll<T extends readonly any[]>(values: T): Promise<AwaitedPromiseAllResult<T>>;

Let’s break this down:

Introduce a generic T with a any[] constraint that we can manipulate
Assign values to T and wrap the result of T in our custom AwaitedPromiseAllResult type
Within AwaitedPromiseAllResult we do the following

// spread T so we can get the first item in the array and the rest
type AwaitedPromiseAllResult<T extends readonly any[]> = T extends [infer TFirst, ...infer TRest] 
 // Unwrap the promise on the first item and repeat this process for the rest
 ? [Awaited<TFirst>, ...AwaitedPromiseAllResult<TRest>] 
 // when there is only one item left in the array, unwrap it and you are done
 : [...Awaited<T>];

When testing our AwaitedPromiseAllResult type separately it seemed to work:

but when using the type in the function we get a whole different result… why, Why, WHY 😡

Here is why:

Note that we constrain our generic as an any[] and then assign our values param as that. In other words [1, 2, Promise<3>] would become number | Promise<number>[].

Okay, that makes sense. So what if we spread the any[] as the values type instead? Well 🙂...

declare function PromiseAll<T ...>(values: [...T]): ...;

Awesome that works, but what happens if I override the generic constraint with something else?

The solution

This is where I gave up and looked at the solutions. All of them were basically a different flavour of the same thing. But before I just show you the solution allow me to stretch this post unnecessarily out a little more.

We’ve come to the reason that I was implored to share this with you all. As you’ve seen in the previous section, by spreading an any[] type you get a tuple. Which is an array of known types. And as you might know, an array is an object. Why is this important? Because Typescript is smart enough to treat it as such and allow you to map over the indexes of an array like you would the properties of a key/value pair object 🤯.

Or in code words… basically this:

type MapArrayAsObjectToIndexes<T extends any[]> = {
  [TKey in keyof T]: TKey
}

Yup 🙂

And finally the solution:

// as before use a generic `T` and convert it to a tuple for `values`'s type
declare function PromiseAll<T extends any[]>(values: readonly [...T]): Promise<{
  // map over the indexes of the array and return their unwrapped values 
  [k in keyof T]: Awaited<T[k]>
}>

Conclusion

Typescript is fun to play with. And I know a lot of people might have already known all the things that I wrote about, but if there were only one or two people who found this interesting I will feel satisfied. But the bigger reason I shared this is to encourage you to play around. Understanding how Typescript's type system works has greatly benefited our day-to-day work even though we don't implement things in this way directly. So have fun and keep learning.

Prioritization: you don’t need a framework, just frame your work

Joshua Ordehi — Thu, 02 Feb 2023 13:16:33 +0000

This is the first of what may become a series of posts about goal-setting, prioritization, and teamwork. During my time at Hotjar, I’ve realized the need to implement better practices for these fundamental aspects of life and work. Sharing a few of my learnings (and failures) is an excellent way to pay it forward and keep myself accountable. Let’s do this!

TL;DR: often, doing less is more. Challenge yourself to go back to basics, prioritizing urgent and important tasks every morning to defog your mind before you dive into your day. Use the Eisenhower Matrix or a similar method to surface critical tasks and shed the rest.

The Challenge

I’m setting a challenge for myself and anyone else who’s interested in trying it out.

Over the next month, choose a productivity strategy that makes sense for you (I settled on the Eisenhower Matrix.) If you already have an approach, share how you use it and how it helps you.
Prioritize your tasks at the beginning of the day, before any work. Pick tasks you’ll realistically complete today or tomorrow, at the latest, and think hard about what’s worth doing first.
If you commit to this challenge, drop us a comment. We’ll catch up again in a month to share our experiences.

Nice to have:

Keep tasks to a minimum, stay within eight per quadrant, and, ideally, don’t exceed five
Only spend up to 15 minutes prioritizing

Growing pains

As companies grow to the point where people don’t know how to pronounce each other’s names anymore, something must unify our efforts toward a common north. Otherwise, we run the risk of rowing in as many directions as there are teams or even contributors!

I find it helpful to go back to the essence of a concept when I want to improve my approach to it: by definition, a team works together toward a common goal.

If you’re working on something that doesn’t contribute to a goal that’s commonly accepted by a group of people, you’re not working in a team—even if you’re under the name of one.

Essential aspects of good teamwork:

Goal-setting: how else can we make sense of our work?
Regular alignment: this is to ensure our work approaches those goals and each teammate is accountable for tasks that best suit their abilities and needs
Task tracking: everyone gets a task list they’re accountable for completing, which is visible to everyone else and scored for priority

These measures also help avoid burnout—you’ll notice when someone has too much on their plate, is blocked, or if their productivity has suddenly diminished, all of which can prompt you to provide timely assistance.

I’ve been researching goal-setting and prioritization frameworks recently, familiar concepts like SMART, RICE, and SWOT. I mean, there are a lot out there!. But one trend I noticed is that many of these approaches need to be simplified if a team wants to reduce time to success and maximize productivity.

Do you know how hard it is to get people out of their particular habits, ways of working, and day-to-day idiosyncrasies? It's not fun. So how can I get them to eat the RICE when they need to plug in their pressure cooker first, or learn how to operate it?

That’s why I’m going back to basics, starting with the simplest, most straightforward approach: prioritizing daily work. Rather than using a framework, I’m framing priorities before I work.

Frame priorities before work

Before you do anything else, get in the right mindset every day. Ask yourself something straightforward, for instance:

What’s urgent, and what do I need to do now?

I try to use this mental model:

Time-sensitive > Multiple categories of users/teams impacted > Users impacted > My department impacted > My team impacted > Only myself impacted

I have less reason to prioritize a task the further I move right when there’s work in a category to the left.

As you set tasks, ask yourself:

Is this really what I should be doing now? Why?

Notice how this approach is still influenced by goals. While you should ideally have planned your goals before prioritizing your work, my aim is to simplify organizing tasks, so it’s easier for me to shed the deadwood later.

In other words, rather than asking you to make sudden changes to your routine and set new goals, which you’re less likely to do, I’m proposing you keep your pace but clear some of the fog by adding organization and clarity to your schedule.

Urgency vs. importance

First, stop fooling yourself that every fantastic idea that comes to mind is urgent or important. We like to feel productive and to keep ourselves busy—ideally visibly so. I like this quote by Tim Ferris:

“Being busy is a form of laziness—lazy thinking and indiscriminate action.”

The bottom line is that you want to be relaxed, focused, and working on the most impactful initiatives daily. In the words of our very own Mo, Hotjar’s CEO:

“Everyone, working on the most important things, every day.”

Mo also says, “If everything is important, then nothing is.” We need to differentiate between urgent, important, and stuff we just shouldn’t do. As an aid, you can use a method like the Eisenhower Matrix, as I am:

The Eisenhower Matrix (source)

Do first: tasks that are important for your life and career and need to be done today or tomorrow at the latest
Schedule: important tasks but less urgent—list tasks that you need to put in your calendar here
Delegate: less important to you than they are to others, usually ad-hoc. Keep track of delegated tasks by email, phone, or within a meeting to check back on their progress later.
Don’t do: initiatives that are unnecessary and unimportant, and you should not be doing them at all

At its core, the Urgent vs. Important matrix, as it’s also called, tries to help you finish tasks rather than collect them. Remember: you should only list tasks you will complete within the next couple of days or this week.

Here are some templates for this matrix in some of the common project tools I’ve seen folks use here at Hotjar:

Next steps

In my next post, I’ll discuss how you can further differentiate between urgent and important tasks (such as the ABCDE method). I’ll also bring some learnings about how my usage of the Eisenhower Matrix helps or hinders me moving forward.

I hope we can make one another more efficient and happier as we remove unnecessary weight from our days. Feel free to share your experiences and criticisms below. Developing more data-driven approaches is important, but first, we must defog what we already do every day.

Monorepos - pnpm: a performant npm

Mario Beltrán — Tue, 27 Dec 2022 17:00:00 +0000

By the end of 2020, npm released the v7 of their package manager. The most noticeable change was that it now offered support for workspaces. That meant you could finally manage a monorepo with the Node built-in package manager—without the need for extra steps or tools.

At Hotjar, a platform that gives people insights into user and customer behavior, we decided to switch back from yarn to npm to simplify our tooling—so, npm would become our package manager again, now that it gave our team the ability to our team manage a monorepo.

But, it didn’t take long for us to realize that it probably wasn’t the best decision.

What is npm workspaces?

Npm workspaces provide the tooling you need to manage your dependencies in a monorepo. The setup is pretty simple with a new property in your package.json:

// package.json
{
  "name": "workspace-example",
  "workspaces": [
    "./packages/*"
  ]
}

It automates the installation and linking process, and there’s a special --workspace flag to run commands in the context of monorepo projects.

# example: run the tests for the workspace "foo"
npm run test --workspace foo

When Hotjar started using npm workspaces, we had less than 20 workspaces in our monorepo. Today, we have more than 50!

So, we realized npm wasn’t a scalable monorepo tool—the hard way. Our CI started taking way too long (where we got to the point of surpassing our max average runtime), the node_modules size got out of hand, and a simple npm install became a dense daily process. We actually weren’t adding that many dependencies to our monorepo, but working with the workspaces and installing their dependencies became tiring and tedious.

For reference: our node_modules size took 3.2GB of space disk, and installing the dependencies on CI took around 4 minutes. This wasn’t desirable at all, so we started digging into how npm CLI works.

The disadvantages of npm

Npm docs are great in general. However, when it comes to understanding how their CLI works under the hood, you’re going in blind. After many undesirable hours of research, we came to the conclusion that the npm workspace was a great approach for simple monorepos, but not for massive repos like ours at Hotjar. Here’s why:

Npm holds a copy of a dependency every time it’s installed: if we have 100 projects using a dependency, we’ll have 100 copies of that dependency saved on disk. This doesn’t work exactly like that for monorepos, since npm can dedupe dependencies with the same version, but it’ll still duplicate some sub-dependencies with different versions across the dependencies tree.
Npm has blocking stages of installation: for example, it can’t parallelize the installation, so each dependency follows the Resolving → Fetching → Writing flow, and each dependency must finish one stage before we can start another
Npm doesn’t have a proper mechanism to realize a dependency is another workspace from your monorepo: this means that every time you want to install your dependencies, npm will request such a dependency to the registry, then will try to link it locally when no dependency was found there.
Npm hoists dependencies you install to the root of the modules directory: as a result, the source code has access to dependencies that are not added as dependencies to the project. This is known as ‘Phantom Dependencies’ and might break your dependencies or code in several unexpected ways.

A performant npm

The way npm works makes it complicated (if not impossible) to escalate properly for workspaces, causing our dependencies process to slow down, and become unmanageable and unsafe. That’s why Hotjar needed to find a faster, lighter, and safer alternative to npm that also scaled properly for our monorepo—so adding more workspaces didn’t degrade the whole dependencies installation process.

‘Performant npm’ (Pnpm), which is sponsored by Vercel and Prisma and is making a name for itself within the Node ecosystem, is an alternative to npm that caught Hotjar’s special attention. With its promise of being ‘fast, efficient, strict, and supporting monorepos’, it looked like the perfect package manager candidate.

You can see a full benchmark comparison of JavaScript package managers here.

Switching to fast monorepos

After running some tests, we estimated that the improvement in terms of installation, speed, and disk space usage would be massive. It was time to leave npm behind again, so we got down to work. We spent a few weeks working on this migration, and then finally made the switch to pnpm.

The setup is quite simple too:

# pnpm-workspace.yaml
packages:
- 'packages/*'

The whole installation and linking are intelligently automated, and you can run all commands in the context of workspaces with the --filter flag:

# example: run the tests for the workspace "foo"
pnpm --filter foo test

Pnpm also provides advanced features like workspace protocol, or doing partial installs for your workspaces, giving you granular control over your monorepo.

However, the best improvement came in the form of performance. Let’s take a look at what changed after Hotjar made the switch:

Disk space comparison

	NPM (before)	PNPM (after)	Δ delta
Docker image size	4.08GB	2.54GB	37% ⬇️
node_modules size	3.2GB	1.3GB	59% ⬇️

Installation time comparison (in local dev env)

	NPM (before)	PNPM (after)	Δ delta
without cache, without node_modules	4min 50sec	1min 30sec	69% ⬇️
with cache, without node_modules	5min	48sec	84% ⬇️
with cache and node_modules	1min	8sec	86% ⬇️

The result? We eventually got our CI back into its average runtime.

We couldn’t be happier about pnpm and can’t recommend it enough as we continue to use it—it makes your monorepo scalable and maintainable without adding any complexity. Even if you have a small or simple monorepo, you can take advantage of pnpm speed installation improvements. In our next post, we’ll talk about Phantom Dependencies and how they complicated our migration to pnpm.

Cover photo by Mak on Unsplash

Importance of Lazy Loading content

Ryan Calleja — Mon, 12 Dec 2022 09:00:00 +0000

What is lazy loading?

Lazy loading or loading content on demand is the process of identifying resources that are non-blocking to a website and delaying their loading or initialization until the page needs them. Common resources to lazy load include:

Bundle JS files
Vendor or third-party JS files
CSS files
Images

Reused image from AdobeStock

This article outlines the benefits of loading content on demand, how to do it, and when the technique should be applied. We’ll also share how we use lazy loading at Hotjar.

A quick caveat to manage expectations: in providing these tips, we assume all essential performance-related implementations already exist on your website and that, like us, your goal is to optimize page load time and Largest Contentful Paint (LCP) further. That said, let’s dive in.

Why is lazy loading important?

You’ve done everything right: content delivery network (CDN) set-up, file compression enablement, static JS and CSS file minification, and resource caching. You’ve been sure to follow good coding standards and promote the reusability of components. Now, you’re monitoring your website’s performance and hoping for the best.

But even after all this work, your page still takes too long to load.

You measure your loading time using a performance monitoring tool. As you suspected, it returns a poor score. But the problem is that none of your website resources can be removed—they're all crucial elements of your site in one way or another.

This is when lazy loading comes in.

Reused image from My Hero project

Four key benefits of lazy loading

Reducing the initial web page load time by reducing the total size of resources downloaded
Conserving the user's bandwidth, especially keeping in mind mobile data plans on mobile devices
Conserving system resources, as requests to the server are made only when needed
Avoiding unnecessary code execution

You might be wondering: don’t I need all the resources included in our website? Is there anything we could load on demand?

Answer: there's almost always something you don't need immediately.

Start by identifying the level of importance of the resources. Important resources might not be ideal to lazy load, such as:

Main JS Bundle
Main CSS Styles
Fonts
Analytics packages that need to be available on the initial website load

When to implement lazy loading

As your website grows, so does the final bundle of JS and CSS files. To use this technique effectively, you need to split the JS bundles and CSS files as much as possible. The traditionally bulky file holding JS, and another hefty file containing CSS, are no longer an option for the modern front-end tech world we live in. Once you’ve completed this critical step, you’re ready to reap the benefits of lazy loading.

Seven scenarios where lazy loading is beneficial:

Loading JS bundles that aren't used on the currently viewed page but are used on other pages of the website
Loading JS components and any library backing them up immediately instead of when that component is viewable on the page
Using a third-party JS library which is only required on a specific page. Such libraries can be for ready-made features, JS utility libraries, animations, analytics, or monitoring purposes.
Loading CSS-style files for the entire website rather than loading only the style files needed for the viewable page
Loading a collection of images that aren't visible to the user until the user scrolls to a particular part of the page
Loading of a resource when a specific DOM event has been triggered, such as resize or click
At a specific state within your website, such as after submitting a form and showing a submission success component that might be animation-heavy

How Hotjar uses lazy loading in a React application

At Hotjar, we constantly strive to keep our website—our most important sales rep—at a solid performance score.

Why do we care so much about performance? Because it results in:

Increased conversion rates
More page views
Higher search engine rankings
Lower server bandwidth costs
Smoother usability on different devices

Our website is a Next.js application. Thus, we rely heavily on the ReactJS library and its available components.

For images, we use the Next.js-provided component, which by default lazy loads automatically. This has meant we no longer have to worry that our website is suffering from overheads not displayed at that moment in time. Instead, images automatically load and display once the scroll position of a specific image reaches the browser's viewport.

Our milestone

When analyzing our resources on page load, Webpack Bundle Analyzer helped us identify a specific external library used by our website that was adding an overhead of 67.3kB on every page refresh. As you can see below, Webpack Bundle Analyzer’s output reveals lottie.js as one of the large libraries immediately downloaded on our page load.

(We also discovered this library was only used at the bottom of the page, to render a nice animation once a subset of components was in the viewport.)

Lazy loader implementation

Our idea was to create a new component (called LazyLoadOnScroll) to wrap other React components. The logic behind it uses a hook that benefits from the browser’s Observer tools, specifically the IntersectionObserver. Any components wrapped with this won’t be rendered, and any underlying external files, libraries, or JS files won’t be downloaded.

Once the user starts to scroll and this component is about to come into view, the IntersectionObserver returns a value of 'true', meaning the component is about to intersect the viewport. At this point in time of scrolling, everything begins downloading, and React renders the component accordingly, making it appear to the user as if items are seamlessly being served on demand.

**
The new files introduced are the following:**

useIntersectionObserver: the hook with the responsibility of observing the provided component ref via an IntersectionObserver instance to return 'true' or 'false' as to whether the ref has intersected the viewport. This can also be extended by providing additional config props, but our use case required only the most basic implementation. This also includes a check to make sure the browser supports IntersectionObserver; otherwise, it will return 'true' to render the content immediately every time.

import { useState, useEffect } from 'react';

/**
 * Providing a useRef to a component, this hook will return a value of true or false, based on whether
 * the ref prop, is visible in the viewport or not
 * @param ref
 * @returns {boolean}
 */
const useIntersectionObserver = (ref) => {
  const [isIntersecting, setIntersecting] = useState(false);

  const supportsObserver = () => {
    return typeof IntersectionObserver !== 'undefined';
  };

  useEffect(() => {
    let observer;
    if (supportsObserver()) {
      observer = new IntersectionObserver(([entry]) => setIntersecting(entry.isIntersecting));
      if (ref && ref.current) {
        observer.observe(ref.current);
      }
    } else {
      setIntersecting(true);
    }

    return () => observer?.disconnect();
  }, [ref]);

  return isIntersecting;
};

export default useIntersectionObserver;

LazyLoadOnScroll: this is a React functional component that uses the useIntersectionObserver hook. It contains a statement checking whether the hook is being run on a server. If it is on a server, Next.js ‘Server Side Renderer’ will present the page to the client. The hook always returns 'true' so as not to lazy load anything, and the full page is generated for SEO purposes. When this component is initiated, it sets the children to render once the hook returns 'true'. The process occurs once, and the component does nothing else but keep the children in the rendered document.

import { useEffect, useRef, useState } from 'react';
import useIntersectionObserver from 'useIntersectionObserver';

/**
 *
 * @param props             Child JSX element
 * @returns {JSX.Element}
 * @constructor
 */
export const LazyLoadOnScroll = (props) => {
  const { children } = props;
  const ref = useRef();
  const isNotBrowser = typeof window === 'undefined';
  const refIntersected = useIntersectionObserver(ref);
  const [visible, setVisible] = useState(isNotBrowser);

  useEffect(() => {
    if (refIntersected) {
      setVisible(refIntersected);
    }
  }, [refIntersected]);
 return <div ref={ref}>{visible && children}</div>;
};

Putting it into practice

Using it is as simple as wrapping any component you want to lazy load. Plus, there’s no need to worry about extra overheads on page load.

<LazyLoadOnScroll key={key}>
  <BigComponentToBeLazyLoaded prop1={} prop2={} etc... />
</LazyLoadOnScroll>

The beauty of measuring

For our performance measurement, we've used Chrome's devtool, Lighthouse, to generate a report before and after implementation. Results were fascinating, and I’m really excited to share the Desktop scores with you.

From an overall score of 87 on performance and an LCP of 1.9 seconds…

…we managed to go up to an overall score of 93 on performance and an LCP of 1.5 seconds.

These results gave us the assurance that loading content on demand does impact our overall website experience. Applying it further to other corners of our site will ensure that no unnecessary overheads block our pages from loading as fast as possible—helping us give our users an experience they’ll love.

5 easy steps for choosing the right goals

Stephanie Mifsud — Mon, 03 Oct 2022 06:13:17 +0000

Thinking about goals you want to achieve in six months can be challenging. Here are some questions and suggestions that can help you find the mental space for paving the way to a better version of yourself.

At Hotjar we have performance reviews every six months. This process generates a lot of peer feedback on one’s strengths and career opportunities—plus it provides a space for thinking about future growth. Susanna has posted some great tips on how to digest this feedback and make it actionable. As peer feedback tends to get more generic as one becomes more senior, this post outlines some resources that are helpful in specific goal setting for the next few months.

I invite you to grab a cuppa and some pen and paper and follow these few steps to identify which goals matter for your career progression.

1. [5m] Brain dump—what’s on your mind?

Our mind is usually cluttered with thoughts about the present or near future. Let’s start by listing those thoughts so we can consider them later.

Make a list of the things on your mind, things you have been postponing, things that worry you because they are close to their breaking point, pending follow-ups, etc. Write it all down. Now put that list aside for now.

2. [15m] Tech laundry list—what needs to be done

Write out (or type out) a laundry list of tasks/projects you think the team needs to work on in the near future.

Think of it as a backlog of tech tasks to bring the code and infra to a better place where you can be proud of how well it serves our users.

You might want to look into the team backlog for inspiration. If you have a North Star document for your team, now is a good time to resurrect it and see what steps you need to take to reach it.

3. [20m] Focus on the destination — keep your eye on the prize

Now it’s time to think about how you want to grow in your current role—or what you need to do to switch roles if you are aiming for that.

Consider these questions:

What are your career aspirations?
What do you want to know in a year that you don’t know now?
What interests you? What articles/books do you read?
Look at the seniority criteria for your current role and your aspired role. What skills do you need to hone to help your growth?
What opportunities came out of your last performance review?

You guessed it, put all these responses in a list, we’ll need it later.

4. [25m] Look for the opportunities — watch for blind spots

After this introspective phase, it’s time to zoom out, to take into consideration higher-level goals. Different companies have different ways of communicating strategy and focus areas. Whether it’s KPIs, OKRs, KPAs or something else, dig up strategic guidance and investigate how it dribbles down to your team and role.

There might also be more horizontal initiatives to become familiar with such as Chapter Group Projects, North Star Initiatives and Golden Path direction.

Here you may see gaps that you are interested in filling, for example. suggesting a new chapter topic or pitching a new initiative for your team in alignment with the Department OKRs. These are excellent material for open-ended goals (see section 5)

5. [30m] Write your plan — pulling it all together

The plan

The follow up of a performance review is a development plan. A development plan is how we describe and track goals for the next six months. It provides a basis for one-on-one conversations on growth and helps your lead highlight practical opportunities for reaching those goals.

A development plan is in essence a set of personal OKRs. It includes a couple of broad goals — think of these as the areas you want to focus on. For each goal, we set two outcomes, where we describe the better situation we want to be in in a few months' time. These outcomes are split into specific actions that will lead to the improved situation we want to achieve.

Broad goals

Take a look at your answers and ideas in numbers three and four. From this, try to come up with two broad and open-ended goals.

The best goals will overlap between what you want to achieve and what the company is trying to achieve. Try to find aligning themes, as it will increase your chances of success.

Examples of broad open-ended goals

Identify opportunities for improvement/innovation in our area of ownership
Improve knowledge sharing and collaboration around data visualization
Invest more time in increasing technical expertise

Outcomes and outputs

Now try to come up with an outcome-based objective. This is probably the hardest part.

So I suggest you try to aim for a direction here, rather than a specific output. This allows for flexibility in your approach—and as time goes by you can adapt the outputs which achieve the outcome if the original ones no longer become outdated.

The list you made in step two will help you here. Your list likely contains actions or tasks, but you can group them if they achieve a common outcome, for example:

Increasing our test coverage by 50%
Working with multiple teams on a shared vision for DataViz
Build awareness to be able to propose a NorthStar direction

Putting it all together

The result will look like this:

Invest more time in topics related to the expertise criteria

Build awareness to be able to propose a North Star direction (outcome-based objective)
Complete a Micro FE course (specific task)
Document modern libraries and how our code can benefit from them (specific task)

If you get stuck in the writing part, just list the open-ended goals and specific tasks, and reach out to your lead or manager to help you put it all together in a coherent way.

If you made it to this point, you might have noticed that you haven’t used the list you made in stage one. Take a look at it again — did any long-term themes surface in your goals? Do you need to tweak anything in your development plan based on this list? Probably not, the aim of this list was mostly to help clear your mind of the imminent needs and focus a little bit more on the long term!

Monorepos - sharding Jest in CI

Mario Beltrán — Tue, 27 Sep 2022 15:47:42 +0000

Monorepos—single repositories that store code and assets for multiple software projects—are trendy in the frontend engineering scene. We at Hotjar are splitting our main app (Insights Webapp) from one huge monolith into small projects within a monorepo. One of the reasons we’ve done this is to make project development more scalable, regardless of the number of areas involved and the engineers working in them.

We mainly instrument our monorepo with pnpm for the dependencies and Nx for the tooling and running tasks. However, orchestrating a monorepo is challenging, and scaling it correctly is more complicated than you’d think.

Running our tests in CI within a monorepo in the optimal amount of time was a complex process that took many iterations to refine.

The old approach

We run the tests only for monorepo workspaces affected by the changes in a branch, so we don’t waste time running tests for code that is not affected at all. Nx is great at this—it identifies the different workspaces of the monorepo affected by the files changed in our PRs.

However, as mentioned earlier, we are in the process of breaking down the monolith into small pieces. We have many small, modern monorepo workspaces (let’s call them modernA, modernB, etc.), and two massive legacy workspaces (we’ll refer to them as legacyX and legacyY).

The runtime difference between modern and legacy tests is massive: the former have only a few tests that are specific to their scope, while the latter have hundreds of tests still covering many areas owned by various squads.

So, this was our approach: we parallelized our test runs in six jobs within GitLab CI, distributing the tests by workspaces. Since legacyX and legacyY are the slowest test runs—and almost always affected by Nx (because they still contain the core and shell of the webapp)—we isolate them in their own jobs. Then, we have four jobs remaining for the rest of the modern workspaces affected by the PR changes.

With this in mind, let’s assume a PR where our changes affected legacyX, legacyY, modernA, modernB, modernC, modernD, modernE, modernF, modernG, and modernH. The tests would be grouped like this in CI jobs:

legacyX
legacyY
modernA, modernB
modernC, modernD
modernE, modernF
modernG, modernH

So far, so good. The problem comes when we take a look at the runtimes in CI:

We can see that jobs 3-6 are reasonably fast, but 1 and 2 became a bottleneck! We can’t break them down any further (to do that, we need to keep extracting functionality into its own workspace). Our CI won’t be completed until the slowest test job is finished. So, in this case, our CI will take eight minutes, no matter how fast the remaining workspace tests are.

This is because all our monorepo workspaces contain different numbers of tests–jobs 1 and 2 are running many, many tests for the legacy workspaces, while the others are running relatively small modern workspaces with fewer tests.

The new approach

Our test framework and runner for JS/TS is Jest. In v28, they introduced a new feature for sharding test runs: Jest now includes a new --shard CLI option that allows you to run parts of your test across different machines—it’s one of Jest’s oldest feature requests.

This feature is handy for distributing our test suites in homogeneous chunks across our parallel jobs. We were already running tests split by workspace through Nx, but this didn’t give us granular control over how many tests we could run per job. Some ran hundreds of tests (e.g., the legacy workspaces), taking eight minutes to complete; others executed fewer than 20 tests (e.g., recently-created modern workspaces), taking only a minute to complete.

Although the tests are no longer grouped by monorepo workspaces on each job, we still keep the ability to run the tests only for affected workspaces. Thanks to the Nx print-affected command, we can use the list of projects affected to manually filter workspaces that need to be checked.

With this new approach in mind, and the same PR used as an example above, the tests would be distributed in CI jobs like this:

legacyX 1/6, legacyY 1/6, modernA 1/6, modernB 1/6, modernC 1/6, modernD 1/6, modernE 1/6, modernF 1/6, modernG 1/6, modernH 1/6
legacyX 2/6, legacyY 2/6, modernA 2/6, modernB 2/6, modernC 2/6, modernD 2/6, modernE 2/6, modernF 2/6, modernG 2/6, modernH 2/6
...
...
...
legacyX 6/6, legacyY 6/6, modernA 6/6, modernB 6/6, modernC 6/6, modernD 6/6, modernE 6/6, modernF 6/6, modernG 6/6, modernH 6/6

This leads to a pipeline result like the one below:

The runtimes are more consistent simply because we mixed tests from all the monorepo workspaces together! Now, the slowest test job takes just six minutes, saving us a solid two minutes from the CI runtime.

These numbers are not just examples—they’re real runtimes from our CI. This, in particular, was our worst-case scenario: running the tests for all the workspaces in our monorepo. For the average scenario (seven to eight workspaces affected, including the two legacy ones), we can save almost four minutes per pipeline—a potential save of 50%!

The Jest shard feature didn’t just make our tests’ runtime more consistent in CI; it also made it more scalable. Now, it doesn’t matter if our tests grow by the number of tests per workspace, or by new workspaces introduced to the monorepo: they’ll be distributed consistently between available parallel jobs.

This is a huge win, especially compared to the previous approach. In the past, we would always have a job taking eight minutes to complete, and the remaining jobs taking different, haphazard completion times on each PR, according to the number of tests assigned.

This new approach was also surprisingly easy to set up in GitLab CI with a custom script, which looks like this:

// calculate-affected-projects.js (snippet simplified for convenience)
const exec = require('node:child_process').exec;

exec('nx print-affected --target=test', (err, stdout) => {
    // do something if `err`

  const affectedProjects = JSON.parse(stdout.toString().trim()).projects;
  const filterAffectedProjects = affectedProjects.map((projectName) => `--filter ${projectName}`).join(' ');

  console.log(filterAffectedProjects);
});

# .gitlab-cy.yml (snippet simplified for convenience)
js-tests:
  parallel: 6
  script: 
    - export FILTER_AFFECTED_PROJECTS=$(node calculate-affected-projects.js)
    - pnpm --no-bail --if-present $FILTER_AFFECTED_PROJECTS test --ci --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

Trade-offs

Everything that glitters isn’t gold, though! This new approach with Jest shard has some implications worth mentioning:

Challenging upgrade to Jest v28

This is dependent on your codebase and your current version of Jest. At the time, we were still on Jest v26. Upgrading two majors was hard, not only because of the breaking changes introduced in v27 and v28, but also due to Jest and js-dom working more closely to a real browser on every upgrade. This led to flaky tests, side effects, and unexpected behaviors during the upgrade that were complicated to debug and fix. The result was worth it, but if you’re a few majors behind Jest v28, remember: you may need to put in more effort than you think.

Inability to take advantage of Nx computation cache

Nx uses a computation cache internally, so it never rebuilds or re-runs the same code twice. This can save a lot of time if the affected monorepo workspaces didn’t change, so Nx can directly output the previous test results.

But with the new Jest shard approach, we can’t take advantage of it. This isn’t a huge deal for us because we weren’t using this functionality yet. Even if we did, the two legacies’ monorepo workspaces are constantly changing, invalidating the computation cache. It was easy for us to ditch the feature, but perhaps you rely on it heavily. If so, we suggest measuring the impact of using Nx computation cache vs. Jest shard for running your tests before changing the way you run them on CI.

Lack of coverage threshold

For us, this has been the biggest downside: the Jest shard option is not compatible with the global coverage threshold. Surprisingly, this is not mentioned anywhere in the docs, but there is this issue open in their repo. Hopefully, this feature will become compatible with the shard flag soon. In the meantime, we’ve had to disable the global coverage threshold while we explore other ways to set it.

Longer total computed runtime on CI

If you compare the total time spent by the six jobs in the old approach (~23 minutes) and the new approach (~30 minutes), the former took less computation time from our CI. This is not causing a significant impact for us, but it could in other cases, especially if you have a tightly restricted minutes-per-month plan to run your CI.

That’s it! We hope you found this post interesting—not just because of the new Jest shard feature, but also the learnings around our take on monorepos and the trade-offs that come with different approaches.

Cover photo by Marcin Jozwiak on Unsplash

Best-self review: how to process feedback in an actionable way

Susana Martins — Tue, 13 Sep 2022 08:49:10 +0000

Every six months at Hotjar, employees have their best-self review, which consists of a self and peer review. The review forces us all to reflect on two things: how well we’re living up to Hotjar’s core values and how we’re performing against our seniority criteria. We’re required to give examples of things we’ve accomplished during the period, and also think about how we can improve and continue to grow. Our peers are also given an opportunity to share their thoughts and feedback. Finally, we discuss these results at a performance review meeting with our lead.

Although this time can feel tense and high-pressure, ultimately, it helps us grow and develop our skills. To that end, it’s every Hotjarian’s responsibility to find ways of processing all the information given to us in a way that translates to concrete actions or steps we can take to achieve our goals. Of course, this is easier said than done.

I want to share my personal process and experience, and what works for me. Each person is different, so my method may not necessarily be beneficial for everyone—but it’s something to consider if you, like many others, dread the review cycle and find it difficult to comprehend and respond to feedback received in the moment.

Changing my approach to reviews

I’m a nervous and anxious person. The anticipation of my lead sharing feedback with me is inevitably very stressful. In the past, I’ve been guilty of joining the meeting, sitting there, and partially listening—while at the same time trying to read the feedback cards and managing my anxiety. Because I’m receiving the info at that moment, I don’t have much to say or comment on because I don’t have the capacity to process it on the spot effectively.

This is not a very useful way to spend my and my lead’s time. So we decided to try something different: in my last review, my lead shared the feedback with me a few days before we met, giving me the time to process this information in my own way.

It worked.

For the first time, I was the one doing the majority of the talking in my review. I was the one presenting the feedback and ideas for changes I could make to achieve my goals to my lead—not the other way around.

I felt more proactive, and the time was more productive: I could ask about specific aspects that were unclear to me, or focus on things I wanted to improve but needed more guidance on how.

I organized all the information on a Miro board, developing this template:

A breakdown of my Miro

My board consists of five sections:

1. To improve

Here, I organized the big topics or themes—things that I repeatedly mentioned—on bigger green cards, and placed the specifics around them on smaller yellow cards.

For example: a green card might say impact the company. In turn, its yellow cards might say lead a Chapter Week initiative or share more learnings in common channels.

I used arrows to help me correlate between them and added a few notes where necessary.

2. To discuss with peers

Sometimes, feedback can be unexpected or unclear. It’s crucial to clarify exactly what my peers are trying to say so I can actually do something about it. I added those cards so I could meet with my colleagues and have these discussions directly.

3. To continue

The good stuff!😊 Even though we tend to focus on what we need to improve, it’s good to reflect on the things we’re good at so we can keep doing them. It’s also something that makes us feel good, so we shouldn’t ignore our strengths!

4. Goals

This is a big one: if we don’t know what we want to achieve, it’s harder to come up with concrete action items. It’s important to work towards a goal, even if that goal changes in time (which is both natural and expected, especially with long-term goals).

It helps to select the most important aspects from the review and focus on these. So I added two sections: one for the short-term goal (for example, promotion to level X in 6 months) and another for long-term, looking two or three years into the future.

5. Actions

This is where I added concrete actions to take based on the green and yellow cards from the “To improve” section.

I kept the green ones (the themes) and added blue cards for each action I wanted to take. For example, related to the impact the company example I mentioned earlier, I could add lead the next Chapter Week initiative on [this date]. As you can see, this is similar to the yellow cards, but it’s more concrete. It’s also filtered because not all the things mentioned in the yellow cards resulted in actions for this specific time.

While populating these sections, I left comments on things I wasn’t sure of or wanted to discuss with my lead, and we focused on these during our meeting time. I found this approach incredibly helpful. Unlike previous meetings, which were often spent on things I already knew how to address, or weren’t that relevant for what I wanted to achieve, this meeting ended with my questions answered and a clear set of actions to take or ideas to refine. And that’s exactly how it should be—after all, the purpose of the review is to help me grow.

Making my review work for me

Since my last review, I’ve bookmarked the Miro board, and I’m marking progress on my action items. Every now and then, I revisit it to see what I’ve already achieved and what I can still work on before my next review. I also find this helps me organize my personal development time, and I can keep this information in mind when I write my next self-review.

The feeling that I’m directly addressing the feedback I receive during the review instead of taking random actions has been a game-changer. I have a completely different attitude towards my review. While the nerves and anxiety haven’t completely dissipated, I feel much more comfortable—and even confident. Maybe this approach could have the same impact on someone else.

Prioritizing brilliantly: better alternatives to grind and guilt

Stephanie Mifsud — Thu, 25 Aug 2022 15:27:00 +0000

As software engineers, we’re always in pursuit of high-quality solutions against a ticking clock. But spending too much time perfecting solutions before delivery increases cost and delays learnings.

What’s more, the standard approach of grinding to the finish line is not sustainable. I quickly realised I had to learn how to pace myself for a marathon, not a sprint, while also enjoying the views. In other words, I needed to work smarter rather than harder.

In this post, I would like to share a few engineering-adjacent skills I’ve found useful for a successful career as a Senior Engineer and Team Lead.

What I learned (and how)

At Hotjar we use industry benchmarks that set elite performers apart from the rest. These benchmarks favour an iterative approach to learning quickly, reducing waste, and keeping up the momentum.

I’ve also picked up a few tips from my mentors, who inspire me to be intentional in my work and align my efforts in the same direction as my team or company. The book Coherence by Dr. Alan Watkins has also been a great inspiration during this ongoing journey.

Both these sources have helped me develop a few valuable ideas of my own.

Align your goals

One of my major time wasters was flitting from one thing to another without first gauging the importance of each task. I needed a mental representation of the priority areas, to differentiate between what was a high-impact activity for my goals and what was merely a distraction. I started by combining all my goals for the quarter.

It’s all too easy to be pulled into different directions by:

OKRs
Tech briefs and guidance
Performance Review opportunities
My own interests and goals

So now, I put all these goals on one list. Then, I pick a theme to focus on for the month. The theme helps me prioritise activities that increase the impact of my work—especially if some groundwork is required before focusing on a particular area. For example, if the theme is to tackle tech debt, it also helps to look at the north star direction.

For an individual contributor, these goals may be squad-related, chapter-related, north star-related, etc. Maintaining a laundry list of bottlenecks or implementations reaching a stress point also helps allocate the right amount of effort. Some of these areas might be aligned with the squad’s current focus area, and thus tech debt can be addressed as preparation for feature work. If not, laundry lists at the very least make good material for a proposal for upcoming focus areas.

Alternate between breadth and depth

Goals and themes require frequent alignment and refinement to continue serving their purpose. I often ask questions when our decisions seem to be drifting away from our original squad goals. As we update our direction with learnings and results from data, it helps the team align and evolve their understanding of what’s next. Then we shift our focus to the tasks in our backlog and identify what’s missing for us to reach the goal we’ve identified.

We’ve reached a point several times where we know in our hearts that we are not going to manage the original target we set. Or perhaps we’ve gained new knowledge that changes our approach. This is either an invitation to grind to the original deadline or rise to the challenge of slicing the deliverable differently.

Some questions I ask here are:

What resources do I have at my disposal to help me de-risk this work?
Can I spike the riskiest part to check for feasibility?
What is the minimum work I can do to get the whole system working from end to end?
Will excluding part X from this work still make it a successful release?

Slicing work into smaller deliverables maintains momentum and ensures timely feedback, so a pivot will not lead to a lot of waste. (Sometimes we even replace the original task with one of higher value to our customers as we adapt to new learnings.)

Use sync-meeting agendas for broad-picture thinking

The switch between breadth and depth is costly—and sometimes painful—because the brain wants to return to its previous train of thought. I like to use sync meetings as a time to address this: syncs can help switch focus faster. If I know a planning meeting is coming up, I spend 30 minutes preparing my contributions. The end of a sync meeting is also an excellent time to revise what was decided and identify any gaps or misalignments.

Ask for feedback

After aligning on the right priorities and keeping a bird's-eye view on the end goal, the next time-suck I realized I needed to avoid was perfection. Contrary to popular belief, perfection is more a vice than a virtue! I remember a time when I used to polish and perfect my work before asking for feedback, mostly out of fear that my work would appear sloppy. However, getting someone to give me feedback on the initial direction, scope, and size proved extremely helpful in structuring my thoughts.

When a draft is spending too much time gathering dust on my desk, I ask my lead or peers whether it’s valuable to continue pursuing it.

Set the right expectations

Naming conventions matter to help set the right expectations from reviewers. Here are a few examples:

Artifact name	Feedback expectations
A brain dump	High-level feedback is required to validate the direction of ideas.
DRAFT: Merge Request	The reviewer will not nitpick with the incomplete parts but instead provide direction on boundaries and code structure.
A proposal or RFC	Needs to be well thought out, and address a clear business case, to receive feedback on specifics.

Iterate quickly

Perfectionism has value when it comes to the polishing stage of an article or deliverable. However, the prior WIP stages are how we work together to achieve a higher impact. Hiding my work simply because it wasn’t perfect yet was not helping me. Iteration is more powerful than perfectionism because it refines the deliverable to the right audience, whether it is the company or the customer.

Energy Management

All the strategies I’ve discussed continue to help me reduce waste and distraction—but being focused still takes up a lot of my energy.

Energy management is more impactful than time management. Having hard stops in my day allows me energy for activities that make my life more meaningful—like spending time with friends, family, and pets. This increases my motivation and sense of self overall. And restoring myself before I crash helps me work more consistently.

Crashing is expensive; it takes time to recover from. I’m still learning to notice the signs I’m slowing down and find healthy pick-me-ups.

I’ve also redefined success to represent my whole person—I’m not simply a brain attached to a body. Our life is so digital that sometimes our bodies become an afterthought. We eat and move on autopilot. Being intentional about improving my physical and mental strength has made my energy more consistent and reliable, helping me do better at work and in my personal life.

Hotjar's migration from AngularJS to React - Our approach

Steve Pitchford — Thu, 30 Sep 2021 09:13:24 +0000

Hotjar is a tool that helps people understand how their users are behaving on their site, what they need, and how they feel. You can find out more about Hotjar and the services we provide at hotjar.com.

From a technical perspective, we provide a rich single-page application (SPA) for displaying data collected from our user’s sites and data provided by their users. Our application was initially written in AngularJS as far back as early 2014. As a bootstrapped startup, the first version of the application was created by a very small team of full-stack engineers. As the company matured, we switched to more specialised roles and now we have a dedicated team of 26+ Frontend Engineers and continuing to grow.

Reasons to migrate away from AngularJS

AngularJS is an older framework that didn’t keep up with modern development practices. Features like lazy loading aren’t very easy to implement and require modification to the application to get it working.
AngularJS is reaching its end of life and will no longer get support.
Due to the framework reaching the end of life it is becoming progressively harder to find developers with recent experience in this framework, due mainly to the two points above.
A lot of lessons were learned from the implementation of AngularJS and these problems were addressed in other frameworks. We want to leverage these better design patterns to make our application scale and easier to maintain.

Why React?

We discussed several framework options including Angular, Vue, Ember, and React.

Angular didn’t feel like the right fit for most developers in the team despite there being a more defined migration path for AngularJS -> Angular. The reason it didn’t feel like a good fit for us as a team was that we felt that the opinionated nature of Angular didn’t align with our goal of allowing teams autonomy in how they develop features and that it would be too restrictive.

Vue was still somewhat new at the time, and no one in the team at the time had any experience using Vue.

Ember is a powerful framework but, as with Vue, no one had any experience using it.

We had developers who had used React in the past and were familiar with the design patterns used in React. Given React’s popularity and community, we wanted to leverage this for both JS libraries and for the pool of developers that we could hire from that already had lots of experience with React.

Our React setup

We opted not to go for a full Redux app setup since we wanted to keep the state local and avoid oversharing the state between different parts of the application if it wasn’t needed, this encourages teams to work independently from each other. We preferred to pull the state up the component tree when needed.

We use TypeScript with React as we find it adds an extra layer of safety to our development. It takes longer to set up components but the payoff exceeds the extra work. We however do have problems with the React/AngularJS boundary since the AngularJS code is not in TypeScript. This means we lose our type safety when passing data to and from the AngularJS application.

We use react-query to handle caching of API requests to avoid over-fetching of data from the backend which, in a way, acts as a global store. Since the query cache is immutable and all changes trigger updates within components we need to worry less about the state being modified in an unexpected way by some other part of the app.

Complete rewrite vs Incremental migration

Complete rewrite

PROS:

A complete rewrite is great because you can ship a new shiny application to your end-users when you finish.
You can develop the application with a clean architecture since you don’t have to carry around any baggage from the legacy application.

CONS:

You have to either halt the development of new features to your customers, or you need to develop features twice so that the new application keeps feature parity with the older one.
You are more prone to introducing regressions and bugs since you are writing tests from scratch and don’t have existing test suites you can leverage to ensure that flows continue to work as expected.
Micro frontends may have solved some of these issues. However, using micro frontends within the AngularJS app isn’t trivial and would still have required rewrites of entire pages or sections of the application. This would still require halting the development of new features while the migration happened.
It is hard to estimate the size of a complete rewrite since there are usually a lot of unknowns.
It usually comes with a huge initial cost of designing the architecture of the application and making application-wide decisions before we even start coding; then when the plan meets reality, it needs to be adjusted and you either end up with an inconsistent codebase or rewrite parts over and over again; this might be the case for the alternative approach as well, though.

Incremental migration

PROS:

You can develop features at the same time as code is migrated to a new framework. This is great because customers continue to get new value.
You can leverage existing test suites such as end-to-end tests to ensure that features still work as expected.
It is easier to size the work needed to migrate since migration work can be broken down into much smaller defined tasks.
It gives you time and opportunity to adjust the approach or architecture over time, evaluate how your ideas work in practice, and change them along the way; it’s still possible with a complete rewrite but it may be more difficult there; with incremental migration, the initial cost is pretty low and you’re not even required of making decisions beforehand - you do it only when you really need it.

CONS:

You carry around a lot of baggage and dependencies from the old codebase as parts of the application may still depend on code that lives in the old codebase; this may have a performance cost for the end-user.
You can’t easily implement new designs into your code since it needs to match the rest of the legacy application.
It takes significantly longer to migrate this way since we have to introduce some workarounds to get data passed between the different frameworks.

We chose to take the incremental approach since at the start of the migration we didn’t have a very big frontend team and we wanted to be able to continue delivering features to our customers.

Despite the drawbacks of incremental migration, we feel like this was a good choice for us as a company and that it has paid off in the long run.

React-2-angular

To approach an incremental migration we needed a way to embed react components within the AngularJS application. We found the great library react-2-angular that lets us create React components that can be embedded within an AngularJS page. This also allows us to pass in dependencies to the react components as props so we can leverage our existing AngularJS services to pass data and API function calls to our react components.

Migrating UI First and Design Systems

It was decided to first attempt to start migrating the UI layer of the application to React while keeping state and some UI logic in AngularJS. This meant we didn’t have to migrate some of our really large AngularJS controllers that had grown over the years and never been nicely split into smaller components. It also had the benefit of allowing us to build pure(ish) components that largely didn’t handle any state, except for maybe state used by controlled inputs. This, in turn, enabled us to get the UI of the app to a point where it is more easily modified across the app while teams work on migrating the controller and service layers of the application.

At a certain point in the migration process, the company decided to create a Design System implemented in React that would standardize all the common UI patterns used through the app. This made the migration of the UI layer even easier since large sections of the UI could be constructed using components from the Design System.

Up until this point, teams had been building reusable components within the project and we didn’t want to throw these away, so these components we used to identify common patterns. We were also able to take these reused components and copy them into the Design System and give them clearly defined type definitions and make them consistent with the Design Language

How Does React Work? Brief History - PART 1

Adam Płócieniak — Thu, 12 Aug 2021 07:19:16 +0000

The article’s goal is to shed some light on the framework internals and historical insights standing behind the implementation decision made by the React Team at the time. I assume you are already aware of basic JavaScript / React and JSX concepts. So let’s kick off with some history first.

It’s 2013. The React framework is born (version 0.3.0 is rolled out to the public) and it’s immediately loved by the community. It stands out with its simplicity, one-way-data-binding, and - what’s obvious - declarative API powered by original, attractive to the community syntax extension, JSX.

React’s JSX representation of an example app

The following months and years bring new releases; bugs are fixed, features and improvements are added. The community grows, the tooling ecosystem supporting its development flourishes. React Native is embraced by mobile native developers, which brings even more popularity to the framework. But there is one thing around the framework that does not change at all. Its internal logic (so-called Reconciliation Algorithm) is responsible for all of the “magic” - starting from when an application's state changes until direct DOM updates are performed.

Input and output of the Reconciliation algorithm

Briefly, here’s how it works:

(1) Every state change (e.g., clicking “Buy product” from the example application) forces building a so-called Virtual DOM, representing the current state of the application. It's a structure of components.

(2) Then, the newly created Virtual DOM tree is compared with a previously generated Virtual DOM tree representing the application’s state already displayed to a user. Discrepancies between those Virtual DOM structures are simple information, what we should change, e.g.:

update attribute X for A element,
remove element B,
or append element C as a child of element A.

(3) The information is utilized by separate modules - renderers (for browsers, it’s react-dom) which applies the minimal set of changes necessary to update UI (the DOM nodes for browsers).

And that’s how React works, briefly.

But how is the Virtual DOM built? If we had to write the function responsible for that on our own, it might look like:

High-Level Pseudocode of React’s Reconciliation Algorithm

The render function drills down through the entire React Elements structure (Virtual DOM) and reconciles (or works on) every element.

Let’s stop for a moment here. What is a React Element? A React Element is a simple building block for our application. It’s what we return from our components, like <Loader /> in our example application. It’s worth mentioning that React Elements are not DOM Elements. Whenever I refer to an “element” below in the article, I mean React Element. If we talk about DOM elements, I will explicitly use the “DOM” to avoid misunderstanding.

Back to the reconciliation. Reconciling here means doing some element-related work. For component elements, it includes:

Invoking the render() method (for class-based components) or calling functional component’s function reference with given props,
managing internal state,
invoking Lifecycle Methods (e.g., getDerrivedState),
marking changes that need to be applied later on,
and many more.

By calling render(<App />) (I recommend getting familiar with our example application specified above, we will use this structure later on), we re-create the tree structure from top to bottom, using render function:

Starting from render(App), we work on the App component,
then we have some work to do with its child (Content),
then with its child’s children: Loader, Product,
and so on, until we reach the last leaf of the structure of the elements.

It’s 2016. Everyone is celebrating the framework's 3rd birthday (version 15.0.0 is released), but its parents (React Core Team with its lead, Sebastian Markbåge - we will talk about him later in the article) are slightly worried about the future of the framework. But is there a reason to be concerned?

It appears that the React has some “innate heart disease”, which limits its organic growth. Let’s have a brief look:

High-Level pseudocode of React’s core algorithm function vs. Fibonacci function

What’s common among render() and fib() functions?

You’re right. It's a recursion. The heart of the React framework relies strongly on recursion. But is it a problem at all?

Web browsers are equipped with a single thread. We can do one operation at a time, so React operates in a limited environment. Although computing fib(4) is not a challenge at all, computing fib(4000) definitely is. Same for the Reconciliation Algorithm - building a Virtual DOM based on a thousand elements is a real challenge. Synchronous code execution blocks the main thread, so JavaScript’s event loop has to wait until the end of execution. During that time, none of the following activities can be performed:

User input (e.g., handling user click event callback)
Animations, layout calculations, repaints
Handle incoming data (HTTP, WebSocket)

There is a brilliant talk about event loop here, so if you are not familiar with it or need a refresher, it’s definitely worth watching.

Let’s talk now about the JS Stack; how does it look for both the fib and render functions?

Fibonacci function stack frames vs. Render function stack frames

The JS Stack grows as we move deeper in the structure, so the process simply can’t be paused because there is no straightforward way to do that in a recursion. We reconcile all of the elements in one shot or none at all. What’s more, React's computation output is pushed onto the JS stack, so it's ditched immediately after the render function returns. There is no way to reuse this work later on if it’s needed.

Imagine a case of a heavy application with a massive number of components. We are in the middle of the Reconciliation Algorithm, and a user clicks a button. Such action is critical from a UX standpoint and should be handled immediately. But what happens?

Clicking dispatches a DOM event.
The event callback lands in a queue and waits (until the JS Stack is empty) to be processed.
But the JS stack is “overwhelmed” by heavy React-related work, so the event callback waits…, waits, and waits for its turn until the Reconciliation Algorithm is done.

(source)

There is an excellent Sierpinski triangle example application on Github. It is a more tangible showcase of the problem. Keep in mind that it’s 2016, so the application is built on top of React 15.x. Here is how the application looks like:

Sierpinski triangle example application view

Each dot is a component displaying a number. Among state updates, there are a lot of other computations, including:

animations (layout computations, painting),
deliberate delays,
a whole bunch of artificial and pointless state changes.

All of these simulate a heavy application. And here is the application. Pretty sluggish, huh? Here’s how the top of the JS Stack looks like (I recommend watching this short video).

React 15.x JS Stack (source)

A synchronous and time-consuming function (in the Sierpinski’s triangle example, for my equipment, each “Task” takes ~300ms) reconciles the entire application from the top to the bottom of the elements tree, no matter what.

The framework here is relentless. It overwhelms the main thread, which can’t perform any other types of work (animations, user’s input). It introduces a significant impact on the browser’s performance. It’s hard to build on the top of such architecture, isn’t it?

And this is a huge limitation for the React Team.

Of course, this is an example application built to show the problem. But we can quickly observe such pitfalls when the number of components grows in our real-world applications. Long and heavy lists are a perfect example here. It’s also why the helper libraries, like react-virtualized (or its lightweight version, react-window), emerged and gained noticeable popularity by supporting the framework. Those libraries are officially recommended on React docs pages.

It’s also worth noticing that React is clever enough, and implements caching, exposes the keys API, introduces some trade-offs to reinforce performance which is a real deal, but still - it’s not enough to move forward (If you are interested in in-depth details of the algorithm, visit official docs here).

In the next part, I will explain what approach the React Team took to address those limitations.

Why should you write good commits?

Gil Goncalves — Wed, 28 Jul 2021 08:19:35 +0000

As software engineers who use git or other similar version control systems, we tend to write a lot of commits. Is investing in good commits a worthwhile investment?

Why should you write good commits?

As engineers, it’s easy to focus on just the code because, after all, that’s what we spend most of our time reading and changing. I’m going to argue that Git isn’t simply a smart diff merger to help us put all code in the same place nor is it just a simple version control system, it can be so much more.

Clearing your head whilst coding

One way I use git commits is to help myself understand what I was doing the day before when writing/changing a lot of code. It gives me a descriptive context that I can read in order to understand what I was doing the day/hours before quicker than just reading the code. This is also useful as a way to share with your colleagues some change you want some feedback on, what better place to write your intentions than in the commit message itself?

Example:

Documenting the codebase

If you use git to manage your codebase, then every single line of code is being documented by the commit messages. Along with each change, there’s a git commit message, and if we’re able to document each change with some text describing it in a way that’s not tied to any external services, why not leverage that as a way to document your whole codebase as you write it.

Easier debugging

Let’s pretend that you’re working in a codebase where each commit clearly explains the reason why it was added. You are now looking at a piece of code and you want to delete it, but you don’t know if you can. Using git blame to check the commit itself lets you understand when and why that change happened. You can then know whether you can remove that code (if it’s not needed because it was added as part of an experiment that has finished, for example) or whether you should keep it (it’s really important because the specific version of your DB requires that for example).

Another git tool that can help us with debugging is git bisect, which helps you find out which commit introduced a bug by helping you check commits using binary search. In this situation, it really helps having atomic commits that have one change only and that are well written so it’s easier for the person debugging to figure out which commit introduced the bug, but also why we made that specific change to answer questions such as: can we revert it?

And speaking of revert, that’s another good example of how good commits can help you. If each change is isolated in a commit, then it’s easy to git revert just that specific commit (or some) to fix a problem in production, when speed is crucial.

OK, then what is a “good commit”?

Let’s split the commit into two parts, the code change and the message.

The code change should be “atomic”, meaning it should not break tests and it should be contained in a particular change.

Good example:

Implementing a new API endpoint including tests and documentation

Bad examples:

Implementing a new API endpoint, tests and documentation in three separate commits
Implementing a change to the code and another commit to “fix tests”

With atomic changes, it’s easier to run git bisect for finding out problems since you know that every commit keeps the build green, so you won’t have noise with whatever command you’re running to find a particular commit when bisecting and it’s easy to git revert it for the exact same reason.

As for the message, it should focus on explaining why the change is being added to the codebase rather than what it’s doing since that’s already described in the change itself, this way when you check a commit for a piece of code you’re curious about, you can use it to understand why that code was written and its purpose (when it’s unclear from the code itself).

What’s next?

OK, I managed to convince you and you want to write commit messages that explain the reasoning behind an atomic code change to the code, what can you do next?

When writing a commit message, ask yourself whether the message explains well the reason for the change to be added to the codebase.
When making a commit, consider if it can be amended with another commit or if it’s an independent change that doesn’t break the tests.
Make reviewing merge requests commit by commit a part of the code review process so that you’re only merging good commits into your codebase.
Share this article with your team and friends (who use git) :)

Resources

Because I’m standing on the shoulders of giants, here is a selection of talks/articles that helped me write this article and have helped me improve my own git commits during my career.

Joel Chippindale | Simplify writing code with deliberate commits talk
How atomic commits make you a better coder
Pro git book | "Rewriting history"
A good talk on git including a story of why git commit messages are useful
The pro git book is free and a great resource to learn more about git: https://git-scm.com/book/en/v2
Tom Stuart offers writing git commits as a solution to “Get off the tightrope”