Forem: Stefano Magni

How Preply improved INP on a Next.js application (without React Server Components and App Router)

Stefano Magni — Thu, 13 Feb 2025 09:54:29 +0000

This article has been originally posted on Preply's engineering blog.

To improve our two most important pages from an SEO and SEM perspective, we started digging into how to improve their INP, a metric to assess web pages’ interaction speed. It was an R&D project driven by a clear goal baked by tons of data and assumptions. We spent a lot of time identifying what to optimize and made one-line changes and big refactors. We jumped onto React Server Components and Next.js App Router. We succeeded but also failed frequently. In this article, we will share the whole journey and the takeaways.

Introduction

Why care about page speed?

Page speed is crucial for web pages because:

From an SEO perspective, page speed is a ranking factor for search engines.
From an SEM perspective, page speed can influence your Quality Score on platforms like Google Ads. A higher Quality Score can lead to lower cost-per-click (CPC) and better ad ranks.
From a user perspective, a faster page speed can lead to better user engagement and higher conversion rates.

Preply.com is a B2C and B2B tutoring platform, so SEO and SEM optimizations are part of our everyday job. Page speed impacts the effectiveness of our efforts in these two areas. Page speed is measured through Web Vitals, which quantify a website’s user experience. One of the Core Web Vitals is INP, the worst Web Vital of Preply.com.

What is INP?

The official definition of Interaction to Next Paint (INP):

INP is a metric that assesses a page’s overall responsiveness to user interactions by observing the latency of all click, tap, and keyboard interactions that occur throughout the lifespan of a user’s visit to a page. The final INP value is the longest interaction observed, ignoring outliers.

Practically speaking, in a simplified way, and thinking of a “normal” MPA (Multi-Page application):

A user visits a web page and interacts with it.
INP measures, in milliseconds, all the synchronous JavaScript executed after the interaction.
The longest interaction is the user session’s INP for the web page.
The overall page’s INP is the 75th percentile of all the user sessions’ INP, split by desktop and mobile devices, over the last 28 days.

Some key things to keep in mind:

It doesn’t matter what the users do on the web page. Some close the privacy banner, and some spend 15 minutes on a page doing many things. The slowest interaction is their INP.
Users don’t have the high-end devices we typically use to develop digital products. Regarding mobile devices, some users have very old and very slow smartphones (the number of these heavily depends on the markets you operate in). So your users could deal with some performance issues you were probably unaware of. The slower the device, the worse the INP.

Thanks to the recent Google Chrome updates, working locally with INP is quite straightforward, look at the following screenshot of the devtools’ Performance tab, which includes INP.

The Google Chrome devtools’s Performance tab shows the Core Web Vitals for the current session, especially INP, and the slowest interaction that caused it.

Preply.com’s tech stack

All Preply.com’s indexed pages are managed through a Multi-page Next.js 13 application (React 18) rendered on the server through the Pages router. The codebase is quite big and has accumulated a lot of technical debt.

The Next.js app has thousands of indexed pages, but the two most important ones, from either an SEO or SEM perspective, are the Home page and the Search page. They are two very different pages:

The Home page has few lines of code and is mostly static. The only interactive elements are a slider and the privacy policy banner.
The Search page is huge, complex, and 90% interactive, and it includes modals to filter tutors, and book lessons.

Web Vitals on both pages are ok’ish, but there’s room for improvement on mobile devices’ INP. The project’s goal is to improve INP from the yellow zone to the green one.

Our internal dashboard reports the INP for the Home page (~250 ms before the optimizations, 185 ms after the optimizations) and the Search page (~250 ms before the optimizations, 175 ms after the optimizations).

We calculated that moving the two pages INP in the green zone could save us $200K/year. Please note that’s a ballpark number we identified by analyzing our current SEO situation, our competitor’s Web Vitals and estimating how much we can improve there.

Our initial hypotheses

We had two main hypotheses on how we could have improved INP:

We need to reduce React’s hydration to get the pages ready and interactive sooner.
We need to improve the performance of our own JS/React code.

Every change we introduced was the result of extensive analysis. Performance optimizations on existing big products usually require weeks of analysis and data intersection to find the root issues and only hours to fix them. This project is no exception: we started with many unknowns, and we moved through them, failure by failure, before finding out how to improve INP.

Reducing React hydration

Hydration is React’s process of making static HTML interactive on the client side. This is a common step for every SSR (Server-Side Rendered) website. Before hydration, the React app was not interactive. After hydration, it is fully interactive. But what’s in the middle?

When users interact with the page, and hydration is in progress, React records and replays their interactions. This is very convenient from a developer perspective, but it comes with a cost: interactions’ INP is worse than normal. The next graph shows how a hypothetical 80-ms INP interaction becomes a 235-ms INP interaction if it happens during the hydration phase.

The diagram shows how a usually fast interaction can become slow during hydration.

The above is an example. We can’t tell if the full hydration time is added to INP, but our manual tests showed that some delay is added to the normal INP.

The most hyped recent React feature is Server Components (RSC). Thanks to RSC, you can selectively make part of your page static, executing it only on the server, without the client-side hydration, similar to how good old PHP does. Next.js supports RSC through the App Router, and migrating the two pages we worked on was the initial big bet.

Let me anticipate one key conclusion: by looking at our metrics, we found out hydration should not be a big INP offender since it’s happening quite fast (120 ms on average), and we expect most of the users to interact after this quick phase:

At the time of writing, the INP for the Search page is 248 ms, and React hydration is 120 ms.

Please note that our app leverages React 18, which introduced various Hydration improvements. We will touch this topic later in the article.

Despite that, we migrated the Home page to the Next.js App Router anyway to gain experience. Ultimately, one of the project’s most important goals was to collect all the possible takeaways to determine the best approach for all the other existing pages in the future.

All the data we leveraged to improve INP

The “What do the users do on the web pages? What do they interact with?” questions are crucial to determining what to optimize. To find out, we needed a ton of data. Preply is a very data-oriented company; we had all the data upfront. Without all this data, bringing INP to the green zone in less than a quarter would have been impossible.

To improve INP, we used:

The official web-vitals library is from the Google Chrome team. We collect all the Web Vitals data at every session and create a global dashboard in DataDog to show the Web Vitals data.

The generic DataDog dashboard we built on top of web-vitals library’s data.

Next.js’ custom metrics (especially the hydration and render duration).
All the mentioned data is available and queryable in SnowFlake, so everyone in Preply can analyze it and create custom dashboards.
Hotjar, to show and analyze the heat maps.
The standard browser dev tools for profiling. React Developer Tools and React Scan to catch unnecessary re-renders.

For your information, we removed DataDog’s RUM and Sentry Web Vitals because they were redundant compared to point 1 (the web-vitals library). We tried them out, but in the end, we all rely on our custom Snowflake instance when it comes to querying and aggregating data (all data passes through Snowflake, and then we forward some data to DataDog), so it was a natural choice to keep everything centralized and remove redundancy.

One impactful detail regarding the data the web-vitals library tracks: one of our most important dashboards showed a high number (75%) of <unidentified> selectors. This defeats the purpose of the dashboard itself. The reason is a known Google Chrome issue, which makes it harder to collect the selector of the DOM elements when they are part of modals that disappear right after interactions. The web-vitals authors suggested working around this issue in this GitHub comment. Thanks to all the data we intersected, we were able to improve INP even without fixing the selector issue.

The custom dashboard with the selectors of the INP offenders shows a very high number of unidentified elements.

How we improved INP

The list of changes that improved INP on the Search page:

Change	INP before	INP after	Net improvement
Upgrading React from 17 to 18	~460 ms	~320 ms	-140 ms
Virtualizing a long list	~320 ms	~280 ms	-40 ms
Debouncing keyboard events	~280 ms	~260 ms	-20 ms
Memoizing heavy components	~260 ms	~230 ms	-30 ms
Fixing state management non-EU privacy policy banner	~230 ms	~215 ms	-15 ms
Avoiding the tooltip to re-render the whole page	~215 ms	~205 ms	-10 ms
Updating Usercentrics for the EU privacy policy banner	~205 ms	~200 ms	-5 ms
Re-optimizing what we already optimized	~200 ms	~185 ms	-15 ms

The list of changes that improved INP on the Home page:

Change	INP before	INP after	Net improvement
Rewriting the Slider and Improving the Accordion components	~250 ms	~220 ms	-30 ms
Updating Usercentrics for the EU privacy policy banner	~220 ms	~180 ms	-40 ms
Migrating the Home page to the App router	~180 ms	~170 ms	-10 ms

We didn’t see any meaningful improvement by:

Leveraging React’s useDeferredValue.
Optimizing the Design System components.
Cleaning up the Search page’s code.
Quickly optimizing Preply Chat for the logged-in users.
Removing DataDog RUM.

Finally, what we didn’t do:

Offloading the third-party scripts with Partytown.
Jumping on the new React Compiler.

You can find more details in the next sections.

React 18 upgrade (INP decreased by 140 ms)

In reality, this was not part of the late 2024 page speed project this article is about, but of course, it’s worth mentioning. At the beginning of 2024, all the Product teams, coordinated by Preply’s Devex team, worked together to upgrade React to v18. This upgrade promised to bring meaningful hydration and performance improvements by itself, even without leveraging React Suspense, and so it was. In the days after deploying the version bump, we started seeing a much better INP.

The INP of the Search page before the React 18 migration (~460 ms) dropped significantly immediately after the React 18 upgrade was released (to ~323 ms).

The “Understanding Hydration in SSR with React 18’s New Architecture” article provides a longer explanation of React 18's hydration improvements.

Virtualizing a long list (INP decreased by 40 ms)

The Search page’s longest list is the Tutor’s Country of Birth. It lists ~300 countries.

The Country of Birth’s UI on Preply’s Search page.

It doesn’t sound like a huge list. Still, the 300 React components and their sub-children take significant time to be added to the DOM on a slow smartphone (please note: we are speaking about the React->DOM reconciliation that adds all the new elements to the DOM, not the render step itself). We used React Virtuoso since it was already used in our front-end projects, but virtualization libraries are all quite similar.

FYI: We discarded it because it’s not yet fully supported by the browser Preply.com supports, but CSS content-visibility has great potential to leverage CSS native virtualization. You can read more in the “Improving rendering performance with CSS content-visibility” article.

Debouncing keyboard events (INP decreased by 20 ms)

Thanks to the data we collected through the web-vitals library, we split the keyboard events, which showed a constantly bad INP. There are three input fields on the Search page, and typing something there was painful on a slow device. As you can see in the next screenshot, debouncing them quickly brought noticeable INP improvements.

Before: 632 ms INP while typing on the main learning subject’s input field, with most keyboard events resulting in a >300 ms INP. After, the same events result in a 72 ms INP.

Memoizing heavy components (INP decreased by 30 ms)

Two big components, namely the “How Preply works” and the “Users’ reviews” component, took a lot of time to re-render, increasing the INP of the interactions that resulted in whole page re-renders (for instance: when you close the Filters modal and the new filters are applied) You can see their impact in the next screenshot of the React DevTools’ profiler.

The React DevTools’ profiler, in flamegraph view, shows the impact of the HowPreplyWorks and Reviews components.

Improving INP here was straightforward: the two components were static, and memoizing them was enough to improve INP.

Fixing state management non-EU privacy policy banner (INP decreased by 15 ms)

The issue was, again, a “from global to local React state” problem. Every new user’s first interaction is with the privacy banner. On both the Home and Search pages, the banner shown to non-EU users re-renders the whole page when closed. The issue was quite straightforward: the React state to render it or not was stored at the page level, way too high in the React tree. An intermediate component containing only the privacy banner logic (to save the preference in the local storage, then hide the banner) was enough to improve INP.

Avoiding the tooltip to re-render the whole page (INP decreased by 10 ms)

We noticed something weird: the first interaction with the Search page (not the Home page) is slow. It didn’t matter what you interacted with — the first interaction with the page was always slow, but subsequent interactions were faster. The real aha moment was when we realized that even touching a non-interactive element (like the page’s background) had a bad INP.

How we proceeded:

Through the browser’s performance tools, we recorded an interaction that clearly showed the issue.
The recording showed the issue was with the touchstart event only.
We removed all the touchstart handlers on the main DOM elements (html, body, <div id="__next">) one by one.

The browser dev tools visualize the event handlers, allow you to remove them, and jump to the listening JS module. Initially, there were more than 30 listeners.

All the handlers were responsible for part of the slow INP, but one of them caused the whole page’s re-render.
In the Event Listener tab, the browser dev tools initially pointed us to Sentry’s listeners (which can also be used to measure Web Vitals, by the way).
After disabling Sentry, the issue remained, and the Event Listener tab pointed us to a custom tooltip we built on top of Radix UI’s Tooltip.

The reason was a basic TypeScript oversight:

The initial tooltip’s state was const [open, setOpen\] = useState(props?.open) which means open is boolean | undefined.
The onClickOutside handler (called when you click anywhere) did setOpen(false). So open switched from undefined to false, resulting in a children re-render.
We fixed the issues just by changing the initial state const [open, setOpen] = useState(props?.open ?? false), removing undefined from the possible states.

As a rule of thumb, I always suggest “help TypeScript to help you” by reducing the possible types: Unions instead of generic types, Discriminated Unions instead of optional properties when possible, no falsy types mixed with booleans, etc. You can read more about this topic in my “How I ease the next developer reading my code” article.

Updating Usercentrics for the EU privacy policy banner (INP decreased by 5 ms for the Search page, 40 ms for the Home page)

For the EU users, we leverage Usercentrics. Specifically, we were using the V2 version. Similar to the previous point, the privacy banner was okay from an INP perspective, apart from when you closed it without accepting/changing privacy preferences. Closing the banner executed a huge amount of JS code that was out of our control and impacted many of Preply’s users.

The Usercentrics team suggested we migrate to their V3, which solved this INP issue. But to do that, we also needed to update the GTM (Google Tag Manager) version to load the Usercentrics banner. We teamed up with those responsible for GTM and helped them with the migration, dropping another INP offender that impacted all of Preply.com’s pages.

It’s worth noting that updating Usercentrics significantly impacted the Home page, where the privacy banner is one of only a few interactive elements. However, the effect on the highly interactive Search page was minimal.

Re-optimizing what we already optimized (INP decreased by 15 ms)

The improvements mentioned above eliminated the worst INP. Then, we had a few weeks (see the graph) with no noticeable improvements. We were hitting the end of the 80/20 Pareto rule, and all the next fixes would have brought lower benefits while requiring more implementation time (which we didn’t have, the project was meant to be completed in 2024).

The INP graph shows that after many quick wins, we had no improvements for a few weeks.

We then decided to return to the already optimized issues and optimize them even more. Ultimately, they proved to be low-hanging fruit because they impacted the core user flows, why not optimize those flows even more?! So we started using a 20x CPU slowdown to measure performance improvements, which is surely closer to the budget devices we are optimizing for compared to the previous 6x slowdown (a 6x slowed-down high-level MacBook Pro is still faster than most devices on the market, especially when speaking of mobile devices).

It worked, and we could push down INP even more and match the project’s OKRs 🎉.

Migrating the Home page to the App router (INP decreased by 10 ms on the Home page)

Next.js App Router and RSC (React Server Components) improved INP only slightly. We expected more, but it was a mistake of ours: in our initial POC with App Router, we wrongly used TBT (Total Blocking Time) as a proxy metric for INP, that’s why our expectations were erroneous.

But there’s more to tell! Just saying “App Router improved slightly” is a bit unfair because its potential is higher. Why? Well, I told you that we migrated, not rewritten, the Home page!

So, we built a POC with a greenfield Next.js app. We chose only RSC-compatible libraries (for internationalization, for example), and rewritten part of the page with data-caching in mind. The results (performance is measured with a 3G network and 20x CPU throttling, locally, in prod mode) are impressive:

	Pages Router	App Router	Improvement %
JS bundle size	1.2 MB	173 KB	85% less JS shipped
HTML response time (shows how fast the server generates and sends the HTML)	51.8 s	15.7 s	70% faster server execution
Web Vitals: FCP	5.8 s	1.9 s	65% better
Web Vitals: LCP	5.8 s	2.8 s	50% better
Web Vitals: INP	Most interactions: 80-100 ms	Most interactions: 30-40 ms	60% better

These results give justice to the App Router! Then, we all agreed that the optimal and long-term solution (not feasible in the project’s lifespan) would be to start from a fresh Next.js project and rewrite all the features hacked into Next.js in years and years of fast growth.

useTransition, await-interaction-response, etc.

During the project, we tried to go deep, find the root cause of the issues, and fix them. We didn’t want to hide the dust under the carpet and worsen the problem.

Despite that, we encountered situations where the lack of proper design wasn’t fixable in the project’s lifespan, and “lying” to the browser was the only way to speed up some interactions. What I mean by “lying”:

In the great Demystifying INP: New tools and actionable insights article, Vercel (Next.js’ creators) shared the await-interaction-response package. This is the source code:

/**
 * Returns a promise that resolves in the next frame.
 */
export default function interactionResponse(): Promise<unknown> {
  return new Promise((resolve) => {
    setTimeout(resolve, 100); // Fallback for the case where the animation frame never fires.
    requestAnimationFrame(() => {
      setTimeout(resolve, 0);
    });
  });
}

Essentially, it’s a small hack to avoid blocking the main browser’s thread and postponing the execution of your heavy code. This improves INP, of course, but in our case, it worsens the problem because it’s a sort of “Don’t worry, the lack of design and development anti-patterns are fine, just use this workaround” message, which we don’t want to spread.

React’s useTransition is more elegant because it’s embedded straight in the framework, but in our case, using it hides the root problems instead of helping/teaching how not to introduce them upfront and removing the existing performance bottlenecks.

But for those looking for short-term and quick solutions… well, keep them in mind 😅.

Just one note: if you are considering implementing a custom yielder, please don’t rely on requestIdleCallback! When working on the Design System Visual Coverage, we saw that the browser frequently invokes the passed callback after several seconds due to other intensive processes. In The Implementation Details of Preply’s Design System Visual Coverage article, you can read all the performance optimizations we made there.

Leveraging React’s useDeferredValue (no INP improvements)

We were a bit surprised this didn’t work. React’s useDeferredValue keeps input fields snappy when the result of the users’ typing (the list of languages to learn, in our case) requires a significant amount of time to be updated on the page. After refactoring one of our input fields, we noticed no improvements compared to when the input field’s value was debounced. But why?

We realized that useDeferredValue works as a charm when the items to render are heavy at the React rendering phase (essentially, when React executes the component’s functions that return the JSX) but soft on the DOM reconciliation phase (where React adds/updates/removes the elements from the DOM). Our case was the opposite, the components are fast enough at rendering but heavy regarding how many DOM elements they produce (you can read a quick thread on X about this), vanishing useDeferredValue’s benefits.

Optimizing the Design System components (no INP improvements)

Path (Preply’s Design System) is used a lot: the Search page counts 875 usages of the Path’s components, and 76% of what the users see comes from Path; you can dig more into how we collect the stats in the Visual coverage: Why and How Preply Measures the Impact of the Design System article. A particularly heavy part of the Search page’s render tree highlighted that the two most used Path’s components (LayoutFlex and Text) had room for improvement when transforming the React props into CSS class names, impacting the overall page’s INP.

Refactoring the Path’s components doesn’t happen overnight, and we spent days heavily testing and refactoring the LayoutFlex component to avoid any regressions across the whole website. All this heavy work created the foundations for how the Design System team can safely test and refactor the components, but from an INP perspective, it brought no improvements. Our analysis was wrong.

Cleaning up the Search page’s code (no INP improvements)

The hundreds of experiments the Product teams launched in recent years were still part of the codebase. Some were just dead code, some were imported dynamically only if the users were in one of the a/b cases, but never removed when the experiments were scaled or killed. Some were always imported, negatively impacting the JS bundle.

We removed 15% of the Search page codebase (7K lines of code) for two weeks, but this didn’t show INP improvements. This has been a recurring theme during the project (later on, we speak about the Next.js Ap Router), but shipping less JS never improved INP for our Search page (please note I’m speaking only of INP, shipping less JS is always good).

Quickly optimizing Preply Chat for the logged-in users (no INP improvements)

Logged-in users can access Preply Chat from the header, a modal that opens inside the same page. From Hotjar, we got the idea that many users were opening the Preply Chat.

A screenshot of the Search page shows the Chat button at the top for logged-in users.

We didn’t expect it, given how indexed pages and heavy SEM campaigns bring tons of new and not logged-in users to the Search page, but from some quick queries, we realized 80% of the Search page views come from logged-in users 🤯. Given that Preply Chat is complex, we used the mentioned await-interaction-response, postponing some of the Preply Chat actions. We didn’t see the INP improvements we expected, meaning that our assumptions from intersecting all the data were again wrong.

Removing DataDog RUM (no INP improvements)

After extensive local tests, and even if it looked weird, DataDog RUM seemed to be causing worse INP. Luckily, the tool was redundant for us (since we mostly rely on the web-vitals library), and we could remove it. Once more, after doing so, we didn’t notice any improvement.

Offloading the third-party scripts with Partytown (we gave up)

We load many third-party scripts, some in a developer-controlled way and ~40 through GTM (Google Tag Manager) without developer control. Despite knowing that the scripts loaded through GTM impact the page negatively, we gave up on trying to move them to Web Workers with Partytown. Here’s why:

Once loaded by GTM, some scripts append more scripts to the DOM, which are executed in the main thread, partially defeating the purpose of adopting Partytown.
Years and years of scripts added through GTM, and maybe frameworks/integrations built on top of them, almost guarantee that we break something without a clear correlation between what we break and what results are wrong.

Given how we all depend on our data and the short amount of time available for the project, we decided to give up, given the unclear return on investment of this work.

Jumping on the new React Compiler (we didn’t even try)

The React Compiler is under construction at the time of this writing. However, the limitations we discussed with the Devex team were the Next.js v14->v15 upgrade and the React v18->v19 upgrade, which required more time than ours and didn’t fit the project’s time scope. Wakelet shared an early feedback about the Web Vitals improvements they got from adopting the new compiler.

Did all this work also improve the UX?

The answer is: YES! Working with 6x and 20x CPU slowdowns, the UX improvements are evident, though there’s still room for further enhancements. INP proves to be a great metric from this perspective; it acts as a true litmus test for how painful it is to use your product when users don’t have access to top-notch devices. While all Web Vitals are valuable, INP uniquely measures the user experience throughout the entire session, regardless of whether the user’s journey is brief and superficial or deep and long.

Miscellaneous

Many of the project’s details have been omitted from this article because they are not crucial or relevant to readers. Let me just mention some quick facts/suggestions:

We saw the first INP improvement after more than one month of work, out of a quarter.
The weekends were crucial to measure the impact of our work because we saw dashboards needed 24–48 hours to reflect the INP changes.
We couldn’t stop other Product teams from running experiments on the pages due to an overlapping important Product goal. It wasn’t optimal, but the generated friction wasn’t so high.
The Devex team worked hard to upgrade Next.js from 13 to 14 and leverage the faster Turbopack for local development.
The project started with the goal of optimizing both INP and TTFB. Then the two metrics were split into two different projects.
Jumping on the App Router required us to refactor all the LESS modules to SCSS (also in the Design System).

Always validate your assumptions on more than one PC: it happened more than once that one of us identified possible improvements, but the same improvements were invisible on other laptops (because of background zombie processes, because a user becomes part of a data sampling that slows down the website, etc.).

Medium and long-term plans

All the mentioned improvements are short-term, and the benefits can vanish quickly if we don’t make performance a first-class citizen when developing the Product. We haven’t formalized all the activities for this goal, but we have some in mind:

(you saw it in action through this article) Monitor Web Vitals to catch regressions.
(already done) Add Web Vitals to our experiments framework’s dashboard and the other Product metrics we all care about.
Add performance data to fill the PR template.
(already in progress a the time of writing) Run workshops to share the takeaways and spread React best practices with all our frontenders.
Run some workshops to share how to deal with the existing Product’s refactors from a performance perspective.
Discuss the role of App Router in our product and our infrastructure around it.

Takeaways

✅ Tracking tons of RUM (Real User Monitoring) data and having it at hand is key.

✅ Adopting a 100% data-driven approach from the beginning is crucial. As you can see above, the optimizations that solved the INP problems are not rocket science, but it is important to identify them before jumping on the code.

✅ Don’t blindly follow what others did and shared (including this article); don’t take for granted that the other, more hyped framework solves your problem. Ultimately, we achieved our INP goals by speeding up our existing code.

Some more resources

The great people who worked on the page speed project

This project took a village, and I want to thank everyone involved 🤗

Maksym Ridush, for the initial deep analysis that resulted in the page speed project 👏.

From the team responsible for the Search page:

Artem Yermak, thank you for the endless support and for improving INP 💪.
Ivan Biletskyi, for the outstanding support, reviews, and knowledge-sharing ❤️.

From the team responsible for the Home page:

Carles Ballester, for the Home page migration and working on the endless experimentation issues 🤘(and Robert Serrat Morros for supporting Carles).
Michael Castro, the Engineering Manager who’s also a front-end expert you don’t expect 😊
Alejandro Lampropulos, for the GTM and UC support 🙏.

Preply’s Design System’s engineers, who have been borrowed to the page speed project:

Oleksandr Kozlov, the real Next.js guru 🤯.
Seif Kamal, who constantly challenges and analyzes every single data 🚀.
Myself 😁.
Bogdan Brindusan, for jumping on and managing such a complex and out-of-the-comfort-zone project.

Vadym Vlasenko, for always digging into the details and challenging us.

All the people from different teams who helped us

Mate Papp for everything DevEx, including the Next.js v14 migration.
Kyryl Sablin, from the SRE team, thank you for the constant help and support.

And, of course, the reviewers who helped make this article better 🤗

Sébastien Lorber, for the suggestions, comments, and your infinite news source: This Week in React ⚛️.
Nicolas Beaussart, for the detailed review and the time you always dedicate to my articles 😳.
Andy Jessop and Omri Lavi, for your interest and comments 🤗.
Dmitry Belyaev, for reading this even if you were on vacation 🌴.
Maxi Ferreira, the author of one of my favourite newsletter: Frontend at Scale.
Nadia Makarevich, the Advanced React course’s creator and Developer Way’s author.

From improving our engineering culture to becoming an AI-native company… Would you like to join us and work in a purpose-driven organization where work, growth, and learning happen at the same time? Preply continues growing and we are actively looking for talented candidates to join our Engineering team! If you are excited about taking on a new challenge, check out our open positions here.

The Implementation Details of Preply’s Design System Visual Coverage (part II)

Stefano Magni — Thu, 03 Oct 2024 11:32:58 +0000

This article has been originally posted on Preply's engineering blog.

Implementing the coverage algorithm, working on the process that resulted in having the dashboard, working with teams to increase the coverage, implementing it for the App, etc., has required months of work. In this (very long) article, we will discuss all the project’s technical details. We also made the visual coverage code available on GitHub at preply/design-system-visual-coverage, so you can use it for your product too.

If you are new to the topic, please read the first non-technical article explaining the whys and hows behind the design system visual coverage project.

You may have heard of the project:

At the Into Design Systems conference: here you can find the article they wrote about it, and the resources we shared).
At the JSDay conference: here is the talk description, and the slides.
At one of Belka’s AMA: here is the recording (in Italian).
At one of the Commit to Growth’s episodes: here is the recording. The podcast is curated by one of Preply’s engineers: Yasemin çidem

Web implementation

The visual coverage performs two main operations:

Parsing the DOM to gather all the elements and their boundaries
Creating a bitmap out of the element boundaries and calculating the coverage

As explained in the first article, we measure the visual coverage on users’ devices. Browsers provide all the APIs we need to avoid getting into the user’s UX: requestIdleCallback and Web Workers. The following graph shows how we decided to run the calculation.

This timeline graph shows when and how we want to retrieve all the DOM element properties (when the user is idle) and that the heavy pixel count happens in a Web Worker.

Some of our pages are SSR’ed; others are part of a regular SPA; we are exploring using RSC… But from a visual coverage perspective, it doesn’t matter. We measure the visual coverage on the client, on all the user’s devices every five minutes (at .00, .05, .10, and so on), across all Preply.com’s pages, regardless of what the users are doing (scrolling the page or taking an online lesson). At the moment, we are collecting 150K-350 K events per day. This high number of events guarantees that every single Preply.com page is tracked.

Preparatory steps

Step 1: We updated all the Design System components to include a dedicated DOM attribute: data-preply-ds-component, whose value is the name of the React component. This allows us to distinguish Design System DOM elements from the other ones. Also, this detaches the coverage script from the React.js nature of the website, and dedicated build and deployment steps were not needed.

Inspecting Preply.com’ pages reveals all the design system component data attributes.

Step 2: Create the visual coverage containers. In our case, this translated into setting the data-preply-ds-coverage attribute to all the pages, assigning them a name and the team the page belongs to. In the JSON, the page is generically called component because the same logic applies to smaller components (like the unified header, the calendars, the chat widget, etc.) and allows splitting the responsibility of the same page across different teams.

Inspecting Preply.com’ pages reveals all the design system coverage container data attributes.

Using a bitmap to recreate the page’s pixels

Counting the number of colored pixels in the DOM is a slow operation, even if you work with a canvas (we haven’t tried OffscreenCanvas, though). So, we thought of a way to “recreate” the page with simple data we could mutate and aggregate quickly. An array of colors (technically speaking: a bitmap) would be perfect!

The idea is:

Traversing the whole DOM reading all the bounding rects (top, left, width, height) of every element.
Creating a bi-dimensional array in which every item represents whether a pixel of the page comes from the Path Design System or not.
Counting the pixels in the array.

To show how the bi-dimensional array could look, consider a hypothetical 20x20 pixels page with some flex containers, a heading, two buttons, and a footer that are Path Design System components.

[
  [🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥], // <-- non-DS components' borders are red
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟥],
  [🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥],
  [⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️],
  [🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥],
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟥], 
  [🟥,⬜️,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,⬜️,🟥],
  [🟥,⬜️,🟩,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩,⬜️,🟥],
  [🟥,⬜️,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,⬜️,🟥],
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟥],
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩,🟩,🟩,🟩,🟩,⬜️,🟩,🟩,🟩,🟩,🟩,⬜️,🟥], // <-- DS components' borders are green
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩,⬜️,⬜️,⬜️,🟩,⬜️,🟩,⬜️,⬜️,⬜️,🟩,⬜️,🟥],
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩,🟩,🟩,🟩,🟩,⬜️,🟩,🟩,🟩,🟩,🟩,⬜️,🟥],
  [🟥,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟥],
  [🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥,🟥],
  [🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩],
  [🟩,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩], // <-- The rest of the page is white
  [🟩,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩],
  [🟩,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,⬜️,🟩],
  [🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩,🟩],
]

For the sake of simplicity, the example shows a 1-pixel border for all the components. In reality, the component weights (see the previous article’s chapter about the component weights) decide the width of the border.

A bitmap is generated for each visual coverage container. In the case of nested containers (like a page that belongs to team A but contains a component that belongs to team B), the area of the nested container looks empty in the parent container’s bitmap.

Please note: there are two approximations in the DOM->array transformation. The algorithm does not respect:

CSS stacking contexts (aka z-index): every element is above the previous one. This isn’t a big deal, given the nature and structure of Preply.com’s pages (which have a document-like structure, rarely relying on z-index, apart from the Classroom).
Content overflow: this limitation can be easily overcome by using containers, which limit the boundaries of the children.

We haven’t tested the visual coverage code on heavy fixed, and non-scrollable UIs. If you do, please let us know 😊.

Performance optimizations

We quickly validated the idea with a throwable POC (this is where ChatGPT shines). Still, we stepped into severe performance optimizations to move from the initial 150 blocking milliseconds for a tall page (on my machine™) to the final 50 non-blocking milliseconds. Also, we moved from 30 MB to 300 KB regarding memory footprint. To achieve these improvements, we also involved Massimiliano Mantione, a former Google Chrome V8 Engineer, and Matteo Ronchi from WorkWave (software architect), who is now using the same visual coverage approach on a very different-from-Preply product.

We applied some obvious optimizations (like skipping SVG contents or hidden elements) and some less obvious ones, like:

The array should be monodimensional instead of bi-dimensional. This optimizes memory access since accessing a memory address to find another memory address (required to access an item of the array stored inside another array) is not optimal for the CPU.
Every item of the array must use the minimum possible number of bits. Strings are wrong from this point of view because their memory usage is dynamic by default. Typed Arrays is the best choice.
If you write branches-free code (if conditions are branches), the JIT compiler can optimize your code, resulting in C++-like performances.
The “count the pixels” code could be moved to a Web Worker and run without interruptions. This does not greatly improve performance but significantly impacts the code’s readability. The equivalent calculation made through requestIdleCallback forces you to write interruptible code, which is inevitably way harder to read than straightforward synchronous code running in a Web Worker.

All the above allowed us to reduce a lot the execution time, and some tests on old devices showed the following:

OnePlus 6 (top-notch 2018 phone): 650ms.
Huawei P9 (cheap 2016 phone): 900ms. Interestingly, the Huawei P9 (4x times slower in everyday use) performs <50% worse than the OnePlus 6. The hardware is okay with this kind of browser operation.

Is it necessary to make the script 100% non-blocking? The answer is yes! Our initial implementation included a very small blocking phase. Still, two users (out of the 1% Preply users included in the initial experiment) faced a prolonged blocking phase, with a worst-case duration of more than 400ms. See the two spikes in our monitoring graph.

By making the script asynchronous, we have to live with the fact that if we split reading the DOM over multiple idle callbacks, we could count the coverage on a half-changing page. We think this is tolerable, especially given that if the count is 100% non-blocking, we can run it whenever we want (also immediately after the React application is initialized), which increases the chances of intercepting users who quickly move through pages (if you count the coverage infrequently, some pages could never get included in the stats).

To limit this issue, we stop counting the coverage when the user clicks, when the URL changes, etc.

Sample rate and events cardinality

As mentioned, we measure the visual coverage on all the user’s devices every thirty minutes (at .00, and at .30), across all Preply.com’s pages, whatever the users are doing. At the moment of writing, we collect 150K-350K events per day. They are sent to a custom endpoint and forwarded to our paid DataDog.

We don’t pay DataDog based on the number of events but their cardinality. The cardinality is inherently high because the most essential event data are the team (16, at the moment of writing) and the component (the page or component name. They are more than 100). We have sacrificed other data to reduce the event’s cardinality and not incur additional costs (like the user type, the release version, etc.).

App (React Native) implementation

Let’s start with two considerations that differentiate tracking the coverage on Preply’s app compared to Preply.com:

When we started working on the app, React Native didn’t provide handy performance-oriented APIs like requestIdleCallback or Web Workers. We can’t easily be 100% sure we don’t impact Preply’s users’ UX, so we don’t measure the Design System’s visual coverage in production.
The app’s E2E tests cover 100% of the use cases.

Please note: requestIdleCallback for React Native’s new architecture was released in August 2024 (see the PR) as part of React Native v0.75. After we upgrade the app to React Native v0.75.0, we can also release the coverage count for the app in production.

We then leveraged the app’s E2E tests to count the coverage and send the results to DataDog.

The majority of the implementation is the same as the Web counterpart. There are only two differences compared to the Web implementation:

How we retrieve the UI elements’ boundaries. By design, the web offers an open and parseable tree, but mobile apps don’t.
How do we identify Path Design System components? This heavily depends on the first point.

After some experiments, we decided to

Leverage some straightforward Swift code to gather all the elements’ boundaries (at the moment, we are counting the coverage only on iOS)
Identify Path Design System components through React Native testID, which converts to iOS’ accessibilityIdentifier. This also required updating our test utilities because testID is used in the E2E tests to interact with some elements.
Launch the coverage count from E2E tests through a dedicated deep link, one of the few options for triggering an internal function from an external testing tool.

What about view hierarchy files?

We experimented using the iOS View Hierarchy. On paper, it would have worked (see screenshot) since it contains all the data we need, but unfortunately, it’s a binary file and can only be read if it is XCode.

The view hierarchy of Preply’s App’s main screen contains all the data we need. However, this data is inaccessible outside of XCode.

The Design System visual coverage score and dashboard

The official Preply’s Design System visual coverage dashboard.

We have two dashboards:

The official one is used and referenced by everyone to check the current coverage, identify possible areas for improvement, and eventually set teams’ OKRs.
A monitoring one is used by the Design System team to identify errors, slower calculations, distribution over different devices, etc.

How We Test the Visual Coverage Code

Only some key parts of the whole code are tested. This is enough for me to be sure the critical parts of the code work. The most exciting part is how the main web function (createCalculateDsVisualCoverages) is tested: through Vitest browser mode. Testing the visual coverage code based on such native but complex browser APIs (requestIdleCallback and Web Workers) is key to writing reliable unit tests. You can check the test code here.

Apart from that detail, all the code heavily relies on dependency injection to test, which is the easiest possible way.

How Teams Can Play with the Visual Coverage

We expose some utilities on the global object (window) to allow teams to play with the visual coverage. exposeGlobalDsVisualCoverageObject is the function that exposes them. The most important ones are the following:

__PREPLY_DS_COVERAGE.runAndLog(): it runs the calculation, it logs everything in the console, but it does not send calculation events to DataDog.
__PREPLY_DS_COVERAGE.runAndVisualize(): The same as above, but also add some visible rectangles to the page.
__PREPLY_DS_COVERAGE.runAndVisualizeContainer(componentName: string): The same as above but only for a single container. This is particularly useful for modals.
__PREPLY_DS_COVERAGE.reset(): Removes the colored rectangles.

So, everywhere (on prod, staging, Storybook, locally), product teams can just launch __PREPLY_DS_COVERAGE.runAndVisualize() in the browser’s console to get the calculation.

Test it on Your Product

After reading the why behind this project in the initial article, and digging deeper into all the tech details here, the next step is to experiment with the visual coverage yourself. In the preply/design-system-visual-coverage repository, we shared:

The entire implementation of visual coverage (we will update it periodically as we evolve it) lives in our design system repository.
All the code that consumes the visual coverage APIs in the product and converts the data to DataDog.

If you try the same approach in your product, you tweak the code, etc., please share what worked and what did not for you 🤗

Special Thanks

I want to publicly kudos all the people involved in this project:

The whole design system team: Justine, Seif, Alex, Javi, Alisa. I love this team 😍
Vadym, Igor, Josh, Dmytro, Dominika, for the support and feedback 😊
Oleg, for creating the dashboard and helping with all the data.
Daniel, Alberto, for the early internal feedback.
All the Preply front-end engineers for supporting the design system, and this initiative. All the Preply app engineers for helping with the React Native implementation.
Javier, Serhii, Volodymyr, for supporting me with the React Native implementation.
Matteo, for trying the visual coverage with a different company.
Massimiliano, for the performance suggestions.
Dmitry, Omri, for the feedback.
Nicolas, for the frigging detailed review ❤️.

Would you like to join me and work in a purpose-driven organization where work, growth, and learning happen at the same time? Preply continues growing and we are actively looking for talented candidates to join our Engineering team! If you are excited about taking on a new challenge, check out our open positions here.

Visual coverage: Why and How Preply Measures the Impact of the Design System (part I)

Stefano Magni — Thu, 03 Oct 2024 11:26:46 +0000

This article has been originally posted on Preply's engineering blog.

We implemented a custom way to measure the impact of the Design System, and we measure it on user’s devices. Why? Why were the existing solutions and approaches not enough? Why is it so crucial for us that everyone, management included, can understand and visualize the design system’s impact?

We worked for months to come up with our own way to measure the design system impact, and we shared all the whys and the hows in this article. A longer, technical one (The Implementation Details of Preply’s Design System Visual Coverage), explains what we did. Last but not least, the code is open source (https://github.com/preply/design-system-visual-coverage).

You may have heard of the project:

At the Into Design Systems conference: here you can find the article they wrote about it, and the resources we shared).
At the JSDay conference: here is the talk description, and the slides.
At one of Belka’s AMA: here is the recording (in Italian).
At one of the Commit to Growth’s episodes: here is the recording. The podcast is curated by one of Preply’s engineers: Yasemin çidem

Why Did We Need a Custom Way to Track the Impact of the Design System?

Preply is a data-driven company. All the teams elaborate on a long-term strategy (following the Good Strategy, Bad Strategy book approach) and define the metrics that allow them to measure their successes toward that strategy. The metrics are evaluated, challenged, discussed, tweaked, and approved. The design system team makes no exceptions.

We found that none of the existing ways to track the impact of design systems could tell the actual influence Path (Preply’s Design System) has on Preply’s users. Explaining static analysis data (tokens and components used in the codebase) to the CEO is hard. We needed to measure the impact on what the users see and interact with.

Since 2021, static analysis has shown a positive trend regarding the design system’s adoption, reporting a boost during Preply’s 2023 re-brand. However, this data expresses the influence on the codebase, not the users.

A screenshot of the internal design system stats we collected. The labels at the top left are the internal project names.

What Have We Implemented? The Design System Visual Coverage

The idea is:

The borders of the DOM elements are colored green for those generated by Path design system components and red for the others.
The borders’ size depends on the importance of the components.
The formula is green pixels / (green pixels + red pixels) (the final formula slightly changed, but this is the core concept). So, 100% means that “all the DOM elements of a page come from Path Design System components.”

Here is a screenshot of the visual coverage on Preply.com’s home page.

Here, you can see the visual coverage in action with borders colored and weighted based on the type of components.

The visual coverage tracks how much the Path Design System components are used in Preply’s products.

How Is the Visual Coverage Used?

The visual coverage serves different purposes:

The Design System team can:

Constantly track the outcomes of its initiatives. Please note that visual coverage is not the only metric we use.
Identify where Path Design System components are used the most and the least, and work with Product teams to increase the visual coverage.

The product teams have:

Numbers they can use as OKRs.
Straightforward tools to gather the visual coverage in seconds straight where they work: in production, locally, or in Storybook.
Straightforward tools to split the ownership of the pages across different teams. Most of Preply.com’s pages are made of components that belong to various teams.

To serve the mentioned purposes, we built an internal dashboard.

Our dashboard with teams, pages, and overall visual coverage.

Product Teams’ Ownership and Coverage Containers

On Preply.com, some pages are owned by a single team, whereas different teams own others. Visual coverage containers are areas of the page that are analyzed and counted independently.

The coverage containers in action: in the screenshot, there are three different containers: the Header, the Calendar, and the rest of the page. A different Preply team owns each of them.

While speaking with teams, we have always stressed that there is no good or bad visual coverage. In Preply, Path Design System is heavily used, and when it’s not used very much, there are strong reasons. For instance:

Calendars: Only one team uses calendars, and Path Design System does not offer pre-made calendar components. That team has low coverage compared to the others, and that’s fine!
A payment button that comes from a third-party integration. It looks like a Preply button, but it is not.
Every custom component has unique needs that the design system components do not offer.

At the same time, all the newly implemented product parts or the frequently refactored parts have the highest score, which has grown organically.

This topic is essential because we never ask teams to reach a specific visual coverage score. Visual coverage is a litmus test for many other factors heavily influenced by the Design System team and initiatives, not a goal to achieve at any cost.

Where to Run the Visual Coverage Count? On Users’ Devices!

The implementation of visual coverage (we shared an “implementation details” article here) ensures that it doesn’t affect the user’s UX. That’s why we can track visual coverage in production on users’ devices. This choice made everything more complex from a technical perspective (we also a/b tested it to ensure UX was not affected). Still, it also made the design system team independent in implementing, deploying, evolving, and gathering thousands of data daily.

If we decided to embed the visual coverage inside Preply’s E2E tests, we would have needed product teams to help us. However, product teams have product priorities, and depending on them would have meant delaying the project.

Some considerations for measuring the visual coverage on users’ devices:

Users use different devices with varying screen ratios and sizes, making using a fixed device as a reference impossible.
Users see different things based on the user-generated content and custom interactions with the Product.
The more a page is visited, the more calculations it sends. The final visual coverage score is an average of all the calculations.

As a result, the visual coverage is not 100% stable and can vary daily for the same page. We embraced this variability because the coverage would reflect what our users see!

Pages’ popularity impact

Again, the more a page is visited, the more calculations it sends. I want to stress this topic because it’s crucial to visual coverage. When we count the visual coverage, we send two pieces of data to the server: the number of pixels of the Path Design System components’ perimeters and the number of pixels of the other DOM elements. In DataDog (used for our dashboard), we calculate the average.

Imagine this scenario (numbers are imaginary):

The Home page is visited 5 times, and it sends the following events:

One event for the Design System pixels count. Let’s imagine the value is 10K
One event Non-Design System pixels: 5K

In total, the Home page sent 10 events. The Home page’s visual coverage is 67%.

The Setting page is visited 1 times, and it sends the following events:

One event for the Design System pixels count. Let’s imagine the value is 1K
One event Non-Design System pixels: 20K

In total, the Settings page sent 4 events. The Settings page’s visual coverage is 4%.

The math done in the dashboard is: all the Design System pixels count / divided by all the pixel counts. (10K*5 + 1K*2) / (10K*5 + 1K*2 + 5K*5 + 20K*2) = 44%. Can you see how the high popularity of the Home page weighs more than the low popularity of the Settings page? The final 44% is closer to the Home page’s 67% than the Settings page’s 4%.

Now, consider that our public and indexed pages are visited millions of times, and other “private” pages are visited hundreds of thousands of times. Other B2B pages could have “just” tens of thousands of visits. This is crucial information because, when we do the math, the teams behind the B2B pages could be entirely obfuscated by the popularity of other teams’ public pages. That’s why the Design System team cares more about the average per team than the final number (at the time of writing, our visual coverage is 77%, but most of the teams are above 85%) because the per-team visual coverage allows more granular monitoring.

The Components’ Weights

What counts for the visual coverage are the component’s perimeters. The perimeters can’t be the same for all the components for some reasons:

Path Design System components are UI components, while the pages contain hundreds of layout containers, which don’t impact the UX. UI components are smaller than containers, so the latter weigh more if the weight is the same.
A Design System’s divider doesn’t impact the UX as much as a searchable dropdown. At the same time, a Dropdown saves developers weeks of work while a Divider saves seconds.

So, the component weights are crucial for the visual coverage. We had a workshop to define the component’s importance. The idea is

To identify a 0–5 score for the component’s impact in terms of UI, UX, and accessibility.
We did it empirically to identify the importance of every factor: every team member voted and gave a 1–3 score to every factor; the importance is the sum of the votes.
Then, multiply the scores we identified at point 1 for the importance identified at point 2.
The result is a relative component’s score. In our case, it goes from 0 to 60.

The spreadsheet we used to identify the weight to assign to every component.

The actual weight we use for the visual coverage is reduced by a factor we identified through multiple iterations. For each iteration, we analyzed the score of the most representative pages (some of them are the most important ones, and some have low Path Design System’s components usage), empirically looking for the right coverage for the pages.

Other Design System Metrics

While elaborating on a long-term Design System strategy and a yearly roadmap, we thought of which meaningful metrics we could use to track the Path Design System’s success. The visual coverage seems to group all of them (because an increase in the visual coverage is the litmus of many other initiatives). Still, at the same time, it can’t reflect everything the design system team does. The metrics we care about are:

The Design System NPS score, of course.
Designers’ and developers’ efficiency: initially, this is a soft metric we ask through surveys without a real process in place to calculate it
The percentage of Path Design System foundations and components documented.
Path Design System platform features parity: the three platforms are Figma, Web, and App. Path Design System should support all of them at the same level (considering some unique capabilities and needs. For instance, Figma does not require Path’s components to be SSR/RSC-friendly).

And, more in general, we monitor

Path’s components usage: static analysis is insightful for the design system team.
Figma’s components detachments through Figma Analytics.

We also discarded implementing some metrics:

Figma adoption: We differentiate coverage (usage) from adoption (correct usage), but after some experiments and chats with other Design System teams, it seems too early for us to work on it, given the time required to implement and maintain something reliable.
The amount of CSS introduced per PR: This is one of the metrics Cristiano Rastelli identified in his Measuring the Impact of a Design System, which is the best resource on this topic. We suggest reading the article to find links to other companies’ activities to track their Design System impact.

Implementation details

Implementing the visual coverage, collecting feedback, improving it, validating it through a company-wide visual coverage increase initiative, etc., required months of work. Other companies we discussed showed interest, too.

If you want to experiment with it, we've got you covered:

Read (or send it to your engineers) the second part of this article, which is dedicated to the implementation details.
Fork, tweak, and use the visual coverage code.

Let us know what you think, and contact us at design-system [AT] preply.com, or directly with me on X, GitHub, or LinkedIn.

Resources

Preply looks very different from September 2023, and Path Design System is now quite connected to the new brand. If you are curious about how Preply executed a company-wide revolution in a few months, please read Maryna’s dedicated article, Turning the Page: Preply’s Successful Journey Through the Rebrand).

Special Thanks

I want to publicly kudos all the people involved in this project:

The whole design system team: Justine, Seif, Alex, Javi, Alisa. I love this team 😍
Vadym, Igor, Josh, Dmytro, Dominika, for the support and feedback 😊
Oleg, for creating the dashboard and helping with all the data.
Daniel, Alberto, for the early internal feedback.
All the Preply front-end engineers for supporting the design system, and this initiative. All the Preply app engineers for helping with the React Native implementation.
Javier, Serhii, Volodymyr, for supporting me with the React Native implementation.
Matteo, for trying the visual coverage with a different company.
Massimiliano, for the performance suggestions.
Dmitry, Omri, for the feedback.
Nicolas, for the frigging detailed review ❤️.

Production-grade gradient bordered, transparent, and rounded button

Stefano Magni — Thu, 21 Sep 2023 15:29:44 +0000

There are a lot of articles out there on making gradient-bordered buttons, but the ones I read do not provide a production-grade solution. At Preply, we needed to have a partially transparent background button with rounded corners, and with gradient border. We need to get it in our Design System, to think about a React Native solution. I know this article is long, but it includes the links to all the resources I read, the solutions I discarded, a link to the Figma file containing the gradient border, and a Codepen where you can play with it.

If you follow me: this article is quite different compared to the high-level, senior-oriented articles I usually write (here is the full list of them) 🙏 but I needed to write it down to be sure it helps other developers in the future.

I had to implement a new variant of the Preply button for our internal Design System: a semi-transparent button with a gradient-rounded border.

The existing solutions

There are some solutions out there but we needed to discard most of them because they do not match the following requirements:

The button must be transparent: this prevents us from using the most common solution presented in the How To Make a Button with a Gradient Border and Gradient Text in HTML & CSS post. The solution presented here relies on two elements, with a pseudo-element that has the same background color as the page background covering the real element that has a gradient background.
The button must be half transparent: the solution presented on StackOverflow's Button gradient borders with transparent background does not work since it gives a false idea of transparency and fits just one possible background.
It must have wider browser support: using engine-only CSS properties like -webkit-mask as suggested in Border with gradient and radius does not work well for us.
It must have rounded corners: this discards using CSS border-image (here is a good CSS Tricks article about it) since it's not compatible with border-radius.

it should have a fallback for the browsers that do not support the gradient border.
the border should not be a raster image to avoid pixelation.
It should be supported by React Native too: more on this topic will follow, forget about it for now.

The viable solution

The last mentioned one, border-image, in reality, meets all the requirements if mixed with a couple of tricks:

We can use an SVG instead or a raster image: pixelation is not a problem anymore.
It's widely supported: at the time of writing, Can I use reports 96.62% (95.43% if unprefixed) browser usage compatibility.

Through border-image-slice: stretch we can define the portions that should scale when the button gets bigger (the sides) and the ones that should not (the corners).
We can use an SVG that includes rounded corners to simulate the final rounded corners.
We can work around the few bugs reported by Can I use easily (we simply need to use the shorthand syntax) and they do not impact border-image-slice: stretch.

Step-by-step implementation

Step 1: Creating the SVG in Figma. This is the Figma file containing it.

Essentially, it's a 100x100 rounded square with the desired border. I made it 100x100 to avoid any kind of coordinates-related problem mentioned in the CSS Tricks article.

Step 2: Optimize the SVG. I was looking for the best way to embed the SVG (external or embedded image?) and the CSS Tricks article points to the great Probably Don’t Base64 SVG that, in turn, points to Optimizing SVGs in data URIs. You can leverage the findings from the latter through mini-svg-data-uri. The steps are

Reducing the SVG size: easily doable through SVGOMG
Passing the result through mini-svg-data-uri: $ npx mini-svg-data-uri image.svg image.svg.uri
Copy-pasting the image.svg.uri content

The result is

border-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' width='100' height='100' fill='none'%3e%3cpath stroke='url(%23a)' stroke-width='2' d='M8 1h84a7 7 0 0 1 7 7v84a7 7 0 0 1-7 7H8a7 7 0 0 1-7-7V8a7 7 0 0 1 7-7Z'/%3e%3cdefs%3e%3clinearGradient id='a' x1='20.27' x2='120.715' y1='125' y2='113.589' gradientUnits='userSpaceOnUse'%3e%3cstop stop-color='%23FF7AAC'/%3e%3cstop offset='.367' stop-color='%23FF7AAC'/%3e%3cstop offset='.682' stop-color='%232885FD'/%3e%3cstop offset='1' stop-color='%233DDABE'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e")

Choosing the inline or hosted one leads to a different UX, obviously. When I was playing with the hosted one, I hated that the border came with a little delay because of the time required to load the SVG remotely...

Please note that using mini-svg-data-uri is mandatory, otherwise, the browser does not recognize the SVG. Alternatively, this problem does not happen if you convert the SVG to Base64.

Step 3: Choosing the right border-image-slice and border-image-width values. If you need to do something more complex than the case reported in this guide, please refer to the official documentation (1, and 2) and again to the great CSS Tricks article. But for the sake of this guide, the value for either of them is just 8. Why? Because 8 is the border radius of the button.

If you imagine the border width acting as a sort of "mask" (I highlighted it in yellow in the next images) it must be 8 to be sure it includes the rounder corner. If, instead you set 2 as the real border width, the result is that the "mask" cuts out the rounded corners.

8px	2px

here you can see the final result of using 8 (correct) and 2 (incorrect) as rendered by the browser

So, the code is now

border-image: url("data:image/svg+xml,...") 8 / 8px;

p.s. if you set the border-width to 8 instead of border-image-width, you obtain the same visual effect... but the border is effectively 8 pixels, not 2 pixels like I set for the button! Keep it in mind if there is something wrong and you do not understand why.

Step 4: Choosing the correct border-image-repeat property. It's stretch, as I told you earlier. Essentially, it means "Don't touch the corners but extend the sides to be sure the image covers all the size of the button".

border-image: url("data:image/svg+xml,...") 8 / 8px stretch

Step 5: Enjoy the full code 😊

border-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' width='100' height='100' fill='none'%3e%3cpath stroke='url(%23a)' stroke-width='2' d='M8 1h84a7 7 0 0 1 7 7v84a7 7 0 0 1-7 7H8a7 7 0 0 1-7-7V8a7 7 0 0 1 7-7Z'/%3e%3cdefs%3e%3clinearGradient id='a' x1='20.27' x2='120.715' y1='125' y2='113.589' gradientUnits='userSpaceOnUse'%3e%3cstop stop-color='%23FF7AAC'/%3e%3cstop offset='.367' stop-color='%23FF7AAC'/%3e%3cstop offset='.682' stop-color='%232885FD'/%3e%3cstop offset='1' stop-color='%233DDABE'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e") 8 / 8px stretch;

Browser compatibility

At Preply, we must support a wide range of browsers. By looking at the Can I use table I shared above, it's hard to find a non-supported browser and I can be sure the gradient works everywhere. Anyway, I wanted to double-check it with my eyes and I initially used BrowserStack screenshots to test out the solution quickly on a lot of browsers, but it resulted in a bit of unreliable since it was not able to render most of the screenshots due to internal BrowserStack problems... Anyway, I'm happy because when it was able to render the normal button, it was also able to render the gradient one. Here are some of the screenshots on some "not-so-common" browsers

Samsung Galaxy S20	MacOS / Safari 13	Windows 10 / Edge 18

Fallback

There are for sure some browsers out there that do not support border-image. There are two options here:

Using CSS Feature Queries (@supports)

@supports (border-image: url("data:image/svg+xml,...") 8 / 8px stretch;) {
   border-image: url("data:image/svg+xml,...") 8 / 8px stretch;
}

This is the solution that gives you more freedom since you can literally change all the button properties you want outside and inside the @supports block, you are not limited to changing just border-image. Please check the Feature Queries browser compatibility, even if they are almost the same as the one of border-image.

Simply choose a fallback border-color. border-image overwrites the border-color, but the browsers not able to interpret border-image will use border-color. Here is an example of Internet Explorer 10.

I opted for the second option because the LESS + PostCSS transpilation process breaks the @supports property and I have not found a solution yet.

React Native

border-image does not work with React Native (which simply renders the normal border color instead) and the app developers pointed me to some different solutions.

If the button has a gradient border but does not have a transparent background: they would use a combination of react-native-linear-gradient with an element with an opaque background over it.
If the button must also be transparent, also masked-view would be needed.

I reported both cases because adding two dependencies for just one button could be overkill. When the button is added to the app, the app developers will decide what to do based on where the button is used. They will maybe opt for implementing the button in the app codebase instead of the Design System one, to avoid putting too many dependencies on the Design System codebase.

But the button is not used in the app at the time of writing so we have the time to go back to the designers asking what they prefer to do.

Additional notes

Don't forget about background: transparent; to make the button transparent, of course 😊.
Our Design System System's button already sets it but if the button has a background that's not totally transparent, remember to set the border-radius to 8px too to avoid the background coming out of the rounded borders.

Codepen liknk

You can play with the final result in this Codepen 😊.

Credit where credit is due

Thank you so much to all the creators sharing their solutions and tools! I would have spent maybe ten times the time on this without their explorations! ❤️

Some things I learnt from working on big frontend codebases

Stefano Magni — Thu, 01 Jun 2023 15:15:36 +0000

Until now (May 2024), I had three experiences working on very big front-end (React+TypeScript) codebases: WorkWave RouteManager, Hasura Console, and Preply.com. The former two are ~ 250K LOC, while Preply.com is close to 1M LOC, and the experiences are very different. In this article, I report the most important problems I saw while working on them, things that usually are not big deals if working on smaller codebases, but that become a source of big friction when the app scales.

Photo by Sander Crombach on Unsplash

Changelog

May, 2024
- add Non-straightforward CI scripts
- add Components accepting className
- add Not tracking architectural decisions
- add Spreading external dependencies and implementation details
- add Hide stores implementation details
- add Consuming Swiss army knifes
- add Major product changes and refactors
- update Never updating the NPM dependencies
June, 2023
- first article publication

My direct experience

First of all, let me describe the main characteristics of the two projects:

WorkWave RouteManager: the product is very complex due to some back-end limitations that force the front-end to take care of a bigger complexity. Anyway, due to the strong presence of the great front-end Architect (that's Matteo Ronchi, by the way), the codebase can be considered front-end perfection. The codebase is completely new (rewritten from scratch from 2020 to 2022), and trying and using the latest tools happened on a high cadence (for instance: we started using Recoil way sooner than the rest of the world, we migrated the codebase from Webpack to Vite in 2021, etc.), and the coding patterns are respected everywhere. The team was made by the four front-end engineers, including the architect and I. Here I was the team leader of the front-end team.
Hasura Console: the complexity of the project is not so high but the startup needs (pushing out new features as soon as possible) and the very back-end nature of the platform later resulted in huge technical debt and antipatterns that became into big friction points for the developers working on the front-end. The team was made of 12 front-end engineers, and later on the company decided to ditch the front-end project creating a 50x smaller one, and keep only the back-end/CLI projects. Here, I joined as a senior front-end engineer and then I became the tech lead of the platform team.
Preply.com: Preply scaled following a very experiment and data-driven approach, given its B2C nature and the million of users taking lessons on the platform every day. The natural business orientation lead to heavy outdated front-end dependencies and hard-to-work-with front-end projects. At the same time, Preply's goal to become a strong brand, to grow in the B2B market, and the tireless dedication to its internal culture and employees' satisfaction, drove the company to care a lot about internal tech excellence, to create a DevEx team inside the Platform team, and to lead some specimen tech initiatives. The company counted ~40 frontend-end engineers, some of them dedicated to React Native. I joined the platform team as a senior front-end engineer and then I moved to the Design System team.

Following, is a non-exhaustive list of examples coming from some of the characteristics/activities/problems I saw, grouped by categories.

Generic approaches
- Managing more cases than the needed ones
- Leaving dead code around
- Internal code dependencies and boundaries
- Implicit dependencies
- Spreading external dependencies and implementation details
- Big modules
- Code readability
- Uniformity is better than perfection
Working flow
- Not tracking architectural decisions
- No PR description and big PRs
- Suggesting big changes and approaches during code reviews
- When to fix the technical debt?
- Major product changes and refactors
- No front-end oriented back-end APIs
- Never updating the NPM dependencies
TypeScript
- Bad practice: Generic TypeScript types and optional properties
- Type assertions (as)
- @ts-ignore instead of @ts-expect-error and broad scope
- any instead of unknown
- ESLint rules kept as warnings
- Validating the external data
React
- HTML templating instead of clear JSX
- Lot of React hooks and logic in the component's code
- Components accepting className
- Hide stores implementation details
- Consuming Swiss army knifes
Tests
- Bad tests
- E2E tests everywhere
Developer Experience
- Deprecated APIs
- Care about the browser logs
- Developer alerts for unexpected things
- React-only APIs
- Non-straightforward CI scripts

Generic approaches

Managing more cases than the needed ones

This innocent approach leads to big problems and a waste of time when you have to refactor a lot of code trying to maintain the existing features. Some examples are:

Components/functions with optional props/parameters and fallback default values: when you need to refactor the components you need to understand what are the indirect consumers of the default values... But what happens if the usage of the default values is driven by network responses? You need to understand and simulate all the edge cases! And what happens if you find out that the default values are not used at all? I once saw a colleague of mine wasting four hours during a refactor for an unused default value...
Types that are typed as a generic string or generic record<string, any> when in reality the possible values are known in advance. The result is a lot of code that manages generic strings and objects while managing the real finite amount of cases would be 10x easier. Again, when you need to refactor the code managing "generic" values, you are going to waste time.

I touched on these topics in my How I ease the next developer reading my code article.

Leaving dead code around

You refactor a module, you remove an import of an external module and you are fine. What happens if the module was the last consumer of the external one? The external module becomes dead code that will not be embedded in the application (nice) but that will confuse everyone that's going around the codebase looking for solutions/utilities/patterns and will confuse the future refactorer that will blame anyone that left the unused module there!

And obviously, it's a waterfall... the external module could import other unused modules and they could depend on an external NPM dependency that could be removed from the package.json, etc.

Internal code dependencies and boundaries

Not enforcing (through ESLint rules or through a proper monorepo structure) strong boundaries among product features/libraries/utilities bring unexpected breaks as a result of innocent changes. Something like FeatureA imports from internal modules of FeatureB that imports from internal modules FeatureA and FeatureC, etc. brings you to break 50% of the product by changing a simple prop in a FeatureA's component. And if you have a lot of JavaScript modules never converted to TypeScript, you will also have a hard time understanding the dependency tree among features...

I strongly suggest reading React project structure for scale: decomposition, layers and hierarchy.

Implicit dependencies

They are the hardest things to deal with. Some examples?

Global styles that impact your UI's look&feel in unexpected ways
A global listener on some HTML attributes that does things without the developer knowing about them
A generic MSW mock server that all the tests used but it's impossible to know what handlers are used by what tests

Again, poor the refactorer that will deal with those. Instead, explicit imports, speaking HTML attributes, inversion of control, etc. allow you to easily recognize who consumes what.

Spreading external dependencies and implementation details

External dependencies should be hidden and consumed by controlled code if you write a custom function called addOneDayToDate compared to spread dateFns.addDays(currentDate, 1) everywhere is better because the function depends on DateFns, which is centralized, easy to test, and change.

Big modules

This is another very subjective topic: I prefer to have a lot of small and single-function modules compared to long ones. I know that a lot of people prefer the opposite so it's mostly a matter of respecting what is important for the team.

Code readability

I'm a fan of the The Art of Readable Code book and after spending 2.5 years working on a big and complex codebase with zero (!!!) tests, I can tell how much code readability is important.

This also really depends on the number of developers working on a codebase, but I think it's worth investing in some shared coding patterns that must be enforced in PRs (or even better if they can be automated through Prettier or similar tools).

I publicly shared the ones we were using in WorkWave in this 7-article series: RouteManager UI coding patterns. The internal rule we had was that "patterns must be recognizable in the code, but not authors".

No silver bullets here, the important thing IMO is that readability and refactorability are kept in mind by everyone when writing code.

Uniformity is better than perfection

If you are about to refactor a module but you do not have time to refactor also the two modules that are coupled to it... Consider not refactoring it to leave the three modules uniform among them (uniformity means predictability and less ambiguity).

Working flow

Not tracking architectural decisions

Architectural decisions and changes are key to comprehending why a project was designed and how it evolved over time. Usually, these decisions are not 100% reflected in the codebase since big codebases always require incremental approaches.

It is important to track those decisions to avoid dealing with approaches partially applied, refactors partially done, etc., without a precise idea about the timeline of those decisions and what they were trying to solve.

Usually, this problem explodes when the engineers who remember those decisions leave the company, and the new ones are doomed to remain ignorant forever.

On a small scale, this also refers to the awkward changes and/or shenanigans you make to get something done. See this great Josh W. Comeau's example.

// Detect dark theme var iframe = document.getElementById('tweet-1695066496450666995-468'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1695066496450666995&theme=dark" }

No PR description and big PRs

That's such an important topic that I wrote four articles about it. Start with the most important one: Support the Reviewers with detailed Pull Request descriptions

And if you are curious you can dig into some real-life examples I documented here

Suggesting big changes and approaches during code reviews

PRs are not the best place to suggest big changes or completely change the approach because you are indirectly blocking releasing a feature or a fix. Sometimes is crucial to do it, but maybe the initial analysis and estimation steps, pair programming sessions etc. works better to help shape the approach and the code.

When to fix the technical debt?

That's a great question, no silver bullet here... I could only share my experience until now

In WorkWave we were used to dealing with technical debt on a daily basis. Fixing tech debt is part of the everyday engineers' job. This can slow down the feature development in favour of having a deep knowledge of the context and keeping the codebase in a good shape. It's like knowing that you are slowing down today's development to keep tomorrow's development at the current pace.
In Hasura, we cannot deal with technical debt due to the needs to deliver new features. This transformed in a lot of frontenders going slower compared to their potential, sometimes introducing bugs, and offering an imperfect UX to the customers. It happened after years, obviously.
In Preply, engineers can dedicate 20% of their time to tech excellence initiatives, some of them driven by the company and other ones proposed by the teams themselves.

You can read more about a good example of Hasura's problems in my Frontend Platform use case - Enabling features and hiding the distribution problems article. Also, you could read what happened to our E2E tests here after all the tech debt problems we were facing.

Major product changes and refactors

Major product changes (the complete rewrite of WorkWave RouteManager, Preply's complete rebrand, etc.) are also perfect for introducing refactors or clearing tech debt that has been there for ages. The reason is that all the knowledge accumulated in the previous years gives us a more comprehensive vision of what is needed and what needs to be cleared, leaving the new product way better than when it started (a sort of greenfield project inside an existing product).

No front-end oriented back-end APIs

By "no front-end oriented" I mean APIs not designed with the end customers' UX in mind and a lot of complexity pushed to the front-end in order to keep the back-end development lean (ex. Embedding a lot of DB queries in the front-end avoiding exposing a new API from the back-end). This approach is natural during the initial evolution of a product but will lead to more and more complex front-ends when the product needs to scale.

Never updating the NPM dependencies

Again, based on my own experiences:

In WorkWave I used to updating the external dependencies on a weekly basis. Usually, it takes me 30 minutes, sometimes 4 hours.
In Hasura, we were used not to update them, finding ourselves, enabling legacy-peer-deps by default, leveraging NPM's overrides and being unable to update any GraphQL-related dependency. Other than having a lot of PRs that completely break the build because of a new dependency.
In Preply, the outdated TypeScript version made impossible to enable exactOptionalPropertyTypes and noUncheckedIndexedAccess which caused more than one production incident. At the same time, the need to become SOC2 compliant (necessary to expand in the B2B market) got caring about the dependencies a first-class citizen (after a big months-long initiative to update all of them).

And since maintaining dependencies has a cost, you should carefully consider if you really need an eternal dependency or not. Is it maintained? Does it solve a complex problem I prefer to delegate to an external part?

As an alternative approach, valid only for very very small projects, you can also consider to copy/paste the code of some dependencies inside a "vendor" directory, linking the original project and tracking which version the code belongs to (at the cost of not being able to update it and that other must know they should not install the same dependency).

TypeScript

Bad practice: Generic TypeScript types and optional properties

It is very common to find types like this

type Order = {
  status: string
  name: string
  description?: string
  at?: Location
  expectedDelivery?: Date
  deliveredOn?: Date
}

that should be represented with a discriminated union like this

type Order = {
  name: string
  description?: string
  at: Location
} & ({
  status: 'ready'
} | {
  status: 'inProgress'
  expectedDelivery: Date
} | {
  status: 'complete'
  expectedDelivery: Date
  deliveredOn: Date
})

that is more verbose but acts as pure domain documentation, removes tons of ambiguity, and allows writing better and clearer code.

The topic is so important and has so many great advantages that I wrote a dedicated article to the topic: How I ease the next developer reading my code.

Type assertions (`as`)

Type assertions are a way to tell TypeScript "shut up, I know what I'm doing" but the reality is that barely you know what you are doing, especially thinking about the consequences of what you are doing...

This happens very frequently in tests, where big objects are "typed" with type assertions... Resulting in the object going outdated compared to the original type... But you realize it only when the tests will fail and you now left room for a lot of future doubts about the test failures...

The solution: type everything correctly and eventually prefer @ts-expect-error with an explanation of the error you expect.

Read Why You Should Avoid Type Assertions in TypeScript to know more about the topic (and keep in mind that also the JSON.parse example shown there can be typed by using Zod parsers).

`@ts-ignore` instead of `@ts-expect-error` and broad scope

@ts-expect-error issues could be auto-fixable in the future, compared to @ts-ignore (that's another way to shut up TypeScript).

More, @ts-expect-error should be applied to the smallest possible scope to TS accepting unintended errors.

// ❌ don't
// @ts-expect-error TS 4.5.2 does not infer correctly the type of typedChildren.
return React.cloneElement(typedChildren, htmlAttributes); // <-- the whole line is impacted by @ts-expect-error

// ✅ do
return React.cloneElement(
  // @ts-expect-error TS 4.5.2 does not infer correctly the type of typedChildren.
  typedChildren, // <-- only typedChildren is impacted by @ts-expect-error
  htmlAttributes
);

`any` instead of `unknown`

TypeScript's any gives you freedom (that's generally bad) of doing everything you want with a variable, while unknown forces you to strictly guarantee runtime the runtime value before consuming it. any is like turning off TypeScript while unknown is like turning on all the possible TypeScript alerts.

ESLint rules kept as warnings

ESLint warnings are useless, they only add a lot of background noise and they are completely ignored. Rules should be on or off, but never warnings.

Validating the external data

In the software world, the rule of "never trust what the frontend sends to the backend" is crucial, but I'd say that in a front-end application armed with TypeScript types, you should not trust any kind of external data. Server responses, query strings, local storage, JSON.parse, etc. are potential sources of runtime problems if not validated through type guards (read my Keeping TypeScript Type Guards safe and up to date article) or, even better, Zod parsers.

React

HTML templating instead of clear JSX

JSX which includes a lot of conditions, loops, ternaries, etc. are hard to read and sometimes unpredictable. I call it "HTML templating". Instead, smaller components with a clear separation of concerns among the components are a better way to write clear and predictable JSX.

Again, I touched on this topic in my How I ease the next developer reading my code article.

Lot of React hooks and logic in the component's code

I'm a great fan of hiding the React component's logic into custom hooks whose name clearly indicates the scope of the hook and the consuming it inside it. The reason is always the same: long code before the JSX makes reading the JSX harder.

Components accepting `className`

Components are designed to encapsulate and hide some logic and give the external world the result of this logic. Their UI is part of the encapsulated APIs the consumer should not be able to change. Usually, components also accept className to allow consumers to customize small parts of the component's UI (this is the initial goal). Instead, the result is an uncontrolled and hard-to-predict backdoor to rape all the UI details of the components and their children in seconds.

Like all the JavScript details, the styling details should be encapsulated and hidden, exposing only some generic configurations to the consumer. These configurations explicitly mark what the component offers and what the consumer wants to obtain (like variants, type, mode, etc.).

As a refactorer, when you see a component accepts className, you already know how hard your life will be.

Hide stores implementation details

Something I saw working like a charm on WorkWave RouteManager is hiding the stores under modules that export React-only, store-independent APIs. We started using Recoil way before the rest of the world, and later on, we migrated to Valtio because it better covered our needs. The migration was painless because Recoil was just an implementation detail of modules that export pure React APIs like useSelection.

Consuming Swiss army knifes

Some React components are generic by design due to the huge number of cases they manage (think of a table, a date picker, a modal, etc.). This makes hard to track what consumers need out of the 100 features those generic components do. As a result, refactoring those components or the consumers is hard. My suggestion is to create intermediate and vertical components that act as proxies for the more complex ones.

The vertical components' name and description allow the reader to understand what they do and need without digging into the details of how the original complex components are consumed (for instance: UserList which uses just the sorting options of Table is clearer compared to digging into how Tutors, Students, and Managers pages use Table).

Tests

Bad tests

As a test lover and instructor (I teach about front-end testing at private companies and conferences) I can say that bad tests are the result of lacking experience on this topic, and the only solution is help, mentoring, help, mentoring, help, mentoring, etc.

Anyway, the false confidence that tests can offer is a big problem in every codebase.

I suggest reading two of my articles:

E2E tests everywhere

E2E tests do not scale well because of the need for real data, a real back-end, etc.

From this perspective, Preply is a great example (and the only successful one I saw in my working life) of what you can achieve when the user experience is considered crucial by the leadership: E2E test are mandatory and the strong Continuous Delivery approach lead to 98%+ stability of the E2E test suite that ensure a lot of happy path are always functional.

Also, in this case, I suggest reading some of my articles:

Developer Experience

Deprecated APIs

When code is marked as @deprecated, the IDE shows it as strikethrough'ed and present the documentation, helping the developers realize that they should not use it.

An example:


/**
 * @deprecated Please use the new toast API /new-components/Toasts/hasuraToast.tsx
 */
export const showNotification = () => { /* ... */ }

Care about the browser logs

Console warnings (coming from ESLint, from TypeScript, from React, from Storybook, etc.) add a lot of background noise that mixes with the important logs you could trace. Care and remove them in order to avoid the developers ignoring your own important alerts due to the high noise.

Developer alerts for unexpected things

Runtime things (ex. server responses) could not be aligned with the front-end types. If you do not want to break the user flow by throwing an error, at least track the error through something that could alert you about it (like Sentry, or whatever other tool), so a short time will pass between the error coming out and you fixing it.

React-only APIs

If you are creating an internal library, prefer to expose only React APIs. The big advantage is that you count on React's reactivity system, and managing dynamic/reactive cases in the future will be easier because you are sure the consumers of your React APIs are re-rendered for free and always deal with fresh data.

Non-straightforward CI scripts

CI pipelines should just launch scripts present in the package.json without additional logic that increases the cognitive load and makes it harder to replicate errors locally or in another environment. Think about the painful process of trying to decipher what a CI step does to replicate it locally to dig into the root cause of the issue. Maybe CI uses a tool you do not master, maybe CI uses a particular configuration, and all these take you depending on other colleagues/teams who own everything CI.

CI pipeline should only care about setting up everything with the correct Node.js version (set by the frontenders who maintain the codebase) and launching some CI-dedicated scripts (for instance: ci:lint, ci:build, ci:ts-check, ci:test:unit, ci:test:e2e, etc.). This decouple the scripts launched in CI from who knows better the JS ecosystem, getting everything simpler.

Credit where credit is due

Thank you so much to M. Ronchi and N. Beaussart for teaching me so many important things in the last few years ❤️ a lot of content included in this article comes from working with them on a daily basis ❤️

Frontend Platform use case - Acting before a real Product need

Stefano Magni — Wed, 31 May 2023 13:05:05 +0000

This article is about what happens when you anticipate Platform activities before a real company need arises.

Photo by Trevor McKinnon on Unsplash

The problem

In the Hasura Console, Datadog was initially used to track some internal errors and alerts. Then, the plan was to move to Sentry and, in parallel, to Heap to allow the company to create funnels and track user activities to base future Product decisions on data. The company could opt for other tools in the future, obviously.

This changes in terms of tools and not having a unified direction about how to practically implement them in the Console UI (the main Hasura's front-end project) led to a lot of patterns used by different developers.

More: since Hasura treats very sensitive data (the env secrets and database details of the clients) it's crucial that this data are not leaked to third-party services.

The sensitive data problem and the data redaction applies specifically to how Hasura uses Heap, since by default Heap (and a lot of other similar tools) tracks everything the users do in the browser to allow later creating and analysing funnels. At the same time, the Console UI exposes some sensitive data (the database table names, for example) in some HTML attributes (some ids, some data-testids, etc.) that must not be shared with third-party services.

When Heap has been added to the Console UI, Heap auto-tracking has been disabled in favor of manually adding some dedicated HTML attributes (ex. data-heap) that then allow the stakeholders to create funnels like "the user clicked on the element that responds to the [data-heap='Save action'] selector". This is a great thing for engineers too, because knowing what the users do in the Console UI allows also to start discussions with stakeholders about "eventually removing some not-so-used features that force us to keep some not-so-maintained dependencies and some not-so-modern code".

This approach is safe from a data leak perspective, but does not scale from a feedback loop point of view. Due to the internal release management, normally no less than two weeks pass from a stakeholder saying "I'd like to track this button" and the new HTML attribute present in the Hasura products.

The solution

In the Platform team (I was one of the frontenders of the Platform team, along with N. Beaussart and N. Inchauspé) we decided to

Create some analytics-related React APIs to centralize adding the tracking HTML attributes and easily switch from a tracking tool to another one in the future without touching the whole codebase
Move the data redaction from the app-level to every single feature level, in order to simplify the work of every Feature team. Every Feature team then had to
1. Analyse the data leaked by their own features in the HTML attributes
2. Remove the data leak
3. Remove the data-redaction from every single feature

As a result, the stakeholder could gain automatic tracking and focus on more interesting things like deciding if to expand tracking to other distributions (like on-prem) or not.

The first point was the easy part, the developers working on the Console easily adapted to use the new <Analytics/>, getAnalyticsAttributes, programmaticallyTraceError APIs etc.

The second point, instead, required a lot of refactors, dig into the features code (not something very easy in the Hasura Console), and a lot of manual tests to be sure no features are broken. But this is the kind of tasks for the Platform team, right? Especially if you think about unleashing the Feature teams' potential in terms of re-enabling the auto tracking, right? Well, not exactly...

Why it did not work

The work every Feature teams should have done (fixing leaking sensitive data and reenabling the auto tracking) requires time, and this time should be planned and prioritised. What happened is that I found myself pushing for this activity, the Head of Analytics pushing too... But the company had not a strong interest in prioritising everything analytics compared to other activities!

Not all the work is wasted! In fact, the the new React APIs are in use and they leave room for future evolutions. But the most impacting process, removing the block to auto-tracking, remained uncompleted despite the initial big effort from the Platform team.

Conclusion

I wanted to share this experience because it is the perfect example for when a Platform team does something not very coupled to the Product goals. This kind of problem could happen especially if the Platform team has not a Product Manager, something I spoke about in my Frontend Platform use case - Creating a roadmap without a Product Manager article.

Frontend Platform use case - Enabling features and hiding the distribution problems

Stefano Magni — Wed, 31 May 2023 06:19:51 +0000

Like other similar products, Hasura is distributed and can be consumed in a lot of different ways, due to different client’s needs and the evolution of the product during the last years.

The lack of a centralized way to identify the server type and version highly impacts working on the Hasura Console (the most important Hasura’s front-end product) and enabling a feature or not. Things get even worse when the distribution matrix crosses the pricing tiers and their frequent changes.

Here is a walkthrough the plan to tackle the problem by offering new React APIs to the Hasura frontenders.

Photo by John Barkiple on Unsplash

How the Hasura Console identifies the server "type"

The Hasura Console "assets" (CSS, JS, static files) are in charge of the Console build tools but the HTML serving them unfortunately is not. The various servers were used to add some logic and some environment variables in the global scope (populating a window.__env object), load the Console assets, and the Console can consume the window.__env object and decide what to show to the users and what not.

How the abovementioned situation became such a big problem for the Console developers? Because of the Hasura distribution options:

The Hasura server embeds the Console assets in order to allow the clients to work offline
At the same time, the Hasura server always tries to load the assets from the CDN before fallback to the embedded ones (to fix hotfixes on the fly)
The Hasura CLI acts as a server that in turn talks to the Hasura server. When served by the Hasura CLI, the Console talks with the Hasura CLI and the Hasura server based on the actions to do
There is no way to force the clients to update their Hasura CLI, resulting in a lot of outdated distributions
In either the Hasura Server or Hasura CLI, the client could be logged as an "Enterprise" user
The Hasura Console can be served by Lux too, the server behind Hasura Cloud
Hasura Cloud has different pricing tiers that evolved over the years
Hasura was introducing a new "Enterprise" distribution and licensing model that differs from the previous one

If you mix it with the fact that

The window.__env object is not documented nor typed
The domain names for the plans themselves are not clear (what a "Pro" Console is is a source of confusion too)
How to launch the application in all the modes/types (they turned out to be 24 different combinations) is hard
Some "new" server implementations presented some "old" domain names with the goal of easing the Console's developer life

you will not be surprised if there were a lot of creative ways to identify the current mode/plan, including a lot of duplication, a lack of centralized management, and a lot of subsequent PRs needed to adjust enabling features after the release.

The feature-first APIs

During one of the internal Front-end Office Hours, we discussed the generic idea to stop dealing with plans/modes/types, and creating some feature-first APIs that hide the implementation details of identifying the plans and that allows the developers to simply tell something like "I need to show this feature in Cloud, that's all". Since this was one of the most reported problems by the frontenders (if you are curious about how we identified the problems to work on, read Frontend Platform use case - Creating a roadmap without a Product Manager, we (the Platform team) immediately jumped on it.

The generic plan was something like:

Quickly creating a POC to be discussed internally and understand if it would have fulfilled the desired needs
Identifying all the existing modes/plans/types/tiers impacting the Console/features and documenting them
Analyzing the environment variables and APIs served by the different servers could have helped us
Implementing a new React wrapper (<LoadHasuraPlan />) around the application that identifies the "Hasura plan"
Dealing with just one/two Hasura plans to later iterate and all the other plans
Creating the "feature-first" APIs: some React components and hooks that allow to easily deal with when a feature is enabled or not (and if not, also telling "why" to the Hasura developers to allow them to propose some alternative or upselling UIs)
Creating some testing and, especially, Storybook utilities to help the Hasura developers to simulate all the edge cases straight from their working tool of choice
Easing the Hasura developers to add and configure new features
Easing the Hasura developers to add and configure new ways to detect the user/license properties
Refactoring the existing implementations of the most important one/two plans to dogfood the new APIs
Presenting the APIs to Hasura developers to gain their feedback and keeping them updated about the current progress of the new APIs
Going back to point 5 to deal with all the Hasura plans one by one
(maybe) Sitting at a table with the back-end developers to expose a new and documented API dedicated to moving the implementation details from the Console to the server

Let me elaborate on some of the above steps.

The high-level APIs

At a very high level, this diagram summarizes the new APIs and how to use them from the App.

(since dev.to compresses the images, here it the link to the original Excalidraw)

A <LoadHasuraPlan /> wrapper that identifies the Hasura plan and wraps the app with a React Context Provider whose value is a Zustand store that stores the Hasura plan details
A useIsFeatureEnabled React hook usable everywhere in the app that received the name of the feature, retrieves the Zustand store from the React Context and passes both the store and the feature object to a checkCompatibility function
All the implementation details are just a sort of state machine to identify the plan and a lot of if statements to check if the feature is enabled for the current Hasura plan (the TypeScript part includes more complexity, you can find more info about it later in the article)

Please note: despite Zustand allowing us to expose Vanilla JS APIs, we decided from the very beginning to expose only React APIs. The big advantage is that if we offer only React APIs, we can count on React's reactivity system! That means that managing dynamic/reactive cases like "the user activated the license during the Console lifetime" is a no-brainer because we update the store and all the component/hook consumers automatically re-render.

Otherwise, exposing Vanilla JS APIs

Would have forced us to expose some subscription-like APIs to be sure the API consumers always work with the latest data
Would have opened the doors to a lot of freedom and creativity for all the Console's developers. While speaking about working on big codebases, and speaking from the standpoint of who usually have to later deal with a lot of refactors due to the mentioned creativity... It is way better to limit the options

The initial POC

You can find the initial POC here but the main goal of creating a POC was

To validate the fallback/upselling APIs
To validate the idea with the Console developers and to collect their feedback since they are the real users of the new APIs and they have all the feature-related complexity in mind

At the end of the POC's README, you can find all the feedback we collected (asking for different API shapes, asking about the Feature Flags, etc.).

Identifying all the existing modes/plans/types/tiers impacting the Console/features

This step required a lot of back and forth with a lot of other stakeholders to identify the available options, all the differences, some of the design choices behind the current state, etc. Then, I spent some time on a lot of trial and error to document all the cases and elaborate some steps to correctly identify all the Hasura plans. It turned out that the window.__env object plus a series of three XHR requests in a row (in the worst-case scenario) are enough to correctly identify all 24 possible distribution combinations.

Implementing the new `<LoadHasuraPlan />` React wrapper

The main characteristic of the wrapper are:

It can be completely disabled through a passThrough prop. This is important to get it merged on the main branch as soon as possible way before the new APIs are ready. Putting everything on main instead of long-living branches is crucial in terms of maintainability and size of PRs, and it's the logic at the base of Feature Flags for instance.
It can be enabled through some secret, in-app, developer tools to test it out everywhere at any time. You can refer to the great Make your own DevTools article if to learn how to effectively create something similar.
The internal list of static (like the window.__env object) and dynamic (like the license and the pricing tiers) resources consumed to identify the current Hasura plan must be extended easily by everyone to cover future needs.
It must be tested thoroughly to be sure it covers all the existing and eventually unknown edge cases. This is crucial because the crazy number of combinations leads for sure to some unmanaged cases and if the wrapper does not handle them could make the Hasura customers block from using the Console!

That's why I spent a good amount of time writing a lot of unit tests like this

it('When the server lacks the server_type, the EE license API fail, and the consoleType is pro, then set the Hasura plan as EE Classic and render the children', async () => {
  // Arrange
  const versionApiResponse: VersionApiResponsePayload = {
    version: '',
  };
  const rawServerEnvVars: ServerEnvVars = { consoleType: 'pro' };

  // Act
  server.use(
    rest.get('*/v1/version', (req, res, context) =>
      res(context.json(versionApiResponse))
    ),
    rest.get('*/v1/entitlement', (req, res, context) =>
      res(context.status(404))
    )
  );

  render(
    <LoadHasuraPlan rawServerEnvVars={rawServerEnvVars}>
      <>
        <RenderHasuraPlanName />
        <RenderEeLicenseStatus />
      </>
    </LoadHasuraPlan>,
    { wrapper: TestWrapper }
  );

  // Assert
  expect(
    await screen.findByText('Hasura plan: eeClassic')
  ).toBeInTheDocument();
  expect(
    screen.getByText('EE license is not expected in this environment')
  ).toBeInTheDocument();
});

Creating the "feature-first" APIs

The most interesting part here is not about the APIs themselves but about the hidden TypeScript gymnastics. Let me explain what was the goal we had in mind: think about a feature like OpenTelemetry integration, which compatibility is defined by the following object

const openTelemetry = {
  ce: 'disabled',
  ee: {
    withLicense: 'enabled',
    withoutLicense: 'disabled',
  },
} as const satisfies Compatibility;

When the developer calls isFeatureEnabled('openTelemetry') and the current Hasura plan is ce the result must be, from a TypeScript perspective:

{
  status: 'disabled,
  doMatch: {}, // ← empty object
  doNotMatch: { ee: { withLicense: true } }, // ← does not include `withoutLicense`
  current: { hasuraPlan: { name: 'ce' } },
}

Why have I said, "from a TypeScript perspective"? Because TypeScript means autocompletion for known properties and error for unknown ones.

Why is it so much important to ensure, at the type level, that the doNotMatch object does not include the ee.withoutLicense property? Because the developer must be prevented from doing something like this!

function OpenTelemetry() {
  const {
    status,
    doNotMatch,
  } = useIsFeatureEnabled('openTelemetry');

  if (status === 'disabled') {
    if (doNotMatch.ee.withLicense) {
        return (
          <div>Activate your license to use OpenTelemetry!</div>
        );
      }

    if (doNotMatch.ee.withoutLicense) {

        // WHAAAT? This edge case does not exist!!!!!! 😱😱 But the developers could try to cover
        // all the edge cases without realizing some of them are impossible!
        // TypeScript could prevent this error by throwing an error that `ee.withoutLicense` does
        // not exist for the `OpenTelemetry` feature! 🎉
        return (
          <div>Deactivate your license to use OpenTelemetry!</div>
        );
      }
    }
  }

  return <div>Enjoy OpenTelemetry!</div>;
}

To be honest: I am not sure that, in the end, the TypeScript gymnastics we implemented (customizing a couple of utility types of the great type-fest) are worth it, but in the first implementation we kept them.

Creating some testing and Storybook utilities

The Hasura Console developers spend most of their time working in Storybook, hence offering some ad-hoc utilities to write all the component stories for all the different Hasura is crucial in terms of Developer Experience.

We initially opted for exposing some Storybook Decorators that allow simulating the Hasura plans, without also offering some in-Storybook UI addons to switch between plans at runtime. The goal was to offer something very practical without implementing something expensive before validating the overall idea. So we just:

Expose a hasuraPlanDecorator from the new IsFeatureEnabled directory
The hasuraPlanDecorator accepts the name of the Hasura plan to simulate
Based on the name of the plan, hasuraPlanDecorator returns a React Context Provider whose value is an externally-initialized Zustand store
The basic plan is used for every story
Every story can import and customise its own decorators passing a different plan name

The whole implementation is something like

type MockedPlan = {
  name:
    | 'ce'
    | 'eeWithoutLicense'
    | 'eeWithActiveLicense'
    | 'eeWithExpiredLicense'
    | 'eeWithDeactivatedLicense'
    | 'eeWithLicenseInGracePeriod';
};

// --------------------------------------------------
// MOCK PROVIDERS APIS
// --------------------------------------------------

const HasuraPlanMockProvider: React.FC<{
  mockedPlan: MockedPlan;
}> = props => {
  const { mockedPlan, children } = props;

  switch (mockedPlan.name) {
    case 'ce':
      return (
        <MockStoreContextProvider
          params={createNewStore({
            hasuraPlan: { name: 'ce' },
            serverEnvVars: {}, // The server env vars (aka window.__env) is not mandatory
          })}
        >
          {children}
        </MockStoreContextProvider>
      );

    // etc.

  }
};

// --------------------------------------------------
// STORYBOOKs APIS
// --------------------------------------------------

export const hasuraPlanDecorator =
  (mockedPlan: MockedPlan) => (Story: React.FC) =>
    (
      <HasuraPlanMockProvider mockedPlan={mockedPlan}>
        <Story />
      </HasuraPlanMockProvider>
    );

And every story can simulate a different plan with something like this

export const OpenTelemetryStory: ComponentStory<typeof OpenTelemetryEeRequired> = () => {
  return <OpenTelemetryEeRequired />;
};

OpenTelemetryStory.decorators = [hasuraPlanDecorator({ name: 'eeWithoutLicense' })]; // <-- simulating a custom Hasura plan

OpenTelemetryStory.storyName = '💠 Primary';

The same APIs can be leveraged for the unit tests too.

Refactor the existing implementations of the most important one/two plans to dogfood the new APIs

This is the longest part of the first iteration. The fast way to do that is to simply change the core of all the abstractions that have been created to tackle the problem. But this is suboptimal and does not allow properly dogfooding the new APIs. The right thing to do, instead, is to deeply understand the existing abstractions, go back to the problem that led to creating the abstractions and then solve the problem in an easier way thanks to the new API.

More: it also means updating all the tests and stories involved to test and document the refactored components and functions.

The result is hundreds of lines of code removed and a lot of functional (manual) tests required to be sure no features are broken. Unfortunately, it also means digging into some existing bugs that now arise thanks to the full understanding of the different distribution combinations.

You could wonder: why not just using Feature Flags?

Because a hypothetical feature flag like "Enable OpenTelemetry for the EE users" is not helpful if you are not able to properly identify who are the EE users. The whole problem presented in this article is specific to identifying plans and hence users.

Conclusion

Many thanks to N. Beaussart and N. Inchauspé (the frontenders of the Platform team along with me) for supporting me designing and discussing the plan presented in this article 😊

Frontend Platform use case - Creating a roadmap without a Product Manager

Stefano Magni — Wed, 31 May 2023 06:07:33 +0000

How we created a roadmap for the Frontend Platform team without clear expectations.

Photo by Mitch Mckee on Unsplash

August 2022, Hasura changes the internal team's structure from horizontal teams (the "Console team" with all the frontenders, the "Server team", etc.) to vertical, full-stack "Feature teams" ones that cover a Product need (the "Native DB team", the "GraphQL Services team", etc.). Also, a generic "Product Platform team" has been created to cover all the Product features that do not fall under the umbrella of the various Feature teams. The Platform's team scope also included dealing with high-level tasks and problems that would have allowed the Feature teams to move faster while developing product features.

We (N. Beaussart, N. Inchauspé, and I) were the front-end side of the Platform team and we wanted to ease all the other frontenders' lives by removing some of the huge technical debt that prevent them to work effectively and to release features safely and without introducing a lot of bugs and "fix" PRs.

We then tried to:

Identify what front-end tasks fall under the umbrella of the Platform teams in other companies
Identify the most important activities to work on

Identify what front-end tasks fall under the umbrella of the Platform teams in other companies

Please keep in mind that while Platform teams are very common in a lot of companies, frontenders are not usually included in those teams. As a result, the public resources for front-end Platform activities are limited. Anyway, we found some great resources that it's worth mentioning:

And if you need more generic info: this is an introduction about Platform teams from Gergely Orosz. The articles are behind a paywall but they are some of the best out there https://blog.pragmaticengineer.com/platform-teams/
An article about what allows a Platform team to work well or not https://www.microtica.com/blog/why-your-platform-engineering-team-isnt-awesome
What does it mean to be in a UI Platform team https://yiou.me/blog/posts/thinking-in-platform
A great article about Hubspot platform team https://product.hubspot.com/blog/frontend-platform
The Croz experience at building a successful Platform team
https://teamtopologies.com/industry-examples/building-a-successful-platform-team-at-croz
A Quora answer about the alignment between Platform teams and Feature teams https://www.quora.com/How-do-product-platform-engineers-keep-in-touch-with-the-needs-of-product-engineers/answer/Alexis-Larry

By reading them, we found confirmations about the tasks covered by frontenders in Platform teams, such as:

Fix the most problematic technical debt that slows down the frontenders of the Feature teams
Fix the problems that will slow down the Feature teams in the short term by anticipating the new Product features' needs
Maintain Design Systems
Create some common patterns and best practices and automate getting them respected
Improve and stabilise the various application tests
Evaluate new solutions and technologies to better solve internal technical problems
Manage and optimise the front-end tools and dependencies
Mentor other frontenders and spread knowledge

Due to bug technical debt present in Hasura's front-end projects, how could we (the frontenders of the Platform team) ensure our activities were aligned with the short-term Product needs? How to create a sort of roadmap and stick with it regardless of the everyday problems that could arise from such a complex Product and a lot of engineers? Go ahead 😊

Identify the most important activities to work on

We proceeded by:

Interviewing all the stakeholders that are directly or indirectly impacted by the Hasura Console (Hasura's main front-end application and the most problematic one), including
1. All the developers working on it on a daily basis
2. All the developers working on it time every now and then
3. All the Engineering Managers of the teams including developers from the previous points
4. The Documentation team
5. The Graphic Design team
Collecting all the answers in a shared spreadsheet
Summarising the most requested area of interventions (the "Engineering priorities") and categorising them
Preparing a presentation for the higher-ups that include
1. A slide for every single activity (15 in total) with a clear explanation of why it was important, what the activity is, how many (in percentage) stakeholders asked for it, and a T-shirt size
2. A table that acted as the focus of the presentation, where we asked the higher-ups to put a "Product priority" to then mixed with the "Engineering priorities" that we identified

The final goal was to list specific activities with

The Engineering "priority"
The Product "priority"
The T-shirt size

And creating a roadmap out of this list was a breeze! This also helped us to identify what to not work on, something not very straightforward when you deal with a lot of problems and requests on a daily basis.

Here is an example of the table we created with a bunch of activities we identified:

Area	Activity	T-shirt size	Eng. ⚠
Perf+stability	Sentry: Mapping feature directories to teams.	👕👕	⚪
Architecture	Providing tools to respect the libraries strategy in the Console.	👕👕👕	🔴
Architecture	Update GraphQL and GraphiQL	👕👕👕👕	🔴
DX	Better docs and tools to run the Console in all modes/types.	👕👕	⚪
Regression	Storybook Tests: Reduce the drag in testing different Console modes/types.	👕👕	🟡

Conclusion

The higher-ups highly appreciated the initiative and we immediately started working on some of the activities.

Hasura E2E tests chronicles, February 2023

Stefano Magni — Wed, 08 Feb 2023 10:49:22 +0000

I joined Hasura in May 2022, and one of my first tasks was to fix the E2E tests of the Hasura Console, main Hasura's front-end application.

The main problems were: they were slow, and they were flaky. Then, by digging into the topic, there was more to say, more to decide, more to fix, and more to do. Let me elaborate a bit more:

The E2E tests were slow: a lot of cy.wait(10000) (yes, ten seconds) everywhere.
The E2E tests were flaky: a couple of months before I joined Hasura, the whole company complained about the CI jobs' flakiness, preventing the teams from merging their PRs. The problem has been workaround'ed by enabling Buildkite (our CI tool) to retry the E2E test job.
The E2E tests were cryptic. Understanding the E2E tests (and fixing/refactoring them) was hard because they were long, terse, and had many abstractions.
Debugging the E2E tests was challenging because of the many different modes the Hasura Console can launch.
The server team used the E2E tests also to test the server.
Cypress crashed because the Console was too resource-demanding.

Let's go through every single topic, one by one.

Photo by Clay Banks on Unsplash

The E2E tests were slow.

In one of my old articles, "Await, do not make your E2E tests sleep ⏳", I shared why having a fixed amount of waiting/sleep is terrible in the tests, following the gist of it.

Fixed waitings are one of the most common causes of slow tests. The tests get slow by always waiting for 10 seconds when an XHR request is happening, even if it usually takes less than 1 second. And for the rare cases where the XHR request takes more than 10 seconds (imagine a cold server start), the tests will fail.

Instead, the test should wait for something that happens deterministically (the XHR request, an element to appear, etc.). Cypress eases avoiding sleep with its retry-ability (see the docs here) and with built-in timeouts, such as:

up to 60 seconds for a page to trigger the load event when visited
up to 4 seconds for an element to appear before interacting with it
up to 5 seconds for an XHR request to start and up to 30 seconds to end
etc.

Another critical problem of fixed waitings: the developer cannot understand if the test's creator initially wanted to wait for an XHR request, for an element to appear, or for an animation, slowing down the debugging and refactoring processes.

Guess what? Replacing the sleep with proper "wait for the XHR request to happen" was almost impossible because of a series of problems:

The Hasura Console comprises legacy and more modern code with no central server-state management. Since different code ages coexist on the same page, other code parts fetch the same data differently. This results in many useless requests and the impossibility of predicting the order of the XHR requests. No order, no party, because Cypress will fail to wait for these crazy requests with many false negatives.
The Hasura Console can use different servers based on how the consumers use it. The main difference is that the Console can run in server mode (talking to the Hasura GraphQL Engine--HGE-- server) or in CLI mode (talking to a locally running CLI server that, in turn, talks with the HGE server). Different servers mean different URLs, different ports, and different APIs, making it almost impossible to intercept the XHR requests.

So? I see two leading possible solutions:

The one to use most of the time: waiting for something that reflects the fact that the XHR request happened (ex., the success notification, something appearing in the UI) with a longer delay (remember, Cypress waits up to 30 seconds by default for an XHR request and there is a reason), and a comment for the readers that the long delay aims to replace the unfeasibility of intercepting XHR requests.
Intercepting two possible requests instead of one. Following is a snippet coming from our codebase

cy.intercept('POST', 'http://localhost:8080/v1/metadata', req => {
  if (JSON.stringify(req.body).includes('create_action')) {
    req.alias = 'createAction';
  }
  req.continue();
});

cy.intercept('POST', 'http://localhost:9693/apis/migrate', req => {
  if (JSON.stringify(req.body).includes('create_action')) {
    req.alias = 'createAction';
  }
});

cy.getBySel('create-action-btn').click();

cy.wait('@createAction').then(interception => {
  checkMetadataPayload(interception, { name: 'Action payload' });

3. The one I do not suggest: waiting for some requests to happen before proceeding with the test. Why do not I recommend it? Because of the edge cases to care about and the resulting complex code.

Following is an example (get ready for something shocking).

/**
 * Wait for a bunch of requests to be settled before proceeding with the test.
 *
 * Alternatively, https://github.com/bahmutov/cypress-network-idle could be used
 *
 * This is a workaround for "element is 'detached' from the DOM" Cypress' error (see the issue
 * linked below). Since the UI gets re-rendered because of the requests, this utility ensures that
 * all the requests parallelly made by the UI are settled before proceeding with the test. Hence, it
 * ensure the UI won't re-render during the next interaction.
 *
 * What are the requests that must be awaited? By looking at the Cypress Test Runner, they are the
 * following, made parallelly or in a rapid series.
 * 1. export_metadata
 * 2. export_metadata
 * 3. export_metadata
 * 4. test_webhook_transform
 * 5. test_webhook_transform
 * 6. test_webhook_transform
 * 7. test_webhook_transform
 * At the moment of writing, I'm not sure the number of requests are fixed or not. If they are fixed,
 * using the cy.intercept `times` options would result in a more expressive and less convoluted code.
 *
 * To give you an overall idea, this is a timeline of the requests
 *
 *         all requests start                             all requests end
 *         |                 |                            |               |
 * |--🚦🔴--1--2--3--4--5--6--7----------------------------1--2--3--4--5--6-7--🚦🟢--|
 *
 *
 * ATTENTION: Despite the defensive approach and the flakiness-removal purpose, this function could
 * introduced even more flakiness because of its empiric approach. In case of failures, it must be
 * carefully evaluated when/if keeping it or thinking about a better approach.
 * In general, this solution does not scale, it should not be spread among the tests.
 *
 * @see https://github.com/cypress-io/cypress/issues/7306
 * @see https://glebbahmutov.com/blog/detached/
 * @see https://github.com/bahmutov/cypress-network-idle
 */

import 'cypress-wait-until';

export function waitForPostCreationRequests() {
  let waitCompleted = false;

  cy.log('*--- All requests must be settled*');

  const pendingRequests = new Map();
  cy.intercept('POST', 'http://localhost:8080/v1/metadata', req =&gt; {
    if (waitCompleted) return;

    Cypress.log({ message: '*--- Request pending*' });

    pendingRequests.set(req, true);

    req.continue(() =&gt; {
      Cypress.log({ message: '*--- Request settled*' });

      pendingRequests.delete(req);
    });
  });

  Cypress.log({ message: '*--- Waiting for the first request to start*' });

  // Check if at least one request has been caught. This check must protect from the following case
  //
  //            check          requests start           test failure, the requests got the UI re-rendered
  //            |              |                        |
  // |--🚦🔴----⚠️---🚦🟢-------1-2-3-4-5-6-7-1----------💥
  //
  // where checking that "there are no pending requests" falls in the false positive case where
  // there are no pending requests because no one started at all.
  //
  // The check runs every millisecond to be 100% sure that no request can escape (ex. because of a
  // super fast server). A false-negative case represented here
  //
  //         requests start requests end   check              check               test failure, no first request caught
  //         |            | |           |  |                  |                   |
  // |--🚦🔴--1-2-3-4-5-6-7-1-2-3-4-5-6-7--⚠️------------------⚠️------------------💥
  cy.waitUntil(() =&gt; pendingRequests.size &gt; 0, {
    timeout: 5000, // 5 seconds is the default Cypress wait for a request to start
    interval: 1,
    errorMsg: 'No first request caught',
  });

  Cypress.log({ message: '*--- Waiting for all the requests to start*' });

  // Let pass some time to collect all the requests. Otherwise, it could detect that the first
  // request complete and go on with the test, even if another one will be performed in a while.
  //
  // This fixed wait protects from the following timeline
  //
  //           1st request start     first request end       other requests start   test failure, the requests got the UI re-rendered
  //           |                     |                       |                      |
  // |--🚦🔴---1---------------------1----🚦🟢----------------2-3-4-5-6-7-1----------💥
  //
  // Obviously, it is an empiric waiting, that also slows down the test.
  cy.wait(500);

  Cypress.log({ message: '*--- Waiting for all the requests to be settled*' });

  cy.waitUntil(() =&gt; pendingRequests.size === 0, {
    timeout: 30000, // 30 seconds is the default Cypress wait for the request to complete
    errorMsg: 'Some requests are not settled yet',
  }).then(() =&gt; {
    waitCompleted = true;
  });
}

I think it's a pity not to intercept the XHR requests because it could act as a life-saver that, when something is not working, allows you to immediately understand if it's the front-end fault (because of a wrong request payload) or a back-end fault (because of an incorrect response payload, status code, etc.). I can live with it, anyway 😊

The E2E tests were flaky

Flakiness is always a red alert. You should monitor your tests and always fix the E2E tests. If you can't fix them now, skip them, and explain why. Flaky tests undermine the working flow, undermine the confidence and trust in the E2E tests, make everyone hate them, and add more friction than the one they are supposed to remove. No tests are way better than flaky tests.

On the Hasura Console, not only has the flakiness not been fixed but the significant, involuntarily error that has been made is enabling Buildkite retry mechanisms in case of E2E test failures... without checking what was happening under the hood. Why? Because Buildkite retries the failing E2E tests without passing a different id to the Cypress CLI. As a result, Cypress does not run any test in the retry because it knows that a test run with the same id already happened! No test run means no failing tests, which means CI goes green.

Buildkite showing a failure, retrying the tests, then going green because Cypress does nothing.

Can you spot the terrible problem? CI is going green for failing tests, with the developers thinking that the tests were succeeding. The result is not only false confidence but also the E2E tests were less and less aligned with the application (highlighting another problem, no one was running the tests locally while working...).

Another thing to consider: running all the E2E tests, then re-starting Cypress to re-run them means that every PR run the complete test suite, paying for AWS running them and for the Cypress Cloud dashboard, paying the time cost of waiting for the completion, with no value at all!

The Buildkite jobs showing the E2E tests require almost 10 minutes.

The E2E tests were cryptic

Test code must be 100x simpler than application code. It must be a clear goal when writing tests, not a by-product of writing a program (the test) that's 100x simpler by definition.

Anyway, this topic is long and deserves a dedicated article. You can find it here: Improving UI tests' code with debugging in mind.

Debugging the E2E tests was challenging

This is the result of multiple factors:

The E2E tests are hard to read (look at the previous chapter, "The E2E tests were cryptic").
The Console runs in different modes (see the "The E2E tests were slow." chapter), which is not easy to detect at first glance.
The tests are coupled, and test B needs to run after test A. This is another exciting and lengthy debate. I split it into a dedicated article, too. See "One long E2E test or small, independent ones?".

But please keep in mind that debugging E2E tests is always hard. The only game changer I know (and suggest) is Replay, which now allows recording browser tests.

The server team used the E2E tests also to test the server

On the server, the situation was even worse. There were no integration tests that allowed us to be sure the server always respected the contract. As a result, the Console's E2E tests were also used to check that "if the Console works, the server did not break any API".

Apart from the false confidence our Buildkite misconfiguration gives, this prevents the Console tests from scaling. It's good to have some E2E tests, but since they are super slow and sometimes flaky, we need to keep them at a minimal number. Most of the testing focus should be on the front-end and back-end sides, but independently from each other.

You can find all the details and the rationales in the dedicated "Decouple the back-end and front-end test through Contract Testing" article, which is the proposal I made internally to split the Console and server tests.

Summary and the future

So, in 2022:

We started refactoring some tests (but never completed them).
We started the Contract Testing proposal (but have yet to push it seriously).
We tracked and skipped all the flaky tests, willing to fix the CI misconfiguration (we never fixed it because of other CI problems).
We enabled Slack alerts from Cypress to quickly identify other flaky tests, skip them or fix them.
We started using proper Contract Testing methodology on a small portion of features.

Then, an internal Working Group was created to fix our numerous CI problems, and the E2E tests were one of the most problematic ones. And since they are slow, they are flaky, they add no value, they are outdated...

... We opted for the hardest but best decision: to eradicate the E2E tests!

The description of the big PR removing the E2E tests.

Let's start with a clean slate and think about the future! Some points that will positively impact the E2E tests:

We will consider Playwright instead of Cypress due to the speed and fewer resources required. The first round of using it went well. Now we need to check if it fulfills all our needs.
Refactors are happening across some features, pushing more tests to the interaction level (Storybook) level instead of the full-app-in-browser level.
The recent migration to Nx reduced the size of the Console by 70%, which means a faster startup and faster browser tests.
We have too many types/modes the Console can run, some of them will be removed from a Product perspective.
We (the frontenders of the Platform Team) will dedicate time helping and mentoring the frontenders of the Feature teams to write more scalable tests.

But there is one real game changer: the server will auto-generate TypeScript types for all the server objects and APIs! That means that on the Console, we will always be using the latest and correct types, guaranteed by the server, and we can start trusting whatever comes from the server!

Ensuring the server respects its part of the contract means we will not need so many E2E tests, but server-free tests against the server types will be our safety net! This changes everything in terms of the need for a lot of E2E tests!!!

Stay tuned. Maybe next year, we will share how we are doing in terms of front-end testing ❤️

Improving UI tests' code with debugging in mind

Stefano Magni — Tue, 07 Feb 2023 08:47:47 +0000

UI tests are made up of a lot of steps and should accomplish three main things, but two of them are somewhat underestimated:

To test a feature (the obvious one)
To help the reader to understand what the code does (usually underestimate)
To ease debuggability (underestimate, and also requires experience)

I go through some simple yet effective tips to keep in mind when writing UI tests.

Photo by Brett Jordan on Unsplash

Readability

Code's readability is something I care about a lot (at the end of the article, you can find other pieces of mine related to the same topic).

One arguable topic (Page-Object Model's fans will get hurt) is my view about abstraction in tests.

Let's look at a real-life example of a test I had to fix.

// spec.ts file
it('Create Query Action', createQueryAction);

// test.ts file (simplified version)
export const createMutationAction = () => {
  // ...
  clearActionDef();
  typeIntoActionDef(statements.createMutationActionText);
  clearActionTypes();
  // ...
};

// test.ts file contains the clearActionDef, typeIntoActionDef, etc.
const clearActionDef = () => {
  cy.get('textarea').first().type('{selectall}', { force: true });
  cy.get('textarea').first().trigger('keydown', {
    keyCode: 46,
    which: 46,
    force: true,
  });
};

// test.ts file contains also the statements
const statements = {
  createMutationActionText: `type Mutation {
    login (username: String!, password: String!): LoginResponse
  }`,
  createMutationCustomType: `type LoginResponse {
    accessToken: String!
  }
  `,
  createMutationHandler: 'https://hasura-actions-demo.glitch.me/login',
  // ...
}

The rationale behind the short functions is creating small, reusable pieces of code to help other tests that must do similar things on the page.

I think it is not good: because it is hard to build a mental model of what the test does! All the test parts are split into small functions and utilities, while the test's code must be as straightforward as possible.

Can you recall the two underrated points at the top of the article? The idea is that the test should accomplish three main things:

To help the reader to understand what code does
To allow debugging itself with ease

The former requires the test's code to be as dumb as possible, and there is no advantage in having abstraction in the test's code because it leads to spending more time debugging and maintaining the tests instead of the application.

The latter relates to the wrong part of tests: debugging/fixing them. Debugging a UI test is hard because you work with

You front-end application
The browser
A tool that controls the browser
The instructions you provide to the tool that controls the browser

Each of the above elements can fail, and even the more experienced developers can struggle with understanding the source of a failing test.

Hence, E2E testing is complicated. Cypress improves the developers' life (I discussed it in my Some UI testing problems and the Cypress way article), but a straightforward code dramatically helps.

No abstraction at all

I recommend having no abstraction at all (later, I will talk about the exceptions and which abstraction it's good)! I rewrote the above example to something like

it('Test the feature', () => {
  cy.get('textarea').eq(0).as('actionDefinitionTextarea');
  cy.get('textarea').eq(1).as('typeConfigurationTextarea');

  cy.get('@actionDefinitionTextarea').clearConsoleTextarea().type(
    `type Mutation {
        login (username: String!, password: String!): LoginResponse
      }`,
    { force: true, delay: 0 }
  );

  cy.get('@typeConfigurationTextarea').clearConsoleTextarea().type(
    `type LoginResponse {
      accessToken: String!
    }
    `,
    { force: true, delay: 0 }
  );

  // ...
})

The rewritten test does the same as the original one, but when you jump on the test's code, the overhead of jumping back and forth to connect the dots mentally is not needed.

Do you want to know that is typed in the textarea? No hassle, it's there!
Do you want to know what is the textareas used by the text? No hassle, it's there!

When is abstraction good in tests?

In my opinion:

When I want to hide some test oddities that could distract the readers for no value
When they are soft, almost parameters-free, only one-level deep

An example of a test oddity is the following

/**
 * Clear a Console's textarea.
 * Work around cy.clear sometimes not working in the Console's textareas.
 */
Cypress.Commands.add('clearConsoleTextarea', { prevSubject: 'element' }, el => {
  cy.wrap(el).type('{selectall}', { force: true }).trigger('keydown', {
    keyCode: 46,
    which: 46,
    force: true,
  });
});

I created the central cy.clearConsoleTextarea because

It's a workaround 😊
For a newcomer, it is odds reading trigger('keydown') instead of using the more idiomatic cy.clear, and I do not want to leave a comment explaining it everywhere.
The command is made up of 5 lines of code that would get the test's code too long for no reason.

An example of a soft abstraction is the following

function expectSuccessNotification = (title: string) {
  cy.get('.notification-success')
    .should('be.visible')
    .should('contain', title)
}

I like it because

It does not use other abstracted code: if my tests fails at expectSuccessNotification('Table created!') I do not have to go crazy down the rabbit-hole to understand what happens behind expectSuccessNotification.
It accepts only one variable, not a lot of options; neither includes conditions that would get hard in the way of understanding what the code finally does.
It's vertical for a specific use case. It does not try to cover all the notification types, contents, etc., at once. Other vertical functions will do.
If you refactor the notification system, you have a central point to refactor to adapt the tests to the new notification system.

At the opposite, this is what I do not want to have (while speaking about notification utilities)

export const expectNotification = (
  {
    type,
    title,
    message,
  }: {
    type: 'success' | 'error';
    title: string;
    message?: string;
  },

  timeout = 10000
) => {
  const el = cy.get(
    type === 'success' ? '.notification-success' : '.notification-error',
    { timeout }
  );

  el.should('be.visible');
  el.should('contain', title);

  if (message) el.should('contain', message);
};

I'm not too fond of the above example because

It tries to cover too many use cases at a time.
If it fails, you have to deal with conditions that make the whole experience a nightmare.

You can find more best practices we follow internally in the Hasura Console UI coding patterns: Testing article.

Matching the test's code and test runner's commands

The Cypress Test Runner helps understand what is happening in the application and which commands are executed, but when you debug a test, it is hard to have an immediate correlation between the Test Runner and the code. More, the logs do not help in understanding what the test is doing in feature terms (ex., the logs say "types in the textarea" but do not say "Type in the Type Configuration textarea"). So, detecting the root of a failure is hard. Cypress records videos for the failing tests, but it is useless if the reader/debugger does not have an immediate correlation between the logs and what the test is doing in plain English.

Look at something like this

I add a log reporting what the test is doing, allowing a direct correlation between the test's code and the Test Runner (cy.log('**--- Type in the Webhook Handler field**');).

Please note that you can pass more arguments to 'cy.log', and they are logged right in the devtools' console when you click on the logged command.

Storybook and Playwright already have the concept of step utilities that allow explaining in English what is happening in the test. Cypress does not have the same option, so the cy.log I propose is valuable, in my opinion.

It's worth mentioning that even if Cypress does not have step, Filip Hric's cypress-plugin-steps is a valid alternative.

Use clear selectors

Look at this code

cy.get('textarea')
  .eq(0)
  .type(`{enter}{uparrow}${statements.createMutationGQLQuery}`, {
    force: true,
  });

What is cy.get('textarea').eq(0)? In the absence of better selectors, I suggest hiding them under Cypress aliases, such as

// Assign an alias to the most unclear selectors for future references
cy.get('textarea').eq(0).as('actionDefinitionTextarea');
cy.get('textarea').eq(1).as('typeConfigurationTextarea');

and then referring to them this way

cy.get('@actionDefinitionTextarea').clearConsoleTextarea().type(/* ... */);

to improve the readers' life.

Reducing data-testid attributes

I do not want to speak about the value for the tests themselves and for their confidence, but only about the effect of data-testid attributes over the debugging phase.

If an element with a data-tesid cannot be retrieved from the page, the possible problems are

The element is not there.
The element is there, but it does not have the attribute.
The element is there, and it has the attribute but not the expected value.

All the above problems cause the developer to re-launch the tests, inspect the elements, look for the test-related attributes, etc. Instead, if the tests are based on the textual contents, a screenshot is enough to understand if the text searched by the test is not there or if it is wrong.

Also, some more cons for the engineers that have to deal with data-testid's

Test-related attributes must be maintained during refactors, but it is not easy when you have hundreds of them.
Test-related attributes are helpful if they are unique on the page. Another thing that is not easy to guarantee when you have hundreds of them

My suggestion is to use data-testid attributes only for:

Sections, not elements (ex. the Header, the Footer, , etc.) that allows reducing the scope of text-based searches. Here is an example

cy.get('[data-test="Actions list"]').within(() => { // <-- reduce the scope
  cy.contains('login') // <-- the "login" text could exist more times in the page
})

Non-text-based elements: icons, images, etc.

Last but not least: I suggest valuing them with user-like values, not programmer-like ones (ex. "Actions List", not "actionsList"), especially when the section shows that exact text. This allows a direct connection between the code of the test, the Cypress' Test Runner, and the page's text content.

Grouping related actions

Reading a flat list of interactions generally does not help comprehend the page's structure the test is running against.

For instance

get 1 and click
get 2 and click
get 3 and click
get 4 and click
get 5 and click
get 6 and click
get 7 and click
get 8 and click

Instead, un-flattening the list could help the readers to create a mental model of where the involved parts reside

within block 1
- get 1 and click
- get 2 and click
- get 3 and click
within block 2
- get 4 and click
- get 5 and click
- get 6 and click
get 7 and click
get 8 and click

Again: Storybook and Playwright already have the concept of step utilities that allow grouping actions, the above suggestion is handy with Cypress.

I touch code's readability in the following articles

and also I touch the topic of getting your code clear in the next ones

Last but not least: I loved and I suggest "The Art of Readable Code" book.

One long E2E test or small, independent ones?

Stefano Magni — Mon, 06 Feb 2023 08:02:44 +0000

While speaking about testing a CRUD app, how should we organize the "create", "modify", and "delete" E2E tests?

The complete list of options are:

To have three small E2E tests dependent on the execution order (test B takes for granted that test A run) - The only bad solution, I'm going to explain why.
To have three small E2E tests independent from the execution order (test B works regardless from whether test A was launched or not) - Theoretically, the best solution. Still, it requires a lot of boilerplates also to be fast.
To have one extended E2E test that does everything - A good tradeoff for the case presented in this article.

Photo by Simon Kadula on Unsplash

It depends, and most of the problems I present are related to the implicit issues of the E2E tests, a strong signal that we should write only a few of them. As a Front-end Engineer, I strongly prefer to invest my time in server-free tests, not E2E ones. Go ahead, and you will understand why.

Please note: this results from working on Hasura's Console E2E tests.

1 - To have three small E2E tests dependent on the execution order (test B takes for granted that test A run)

The test flow would be something like this:

START (the application state is empty)
Test 1: create the entity
Test 2: modify the entity
Test 3: delete the entity
END (the application state is empty)

In this case, the tests are not independent but based on execution order. To test a CRUD flow, the three primary tests are "create an entity", "modify an entity", "delete the entity". The second test ("modify the entity") takes for granted that when it starts, the application state is okay because it runs after the "create the entity" one. "delete the entity" must run after the "modify the entity" too, etc.

Coupling multiple tests together is an anti-pattern because of:

False negatives: The tests will fail in a row once one fails.
Hard to debug: understanding the root of a failure is more complicated because of higher ambiguity. Did the test fail because of its code? Or because the state of the previous test changed? Then, you have to debug two tests when one fails.
Hard to debug (again): the developers waste a lot of time because they can not run a single test nor use skip and only to launch a portion of them.
Hard to refactor: The tests cannot be moved elsewhere. If the code of the tests becomes too long, too complex, etc., you cannot move it to a dedicated file/directory because it depends on the previous one.
Hard to read: The readers cannot know what a test does because they must also know the previous tests. You have to read two tests instead of one, which is not good.

I do not recommend writing tests coupled this way, but I want to include them to be sure you realize why.

2 - To have three small E2E tests independent from the execution order

To get every test independent, every test should create the application state that it needs to run, then clear it after completion. The flow presented in the previous chapter should become something like (in italic the new steps compared to the original create->modify->delete three tests in a row)

START (the application state is empty)
Test 1: create the entity
1. BEFORE: Load the page (the application state is empty)
2. create the entity
3. AFTER: Delete the *entity (the application state is empty)*
Test 2: modify the entity
1. BEFORE: create the *entity (through APIs)*
2. BEFORE: Load the page (the application state is empty)
3. modify the entity
4. AFTER: Delete the *entity (through APIs, the application state is empty)*
Test 3: delete the entity
1. BEFORE: create the *entity (through APIs)*
2. BEFORE: Load the page (the application state is empty)
3. delete the entity
4. AFTER: Delete the action (the application state is empty)
END (the application state is empty)

By doing so, every test is independent. Please note that the before and after actions are done directly by calling the server APIs. Doing them through the UI would be too slow.

Anyway, the presented approach's problem is that tests become slower because every test creates the entity, and every test visits the page. When the application takes 10 seconds to load (it was initially the case of Hasura's Console), reloading the app is a problem.

To get the best of both worlds (independent but fast tests), we should evolve the above flow to

Exploit the previous test's application state.
But also create the needed application state if no tests have run.

Something like (in italic the new steps compared to the flow presented in the previous chapter)

START (the application state is empty)
Test 1: create the entity
1. BEFORE: Does the *entity exist?*
  1. NO: it's ok!
  2. YES: delete the entity (through APIs)
2. BEFORE: Load the page (the application state is empty)
3. create the entity
Test 2: modify the entity
1. BEFORE: Does the *entity exist?*
  1. YES: it's ok!
  2. NO: create the entity (through APIs)
2. BEFORE: Does the entity already includes the change the test is going to make?
  1. YES: it's ok!
  2. NO: modify the entity (through APIs)
3. BEFORE: Are we already on the correct page?
  1. YES: it's ok!
  2. NO: load the page
4. modify the entity
Test 3: delete the entity
1. BEFORE: Does the entity exist?
  1. YES: it's ok!
  2. NO: create the entity (through APIs)
2. BEFORE: Are we already on the correct page?
  1. YES: it's ok!
  2. NO: load the page
delete the entity
END

Now, if you run all the tests in a row, each of them leverages the existing application state. If you run just the "modify the entity" for instance, it creates whatever it needs, then runs the test itself.

Now we have both test independence and test performance! Cool!

Well... Did you notice the amount of code we need to write? The cypress-data-session plugin comes in handy but

You have a lot of cypress-data-session related boilerplate
You have to maintain, in the E2E tests, a lot of API calls that could go out of sync with the ones used in the main application in a while

Here is an example of the cypress-data-session related boilerplate

import { readMetadata } from '../services/readMetadata';
import { deleteHakunaMatataPermission } from '../services/deleteHakunaMatataPermission';

/**
 * Ensure the Action does not have the Permission.
 *
 * ATTENTION: if you get the "setup function changed for session..." error, simply close the
 * Cypress-controlled browser and re-launch the test file.
 */
export function hakunaMatataPermissionMustNotExist(
  settingUpApplicationState = true
) {
  cy.dataSession({
    name: 'hakunaMatataPermissionMustNotExist',

    // Without it, cy.dataSession run the setup function also the very first time, trying to
    // delete a Permission that does not exist
    init: () => true,

    // Check if the Permission exists
    validate: () => {
      Cypress.log({ message: '**--- Action check: start**' });

      return readMetadata().then(response => {
        const loginAction = response.body.actions?.find(
          // TODO: properly type it
          action => action.name === 'login'
        );

        if (!loginAction || !loginAction.permissions) return true;

        const permission = loginAction.permissions.find(
          permission => permission.role === 'hakuna_matata'
        );

        // Returns true if the permission does not exist
        return !permission;
      });
    },

    preSetup: () =>
      Cypress.log({ message: '**--- The permission must be deleted**' }),

    // Delete the Permission
    setup: () => {
      deleteHakunaMatataPermission();

      if (settingUpApplicationState) {
        // Ensure the UI read the latest data if it were previously loaded
        cy.reload();
      }
    },
  });
}

and here is an example of the API call to create the entity

/**
 * Create the Action straight on the server.
 */
export function createLoginAction() {
  Cypress.log({ message: '**--- Action creation: start**' });

  cy.request('POST', 'http://localhost:8080/v1/metadata', {
    type: 'bulk',
    source: 'default',
    // resource_version: 138,
    args: [
      {
        type: 'set_custom_types',
        args: {
          scalars: [],
          input_objects: [
            {
              name: 'SampleInput',
              fields: [
                { name: 'username', type: 'String!' },
                { name: 'password', type: 'String!' },
              ],
            },
          ],
          objects: [
            {
              name: 'SampleOutput',
              fields: [{ name: 'accessToken', type: 'String!' }],
            },
            {
              name: 'LoginResponse',
              description: null,
              fields: [
                {
                  name: 'accessToken',
                  type: 'String!',
                  description: null,
                },
              ],
            },
            {
              name: 'AddResult',
              fields: [{ name: 'sum', type: 'Int' }],
            },
          ],
          enums: [],
        },
      },
      {
        type: 'create_action',
        args: {
          name: 'login',
          definition: {
            arguments: [
              {
                name: 'username',
                type: 'String!',
                description: null,
              },
              {
                name: 'password',
                type: 'String!',
                description: null,
              },
            ],
            kind: 'synchronous',
            output_type: 'LoginResponse',
            handler: 'https://hasura-actions-demo.glitch.me/login',
            type: 'mutation',
            headers: [],
            timeout: 25,
            request_transform: null,
          },
          comment: null,
        },
      },
    ],
  }).then(() => Cypress.log({ message: '**--- Action creation: end**' }));
}

So, having independent tests is essential, but it comes with a cost.

That's why, for this specific problem, I then opted for the last option...

3 - To have one extended E2E test that does everything

Pros: a lot of boilerplate files can be removed.

Cons: Working with the tests becomes slower (you cannot launch only the third test anymore)

Compared to the boilerplate we need to write and the code we need to maintain, it is worth unifying them. After all, the specific CRUD flow I was working on took ~20 seconds.

START (the application state is empty)
Test: CRUD
1. BEFORE: Delete the entity if it exists (the application state is empty)
2. BEFORE: Load the page
3. create the entity
4. modify the entity
5. delete the entity
6. AFTER: Delete the entity if it exists (the application state is empty)
END (the application state is empty)

And at the same time, it makes cypress-data-session useless. Hence one less dependency to keep updated.

Conclusions

Working with E2E tests is hard. Dealing with real data, real application state to clear, etc., has a cost. I know that E2E tests are the only ones that give complete confidence, but as a Front-end Engineer (remember, I'm not a QA Engineer), I strongly prefer to work with server-free tests.

If you found this article interesting, the following articles of mine could help you too

Improving Hasura's Internal PR Review process

Stefano Magni — Tue, 06 Dec 2022 09:52:56 +0000

During 2022, thanks to Haystack, we realized our Pull Requests stayed open, on average, for seven days.

As you can imagine, this is troublesome because:

It slows down the Software Development Life Cycle, preventing us from merging PRs and releasing new features for our users.
It requires a long back and forth between the PR author and the reviewers, sometimes resulting in the need to reload the context in people's minds again and again.
The PR’s code becomes outdated, often requiring repeated merges/rebases and resolving conflicts.

We all see the problem, but what causes it? Inspired by GitLab's model, we created a dedicated internal Working Group (inspired by GitLab's Working Groups) to dig into the human aspects of this problem.

How the Working Group proceeded

Despite this being only the second internal Working Group, we already have a standard way to proceed:

State the problem and explain why you think it's important
Look for a sponsor among all the engineering managers
Define the exit criteria
Create a dedicated Slack channel
Get people involved

When speaking about the specific "PR Review Working Group", we decided to:

Schedule some early meetings: people interact the most during sync meetings compared to async documents
Elaborate on the problem and the possible solutions
Request feedback
Interview some internal engineers: we opted for interviews instead of sending out a survey
Find some best practices
Present the best practices internally

So, Rob Dominguez, Daniel Chambers, and I (Stefano Magni) started working on the Exit Criteria, drafted the first notes and the first opinionated solutions, asked for feedback, etc.

The exit criteria

Defining the exit criteria of the Working Group is crucial to work toward a goal. We determined the following indicators:

Identifying some best practices to follow to get PRs merged as soon as possible.
Identifying the best practices in newly-opened PRs.
Significantly shortening the average lifecycle of a PR.
Improving the relationship between author and reviewer on both ends as measured by a monthly survey.

Over the next several months, we'll monitor the progress and adoption of points 2 through 4. For the remainder of this article, we'll focus on how we identified these best practices.

The root causes the Working Group found out

We all know that a slow PR review process is terrible, but what are the root causes for this slowness? What happens between a developer creating the PR and getting the PR merged? Well, let’s list them in order of when they occur during the end-to-end PR creation and review process:

Missing or not enough detailed PR description: Reviewing a PR is mostly about understanding the author’s context and validating their changes against this context. When a description is missing or too short, it deprives the reviewers of context, making it more difficult to understand and adding a lot of drag to the whole process.
Big PRs: PR size directly impacts the review time. Working for a full sprint and then opening a single PR guarantees the PR will not reviewed promptly.
Code Owners: GitHub Code Owners help to define the specific people or groups that own part of the codebase. Until recently, PR reviews from code owners were mandatory, which prevents merging a PR quickly.
Lack of ownership: When GitHub assigns reviews to a code owners group instead of an individual, nobody feels the review is their responsibility because they believe someone else will pick it up. An author won't waste time waiting for someone to review; they'll naturally move on to their next task, which, as we'll see later, causes them to lose their context on their work.
Notifications missing: The modern developer has so many services competing for their attention, and, in the deluge of notifications, it can be difficult to parse what is essential and what is simply noise. We determined that important messages notifying us of pending reviews or feedback on PRs went unnoticed too often.
PR review time not explicitly allocated: Reviewing PRs takes time, primarily because of the complexity of Hasura's system. When planning the sprints, estimating features, etc., we must never forget to dedicate part of the day to reviewing each other people's PRs.
Timezone overlaps: As a fully-distributed, remote-only company, Hasura's engineers work worldwide. Short overlap adds some delays when you need to interact with people on the opposite side of the world.
CI issues: CI is a gift and a curse, and it can make our review process more streamlined and give reviewers more confidence when used correctly. However, automated steps exceeded the scope of our WG; another group recently began work on this task in conjunction with our Infrastructure team to improve the developer experience.

The best practices

One by one, let's check the best practices we identified for every mentioned problem.

PR description missing or insufficiently detailed

PR authors must try to walk in the reviewer's shoes. And the reviewers need to know a lot of things like

What has changed?
Why did you choose this solution? More importantly, why did you not select other solutions?
What is the dependency graph of the changes? Where should I start reviewing the changes?
How should I test/verify?

In one word: context! The PR author must describe the PR context in detail, adding images, videos, Code Tours, diagrams, pseudo-code; whatever helps reviewers understand the PR changes.

It's important to understand that this activity could take one hour or more, but every minute dedicated to describing the PR saves every reviewer two minutes. Also, the description acts as documentation for posterity and could help the PR authors when they have to return to the PR after some time.

Thanks to a detailed description, reviewers can dedicate their time by checking that the developers’ code reflects their intentions instead of guessing what was in the developers’ minds when they wrote the code.

Tests are also an essential part of the PR; they should be present and reflect the changes introduced by the PR. Tests serve as a quick summary answering the question, “What should the code do?”

Big PRs

Authors need to create reasonably-sized PRs. All projects - or even subsets of codebases - are different, but the size of a PR exponentially impacts the time required to review them. The best way to create easily-parsed PRs is... writing a detailed description! Why? Because if you are describing your PR in detail, and you realize that you are spending hours describing it, chances are the PR is too big.

Code Owners

Before August 2022, we organized Hasura into stack-oriented teams (the Server team, the Console team, etc.), and code-owners groups reflected this internal structure. Since this created too many cross-team dependencies (resulting in slowing down E2E features implementation), we reorganized into feature-focused teams that include all the necessary people to implement and release features independently.

As a result, if GitHub automatically assigns your PR to the Console code-owners group, there's a high probability that you have at least a couple of Console developers on your team! And since they participate in your standup and align with the team goals, they are the best candidates to review your Console-related PR.

The suggestion is always to use Code Owners as a hint to understand what knowledge people should have to review your PR, then find the people with that knowledge in your team and assign the PR to them.

In the future, code owners at Hasura will reflect the new teams’ structure, but this is only partially possible at the time of writing.

Lack of ownership

Lack of ownership happens in two different moments:

When no one reviews a PR
When the PR authors forget about their PR

We can automatically resolve the former case with what we shared in the previous point via assigning a specific person to review a PR. In contrast, the latter happens when the PR authors become tired of waiting for a review and start working on new tasks.

Whenever possible - aka, when getting the PR merged is a matter of hours - the PR authors should not start working on new tasks, but they should opt for getting the existing PR merged. Working on new tasks means forgetting and de-prioritizing the current PR, while striving to get it merged to ensure it will bring to completion.

What happens when a review takes too long? Simply put: authors get stressed. However, this response is critical to get them to become self-advocates and draw attention to the issue:

Raise their hands.
Send messages.
Ask for help.
Report the problem to the managers and in the retros.
Escalate if the problem persists.

We know this process could be better, but it also works towards resolving the issue. Working on new tasks, instead, buries the problem under the rug.

Notifications missing

We solved this problem by:

Firstly, enabling GitHub real-time alerts since it's the best way to receive only the important notifications in Slack

Secondly, we enabled Haystack daily reports in Slack, so that teams never forget their waiting, stale or large PRs.

PR review time not explicitly allocated

We suggested that:

Individuals should enable GitHub real-time alerts, since it is the best way to receive only important notifications in Slack.
Teams should prioritize reviewing PRs (who cares if a team creates a lot of PRs without getting them merged?)
Teams should raise the problem of lack of review time during retros
Teams should set correct expectations in terms of workload the team can afford, also considering the code reviews
Team managers always remember that soft code reviews often result in bugs hitting our users in production

Timezone overlaps

We reduced this problem almost to zero naturally by assigning PR reviews to peers from the same team instead of code owners. We built feature teams with timezone compatibility in mind.

What happens if we do not follow the above best practices?

Simply put:

We start working on many features, but we release only a few.
We release features at a slower pace.
We end up with a backlog of stale PRs that haven't seen activity in months and wither away, neglected on GitHub's servers.

The presentation

Instead of opting for the standard slide deck, we made a more engaging presentation focused on a story: a Hasura developer opening a PR and facing all the PR review-related problems. We wanted attendees to garner empathy - a company core value - for devs on either side of this process.

To make the story more engaging, we relied heavily on gifs and exaggerated - only slightly! - some of the impediments we all face in this industry.

We created some vertical slides to have the slides side-by-side with the presenter (using a virtual camera and OBS, you can find all the instructions in this tweet about how to do the same).

Here is a picture of the presentation layout:

What's next?

Once we identified the best practices, we immediately put the "Starting to review PRs" topic in our onboarding docs to ensure new hires understand the importance of these best practices within their first few weeks.

Over the next few months, we'll continue analyzing metrics via Haystack to understand if the average number of days a PR stays open decreases. If not, we will try to understand the developer's difficulties applying the best practices mentioned.

Special thanks

I (Stefano Magni) volunteered to create and drive the Working Group, but I want to thank Rob Dominguez for raising the quality of our work by doing a lot of under-the-hood things. His presence and energy always motivated me too!

Thanks a lot also to Daniel Chambers. Daniel got involved in all the discussions, provided a lot of feedback, and volunteered to interview some of our developers. Thanks, Daniel!

Finally, thanks to all the people we interviewed and those who participated in every async and sync discussion ❤️.

Forem: Stefano Magni

How Preply improved INP on a Next.js application (without React Server Components and App Router)

Introduction

Why care about page speed?

What is INP?

Preply.com’s tech stack

Our initial hypotheses

Reducing React hydration

All the data we leveraged to improve INP

How we improved INP

React 18 upgrade (INP decreased by 140 ms)

Virtualizing a long list (INP decreased by 40 ms)

Debouncing keyboard events (INP decreased by 20 ms)

Memoizing heavy components (INP decreased by 30 ms)

Fixing state management non-EU privacy policy banner (INP decreased by 15 ms)

Avoiding the tooltip to re-render the whole page (INP decreased by 10 ms)

Updating Usercentrics for the EU privacy policy banner (INP decreased by 5 ms for the Search page, 40 ms for the Home page)

Re-optimizing what we already optimized (INP decreased by 15 ms)

Migrating the Home page to the App router (INP decreased by 10 ms on the Home page)

useTransition, await-interaction-response, etc.

Leveraging React’s useDeferredValue (no INP improvements)

Optimizing the Design System components (no INP improvements)

Cleaning up the Search page’s code (no INP improvements)

Quickly optimizing Preply Chat for the logged-in users (no INP improvements)

Removing DataDog RUM (no INP improvements)

Offloading the third-party scripts with Partytown (we gave up)

Jumping on the new React Compiler (we didn’t even try)

Did all this work also improve the UX?

Miscellaneous

Medium and long-term plans

Takeaways

Some more resources

The great people who worked on the page speed project

The Implementation Details of Preply’s Design System Visual Coverage (part II)

Web implementation

Preparatory steps

Using a bitmap to recreate the page’s pixels

Performance optimizations

Sample rate and events cardinality

App (React Native) implementation

What about view hierarchy files?

The Design System visual coverage score and dashboard

How We Test the Visual Coverage Code

How Teams Can Play with the Visual Coverage

Test it on Your Product

Special Thanks

Visual coverage: Why and How Preply Measures the Impact of the Design System (part I)

Why Did We Need a Custom Way to Track the Impact of the Design System?

What Have We Implemented? The Design System Visual Coverage

How Is the Visual Coverage Used?

Product Teams’ Ownership and Coverage Containers

Where to Run the Visual Coverage Count? On Users’ Devices!

Pages’ popularity impact

The Components’ Weights

Other Design System Metrics

Implementation details

Resources

Special Thanks

Production-grade gradient bordered, transparent, and rounded button

The existing solutions

The viable solution

Step-by-step implementation

Browser compatibility

Fallback

React Native

Additional notes

Codepen liknk

Credit where credit is due

Some things I learnt from working on big frontend codebases

My direct experience

Table of contents

Generic approaches

Managing more cases than the needed ones

Leaving dead code around

Internal code dependencies and boundaries

Implicit dependencies

Spreading external dependencies and implementation details

Big modules

Code readability

Uniformity is better than perfection

Type assertions (`as`)

`@ts-ignore` instead of `@ts-expect-error` and broad scope

`any` instead of `unknown`

Components accepting `className`

Implementing the new `<LoadHasuraPlan />` React wrapper