Forem: ‹div›RIOTS

Why devs should build with Design Systems

RBerthier — Wed, 09 Feb 2022 09:53:07 +0000

Design systems are now a hot topic among front-end teams. Design Systems have switched from best-in-class solutions for big enterprises to must-have for any front-end team.

Initially, Design Systems have been popularized by designers (with tools like Figma/Sketch), to help them build reusable components to improve their work efficiency and consistency.

From a developer perspective, Design Systems could somehow be compared to Components Libraries, as they are widely used as reusable components sources to build web products. So why should developers consider building with a Design System on the code side instead?

Design Systems are not just components libraries. They also include design tokens, documentation, and a design kit. Check our article What is a Design System ? if you want to learn more about this.

Let’s discover some key benefits of building a Design System for developers.

🤝 Workspace for collaboration

One of the most important benefits of working with a Design System is to set up an isolated and collaborative environment to ensure the quality and the adoption of all the assets (tokens, components, documentation, designs …) that are built and consumed by the team.

A good Design System platform should provide core features to foster and streamline collaboration within the developer team, but also with the whole front-end team.

For instance, Backlight offers real-time preview and collaboration, branch/pull-request management, asynchronous messaging, visual reviews, …

🧩 Benefits from design tokens

Design tokens are core elements of the design language used both by designers and developers to build components in Design Systems.

We can refer to tokens as "constants" hosting common values in the Design System codebase. An interesting thread about it here.

For instance, here are some common tokens: colors, fonts, spacing, border, radius, opacity, shadows, z-index, …

A Design System is a source of truth for tokens, built-in collaboration with designers and developers. All Design System components built on top of tokens benefit from auto updates when tokens are modified.

So having a Design System, instead of a standalone component library, makes it easier and more efficient to build consistent and maintainable components.

Tips: Design system tokens on the code-side can be synchronized with the Design System on the design-side thanks to some dedicated tools (like Specify) or built-in features in Design System tools (like Backlight).

📦 Benefits from a component library inside a Design System

Component Libraries (like Material UI) have become very popular among developers, as they help speed up the development and bring more reusability/consistency. The challenge of building with component libraries lies in the customization it requires to fit specific design guidelines.

Building a custom component library inside a Design System either from scratch or on top of a popular component library, can have great benefits.

Design Systems are natively built to offer a nested environment to split the work on components from the work on products. Using a Design System can help prevent developers from customizing components directly in production code, which could lead to consistency and maintenance issues in the long run. Developers are working on components solely in the Design System, while consuming components in production code. It makes customization of components easier to manage and ensure better quality over time.

Building and updating the components library in the Design Systems will also benefit from collaboration reviews and testing workflows, before being shipped in production code. It improves the reliability of new releases and automates components maintenance in production. Design Systems are even more powerful when working in cross-products or multi-brands environments.

For instance, Backlight provides a full environment to work solely on Design Systems, with Github/Gitlab repository versioning and NPM package publishing features to release bullet-proof assets ready-to-use in production.

Having the component library inside the Design System also offers the possibility to enhance documentation with live components and playgrounds directly integrated with code snippets from the component library. It makes documentation easier to maintain.

🎨 Benefits from design kits inside developer Design System

A Design System also provides design kits assets (brand guidelines, brand assets, component designs, …). In a Design System on the developer side, a design kit could be a mapping with the Design System on the design side, either through links to related design assets (Figma/Sketch files for instance) or with some synchronization (provided by design-to-code / code-to-design features).

The developer team can benefit from a Design System with such design kits, as it will help tackle the handoff issue with designers. As the Design System provides design for each component it makes it easy for developers to always have the last related design version at hand.

Backlight offers the capability to add design links to each component of the Design System.

📒 Benefits from documentation for developers

Documentation is a core element of a Design System. This is a collaborative workspace where the whole front-end team shares basics, guidelines, and how-to on the design and code.

It helps foster collaboration and information sharing across the team. Documentation is a powerful tool for developers to find information on how to use design tokens and the component library. It’s especially useful when onboarding new developers.

Backlight has built-in capability to build collaborative and live documentation from Design System components and tokens.

⚡️ Boost developers efficiency

As for designers, developers benefit from a great boost of productivity when building products with Design Systems. Reusable components from Design Systems help save valuable time from developers each time they need to use components (not reinventing the wheel, or wasting time finding the right component).

This study shows for instance that it’s 47% faster to build with a Design System instead of building from scratch, with also better consistency in the end.

🚧 Lower code maintenance

Working with a Design System on the developer side also means saving loads of time to ensure product consistency and maintenance. When new requirements are coming from the design team (like changing a primary color or a component design/behavior …), it can be quite tedious and time-consuming to upgrade all assets in production when not using a Design System. On the other hand, a Design System offers a collaborative environment to develop components safely, by being able to build, upgrade and test them in isolation through multiple contexts/conditions, making later product development and maintenance lighter.

In Backlight, just update your token/component, create a pull-request, merge, ship an updated npm package, and voila all your related components in production are updated automatically. No need to waste time maintaining components in production! 🧙‍♂️

🎯 Better consistency

Using components from Design Systems will ensure that production components are consistent, as the components should be loaded and updated directly from the Design System. All similar components are strictly the same and all basic tokens are bound with the Design System.

⭐️ Better quality and accessibility

Quality and accessibility are key aspects, with technical challenges to tackle. Building components in a nested environment, like in a Design System, is a good way to ensure that components are well crafted and tested collaboratively, before being safely shipped in production.

😀 Conclusion

Building with a Design System can have multiple benefits for developers. Eventually, the Design System will become a source of truth that is a source of trust for the whole team. Design Systems ultimately empower developers with collaboration, speed, quality, and consistency.

If you want to start playing with Design Systems, give a try to one of the starter kits provided by Backlight. Many frameworks are supported, coming with loads of tokens and components ready to use. Perfect for a quick jump-in.

A story of how we migrated to pnpm

🎙 m4dz — Wed, 26 Jan 2022 09:13:13 +0000

It all started with me trying to improve our Continuous Integration pipeline. I'm a strong believer in having proper CI - the threshold for how much to invest in unit & integration tests is always tricky to set, but to me the bare minimum should be to have linting and type checking run on every commit.

Now, having that bare minimum is great, but it also needs to be as fast as possible. When you want commits & reviews to be fast, CI cannot be the one thing holding you back.

Yet... This is what we would see in the best case scenario on that bare minimum linting & type checking job:

1 minute and 11 seconds just to install dependencies. Obviously, the job has to do more afterwards, and that's where I'd prefer it to spend time.

But wait, there's more. This was the best case scenario. You may know that package managers have caches, and a known trick to speed installs is to save that cache after CI runs, so it can be reused for subsequent runs. An easy way to do that nowadays is to use actions/node-setup's caching capabilities.

However the cache cannot always be used. As soon as the lock file changes, typically when adding dependencies, the cache isn't reused because the cache's hash is usually computed based on the lock file. We would then get:

6 minutes and 31 seconds 🐌.
That's when we really thought we needed to do something.

Where we stood with Yarn

We have been using Yarn 2 for quite some time, having originally switched to it for its native workspace support which is great for monorepos as we happen to have one. Because we use a lot of different dev tools (in no particular order - Vite, Vitepress, Astro, esbuild, Webpack, Eleventy, Firebase tools, Tailwind...) and many more actual dependencies. It's easy to understand how many dependencies we're bound to have when you see all the frameworks we support, whether on WebComponents.dev or on Backlight.

You may know Yarn 2 for introducing the Plug'n'Play linker. To make it short, it completely forfeits the idea of the node_modules resolution mechanism and tells Node to depend on Yarn for dependency resolution.
It's a really interesting idea but dropping node_modules is a compatibility challenge which kept us away from trying it. We stuck and are sticking to node_modules for now.

Anyway, because Yarn 3 had been released for a few months with performances improvements, we decided to give it try to see if that would speedup our builds.

Trying yarn 3

Upgrading to Yarn 3 is fairly simple:

> yarn set version berry

➤ YN0000: Retrieving https://repo.yarnpkg.com/3.1.1/packages/yarnpkg-cli/bin/yarn.js
➤ YN0000: Saving the new release in .yarn/releases/yarn-3.1.1.cjs
➤ YN0000: Done in 0s 758ms

And there we go, we were upgraded to Yarn 3.

I'll spare you another pair of screenshots, but that got us down a bit, to 4 minutes 50 seconds without cache and 57 seconds with cache.

I'm sparing you the screenshots for a good reason - I did mention we have been using Yarn 2 in that monorepo for a while. We've also been adding so many packages in different workspaces that we ended up with a lot of duplicated dependencies, ie with multiple versions of the same packages.

So just for the sake of the comparison and because our original point was to speedup install times, I went ahead and completely removed the yarn.lock file and tested again.

With cache, down to 50 seconds:

And without cache, we got down to 4 minutes and 1 second:

It's fair to say we've sped up our builds quite a lot already, but we wanted to go further yet.

Now, that's where we decided to try out pnpm. But as pointed out by @larixer, there are also options you can set to speedup Yarn. We tried them out to make it a fairer comparison.

@larixer mentions the 3 following options:

nmMode: hardlinks-global
enableGlobalCache: true
compressionLevel: 0

And they do help a lot, especially without cache where we go down to 1 minute 10 seconds:

It is also slightly faster with a cache, yielding 45 seconds:

So if you are running Yarn, do considering trying them out! Chances are they will greatly improve your install times.

Anyhow, let's jump into pnpm!

Enter pnpm

pnpm stands for Performant NPM. Its adoption has been really steady as its close to the 15k stars at the moment on Github. It also comes with out of the box support for workspaces, making it easier for us to consider.

As its name indicates, it really emphasizes performances, both regarding disk space and installation times. In all the figures provided, whether from pnpm or from Yarn you can see that pnpm really comes out faster most of the time.

There seems to be two main reasons for it.

One, being performance-oriented, its implementation targets speed. You may have seen when installing with yarn or npm timings for each of the resolution/fetch/link steps. It appears that pnpm is not doing those steps sequentially globally, but sequentially for each package in parallel which explains why it's so efficient.

The other reason is the way it deals with the node_modules folder.

Centralized addressable cache

pnpm calls it a content addressable filestore, and we know other package managers like yarn or npm have caches as well, which allow you not to have to re-download.

The difference with pnpm's is that this cache is also referenced by your node_modules files, which are effectively hard-links to that cache. A hard-link means your OS will report those files as being actual files - but they're not. So the actual disk usage occurs in pnpm's cache, not in your node_modules folder. You save space, and installation time, because there's way less IO involved in setting up that infamous node_modules folder! 🪄

Non-flat node_modules

What's also interesting is the way the node_modules is organized with pnpm. npm and yarn (when using the node_modules linker) tend to do hoisting to save space as they're not using links. Hoisting is the act of installing a dependency in a parent directory rather than where it is depended on. So if you have a dependency that can be resolved to the same version pulled by two other packages, they'll try to hoist that dependency to avoid storing that same dependency twice in your node_modules.

The behavior of pnpm is different, somewhat more consistent. It's always setting up the node_modules structure the same way. First, it's non-flat. So running pnpm install vite in an empty folder will result in the following node_modules:

> tree node_modules -L 1
node_modules
└── vite -> .pnpm/vite@2.7.10/node_modules/vite

So our node_modules only contains vite and not all of its dependencies. This may seem unusual, but this avoids phantom dependencies. Phantom dependencies are dependencies that you end up being able to use without explicitly depending on them. This is a rather dangerous practice, because you do not control those - you may update the original dependency, just upgrading it to a new patch, but its dependencies may have been upgraded to major versions breaking your own code!

In our previous example, my source code won't be able to require any other dependency but vite as it's the only one which was effectively installed to the top of my node_modules.

Now we can see that this folder is actually linking to another folder in node_modules/.pnpm: this is pnpm's Virtual Store where you will find all the packages installed in your project.

If we take a peek at this folder:

> tree node_modules/.pnpm/vite@2.7.10 -L 2
node_modules/.pnpm/vite@2.7.10
└── node_modules
    ├── esbuild -> ../../esbuild@0.13.15/node_modules/esbuild
    ├── postcss -> ../../postcss@8.4.5/node_modules/postcss
    ├── resolve -> ../../resolve@1.21.0/node_modules/resolve
    ├── rollup -> ../../rollup@2.63.0/node_modules/rollup
    └── vite
        ├── bin
        ├── CHANGELOG.md
        ├── client.d.ts
        ├── dist
        ├── LICENSE.md
        ├── node_modules
        ├── package.json
        ├── README.md
        ├── src
        └── types

So, vite itself and its dependencies were installed to node_modules/.pnpm/vite@2.7.10/node_modules.
The magic that makes it all work is that Node, when resolving packages, considers the target of the symlink instead of using the symlink's path itself. So when I do require('vite') from an src/index.js file, Node finds the node_modules/vite file by iterating on parent directories looking for a node_modules folder containing vite but actually resolves it to the source of the symlink:

> node -e "console.log(require.resolve('vite'))
/tmp/foobar/node_modules/.pnpm/vite@2.7.10/node_modules/vite/dist/node/index.js

That means that any further package resolutions needed will effectively be done from this folder - so if that /tmp/foobar/node_modules/.pnpm/vite@2.7.10/node_modules/vite/dist/node/index.js file requires esbuild it will find it in node_modules/.pnpm/vite@2.7.10/node_modules/esbuild!

This is also why some dependencies don't play nice with pnpm: because they don't resolve symlink targets. But we'll get to that later.

Note: peer dependencies have a special handling in pnpm, if you're interested about it I suggest this read.

Now that we have a rough understanding of how pnpm works, let's try using it! 🚀

Migrating to pnpm

pnpm import

pnpm comes with a command to import yarn's locked dependencies:

https://pnpm.io/cli/import

There's just one gotcha when you're using it in a monorepo: the workspaces have to be declared in your pnpm-workspace.yaml first. If you don't, then at best pnpm import will only import the dependencies declared in your root file.

Dependencies which have undeclared dependencies

Another kind of problem we ran into is some dependencies having undeclared dependencies. When using yarn it was not a problem because those undeclared dependencies are sometimes very used. For example, after the migration we realized mdjs-core had not declared its dependency on slash.

A simple way to fix this is again through the readPackage hook we mentioned in the previous section. There, you can simply declare the dependency explicitly for mdjs-core:

if (pkg.name === '@mdjs/core') {
  pkg.dependencies = {
    ...pkg.dependencies,
    slash: '^3.0.0',
  };
}

shamefully-hoist when tools don't play along

We talked about the non-flat node-modules earlier. This structure is unfortunately not compatible with every Node tool.

An example of this is Astro which at the moment recommends using shamefully-hoist.
Kind of a funny name, meant to dissuade you from using it :-)

As the name implies, this one will hoist all your dependencies in your root node_modules, fixing any incompatibility you may have with dev tools not playing along with the nested node_modules. This typically happens because they don't resolve symlinks to their target.

At the time of this writing, Astro requiring it, if you're not using it will fail at loading its dependencies, with a

Error: The following dependencies are imported but could not be resolved:

  react (imported by /not-relevant/testimonial-card/src/index.tsx)
  svelte/internal (imported by /not-relevant/double-cta/dist/DoubleCta.svelte.js)

Instead of going this way, I preferred adding manually the missing dependencies to the workspace using Astro. It's a hack, but one I prefer living with than using shamefully-hoist globally as it would cancel out the advantages of the non-flat node-modules.

How fast is it

I know, that was the whole point of us trying out pnpm - let's see how fast it is!

So, when the cache is hit, we get down to 24 seconds:

And when the cache cannot be used, we get down to a whopping 53 seconds:

Summarizing the results:

	Without cache	With cache
yarn 2 (without dedupe)	6min 31s	1min 11s
yarn 3 (without dedupe)	4min 50s	57s
yarn 3	4min 1s	50s
yarn 3 (optimized)	1min 10	45s
pnpm	58s	24s

Honestly, I'm particularly impressed about the results when there's no cache.
I would have expected network to be the bottleneck for both yarn or pnpm in that case, but somehow pnpm still really shines there, while also being faster (at least for us) when the cache is used too!

Now I'm happy - the CI is snappy, at least way snappier than it was, and our local install times also benefitted from it. Thank you pnpm!

Vite in the browser

🎙 m4dz — Wed, 19 Jan 2022 17:39:06 +0000

TL;DR

We made browser-vite - a patched version of Vite running in the browser with Workers.

How it works - in a nutshell

A Service Worker: replaces Vite's HTTP server. Capturing the HTTP calls of an embedded iframe from example.
A Web Worker: Run browser-vite to process off the main thread.
Calls to the file system are replaced by an in-memory file system.
Import of files with special extensions (.ts, .tsx, .scss...) are transformed.

The challenges

No real File System

Vite does a lot with files. The files of the project but also config files, watchers, and globs. These are difficult to implement in the browser with a shimmed in-memory FS. We removed watchers, globs, and config file calls to limit the complexity and the surface API.

The project files stay in the in-memory FS that browser-vite and vite plugins can access normally.

No "node_modules"

Vite relies on the presence of node_modules to resolve dependencies. And it bundles them in a Dependencing Pre-Bundling optimization at startup.

We didn't want to run a node_modules folder in the browser's memory because we think it's just too much data to download and store into the browser's memory. So we carefully stripped out node resolvers and Dependencing Pre-Bundling from Vite.

Users of browser-vite have to create a Vite plugin to resolve bare module imports.

Our products: Backlight.dev, Components.studio and WebComponents.dev, are running a server-side bundler optimizer for the past 2 years now. We created a Vite plugin for browser-vite to resolve node dependencies automatically. As of the date of this post, this server-side bundler is not open-sourced.

Regex "lookbehind"

Some regexs in Vite are using lookbehind. This works great locally when executed by Node.js, but it's not supported in Safari.

So we rewrote the regexs for more browser compatibility.

Hot Module Reload (HMR)

Vite uses WebSockets to communicate code changes from the server (node) to the client (browser).

In browser-vite, the server is the ServiceWorker + Vite worker and the client is the iframe. So we changed the communication from WebSockets to a post message to the iframe.

For this, the client side code of Vite in iframe has been replaced by a special browser version handling messages outside of WebSockets.

How to use it

As of the time of this writing, it's not a plug and play process. There is a lot to figure out by reading Vite's internal processing in order to use browser-vite.

Note: This post may become obsolete over time, so make sure you check
browser-vite's README for always up to date information on browser-vite's usage.

Installation

Install the browser-vite npm package.

$ npm install --save browser-vite

$ npm install --save vite@npm:browser-vite

To channel "vite" imports to "browser-vite".

iframe - window to browser-vite

You need an iframe that will show the pages served internally by browser-vite.

Service Worker - the in-browser web server

The Service Worker will capture certain URLs requests coming from the iframe.

Here is an example using workbox.

workbox.routing.registerRoute(
  /^https?:\/\/HOST/BASE_URL\/(\/.*)$/,
  async ({
    request,
    params,
    url,
  }: import('workbox-routing/types/RouteHandler').RouteHandlerCallbackContext): Promise<Response> => {
    const req = request?.url || url.toString();
    const [pathname] = params as string[];
    // send the request to vite worker
    const response = await postToViteWorker(pathname)
    return response;
  }
);

Mostly posting a message to the "Vite Worker" using postMessage or broadcast-channel.

Vite Worker - processing request

The Vite Worker is a Web Worker that will process requests captured by the Service Worker.

Example of creating a Vite Server:

import {
  transformWithEsbuild,
  ModuleGraph,
  transformRequest,
  createPluginContainer,
  createDevHtmlTransformFn,
  resolveConfig,
  generateCodeFrame,
  ssrTransform,
  ssrLoadModule,
  ViteDevServer,
  PluginOption
} from 'browser-vite';

export async function createServer(
  const config = await resolveConfig(
    {
      plugins: [
        // virtual plugin to provide vite client/env special entries (see below)
        viteClientPlugin,
        // virtual plugin to resolve NPM dependencies, e.g. using unpkg, skypack or another provider (browser-vite only handles project files)
        nodeResolvePlugin,
        // add vite plugins you need here (e.g. vue, react, astro ...)
      ]
      base: BASE_URL, // as hooked in service worker
      // not really used, but needs to be defined to enable dep optimizations
      cacheDir: 'browser',
      root: VFS_ROOT,
      // any other configuration (e.g. resolve alias)
    },
    'serve'
  );
  const plugins = config.plugins;
  const pluginContainer = await createPluginContainer(config);
  const moduleGraph = new ModuleGraph((url) => pluginContainer.resolveId(url));

  const watcher: any = {
    on(what: string, cb: any) {
      return watcher;
    },
    add() {},
  };
  const server: ViteDevServer = {
    config,
    pluginContainer,
    moduleGraph,
    transformWithEsbuild,
    transformRequest(url, options) {
      return transformRequest(url, server, options);
    },
    ssrTransform,
    printUrls() {},
    _globImporters: {},
    ws: {
      send(data) {
        // send HMR data to vite client in iframe however you want (post/broadcast-channel ...)
      },
      async close() {},
      on() {},
      off() {},
    },
    watcher,
    async ssrLoadModule(url) {
      return ssrLoadModule(url, server, loadModule);
    },
    ssrFixStacktrace() {},
    async close() {},
    async restart() {},
    _optimizeDepsMetadata: null,
    _isRunningOptimizer: false,
    _ssrExternals: [],
    _restartPromise: null,
    _forceOptimizeOnRestart: false,
    _pendingRequests: new Map(),
  };

  server.transformIndexHtml = createDevHtmlTransformFn(server);

  // apply server configuration hooks from plugins
  const postHooks: ((() => void) | void)[] = [];
  for (const plugin of plugins) {
    if (plugin.configureServer) {
      postHooks.push(await plugin.configureServer(server));
    }
  }

  // run post config hooks
  // This is applied before the html middleware so that user middleware can
  // serve custom content instead of index.html.
  postHooks.forEach((fn) => fn && fn());

  await pluginContainer.buildStart({});
  await runOptimize(server);

  return server;
}

Pseudo code to process requests via browser-vite

import {
  transformRequest,
  isCSSRequest,
  isDirectCSSRequest,
  injectQuery,
  removeImportQuery,
  unwrapId,
  handleFileAddUnlink,
  handleHMRUpdate,
} from 'vite/dist/browser';

...

async (req) => {
  let { url, accept } = req
  const html = accept?.includes('text/html');
  // strip ?import
  url = removeImportQuery(url);
  // Strip valid id prefix. This is prepended to resolved Ids that are
  // not valid browser import specifiers by the importAnalysis plugin.
  url = unwrapId(url);
  // for CSS, we need to differentiate between normal CSS requests and
  // imports
  if (isCSSRequest(url) && accept?.includes('text/css')) {
    url = injectQuery(url, 'direct');
  }
  let path: string | undefined = url;
  try {
    let code;
    path = url.slice(1);
    if (html) {
      code = await server.transformIndexHtml(`/${path}`, fs.readFileSync(path,'utf8'));
    } else {
      const ret = await transformRequest(url, server, { html });
      code = ret?.code;
    }
    // Return code reponse
  } catch (err: any) {
    // Return error response
  }
}

Check Vite's internal middlewares for more details.

How does it compare to Stackblitz WebContainers

"WebContainers:
Run Node.js natively in your browser"

Stackblitz's WebContainers can also run Vite in the browser. You can elegantly go to vite.new to have a working environment.

We are not experts in WebContainers but, in a nutshell, where browser-vite shims the FS and the HTTPS server at the Vite level, WebContainers shims the FS and a lot of other things at the Node.js level, and Vite runs on it with a few additional changes.

It goes as far as storing a node_modules in the WebContainer, in the browser. But it doesn't run npm or yarn directly because it would take too much space (I guess). They aliased these commands to Turbo - their package manager.

WebContainers can run other frameworks too, like Remix, SvelteKit or Astro.

It's magical ✨ It's mind-blowing 🤯 We have massive respect for what the Stackblitz team has built here.

One downside of WebContainers is that it can only run on Chrome today but will probably run on Firefox soon. browser-vite works on Chrome, Firefox and Safari today.

In a nutshell, WebContainers operates at a lower level of abstraction to run Vite in the browser. browser-vite operates at a higher level of abstraction, very close to Vite itself.

Metaphorically, for the retro-gamers out there, browser-vite is a little bit like UltraHLE 🕹️😊

(*) gametechwiki.com: High/Low level emulation

What's next?

browser-vite is at the heart of our solutions. We are progressively rolling it out to all our products:

Backlight.dev
Components.studio
WebComponents.dev
Replic.dev (New app coming up very soon!)

Going forward, we will continue to invest in browser-vite and report back upstream. Last month, we also announced that we sponsored Vite via Evan You and Patak to support this wonderful project.

Want to know more?

GitHub Repository: browser-vite
Join our Discord server, we have a #browser-vite channel going on 🤗

How ESLint Can Enforce Your Design System Best Practices

🎙 m4dz — Wed, 19 Jan 2022 17:34:31 +0000

If you're creating a design system component library for your company or the open-source community, there's a good chance that you have strong opinions on how end-users should consume your design system.

To ensure that your design system is used in its intended way, and to reduce the number of possible bugs, you might want your users to adhere to your best practices. The following are two examples of possible best practices:

Avoiding inline styles in your elements
Ensuring that tooltips don't contain interactive content.

If you're the only person designing, developing, and consuming your design system, then you can sleep comfortably knowing that your design system is being used exactly as intended.

Chances are you're not the only person developing the design system and you'll certainly not be present when someone consumes it. How can you feel sure that everyone abides by your design system's best practices? You could cross your fingers and trust that your end-users read the documentation, heed your warnings, and never fail to abide by your rules.

Unfortunately, this is often not the case and it's very easy to miss warnings or misunderstand how to properly use a tool. I've been there!

Fortunately, a great way to encourage your consumers to follow your best practices is through the use of ESLint, a static analysis tool to find problems in your code.

By default, ESLint ships with a handful of general best practices, called rules and will display red squigglys in your IDE if the rules have been violated. Some of these rules include:

No duplicate keys in objects
No unreachable code
No unused variables

However, the rules you enable in your project don't need to come directly from ESLint. Popular libraries like Cypress, Lodash, and React have ESLint configurations that anyone can use in their own projects to ensure users adhere to best practices. If you're an intrepid explorer of the JavaScript language you can go a step further and create custom rules specific to your design system which you can export for other people to use in their projects. That's exactly what we'll do in these articles.

In this article, we'll spend a bit of time understanding how tools like ESLint parse JavaScript down into a data structure called an abstract syntax tree (AST). We'll then touch on how ESLint rules work and how to parse our Lit templates into HTML. Finally we'll start creating our rules. We'll even use ESLint's built-in testing tool to make sure our rules work under a variety of conditions.

The pre-requisite for this article is some JavaScript + HTML knowledge. A little experience using ESLint and Lit may come in handy but isn't necessary.

What’s an Abstract Syntax Tree?

For those, like me, who haven't gotten their hands dirty with compilers before, conceptualising how the human-readable language we write in our IDE gets understood (and transformed) by tools like Webpack, Prettier, and Babel can feel like magic.

Under the hood, when a tool like ESLint wants to start performing actions against your JavaScript it parses your code. Parsing is the process of taking the JavaScript you've written and turning it into a tree representation of the code, an abstract syntax tree (AST).

This process of parsing is split into two parts, tokenization and tree construction.

Tokenization takes the code and splits it into things called tokens which describe isolated parts of the syntax.

Tokens for a JavaScript program like:

const helloWorld = 'hello world';

will look something like this:

[
  { type: 'IdentifierName', value: 'const' },
  { type: 'WhiteSpace', value: ' ' },
  { type: 'IdentifierName', value: 'helloWorld' },
  { type: 'WhiteSpace', value: ' ' },
  { type: 'Punctuator', value: '=' },
  { type: 'WhiteSpace', value: ' ' },
  { type: 'StringLiteral', value: "'hello world'", closed: true },
];

JS Tokens

I used js-tokens as a quick way of tokenizing my JS for this example but we won't be dealing with tokenization directly ourselves in this article.

The second step in the parsing process is tree construction, which reformats the tokens into an AST. The AST describes each part of the syntax and its relationship to the others.

We can visualise this relationship by parsing the following JavaScript statement:

const component = html`<h1>Creating custom ESLint rules</h1>`;

It would get transformed into an AST, with the following structure:

Tools like Babel and Prettier turn your written JavaScript into an AST to analyse and transform the code we've written. Babel uses the AST to transpile our code into a browser-friendly version of JavaScript, while Prettier uses the AST to reformat your code.

Getting Curious with the AST Explorer

To really explore what an AST looks like, have a play with the AST explorer. Get familiar with the AST explorer as we're gonna be using it loads later in the article.

Write a simple statement, like the following:

const helloWorld = 'hello world';

You'll see that the top level of the tree describes the entire program and we can look into the body array to see the individual constituents of our above statement represented in the AST.

If you hover over the VariableDeclaration you can see that the entire statement on the left gets highlighted. If we go a level deeper into the declarations array you'll see an additional node VariableDeclarator. If we keep going we'll eventually reach rock bottom. In the case of our hello world statement, it's with the variable's Identifier and the variable's Literal value.

Let's revisit our component from earlier:

const component = html`<h1>Creating custom ESLint rules</h1>`;

If you step through the tree in the AST explorer you can see that the structure matches our image from earlier. Pay particular attention to the TaggedTemplateExpression node and the TemplateLiteral node. These are the ones that will come in handy when we write our ESLint rules.

Our call to the html function is an expression, but it looks a little different from other function definitions. Let's see how the AST differs with an expression like the following:

function heyThere() {
  return 'hey';
}

heyThere();

If we hover over the heyThere() ExpressionStatement, we see that the properties match our html ExpressionStatement. The main difference is that the value in the expression property looks different. The expression this time is a CallExpression, which has a set of properties different to that of our TaggedTemplateExpression.

If we look back at our TaggedTemplateExpression, we can see that we have properties like tag and quasi.

The tag gives us some details about the function name. Which in this case is html.

This means when writing our ESlint rule we'll be able to do something like this:

// Some ESLint psuedo-code
function createRule() {
  return {
    TaggedTemplateExpression(node) {
      const isLitExpression = node.tag.name === 'html';

      if (isLitExpression) {
        // rest of the rule
      }

      // do nothing
    },
  };
}

Finally, if you look into the TaggedTemplateExpression object, you'll see a property named quasi. This property contains our two noteworthy properties expressions and quasis. Take the following expression:

The blue underlines, the first and third respectively, will live in the quasis array and they'll be in the order that they're written in your template literal.

The green underline, the second one, will live in the expressions array, and provides a reference to the name of the variable. Like the quasis, the items in the array are in the order that they're defined. This makes it very easy to reconcile your template literal later on.

Quasis

When accessing the value in your quasis, you'll see the string available as either raw or cooked. These values determine whether escape sequences are ignored or interpreted. Axel Rauschmayer covers this in a little more detail in this article.

Here's a question for you, what happens if the first character of our template literal is an expression? How is this represented in our AST? Try the following snippet in the AST explorer:

const helloWorld = `${name}, how you doin'?`;

Take a little more time exploring quasis and expressions if they still feel unfamiliar to you.

Fortunately, we won't need to directly deal with the parsing process when writing our ESLint rules. We've covered a lot of ground because having a high-level understanding of how the tooling works, makes for a much more intuitive development experience later on.

Super Tiny Compiler

If you're interested in learning a little more about the whole compilation process, the Super Tiny Compiler is a really fun way of building your own JavaScript compiler using only a couple of hundred lines of code.

How Do Eslint Rules Work?

The Visitor Pattern

Fortunately, we don't need to make any transformation when writing ESLint rules and instead we write our checks against specific node types in our code. These nodes are slices from our code's AST.

Once ESLint has parsed your code into an AST, it then traverses your tree, visiting each node along the way. For those familiar with programming design patterns, you might recognise this pattern as the visitor pattern.

The visitor pattern is a way of running some new logic against an object without modifying the object. ESLint uses the visitor pattern to separate the code used to run checks against your code from the AST.

Let's take a look at the visitor pattern in action.

You can see, that I've implemented the visitor using 3 blocks of code:

ast.js: The AST for const name = 'andrico'
traverser.js: An algorithm that traverses the nodes of our AST.
visitors.js: An object of methods where a given method fires once the traverser reaches its corresponding node. In our case, when the traverser reaches a VariableDeclarator node, it fires off our visitor function.

Let's break down the traverser a little more:

We begin in index.js by creating an instance of our Traverser class and passing through our AST and our visitors. Under the hood, our Traverser class stores our AST and visitors as instance variables for us to use later.

We then invoke the instance's traverse method. If you move to the traverser.js file, you can see that when we invoke traverse 5 things can happen:

The node is null, which will happen when we manually invoke the traverse method without any arguments. When this happens, we kick off the traversal function using the AST we stored during the class's initialisation.
The node has a type of Program, which will happen for the top-level nodes in our AST. When this happens we recursively call the traversal method on the child nodes.
The node has a type that matches a visitor function. When this happens, we fire our visitor function and pass through the node as an argument.
The node has additional declarations, so we continue calling our traversal function on those child declarations.
Our node satisfies none of these conditions, which will cause our traversal method to exit.

In the context of our const name = 'andrico' example, our traversal function will continue making its way through the AST until it reaches the VariableDeclarator, where it will invoke the visitor we defined in visitors.js. In this visitor we check to see if the value is Andrico and if it is, we log a message saying that it's an invalid name (though I kinda like it).

Open the console in the CodeSandbox and see what it outputs. Try changing the check in your visitor and see what happens, if anything.

The good news is that ESLint handles the traversal logic for our JavaScript. The other good news is that we'll need to implement the traversal logic for our parsed HTML. 😄

Open WC's eslint-plugin-lit-a11y

This section was heavily informed by my recent involvement with Open WC's eslint-plugin-lit-a11y and eslint-plugin-lint. If you'd like to learn more about (or try a handful of) ESLint rules focused on web components, these are your go-to repos.

What Does an Eslint Rule Look Like?

Writing an ESLint rule doesn't require anything fancy, it's just a plain ol' JavaScript object. The object's top-level can receive two properties: meta and create.

meta provides the metadata for the rule.

The create property is a function that returns an object of visitors that ESLint calls when it visits each node. This follows the same principle as the snippets in the codesandbox. And much like the demo in our codesandbox, the name of each visitor function is the name of the node that we want to visit.

In fact, we can even repurpose the pseudo-code from earlier, and decorate it with the ESLint specific boilerplate:

module.exports = {
  create: function create() {
    return {
      TaggedTemplateExpression(node) {
        const isLitExpression = node.tag.name === 'html';

        if (isLitExpression) {
          // rest of the rule
        }

        // do nothing
      },
    };
  },
};

The create function also provides a context object, which provides some additional helpers and information about the current rule. The helper we're most concerned with right now is the report() method. We can call report whenever we want an ESLint error to display in the console or the IDE.

Context.report takes an object with a handful of properties, but we're most interested in the following:

message: the description of the problem
node: the AST node related to the problem

Additional information

We can pass through additional information, like the line of code we want to report the error on, but that's outside the scope of this tutorial.

Before continuing, why not think about adjusting the pseudocode above to display an ESLint error when a tagged template is invoked, and the template literal has no content, like this:

const expression = html``;

With a basic understanding of JavaScript's AST, the visitor pattern, and the anatomy of an ESLint rule, the only thing left to cover is how to parse our template string into HTML before we can start creating our rules.

For a more in-depth read into the anatomy of an ESLint rule, there's no better place to look than the official docs.

How Can We Transform Our Templates into HTML?

When using ESLint, we have the luxury of ESLint providing us with our parsed JavaScript AST. And while ESLint can't parse our HTML, we can use a library like [parse5](https://github.com/inikulin/parse5) to parse a valid HTML string into a data structure, not unlike our JavaScript AST.

The AST explorer we've spent so much time exploring even has settings for displaying HTML ASTs.

Since one of our rules is going to prevent us from passing through inline styles, let's see how the following gets represented as an AST:

<div style="display:inline;">Main content</div>

If we dive into the AST and look for our div, we can see we're presented with some useful information. The most notable are:

tagName: Which is the name of the html element. (in this case div).

attrs: Which is an array of attributes, represented as a key-value pair. Our div's attrs property holds a single item. The item has a name of style and a value of display:inline;.

Using this information we can already start seeing how to piece together everything we've learned to create our first lint rule.

Here's how we can parse our JavaScript templates using the parse5 library:

import parse5 from 'parse5';

// We're defining out HTML templates
const htmlString = `<div style="display:inline;">Main content</div>`;

// We're passing through an HTML snippet to parseFragment, which returns our HTML AST
const parsedFragment = parse5.parseFragment(htmlString);

// We access the first child because the top-level contains metadata we don't need right now.
const div = parsedFragment.childNodes[0];

// We check to see if there are any style attributes in our div
const hasStyleAttr = div.attrs.some((attr) => attr.name === 'style');

// If there are, we report an error
if (hasStyleAttr) console.log('FAIL');

Thanks to tools like parse 5 and ESLint, we can offload a lot of the complex processing, and focus on writing the code for our specific rules. This is what we'll start doing from the next article onwards.

Vibe Check

Design systems supercharged

RBerthier — Tue, 07 Dec 2021 15:39:31 +0000

Building great front-end products is hard and time-consuming.

Web technologies are evolving at high pace, designers and developers struggle to collaborate on handover, building consistent web components requires loads of works, documentation is too often not up-to-date, maintaining apps over time is an Herculean task, ...

You know the story.

Implementing a Design System is a good solution to tackle these challenges. The Design System is the source of truth for all reusable tokens and components to be consumed in web apps. Building and managing a Design System, as a part of a production pipeline, will greatly improve team efficiency and product consistency.

A good Design System should help teams collaborate, improve reusability, manage technologies evolution, build documentation, ship faster / better, and lower maintenance. But building and maintaining the tooling to manage a Design System remains quite time consuming too ...

That's why we launched Backlight.dev, a powerful and collaborative platform to build Design Systems on the code side. Backlight helps front-end teams to deliver value by focusing on their real work, not on maintaining tools.

Backlight comes with some nice features :

All-in-one : tokens, components, code, stories, tests, documentation, designs, ...
Technologies / frameworks agnostic
Real-time preview rendering for tokens, components and documentation
Real-time and asynchronous collaboration
Live documentation (MDX with embedded components and code snippets)
Versioning and repo management with Github / Gitlab / Bitbucket
Code review and visual diff for Pull-Request
NPM package export
In browser or local CLI
No tools or pipeline to deploy / manage

And best of all, we have a free plan to test Backlight.

Let's give it a try ? :) Start now

Front-end is for heroes. At ‹div›RIOTS, we believe front-end is a beautiful challenge and we are on a mission to empower front-end teams with great tools. Backlight is one of them. We also launched some free sandbox products to play with web components and tokens (check them out here).

Our experience with Astro

Georges Gomes — Fri, 09 Jul 2021 17:10:22 +0000

We built divRIOTS.com with Astro.
"Here we go, another framework is out and another dude is making a website and blogging about it"

Let's see if we can make this interesting. 😉

What is Astro?

If you already know Astro, you can skip to the next chapter.

Astro is a Static Site Generator (SSG). Build your pages in .astro files (very close to HTML/JSX) and sparkle them with React/Vue/Preact/Svelte components. It deals with file-based routing and templates neutrally. Bring your component framework. In my words

There is more to it. See Astro's introduction blog post.

And if you have time, there is a 90min video and Transcript about it.

Why we chose Astro

Not because it's the new/latest shiny framework 😀

I shared my thought in April when I first saw Astro.

// Detect dark theme var iframe = document.getElementById('tweet-1380801812656226304-686'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1380801812656226304&theme=dark" }

When it was time to develop the new divRIOTS.com website, we searched for the best option.

Our requirements were:

Simple - It's not going to be a massive site.
Runs on JavaScript - Ecosystem we know well.
Generate 100% static HTML - Good performance, good SEO.
File-based routing - Very convenient
Allow component-driven development - That's how we like to build
Markdown support - For the blog posts

There are many static site generators.
But, believe it or not, there aren't many options matching our requirements.

Most component-driven options will come with some relatively heavy JavaScript
payloads for hydration, even if the content is 100% static.

On the other end, truly JavaScript-less SSG will use template-engine like Nunjucks or Liquid. They are amazing options but it's another language and another paradigm. Not component-driven.

We just wanted a good Static Site Generator with a JavaScript Developper Experience and 100% HTML output by default. And Astro is precisely that.

Pure Astro

divRIOTS.com is built in 100% Astro.

No React, No Vue, No Svelte, None of the partial hydration or islands capabilities of Astro.

Just .astro files.

You don't need to use the cutting-edge hydration capabilities of Astro to already benefit from it.

Astro comes with an elegant component model and a solid CSS pipeline with Scoped CSS, CSS Modules, and Sass support.

OUT-OF-THE-BOX.

Underrated feature

Astro's CSS Bundling is probably its most underrated feature. It never makes it to the headline but it's by far my favorite!

In Astro, you just layout <style> tags in your astro components where you need them and add lists of <link ref="stylesheet"> in your <head>.

For example, divRIOTS.com uses 2 global css in <head> in the most idiomatic way.

<link href="/css/reset.css" rel="stylesheet" />
<link href="/css/global.css" rel="stylesheet" />

None of these .css files are minified and calling them separately doesn't provide the best performance result.

But when built for production with astro build, <style> tags and <link ref="stylesheet"> are minified and bundled automatically.

If a style only appears on one route, it’s only loaded for that route. (/_astro/[page]-[hash].css)
If a style appears on multiple routes, it’s deduplicated into a (\_astro/common-[hash].css)

In production, pages have:

<link href="/_astro/common-[hash].css" rel="stylesheet" />
<link href="/_astro/mypage-[hash].css" rel="stylesheet" />

/_astro/common-[hash].css is the same on every page. It's cached and not re-downloaded during navigation on the site. It's hard to have a better result.

This means that I can write styles the way it makes sense for readability and maintenance and let astro build take care of best performance.

I don't know any static site generator capable of doing [pure] CSS Bundling and minification so seamlessly.

More details in Astro's Styling Guide #bundling.

Performance results

The output is 100% optimized HTML/CSS. It's hard to be slow 😀

What's missing in Astro

In my humble opinion, not much. divRIOTS.com is proof of that.

But here is my wish list:

JavaScript processing

Like Astro's CSS Bundling, I would like my <script> tags transpiled, bundled, chunked, and minified in the best possible way.

Transpiled : I can write ES202X code and get a more compatible output.
Bundled : I can import bare modules from node_modules
Chuncked : If modules are used on many pages, move them into a single common-chunk.js
Minified : Everybody wants small JavaScript - always.

With this, I don't need a webpack or gulp configuration on top of Astro.

GitHub issue #370

Image processing

Like JavaScript, image optimization is another fairly complex build process to add on top of static site generators. Having out-of-the-box support would help get maximum performance with minimal effort.

GitHub Issue #492

"Permalink” for certain pages

Today all pages are generated as /slug/index.html, but some pages need to be generated as /slug.html instead. Like /404.html.

GitHub Issue #465

Closing thoughts

Astro is more than a SSG

As described in my tweet about Astro, another compelling feature of Astro is his neutrality to frameworks.

Astro takes care of routing, layouts, data management and SSR infrastructure and you can bring your components from any other framework (currently React, Vue3, Preact, and Svelte) but still keep zero JavaScript runtime on the output if you want.

It makes your site last longer as component frameworks come and go. It also makes your component last longer as you don't need to migrate them from one framework to another. Just use them as long as you want.

I called Astro an "Agnostic Meta-Framework". And I think we will see other solutions emerge in this space because it makes a lot of sense to decouple the meta-frameworks from the rendering libraries.

Another Astro website is coming up

Backlight.dev, our upcoming product to build and manage Design Systems on the code-side, will be revealed soon.

The full landing is also made in Astro but taking it to a whole new level 🚀

Stay tuned!