Forem: Florian Rappl

netpack

Florian Rappl — Thu, 09 Jan 2025 23:51:37 +0000

This will be a short post... but hopefully with a longer one following up.

During my winter vacation I got some time to do one experiment:

Can web tooling created using C#/.NET yield comparable performance to Rust / Go / Zig ...?

So I did some coding... (that you can find on GitHub)

The Process

I started with a crude bundler logic:

Open files
Read their content
Use a regular expression to detect, e.g., import statements in JS files
Resolve linked modules
Open resolved / existing package.json files to identify module paths

The result was easy: Using AoT (ahead-of-time compilation) .NET can certainly be used for highly performing web projects.

So I continued a bit with the experiment; replacing regular expressions with actual code understanding.

TLDR; Results

The answer is: yes! 🤯

The bundler is at the moment feature-incomplete, but the first results are quite strong. The benchmarks shown in the README indicate that the performance is definitely in the same ballpark as other tools. So fast enough.

More Details

Personally, I think that C#/.NET is much less complicated than Rust and more powerful than Go. It comes with some drawbacks as well - not gonna lie.

The main reason why C#/.NET can be viable in that space is AoT. Without AoT the startup performance (as well as runtime requirements) is killing the whole idea.

AoT, on the other hand, comes with some challenges. Some libraries cannot be used or require some work to be integrated. Hence, some of the flexibility of .NET cannot be used.

For the largest test project also used by tools such as rspack we get these results:

Note that even the bundler is feature incomplete, it is crafted enough to produce a valid result on the project. So even though all results are at the moment preliminary there is at least some validity to it.

Test	esbuild	rspack	Vite	netpack
Small lib	326ms	611ms	601ms	359ms
Small project	670ms	912ms	1658ms	418ms
Medium project	1931ms	2877ms	10601ms	974ms
Large project	2189ms	2422ms	13710ms	1357ms

So yes, netpack already beats the competition and even has potential for even better performance. While it can be optimized further, it will also loose some performance when things such as sourcemaps or tree shaking are introduced. Right now I am positive that in total it should be around the same as of now due to the potential optimizations (such as streaming in the JS AST generation).

The biggest hurdle at the moment is that it only supports JS(X) - no TypeScript yet (it tries to parse these files, but once types are used it will fail). It would be "fairly" easy to support, but I would need to fork Acornima for that and that's something I'd only do if there is enough buzz around the project.

Outlook

There are many more things that would be superb to get into this. Some basics need to be cleared first though. Things such as sourcemaps, TypeScript support, or maybe a configuration system would be great.

There are some things in this experiment that no other bundler does. For instance, if your HTML entry point has an importmap, then the entries in the importmap are automatically taken as externals. Likewise, you can set certain dependencies as shared - in this case there are automatically importmap entries / an importmap created in the resulting HTML. Quite neat.

In the future the bundler will have native (i.e., out-of-the-box) support for SASS, CSS modules, CSS-in-JS, as well as module federation and native federation.

What are your thoughts? Do you think this is a viable idea or just trash? Is a fast .NET-native bundler with sensible defaults something we need?

Great edition - thanks to all the authors for their hard work!

Florian Rappl — Wed, 04 Dec 2024 08:31:48 +0000

What's Hot in Web Dev?

Florian Rappl ・ Dec 4

#webdev #javascript #frontend #node

What's Hot in Web Dev?

Florian Rappl — Wed, 04 Dec 2024 08:31:02 +0000

This year I started a new series on LinkedIn - "Advanced Links for Frontend". Each issue has 10 links to outstanding posts / articles. This bundle contains the links from the last 5 issues (issue 66 to issue 70).

I hope you enjoy this collection. Let me know in the comments which of these articles is your favorite (and why).

Issue 66

How Bun supports V8 APIs without using V8 (part 2) (https://bun.sh/blog/how-bun-supports-v8-apis-without-using-v8-part-2) by Ben Grant

What is the best React Native list component? (https://expo.dev/blog/what-is-the-best-react-native-list-component) by Simon Grimm

Centering things: a solved problem? (https://fullystacked.net/centering-things/) by Ollie Williams

WebVM 2.0: A complete Linux Desktop Environment in the browser via WebAssembly (https://labs.leaningtech.com/blog/webvm-20) by Alessandro Pignotti

I Followed the Official AWS Amplify Guide and was Charged $1,100 (https://elliott-king.github.io/2024/10/amplify-overcharge/) by Elliott King

The Different (and Modern) Ways to Toggle Content (https://css-tricks.com/the-different-and-modern-ways-to-toggle-content/) by Daniel Schwarz

Using Typescript in node.js scripts without actually writing Typescript (https://www.jonathancreamer.com/using-typescript-in-node-js-scripts-without-actually-writing-typescript/) by Jonathan Creamer

Overflow Clip (https://ishadeed.com/article/overflow-clip/) by Ahmad Shadeed

Conditional Props in React Using Type Discrimination (https://elanmed.dev/blog/conditional-props-using-type-discrimination) by Elan Medoff

Micro Components - a different approach to a simpler component-based web (https://www.wearedevelopers.com/magazine/micro-components---an-alternative-to-web-components-and-frameworks/) by Chris Heilmann

Issue 67

JavaScript's ??= Operator: Default Values Made Simple (https://www.trevorlasn.com/blog/javascript-nullish-coalescing-assignment-operator) by Trevor Lasn

Fit-to-Width Text: A New Technique (https://kizu.dev/fit-to-width/) by Roman Komarov

Self-contained Executable Programs with Deno Compile (https://deno.com/blog/deno-compile-executable-programs) by Ryan Dahl and Andy Jiang

New in React 19 - Pre-warming (https://github.com/facebook/react/issues/29898) by Ricky Hanlon

Will we care about frameworks in the future? (https://paul.kinlan.me/will-we-care-about-frameworks-in-the-future/) by Paul Kinlan

Introducing the vlt Package Manager & Serverless Registry (https://blog.vlt.sh/blog/introducing-vlt-and-vsr) by Darcy Clark et al.

The Saga Pattern in Elixir (https://peterullrich.com/saga-pattern-in-elixir) by Peter Ullrich

Generating Random Mazes with JavaScript (https://cloudfour.com/thinks/generating-random-mazes-with-javascript/) by Paul Hebert

Beautiful focus outlines (https://medienbaecker.com/articles/focus-outlines) by Thomas Günther

The Tragedy of Running an Old Node Project (https://abdisalan.com/posts/tragedy-running-old-node-project) by Abdisalan Mohamud

Issue 68

Long Frames and INP: Understanding Post-Load Performance (https://dev.to/viniciusdallacqua/long-frames-and-inp-understanding-post-load-performance-2maa) by @viniciusdallacqua

Reporting Core Web Vitals With The Performance API (https://www.smashingmagazine.com/2024/02/reporting-core-web-vitals-performance-api/) by Geoff Graham

React Router v7 (https://remix.run/blog/react-router-v7) by Michael Jackson

The Shadow DOM is in the front (https://abstract.properties/the-shadow-dom-is-in-the-front.html) by Sufian Rhazi

Tailwind CSS v4.0 Beta 1 (https://tailwindcss.com/blog/tailwindcss-v4-beta) by Adam Wathan

Announcing TypeScript 5.7 (https://devblogs.microsoft.com/typescript/announcing-typescript-5-7/) by Daniel Rosenwasser

Handling Cookies is a Minefield (https://grayduck.mn/2024/11/21/handling-cookies-is-a-minefield/) by April King

This Website is Hosted on Bluesky (https://danielmangum.com/posts/this-website-is-hosted-on-bluesky/) by Daniel Mangum

Zero Disk Architecture (https://avi.im/blag/2024/zero-disk-architecture/) by Avinash Sajjanshetty

Google’s Chrome Worth Up to $20 Billion If Judge Orders Sale (https://www.bloomberg.com/news/articles/2024-11-18/doj-will-push-google-to-sell-off-chrome-to-break-search-monopoly) by Leah Nylen and Josh Sisco

Issue 69

Webflow Design Language: Bridging The Gap Between Design & Code (https://webflow.com/blog/webflow-design-language) by Merrick Christensen

Building Reddit’s Frontend with Vite (https://www.reddit.com/r/RedditEng/comments/1dhztk8/building_reddits_frontend_with_vite/) by Jim Simon

Interop and Hard Problems (https://bkardell.com/blog/debt.html) by Brian Kardell

Vite 6.0 is out! (https://vite.dev/blog/announcing-vite6.html) by Evan You

A Complete Guide To Live Validation UX (https://www.smashingmagazine.com/2022/09/inline-validation-web-forms-ux/) by Vitaly Friedman

Placeholders in Form Fields Are Harmful (https://www.nngroup.com/articles/form-design-placeholders/) by Katie Sherwin

Browser APIs: Must-Know Secrets for Every Frontend Developer! (https://dev.to/mukhilpadmanabhan/browser-apis-must-know-secrets-for-every-frontend-developer-55lk) by @mukhilpadmanabhan

I learned React Native as a web developer, and I got everything wrong (https://fernandorojo.co/mistakes) by Fernando Rojo

Why TanStack Start is Ditching Adapters (https://tanstack.com/blog/why-tanstack-start-is-ditching-adapters) by Tanner Linsley

Vitest vs. Jest (https://www.speakeasy.com/post/vitest-vs-jest) by Nolan Sullivan

Issue 70

Why am I writing a JavaScript toolchain in Zig? (https://injuly.in/blog/announcing-jam/index.html) by Srijan Paul

November 20, 2024 Release (https://react-spectrum.adobe.com/releases/2024-11-20.html) by React Aria

Effortless UI Spring Animations: A Two-Parameter Approach (https://www.kvin.me/posts/effortless-ui-spring-animations) by Kevin Grajeda

If Not React, Then What? (https://infrequently.org/2024/11/if-not-react-then-what/) by Alex Russell

Comparing AWS S3 with Cloudflare R2: Price, Performance and User Experience (https://kerkour.com/aws-s3-vs-cloudflare-r2-price-performance-user-experience) by Sylvain Kerkour

The Must-Have Obsidian plugins for 2025 (https://www.dsebastien.net/2022-10-19-the-must-have-obsidian-plugins/) by Sebastien Dubois

A Layered Approach to Speculation Rules (https://csswizardry.com/2024/12/a-layered-approach-to-speculation-rules/) by Harry Roberts

Nothing Is Something (https://blog.jim-nielsen.com/2024/nothing-is-something/) by Jim Nielsen

Increasing Vite's potential with the Environment API (https://green.sapphi.red/blog/increasing-vites-potential-with-the-environment-api) by sapphi-red

Existential React questions and a perfect Modal Dialog (https://www.developerway.com/posts/hard-react-questions-and-modal-dialog) by Nadia Makarevich

Conclusion

These are all outstanding articles by masterful authors. I enjoyed reading them all - I hope you did find something in there, too.

👉 Follow me on LinkedIn, Twitter, BlueSky, or here for more to come.

🙏 Thanks to all the authors and contributors for their hard work!

🤯 50 Articles to Become a Web Dev Guru

Florian Rappl — Fri, 15 Nov 2024 19:15:31 +0000

I hope you enjoy this collection. Let me know in the comments which of these articles is your favorite (and why).

Issue 61

Liskov’s Gun: The parallel evolution of React and Web Components (https://www.baldurbjarnason.com/2024/liskovs-gun/) by Baldur Bjarnason

Mozilla rebrand takes from early internet aesthetics, to move its cause to reclaim the web (https://www.itsnicethat.com/articles/jkr-mozilla-rebrand-graphic-design-project-240924) by Ellis Tree

What's New in Ruby on Rails 8 (https://blog.appsignal.com/2024/10/07/whats-new-in-ruby-on-rails-8.html) by Damilola Olantuji

A modest critique of Htmx (https://chrisdone.com/posts/htmx-critique/) by Chris Done

HTML is for people (https://htmlforpeople.com/) by Blake Watson

17 Must-know React Projects for Developers 👩‍💻 🔥 (https://dev.to/tolgee_i18n/17-must-know-react-projects-for-developers-28a2) by @anmolbaranwal

The story of web framework Hono, from the creator of Hono (https://blog.cloudflare.com/the-story-of-web-framework-hono-from-the-creator-of-hono/) by Yusuke Wada

Sites Now Become Interactive 50% Faster (https://www.framer.com/blog/sites-interactive-faster/) by Jacob Groß

Drag to Select (https://www.joshuawootonn.com/react-drag-to-select) by Joshua Wootonn

Why I’m skeptical of rewriting JavaScript tools in “faster” languages (https://nolanlawson.com/2024/10/20/why-im-skeptical-of-rewriting-javascript-tools-in-faster-languages/) by Nolan Lawson

Issue 62

Our Journey with Caching (https://nextjs.org/blog/our-journey-with-caching) by Sebastian Markbåge

Offline UX Patterns (https://github.com/Expensify/App/blob/main/contributingGuides/OFFLINE_UX.md) by Expensify

World Wide Web Foundation closes so Tim Berners-Lee can spend more time with his protocol (https://www.theregister.com/2024/09/30/world_wide_web_foundation_closes/) by Thomas Claburn

why I'm turning off Firefox ad tracking: the PPA paradox (https://blog.zgp.org/why-turn-off-firefox-ad-tracking/) by Don Marti

Hyper-Scale Activated! Ship Your Own FaaS 🤖 (https://dev.to/smapiot/hyper-scale-activated-ship-your-own-faas-40kg) by @florianrappl

Next.js 15 (https://nextjs.org/blog/next-15) by Delba de Oliveira et al

AWS Data Center Latencies - Visualized (https://benjdd.com/aws/) by Benjamin Dicken

Where web components shine (https://daverupert.com/2024/10/super-web-components-sunshine/) by Dave Rupert

Netlify joins OpenNext (https://www.netlify.com/blog/netlify-joins-opennext/) by Mathias Biilmann

New Architecture is here (https://reactnative.dev/blog/2024/10/23/the-new-architecture-is-here) by The React Team

Issue 63

Expo SDK 52 beta is now available (https://expo.dev/changelog/2024/10-24-sdk-52-beta) by Brent Vatne

That's Not an Abstraction, That's Just a Layer of Indirection (https://fhur.me/posts/2024/thats-not-an-abstraction) by Fernando Hurtado

An Early Glimpse of TanStack Start (https://www.youtube.com/watch?v=AuHqwQsf64o) by Tanner Lindsey

How we shrunk our Javascript monorepo git size by 94% (https://www.jonathancreamer.com/how-we-shrunk-our-git-repo-size-by-94-percent/) by Jonathan Creamer

Making content-aware components using CSS :has(), grid, and quantity queries (https://piccalil.li/blog/making-content-aware-components-using-css-has-grid-and-quantity-queries/) by Eric Bailey

Remix Todo App: Part 1 - Building the App Layout and Structure (https://www.udohjeremiah.com/blog/remix-todo-app-part-1-building-the-app-layout-and-structure) by Udoh Jeremiah

Using Rust in Non-Rust Servers to Improve Performance (https://github.com/pretzelhammer/rust-blog/blob/master/posts/rust-in-non-rust-servers.md) by Pretzelhammer

Interactive Select component using Phoenix LiveView and vanilla JS Hook (https://aurmartin.fr/posts/phoenix-liveview-select/) by Aurélien Martin

Unleash the Power of Scroll-Driven Animations (https://css-tricks.com/unleash-the-power-of-scroll-driven-animations/) by Geoff Graham

Incremental adoption of micro-frontends with Cloudflare Workers (https://blog.cloudflare.com/fragment-piercing/) by Peter Bacon Darwin, Dario Piotrowicz, James Culveyhouse, Igor Minar

Issue 64

Headless, boneless, skinless & lifeless UI (https://nerdy.dev/headless-boneless-and-skinless-ui) by Adam Argyle

On Crafting Painterly Shaders (https://blog.maximeheckel.com/posts/on-crafting-painterly-shaders/) by Maxime Heckel

Smarter than 'Ctrl+F': Linking Directly to Web Page Content (https://alfy.blog/2024/10/19/linking-directly-to-web-page-content.html) by Ahmad Alfy

Simplfying Islands Architecture (https://preactjs.com/blog/simplifying-islands-arch/) by Siddharth Gelera

Going through the “State of CSS” results… (https://www.joshwcomeau.com/email/2024-10-22-css-survey/) by Josh W. Comeau

Rendering Outlines with a Post-processing Shader (https://www.atomwolf.org/posts/rendering-outlines-with-a-post-processing-shader/) by Adam Wolf

Should JavaScript be split into two languages? New Google-driven proposal divides opinion (https://devclass.com/2024/10/22/should-javascript-be-split-into-two-languages-new-google-driven-proposal-divides-opinion/) by Tim Anderson

Introducing Wasmer 5.0 (https://wasmer.io/posts/introducing-wasmer-v5) by Syrus Akbary

Should masonry be part of CSS grid? (https://ishadeed.com/article/css-grid-masonry/) by Ahmad Shadeed

Recap: Next.js Conf 2024 (https://vercel.com/blog/recap-next-js-conf-2024) by Lee Robinson, Delba de Oliveira

Issue 65

How headless components became the future for building UI libraries (https://www.subframe.com/blog/how-headless-components-became-the-future-for-building-ui-libraries) by Irvin Zhan

Hyperkit (https://www.chrsgrrtt.com/hyperkit) by Chris Garrett

Converting any website to a React component (https://www.magicpatterns.com/blog/any-website-to-react-component) by Teddy Ni

HTML Form Validation is heavily underused (https://expressionstatement.com/html-form-validation-is-heavily-underused) by everdimension

Using Shadcn UI without a Tailwind Config File (https://www.luisball.com/blog/shadcn-ui-with-tailwind-v4) by Luis Ball

Debugging Microservices & Distributed Systems (https://sentry.io/resources/debugging-microservices-and-distributed-systems/) by Sarah Guthals

RFC: An updated style guide for the year 2024 (https://github.com/angular/angular/discussions/58412) by Jeremy Elbourn

My solar-powered and self-hosted website (https://dri.es/my-solar-powered-and-self-hosted-website) by Dries Buytaert

HTTP Tunnel and Proxy in JavaScript (https://dbushell.com/2024/10/22/http-tunnel-proxy/) by David Bushell

What's a Single-Page App? (https://jakelazaroff.com/words/whats-a-single-page-app/) by Jake Lazaroff

Conclusion

These are all outstanding articles by masterful authors. I enjoyed reading them all - I hope you did find something in there, too.

👉 Follow me on LinkedIn, Twitter, or here for more to come.

🙏 Thanks to all the authors and contributors for their hard work!

Hyper-Scale Activated! Ship Your Own FaaS 🤖

Florian Rappl — Fri, 18 Oct 2024 12:34:36 +0000

In the past decade I've done a lot of work on distributed web applications. In that area you definitely require processes that work implicitly - fully decoupled, with autonomy for individual teams and modules.

The key question quite often is:

How awesome would it be to just extend your application at runtime without any downtime whatsoever?!

In this article I want to look at how we can extend a Node server using Express.js to act like a FaaS provider (such as AWS Lambda or Azure Functions).

Prerequisites

To follow this article you'll need the following:

Node.js with npm installed (I recommend version 22 or higher - we'll use fetch in Node, which is available since version 18)
A code editor
If you want to clone the sample project you'll need to have git installed (but you can download it as a ZIP from GitHub)
Access to the Piral Cloud Feed Service - you can use it for free using a Microsoft or GitHub account

Architecture

Before we start to write some code let's look at the anticipated architecture:

Using the Piral Cloud Feed Service we get access to a dynamic service / module registry, which we can leverage to load a list of modules, evaluate them directly from their URLs, and then use them to populate a list of (sub-)routers.

The routers are necessary to extend the default router middleware. Let's see the whole thing in action. We start by writing the server.

Writing the Server

As server we use a pretty standard and lightweight Express.js server. It can be created by running the following commands in a new directory.

We first initialize a new npm project:

npm init -y

Now we install the required dependency:

npm i express

We add a new file (index.js) with the following content:

const express = require("express");

const port = 3000;
const app = express();

app.get("/", (req, res) => {
  res.send("Hello world!");
});

app.listen(port, () => {
  console.log(`Running at http://localhost:${port}`);
});

And we finally run the application:

node index.js

Go to localhost:3000 in the browser of your choice and see that everything is running as it should:

Great! But there is a bit more to it for making it truly dynamic...

Making the Server Dynamic

So what we are after is to run modules that we can just publish whenever we feel the need for it. The first thing to look for is to avoid having routing issues.

To prevent different modules for occupying the same route we will actually namespace the routes by their module names. In Express.js this also allows us to use dynamic Router objects, which are bound to their module names.

In code this can be done like that:

const routers = {};

app.use("/:namespace", (req, res, next) => {
  const { namespace } = req.params;
  const router = routers[namespace];

  if (router) {
    return router(req, res, next);
  }

  return next();
});

We introduce a middleware that is sensitive to a namespace parameter. In a productive setup we might want to constraint this a bit further, e.g., /api/:namespace or /apps/:namespace, to avoid clashes between in-built functionality and the modules.

The job of the middleware is to use the retrieved namespace value by looking at all available sub-routers. If a sub-router is matched we will use the sub-router as follow-up middleware. Otherwise, we continue with the next middleware. Usually, the next middleware would be the "not-found-route" from Express.js.

The question to answer now is how to populate the routers object. How can we obtain these sub-routers?

What we can do is to introduce a function to obtain the information from a discovery service. This service gets us all the relevant pieces of information - we then only need to load / run the referenced modules.

The necessary code to built something like that is:

const feed = "https://feed.piral.cloud/api/v1/pilet/express-demo";

function makeId(item) {
  return `${item.name}@${item.version}`;
}

async function loadPlugins() {
  console.log("Loading plugins ...");
  const res = await fetch(feed);
  const { items } = await res.json();

  for (const item of items) {
    const id = makeId(item);
    console.log(`Integrating plugin "${id}" ...`);
    await installPlugin(item);
    console.log(`Integrated plugin "${id}"!`);
  }

  watchPlugins();
}

loadPlugins();

What are we doing here?

We want to start loading the plugins / sub-routers as fast as possible. So we immediately run the loadPlugins function. Ideally we use the top-level await here; if it's not available (or if we want to save a bit of startup time) we just call it as shown.
The loadPlugins fetches the information from the discovery service.
Once the information is fetched we go over the contained items array. Each item is treated like the meta information of a plugin - so we just call installPlugin to use the found meta information.
Finally we also want to react when something changes - so the loadPlugins part concludes by watching for changes using the watchPlugins function.

Important Note
If you want to follow this on your own then replace https://feed.piral.cloud/api/v1/pilet/express-demo with your own discovery feed URL. Using the Piral Cloud Feed Service you can see the URL on the feed overview page.

Let's look at the installPlugin first:

const current = [];

async function installPlugin(item) {
  const { name, link } = item;
  const router = express.Router();
  const { setup } = await loadModule(link);
  typeof setup === "function" && setup(router);
  routers[name] = router;
  current.push({ id: makeId(item), name });
}

Doesn't look too complicated, does it? What's happening here?

We decompose the item - but we are only interested in the name and link (module reference / URL) of the module.
We create a new Express.js Router instance (the "sub-router").
We load the respective module - it should get us a function called setup (now this is a design choice; you could either name it differently, make it a default export, or export a Router instance directly...
If we really obtained a setup function we call it with the Router instance. This can now be modified properly.
Finally, the created router is assigned to the routers object and the current array is extended with the proper entry.

As you might have guessed the current array is for book-keeping purposes. It allows us to easily patch / change the routers in case of changes. For this, we'll need to look at the watchPlugins function:

const WebSocket = require("ws");

const changeEventTypes = ["add-pilet", "update-pilet", "remove-pilet"];

function watchPlugins() {
  console.log("Watching plugins ...");
  const ws = new WebSocket(feed.replace("http", "ws"));

  ws.on("error", console.error);

  ws.on("message", async (data) => {
    const msg = JSON.parse(Buffer.from(data).toString("utf8"));

    if (changeEventTypes.includes(msg.type)) {
      const res = await fetch(feed);
      const { items } = await res.json();
      const removeItems = current.filter(
        ({ id }) => !items.some((n) => makeId(n) === id)
      );
      const addItems = items.filter(
        (item) => !current.some(({ id }) => id === makeId(item))
      );

      for (const item of removeItems) {
        await uninstallPlugin(item);
      }

      for (const item of addItems) {
        await installPlugin(item);
      }
    }
  });
}

For this code to run we rely on the ws library. So we'll need to install it first:

npm i ws

With ws active we can have a look at what the code does:

We define some events we want to listen to. The provided events are send via the WebSocket channel in case the contained modules change.
Once a message is received we transform it (from raw bytes to a JSON object) and check if the type matches one of the events defined in (1).
We could now use the information from the event, but some events (e.g., update-pilet) don't tell us what changed. In such cases we could be in a problematic zone. So we follow the safe route and just re-load all the modules.
Finally, we need to compare the current information from the discovery service vs the stored / previous information. For the changed (removed, updated) modules we remove them using the uninstallPlugin. All the ones that are not yet instantiated we add them using the installPlugin function.

The uninstallPlugin looks as follows:

async function uninstallPlugin(item) {
  delete routers[item.name];
  current.splice(current.indexOf(item));
}

That was easy, right? But so far we avoided the most complicated topic. How are we evaluating those modules?

Evaluating Modules from URLs

In the browser you can always evaluate ESMs - they are coming from an URL by default. The same is true in runtimes such as Deno. But in Node.js this is not possible. So should we give up?

As it turns out evaluating URLs is actually not so difficult. We need 2 things:

Code to use the vm module. We need to rely on the SourceTextModule to create modules on the fly.
The experimental flag --experimental-vm-modules for running Node. This way, the SourceTextModule is available at runtime.

Using those two ways we can add the following code to our server:

const vm = require("vm");

async function linkModule(url, ctx) {
  const res = await fetch(url);
  const content = await res.text();
  const mod = new vm.SourceTextModule(content);
  mod.context = ctx;
  await mod.link((specifier) => {
    const newUrl = new URL(specifier, url);
    return linkModule(newUrl.href, ctx);
  });
  await mod.evaluate();
  return mod;
}

async function loadModule(url) {
  const ctx = vm.createContext();

  try {
    const res = await linkModule(url, ctx);
    return res.namespace;
  } catch (ex) {
    console.warn(`Failed to evaluate "${url}":`, ex);
    return {};
  }
}

Finally, we have everything together and can just run the server with the flag:

node --experimental-vm-modules index.js

We should see output like this in the console:

Loading plugins ...
Running at http://localhost:3000
Integrating plugin "app1@2.0.0" ...
Integrated plugin "app1@2.0.0"!
Integrating plugin "app2@1.0.4" ...
(node:23052) ExperimentalWarning: VM Modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Integrated plugin "app2@1.0.4"!
Integrating plugin "app3@1.0.0" ...
Integrated plugin "app3@1.0.0"!
Watching plugins ...

Here, we are already including a couple of modules (referred to as plugins). Let's see what we can do here - how we can write, deploy, and update them.

Example Functions

One Module

The first example is a really simple one; just a single function to return a constant string:

export function setup(router) {
  router.get("/foo", (req, res) => {
    res.send("Hello from app1 v2: /foo");
  });
}

Importantly, we initialize a new project and define the package.json like this:

{
  "name": "app1",
  "version": "1.0.0",
  "module": "index.js",
  "scripts": {
    "deploy": "publish-microfrontend --url https://feed.piral.cloud/api/v1/pilet/express-demo --interactive"
  },
  "devDependencies": {
    "publish-microfrontend": "^1.6.2"
  }
}

The publish-microfrontend script is used to deploy the module. This little script just packages the current project and invokes a POST against the provided discovery service URL. The --interactive flag then allows us to dynamically log into the discovery service to deploy our function.

We'll see how this process looks in a second. Let's look at another example.

Two Modules

While publishing the package with a single module works as expected the question is: Does it also work with more modules? The answer is yes.

Pretty much the same definition in the package.json:

{
  "name": "app2",
  "version": "1.0.0",
  "module": "lib/index.js",
  "scripts": {
    "deploy": "publish-microfrontend --url https://feed.dev.piral.cloud/api/v1/pilet/express-demo --interactive"
  },
  "devDependencies": {
    "publish-microfrontend": "^1.6.2"
  }
}

But now lib/index.js is different:

import { compute } from './other.js';

export function setup(router) {
  router.get("/compute", (req, res) => {
    const { a, b } = req.query;
    const c = compute(+a, +b);

    if (!isNaN(c)) {
      return res.status(200).send(`${c}`);
    }

    return res.status(400).send(`Only numbers allowed.`);
  });
}

Not only is this a bit more logic - it also uses a function from another module located at lib/other.js:

export function compute(a, b) {
  if (typeof a === "number" && typeof b === "number") {
    return (a + b) * (a - b);
  }

  return NaN;
}

As you might have guessed our ESM URL loader just works as it should. No questions asked. Great!

We won't answer the burning question: Can it do three modules? I guess we know the answer, but if you want to find out get active now!

In the meantime we go already to the complex case of having the modules written in TypeScript - and including some dependencies.

TypeScript Modules with Dependencies

Consider the code at src/index.ts:

import cors from 'cors';
import type { Router } from "express-serve-static-core";

export function setup(router: Router) {
  router.use(cors());

  router.get("/", (req, res) => {
    res.status(404).send(`NOT FOUND.`);
  });
}

Here we are using types and dependencies. This won't work - we don't do any dependency resolution and we don't transpile TypeScript. So should we be worried? Not at all!

Let's bundle it using esbuild:

esbuild --outdir=dist src/index.ts --bundle --platform=node --format=esm

Even better - we can just extend the package.json to take care of this in the deploy task:

{
  "name": "app3",
  "version": "1.0.0",
  "module": "dist/index.js",
  "scripts": {
    "build": "esbuild --outdir=dist src/index.ts --bundle --platform=node --format=esm",
    "deploy": "npm run build && publish-microfrontend --url https://feed.dev.piral.cloud/api/v1/pilet/express-demo --interactive"
  },
  "devDependencies": {
    "@types/cors": "^2.8.17",
    "@types/express": "^5.0.0",
    "esbuild": "^0.24.0",
    "publish-microfrontend": "^1.6.2"
  },
  "dependencies": {
    "cors": "^2.8.5"
  }
}

This way, we can just write anything - bundle it using esbuild and ship it with publish-microfrontend.

Now let's step back a bit and see how we actually deploy the modules:

Deployment

Maybe let's see first how it looks in the discovery service when we deployed all of our modules:

As already mentioned for deploying such a module we can use the available publish-microfrontend npm package:

npx publish-microfrontend --url <your-discovery-service-url> --api-key <your-key>

In cases such as the ones above we might swap the --api-key with an --interactive flow, where we need to use the browser for obtaining a user token instead.

npx publish-microfrontend --url <your-discovery-service-url> --interactive

Let's see how this process looks in real life:

In the end having such a powerful service registry between our server and the respective modules is not only clean, but also useful for flexibility reasons.

Advanced Dynamics

We already saw that the dynamics with respect to partial rollbacks are in place from the beginning. In general, the service registry's portal gives us the ability to control every bit of the delivery process:

The version selector in particular is quite useful. Besides the possibility of making rollbacks, we can also do feature flags, A/B testing, blue-green deployments, or canary releases.

Quite powerful stuff...

This allows us, e.g., to have multiple instances of our Express.js server, where each instance might get a different set of modules. Sounds crazy (how can this be useful?!) but if you don't consider these instances equal (i.e., meant for scaling), but rather as different sites (e.g., one for US, one for EU etc.) then backend behavior might be different - at least in some areas.

Beforehand, that was a lot of overhead. You had to produce countless variations of the same server - now you can just have one and have the different behavior qualified through the loaded modules.

Conclusion

In this article you've seen how you can easily extend your Express.js server into a FaaS platform allowing you to add, update, and remove endpoints on the fly.

Of course, the provided sample is rather simplistic and does not solve some of the issues including advanced isolation, memory footprint (unload and restrictions of modules), local module development using an emulator, and many others. But it's a very convenient start - showing what's possible with just a few lines of code.

Check out the sample code at github.com/piral-samples/piral-cloud-express-plugins-demo.

🎃 50 Articles to Avoid Web Dev Nightmares

Florian Rappl — Tue, 15 Oct 2024 13:28:34 +0000

I hope you enjoy this collection. Let me know in the comments which of these articles is your favorite (and why).

Issue 56

The CSS radial composer (https://zumerlab.github.io/orbit-docs/) by Orbit
Gravity is everywhere! For some reason I feel attracted to this.

Remix's concurrent submissions are fundamentally flawed (without causal ordering) (https://dashbit.co/blog/remix-concurrent-submissions-flawed) by José Valim
Honestly, I am not surprised.

ts-blank-space (https://bloomberg.github.io/ts-blank-space/) by Ashley Claymore
This can be really useful, but I wonder why not just esbuild is used. It's fast and can also be used for more.

date-fns v4.0 is out with first-class time zones support! (https://blog.date-fns.org/v40-with-time-zone-support/) by Sasha Koss
Well, having time zones support is one of the things I'd consider necessary for any good time library.

Fastify v5 is Now Officially Released! (https://openjsf.org/blog/fastifys-growth-and-success) by Fastify Team
Fastify is great and presumably under-appreciated in the Node.js community.

WebKit Features in Safari 18.0 (https://webkit.org/blog/15865/webkit-features-in-safari-18-0/) by Jen Simmons
Keep an eye on this - view transitions and WebXR are huge! Fantastic release!

Biome v1.9 Anniversary Release (https://biomejs.dev/blog/biome-v1-9/) by Victorien Elvinger
Alright I can happily say that finally Biome is also becoming interesting to me. Goodbye Prettier + eslint!

A milestone for TypeScript Performance in TanStack Router (https://tanstack.com/blog/tanstack-router-typescript-performance) by Christopher Horobin
Great write-up with some insights on how the type checker / TypeScript performance can be evaluated and improved.

New Values and Functions in CSS (https://dev.to/alvaromontoro/new-values-and-functions-in-css-1b9o) by @alvaromontoro
Well, some of these functions are not really new, but it's always good to be reminded about them.

Deno 2.0 Release Candidate (https://deno.com/blog/v2.0-release-candidate) by Andy Jiang and Bartek Iwańczuk
The v2 promises to be a really well-rounded release. Time for everyone to take another look at Deno!

Issue 57

How Discord Reduced Websocket Traffic by 40% (https://discord.com/blog/how-discord-reduced-websocket-traffic-by-40-percent) by Austin Whyte
A lot of micro optimizations and lazy loading (or "sending") for the win!

You Can't Build Interactive Web Apps Except as Single Page Applications... And Other Myths (https://htmx.org/essays/you-cant/) by Tony Alaribe
Never heard that myth - but it's fine for me to claim that fire has been rediscovered.

Component Composition is great btw (https://tkdodo.eu/blog/component-composition-is-great-btw) by Dominik Dorfmeister
Truly. What's also great is to have the right company boundaries.

Compile and run C in JavaScript (https://bun.sh/blog/compile-and-run-c-in-js) by Jarred Sumner
More like: Compile and run C in Bun. As usual, I see the devil being in the detail.

The Nine Node Pillars (https://www.platformatichq.com/node-principles) by Matteo Collina et al.
It's great to recap those pillars, even though I did not find anything really Node.js specific (most of the points could be assigned on to any web server).

Module Federation Vite the new v1 is here 🎉 (https://www.learnbydo.ing/diary/2024-09-23-module-federation-vite-v1) by Giorgio Boa
Happy to see that progress is made! Now we only miss all the other bundlers ...

The Golden Ratio in CSS (https://dev.to/madsstoumann/the-golden-ratio-in-css-53d0) by @madsstoumann
Sweet - starts with some boxes and ends with a lot of rings. I love it!

Sticky Transitioning Veggie Hamburger 🍔 (https://dev.to/florianrappl/sticky-transitioning-veggie-hamburger-7dd) by @florianrappl
Put your web app on a diet and reduce the amount of JS - especially when not needed.

Why Typed Linting Needs TypeScript Today (https://www.joshuakgoldberg.com/blog/why-typed-linting-needs-typescript-today/) by Josh Goldberg
Every once in a while somebody tries to reimplement TypeScript - so far everyone failed. Yes, TypeScript is required.

Improving rendering performance with CSS content-visibility (https://nolanlawson.com/2024/09/18/improving-rendering-performance-with-css-content-visibility/) by Nolan Lawson
This is indeed magical. I see some difficulties here, but certainly no less than doing actual lazy loading / virtualization in JavaScript.

Issue 58

Ever faced the problem of dead space through gaps using flexbox or grid? Then read about
Sanding UI (https://blog.jim-nielsen.com/2024/sanding-ui/) by Jim Nielsen

As you are all aware Ahmad has become my favorite author for CSS related articles. Recently he gave his website some polish. Read about the
Redesign Case Study (https://ishadeed.com/article/redesign-2024/) by Ahmad Shadeed

Now that we conquered client islands it's time for server islands to become standard. One way is discussed in
Partial Prerendering for Everyone with Cloudflare Workers (https://sunilpai.dev/posts/ppr-for-everyone/) by Sunil Pai

It was quite a week for web components. We started with
Web Components Are Not the Future (https://dev.to/ryansolid/web-components-are-not-the-future-48bh) by @ryansolid

But as usual on this topic there had to be different opinions. Entering
Web Components Are Not the Future — They’re the Present (https://www.abeautifulsite.net/posts/web-components-are-not-the-future-they-re-the-present/) by Cory LaViska

In the end both have their points and perspectives, but like the old XKCD comic it was the job of a third person to summarize in
Web components are okay (https://nolanlawson.com/2024/09/28/web-components-are-okay/) by Nolan Lawson

Years ago we tried to introduce scoping in CSS - but without much success. Now the working group is back with power... Let's try
Understanding the Concept of Scoping in CSS (https://dev.to/bilkeesu96/understanding-the-concept-of-scoping-in-css-1m9g) by @bilkeesu96

A major downside of classic cron jobs is the failure cascade. This can be countered by some worker frameworks - for instance offering something dubbed durable functions.
What are Durable Functions? A visual JavaScript primer (https://www.inngest.com/blog/durable-functions-a-visual-javascript-primer) by Lydia Hallie

Having tests is both - a pain and necessity. Especially in the frontend things tend to become difficult - relying on the browser and user behavior for many features. Let's recap
Testing Frontend — Lessons from over a million lines of TypeScript at Palantir (https://www.meticulous.ai/blog/lessons-from-a-decade) by Quentin Spencer-Harper

Not only the web component department delivered... If you missed the excitement delivered by the WordPress community then read
The messy WordPress drama, explained (https://www.theverge.com/2024/9/27/24256361/wordpress-wp-engine-drama-explained-matt-mullenweg) by Emma Roth

Issue 59

Building a Single-Page App with htmx (https://jakelazaroff.com/words/building-a-single-page-app-with-htmx/) by Jake Lazaroff
Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.

Keeping an eye on changes to MDN’s Browser Compatibility Data (https://bcd-watch.igalia.com/) by igalia
This is quite neat - not only knowing what changed, but also when it changed.

*OpenFreeMap *(https://openfreemap.org/) by Zsolt Ero
Eagerly waiting to use this with Leaflet - but the Mapbox demos look great, too.

Bridging the hard and the soft (https://wattenberger.com/thoughts/hard-and-soft) by Amelia Wattenberger
Superb read - it's all about the interface.

ESLint now officially supports linting of JSON and Markdown (https://eslint.org/blog/2024/10/eslint-json-markdown-support/) by Nicholas Zakas
I am not getting enough warnings already.

Tauri 2.0 Stable Release (https://v2.tauri.app/blog/tauri-20/) by Tillmann Weidinger
It's now just a much better DX.

Running Clang in the browser using WebAssembly (https://wasmer.io/posts/clang-in-browser) by Syrus Akbary
I need to repeat: Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.

Unleash JavaScript's Potential with Functional Programming (https://janhesters.com/blog/unleash-javascripts-potential-with-functional-programming) by Jan Hesters
Personal opinion: Don't get too much into FP, but just take the best pieces and use them.

A Local-First Case Study (https://jakelazaroff.com/words/a-local-first-case-study/) by Jake Lazaroff
Local-first is underappreciated but should be a cornerstone for good DX.

Why Gumroad Didn't Choose htmx (https://htmx.org/essays/why-gumroad-didnt-choose-htmx/) by Sahil Lavingia
It's fine. No one needs to choose everything. Everything has trade-offs and I believe in choice.

Issue 60

If you look for icons to be used in your next project - then maybe start directly at
15+ Best Icon Libraries of 2024 (https://dev.to/vinishbhaskar/best-icon-libraries-28ce) by @vinishbhaskar

Ever wondered how to get or set up a great CRM for free? if so, check out
Atomic CRM (https://marmelab.com/atomic-crm/) by Marmelab

In case you missed the preview you can now enjoy the full next gen Node.js competitor;
Announcing Deno 2 (https://deno.com/blog/v2.0) by Ryan Dahl

Are you sometimes hesitant to just install npm packages? Paranoid, maybe not - here are some
Nightmares on npm: How Two Malicious Packages Facilitate Data Theft and Destruction (https://socket.dev/blog/nightmares-on-npm-how-two-malicious-packages-facilitate-data-theft-and-destruction) by Kush Pandya

I think there is a great new framework in town (no week without it!) - server-first with JSX to web components for islands of interactivity. Entering
The Web Platform Framework (https://brisa.build) by Aral Roca Gomez

Ever thought that logs can provide much more insights, but are too clumsy to work with? Maybe strict sequential logs are not the answer. One approach can be found by reading an
Introduction to Causal Logs (https://joelgustafson.com/posts/2024-09-30/introduction-to-causal-logs) by Joel Gustafson

Besides having a new framework each week there is also another attempt on explaining and visualizing how React renders (or reconciliates) your view - so let's dive into
The Interactive Guide to Rendering in React (https://ui.dev/why-react-renders) by UiDev

Do you believe that JavaScript on the server is essentially PHP? I did not - but if you did then read
React on the server is not PHP (https://www.artmann.co/articles/react-on-the-server-is-not-php) by Christoffer Artmann

Do you like mysteries? Then try solving the riddle of
Node.js, pipes, and disappearing bytes (https://sxlijin.github.io/2024-10-09-node-stdout-disappearing-bytes) by Sam Lijin

I have a feeling that the server-side story for React gets more fragmented and unnecessarily complex each day - even without talking about
Serverless servers and the challenge of new React architecture (https://bobaekang.com/blog/serverless-servers-and-the-challenge-of-new-react-architecture/) by Bobae Kang

Conclusion

These are all outstanding articles by masterful authors. I enjoyed reading them all - I hope you did find something in there, too.

👉 Follow me on LinkedIn, Twitter, or here for more to come.

🙏 Thanks to all the authors and contributors for their hard work!

Sticky Transitioning Veggie Hamburger 🍔

Florian Rappl — Wed, 25 Sep 2024 08:40:42 +0000

In our efforts to make the page faster and even faster we also explored different ways of just omitting user interactivity altogether. One place we looked at intensively was the header (or general layout). The header contained a bit of interactivity in form of a React component.

The job of the header component is:

on mobile show a persistent header bar with a hamburger instead of the normal entries
on desktop show all the menu entries, but switch the appearance once scrolling occurs (full height to less height with a shadow below)
the menu on mobile should not have any scrolling (i.e., remove scrollbar if any)

In practice this looks like the following on desktop:

And for mobile the appearance changes to this:

Let's see how we went from the original to a non-JS version that behaves the same.

Original Meat Burger

Alright, we start with the original version of our header (the "hamburger"). To stay rather generic for this article we prepared a small sample that evolves with every step.

We start with the original:

Most of the code is in the CSS-in-JS. You don't need to use the CSS-in-JS here (we actually use SASS), but it easily brings the two parts (CSS and React) that are needed for the component together.

The critical part is that within the component we need to use some hooks (i.e., interactivity) to make it work:

export const PageHeader = ({ children }) => {
  const [open, setOpen] = React.useState(false);
  const header = useStickyHeader();
  useLockBodyScroll(open);

  // ...
};

The goal is thus to reduce it. Let's start with the easy thing first: How to get the hamburger menu working without JS.

Less Meat - Same Taste

How can we toggle two states using CSS and no JS? Well, first we need to understand that CSS inherently allows us to distinguish between states. For instance, CSS allows to select based on

empty
hovered
focused

and more states directly. Among those are also special states of some form fields, e.g., if a checkbox is checked.

So instead of toggling based on a data- attribute, i.e., the following code in CSS:

header[data-open="true"] {
  display: flex;
  height: 100%;
}

we can toggle this based on another selector:

header:has(#header-hamburger-open:checked) {
  display: flex;
  height: 100%;
}

This assumes that a new element with the ID header-hamburger-open has been introduced. This is a checkbox that is hidden to the user. The effect of hiding the checkbox can be done quite easily using CSS, too:

#header-hamburger-open {
  visibility: collapse;
  height: 0;
}

Personally, I like to use visibility: collapse as it transports the meaning the best. Unfortunately, for most elements Webkit-based/derived browsers will behave for collapse like for hidden - i.e., reserving the original space but not rendering the element.

Only Firefox behaves in a way that aligns with the original meaning (but contrary to the spec - which is actually weird in this scenario). That's why we need the additional height: 0 to also prevent allocating same space on Webkit-based browsers such as Chrome.

With this in mind we replace the original button inside the navbar-toggler with the following code:

<input type="checkbox" id="header-hamburger-open" />

<div className="navbar-toggler">
  <label
    htmlFor="header-hamburger-open"
    data-toggle="collapse"
    aria-controls="navbar-menu-content"
    aria-label="Toggle navigation"
  >
    🍔
  </label>
</div>

When clicking on the label the htmlFor attribute is used by the browser. This will lead to toggling the checkbox, which - as per our CSS structure - will toggle the menu.

You can play around with this in the updated sample.

Now that the easy part is done - let's look at the final step for making the truly JS-free: Having the sticky positioning fulfilled.

Veggie Version - Original Taste

The main complication when moving the sticky header from a JS transition to a CSS one is that CSS has no idea about scrolling.

Let's see the original hook / code to trigger the transition:

function useStickyHeader() {
  const header = React.useRef(undefined);
  React.useLayoutEffect(() => {
    if (header.current) {
      const sticky = header.current.offsetTop;
      window.onscroll = () => {
        if (window.pageYOffset > sticky) {
          header.current.classList.add("sticky");
        } else {
          header.current.classList.remove("sticky");
        }
      };
    }

    return () => (window.onscroll = undefined);
  }, [header.current]);
  return header;
}

So by applying the CSS class sticky we transform the original header to the sticky one. Without JavaScript - this won't be possible.

Maybe - so the idea - it is possible to have something set from the beginning. So the idea is to have the right layout and not rely on a scrolling-based transition.

The starting point should be the CSS declaration position: sticky. Using position: sticky we can tell the browser to make the menu sticky (in relation to its scrolling container) without relying on a transition.

The problem with position: sticky is that we can only tell the browser what "position" we want to settle in. For instance, we can tell the browser top: -10px to keep moving the header bar until it's 10 pixels inside the containers hidden area, i.e., shifted to the top by 10px. There is no way to add another shadow or other things we might be interested in...

This is the point where we could call it a day, marking the task as "impossible" and move on.

Luckily, there are still some tricks we can apply in this case. Let's look at what we want:

In other words, we want to "reveal" part of the shadow when we move up. The moving up part we get covered by using position: sticky as outlined - now the shadow part could be tricked in by really introducing such layers.

What we mean specifically is that we could introduce two more "virtual" elements (using ::before and ::after pseudo selectors):

one element moving together with the sticky header (i.e., absolutely positioned) - initially "hiding" the shadow
one element as a shadow that is being fixed; thus not moving with the sticky header (initially being hidden behind the absolutely positioned element)

Note that both need to be in the foreground. This means we have a layering such as:

The real header (blue) is now shorter, but extended with the overlay element (green), which is initially hiding the shadow (purple). Once the page (red) scrolls, the header scrolls up to a certain point. Together with it the overlay is supposed to scroll, thus revealing the shadow:

In theory that should work - but in reality it will all be determined by the specific values we pick for the positioning of these elements.

In CSS the following is needed to introduce the two virtual elements:

header + *::before {
  position: fixed;
  width: 100%;
  height: 45px;
  content: '';
  background: #ccc;
  box-shadow: 0 3px 5px rgba(57, 63, 72, 0.3);
  top: 0;
  left: 0;
  z-index: 9999;
}

header + *::after {
  position: absolute;
  width: 100%;
  height: 55px;
  content: '';
  background: #ccc;
  top: 0;
  left: 0;
  z-index: 9999;
}

The header itself is positioned in a way that makes it be in front of these layers:

header {
  width: 100%;
  position: sticky;
  top: -5px;
  z-index: 10000;
}

This creates the effect we are looking for - and without requiring any JavaScript whatsoever.

Conclusion

Going to a full non-JS version of our header made the lighthouse score even better - being a full 100 on most / all pages. Now, whenever JS is needed it would be just there to enhance a certain view or give more details. There is no more JavaScript to actually enable navigation or other crucial means.

As far as the sticky header is concerned, the shown trick only works in scenarios like ours. If you need more transformations then finding a good workaround might be more difficult. In the future that may change, but for now position: sticky is quite limited and requires such tricks to also have scenarios such as ours covered.

🧠 50 Articles to Master Web Dev

Florian Rappl — Fri, 20 Sep 2024 16:25:02 +0000

Image by freepik

I hope you enjoy this collection. Let me know in the comments which of these articles is your favorite (and why).

Issue 51

Fixing a Bug in Google Chrome as a First-Time Contributor (https://cprimozic.net/blog/fixing-a-bug-in-google-chrome/) by Casey Primozic
The last time I had a look at the Chromium codebase was years ago - and it is massive. So much respect to the author for getting in there!

The Only Widely Recognized JavaScript Feature Ever Deprecated (https://www.trevorlasn.com/blog/the-only-javascript-feature-that-was-deprecated/) by Trevor Indrek Lasn
I am happy to say that I don't miss "with". I actually only used it once; and even though it made sense for in that case, there would have been other options.

React Email 3.0 (https://resend.com/blog/react-email-3) by Gabriel Miranda
Fantastic changes, superb release!

Implementing React From Scratch (https://www.rob.directory/blog/react-from-scratch) by Robby Pruzan
Been there, done that. As usual, coming up with something that works is not hard - coming up with something that works as good as the original is difficult.

Developer tools are different than tools for any other profession (https://mxstbr.com/notes/developer-tools) by Max Stoiber
If Max says so!

Scroll Snap Events (https://developer.chrome.com/blog/scroll-snap-events) by Adam Argyle
Funny - just a couple of weeks after I'd needed that.

State of JavaScript: Insights from the latest JavaScript developer survey (https://www.infoworld.com/article/3486850/state-of-javascript-insights-from-the-latest-javascript-community-survey.html) by Matthew Tyson
Surprised that the Bun bundler gets a positive rating. That it does not even print some elemental error messages is a red flag for me.

50 TypeScript F!ck Ups Book (https://github.com/azat-co/50-ts/) by Azat Mardan
I like the collection. Definitely worth supporting the author!

Common misconceptions about how to optimize LCP (https://web.dev/blog/common-misconceptions-lcp) by Brendan Kenny
The images part, unfortunately, is underrated. Having just optimized a web app last week the images optimization part took the project fom a mid 70s rating to a 99 rating in lighthouse. In general I agree though.

Elastic Overflow Scrolling (https://css-tricks.com/elastic-overflow-scrolling/) by Dave Seidman
While I don't see it being essential for the Y axis, I think this technique is useful for carousels / on the X axis.

Issue 52

Compilation of JavaScript to Wasm, Part 3: Partial Evaluation (https://cfallin.org/blog/2024/08/28/weval/) by Chris Fallin
The topic sounds more exotic than it is - personally, I think this will be quite standard in a couple of years.

Material UI v6 is out now 🎉 (https://mui.com/blog/material-ui-v6-is-out/) by Material UI
Great to see that MUI is still progressing!

Announcing Rspack 1.0 (https://rspack.dev/blog/announcing-1-0) by rspack
Wonderful milestone - great project! More than 80% of the top 50 plugins for Webpack are supported - so chances are high that you can migrate quite smoothly.

Frontend Resources V2! 🚀 (https://dev.to/miguelrodriguezp99/frontend-resources-v2-57mj) by @miguelrodriguezp99
Collection of useful tools and libraries to simplify your development life.

New to the web platform in August (https://web.dev/blog/web-platform-08-2024) by Rachel Andrew
In the west the ruby element is quite unknown, but even here it can be super useful.

When 3rd party JavaScript attacks (https://changelog.com/jsparty/336) by Simon Wijckmans
When has a third-party JavaScript never attacked?!

JavaScript Generators Explained, But On A Senior-Level (https://www.reactsquad.io/blog/understanding-generators-in-javascript) by Jan Hesters
They are certainly useful - especially in the pull-stream scenario.

Component testing in Storybook (https://storybook.js.org/blog/component-testing/) by Michael Shilman
Yeah sure - I am not surprised to see this in the Storybook blog.

Faster Pages with React ✨ (https://dev.to/smapiot/faster-pages-with-react-h8j) by @florianrappl
If you really need more performance - you can always get more performant.

The secret inside One Million Checkboxes (https://eieio.games/essays/the-secret-in-one-million-checkboxes/) by Nolen Royalty
That was actually a superb read. What a journey! Seems like building this little site has given Nolen stories for the next decade.

Issue 53

CSS-only infinite scrolling carousel animation (https://blog.logto.io/css-only-infinite-scroll) by Gao Sun
Since I have played around with this myself two weeks ago I thought the article was great.

12 Keys to Write Senior-Level Tests (https://www.reactsquad.io/blog/testing-desiderata-in-javascript) by Jan Hesters
I think this one is the list to follow after the 10 commandments.

MDX with Translations 🐠 (https://dev.to/smapiot/mdx-with-translations-157c) by @florianrappl
MDX might be the secret sauce that actually provides the bridge between "just content" and "just code".

Inside ECMAScript: JavaScript Standard Gets an Extra Stage (https://thenewstack.io/inside-ecmascript-javascript-standard-gets-an-extra-stage/) by Mary Branscombe
What else is there to do? Do we really want or need a Frankenstein's monster of a language?

What’s new in React 19 (https://vercel.com/blog/whats-new-in-react-19) by Michael Novotny
I'm wondering that myself. Last time I checked it's rather "what will be new in React 19".

Preloading files to reduce download chains in the browser (https://www.lkhrs.com/blog/2024/preloading/) by Luke Harris
Or just batch them in the backend / at compile-time.

The future is standalone! (https://blog.angular.dev/the-future-is-standalone-475d7edbc706) by Alex Rickabaugh
I think the future is standalone since more than a decade, but I am happy that finally Angular recognizes this, too.

Introducing @bramus/style-observer, a MutationObserver for CSS (https://www.bram.us/2024/08/31/introducing-bramus-style-observer-a-mutationobserver-for-css/) by Bramus Van Damme
While I applaud the effort I will definitely avoid scenarios that force me into using it.

i-html (https://www.keithcirkel.co.uk/i-html/) by Keith Cirkel
This is great - but pi-component can already do that (and more). Still happy that the need for fragment inclusion is becoming more recognized.

Build-time Components (https://codehike.org/blog/build-time-components) by Rodrigo Pombo
Alright - what did I say about MDX?! But great write-up and it also goes into the different composition modes.

Issue 54

Look out, kids: PHP is the new JavaScript (https://www.mux.com/blog/php-is-the-new-javascript) by Dave Kiss
No, it's not. But generally it's nice to see PHP (and its ecosystem) evolving.

An SSR Performance Showdown (https://blog.platformatic.dev/ssr-performance-showdown) by Matteo Collina
Long story short: If you need something simple, choose something that does less work.

Managing Angular (https://blog.mgechev.com/2024/08/25/managing-angular/) by Minko Gechev
Certainly a great read. The only thing I missed is the objective - what's the goal? Seems to me that Angular is still playing catchup.

Oops, I accidentally made our website faster by switching to Remix (https://echobind.com/post/oops-i-accidentally-made-our-website-faster-by-switching-to-remix) by Alex Anderson
Maybe drop frameworks all together to make it even faster!

Building the Same App Using Various Web Frameworks (https://eugeneyan.com/writing/web-frameworks/) by Eugene Yan
Ah another comparison. Seems like this is the main trend for the current week. Spoiler: Performance is not considered in this one.

New Environment API (https://main.vitejs.dev/guide/api-environment) by Vite
With the next iteration of Vite we'll get some exciting new APIs!

11ty is joining Font Awesome (https://www.11ty.dev/blog/eleventy-font-awesome/) by Zach Leatherman
With this partnership the foundation of 11ty is solidified even more. Great work!

Svelte 5 signals fix its glitchy and inconsistent reactivity (https://www.webdevladder.net/blog/svelte-5-signals-fix-its-glitchy-and-inconsistent-reactivity) by Ryan Atkinson
Another reason to switch to Svelte 5 as soon as possible - runes are really wonderful!

Rethinking CSS in JS (https://dev.to/black7375/rethinking-css-in-js-5dip) by @black7375
One of the longer articles at dev.to, but most certainly worth the read.

Tailwind CSS vs Pico CSS (https://edofic.com/posts/2022-01-18-tailwind-vs-pico/) by Andraž Bajt
I am not sure if this is a fair comparison at all, but we are back to "if you need less features, use something with less features".

Issue 55

Understanding Concurrency, Parallelism and JS (https://www.rugu.dev/en/blog/concurrency-and-parallelism/) by Uğur Erdem Seyfi
In the last 15 years (and surely beyond that) this has been discussed every year - but I feel the topic has never been more relevant than today.

MDX in Next.js (https://spacejelly.dev/posts/mdx-in-nextjs) by Colby Fayock
MDX is indeed a great format for content - it's very close to the sweet spot between just static markup and rich interactivity.

Cleaner JavaScript promises with safe-await (https://alexjpate.com/posts/cleaner-promises-with-safe-await) by Alex Pate
I am not sure it that's indeed cleaner (for everyone's taste), but I like it. Note that something similar is debated in TC39.

Announcing TypeScript 5.6 (https://devblogs.microsoft.com/typescript/announcing-typescript-5-6/) by Daniel Rosenwasser
Whoop whoop. Let's all upgrade. Seriously - good new version, but nothing life changing. Back to work!

The web's clipboard, and how it stores data of different types (https://alexharri.com/blog/clipboard) by Alex Harri
Years ago this was a great struggle, but the async clipboard API is indeed quite nice and useful. Great writeup!

CSS display contents (https://ishadeed.com/article/display-contents/) by Ahmad Shadeed
My favorite CSS author is back - and now this picks up one of my favorite CSS features.

Learning to Reason with LLMs (https://openai.com/index/learning-to-reason-with-llms/) by OpenAI
Keep in mind that it's unreasonable to call yourself open but do everything closed. There is an open source variation of this, too.

The Undeniable Utility Of CSS :has (https://www.joshwcomeau.com/css/has/) by Josh Comeau
Honestly, :has is exactly the selected we needed. The number of (previously) "impossible" selectors that I created with it speaks for itself.

Passwords have problems, but passkeys have more (https://world.hey.com/dhh/passwords-have-problems-but-passkeys-have-more-95285df9) by David Heinemeier Hansson
I was also quite excited about passkeys first. But I never use them for the exact reasons that are outlined in the post.

A (more) Modern CSS Reset (https://piccalil.li/blog/a-more-modern-css-reset/) by Andy Bell
I have not used a CSS reset in ages. Might be useful - but just being explicit with some declarations (esp. if inconsistencies between browsers are seen) is usually enough.

Conclusion

These are all outstanding articles by masterful authors. I enjoyed reading them all - I hope you did find something in there, too.

👉 Follow me on LinkedIn, Twitter, or here for more to come.

🙏 Thanks to all the authors and contributors for their hard work!

MDX with Translations 🐠

Florian Rappl — Tue, 03 Sep 2024 15:13:11 +0000

In recent years the need for some easy yet flexible format for simple static content has grown. Originally, HTML was meant for that - but let's be honest: it is neither simple, nor flexible. There have been plenty replacement proposals - as pretty much every CMS comes with its own dialect.

One good contender for static content that can be read and rendered nicely is Markdown. However, it has quite a bit of shortcomings. Most notably, Markdown is not fully specified - leading to multiple dialects that exist. Furthermore, many necessary things are only available in form of non-standard extensions. Finally, using custom components within Markdown is not possible.

This is where MDX comes into play. It's essentially a hybrid of JSX / React and a well-specified dialect of Markdown. Essentially, this allows writing texts that are as simple as plain Markdown, as well as creating complex components as you might have done in full JSX.

Website Structure

We build upon the structure set up in our recently published article faster pages with React and our original blog post from 2019. In this structure we have all the pages placed in their path-relative order using React for each page:

Now there are two things that we want to take care of:

Improve the content writing (and reading) for each employee at smapiot
Simplify translations; right now the structure requires duplicating pages (placing them with different / translated content in the "de" and "en" folders).

For the former we want to leverage MDX - for the latter we need to come up with a solution that allows us to have a flat structure.

Integrating MDX

Bringing MDX into the solution is actually quite straight forward. The @mdx-js/mdx package contains everything to compile and work with MDX files. However, as we are already using a bundler (Vite), we can just rely on the specific plugin that exists for that bundler.

npm i @mdx-js/rollup --save-dev

Why are we installing a package called rollup if our bundler is Vite? It turns out, that Vite is actually just a build-tool - for the actual bundling two other things are used:

esbuild to bundle/process anything in the third-party packages (as well as use it, e.g., on the config file)
rollup to bundle/process the user code

Hence the rollup plugin - which is all we need.

The integration into our vite.config.mjs file looks like this:

import codegen from 'vite-plugin-codegen';
import mdx from '@mdx-js/rollup';
import { resolve } from 'path';

export default {
  build: {
    assetsInlineLimit: 0,
  },
  resolve: {
    alias: {
      '@': resolve(__dirname, 'src'),
    },
  },
  plugins: [
    codegen(),
    mdx(),
  ],
};

Note that we are not providing any options right now. It just works!

So what can we do here? Consider one of our previous pages, the legal disclaimer (who doesn't love legal pages?!):

import * as React from 'react';
import { Page } from '@smapiot/components/lib/basic/Page';

export const meta = {
  title: 'Legal Disclaimer',
  legal: 'Legal Disclaimer',
};

const title = 'smapiot - Legal Disclaimer';

export default () => (
  <Page title={title}>
    <section className="container">
      <h1>Legal Disclaimer</h1>
      <h3>Liability for Contents</h3>
      <p>
        The contents of our pages have been compiled with the greatest care. However, we cannot guarantee for accuracy,
        completeness or topicality of the contents. As service providers, we are liable for own contents of these
        websites according to sec. 7, paragraph 1 German Telemedia Act (TMG). However, according to sec. 8 to 10 German
        Telemedia Act (TMG), service providers are not obligated to permanently monitor submitted or stored information
        or to search for evidences that indicate illegal activities. Legal obligations to removing information or to
        blocking the use of information remain unchallenged. In this case, liability is only possible at the time of
        knowledge about a specific violation of law. Illegal contents will be removed immediately at the time we get
        knowledge of them.
      </p>
      <h3>Liability for Links</h3>
      <p>
        Our web pages include links to external third party websites. We have no influence on the contents of those
        external websites, therefore we cannot guarantee for those contents. Providers or administrators of linked
        websites are always responsible for their own contents. The linked websites had been checked for possible
        violations of law at the time of the establishment of the link. Illegal contents were not detected at the time
        of the linking. A permanent monitoring of the contents of linked websites cannot be imposed without reasonable
        indications that there has been a violation of law. Illegal links will be removed immediately at the time we get
        knowledge of them.
      </p>
      <h3>Copyright</h3>
      <p>
        Contents and compilations published on these websites by the providers are subject to German copyright laws.
        Reproduction, editing, distribution as well as the use of any kind outside the scope of the copyright law
        require a written permission of the author or originator. Downloads and copies of these websites are permitted
        for private use only. The commercial use of our contents without permission of the originator is prohibited.
        Copyright laws of third parties are respected as long as the contents on these websites do not originate from
        the provider. Contributions of third parties on this site are indicated as such. However, if you notice any
        violations of copyright law, please inform us. Such contents will be removed immediately.
      </p>
    </section>
  </Page>
);

This isn't too bad! What we can now do is to rename it from Disclaimer.tsx to Disclaimer.mdx and change its content to become:

export const meta = {
  title: 'Legal Disclaimer',
  legal: 'Legal Disclaimer',
};

<section className="container">
  # Legal Disclaimer

  ### Liability for Contents

  The contents of our pages have been compiled with the greatest care. However, we cannot guarantee for accuracy, completeness or topicality of the contents. As service providers, we are liable for own contents of these websites according to sec. 7, paragraph 1 German Telemedia Act (TMG). However, according to sec. 8 to 10 German Telemedia Act (TMG), service providers are not obligated to permanently monitor submitted or stored information or to search for evidences that indicate illegal activities. Legal obligations to removing information or to blocking the use of information remain unchallenged. In this case, liability is only possible at the time of knowledge about a specific violation of law. Illegal contents will be removed immediately at the time we get knowledge of them.

  ### Liability for Links

  Our web pages include links to external third party websites. We have no influence on the contents of those external websites, therefore we cannot guarantee for those contents. Providers or administrators of linked websites are always responsible for their own contents. The linked websites had been checked for possible violations of law at the time of the establishment of the link. Illegal contents were not detected at the time of the linking. A permanent monitoring of the contents of linked websites cannot be imposed without reasonable indications that there has been a violation of law. Illegal links will be removed immediately at the time we get knowledge of them.

  ### Copyright

  Contents and compilations published on these websites by the providers are subject to German copyright laws. Reproduction, editing, distribution as well as the use of any kind outside the scope of the copyright law require a written permission of the author or originator. Downloads and copies of these websites are permitted for private use only. The commercial use of our contents without permission of the originator is prohibited. Copyright laws of third parties are respected as long as the contents on these websites do not originate from the provider. Contributions of third parties on this site are indicated as such. However, if you notice any violations of copyright law, please inform us. Such contents will be removed immediately.
</section>

Much easier to read! Note that unnecessary imports such as react could be removed, while the boilerplate wrapper component Page will now be automatically inserted.

But there was a second thing we needed, right? What about improvements for the localization / page structure?

Let's see what we can do here!

Providing Translations

What if we would just put the localization strings all in a little sidecar? For instance, for a file like Disclaimer.mdx we'd also have a file Disclaimer.yml. In this file we could place translations according to the languages.

The whole structure could then look like this:

Where an individual file, e.g., for Disclaimer.yml could be written as:

de:
  content$: |
    # Haftungsausschluss

    ### Haftung für Inhalte

    Die Inhalte unserer Seiten wurden mit größter Sorgfalt erstellt. Für die Richtigkeit, Vollständigkeit und Aktualität der Inhalte können wir jedoch keine Gewähr übernehmen. Als Diensteanbieter sind wir gemäß § 7 Abs.1 TMG für eigene Inhalte auf diesen Seiten nach den allgemeinen Gesetzen verantwortlich. Nach §§ 8 bis 10 TMG sind wir als Diensteanbieter jedoch nicht verpflichtet, übermittelte oder gespeicherte fremde Informationen zu überwachen oder nach Umständen zu forschen, die auf eine rechtswidrige Tätigkeit hinweisen. Verpflichtungen zur Entfernung oder Sperrung der Nutzung von Informationen nach den allgemeinen Gesetzen bleiben hiervon unberührt. Eine diesbezügliche Haftung ist jedoch erst ab dem Zeitpunkt der Kenntnis einer konkreten Rechtsverletzung möglich. Bei Bekanntwerden von entsprechenden Rechtsverletzungen werden wir diese Inhalte umgehend entfernen.

    ### Haftung für Links

    Unser Angebot enthält Links zu externen Webseiten Dritter, auf deren Inhalte wir keinen Einfluss haben. Deshalb können wir für diese fremden Inhalte auch keine Gewähr übernehmen. Für die Inhalte der verlinkten Seiten ist stets der jeweilige Anbieter oder Betreiber der Seiten verantwortlich. Die verlinkten Seiten wurden zum Zeitpunkt der Verlinkung auf mögliche Rechtsverstöße überprüft. Rechtswidrige Inhalte waren zum Zeitpunkt der Verlinkung nicht erkennbar. Eine permanente inhaltliche Kontrolle der verlinkten Seiten ist jedoch ohne konkrete Anhaltspunkte einer Rechtsverletzung nicht zumutbar. Bei Bekanntwerden von Rechtsverletzungen werden wir derartige Links umgehend entfernen.

    ### Urheberrecht

    Die durch die Seitenbetreiber erstellten Inhalte und Werke auf diesen Seiten unterliegen dem deutschen Urheberrecht. Die Vervielfältigung, Bearbeitung, Verbreitung und jede Art der Verwertung außerhalb der Grenzen des Urheberrechtes bedürfen der schriftlichen Zustimmung des jeweiligen Autors bzw. Erstellers. Downloads und Kopien dieser Seite sind nur für den privaten, nicht kommerziellen Gebrauch gestattet. Soweit die Inhalte auf dieser Seite nicht vom Betreiber erstellt wurden, werden die Urheberrechte Dritter beachtet. Insbesondere werden Inhalte Dritter als solche gekennzeichnet. Sollten Sie trotzdem auf eine Urheberrechtsverletzung aufmerksam werden, bitten wir um einen entsprechenden Hinweis. Bei Bekanntwerden von Rechtsverletzungen werden wir derartige Inhalte umgehend entfernen.
en:
  content$: |
    # Legal Disclaimer

    ### Liability for Contents

    The contents of our pages have been compiled with the greatest care. However, we cannot guarantee for accuracy, completeness or topicality of the contents. As service providers, we are liable for own contents of these websites according to sec. 7, paragraph 1 German Telemedia Act (TMG). However, according to sec. 8 to 10 German Telemedia Act (TMG), service providers are not obligated to permanently monitor submitted or stored information or to search for evidences that indicate illegal activities. Legal obligations to removing information or to blocking the use of information remain unchallenged. In this case, liability is only possible at the time of knowledge about a specific violation of law. Illegal contents will be removed immediately at the time we get knowledge of them.

    ### Liability for Links

    Our web pages include links to external third party websites. We have no influence on the contents of those external websites, therefore we cannot guarantee for those contents. Providers or administrators of linked websites are always responsible for their own contents. The linked websites had been checked for possible violations of law at the time of the establishment of the link. Illegal contents were not detected at the time of the linking. A permanent monitoring of the contents of linked websites cannot be imposed without reasonable indications that there has been a violation of law. Illegal links will be removed immediately at the time we get knowledge of them.

    ### Copyright

    Contents and compilations published on these websites by the providers are subject to German copyright laws. Reproduction, editing, distribution as well as the use of any kind outside the scope of the copyright law require a written permission of the author or originator. Downloads and copies of these websites are permitted for private use only. The commercial use of our contents without permission of the originator is prohibited. Copyright laws of third parties are respected as long as the contents on these websites do not originate from the provider. Contributions of third parties on this site are indicated as such. However, if you notice any violations of copyright law, please inform us. Such contents will be removed immediately.

For this file a set of conventions exist:

The top-level keys must be valid languages (right now only de and en)
The keys can be nested, i.e., objects / arrays are allowed
When a key ends with a $ sign it will be interpreted as Markdown (specifically, MDX)

How does the MDX file look with such a sidecar file?

export const meta = {
  title: 'Legal Disclaimer',
  legal: 'Legal Disclaimer',
};

<section className="container">
  {locale.content$}
</section>

Now this looks appealing!

For integrating the localization we still need to do something. See the use of locale in the code above. But how does locale enter the landscape in the file? After all, we don't see it!

Turns out we can manipulate the generated code by MDX. Let's configure the tool a bit by adding a custom plugin to the recmaPlugins option. This allows us to have full control over the generated abstract syntax tree (AST), which is then used to generate the actual code representing the MDX file.

import codegen from 'vite-plugin-codegen';
import mdx from '@mdx-js/rollup';
import { resolve } from 'path';
import localize from './src/tools/localize.mjs';

export default {
  build: {
    assetsInlineLimit: 0,
  },
  resolve: {
    alias: {
      '@': resolve(__dirname, 'src'),
    },
  },
  plugins: [
    codegen(),
    mdx({
      recmaPlugins: [localize],
    }),
  ],
};

The imported localize function is the actual plugin - which just yields another function - the so-called transformer. The transformer is responsible for manipulating the AST.

The transform function in the next snippet receives the AST and the virtual file (i.e., it's name, content, ...) of the MDX file that is currently being handled.

Let's see the code before we go over what it does:

import { compileSync } from '@mdx-js/mdx';
import { loadLocale } from './localization.mjs';

function fromMarkdown(content) {
  let result = {};
  compileSync(content, {
    development: false,
    recmaPlugins: [
      () => (ast) => {
        const [, fn] = ast.body;
        result = {
          type: 'CallExpression',
          optional: false,
          callee: {
            ...fn,
            id: null,
          },
          arguments: [
            {
              type: 'ObjectExpression',
              properties: [],
            },
          ],
        };
      },
    ],
  });

  return result;
}

function getExpression(content, key) {
  switch (typeof content) {
    case 'object':
      if (Array.isArray(content)) {
        return {
          type: 'ArrayExpression',
          elements: content.map((item, i) => getExpression(item, `${i}`)),
        };
      }

      return {
        type: 'ObjectExpression',
        properties: Object.entries(content).map(([name, value]) => ({
          type: 'Property',
          key: {
            type: 'Identifier',
            name,
          },
          value: getExpression(value, name),
          kind: 'init',
        })),
      };
    case 'number':
    case 'boolean':
      return {
        type: 'Literal',
        value: content,
      };
    case 'string':
      if (key.endsWith('$')) {
        // uses potentially markdown
        return fromMarkdown(content);
      }

      return {
        type: 'Literal',
        value: content,
      };
    case 'undefined':
    default:
      return {
        type: 'Literal',
        value: null,
      };
  }
}

async function transform(ast, vfile) {
  const language = process.env.WEBSITE_LOCALE || 'en';

  const source = vfile.history[0];
  const locale = await loadLocale(source, language);

  const idx = ast.body.findLastIndex((m) => m.type === 'ImportDeclaration');
  const imprt = ast.body.find((node) => node.type === 'ImportDeclaration' && node.source.value === 'react/jsx-runtime');

  const requiredImports = [
    ['_jsx', 'jsx'],
    ['_jsxs', 'jsxs'],
    ['_Fragment', 'Fragment'],
  ];

  for (const requiredImport of requiredImports) {
    const [alias, original] = requiredImport;

    if (!imprt.specifiers.find((node) => node.type === 'ImportSpecifier' && node.imported.name === original)) {
      imprt.specifiers.push({
        type: 'ImportSpecifier',
        imported: {
          type: 'Identifier',
          name: original,
        },
        local: {
          type: 'Identifier',
          name: alias,
        },
      });
    }
  }

  ast.body.splice(idx + 1, 0, {
    type: 'VariableDeclaration',
    kind: 'const',
    declarations: [
      {
        type: 'VariableDeclarator',
        id: {
          type: 'Identifier',
          name: 'locale',
        },
        init: getExpression(locale, language),
      },
    ],
  });
}

const plugin = () => transform;

export default plugin;

Looks more complicated than it needs to be! In the end, it all boils down to changing the AST to

obtain the locale for the document
insert potentially missing named imports (to fully support the generated Markdown / content, if any) into the react/jsx-runtime import
insert the locale declaration - first thing after the imports (this way you can use locale pretty much everywhere in the document
initialize the locale variable to be the object we know; with exception of Markdown strings (suffixed with $): these are transformed into an IIFE keeping the original MDX code

A good way to see what we are doing can be observed by using the MDX playground.

As you can see the compiled code is a bit strange, but can be mapped nicely to our original MDX file.

In contrast, the view that we are actually most interested about is the stage before the code is in that compiled state. It's the "esast" view in the playground (i.e., the AST of the ESTree stage).

What we want to achieve with our plugin is that we manipulate the start. Unfortunately, directly in MDX code we'd need to use an export for that, but by manipulating the AST directly we actually don't need it.

To obtain the locale variable / object we the loadLocale function from a dedicated module.

This module does:

Read and parse the local / file specific translations
Obtain the language-specific translations of the local file
Read and parse the global translations
Obtain the language-specific translations of the global file
Merge with local translations being regarded higher

For obtaining the language-specific translations we choose a base translation (in our case en) and "walk" through the object. The current language (let's say de) then either overrides the base translation or just uses the snippet obtained from the base translation.

You can think of this process as a deep merge with the base translation being the original.

import { readFile } from 'fs/promises';
import { resolve } from 'path';
import { parse } from 'yaml';

async function getLocalization(path) {
  try {
    const content = await readFile(path, 'utf8');
    return parse(content);
  } catch (e) {
    console.error('Error reading YAML file:', e);
    return {};
  }
}

function mergeLocale(result, baseLocale, newLocale) {
  if (!newLocale) {
    Object.assign(result, baseLocale);
  } else {
    Object.entries(baseLocale).forEach(([name, value]) => {
      const c = newLocale[name];

      if (typeof value === 'object') {
        mergeLocale((result[name] = {}), value, c || {});
      } else if (typeof c === 'string' || c) {
        result[name] = c;
      } else {
        result[name] = value;
      }
    });
  }

  return result;
}

function getLocale(locales, lang) {
  const baseLocale = locales.en || {};

  if (lang !== 'en' && lang in locales) {
    const newLocale = locales[lang] || {};
    return mergeLocale({}, baseLocale, newLocale);
  }

  return baseLocale;
}

export async function loadLocale(source, language) {
  if (source.endsWith('.mdx')) {
    const globalFn = resolve(import.meta.dirname, '..', 'global.yml');
    const targetFn = source.replace('.mdx', '.yml');

    const globals = await getLocalization(globalFn);
    const locales = await getLocalization(targetFn);

    return {
      ...getLocale(globals, language),
      ...getLocale(locales, language),
      language,
    };
  }

  return { language };
}

The global translations are placed in the folder above the pages. The name of the file is global.yml.

For the global.yml we only have some very common translation terms stored. An example:

de:
  germany: Deutschland
en:
  germany: Germany

Being able to just change a single file when new translations come (or just changing the structure in one place independent of language changes) is a big deal. For us this was quite important and the new structure reflects that.

Conclusion

With the new approach the pages are much easier to write than beforehand. Also, since we now have a static translation system in place we do not need to duplicate the pages or come up with some complicated system for intermediate components. It's all already done at compile-time in the given structure.

Faster Pages with React ✨

Florian Rappl — Fri, 30 Aug 2024 08:17:31 +0000

Photo by Mathew Schwartz on Unsplash

This is the story of our homepage make over to reach a solid lighthouse score. If you want to read how we made the homepage in the first place then have a look at our blog post from 2019.

Many of the things I write about here can be seen in the Fast Pages with React article.

The Origins

Our homepage started off as a React SPA. Why? It was 2019 and the UI designer created the components all in React. Plus, it was a super smooth experience that was actually quite acceptable.

However, while some may argue that a SPA is not good for SEO this was not the reason why we wanted to have a pre-rendered page. SEO seemed actually fine, but some more simple crawlers did not find all the information as it was hidden behind some JavaScript.

So what can we do here? Well, we can just render everything at build-time using an static-site generation (SSG) approach.

Going into SSG

We did not want to change the overall content or engine behind the page. After all, the DX was great and we thought that reusability of the components is more important than migrating to the latest SSG framework.

After the build (back then using the Parcel v1 bundler) was done, we had a post-build process that took the generated index.html and went through all the detected pages. As our routing is declarative (we already generated the routes from the file system paths) finding the pages was easy.

For each page we ran simple script that does the following things:

Teach Node.js how ESM works (esm module)
Allow using TypeScript via ts-node (our source is using TypeScript)
Add some globals like document or localStorage
Register some additional extensions, e.g., for resolving images as modules in Node.js (these images should resolve to the already generated images from the bundler)
Evaluate the page - using renderToString
Replace the app's content container with the pre-rendered page
Save the HTML of the modified app in a new file that matches the path of the page

In code this worked as follows:



const { readFileSync, writeFileSync, mkdirSync } = require('fs');
const { basename, dirname, resolve } = require('path');

require = require('esm')(module);

require('ts-node').register({
  compilerOptions: {
    module: 'commonjs',
    target: 'es6',
    jsx: 'react',
    importHelpers: true,
    moduleResolution: 'node',
  },
  transpileOnly: true,
});

global.XMLHttpRequest = class {};
global.XDomainRequest = class {};
global.localStorage = {
  getItem() {
    return undefined;
  },
  setItem() {},
};
global.document = {
  title: 'sample',
  querySelector() {
    return {
      getAttribute() {
        return '';
      },
    };
  },
};

const React = require('react');
const { MemoryRouter } = require('react-router');
const { renderToString } = require('react-dom/server');

React.lazy = () => () => React.createElement('div', undefined, 'Loading ...');
React.Suspense = ({ children }) => React.createElement(React.Fragment, undefined, children);
React.useLayoutEffect = () => {};

function setupExtensions(files) {
  ['.png', '.svg', '.jpg', '.jpeg', '.mp4', '.mp3', '.woff', '.tiff', '.tif', '.xml'].forEach(extension => {
    require.extensions[extension] = (module, file) => {
      const parts = basename(file).split('.');
      const ext = parts.pop();
      const front = parts.join('.');
      const ref = files.filter(m => m.startsWith(front) && m.endsWith(ext)).pop() || '';
      module.exports = '/' + ref;
    };
  });

  require.extensions['.codegen'] = (module, file) => {
    const content = readFileSync(file, 'utf8');
    module._compile(content, file);
    const code = module.exports();
    module._compile(code, file);
  };
}

function renderApp(source, target, dist) {
  const sourceModule = require(source);
  const route = (sourceModule.meta.route || target).substring(1);
  const Page = sourceModule.default;
  const Layout = require('../../scripts/layout').default;
  const element = React.createElement(
    MemoryRouter,
    undefined,
    React.createElement(Layout, undefined, React.createElement(Page)),
  );
  return {
    content: renderToString(element),
    outPath: resolve(dist, route, 'index.html'),
  };
}

function makePage(outPath, html, content) {
  const outDir = dirname(outPath);
  const file = html.replace(/<div id="app">(.*)<\/div>/, `<div id="app">${content}</div>`);

  mkdirSync(outDir, {
    recursive: true,
  });

  writeFileSync(outPath, file, 'utf8');
}

process.on('message', msg => {
  const { source, target, files, html, dist } = msg;
  setupExtensions(files);

  setTimeout(() => {
    const { content, outPath } = renderApp(source, target, dist);
    makePage(outPath, html, content);

    process.send({
      content,
    });
  }, 100);
});

The whole process is used as the given module is called from a forked process. So each page is generated in an isolated process.

The following diagram illustrates this process.

What did we win so far? We have a pre-rendered page that works already and transforms to a SPA. Nice. But not enough for us.

Modernizing the Stack

I already described the process as "was". Parcel v1 has not been updated in quite a while and is - from today's point of view - quite slow. It also does not support some modern concepts and should be retired for good.

As a replacement we've chosen Vite. This is a suitable replacement as it comes with a super fast development server and a very well optimized release build.

Also, since we are using codegen for the route retrieval we can still keep using it for Vite.

After all, the whole configuration for Vite to allow the transition is shown below:



import codegen from 'vite-plugin-codegen';
import { resolve } from 'path';

export default {
  build: {
    assetsInlineLimit: 0,
  },
  resolve: {
    alias: {
      '@': resolve(__dirname, 'src'),
    },
  },
  plugins: [codegen()],
};

To avoid Vite's original behavior of inlining smaller assets (which would lead to problems in our SSG behavior) we set the assetsInlineLimit to 0.

One thing we also improved when making the transition from Parcel v1 to Vite is that we introduced path aliases. With the configuration above we can write import '@/foo/bar' instead of going explicitly via the relative path to src/foo/bar. This makes modules more flexible and easier to maintain.

Previously, we used a bit of tools (esm, ts-node, ...) to actually make the SSG happen. With the new setup it was time to reduce the amount of tools and replace them all with esbuild.

The updated code of the SSG kernel module therefore changed a bit, too:



const { writeFile, readFile, mkdir } = require('fs/promises');
const { dirname, resolve, basename, relative } = require('path');
const { createContext, runInContext } = require('vm');
const { compile } = require('./compile');

const React = require('react');
const ReactRouter = require('react-router');
const ReactRouterDom = require('react-router-dom');
const { renderToString } = require('react-dom/server');

React.lazy = () => () => React.createElement('div', undefined, 'Loading ...');
React.Suspense = ({ children }) => React.createElement(React.Fragment, undefined, children);
React.useLayoutEffect = () => {};
React.useEffect = () => {};

async function renderApp(source, target, language, files, dist) {
  const result = await compile({
    dist,
    files,
    target,
    platform: 'node',
    stdin: {
      contents: `
        import { MemoryRouter } from "react-router";
        import Page, { meta } from "./${basename(source)}";
        import Layout from "${relative(dirname(source), resolve(__dirname, '../../layouts/default'))}";

        const page = (
          <MemoryRouter>
            <Layout language="${language}">
              <Page />
            </Layout>
          </MemoryRouter>
        );

        export { page, meta };
      `,
      sourcefile: resolve(dirname(source), 'temp-page.jsx'),
      resolveDir: dirname(source),
      loader: 'jsx',
    },
  });
  const module = {
    exports: {},
  };
  const ctx = createContext({
    exports: module.exports,
    require(name) {
      switch (name) {
        case 'react':
          return React;
        case 'react-router':
          return ReactRouter;
        case 'react-router-dom':
          return ReactRouterDom;
        default:
          console.error('Cannot require', name);
          return undefined;
      }
    },
    setTimeout() {},
    setInterval() {},
    React,
    module,
    XMLHttpRequest: class {},
    XDomainRequest: class {},
    localStorage: {
      getItem() {
        return undefined;
      },
      setItem() {},
    },
    document: {
      title: 'sample',
      querySelector() {
        return {
          getAttribute() {
            return '';
          },
        };
      },
    },
  });
  const code = result.outputFiles.find((m) => m.path.endsWith('.js')).text;

  runInContext(code, ctx);
  const { page, meta } = module.exports;

  return {
    content: renderToString(page),
    meta,
  };
}

async function makeFile(outPath, content) {
  const outDir = dirname(outPath);

  await mkdir(outDir, {
    recursive: true,
  });

  await writeFile(outPath, content, 'utf8');
}

function makePage(outPath, meta, html, content) {
  return makeFile(outPath, html.replace(/<div id="app">.*?<\/div>/s, `<div id="app">${content}</div>`);
}

process.on('message', (msg) => {
  const { source, target, files, language, html, dist } = msg;

  setTimeout(async () => {
    const { content, replacements, meta } = await renderApp(source, target, language, files, dist, html);
    const route = (meta.route || target).substring(1);
    const outPath = resolve(dist, route, 'index.html');
    await makePage(outPath, meta, html, content);

    process.send({
      done: true,
      outPath,
      replacements,
    });
  }, 100);
});

While the overall flow has remained the same, we now convert the page with some imports (e.g., MemoryRouter) to a plain JavaScript script using the CommonJS module system. Therefore, the evaluation with the Node.js vm module does not require esm or ts-node any more. Everything has been taken care of by esbuild.

In the code above the actual esbuild usage is hidden in the compile function, so let's see how we set it up:



const { build } = require('esbuild');
const { codegenPlugin } = require('esbuild-codegen-plugin');
const { resolve, basename } = require('path');

function compile(opts) {
  const { dist, target, stdin, files, platform, entryPoints } = opts;
  const isBrowser = platform === 'browser';
  return build({
    stdin,
    entryPoints,
    outdir: resolve(dist, target.substring(1)),
    write: false,
    bundle: true,
    splitting: isBrowser,
    minify: isBrowser,
    format: !isBrowser ? 'cjs' : 'esm',
    platform,
    loader: {
      '.jpg': 'file',
      '.png': 'file',
      '.svg': 'file',
      '.avif': 'file',
      '.webp': 'file',
    },
    alias: {
      '@': resolve(__dirname, '../..'),
    },
    external: ['react', 'react-router', 'react-router-dom', 'react-dom', 'react-dom/client'],
    plugins: [
      codegenPlugin(),
      {
        name: 'dynamic-assets',
        setup(build) {
          build.onResolve({ filter: /.*/ }, (args) => {
            const name = basename(args.path);
            const idx = name.lastIndexOf('.');
            const front = name.substring(0, idx);
            const ext = name.substring(idx);
            const prefix = front + '-';
            const file = files.find((m) => m.startsWith(prefix) && m.endsWith(ext));

            if (file) {
              return {
                path: file,
                namespace: 'dynamic-asset',
              };
            }

            return undefined;
          });

          build.onLoad({ namespace: 'dynamic-asset', filter: /.*/ }, (args) => {
            const path = `/assets/${args.path}`;
            return {
              contents: `export default ${JSON.stringify(path)};`,
              loader: 'js',
            };
          });
        },
      },
    ],
  });
}

exports.compile = compile;

Quite straight forward. The few things to note are:

Again, the usage of codegen is superb is there is literally a plugin for each bundler. So we can just use the codegen plugin for esbuild and we are covered here.
The dynamic asset plugin that has been defined above looks within the files if the name and extension fits; if it does it will use the already generated one from Vite.
We can reuse the compile function not only for Node.js (the SSG part), but also for compiling some JS functions that run in the browser.

Especially the last point can be crucial. Right now this setup really moves the dynamic part (a SPA) to a fully static one. But we still have some interactive parts in there... So fully hydrating this thing - as we did beforehand - may not be good enough.

But even with this more modern setup one thing was still from 2019: the overall performance.

Lighthouse Issues

In 2019 we still got a great Lighthouse score, but now it's mediocre at best. Lighthouse got a lot more pushy - especially regarding hydration scenarios as we have here.

The score summary is:

Category	Desktop	Mobile
Score	70	50
Best Practices	81	79
First Contentful Paint	0.4 s	1.7s
Largest Contentful Paint	1.6 s	8.4s
Total Blocking Time	0 ms	20ms
Cumulative Layout Shift	1.828	1.899
Speed Index	0.7s	2.1s

There are some findings that seem rather optional, but nevertheless could be implemented rather quickly (or not at all):

Easy: No CSP header (as this is just served as static content we could embed it easily in the HTML as meta tag)
Tedious: On mobile some images are served with a lower resolution (presumably going into the picture tag with different source elements to return the "right" resolution would fit)
Impossible: Deprecated API used (unload handler; use pagehide event instead - actually we don't use this: the problem is coming/injected from a Chrome extension)

Naturally, the question is why the score is so low - and what we can do about it. Notably, we have a CLS (Cumulative Layout Shift) and a quite large LCP (Largest Contentful Paint).

Improvements

The first thing to do is to lower the amount of work that the JS engine has to do. For this we needed to rethink our hydration strategy. The standard one from a React SPA is to hydrate everything. But this comes with a lot of overhead and a quite slow start.

A much better approach would be to include an island architecture style here. Instead of hydrating everything in the beginning, we only hydrate parts - and only hydrate these parts when we need them (e.g., when they become visible).

How can we do that? Time for magic.

Let's consider the following code:



const Testimonials: React.FC = () => (
  <>
    <div className="container">
      <h1>Testimonials</h1>
    </div>
    <div className="quote-carousel">
      <TestimonialsSlides />
    </div>
  </>
);

This is the testimonials part of the homepage. Everything should be static - except the TestimonialsSlides. This is a Carousel with some content. Everything is defined in React.

What if we could just tell the system to hydrate it? Let's imagine we rewrite the code to look like this:



const Testimonials: React.FC = () => (
  <>
    <div className="container">
      <h1>Testimonials</h1>
    </div>
    <div className="quote-carousel" data-hydrate={TestimonialsSlides}>
      <TestimonialsSlides />
    </div>
  </>
);

The only thing we changed is that we added a data-hydrate attribute. Granted, this should be a string - but (surprise) we won't use the attribute anyway. We will actually change the attribute value at build-time.

The architecture that we have in mind works as follows:

The build-time change in the SSG kernel module is an addition for the replacements:



const { createHash } = require('crypto');

async function getUniqueName(path) {
  const fn = basename(path);
  const name = fn.substring(0, fn.lastIndexOf('.'));
  const content = await readFile(path);
  const value = createHash('sha1').update(content);
  const hash = value.digest('hex').substring(0, 6);
  return `${name}.${hash}`;
}

const matcher = /"data-(hydrate|render|load)":\s+(\w+)[,\s]/g;
const replacements = [];

while (true) {
  const match = matcher.exec(code);

  if (!match) {
    break;
  }

  const lookup = match[0];
  const kind = match[1];
  const componentName = match[2];
  const pos = code.indexOf(`var ${componentName}`);
  const idx = code.lastIndexOf('\n// ', pos) + 4;
  const src = code.substring(idx, code.indexOf('\n', idx));
  const entry = resolve(__dirname, '../../..', src);
  const name = await getUniqueName(entry);
  replacements.push({
    lookup,
    entry,
    name,
    value: `"data-${kind}": "/assets/${name}.js",`,
  });
}

for (const { lookup, value } of replacements) {
  code = code.replace(lookup, value);
}

Nice! With this change we have a replacement of the attributes during the build process. We also identify the replacements to use them later on in one joint build process (such that they produce common chunks):



const { writeFile } = require('fs/promises');
const { basename, resolve } = require('path');
const { compile } = require('./compile');

process.on('message', (msg) => {
  const { replacements, dist, files, target } = msg;

  setTimeout(async () => {
    const result = await compile({
      dist,
      files,
      target,
      platform: 'browser',
      entryPoints: replacements.map((m) => ({ in: m.entry, out: m.name })),
    });

    for (const file of result.outputFiles) {
      const name = basename(file.path);
      const content = file.text;
      await writeFile(resolve(dist, target.substring(1), name), content, 'utf8');
    }

    process.send({
      done: true,
    });
  }, 100);
});

The only thing missing is a replacement for the SPA script. The replacement should be able to work with the data-hydrate attribute, which tells the website to hydrate a container with a specific file:



function integrate() {
  function getProps(element: Element) {
    try {
      return JSON.parse(element.getAttribute('data-props'));
    } catch {
      return {};
    }
  }

  function load(fn: string) {
    const react = import('react');
    const reactDom = import('react-dom/client');
    const mod = import(fn);
    return Promise.all([react, reactDom, mod]);
  }

  document.querySelectorAll('*[data-hydrate]').forEach((element) => {
    const fn = element.getAttribute('data-hydrate');
    const observer = new IntersectionObserver((entries) => {
      entries.forEach((entry) => {
        if (entry.isIntersecting) {
          observer.disconnect();
          load(fn).then(([{ createElement }, { hydrateRoot }, m]) => {
            requestAnimationFrame(() => hydrateRoot(element, createElement(m.default, getProps(element))));
          });
        }
      });
    });

    observer.observe(element);
  });

  document.querySelectorAll('*[data-load]').forEach((element) => {
    const fn = element.getAttribute('data-load');
    load(fn).then(([{ createElement }, { hydrateRoot }, m]) => {
      requestIdleCallback(() => hydrateRoot(element, createElement(m.default, getProps(element))));
    });
  });

  document.querySelectorAll('*[data-render]').forEach((element) => {
    const fn = element.getAttribute('data-render');
    load(fn).then(([{ createElement }, { createRoot }, m]) => {
      createRoot(element).render(createElement(m.default, getProps(element)));
    });
  });
}

if (document.readyState === 'loading') {
  document.addEventListener('DOMContentLoaded', integrate);
} else {
  integrate();
}

That is pretty much it! The great thing is that this script is so small that it can be inlined. Since we also know when / if we need replacements we'd only inline the script when we would need it (i.e., when we might hydrate).

Finally, we externalized react (and others), but we did not specify a replacement. As esbuild produces esm files those externals are placed as standard imports such as:



import * as React from 'react';

We would run into problems if such a thing is evaluated in the browser. The solution is to introduce an importmap in our page:



<script type="importmap">
{
  "imports": {
    "react": "https://esm.sh/react@18.3.1",
    "react-dom": "https://esm.sh/react-dom@18.3.1",
    "react-dom/client": "https://esm.sh/react-dom@18.3.1/client"
  }
}
</script>

Note that the importmap is lazyly loaded anyway. So it will perform optimally out of the box.

With everything in place we can look again at the lighthouse score.

The score summary is:

Category	Desktop	Mobile
Score	78	65
Best Practices	81	79
First Contentful Paint	0.6 s	4.3s
Largest Contentful Paint	1.3 s	9.4s
Total Blocking Time	0 ms	10ms
Cumulative Layout Shift	0	0
Speed Index	0.6s	4.3s

While the score is already much better the LCP and overall speed index actually got worse. The reason the score is better is most notably that the CLS is now 0, which is great.

So where did we go wrong?

Final Touches

While the SPA transition hurt our score, it did not hurt as much as badly hydrated components. One of these components is the main carousel, which is shown immediately.

It turns out the carousel component was only made for SPAs. It dynamically computes the width and position of its slides based on the given content. The following diagram shows its working:

In short we have a container that gets the dimensions of the component. Inside the container we use a content element that has the width of the container multiplied by the number of slides + 2. We add two as we also introduce duplicates for the first and last slide. This way, we can have the experience of a real carousel. From the last slide you can keep on swiping right to get to the first slide and vice versa. Essentially, this allows you to keep scrolling in one direction.

What can we do to improve the code here?

Instead of relying on the runtime to set style properties we return a pre-computed style object. This way, we only change dynamic properties when a scrol operation starts.

Originally, the code had a function like:



const updateOffset = () => {
  const c = container.current?.parentElement;

  if (c) {
    const o = offset.current;
    let transform = 'translateX(0)';
    let transition = 'none';

    if (state.desired !== state.active) {
      const dist = Math.abs(state.active - state.desired);
      const pref = Math.sign(o || 0);
      const dir = (dist > length / 2 ? 1 : -1) * Math.sign(state.desired - state.active);
      const shift = (100 * (pref || dir)) / (length + 2);
      transition = smooth;
      transform = `translateX(${shift}%)`;
    } else if (!isNaN(o)) {
      if (o !== 0) {
        transform = `translateX(${o}px)`;
      } else {
        transition = elastic;
      }
    }

    c.style.transform = transform;
    c.style.transition = transition;
    c.style.left = `-${(state.active + 1) * 100}%`;
  }
};

Now we move to:



const updateOffset = () => {
  const c = container.current;

  if (c) {
    const o = offset.current;
    let transform = 'translateX(0)';
    let transition = 'none';

    if (state.desired !== state.active) {
      const shift = getShift(o, state.active, state.desired, length + 2);
      transition = smooth;
      transform = `translateX(${shift}%)`;
    } else if (!isNaN(o)) {
      if (o !== 0) {
        transform = `translateX(${o}px)`;
      } else {
        transition = elastic;
      }
    }

    c.style.transform = transform;
    c.style.transition = transition;
  }
};

The new getShift function is also much improved:



function getShift(o: number, active: number, desired: number, total: number) {
  if (!o) {
    const end = total - 3;

    if (!desired && active === end) {
      o = -1;
    } else if (desired === end && !active) {
      o = 1;
    }
  }

  if (o) {
    const pref = Math.sign(o);
    return (100 * pref) / total;
  } else {
    const diff = active - desired;
    return (100 * diff) / total;
  }
}

This results in a better UX while also improving the story for SSG / SSR.

Besides the work on the carousel component we also optimized the images and spend some time on content cleanup. Nothing dramatic - just little wins here and there.

The score summary is:

Category	Desktop	Mobile
Score	99	87
Best Practices	78	75
First Contentful Paint	0.6 s	2.6s
Largest Contentful Paint	0.8 s	3.4s
Total Blocking Time	0 ms	10ms
Cumulative Layout Shift	0.009	0
Speed Index	0.6s	2.6s

This is it! Wonderful that such a "little" change in a single component can have such a drastic outcome. But in the end it's all about the details.

While every metric now looks good the best practices got a bit worse. Why? Because we added a small video in form of a YouTube embed (iframe) into the homepage. As YouTube brings quite some cookies and other things Lighthouse is not really happy with the third-party content. So we can actually ignore this.

As wanted to be in the 90s rating this is certainly a success and as desired.

Conclusion

Overall, we kept the codebase stable and just gave the whole source code a bit of a makeover. This was sufficient to reach the next level of DX and performance.

As next step we'll modernize some of our content and re-introduce the SPA transition by hijacking internal links; loading an HTML fragment during a page transition. We'll potentially also replace the importmap with an integrated bundle, using Preact in compat mode as a replacement.

🧠 Don't Miss These 50 Articles

Florian Rappl — Mon, 26 Aug 2024 07:54:56 +0000

Image by freepik

I hope you enjoy this collection. Let me know in the comments which of these articles is your favorite (and why).

Issue 46

Garbage collection and closures (https://jakearchibald.com/2024/garbage-collection-and-closures/) by Jake Archibald
Weak references for the win. Honestly, finding memory leaks is always tricky - even you find the source fast; getting rid of it can be complex.

A Look at What's Coming to PHP 8.4 (https://laravel-news.com/php-8-4-0) by Paul Redmond
The last thing I'd do is bet against PHP.

This month in Servo: console logging, parallel tables, OpenXR, and more! (https://servo.org/blog/2024/07/31/this-month-in-servo/) by Servo
Servo is one of the main reasons Rust exists today - a browser engine that is fast and transportable. It's great to see them back and continuing to do well!

How to Get the Width/Height of Any Element in Only CSS (https://frontendmasters.com/blog/how-to-get-the-width-height-of-any-element-in-only-css/) by Temani Afif
This is getting crazy. Previously this was thought as impossible as CSS is used to compute those values, now that Chromium has full support we are most likely to see it across all browsers.

Styling Tables the Modern CSS Way (https://piccalil.li/blog/styling-tables-the-modern-css-way/) by Michelle Barker
Every once in a while somebody reminds me that tables are still a thing. They are - and styling them properly is actually required to make full use of them.

Web Styles in Native Apps (https://www.telerik.com/blogs/web-styles-native-apps) by Sam Basu
Was not really sure what to expect in here, but I got happy once I saw XAML. Quite cool article if you work in the Blazor / MAUI space.

2024 Developer Survey Results (https://survey.stackoverflow.co/2024) by StackOverflow
Pretty much no surprises here. However, that reminds me that we may be should start another micro frontend survey?! It's about time...

Battle of the Asyncs (https://www.brenelz.com/posts/battle-of-the-asyncs/) by Brenley Dueck
I missed the battle in this one, but generally a good comparison of the different philosophies behind Solid and React. There is more to it, but this is just a solid point.

Please Stop Using Barrel Files (https://tkdodo.eu/blog/please-stop-using-barrel-files) by Dominik Dorfmeister
I agree up to some degree. In general if you use barrel files you should also treat the folders / barrels like its own library; i.e., think of it as a dependency and always visualize how the dependency graph flows.

What we (Deno) got wrong about HTTP imports (https://deno.com/blog/http-imports) by Ryan Dahl
One of the longer ads for JSR. While I am still positive about the JSR I also see some drawbacks.

Issue 47

Plain Vanilla (https://plainvanillaweb.com/) by Joeri Sebrechts
A nice collection of starters and ideas for going fully vanilla. Be aware that following this you might end up with your own framework.

A different way to think about typescript (https://www.rob.directory/blog/a-different-way-to-think-about-typescript) by Robby Pruzan
TypeScript is like a strict librarian who insists on organizing infinite sets of values, just so you can avoid typos in your code!

How Google handles JavaScript throughout the indexing process (https://vercel.com/blog/how-google-handles-javascript-throughout-the-indexing-process) by Ryan Siddle and Malte Ubl
I'm glad this got out. I'm tired of telling everyone that you don't need SSR to be seen by Google.

Patterns for Memory Efficient DOM Manipulation with Modern Vanilla JavaScript (https://frontendmasters.com/blog/patterns-for-memory-efficient-dom-manipulation/) by Marc Grabanski
Spot the pattern? All vanilla. Knowing your DOM API is key.

Benchmarking AWS Lambda Cold Starts Across JavaScript Runtimes (https://deno.com/blog/aws-lambda-coldstart-benchmarks) by Andy Jiang and Igor Zinkovsky
Alright, so Deno is actually quite fast - but what's more important is that Hono is super (even more than Express) lightweight.

Nx 19.5 is here! Stackblitz, Bun, Incremental Builds for Vite, Gradle Test Atomizer (https://nx.dev/blog/nx-19-5-adds-stackblitz-new-features-and-more) by Zack DeRose
Well that's a nice feature set! Incremental builds are super slick, so very happy with this one.

CSS Spring Easing Generator (https://www.kvin.me/css-springs) by Kevin Grajeda
The missing piece for crucial CSS snippet tools collection. Wait...

Common Sense Refactoring of a Messy React Component (https://alexkondov.com/refactoring-a-messy-react-component/) by Alex Kondov
Hey, I'm still missing the final outcome! Spoiler: Instead of one large file we now have 3 new dependencies and 15 files. Just kidding!

An Alpine & Tailwind UI Library (https://devdojo.com/pines) by Tony Lea et al.
Looks quite nice, but I wonder if copying over so much code is really efficient if you want to reuse these components. For single use-cases I think this is wonderful.

React Compiler, How Does It Work? 4 - SSA Transformation (https://yongseok.me/blog/en/react_compiler_4/) by YongSeok Jang
The techniques presented in the article also play a role in many other parsers / compilers / transpilers. Super cool!

Issue 48

Announcing Official Puppeteer Support for Firefox (https://hacks.mozilla.org/2024/08/puppeteer-support-for-firefox/) by James Graham, Henrik Skupin, Julian Descottes, Alexandra Borovova
That was about time - quite happy that's here. Still, I'll keep on using Playwright.

A tiny utility to build JSON schema types (https://github.com/lukeed/tschema) by Luke Edwards
Super useful - well done! Finally I only need to define it once.

Effect Best Practices (https://ethanniser.dev/blog/effect-best-practices/) by Ethan Niser
Using the useful Effect utility lib? This post should be on your radar.

Introducing XState Store (https://tkdodo.eu/blog/introducing-x-state-store) by Dominik Dorfmeister
Dominik keeps on delivering great content! It's half the size of Zustand yet seems very comparable feature-wise.

Gradient Generator (https://www.learnui.design/tools/gradient-generator.html) by Kennedy Design
Last time we had an awesome easing generator - this time it's about gradients.

Integrating Blazor with Existing .NET Web Apps (https://visualstudiomagazine.com/Articles/2024/08/07/Integrating-Blazor-with-Existing-,-d-,NET-Web-Apps.aspx) by David Ramel
Very nice interview for the session scheduled at Live!360 conference in Orlando. Would be even better if micro frontends are mentioned ;).

It's Time To Talk About "CSS5" (https://www.smashingmagazine.com/2024/08/time-to-talk-about-css5/) by Brecht De Ruyte
I'm listening. BTW the last full spec of CSS was 2.1 - since then everything is divided into modules, which have their own level. CSS3 was the first to just be an aggregate of the module levels.

Google Angular Lead Sees Convergence in JavaScript Frameworks (https://thenewstack.io/google-angular-lead-sees-convergence-in-javascript-frameworks/) by Loraine Lawson
I think this is a bit misleading. Just because React now has a compiler does not mean it's converging to Angular...

The Great npm Garbage Patch (https://blog.phylum.io/the-great-npm-garbage-patch/) by Phylum Research Team
Being in touch with Tea protocol myself I am not surprised. It's, unfortunately, essentially crypto garbage. Happy that someone is aware of the problem and reporting the issues for improvement.

The Sneaky Costs of Scaling Serverless (https://www.zachleat.com/web/serverless-cost/) by Zach Leatherman
Spoiler: Serverless can lead to (unnecessary) massive bills if not used correctly.

Issue 49

Do Quests, Not Goals (https://www.raptitude.com/2024/08/do-quests-not-goals/) by David Cain
Right now I am not doing anything besides hiking, swimming, and enjoying the mountains.

Zen Browser (https://www.zen-browser.app/) by Zen
This is what Firefox feels to me. But maybe worth giving it a try, too?!

Tips from 8 months of TanStack/Router in production (https://swizec.com/blog/tips-from-8-months-of-tan-stack-router-in-production/) by Swizec Teller
I still miss the point of using something besides React Router, but I guess that's just a sign of me getting old.

Parallel DOM (https://www.pdom.dev/#how-it-works) by Ashish Shubham
Unfortunately the demos don't work in Firefox - but otherwise looks great if you think about distributing rendering tasks!

An approach to optimizing TypeScript type checking performance (https://www.edgedb.com/blog/an-approach-to-optimizing-typescript-type-checking-performance) by Scott Trinh
Especially check out the general advice section on how to write better performing types.

Optimizing SPA load times with async chunks preloading (https://mmazzarolo.com/blog/2024-08-13-async-chunk-preloading-on-load/) by Mazzarolo Matteo
I'd argue that bundle splitting is great, but instead of just blindly preloading everything you'd only preload parts before they become relevant, e.g., when certain areas are visible or focused.

Prerendering with Preset Vite (https://preactjs.com/blog/prerendering-preset-vite/) by Ryan Christian
This is definitely a useful plugin. Since you can mostly also just alias / replace React by Preact it could be a great way out for simpler pages to avoid requiring JavaScript.

Node.js Takes Steps Towards Removing Corepack (https://socket.dev/blog/node-js-takes-steps-towards-removing-corepack) by Sarah Gooding
Personally, I think it would be better to remove corepack and let it evolve independently.

A case for vanilla JSX (https://vanillajsx.com/) by sdegutis
There is a great debate ongoing in the WICG Web Components GitHub repository that might actually benefit from native JSX (https://github.com/WICG/webcomponents/issues/1069).

20 years since open source release, here comes Ruby on Rails 7.2 with JIT compiler and more (https://devclass.com/2024/08/13/20-years-since-open-source-release-here-comes-ruby-on-rails-7-2-with-jit-compiler-and-more/) by Tim Anderson
20 years ago I found Ruby. Two years later I started using Rails, but even though I then headed in a different direction I still think this is one of the best ecosystems out there.

Issue 50

Optimizing React Applications for Maximum Performance (https://dev.to/surajondev/optimizing-react-applications-for-maximum-performance-5epm) by @surajondev
Honestly, there is a bit more to it, but following these points will give you a great starting point.

ASPNetCore Angular SPA application with SEO friendly (SSR) Title and Meta tags updates (https://medium.com/@haykflexgrigoryan/aspnetcore-angular-spa-application-with-seo-friendly-ssr-title-and-meta-tags-updates-bcef5f75b6ad) by Hakob Grigoryan
Using AngleSharp to optimize an Angular app for SEO? Count me in!

Building Micro-frontends using Blazor and Piral Framework⚡ (https://wdevon99.medium.com/building-micro-frontends-using-blazor-and-piral-framework-c38c5426ccee) by Devon Wijesinghe
Very well written and with a great demo project that you can just use as a starting point.

Bye Bye, Try-Catch Blocks: Meet JavaScript's Safe Assignment Operator Proposal😉 (https://dev.to/dharamgfx/bye-bye-try-catch-blocks-meet-javascripts-safe-assignment-operator-proposal-1j7) by @dharamgfx
This is not yet finalized - but sometimes having the discussion earlier is better than later.

Regexes Got Good: The History And Future Of Regular Expressions In JavaScript (https://www.smashingmagazine.com/2024/08/history-future-regular-expressions-javascript/) by Steven Levithan
This is finalized. While regular expressions in JS will never reach the level of Perl, they are certainly quite decent.

From React to Effect (https://effect.website/blog/from-react-to-effect) by Michael Arnaldi
I was curious at the title, but then it hit me... should have rather been called "Effect for React developers".

Introducing Nuxt Scripts (https://nuxt.com/blog/nuxt-scripts) by Harlan Wilton
Sounds interesting, but I wonder if this could (or should) be extended beyond Nuxt.

spatial compute (https://sunilpai.dev/posts/spatial-compute/) by Sunil Pai
Have we already conquered temporal compute?

JS Dates Are About to Be Fixed (https://docs.timetime.in/blog/js-dates-finally-fixed/) by Iago Lastra
When did I hear that the last time?!

Good Refactoring vs Bad Refactoring (https://www.builder.io/blog/good-vs-bad-refactoring) by Steve Sewell
We've all been there... Most important part: Understand the code, and make sure previously working code can be verified to still be working.

Conclusion

These are all outstanding articles by masterful authors. I enjoyed reading them all - I hope you did find something in there, too.

👉 Follow me on LinkedIn, Twitter, or here for more to come.

🙏 Thanks to all the authors and contributors for their hard work!

🤯 Migration: Completed

Florian Rappl — Tue, 06 Aug 2024 13:20:47 +0000

In the previous four articles (Migration from Classic Hosting to Serverless, Migration of a Multiplayer Game from Hosted to Serverless, Migration of a Dynamic Website to a Static Website, and Migration of Mario 5 to Serverless ) I've introduced you to my plan of migrating away from my dedicated server to a fully serverless infrastructure.

A little bit over 8 months I finished the migration (it's been a while!) of my website florian-rappl.de. So, now it's time to reveal how it went and what's improved (or even got worse).

Before I go into details - let's recap a bit.

Why Migrate?!

Quick recap: What did I expect from this migration:

A more clean code base (finally I can clean up my stuff, maybe remove something and modernize some other parts)
No more FTP or messy / unclear deployments - everything should be handled by CI/CD pipelines
Cost reduction; sounds weird, but the last thing I want to have is a cost increase (today it's about 30 € per month for the hosting and my goal is to bring this below or close to 10 € - note: I pay much more for domains and these costs are not included here as they will remain the same).

There is a bit of background to this: Having my own dedicated server is something I was initially was happy about, however, over all the years the burden of properly maintaining this machine was a bit too high. I have quite some things on my plate and dealing with the (software-side) of a dedicated server was always on the bottom part of my ToDo list.

So, here we are!

The Long Way

Surely, this is "just" a little homepage, but as you can see from the previous articles there are quite some things I'd like to keep working. Overall, the whole migration was prepared all along - with a quick brainstorming first.

This identified what needs to be migrated and how. So having arrived at the end it's time to look at my homepage and what kind of migration it needs.

The Final Piece - My Homepage

In a way this website belongs in a museum. It's out there since 2008 and the inner core never really changed. Surely, there have been some migrations (e.g., from ASP.NET Core MVC to MVC 3 to MVC 4), but all in all the code never really changed.

At the core of the website is a custom CMS that allows me to write, edit, and publish articles on the website. An article may look like this:

While the content is dynamically rendered, other relevant parts of the page (such as the suggestions or tags) are retrieved in relation to the shown article.

The CMS is fully embedded in the website, i.e., the website is also rendering its own admin area that allows to perform all the administrative tasks.

All the data is kept in a MySQL database. There are 28 tables for the whole website - with some tables (such as membership etc.) not being necessary any more. Most of the data is actually stored in the analytics tables, which are then used in the administrative area to display the usage charts.

The red lines have been used by me to indicate what was already successfully migrated.

So let's see how the migration process was done in detail.

Migration Process

First, I identified the general architecture. For me it was clear that in the long run I want to migrate the page to something more static, e.g., based on Astro. But right now, I certainly don't have the time to look over this and actually do it.

As the website is an ASP.NET MVC 4 project that is certainly incompatible with ASP.NET Core I was in a bit of pickle. It was obvious that I cannot use something really cheap (such a Linux App service or Azure Container Instance). I still needed to run on Windows. But on the other hand, I did not want to have a full virtual machine (too expensive, and too much maintenance required).

In the end, I went for using an Azure App Service with a rather cheap plan (that is still fully covered by my monthly Azure allowance). As plan I've taken B1, which is around 30 € per month. Still, this plan already comes with support for custom domains and full-time operation.

Once I deployed the page I've seen that the performance was just superb. Page generation took **0ms! Well, if things are too good to be true - they usually are not true. So let's investigate...

The whole logic is triggered from a certain IHttpModule called TimingModule. The code is like this:

namespace FlorianRappl
{
    using System;
    using System.Diagnostics;
    using System.Web;

    public class TimingModule : IHttpModule
    {
        public void Init(HttpApplication context)
        {
            context.BeginRequest += OnBeginRequest;
            context.EndRequest += OnEndRequest;
        }

        public void Dispose()
        { }

        void OnBeginRequest(Object sender, System.EventArgs e)
        {
            var stopwatch = new Stopwatch();
            stopwatch.Start();
            HttpContext.Current.Items["Stopwatch"] = stopwatch;
        }

        void OnEndRequest(Object sender, System.EventArgs e)
        {
            var stopwatch = HttpContext.Current.Items["Stopwatch"] as Stopwatch;
            if (stopwatch != null) stopwatch.Stop();
        }
    }
}

In short, when a request begins we start a stopwatch. Once it ends - we just stop the stopwatch. We place the result in the current HttpContext. This way, we can access it within our view and render the page with the stop watch timing.

The problem now occurs as the system.web configuration of the Web.config file is not fully considered when running in an Azure App Service. This is a consequence of the App Service running its own runtime - so we don't bring the full runtime. Therefore, only configuration steps after the runtime initialized can be considered.

Consequently, the following sections can be removed from the Web.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.web>
    <httpModules>
      <add name="TimingModule" type="FlorianRappl.Modules.TimingModule, FlorianRappl" />
    </httpModules>
    <!-- rest as-is -->
  </system.web>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true">
      <add name="TimingModule" type="FlorianRappl.Modules.TimingModule, FlorianRappl" />
    </modules>
    <!-- rest as-is -->
  </system.webServer>
  <!-- rest as-is -->
</configuration>

But what to add for bringing back the timing module? Introducing... the PreApplicationStartCode. Just introduce a class like this:

namespace FlorianRappl
{
    using System.Web;

    public class PreApplicationStartCode
    {
        public static void Start()
        {
            HttpApplication.RegisterModule(typeof(TimingModule));
        }
    }
}

Now we can have access to the startup code of the runtime. We use it to register the TimingModule in the application.

With this everything is in ship-shape:

Now that this is working it's time to look at some performance improvements.

Performance Improvements

You may wonder: Why is it slower than beforehand? Well, there are many reasons. For once, we now are running in a shared environment. Beforehand, we had a dedicated server.

However, certainly the larger impact comes from the database. Beforehand, we have been using a MySQL system running on the same machine (i.e., short travel distance), now we run on very cheap Azure Table Storage (i.e., longer travel distance, less query throughput).

Specifically, there are many unnecessary queries running. One thing we can do to improve this is to introduce some caching. Specifically, we can cache the article listings (not the articles themselves) and the tags (they are often correlated and need to be used for coming up with suggestions).

The effect of caching can be seen immediately.

But one thing was still missing... I realized that there is a certain spike in response times when the long polling endpoint is called. In the end, it turned out that ASP.NET MVC is only allowing a single session to retrieve data at a time. If more requests come in from the same session then these requests need to wait.

There is a simple way out of this trouble: Declare the SessionState as ReadOnly. This way, we cannot modify the session associated with the request, but we therefore also don't need to wait to avoid any race condition.

This can look like this (here, all endpoints of the controller are moved to this state):

using System.Web.Mvc;
using System.Xml;

[SessionState(SessionStateBehavior.ReadOnly)]
public class ApiController : BaseController
{
    // ...
}

For an ApiController that does (or should not) work on the session this should be the default behavior.

As far as the cache goes the standard ASP.NET cache behaves rather poorly on the app service. What we can do is to introduce a ConcurrentDictionary to handle this:

private static readonly ConcurrentDictionary<string, object> cache = new ();

This, of course, only works because we are restricted to a single instance. If the website would run in a multi-instance mode we'd need a different solution such as as a Redis cache (alternatively, we could then use the ASP.NET cache as it is automatically synced within an app service).

We introduce two convenience methods to control the cache.

private T GetOrAddCache<T>(String key, Func<T> callback) => 
    (T)cache.GetOrAdd(key, (_) => callback());

private void ResetCache(String key) =>
    cache.TryRemove(key, out _);

As an example the FindAllTags method to retrieve all tags now looks like:

public IEnumerable<Tag> FindAllTags() =>
    GetOrAddCache(nameof(FindAllTags), () => Tags.Query<Tag>().ToList());

Likewise, when we mutate the tags we also need to reset the cache. Here the Delete(Tag) method had to be adjusted:

public void Delete(Tag tag)
{
    Tags.DeleteEntity(tag.PartitionKey, tag.RowKey);
    ResetCache(nameof(FindAllTags));
}

With everything in place we can now compare the behavior of the website with these changes (running in the cloud) to the previous solution.

Comparison

We started with the news overview page taking between 16ms and 25ms. This was on the dedicated server, with the database being integrated (i.e., running on the same machine) as a MySQL database.

Now, we are running in Azure using an Azure App Service. We had to use a Windows ASP.NET MVC 4 compatible App Service plan, which is a bit pricey (total cost would be a bit higher than we previously had) - but due to my monthly Azure allowance still much cheaper than beforehand.

With everything in place we reach times between 4ms and 50ms. The average is between 8ms and 10ms. That the variance is higher is not actually a problem, as long as the worst result is within the 100ms time frame.

As the average time is now much better than beforehand I'm (on average) happy. Still, I potentially could have achieved the same time on the dedicated machine; I only should have looked. But since I thought the performance is good enough I did not spend more time on it.

Next Steps

1️⃣ Migrate away from the custom analytics solution to Azure App Insights (or some other service - ideally one with a possible data import)

2️⃣ Put everything into a SSG such as Astro, i.e., generate the page fully at build (hopefully incrementally to avoid building 1000s of pages every time something small changes)

3️⃣ Use micro frontends to bring in little islands of interactivity that can be updated / deployed independently of the SSG part

The latter would only be interesting for the parts that definitely require JavaScript or data at runtime, such as the current list of articles on other websites ("stream" on the homepage) or things such as little games that are available on the page.

Conclusion

It runs - faster, more modern, and far more cost efficient (for the given subdomains no additional costs occur). The crucial part was to identify a way of providing the content in a mode that fits its purpose best.

With an average memory allocation never exceeding 70% and a peak CPU utilization of 30% I think the chosen plan works out nicely. While the plan would usually cost around 30 € monthly (thus certainly not being cheaper than my dedicated hosting beforehand) the monthly Azure allowance reduces that to 0 €.

So overall the payment for the infrastructure was reduced significantly. Previously I had to pay about 30 € per month for the hosting - now it's just $5. In total I am now at 15% of the original cost. Surely, if the monthly Azure allowance would not be in there - that ratio would not be the same - but let's not forget that the page does not need to run on an Azure App Service plan.

In the long run I will migrate it to a fully static page - thus not requiring much infrastructure and being far more efficient. But this is the content for another story...

🙏 Thanks for following my migration and happy coding everyone!