Forem: Oxford Harrison

A Fresh Take On Reactivity

Oxford Harrison — Wed, 05 Mar 2025 10:42:10 +0000

It's been months of work mocking up a new runtime capability in JavaScript! The new capability lets us have reactivity as an entirely language-driven feature in JavaScript such that using the exact same imperative syntax, we can write programs that automatically reflect changes to state in fine-grained details.

The idea?:

Say you have the following piece of logic...

let count = 10;
let doubleCount = count * 2;
console.log(doubleCount);

a change to count...

setTimeout(() => count = 20, 1000);

(being a change in program state) should be automatically reflected down the line such that with the above, the console has a second output of 40.

This is that thing you want to do in React when you write:

import { useState, useMemo, useEffect } from 'react';

const [count, setCount] = useState(10);
const doubleCount = useMemo(() => count * 2, [count]);

useEffect(() => {
  console.log(doubleCount);
}, [doubleCount]);

setTimeout(() => {
  setCount(20);
}, 1000);

and in a Signals-based framework like Solid when you write:

import { createSignal, createMemo, createEffect } from 'solid-js';

const [ count, setCount ] = createSignal(10);
const doubleCount = createMemo(() => count() * 2);

createEffect(() => {
  console.log(doubleCount());
});

setTimeout(() => {
  setCount(20);
}, 1000);

all of which, however, require special metaphors for the job and ultimately result in a new problem: too many moving parts and so much of a Functional Programming exercise for UI development!

Sit long enough with all current approaches, you realize that there's only so much that's possible without deep language support for reactivity. It comes straight out that the language is its own best enabler of the idea.

Having itched a ton on this over the past few months, today, I am excited to introduce the underexplored programming paradigm here: Imperative Reactive Programming!

This is being developed in this little repo of mine and is ready to be taken for a spin:

webqit / quantum-js

A runtime extension to JavaScript that unlocks Imperative Reactive Programming (IRP) in JavaScript!

Quantum JS

Overview • Creating Quantum Programs • Implementation • Examples • License

Quantum JS is a runtime extension to JavaScript that brings Imperative Reactive Programming to JavaScript!

What's that?

Overview

Where you normally would require certain reactive primitives to express reactive logic...

// Import reactive primitives
import { createSignal, createMemo, createEffect } from 'solid-js';

// Declare values
const [ count, setCount ] = createSignal(5);
const doubleCount = createMemo(() => count() * 2);
// Log this value live
createEffect(() => {
  console.log(doubleCount());
});

// Setup periodic updates
setInterval(() => setCount(10), 1000);

Quantum JS lets you acheive the same in the ordinary imperative form of the language:

// Declare values
let count = 5

…

View on GitHub

Already, you can have much of what you see here working right in your browser via the Quantum JS script:

<script src="https://unpkg.com/@webqit/quantum-js/dist/main.js"></script>

Details right in the readme

Laying Reactive Primitives to Rest

Reactivity has only been possible by means of functions. Reactive systems rely on use of these primitives to gain the ability to track state and drive effects.

By contrast, the language-based model doesn't have this limitation. That effectively renders the essence of these primitives obsolete!

Coincidentally, this is what would be expected as the natural evolution of reactivity: towards less and less of its mechanical moving parts, not more! (People aren't itching to have more!)

It's essentially the path we've been walking for some time now with compilers—Svelte, Vue, React, etc.! (For example, the React compiler now helps you eschew the manual use of useMemo() and useCallback().) Only, we'd know we've hit perfection when there isn't any more primitive to remove!

There's an old quote regarding this benchmark:

"Perfection is attained not when there is nothing more to add, but when there is nothing more to take away." — Antoine de Saint-Exupéry (1939)

And it turns out, people really want to see things zero down to "arbitrary JS":

"It’s building a compiler that understands the data flow of arbitrary JS code in components.

Is that hard? Yeah!
Is it possible? Definitely.

And then we can use it for so many things…" - sophie alpert

The new language-based model is what you see when you walk the compiler path to its end!

Putting the Language to Work in Fun Ways

The first notable thing about the new language-based approach is how you can literally write an arbitrary piece of code... say within a script or function...

<script>
  // Program body:
  // 100s of lines of code here
</script>

function program() {
  // Program body:
  // 100s of lines of code here
}

and express the additional intent of reactivity with only a declaration:

<script shouldbereactive>
  // Program body:
  // 100s of lines of code here
</script>

shouldbereactive function program() {
  // Program body:
  // 100s of lines of code here
}

of course, with shouldbereactive being actually the keyword quantum, in practice:

<script quantum>
  // Program body:
  // 100s of lines of code here
</script>

providing a direct upgrade path from "regular" to "reactive" (and back)—with no changes at all in code.

Here, the quantum declaration opts you in to a new execution mode in the runtime—the "quantum" execution mode—in which the runtime stays sensitive to changes in its own state and automatically gets the relevant parts of the program re-executed.

Being this way a runtime extension (rather than a syntax extension), your involvement in the process is done here. The rest, and bulk, of the story continues within the runtime—and by the "person" that's most specialized in low level things: the runtime itself!

What's next is the many amazing implications that this has for us as developers!

Reactivity in the Full Range of the JS Language

Given a language-level reactive system, think the freedom to use the full range of the JS language without losing reactivity!

Think reactive array mutations, wherein, given the below...

<script quantum>
  let products = [];
  console.log(products.length, products[0]); // 0, undefined
</script>

each of the following operations is automatically reflected in the console:

products.push('T-Shirts');
// console: 1, 'T-Shirts'

products.pop();
// console: 0, undefined

Think reactive object mutations, wherein, given the below...

<script quantum>
  let person = {};
  console.log(person.name, person.age); // undefined, undefined
</script>

each of the following operations is automatically reflected in the console:

person.name = 'Sam';
// console: 'Sam', undefined

Object.assign(person, { name: 'Tam', age: 1 });
// console: 'Tam', 1

Object.defineProperty(person, 'name', { value: 'Ham' });
// console: 'Ham', 1

Think live counters:

<script quantum>
  let count = 0;
  console.log(count);
</script>

with each tick from below automatically reflecting in the console:

setInterval(() => count++, 1000);

Think live loops, wherein, given the below...

<script quantum>
  let products = [];
  for (let p of products) {
    console.log(p);
  }
</script>

each of the following operations automatically advances the iteration:

products.push('T-Shirts');
// console: 'T-Shirts'

products.push('Backpacks');
// console: 'Backpacks'

Think live flow control constructs, wherein, given the below...

let condition;

quantum function program() {
  if (!condition) {
    console.log('Exiting...');
    return;
  }
  console.log('Running...');
}

program();
// console: 'Exiting...'

each of the following operations automatically controls the program flow:

condition = true;
// console: 'Running...'

condition = false;
// console: 'Exiting...'

condition = true;
// console: 'Running...'

Think, think: destructuring assignments, rest assignments, and more.

...and Beyond

Think of some weird things that could now be possible, like "live" module imports in JavaScript! (I.e. Reactivity through the wire!)

Here's a demo of that that works today:

<!-- exporting module -->
<script type="module" quantum id="module-1">
  export let localVar = 0;
  setInterval(() => localVar++, 500);
</script>

<!-- importing module -->
<script type="module" quantum>
  import { localVar } from '#module-1'; // the ID of the module script above
  console.log(localVar); // 1, 2, 3, 4, etc.
</script>

Personally looking forward to how this evolves! For example, might this eventually come to having an import attribute?:

import { localVar } from '#module-1' with { live: true };

Up for a Spin!

It's been super fun building the Quantum JS compiler along with its runtime, and essentially living at the very edge of the JavaScript language—learning every nuance that exists therein! We're now ready to run in your browser!

The fastest way to try is via the Quantum JS script loaded from a CDN:

<script src="https://unpkg.com/@webqit/quantum-js/dist/main.js"></script>

But, of course, that's just one of many ways—depending on use case. For example...

Quantum JS itself has no concept of the DOM, and, as such, no concept of HTML elements like <script>. You'd, instead, need to use the OOHTML implementation of Quantum JS for DOM-based integration:

<script src="https://unpkg.com/@webqit/oohtml/dist/main.js"></script>

That in place, here are some of my favourite examples which you can copy/paste and run directly in your browser (without a build step):

Example 1: A Custom Element-Based Counter
└─────────

In this example, we demonstrate a custom element that works as a counter. Notice that the magic is in its Quantum render() method. Reactivity starts at connected time (on calling the render() method), and stops at disconnected time (on calling dispose)! (You can also find this example in the Quantum JS list of examples.)

<script>
  customElements.define('click-counter', class extends HTMLElement {

  count = 10;

  connectedCallback() {
    // Initial rendering
    this._state = this.render();
    // Static reflection at click time
    this.addEventListener('click', () => {
      this.count++;
    });
  }

  disconnectCallback() {
    // Cleanup
    this._state.dispose();
  }

  quantum render() {
    let countElement = this.querySelector('#count');
    countElement.innerHTML = this.count;

    let doubleCount = this.count * 2;
    let doubleCountElement = this.querySelector('#double-count');
    doubleCountElement.innerHTML = doubleCount;

    let quadCount = doubleCount * 2;
    let quadCountElement = this.querySelector('#quad-count');
    quadCountElement.innerHTML = quadCount;
  }

});
</script>

<click-counter style="display: block; padding: 1rem;">
  Click me<br>
  <span id="count"></span><br>
  <span id="double-count"></span><br>
  <span id="quad-count"></span>
</click-counter>

Example 2: A Custom URL API
└─────────

In this example, we demonstrate a simple replication of the URL API - where you have many interdependent properties! Notice that the magic is in its Quantum compute() method which is called from the constructor. (You can also find this example in the Quantum JS list of examples.)

<script>
  const MyURL = class {

    constructor(href) {
      // The raw url
      this.href = href;
      // Initial computations
      this.compute();
    }

    quantum compute() {
      // These will be re-computed from this.href always
      let { protocol, hostname, port, pathname, search, hash } = new URL(this.href);

      this.protocol = protocol;
      this.hostname = hostname;
      this.port = port;
      this.pathname = pathname;
      this.search = search;
      this.hash = hash;

      // These individual property assignments each depend on the previous 
      this.host = this.hostname + (this.port ? ':' + this.port : '');
      this.origin = this.protocol + '//' + this.host;
      let href = this.origin + this.pathname + this.search + this.hash;
      if (href !== this.href) { // Prevent unnecessary update
        this.href = href;
      }
    }
  }
</script>

<script>
  // Instantiate
  const url = new MyURL('https://www.example.com/path');

  // Change a property
  url.protocol = 'http:'; //Observer.set(url, 'protocol', 'http:');
  console.log(url.href); // http://www.example.com/path

  // Change another
  url.hostname = 'foo.dev'; //Observer.set(url, 'hostname', 'foo.dev');
  console.log(url.href); // http://foo.dev/path
</script>

Example 3: An Imperative List
└─────────

In this example, we demonstrate a reactive list of things. Notice how additions and removals on the items array are statically reflected on the UI! (You can also find this example and its declarative equivalent in the OOHTML list of examples.)

<section namespace>

  <!-- The "items" template -->
  <template def="partials" scoped>
    <li  def="item"><a></a></li>
  </template>

  <!-- The list container -->
  <ul id="list"></ul>

  <script quantum scoped>
    // Import item template
    let itemImport = this.import('partials#item');
    let itemTemplate = itemImport.value;

    // Iterate
    let items = [ 'Item 1', 'Item 2', 'Item 3' ];
    for (let entry of items) {
      const currentItem = itemTemplate.cloneNode(true);
      // Add to DOM
      this.namespace.list.appendChild(currentItem);
      // Remove from DOM whenever corresponding entry is removed
      if (typeof entry === 'undefined') {
        currentItem.remove();
        continue;
      }
      // Render
      currentItem.innerHTML = entry;
    }

    // Add a new entry
    setTimeout(() => items.push('Item 4'), 1000);
    // Remove an new entry
    setTimeout(() => items.pop(), 2000);
  </script>

</section>

And No, We Aren't Done!

There's definitely a bunch of things to come! And there's definitely a bunch of questions to be answered too, including:

How does reactivity actually happen? What is the update model?
How bad, really, was the explicit approach to reactivity?
What does it look like building real world applications?

This post is the first in a series!

Our Star ☆ Button

Would you help our project grow and help more people find us by leaving us a star on github? We, too, have a star button 😅.

Star Us on Github ★

Much Thanks

...to everyone who gave great feedback on drafts of this post, including: Joe Pea, and Alex Russell!

Revisiting the HTML Problem Space and Introducing OOHTML

Oxford Harrison — Wed, 24 Jan 2024 13:01:03 +0000

It's 2024! Is it perhaps time to give attention to HTML?

It's a time when many people are interested in going back to the basics! Even whole teams are looking to make the big return from rethinking best practices! (How we, in fact, got fed such a line remains a mystery!)

But perhaps, no one is talking about the alternative and how still "philosophical" it is to build anything of substance in just HTML, CSS and JS in 2024! In fact, you may contend that a lot still goes unanswered in our web-native playbooks!

I'd agree!

Actually, while it's now possible to streamline your workflow with features like native ES Modules + Import Maps (I can see you on that, Joe Pea), the Shadow DOM (I see that you love that, Passle), CSS nesting, etc., so much is still only philosophical!

For example, more recently, I decided to play the devil's advocate on the subject myself, and, so, threw in an obvious question: how do you do reactivity? (Hoping to hear something intriguing!) But it turns out, we still aren't close! And you're almost always disappointed each time!

It's something we cannot possibly be dogmatic about or evangelize as though there were no limitations when we talk about web-native development. That could make one come across as being out of touch with reality or as being merely dismissive of framework-era innovations!

Web-native development, from one perspective, simply addresses overheads!

That's the whole point about "leveraging native platform capabilities and streamlining your workflow and tooling budget"! — The web-native manifesto.

That's the whole point about native technologies like Web Components, speaking of which:

"One of the best outcomes from delegating our component model to the platform is that, when we do, many things get cheaper. Not only can we throw out the code to create and manage separate trees, browsers will increasingly compete on performance for apps structured this way." — Alex Russel

And should any aspect of that, at some point, begin to go counterproductive, such that you wind up with more boilerplates and homegrown tooling, or more runtime regressions — being oftentimes the counterargument, it should be time again to address overheads, not sugar coat them!

In the spirit of that, here is what I am happy to do today: discuss specific pain points that have yet to be addressed natively and touch each one with the relevant part of a proposal you're sure to be excited about: Object-Oriented HTML (OOHTML)!

OOHTML is a new set of HTML and DOM-level features designed to make the HTML-First + Progressive Enhancement paradigm a viable idea — initially developed out of need at WebQit! If you've ever tried, as much as we have, to make success of the above, you probably would come to the same design and architectural conclusions as below and you might as well be inclined to put your money where your mouth is!

Coincidentally, with vanilla HTML increasingly becoming an attractive option for the modern UI author — amidst a multitude of frameworks, it is a perfect time to revisit the HTML problem space!

This is a long discussion but I can tell you this will be worth your time and contribution! See what's on the agenda:

Question 1: How Do You Do Reactivity?
Question 2: How Do You Make Components?
Question 3: How Do You Re-Use Components?
Question 4: How Do You Pass Data/Props?

Show Full Outline

Question 1: How Do You Do Reactivity?
- Introducing: Native Data Binding for HTML
- Wanna Try?
- Introducing: Quantum Reactivity
- This Works Today!
Question 2: How Do You Make Components?
- Introducing: Namespacing
- Good Thinking?
- Introducing: Style and Script Scoping
- Try Now!
Question 3: How Do You Re-Use Components?
- Introducing: HTML Imports
  - Module Definition
  - Remote Modules
  - Module Imports
  - Scoped Modules
- Cool Yet?
Question 4: How Do You Pass Data/Props?
- Introducing: The Bindings API
- Good Job There?
- Introducing: The Context API
- Exciting Yet?

Question 1: How Do You Do Reactivity?

This isn't something Web Components do today! And I do in fact fear that inventing a WC-only solution to this effect, such that you now have to employ a custom <my-div> element where a regular <div> element would ordinarily suffice, would yet constitute a new form of overhead!

And I'd like to say that what we do in Lit today isn't to me a particularly attractive thing! TL;DR: a runtime solution to a markup-level problem is an absolute non-starter! That's to still miss the essence of a markup language!

You enter a framework like Vue or Svelte and you see something more like the ideal: a markup-based solution — wherein HTML is your first language:

<!-- Vue -->
<script setup>
  let count = 0;
</script>

<template>
  <button>Count is: {{ count }}</button>
</template>

<!-- Svelte -->
<script>
  let count = 0;
</script>

<button>Count is: {count}!</button>

Something more HTML-ish now!

You are able to just write markup and progressively add logic, as against going ahead of the problem with a JS-first approach!

Now, you think of this as a native language feature, and you aren't totally mistaking! For that would be us delegating all of the work to the HTML parser and getting to write code that practically hits the ground running:

button.innerHTML = 'Count is: {count}!';

Which would mean: no compile step; no waiting for some JS dependency to generate the UI!

Introducing: Native Data Binding for HTML

But the Vue/Svelte conventions above aren't where this leads! Not exactly!

First, we do need a binding syntax that upholds, instead of change, the default behaviour of the opening brace { and closing brace } characters in HTML! Whereas these should behave as mere text characters, a new behaviour is introduced above wherein, across the active DOM, every arbitrary text sequence in that form — {count} — is treated as special and automatically transformed. That as a native language feature (i.e. browsers suddenly waking up with this new rule) would be a breaking change to older code that make up a large percentage of the web today:

<body>
  Hello {curly braces}!
</body>

element.innerHTML = 'Hello {curly braces}!';

Consider the dilemma: should raw template tags be default-visible so as to be backwards-compatible with older code? Then, newer code would be having a "flash of unrendered content" and an inadvertent exposure of low-level implementation details! Should they be default hidden, then older code would be broken!

Given how the HTML language currently works, I found a sweet spot with a comment-based approach wherein regular HTML comments get to do the job and double up as dedicated insertion points for application data:

<body>
  Hello {curly braces}!

  <button>Count is: <?{count}?>!</button>

  Or, using the comment syntax you might know: <button>Count is: <!--?{count}?-->!</button>
</body>

element.innerHTML = 'Count is: <?{count}?>. Hello {curly braces}!';

Data-binding expressions are now default-hidden while application data becomes available at any timing! And the idea of having a special behaviour for certain text characters now requires an explicit opt-in using a special opening character sequence <? (or the longer equivalent ). (Of course, syntax could be improved on where possible!)

This is something I really do like and I found that it resonates with many developers!

We next come to attribute bindings and the first challenge becomes whether or not to support text interpolation within attributes as Svelte would have it?:

<img {src} alt="{name} dances." />

Heck! That again would be a breaking change in how curly braces are treated within attributes, just as has been considered above!

Also, how do you hydrate server-rendered attributes such that, given the <img> above, the browser still knows to map the src and alt attributes to certain application data after having been rendered to?:

<img src="/my/image.png" alt="John Doe dances." />

Interestingly, I did set my foot on this path with OOHTML and that turned out messy!

Here is where Vue shines with its idea of having a separate, pseudo attribute do the binding for every given attribute:

<img v-bind:src="src" v-bind:alt="computedExpr">
<!-- Or, for short: <img :src="src" :alt="computedExpr"> -->

which renders to:

<img v-bind:src="src" v-bind:alt="computedExpr" src="/my/image.png" alt="John Doe dances.">

thus, obviating the trouble with the former approach.

But heck! The attribute bloat... the sheer idea of an extra attribute for every given attribute!

What if we simply had one dedicated attribute for everything wherein we follow a key/value mapping syntax?:

<img render="src:src; alt:computedExpr">
<!-- Or, with on-the-fly JS expressions: <img render="src:src; alt:name+' dances.'"> -->

to render to?:

<img render="src:src; alt:computedExpr" src="/my/image.png" alt="John Doe dances.">

That would give us a more compact binding syntax, plus the same benefit as with the Vue approach — of having the original binding logic retained, instead of replaced, after having been rendered!

We only now need to extend our syntax to support a wider range of DOM element features like "class list" and "style". For this, we introduce the idea of directives, or special symbols, that add the relevant meaning to each binding:

e.g. the tilde symbol ~ for attribute binding:

<img render="~src:src; ~alt:computedExpr" src="/my/image.png" alt="John Doe dances.">

the percent symbol % for class binding:

<my-widget render="%active:app.isActive"></my-widget>

the ampersand & for CSS binding:

<div render="&color:app.themeColor"></div>

etc.

I find this very conventional on the backdrop of existing syntaxes like CSS rules! And we've introduced no breaking changes and no funky attribute names!

But how should bindings be resolved? From an embedded <script> tag in scope as we have above?:

<script>
  let count = 0;
</script>

<button>Count is: <?{ count }?>!</button>

That idea might be more suited to the Single File Component (SFC) architecture than to the DOM itself where the actual building blocks aren't SFCs but the very DOM nodes themselves!

Given the DOM, it makes more sense for the said JS references in binding expressions to have to do with the state of those DOM nodes:

<button>Count is: <?{ count }?>!</button>

button.count = 0;

which also resonates with the object/property paradigm of Custom Elements:

<my-counter>Count is: <?{ count }?>!</my-counter>

customElements.define('my-counter', class extends HTMLElement {
  connectedCallback() {
    this.count = 0;
  }
});

But there goes one subtle challenge: using an element's root namespace for arbitrary state management and potentially conflicting with native methods and properties!

It turns out, a more decent approach would be to have a dedicated state object for this:

button.bindings.count = 0;

connectedCallback() {
  this.bindings.count = 0;
}

And to make this even more flexible, we don't have to restrict the resolution scope of references to the immediate host elements. A more flexible approach would be to have things resolve from the closest node in scope who exposes said property, such that:

<body>
  <div>
    <div>
      <button>Count is: <?{ count }?>!</button>
    </div>
  </div>
</body>

document.body.bindings.count = 0;
// Or, even
document.bindings.count = 0;

still works!

This way, it becomes possible for higher-level state (or even the global state that lives at document.bindings) to still be reflected at deeply nested elements in the page without having to mechanically pass data from node to mode down the tree!

Wanna Try?

Simply include the OOHTML polyfill from a CDN!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

And here's something to try on a blank document:

<!DOCTYPE html>
<html>

  <head>
    <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
    <script>

      // Global count
      document.bindings.count = 0;
      setInterval(() => document.bindings.count++, 1000);

      // Local count
      customElements.define('my-counter', class extends HTMLElement {
        connectedCallback() {
          this.bindings.count = 0;
          setInterval(() => this.bindings.count++, 2000);
        }
      });
    </script>
  </head>

  <body>
    <h1>Global count is: <?{ count }?>.</h1>
    <my-counter>Local count is: <?{ count }?>.</my-counter>
  </body>

</html>

And you could try writing the "Declarative Lists" example in the list of examples!

I ask that you share your thoughts and leave us a star 🌟!

You may want to visit the Data-Binding section on the project README to learn more!

Introducing: Quantum Reactivity

Data binding isn't all there is to reactivity on the UI!

While you're able to express JavaScript logic within binding tags:

<h1>Global count is: <?{ count }?>.</h1>
<h2>Global count, doubled, is: <?{ count * 2 }?>.</h2>

you still need a way to write reactive logic in your main application code — i.e. the counter logic itself, above — as things begin to grow and require some form of dependency tracking.

Here's a sampling across frameworks of how our code could look in a reactive programming paradigm:

<!-- Vue -->
<script setup>
  import { ref, computed } from "vue";

  let count = ref(0);
  let doubleCount = computed(() => count.value * 2);
</script>

<template>
  <button>Count is: {{ count }}</button>
  <button>Double count is: {{ doubleCount }}</button>
</template>

<!-- Svelte -->
<script>
  let count = 0;
  $: doubleCount = count * 2;
</script>

<button>Count is: {count}!</button>
<button>Double count is: {doubleCount}!</button>

<!-- Upcoming Svelte 5 -->
<script>
  let count = $state(0);
  let doubleCount = $derived(count * 2);
</script>

<button>Count is: {count}!</button>
<button>Double count is: {doubleCount}!</button>

But it's a whole new paradigm now from the ordinary imperative paradigm you started with! Many "reactive primitives" and much symbolism now, and there isn't a universal syntax for this form of programming!

Svelte 3, in 2019, did try to bring reactivity in vanilla JS syntax, as seen in the middle code above, but that was greatly limited in many ways! And, unfortunately, Svelte is unable to move forward with that idea in v5!

Problem with the above is: that whole idea of manually modelling relationships is an overhead; you're paying a price that need not be paid! Short explainer: you do not need to re-express the dependency graph of your logic when that's a baseline knowledge for any runtime! Plus, you could never do that manually as cheaply, and as accurately as a runtime would!

Long explainer is what I presented in Re-Exploring Reactivity and Introducing the Observer API and Reflex Functions — the previous post in this series! You'll find that we can have reactivity without a shift in programming paradigm!

It turns out, we're able to again delegate this work to the platform and simply enable reactive programming in the ordinary imperative form of JavaScript! Impossible? That is what we sought to explore with the Quantum JavaScript project!

Quantum JS is a runtime extension to JavaScript that lets us write ordinary JavaScript and have it work reactively! Given our markup above, we're able to simply opt-in to reactive programming on the regular <script> element using the quantum attribute:

<script quantum>
  // Global count
  let count = 0;
  let doubleCount = count * 2;
  setInterval(() => count++, 1000);

  document.bindings.count = count;
  document.bindings.doubleCount = doubleCount;
</script>

and for the Custom Element definition, we're able to turn our existing connectedCallback() method to a Quantum function using a special quantum flag:

// Local count
customElements.define('my-counter', class extends HTMLElement {
  quantum connectedCallback() {
    let count = 0;
    let doubleCount = count * 2;
    setInterval(() => count++, 1000);

    this.bindings.count = count;
    this.bindings.doubleCount = doubleCount;
  }
});

It's exciting how we practically now have a universal syntax across both reactive and non-reactive code and can now leverage metal-level accuracy and performance! And as a runtime extension to JavaScript, there isn't the "Svelte 3" sort of limitations like relying on symbolism or top-level let declaration, and others! On the contrary, you are able to enjoy the full range of the JS language — conditionals and loops, spread and "rest" syntax sugars, flow control statements, etc.

This Works Today!

Simply include the OOHTML Polyfill and you have a new useful magic right in the browser! You wanna try the "Imperative Lists" example in the list of examples to have a feel!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

You may also want to visit Quantum JS itself and leave us a star 🌟.

Question 2: How Do You Make Components?

Let's say: Web Components, or, in other words, Custom Elements and the Shadow DOM — being collectively a powerful mechanism for encapsulating behaviour, structure, and styling!

Cool, but... there are times you aren't thinking about a "custom" element or some special technology called Shadow DOM when you say "component"!

Given the tailwind component library as one random reference

the Daisy UI library as another
the Bootstrap starter library as another

a sampling of people's idea of that term should include:

a distinct unit of behaviour
- e.g. a carousel component that enables a certain form of interaction consistently across instances, e.g. rotating through a series of images or content
a distinct structural form
- e.g. a card component that models a certain structure consistently across instances, e.g. featuring an image, a title bar, a description line, a summary area
a distinct visual form
- e.g. a grid component that presents elements in a certain pattern consistently across instances

This means that it isn't always a Web Components' call! For example, what have many "visual" components, like grid above, got to do with Custom Elements or Shadow DOM? And let's just say, this whole idea of subtree isolation which the Shadow DOM is best for, isn't always the idea for many, many "components"!

Consider, for example, structural components, like card above! I often just want a way to "model" the needed relationships, or, in other words, a way to associate the card's features like image and title bar with the card, and nothing more! (Something people have done for years with a naming convention like BEM!)

<div class="card">
  <img class="card__image" src="image.jpg" alt="Image">
  <h2 class="card__title">Card Title</h2>
  <div>
    <p class="card__description">Card Description</p>
  </div>
  <div>
    <p class="card__summary">Card Summary</p>
  </div>
</div>

which would translate to writing CSS selectors that feel "namespaced":

.card { ... }
.card__title { ... }

This means, you more often just want to write modular markup and less often want to opt-in to subtree isolation and a new rendering model!

The ideal authoring experience for HTML, therefore === having a way to do both elegantly: author modular markup by default and let the use case decide when to opt-in to the Shadow DOM!

Introducing: Namespacing

Enter modular markup: we currently have to rely on annoying naming conventions to model relationships. (E.g. BEM, above.) And for the relationships that work with only IDREFs, like the relationship between a <label> and an <input>, and others involving ARIA Relationship Attributes, we are challenged with HTML's global ID system wherein you have to generate IDs that must be unique throughout the document:

<form>

  <fieldset>
    <legend>Home Address</legend>

    <label for="home__address-line">Address</label>
    <input id="home__address-line">

    <label for="home__city">City</label>
    <input id="home__city">
  <fieldset>

  <fieldset>
    <legend>Delivery Address</legend>

    <label for="delivery__address-line">Address</label>
    <input id="delivery__address-line">

    <label for="delivery__city">City</label>
    <input id="delivery__city">
  <fieldset>

</form>

This has you thinking globally even when what you're working on has no global relevance! For a fairly-sized document, the level of coordination needed at the global level is often too unrealistic to happen by hand!

But how about a way to keep those relationships local, or implicitly namespaced? Meet the namespace attribute:

<form>

  <fieldset namespace>
    <legend>Home Address</legend>

    <label for="~address-line">Address</label>
    <input id="address-line">

    <label for="~city">City</label>
    <input id="city">
  <fieldset>

  <fieldset namespace>
    <legend>Delivery Address</legend>

    <label for="~address-line">Address</label>
    <input id="address-line">

    <label for="~city">City</label>
    <input id="city">
  <fieldset>

</form>

Notice how we haven't introduced a breaking change to how IDREFs are resolved in browsers! You need the tilde ~ character to denote "relativity", or "local" resolution.

You're able to use that in selectors to match things within a specified namespace:

fieldset #~address-line {}

document.querySelector('#~address-line'); // null; not in the global namespace
document.getElementById('~address-line'); // null; not in the global namespace

fieldset.querySelector('#~address-line'); // input#address-line; in the fieldset namespace

In JavaScript, you are additionally able to access these elements declaratively using the namespace API:

let { city, ... } = fieldset.namespace;

And you get reactivity for free on top of that — in being able to observe DOM changes happening within given namespace:

Observe.observe(fieldset.namespace, changes => {
  console.log(changes);
});

Good Thinking?

This is all currently possible using the OOHTML Polyfill!

The "Multi-Level Namespacing" example in the list of examples is something you may want to try!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

You may also want to visit the Namespacing section in the project README to learn more!

I again ask that you share your thoughts and leave us a star 🌟!

Introducing: Style and Script Scoping

While namespacing helps us keep IDREFs relative, we still need a way to keep component-specific style sheets and scripts scoped! This is a common idea across frameworks!

Here, we are able to do that using a new scoped attribute:

<div>

  <style scoped>
    :scope { color: red }
  </style>

  <script scoped>
    console.log(this) // div
  </script>

</div>

Just like that!

Now, that comes especially crucial to SPAs!

It is often asked how the scoped attribute on the <style> element compares with the @scope rule in the CSS Cascading and Inheritance Level 6 Module which lets us do something similar:

<div>
  <style>
    @scope {
      p { color: red; }
    }
  </style>
  <p>this is red</p>
</div>
<p>not red</p>

But here's the deal with the attribute-based approach:

Free of an extra nesting level!
Consistent with the syntax for two other things in the scoping agenda: scoped scripts — <script scoped> — and scoped HTML modules — <template scoped> — as we'll see shortly!
Presents one way to interpret a <style> element: as either "entirely scoped" or "entirely unscoped"; which isn't the case with the @scope rule approach:

Code
```
<style scoped>
/* scoped rules */
<style>

<style>
/* unscoped rules */
<style>
```
vs
```
<style>
@scope {
  /* scoped rules */
}
/* unscoped rules */
@scope {
  /* scoped rules */
}
/* unscoped rules */
</style>
```
Notice how the latter requires you to parse the CSS source text itself to know what's going on with a <style> element, vs how the the former removes the guesswork at the attribute level!

Presents us an opportunity to have scoped style sheets that can truly make no global footprint, as in, this time, map to their immediate host element instead of to the document:

Code

<div>
  <style scoped>
  /* scoped rules */
  </style>
</div>

<style>
/* unscoped rules */
</style>

// Given that the scoped style sheet really has no global relevance
console.log(div.styleSheets.length); // 1
// Only the second style sheet making a global footprint
console.log(document.styleSheets.length); // 1

That said, here's how the scoped attribute works for the <style> element:

the :scoped pseudo selector is treated as a reference the style sheet's host element

the relative ID selector, e.g. #~child, is resolved within given namespace:

Code

<div namespace>

  <p id="child"></p>

  <div namespace>
    <p id="child"></p>
  </div>

  <style scoped>
  /* matches only the first "p" */
  #~child { color: red; }
  /* matches all "p" as usual */
  #child { background-color: whitesmoke; }
  </style>

</div>

where multiple identical scoped styles exists, you are able to add an (experimental) shared directive, as in <style scoped shared>, to get only the first instance processed and shared by all host as an adopted style sheet, to optimise performance:

Code
```
<ul>
  <li>
    <style scoped shared></style>
  </li>
  <li>
    <style scoped shared></style>
  </li>
</ul>
```

and here's how the scoped attribute works for the <script> element:

the this keyword is bound to the script's host element
the <script> element is (re)executed on each re-insertion into the DOM

Did that just pique your interest?

Try Now!

Simply include the OOHTML Polyfill and go ahead with scoping!

The "Single Page Application" example in the list of examples is something I'm sure you wanna try!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

You may also want to visit the Style and Script Scoping section in the project README!

And we'd like for you to share your thoughts and leave us a star 🌟!

Question 3: How Do You Re-Use Components?

A component architecture is a way to organise code! That involves being able to work on disparate pieces of an idea and have them automatically come together! It's about the biggest need frameworks fill! Now, something like that in the HTML/DOM land is the <defs> and <use> system in SVG!

You could mention the <template> element which affords us a place to define reusable pieces of markup. But the <template> element is really only half the idea of a templating system; the remaining half left out to imperative DOM APIs:

let fragment = document.querySelector('template').content;
document.body.appendChild(fragment.cloneNode(true));

And when it comes to dealing with remote contents: a different set of APIs:

fetch('/file.html').then(response => response.text()).then(content => {
  document.body.append(content);
});

This has meant doing half of the work in HTML and half in JS!

We do have the HTML Modules proposal, but that's still a JavaScript feature, not HTML!

But here's what I have always wanted: a simple define-and-use system in HTML, just as with the SVG <defs> and <use> concept!

Introducing: HTML Imports

HTML Imports is a system for templating and reusing objects - in both declarative and programmatic terms! It extends the language with a definition attribute — def, complements that with a new <import> element, and has everything working together as a real-time module system!

Here, we get a way to both define and reuse a snippet within same document, exactly as with the SVG <defs> and <use> system we've always had:

<head>

  <template def="foo">
    <div></div>
  </template>

</head>
<body>

  <import ref="foo"></import>

</body>

...while optionally supporting remote content without a change in paradigm:

<head>

  <template def="foo" src="/foo.html"></template>

</head>
<body>

  <import ref="foo"></import>

</body>

Module Definition

You use the def attribute to expose a <template> element, and optionally, its direct children, as definition:

<head>

  <template def="foo">
    <div def="fragment1">A module fragment that can be accessed independently</div>
    <div def="fragment2">Another module fragment that can be accessed independently</div>
    <p>An element that isn't explicitly exposed.</p>
  </template>

</head>

And you are able to nest modules nicely for code organisation:

<head>

  <template def="bar">
    <div def="fragment1"></div>

    <template def="nested">
      <div def="fragment2"></div>
    </template>
  </template>

</head>

Remote Modules

We shouldn't need a different mechanism to work with remote content.

Here, OOHTML extends the <template> with an src attribute that lets us have self-loading <template> elements::

<template def="bar" src="/foo.html"></template>

└ Loaded file:

-- file: /foo.html --
<div def="fragment1"></div>
<template def="nested" src="/nested.html"></template>

└ Sub-loaded file:

-- file: /nested.html --
<div def="fragment2"></div>

And you can lazy-load modules using the loading="lazy" directive; meaning that loading doesn't happen until the first attempt to access the given model:

<template def="foo" src="/foo.html" loading="lazy"></template>

Module Imports

You use the <import> element for declarative module imports:

Notice the before and after

<body> <!-- Before -->
  <import ref="/foo">Default content</import>
</body>

<body> <!-- After -->
  <div def="fragment1"></div>
  <div def="fragment2"></div>
  <p></p>
</body>

<body> <!-- Before -->
  <import ref="/foo#fragment1">Default content</import>
</body>

<body> <!-- After -->
  <div def="fragment1"></div>
</body>

and the HTMLImports API for programmatic module imports:

const result  = document.import('/foo#fragment1'); // module:/foo#fragment1, received synchronously
const divElement = result.value;

document.import('/foo#fragment1', divElement => {
  console.log(divElement); // module:/foo#fragment1, received synchronously
});

document.import('/bar/nested#fragment2', divElement => {
  console.log(divElement); // module:/bar/nested#fragment2;
});

Scoped Modules

Just as with scoped styles and scripts above, you are able to scope <template> elements to create an object-scoped module system:

Notice how paths are resolved as to whether globally or relatively.

<section> <!-- module host -->

  <template def="foo" scoped> <!-- Scoped to host object and not available globally -->
    <div def="fragment1"></div>
  </template>

  <div>
    <import ref="foo#fragment1"></import> <!-- Relative path (beginning without a slash), resolves to the local module: foo#fragment1 -->
    <import ref="/foo#fragment1"></import> <!-- Absolute path (beginning with a slash), resolves to the global module: /foo#fragment1 -->
  </div>

</section>

Paths are also resolved the same way on the HTMLImports API:

// Showing relative path resolution
document.querySelector('div').import('foo#fragment1', divElement => {
  console.log(divElement); // the local module: foo#fragment1
});

// Showing absolute path resolution
document.querySelector('div').import('/foo#fragment1', divElement => {
  console.log(divElement); // the global module: foo#fragment1
});

Consider:

Custom elements can now, sometimes, be designed to have their logic decoupled from markup:

<my-element> <!-- module host -->

  <template def="foo" scoped> <!-- Scoped to host object and not available globally -->
    <div def="fragment1"></div>
  </template>

</my-element>

and in JS:

customElements.define('my-element', class extends HTMLElement {
  connectedCallback() {
    const { value: divElement } = this.import('foo#fragment1'); // the local module: foo#fragment1
    this.shadowRoot.appendChild(divElement.cloneNode());
  }
});

Cool Yet?

This is something you may find really interesting! You'll find that there's a whole lot that a declarative define-and-use system in HTML can change in your daily workflow!

Simply grab the OOHTML polyfill and game on with some of the examples!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

Once you're comfortable with the basics, you may also want to visit the rest of the story in the HTML Imports section in the project README!

And as before, we'd like for you to share your thoughts and leave us a star 🌟!

Question 4: How Do You Pass Data/Props?

In a hierarchy of components, parent components often need to pass data down to child components or, in some scenarios, just expose certain data for consumption by anyone, including child components!

On the first scenario, frameworks that do string concatenation for rendering, e.g. Lit, often let you do parent-to-child data-passing via markup/data interpolation, and everything together is parsed into what becomes the DOM:

// Example in Lit
render() {
  return html`
    <h1>Parent Component</h1>
    <!-- use the dot (.) syntax to pass data as a property -->
    <child-component .data=${this.childData}></child-component>
  `; 
}

By contrast, data-passing on an already constructed DOM tree is really what we're talking about when it comes to the DOM!

Interestingly, you could do something as simple as setting arbitrary properties directly on child elements from within a parent element:

// Inside a custom element
connectedCallback() {
  const child = this.querySelector('#child');
  child.data = this.childData;
}

Only that this isn't very elegant, given that, again, we would be littering an element's root namespace with application data and potentially conflicting with native DOM properties and methods! This is to say, we need a better idea around here!

Introducing: The Bindings API

This is a simple, read/write, data object exposed on the document object and on DOM elements as a bindings property:

// Read
console.log(document.bindings); // {}
// Modify
document.bindings.app = { title: 'Demo App' };
console.log(document.bindings.app); // { title: 'Demo App' }

const node = document.querySelector('div');
// Read
console.log(node.bindings); // {}
// Modify
node.bindings.style = 'tall-dark';
node.bindings.normalize = true;

This is the same Bindings API we introduced in the data-binding section! But it happens to be designed for arbitrary state management - which fits right in with our case:

// Notice our use of the Bindings API 
connectedCallback() {
  const child = this.querySelector('#child');
  child.bindings.data = this.bindings.childData;
}

And above, if we went further to take advantage of the namespace API, we'd be able to access the said child elements declaratively, as against having to imperatively query the DOM:

// Notice our use of the Namespace API 
connectedCallback() {
  this.namespace.child.bind(this.bindings.childData);
}

or...

// We could destructure the above for a more beautiful code
connectedCallback() {
  let { child } = this.namespace;
  let { childData } = this.bindings;
  child.bind(childData);
}

Now as a perk, we get reactivity for free on all of the moving parts above: a way to observe when this.bindings.childData changes and a way to observe when this.namespace.child changes:

// Notice the observers
connectedCallback() {
  const bind = (child, data) => child.bind(data);
  // bind current values
  bind(this.namespace.child, this.bindings.childData);
  // Bind new values on change
  Observer.observe(this.bindings, 'childData', (e) => bind(this.namespace.child, e.value));
  Observer.observe(this.namespace, 'child', (e) => bind(e.value, this.bindings.childData));
}

Better yet, we are able to have that same reactivity happen declaratively if we implemented our connectedCallback() method as quantum function:

// Notice the double star
quantum connectedCallback() {
  let { child } = this.namespace;
  let { childData } = this.bindings;
  child.bind(childData);
}

And when not dealing with class instances, we are able to achieve the same logic, and same reactivity, using a scoped, quantum script:

<div namespace>

  <div id="child"></div>

  <script scoped quantum>
    let { child } = this.namespace;
    let { childData } = this.bindings;
    child.bind(childData);
  </script>

</div>

Good Job There?

You'll find that it makes it easier to reason about state management in the DOM, and provides a better way to write Web Components!

Simply include the OOHTML polyfill from a CDN and game on!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

And here's one take-home idea for a Single Page Application: the code below, wherein on each navigation to a URL, data is programmatically fetched (via fetch()), then JSONed to a JavaScript object and held as global state at document.bindings (which you're able to directly render anywhere in the page):

<!DOCTYPE html>
<html>

  <head>
    <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
    <script>
      async function route() {
        const data = await fetch(`http://localhost:3000/api/${ location.hash.substring(1) }`).then(response => response.json());
        document.bind({ pageTitle: data.pageTitle });
      }
      window.addEventListener('hashchange', route);
    </script>
  </head>

  <body>
    <h1>Page title is: <?{ pageTitle }?>.</h1>
  </body>

</html>

(Now, you may fuse that with the "Single Page Application" example in the list of examples to implement individual page layouts for each URL!)

Maybe you'll have some thoughts to share!

You may also want to visit the Bindings API section on the project README to learn more! And do remember to leave us a star 🌟!

Introducing: The Context API

While we're able to explicitly bind data on DOM nodes using the Bindings API, giving parents a way to pass data down to child components, sometimes the goal is to just expose certain data at a certain level in the DOM tree for consumption by anyone, including child components! This time, instead of the below:

// Inside "parent" component
connectedCallback() {
  let { child } = this.namespace;
  let { childData } = this.bindings;
  child.bind(childData);
}

we now want "child" to request said data from "context":

// Inside "child" component
connectedCallback() {
  let parent = this.parentNode;
  let { childData } = parent.bindings;
  this.bind(childData);
}

much like an inversion of control!

Now, while we've used the immediate parent node for our data source above, it is sometimes impossible to predetermine the exact location up the tree to find said data, in which case, the idea of walking up the DOM tree mechanically - this.parentNode.parentNode... - not only creates a tight coupling between components, but also fails quickly!

That's where a Context API comes in!

You'd find the same philosophy with React, and something similar with Lit, and perhaps something of same sort with other frameworks!

In our case, we simply leverage the DOM's existing event system to fire a "request" event and let an arbitrary "provider" in context fulfil the request. All of this is made available via an API named contexts, exposed on the document object and on DOM elements!

Here, we get a contexts.request() method for firing requests:

// ------------
// Get an arbitrary
const node = document.querySelector('my-element');

// ------------
// Prepare and fire request event
const requestParams = { kind: 'html-imports', detail: '/foo#fragment1' };
const response = node.contexts.request(requestParams);

// ------------
// Handle response
console.log(response.value); // It works!

and a contexts.attach() and contexts.detach() methods for attaching/detaching providers at arbitrary levels in the DOM tree:

// ------------
// Define a CustomContext class
class FakeImportsContext extends DOMContext {
  static kind = 'html-imports';
  handle(event) {
    console.log(event.detail); // '/foo#fragment1'
    event.respondWith('It works!');
  }
}

// ------------
// Instantiate and attach to a node
const fakeImportsContext = new FakeImportsContext;
document.contexts.attach(fakeImportsContext);

// ------------
// Detach anytime
document.contexts.detach(fakeImportsContext);

And everything comes as one standardized API for looking up the document context for any use case! In fact, the Context API is integral to the Namespace API and the Data Binding and HTML Imports features in OOHTML!

Now, we are able to use the Context API to retrieve the said "binding" in our parent-child scenario earlier:

// Inside "child" component
connectedCallback() {
  const requestParams = { kind: 'data-binding', detail: 'childData' };
  let { value: childData } = this.contexts.request(requestParams);
  this.bind(childData);
}

Exciting Yet?

This is covered in detail in the Context API section on the project README!

It goes without saying that this too is possible today using the OOHTML polyfill!

Polyfill

<head>
  <script src="https://unpkg.com/@webqit/oohtml/dist/main.lite.js"></script>
  <!-- other code -->
</head>

Take that as an invitation to explore and share your thoughts! And we'll be more than delighted to have you leave us a star 🌟!

That's now a wrap!

But that's probably a lot to unpack! Well, think of OOHTML as not one, but a suite of small, interrelated proposals with one agenda: towards a more dynamic, object-oriented HTML! While features may be discussed or explored individually, the one agenda helps us stay aligned with the original problem!

As with everything on the web, your contribution is how this gets better!

The polyfill is actively developed and kept in sync with the spec. You'll find that it goes a long way to help us not think in a vacuum! And it's gone as far as help us build interesting internal apps! (And that's a go ahead to give yours a shot! Not to mean that there aren't bugs there waiting to be discovered!)

Finally, here we go:

webqit / oohtml

Towards a more dynamic and object-oriented HTML.

OOHTML

Explainer • Features • Modular HTML • HTML Imports • Data Binding • Data Plumbing • Implementation • Examples • License

Object-Oriented HTML (OOHTML) is a set of features that extend standard HTML and the DOM to enable authoring modular, reusable and reactive markup - with a "buildless" and intuitive workflow as design goal! This project revisits the HTML problem space to solve for an object-oriented approach to HTML!

Building Single Page Applications? OOHTML is a special love letter! Writing Web Components? Now you can do so with zero tooling! Love vanilla HTML but can't go far with that? Well, now you can!

Versions

This is documentation for OOHTML@4. (Looking for OOHTML@1?)

Status

Actively maintained
A working implementation
Proposed at the WICG
Open to contributions

Implementation

OOHTML may be used today. This implementation adheres closely to the spec and helps evolve the proposal through a practice-driven process.

…

View on GitHub

Re-Exploring Reactivity and Introducing the Observer API and Reflex Functions

Oxford Harrison — Sat, 01 Jul 2023 04:57:16 +0000

In the last few years, I have spent an insane amount of time thinking through reactivity on the frontend! In all that we've achieved in the space, the State of Reactivity today still leaves much to be desired:

reactive primitives - in the various mechanisms for intercepting and reflecting change, there is still no easy way around the concept of change detection! From language-level primitives - proxies and accessors - to custom primitives, this has continued to be a tricky game!
syntax and mental model - in the overall language and programming paradigm around the idea, there's still no ergonomic and mentally efficient way to express reactive logic! (Much rigor here often as we try to model normal logic using functions; or try to wield a compiler!)

These challenges greatly impact our work, sometimes in less obvious ways, given how much of the frontend paradigm space this useful magic now occupies, and thus, how much of it accounts for the code we write and our overall tooling budget! For a paradigm so intrinsic to frontend, it couldn't be more compelling to revisit the various painpoints!

To this end, I took a stab:

at re-exploring the fundamental idea of change detection in JavaScript (i.e. reactive primitives) and the overall language and programming paradigm around reactivity (i.e. syntax and mental model) for some "cleaner" magic!
at re-exploring full language support for all of it in each case, given how the idea seems to suffer from meager language support and thus, remains, for the most part, externalized to tooling!

This is a pretty long article - with 3 main sections:

Re-Exploring Change Detection (The design discussion around the Observer API)
Re-Exploring the Language of Reactivity (The design discussion around Reflex Functions)
The Duo and the Prospect for Reactivity

Show Full Outline

Re-Exploring Change Detection
- Accessors and Proxies; the Tricky Parts
  - Accessors Got a Flexibility Problem There
  - Proxies Got an Identity Problem There
  - The Whole Idea of Internal Traps
  - The Whole Idea of "Magic Objects"
- A Way to Think About the Problem
- Rediscovering Prior Art and Nailing the Problem
- Introducing the Observer API
  - An Overview
  - Featuring Path Observability
  - Documentation
  - The Project
Re-Exploring the Language of Reactivity
- The Toll On Syntax and Mental Model
  - Solving for Syntax
  - Solving for Mental Model
  - Solving for Both
- Bringing Reactivity to Imperative JavaScript and Nailing the Problem
- Introducing Reflex Functions
  - An Overview
  - Change Propagation
  - Documentation
  - The Project
The Duo and the Prospect for Reactivity
Acknowledgements
References

Update Jan, '24

Reflex Functions has had a major update since this initial announcement! This includes:

A name change: being now Quantum JS!
A major API revamp: majorly in the return values of Quantum Functions - from being a tuple [ value, reflect ] = sum() to being a "State" object state = sum()!
An entire compiler overhaul towards a more runtime-driven form of reactivity!

Generally, while the ideas shared here under the term "Reflex Functions" are largely the same with its Quantum JS successor, there are important differences in the implementation!

SECTION 1/3

Re-Exploring Change Detection

Change detection comes as the first challenge to reactivity in JavaScript!

The general idea deals with detecting and responding to changes made to JavaScript objects to drive synchronization - sometimes data binding, but generally, any functionality that is dependent on object state. This in JavaScript has been approached in many different ways past and present.

This wouldn't be about Signals and Hooks, because - paradigm mismatch:

Those "functional" primitives are designed with the Functional Programming paradigm in mind, where immutable data and pure functions are emphasized. Object observability, on the other hand, is more closely associated with the Object-Oriented paradigm, where mutable state and object interactions are the idea. And whereas the former basically represents an intermediary event system, the latter deals with object-level change detection via native mechanisms.

Meanwhile, later in the second half of this article, we will be touching on Signals and Hooks.

Compare also: Mutability vs. Immutability

Immutability is a programming principle that treats data as unchangeable. Here, the idea of change is modelled by functions that repesent a pipeline of transformations for whole data wherein each transformation creates a new instance, and observability happens at the event level, well, instead of at the fine-grained level. Mutability, on the other hand, embraces the concept of change, wherein objects are modified in place.

Historically, certain techniques have had their day here! For example, dirty-checking was the technique in Angular; and custom pub/sub mechanisms were the idea in Backbone Models and Ember Models. These and probably other approaches at the time have now falling out of favour for newer, more straightforward solutions!

Accessors and Proxies; the Tricky Parts

Today, object accessors (since ES5) and proxies (since ES6) have greatly revolutionized the problem space! Being integral to the language, these primitives constitute the only way to detect change at the program level:

Accessors

const person = {
  // name accessors
  get name() { ... },
  set name(value) { ... },
  // age accessors
  get age() { ... },
  set age(value) { ... },
};

Proxies

// Original object
const $person = {
  name: 'John',
  age: 30,
};
// Proxy wrapper
const person = new Proxy($person, {
  get(target, key) { ... },
  set(target, key, value) { ... }
});

Such that we're able to hide a ton behind a dot syntax:

person.name = 'James';
person.age = 40;

Giving a clean interface where you'd normally have needed intermediary get()/set() mechanisms is perhaps what makes these so nifty that you definitely want to use them! But then comes additional details to walk through, where you end up with an unexpected amount of boilerplate and a tricky implementation!

Let's add here that there's something on the horizon: Decorators! (On which Dr. Axel Rauschmayer takes a deep dive in their latest, stage 3, syntax.) But it turns out, while decorators might be solving something in the broader scope of meta programming in JavaScript, they aren't adding anything to change detection other than syntax sugars over existing possibilities! So, if the question is object observabilty, I'm the least excited about decorators!

Accessors Got a Flexibility Problem There

It's probably the biggest deterrent to using them: the inflexibility of working per-property and not supporting on-the-fly properties! This requires you to make upfront design decisions around all the possible ways an object could change at runtime - an assumption that's too unrealistic for many dynamic applications; one that fails quickly if you tried!

Accessors were the primary reactive mechanism in Vue 2, and that's probably the best place to see their reallife limitations - the biggest of which is perhaps their inability to detect new property additions! This became a big enough issue for the framework that they didn't hesitate to move away when they had the chance to!

Proxies Got an Identity Problem There

It turns out that for every "proxy" instance, you practically have to account for two different object identities, and that comes very problematic for code that relies on type checks (x instanceof y) and equality checks (x === y). Heck, even two proxy instances with the same object wouldn't equate, due to their different identities!

As David Bruant would have me do here, let's add that with some hack, you could get "instanceof" working correctly: via the getPrototypeOf trap. (But I guess that doesn't make the whole idea any less tricky but more!)

And maybe let's reference Tom van Cutsem on the concept of membranes and Salesforce's implementation of the idea - wherein you're able to have all references to the same target return the same proxy instance, and thus have references pass "equality checks"! (But I guess membranes aren't what you're really out here to do!)

All of this is probably another way to see the problem for reactivity with proxies: tricky, no easy way out!

The Whole Idea of Internal Traps

It comes as a gap between the communication model that those internal methods enable and the typical model around reactivity: these primitives don't bring with them the regular subscription model wherein anyone can subscribe to changes on a subject! They just give you a way to intercept - in terms of internal traps - but not a way to observe from the outside - in something like a .subscribe() method! And in however you go from here, this becomes a tricky game all the way:

Tricky if you were to take those internal methods for generic "observers"! In the case of accessors, you couldn't have another "observer" without re-defining a property, and thus inadvertently displacing an existing one! And in the case of proxies, while you wouldn't be able to displace traps anyway as those are finalized at instance time (i.e. closed-in by the constructor), you also couldn't have another "observer" without, this time, creating another proxy instance over same object, and thus inadvertently ending up with multiple redundant proxies that are each tracking very different interactions - giving you a leaky, bypassable tracking mechanism each time!

Accessors
```
Object.defineProperty(person, 'age', { set() { /* observer 1 */ } });
// Singleton Error: A new observer inadvertently breaks the one before
Object.defineProperty(person, 'age', { set() { /* observer 2 */ } });
```
Proxies
```
let $person1 = new Proxy(person, { /* observer 1 */ });
goEast($person1);
// Singleton Error: A new observer inadvertently creates redundancy
let $person2 = new Proxy(person, { /* observer 2 */ });
goWest($person2);
```
Turns out, those internal methods don't at all add up to a reactive system! Until you actually build one, you don't have one!

Tricky in going from those internal methods to a "reactive system"! You'd need to build an event dispatching mechanism that those internal methods get to talk to, and that couldn't happen without having to walk through new technical details! And there lies many pitfalls! For example, it's quick and easy to inadvertently introduce a breaking change to public object interfaces - e.g. window.document - by polluting/patching their namespace with setters and getters, or with something like a .subscribe() method!

Accessors

Object.defineProperty(window.document, 'custom', { set() { /* observer 1 */ } });

const person = {
  // Namespace pollution
  _callbacks: [],
  subscribe(callback) {
    this._callbacks.push(callback),
  },
  // Accessors
  get name() { ... },
  set name(value) { ... },
};

Proxies

const _callbacks: [],
function subscribe(callback) {
  _callbacks.push(callback);
}
const $person = new Proxy(person, {
  get(target, key) {
    // Namespace pollution
    return key === 'subscribe' ? subscribe : target[key];
  },
  set(target, key, value) { ... }
});

Turns out, there's no direct/neat path to go from "internal methods" to "reactive system"!

The Whole Idea of "Magic Objects"

That comes as the narrative for the form of reactivity that these techniques enable, and the implication is in terms of the final model that these magic objects create: a system where observability is an object-level feature, or in other words, an object-level concern! This comes with much complexity and multiple points of failure at the granular level!

Being a concern at such a granular level of your architecture, "reactivity" now tends to govern every detail of your work: objects have to be purpose-built for observability, and existing ones retrofitted; and in all you do, you must be explicit about reactivity! And because it is hard to gain and easy to lose, you also have to be conscious about how each object flows through your application; and there lies the footguns and bottlenecks:

Tricky to account for across the various points of interaction! Given a system where functionality relies on data and reactivity being bundled together and passed around together, it happens so easily that the latter is lost on transit without notice! That becomes a big problem in any fairly-sized codebase!

Accessors
```
// Using the accessor-based ref() function in Vue.js
import { ref, watchEffect } from 'vue';

const count = ref(0); // Reactivity gained
watchEffect(() => {
  console.log('Count value changed:', count.value);
});

function carelessHandling(count) {
    delete count.value;
    count.value = 2;
}

carelessHandling(count); // Reactivity lost
```
It turns out, any reactive system based on "magic objects" is susceptible to failure at the granular level!
Tricky in dealing with object trees! You couldn't gain reactivity beyond the surface without some form of recursive transformations, and yet all of that would still be subject to the assumption that your object tree is a static structure! Things get extra tricky this time because, in reality, that assumption isn't always the case!

Proxies
```
// Using the depth-capable reactive() function in Vue.js
import { reactive } from 'vue';

// Creating a reactive object tree
const reactiveTree = fetch(resourceUrl).then(res => res.json()).then(json => {
  return reactive(json);
});

// Problem
reactiveTree.entries.push({ ... });
```
Turns out, any reactivity gained from stitching proxies or accessors together is most often than not unpredictable and rickety!

A Way to Think About the Problem

Proxies and accessors constitute the only native way today to do mutation-based reactive programming - not particularly because they perfectly reflect the desired approach, but because they're the only native way! However, this isn't a case of poorly designed language features, this is rather about usecase mismatch! Whereas proxies and accessors have their perfect usecases across the broader subject of meta programming, reactive programming represents a usecase not well captured; or in other words: something that isn't very much their usecase!

Proxies, for example, are mind blowing in what they can do across the broader subject of meta programming, having had a massive technical and academic work go into their design (as can been seen from ES-lab's original design document), and it is on this backdrop they seem to be the silver bullet to reactivity! But it turns out that it is on just a subset of that - typically something around three internal traps: get, set, deleteProperty - that we're building our reactive world, and well, the rest of which isn't fully captured in the design of proxies!

While object observability as a technique may have an overlap with the usecase for proxies and accessors, it isn't the main thing these were designed for, and these primitives conversely don't come ideal for the problem case! If I were to represent that mathemaically, that would be 25% overlap, 75% mismatch:

This leaves us with a desire to find something made for the problem! And it turns out, this isn't the first time we're hitting this wall!

Rediscovering Prior Art and Nailing the Problem

This is where history helps us!

Amidst the remnants lies a design precedent to object observability waiting to be re-discovered: the Object.observe() API, which was depreciated in November 2015! It probably was ahead of its time, but in its 8 years in the mud, there's something time has taught us: the platform needs native object observability, and it's okay to have things coexist for different usecases!

Consider the approach in this API:

Object.observe(object, callback);

Notice that, now, the tracking mechanism is separate from the objects themselves, and what's more, universal for every object! (It's much like the upgrade from .__defineSetter__() and .__defineGetter__() to Object.defineProperty()!) And this basically solves all of our problems today with primitives!

Now, we can put all of that "magic object" idea behind us: observability shouldn't have to be an object-level feature (or object-level concern)!
- We no more have to be explicit about reactivity; or be governed by that concern!
- It shouldn't have to be gained or lost; or be passed around together with data!
Now everything just works on the fly; no more upfront work nor dealing with internal traps!
- No more patching objects, nor transforming object trees!
Now we can interact with real objects across our codebase: things no more rely on intermediary functions or wrappers!
- No more accounting for two different object identities or hacking around that!
- No more dealing with leaky, bypassable tracking mechanism!

And doesn't that nail it? Why shouldn't we explore this further?

Having struggled around object observability myself over the years, I find it necessary to re-explore this prior art today as a new project: the Observer API!

Introducing the Observer API

This is a new effort that re-explores object observability along the lines of Object.observe(), but this time, with a more wholistic approach! It takes a leap at what could be a unifying API over related but disparate APIs like Object.observe(), the "Proxy traps" API, and the Reflect API (which is a complementary API to proxies)!

This is an upcoming proposal!

An Overview

As an improvement over its original form - Object.observe(), the object observability API is being re-imagined!

Whereas the previous API lived off the global "Object" object, the new idea is to have it on the global "Reflect" object as if being one of the missing pieces in a puzzle! (But this is subject to whether everything considered here still falls within the scope of the Reflect API.) But that's just one of two possibilitis considered! The other is to have it on a new Reflect-like object called Observer, which is, this time, a fully-featured reactivity API!

While that unfolds, a prototype of the Observer API exists today as Observer, featuring a superset of each Reflect method in addition to other specialized APIs. Much of the code here is based on this Observer namespace.

Observer API	Reflect API
`apply()`	`apply()`
`construct()`	`construct()`

`observe()`	`-`

`set()`	`set()`
`setPrototypeOf()`	`setPrototypeOf()`

Being fully compatible with the Reflect API, the Observer API can be used today as a drop-in replacement for Reflect.

To begin, here's the observe() method:

└ Signatures

// Observe all properties
Observer.observe(object, callback);

// Observe a list of properties
Observer.observe(object, [ propertyName, ... ], callback);

// Observe a value
Observer.observe(object, propertyName, inspect);

└ Handler

function callback(mutations) {
  mutations.forEach(inspect);
}

function inspect(m) {
  console.log(m.type, m.key, m.value, m.oldValue, m.isUpdate);
}

From here, additional features become necessary! We discuss some of these here and link to the rest.

Featuring Path Observability

A much needed feature for Object.observe() was path observability! The void that initially having it left out created required much boilerplate to fill. A few polyfills at the time (e.g. Polymer's observe-js) took that to heart. And that certainly has a place in the new API!

This time, instead of following a string-based "path" approach - level1.level2 - a path is represented by an array - a Path array instance:

const path = Observer.path('level1', 'level2');

An array allows us to support property names that themselves have a dot in them. And by using Observer's Path array instance, we are able to distinguish between normal "property list" array - as seen earlier - and actual "path" array.

The idea here is to observe "a value" at a path in a given tree:

// A tree structure that satisfies the path above
const object = {
  level1: {
    level2: 'level2-value',
  },
};

Observer.observe(object, path, (m) => {
  console.log(m.type, m.path, m.value, m.isUpdate);
});

object.level1.level2 = 'level2-new-value';

Console

type	path	value	isUpdate
`set`	[ `level1`, `level2`, ]	`level2-new-value`	`true`

And well, the initial tree structure can be whatever:

// A tree structure that is yet to be built
const object = {};

const path = Observer.path('level1', 'level2', 'level3', 'level4');
Observer.observe(object, path, (m) => {
  console.log(m.type, m.path, m.value, m.isUpdate);
});

Now, any operation that changes what "the value" at the path resolves to - either by tree extension or tree truncation - will fire our listener:

object.level1 = { level2: {}, };

Console

type	path	value	isUpdate
`set`	[ `level1`, `level2`, `level3`, `level4`, ]	`undefined`	`false`

Meanwhile, this next one completes the tree, and the listener reports a value at its observed path:

object.level1.level2 = { level3: { level4: 'level4-value', }, };

Console

type	path	value	isUpdate
`set`	[ `level1`, `level2`, `level3`, `level4`, ]	`level4-value`	`false`

If you were to find the exact point at which mutation happened in the path in an audit trail, use the event's context property to inspect the parent event:

let context = m.context;
console.log(context);

And up again one level until the root event:

let parentContext = context.context;
console.log(parentContext);

And you can observe trees that are built asynchronously! Where a promise is encountered along the path, further access is paused until promise resolves:

object.level1.level2 = Promise.resolve({ level3: { level4: 'level4-new-value', }, });

Documentation

Visit the docs for full details - including Timing and Batching, full API Reference, and more.

The Project

The Observer API is being developed as something to be used today - via a polyfill. The polyfill features all of what's documented - with a few limitations!

webqit / observer

A simple set of functions for intercepting and observing JavaScript objects and arrays.

The Observer API

Observe and intercept operations on arbitrary JavaScript objects and arrays using a utility-first, general-purpose reactivity API!

This API re-explores the unique design of the retired Object.observe() API and unifies that with the rest of JavaScript's metaprogramming APIs: Proxies, Reflect, Object!

Observer comes as one little API for all things object observability. (Only ~5.8KiB min|zip)

const obj = {};

// Observe all property changes
Observer.observe(obj, (mutations) => {
  mutations.forEach(mutation => {
    console.log(`${mutation.type}: ${mutation.key} = ${mutation.value}`);
  });
});

Observer.set(obj, 'count', 5);
Observer.deleteProperty(obj, 'oldProp');

Tip

Reactivity is anchored on its programmtic APIs…

View on GitHub

Show support with a star on github! Also, all forms of contributions are welcome at this time.

SECTION 2/3

Re-Exploring the Language of Reactivity

While we may have fixed object observability, we still need to find a way to actually write reactive logic - in how we normally would write JavaScript programs! There's a big difference here, and it's like the difference between playing individual musical notes and composing a song!

But while this constitutes a different problem from what object observability solves, they've got be related, and that's definitely where this journey culminates!

Come to writing application logic...

Here is how we learned to write applicatons (notice: in normal, imperative JavaScript):

let count = 5;
let doubleCount = count * 2;
console.log(doubleCount); // 10

and the challenge for reactive programming is to find a way to express the same such that the logic...

count -> doubleCount -> console.log(doubleCount)

...continues to hold even when a part of the dependency chain is updated!

This magic by itself has had no formal language, yet it has managed to become the most spoken language on the frontend today! But unfortunately, reactivity is a difficult language to speak:

This has involved compiler-driven syntaxes and whole new DSLs, and on a more general note, making the shift to writing applications under an entirely different programming paradigm: Functional Programming! And there lies the big problem for the average developer: that paradigm shift from "normal" JavaScript!

It just happens that there's no easy way around the idea! And when you look: what always goes for reactivity is syntax and mental model! Only, some approaches can be considered better off than others on either of those factors.

The Toll On Syntax and Mental Model

Enter the typical language of reactivity: Functional Programming...

Whereas imperative programs are written in literal terms and are based on a linear execution flow (a sequential, line-by-line evaluation of statements and control flow structures), functional programming requires a series of change detection primitives to model the given application logic!

Given our sample program in context, here's what the "functional" equivalent could look like across frameworks, using a pretty identical set of primitives:

React

import { useState, useMemo, useEffect } from 'react';

// count
const [count, setCount] = useState(5);
// doubleCount
const doubleCount = useMemo(() => count * 2, [count]);
// console.log()
useEffect(() => {
  console.log(doubleCount);
}, [doubleCount]);

// Update
setCount(10);

SolidJS

import { createSignal, createMemo, createEffect } from "solid-js";

// count
let [count, setCount] = createSignal(5);
// doubleCount
let doubleCount = createMemo(() => count() * 2);
// console.log()
createEffect(() => {
  console.log(doubleCount());
});

// Update
setCount(10);

Vue.js

import { ref, computed, watchEffect } from 'vue';

// count
const count = ref(5);
// doubleCount
const doubleCount = computed(() => count.value * 2);
// console.log()
watchEffect(() => {
  console.log(doubleCount.value);
});

// Update
count.value = 10;

Svelte

import { writable, derived } from 'svelte/store';

// count
const count = writable(5);
// doubleCount
const doubleCount = derived(count, $count => $count * 2);
// console.log()
doubleCount.subscribe(value => {
  console.log(value);
});

// Update
count.set(10);

MobX

import { observable, computed, autorun } from 'mobx';

// count
const count = observable.box(5);
// doubleCount
const doubleCount = computed(() => count.get() * 2);
// console.log()
autorun(() => {
  console.log(doubleCount.get());
});

// Update
count.set(10);

What do we have here? A common reactive language, and, well, the problem case: a programming paradigm shift!

What is in traditional JavaScript a literal variable declaration is in the functional approach a very different type of declaration:

// Variable
let items = ['one', 'two'];

// Read/write segregation
const [items, setItems] = createSignal(['one', 'two']);

Also, what is to traditional JavaScript a "mutable" world is to the functional approach an "immutable" world (or you end up defying the reactivity system):

// Mutation world
items.push('three');

// Immutable world
setItems([...items(), 'three']);

And the story goes the same for conditional constructs, loops, etc.!

The focus entirely shifts now from writing idiomatic JavaScript to following paradigm-specific constraints, and in addition, other implementation-specific details:

In the Signals family: that technique around fine-grained reactivity - wherein you have to carefully craft, or "group", logic "around data flow" to actually stay fine-grained. Also, that extra invokation syntax - count(), count.get(), count.value - that goes with what was supposed to be a literal variable.
In the Hooks family (React, Preact): the underestimated dependency array, etc.

All of this on top of the required diligence needed to manually model the control flow and dependency graph of your application logic - by piecing together multiple primitives! Anyone who's been there can tell how the many moving parts and high amount of abstraction and indirection significantly impact the authoring experience and cognitive load!

It turns out that this is a problem with any "runtime" magic as there must always be a thing to put up with: syntax noise and other ergonomic overheads, a difficult-to-grok execution model and other cognitive overheads, etc.!

This has been a strong enough case across frameworks that everyone has had to support their "functional" magic with some other type of magic that tries to solve either for syntax or for mental model!

Solving for Syntax

Vue, for example, will help you automatically unwrap certain refs used in .vue templates, and in a few other contexts. (The idea even made it as a more extensive syntax transform project!) Similarly, in React, work is underway to put memoization and more behind a compiler!

Svelte stands out here as it goes full-blown with the idea of a compiler to let you write whole logic in literal, imperative JavaScript in .svelte templates, on top of its "functional" core:

<script>
  let count = 0;
  $: doubleCount = count * 2;
</script>

<main>
  <button on:click={() => count += 1}>Double Count: {doubleCount}</button>
</main>

Only, it suffers from being unintuitive by the "twist" - the magic - that went into it: compiling from literal, linear syntax to a functional, non-linear execution model, thus defying the general expectation of the "imperative" paradigm it appears to go by!

For example, as documented:

<script>
export let person;

// this will update `name` when 'person' changes
$: ({ name } = person);

// don't do this. it will run before the previous line
let name2 = name;
</script>

And you can see that again here in how the program seems to go bottom-up:

<script>
let count = 10;

$: console.log(count); // 20

// Update
$: count = 20;
</script>

There goes the non-linear execution model behind the imperative syntax! So, it turns out, what you get isn't what you see when you look in the face of your code, but what's documented! Indeed, Svelte scripts "are nothing like 'plain JavaScript' yet people seem to be accepting of those and some even advertising them as such." - Ryan Carniato

Nonetheless, Svelte's vision underscores the strong case for less ergonomic overheads in plain, literal syntax - the bar that every "syntax" quest actually has in front of them! Only, this cannot be rational with a functional, non-linear core - as seen from its current realities!

Solving for Mental Model

React has long stood on what they consider a "simpler" rendering model:

Whereas in normal functional programming, changes propagate from function to function as explicitly modelled, React deviates from the idea in its "hooks" approach for a model where changes don't propagate from function to function as explicitly modelled, but propagate directly to the "component" and trigger a full "top-down" re-render of the component - giving us a "top-down" rendering model!

The pitch here is:

"You don't have to think about how updates flow through your UI anymore! Just rerender the whole thing and we'll make it fast enough." - Andrew Clark

And in relation to Signals:

"That's why I don't buy the argument that signals provide a better DX/mental model." - Andrew Clark

Which from this standpoint should remain highly-protected:

"I think you guys should be way more aggressive in pushing back against this stuff. Throwing out the react programming model and adopting something like signals (or worse, signals plus a dsl) is a huge step backwards and most people haven’t internalized that." - Pete Hunt

And when you look, you see a vision here for a "linear" execution model, much like the linear "top-down" flow of regular programs!

Only, it suffers from being counterintuitive by itself with the "twist" - the magic - that went into it: defying the general expectation of the "functional" paradigm it appears to go by! So, it turns out, what you get isn't what you see when you look in the face of your code, but what's documented, which for many devs makes the Signals approach more grokable, wherein "functional" is "functional":

"its funny, the first time i ever used signals wasn't for perf, it was for understanding where change happens easier. plus you get LSP references to change.

just seems.... most rational" - ThePrimeagen

Being as it may:

"The React thought leadership had long cultivated the narrative of 'our model is more correct', almost becoming a dogma. In reality, many engineers are simply more productive with the signals mental model - arguing on a philosophical level is pointless when that happens." - Evan You

And the whole idea seems to fall short again with the extra useMemo/useCallback performance hack needed to control the "re-render" mdoel!

Nonetheless, React's vision underscores the strong case for less mental overheads in terms of a linear execution model - the very thing we hope to regain someday! Only, this doesn't seem to be what function primitives can provide rationally - as seen from its current realities!

Solving for Both

Given the ongoing quest for reactivity without the ergonomic and mental overheads, the new bar for reactivity is to not solve for one and fall short on the other! We may not be hitting the mark in a single implmenentation now, but not until more and more people discover the power of compilers and make the full shift!

"Smart compilers will be the next big shift. It needs to happen. Look at the optimization and wizardry of modern C compilers. The web needs similar sophistication so we can maintain simpler mental models but get optimized performance." - Matt Kruse

In their ability to take a piece of code and generate a different piece altogether, compilers give us a blank check to write the code we want and get back the equivalent code for a particular problem, becoming the hugest prospect for reactive programming! Soon on the Language of Reactivity, we may never again talk about "functional" primitives or "functional" core! And there's only one place this leads: reactivity in just plain JavaScript; this time, in the literal form and linear flow of the language!

"It’s building a compiler that understands the data flow of arbitrary JS code in components.

Is that hard? Yeah!
Is it possible? Definitely.

And then we can use it for so many things…" - sophie alpert

There goes the final bar for reactivity! It is what you see given half the vision in Svelte: "syntax" (assuming a path fully explored to the end) and half the vision in React: "top-down flow" (assuming a path fully explored to the end); the point where all quests on the Language of Reactivity culminate!

But what if everything in this "reactivity of the future" thesis was possible today, and what's more, without the compile step?

Bringing Reactivity to Imperative JavaScript and Nailing the Problem

You'd realize that the idea of "building a compiler that understands arbitrary JS code" is synonymous to bringing reactivity to "arbitrary" JavaScript, and if we could achieve that, we could literaly have reactivity as a native language feature! Now this changes everything because that wouldn't be a compiler thing anymore!

Can we skip now to that fun part?

We explore this by revisiting where we've fallen short of bringing reactivity to "arbitrary" JavaScript and fixing that! Svelte comes as a good starting point:

In Svelte today, reactivity is based on the custom .svelte file extension! (Being what carries the idea of a reactive programming context in the application!)

calculate.svelte

<script>
  // Code here
</script>

But what if we could bring that reactive programming context to a function...
...for a handy building block that can be easily composed into other things?

function calculate() {
  // Code here
}

And next is how you express reactivity!

That, in Svelte today, relies on the dollar sign $ label - being another type of manual dependency plumbing, albeit subtle:

<script>
let count = 10;
// Reactive expressions
$: doubleCount = count * 2;
$: console.log(doubleCount); // 20
</script>

But what if we simply treated every expression as potentially reactive...
...having already moved into a reactive programming context?

function calculate() {
  let count = 10;
  // Reactive expressions
  let doubleCount = count * 2;
  console.log(doubleCount); // 20
}

And lastly, the execution model!

Updates propagate in Svelte today "functionally", and thus, in any direction, to any part of the program:

<script>
let count = 10;
// Reactive expressions
$: doubleCount = count * 2;
$: console.log(doubleCount); // 40
// Update
$: count = 20;
</script>

But what if we could get updates to propagate "top-down" the program...
...and actually regain the "top-down" linear flow of actual imperative programs?

function calculate() {
  let count = 10;
  // Reactive expressions
  let doubleCount = count * 2;
  console.log(doubleCount); // 20
  // Update
  count = 20;
}

Well, notice now that the update to count in the last line wouldn't be reactive! And that brings us to the question: does anything ever react to anything in this model? The answer lies in what "dependencies" mean in normal programs!

Conventionally, programs run in sequential, linear flow, along which references to "prior" identifiers in scope create "dependencies"! This means that statements should really only be responding to changes happening up the scope, not down the scope (as the case may be in Svelte today)! And that for a function scope would be: changes happening "outside" the scope - up to the global scope!

function calculate(count) { // External dependency
  // Reactive expressions
  let doubleCount = count * 2;
  console.log(doubleCount); // 20
}
let value = 10;
calculate(value);

count is now a dependency from "up"/"outside" the function scope - by which reactivity inside the function scope is achieved! But what would that even look like?

Imagine where the function has the ability to just "statically" reflect updates to its external dependencies:

// An update
value = 20;
// A hypothetical function
reflect('count'); // "count" being what the function sees

not particularly in their being a parameter, but particularly in their being a dependency:

let count = 10; // External dependency
function calculate() {
  // Reactive expressions
  let doubleCount = count * 2;
  console.log(doubleCount); // 20
}
calculate();

// An update
count = 20;
// A hypothetical function
reflect('count'); // "count" being what the function sees

And that brings us to our destination - this point where we've checked all the boxes; where:

The "reactive programming context" isn't some file, but a function - a handy new "reactivity primitive"!
Reactivity isn't some special language or special convention, but just JavaScript - the only real place you'd arrive if you tried!
Execution model isn't anything "functional" by which we defy the semantics of literal syntax, but "linear" - just in how imperative JavaScript works!

All of this in real life is Reflex Functions!

Introducing Reflex Functions

Reflex Functions are a new type of JavaScript function that enables fine-grained Reactive Programming in the imperative form of the language - wherein reactivity is drawn entirely on the dependency graph of your own code!

This is an upcoming proposal! (Introducing Imperative Reactive Programming (IRP) in JavaScript!)

An Overview

Reflex Functions have a distinguishing syntax: a double star notation.

function** calculate() {
  // Function body
}

See Formal Syntax for details.

Function body is any regular piece of code that should statically reflect changes to its external dependencies:

let count = 10; // External dependency
function** calculate(factor) {
  // Reactive expressions
  let doubled = count * factor;
  console.log(doubled);
}

Return value is a two-part array that contains both the function's actual return value and a special reflect function for getting the function to reflect updates:

let [ returnValue, reflect ] = calculate(2);
console.log(returnValue); // undefined

Console

doubled	returnValue
`20`	`undefined`

The reflect() function takes just the string representation of the external dependencies that have changed:

count = 20;
reflect('count');

Console

doubled
`40`

Path dependencies are expressed in array notation. And multiple dependencies can be reflected at once, if they changed at once:

count++;
this.property = value;
reflect('count', [ 'this', 'property' ]);

Change Propagation

Reactivity exists with Reflex Functions where there are dependencies "up" the scope to respond to! And here's the mental model for that:

┌─ a change happens outside of function scope

└─ is propagated into function, then self-propagates down ─┐

Changes within the function body itself self-propagate down the scope, but re-running only those expressions that depend on the specific change, and rippling down the dependency graph!

Below is a good way to see that: a Reflex Function having score as an external dependency, with "reflex lines" having been drawn to show the dependency graph for that variable, or, in other words, the deterministic update path for that dependency:

It turns out to be the very mental model you would have drawn if you set out to think about your own code! Everything works in just how anyone would predict it!

Plus, there's a hunble brag: that "pixel-perfect" level of fine-grained reactivity that the same algorithm translates to - which you could never model manually; that precision that means no more, no less performance - which you could never achieve with manual optimization; yet, all without working for it!

Documentation

Visit the docs for details around Formal Syntax, Heuristics, Flow Control and Functions, API, and more.

The Project

Reflex Functions is being developed as something to be used today - via a polyfill. The polyfill features a specialized compiler and a small runtime that work together to enable all of Reflex Functions as documented, with quite a few exceptions.

webqit / quantum-js

A runtime extension to JavaScript that unlocks Imperative Reactive Programming (IRP) in JavaScript!

Quantum JS

Overview • Creating Quantum Programs • Implementation • Examples • License

Quantum JS is a runtime extension to JavaScript that brings Imperative Reactive Programming to JavaScript!

What's that?

Overview

Where you normally would require certain reactive primitives to express reactive logic...

// Import reactive primitives
import { createSignal, createMemo, createEffect } from 'solid-js';

// Declare values
const [ count, setCount ] = createSignal(5);
const doubleCount = createMemo(() => count() * 2);
// Log this value live
createEffect(() => {
  console.log(doubleCount());
});

// Setup periodic updates
setInterval(() => setCount(10), 1000);

Quantum JS lets you acheive the same in the ordinary imperative form of the language:

// Declare values
let count = 5

…

View on GitHub

Show support with a star on github! Also, all forms of contributions are welcome at this time.

SECTION 3/3

The Duo and the Prospect for Reactivity

The magic that drives Frontend has got all the more powerful and completely intuitive!

From where I am sitting, I can see the amount of heavy-lifting that the Observer API will do for large swaths of today's usecases for "internal traps" and the whole "magic objects" idea! (It's probably time for that "revolution" that never took off!)

And we've certainly needed language support for reactive logic! Think of the different proposals out there around JavaScript-XML-like DSLs - which of course don't translate well to a native ECMAScript feature! And consider how much we've had to rely on compilers today for the idea!

Now that we can get "arbitrary" JavaScript to be reactive with Reflex Functions, a huge amount of that can be met natively! I am more than excited at how much we can regain using just JavaScript in each of those areas where we've relied on tooling!

I presume that experimenting around these new ideas, using the polyfill in each case, will furnish us new learnings around the problem! For now, I am happy to have started the conversation!

Now, did you know that these two reactive primitives could individually, or together, solve very unlikely problems? Here's where you can find early examples:

Reflex Functions / Usecases

If you'd find this an interesting idea, feel free to share!

Acknowledgements

Much thanks to David Bruant, Andrea Giammarchi, and others for the various feedbacks and insights on early drafts of this article.

References

What is reactivity? - a Vue.js definition
A Brief History of Reactivity - a Miško Hevery narrative
Object.observe() - web.dev
On the design of the ECMAScript Reflection API - ES-Lab
What is a membrane? - Tom van Cutsem
Observable Membrane - Salesforce
A Hands-on Introduction to Fine-Grained Reactivity - Ryan Carniato
The Quest for ReactiveScript - Ryan Carniato
Rethinking Reactivity - Rich Harris
React Forget - React
Reactivity Transform - Vue.js

Rethinking the Modern Web

Oxford Harrison — Sun, 12 Feb 2023 21:40:13 +0000

Frontend has an engineering problem!

It is one field that has forged about the most enviable tooling ecosystem by roughly every measure - from growth rate to technical wizardry - and along with that an incredible amount of thought leadership on its every move,

and yet at its bottom line:

the average website performance is only regressing with us!
the average developer productivity is only grinding to a halt!

So it turns out, all of that "vibrant ecosystem" isn't really translating to more "accessible", "functional" apps on the user front, and neither to more "speed" and "productivity" on the developer front!

The closer you get to Frontend's bottlenecks and to how much of that is tooling-induced the more you are wondering if we just have created more problems with our tools than we've solved!

The Counterintuitive Equations!
- ...on the user front
- ...on the developer front
Looking Back: The Paradigm Shift!
- ...from HTML to JavaScript
- ...from the web platform to abstractions
Towards a Faster Web: A New Quest!
Introducing: Web-Native Development!
- Explore with Me...
Notes
Acknowledgements
References

The Counterintuitive Equations!

Consider how self-defeating it gets...

...on the user front

Given real world network and device-capability factors, performance starts with shipping less bytes, especially when it coms to JavaScript - your most expensive asset. (Truth is, all of the extra things like instant navigations and transitions that will delight your users need your app to first be accessible!)

Unfortunately, SPA frameworks, who have this as their primary call, have their engineering model going the opposite way: towards more JavaScript! These may follow different programming paradigms... but all have got the same JavaScript bottom-line!

[HTML, CSS, JS] > Build_Step > [JS, JS, JS] e.g. Svelte, Vue
[JS, JS, JS] > Build_Step > [JS, JS, JS] e.g. React

Your final bundle here is the sum of your framework bundle, plus your essential JS, plus the induced (usually larger) JS from page structure, content, and styling - the obvious math to Frontend's terrifying payloads!

With your application's entire weight now culminating in JavaScript... and its non-JS aspects - page structure, content, and styling - now being accounted for in JS, you're now wrongly set up for the real world! It soon becomes clear how much of a knife edge you're on as your application becomes a real thing and starts to regress with every weight gained!

Not surprising... the idea of a "highly-optimized", "accessible", "functional" application quickly falls apart!

Reporting on this in the 2017 Real-world Web Performance Budgets (here), Alex Russell relates:

"I've seen teams that have just finished re-building on a modern tech stack cringe for an hour as we walk them through the experience of using their 'better', 'faster' experiences under real-world conditions."

Tim Kadlec also notes from field experience:

"What is clear: right now, if you’re using a framework to build your site, you’re making a trade-off in terms of initial performance—even in the best of scenarios. [...] If you are going to use one of these frameworks, then you have to take extra steps to make sure you don’t negatively impact performance in the meantime."

It is often excluded in the conversation, but many are winding up with more tooling-induced performance problems than they ever would otherwise! How entirely counterproductive!

...on the developer front

Performance is all about being able to tame complexity with the least amount of overhead. This calls for the abstractions/tools for managing complexity to hold to "a 'first do no harm' principle... make sure you are at least no less productive with overhead than you were without it." (Swyx)

Question is: are we really improving at managing complexity? Not when it feels like we're drowning in more complexity than ever, with an entire curriculum of deep programming concepts taking the place of what used to be the "HTML" kind of problems, and along with that a doze of compilers - that themselves have to cater to a number of new paradigms, magic syntaxes and dialects!

We must now rigor through the surprisingly high amount of code-level complexity, and pay the price of a compile-step (with Webpack and babel (or similar beast) and a barrage of configurations, plugins and extensions) to have a working web page! But that's not all, we must also bend along with mind-bending runtime behaviours (of hooks and friends), and debug through difficult-to-grok transforms of our code!

At this point... you're now entirely cracking a different grade of nut! Everything easy is hard again¹!

Jelan shares her frustration:

"I'm getting more and more frustrated with all the added layers of complexity needed to work with most common frontend frameworks. I’ve hit a point where it just doesn’t seem like the end justifies the means in the vast majority of cases anymore."

And Chris Coyer puts a fitting analogy to that:

"Ironically, while heaps of tooling add complexity, the reason they are used is for battling complexity. Sometimes it feels like releasing cougars into the forest to handle your snake problem. Now you have a cougar problem."

Look what was going to land you in the pit of success² winding you up with more complexity than you had at first! (On the essential/accidental³ scale, that's you having your "accidental" (tooling-induced) complexity trump your "essential" (real problem-space) complexity!) And again, that's utterly counterproductive!

If there's anything that's obvious, it is the one thing that ruins the experience each time: that "JavaScript"; this time, not the "JS" in the conventional "HTML, CSS, JS" approach, but the "JS" in our new "all-JS" equations:

the "all-JS bottom-line" engineering model - which brings all of page structure, content, and styling to JS (Build_Step => [JS, JS, JS])!
the "all-JS front-line" engineering model - which defaults to JS for authoring page structure, content, and styling ([JS, JS, JS] => Build_Step)!

Looking Back: The Paradigm Shift!

Signaled by React's Rethinking Best Practices pitch back in March 2013, frontend's new era has for the most part sat on what seems to be a universal axiom: the traditional web is awful; abstract it! Everything "traditional" about the web application story - from authoring to runtime - has since been doomed for an abstraction or a re-engineering - with the conventional "HTML, CSS, JS" and the DOM being the first casualty of war, and JavaScript being the new default instinct!

Whole new ecosystems of breakaway technologies have spun, each staying at a false dichotomy with web fundamentals and staying defensive about that! Here's for an insight into the thought process...

from HTML to JavaScript:

Mike Turley (on JSX): Why JavaScript is Eating HTML
Mark Dalgleish (on CSS-in-JS): A Unified Styling Language

from the web platform to abstractions:

Rich Harris: Why I don't use web components
Ryan Carniato: Maybe Web Components are not the Future?

...from HTML to JavaScript

A surprising exodus ensues!

"HTML has been the running punchline in the web development community lately. [...] We are in the midst of a very large problem in the web development field, where HTML is being left in the rearview mirror in place of JavaScript", writes Mark Steadman

React's infamous movement may have had an initial context of just JSX and, later, CSS-in-JS! But unfortunately, this has swept and overturned how a broader share of the frontend tooling space thinks! There's now multiple JS-first agendas everywhere you look - even on web platform proposal boards!

Consider as honorable mentions:

HTML Modules - a proposed "JS-first" import mechanism for "HTML".
CSS Modules - a "JS-first" import mechanism for "CSS".

Slowly, some of what belongs in the HTML problem space are now beginning to land as JavaScript feature proposals - sometimes downright namesquatting HTML!

But lest you asked how "HTML Modules" and friends don't solve a problem: maybe they do! But those would certainly not be the "HTML" kind of problems, but the "accidental limitation" type of problem in the JS-first world! Proposals like this are only symptomatic of having gone the "unconventional" way; you must endlessly build JS-first bridges, seek a JavaScript metaphor for every other HTML/CSS thing, and wish the rest of the web were colored yellow! (You can tell clearly how these aren't a feature, but a means to an end!) And notice how this comes even at the risk of tight-coupling a supposed general-purpose, DOM-agnostic programming language with the DOM! (See how HTML Modules in principle tortures the JavaScript language to produce DOM primitives!)

It has turned out to be not just more JavaScript being smuggled in through the back door, but also more HTML and CSS being thrown out the window! (Probably the greater harm!) Think the case regarding the removal - rather than improvement - of HTML Imports from the spec in favour of ES6 Modules - or possibly now the name-squatting HTML Modules above! It just happened that "the JS-centric zeitgeist won" this over - to put it in Brad Frost's words!

And is someone pointing to the removal of Scoped CSS in favour of Shadow DOM based CSS? We'll come to that!

JS-first has this way led to setting fire on many good thinking around HTML just to have them reborn in JavaScript! An irreversible, all-in proposition!

Needles to say is how therefore many highly-missing features in HTML aren't even in the radar on proposal boards! When you look around, really, how much "developer mind-share" does HTML have anymore to push those?

This has been so deep that if you dared to contend with the status quo and build a specialised career on real UI development skills that means less engineering, you'd find yourself in the marginalised half of the Great Divide and risk not having a career! (So then, many devs have had to acquiesce against their own will considering that there are bills to pay!)

...from the web platform to abstractions

Suddenly, the platform is abandoned!

Born out of contempt and mischaracterization of the web platform, the focus of modern abstractions has come to seem somewhat like re-engineering web languages, replicating platform features and APIs, and duplicating the browser's efforts! The whole idea of using the web platform has remained an unattractive option for breakaway technologies. To put it in Alex Russell's words, "The incentives of framework authors are not aligned with compatibility". I often see "web standards" being touted only where it makes a big news or where the performance gains are the incentive!

Looking back at how React and its ecosystem has stayed detached over the years, Mikeal Rogers relates:

"I remember when React was launched, the whole thing was about DOM diffing. The value of it is this virtual DOM thing. Then we made the DOM fast, and who gives a shit now. But we’re still using React because of – I don’t know. [...] And then now we have Web Components and they can’t adopt it, because they’re on their own pattern, so we can’t take this feature upgrade from the platform. [And] I think there's a ton of other examples of this where the platform starts to catch up, and then the frameworks can’t." - JS Party – Episode #89 | Changelog

We just seem to be disposed to trading the "norm" and yet searching for the "kind" in the abstract world!

And what has been a particularly sad implication of the framework-first culture? The more we've invested in breakaway technologies and side ecosystems, the less we've learned about our real problem space and all the opportunities to actually move the web forward! It turns out, HTML and its ecosystem remains undertooled to date, whereas the framework web continues to flourish! You're now almost guaranteed to get stranded as being in a desert land building anything in vanilla HTML and the DOM:

"It’s really depressing that most useful tools these day are made for React projects and/or require React knowledge to set up and use. This locks out many of us who are not using React for everything & who still prefer the vanilla route for their projects." - Sara Soueidan

You just realise how much everyone seems to have disconnected from the real "problem-space" type of problems and gotten buried in "abstract" problems, leaving us with framework-specific solutions to very common problems!

It turns out, undertooling remains a real deterrent for everyone who has craved the simplicity of the vanilla web. (And this should also be one valid challenge to the challenge thrown here by Remy Sharp when he asked: what's stopping you from using exactly [that] method today?)

With so much having gone wrong in the tooling space, what follows is only expected: a community at large suffering a terrible blind spot for web fundamentals! Ground truths are now debated everywhere, "best practices" are literally going numb (hi Pete), and standards are increasingly losing their place among developers! (Consider a survey: What do you know about Web Standards?.)

Go see a typical framework-speaking dev - even in their seniors of roles; go check the typical learning path of the coming generation; and go see what the modern Frontend job descriptions on job boards are saying! Surprisingly, the modern developer career is now little about web technologies themselves and all "about [the] intricacies of the most popular frameworks", to put it in Frédéric Bonnet's words! See that?

There's a destructive information gap and a culture erosion⁴ now also overtaking us!

Suffice to say, the past decade has been a life of alienation from conventional wisdom, almost entirely spent trading HTML - and burning our ships on the go - in an irreversible bet on JavaScript; turning our backs and throwing dirt on the web platform to bank futures on abstractions! Yet, only a few are really talking about the almost irreversible proposition of this decade-long shift...

"In elevating frontend to the land of Serious Code we have not just made things incredibly over-engineered but we have also set fire to all the ladders that we used to get up here in the first place." - Laura Buns

Towards a Faster Web: A New Quest!

As the ills of the Framework era become more and more palpable, the bells and whistles of the decade are losing their charms on people! Folks are now coming around on our lived reality, and many are actively going back on their life's bets in search of sanity, at the risk of bringing people with torches and pitchforks to their door!

Hajime Yamasaki Vukelic: What Got Me Writing Vanilla JavaScript again
Sam Magura: Why We're Breaking Up with CSS-in-JS

But what about the philosophy at the tooling layer: is it now time for a rethink? It depends on how much of a rethink you are asking of!

On the one hand, folks have been hard at work with identifying and addressing inherent overheads in the current system. (But of course, not with a view to rethinking the overall JS-first philosophy!)

For example, realising that the idea of shipping applications as just JavaScript wasn't working anything good on the user front, React and friends have since slid back to the idea of being able to send HTML over the network. This has called for major architectural rework (think React 18's Streaming SSR architecture); this being in addition to the existing idea of static site generation with build tools like Gatsby. Server-Side Rendering (SSR) and Static Site Generation (SSG) have been such a feat for the status quo!

So, right now, HTML and progressive enhancement have again become an attractive option to what has been the "all-JS or nothing" camp. (In fact, Remix has this as a new bragging right! In its own words: who knew?) Put together, all of the new takes here just represent a new game plan: a long walk back to the basics; a bit of a compromise on its initial takes, but a bit of some greater good: towards a faster, functional Frontend!

On the other hand, a new wave of innovation is taking center stage with a more ambitious take than the ongoing long, painful walk back to the basics: this time, the idea of an outright HTML-first, "progressive enhancement" architecture from start! (SitePen Engineering gives the perfect Intro to HTML-first Frontend Frameworks featuring Qwik, Marko, Astro, 11ty, Fresh, and Enhance; and this is absolutely worth your time!)

I find this especially interesting because, finally, everyone can see that the idea of a faster web didn't have to be the hard, compute-intensive, and yet elusive thing we've had for years! It is now clearer than ever how much of an absolute illusion we've been on... in staying fixated to JS and yet subscribed to HTML, albeit cleverly! You'd also realise: HTML over the network didn't have to be any "modern" wisdom or such a big deal of an innovation, or any feat! Not when this is something the web has had from genesis! (Streaming SSR from day 1 of PHP, anyone?)

Sad that we seem to only live after the fact when it comes to tooling, but thanks to the new wave of frameworks for shedding the light that openly challenges the status quo! Finally, we've come to a point that we can be hopeful about: the prospect of having "accessible", "functional" applications at half the price! (Hopefully related is how we've now for the first time - since the JS-making epoch - recorded a less steep increase in the amount of JavaScript shipped to users!)

So are we good now? Well, not just yet!

Not when it seems that we've come only halfway with the idea: sending pages as HTML but authoring them as JS; addressing the tradeoff problem and its overheads on the user front but leaving it unchanged on the developer front! Clear yet? Much of the "HTML" in our new HTML-first equations is still the tooling-yielded HTML, not the hand-authored HTML; same as before where HTML is treated as a compile target, an implementation detail... something you'd abstract to have a good DX! "Tooling" still remains Frontend's primary means to its "HTML" end!

This sheer idea of getting "HTML" behind a compile wall in favor of an abstraction is also where it begins to tell that we might yet be suffering the decade-long blindspot for the platform in a new way - embracing HTML for just the performance incentives and not for the whole idea of using the platform! Let's just throw it out there: Frontend still isn't aligned with using the platform to give legacy tooling their well-earned rest and take a correction back to simplicity!

Chad Fowler notes how deep-seated this is:

"The older I get, the more I realize the biggest problem to solve in tech is to get people to stop making things harder than they have to be."

So, how about just taking the plunge... and embracing the whole web platform thing? Now, we would not only be unlocking performance on both the user front and the developer front, we would be doing more: unleashing the web's full potential! (And what could be a more ambitious goal?)

Actually, this is why we're here! Can we skip to the good parts?

Introducing: Web-Native Development!

Take this as not the name of a new framework or some anti-framework movement, but as a playbook to unleashing the web's full potential.

Web-native development⁵ is an approach to web development that sees the web platform as an enabler in the whole application story, and in fact, a fundamental key to succeeding at each phase of that story - from authoring to runtime! This comes as a hard reset to the decade-long cultural shift and its pessimistic take on the web platform!

This is really about embracing and leveraging native web technologies, APIs, languages and conventions, etc, and minimizing tooling! For a fact, there comes a time in life when this is all you really want... get things done with less of the drama! "I've been in enough teams in my 20 years of programming to value this part almost more than anything else", writes Andrea Giammarchi!

As the platform and its technologies and languages advance, we often can find multiple opportunities "to shed a lot of this tooling" and defer to native approaches. (JS Party - Episode #89 | Changelog) Good to know is that tools are only as good as being solutions to unsolved problems, not solved problems! Anything in exception soon begins to change the narrative to something counterproductive! Thus, for all we've bet on custom tooling, we must now beat that Sunk Cost fallacy⁶ to explore what's natively available!

At the end of the day, web-native development gets you banking more on the web platform and less on abstractions! You're now on the winning side - rather than on the contending side - of the web's "moving" story! You're now winning as the platform unfolds!

George Katsanos explains how this allows us take back more brain cycles to finally go make something:

"It should be obvious to all of us that if we would focus all our efforts in the Platform and stop reinventing the wheel [...] we would have a lot more free time to focus on the real reason we are in this job which is to deliver fast, stable, secure, user-friendly interfaces."

So, where's a good place to get well-rounded with the web platform? Chrome's web.dev developer centre is a treasure trove of resources on key web design and development subjects, maintained by the Chrome team and other industry experts! (Here's the learning centre.) And for when you need to take the web platform by its individual technologies, here is MDN! (And here's the learning centre.) For showcases, here is one: Using the platform - a delightful piece of a story on going buildless with ES6 modules and going framework-free with Web components, written by Elise Hein!

Up next is: how far does this go in real life? Build a twitter clone with zero tooling? Uhh, that has never been the idea, and we may never get there! The web platform is anything but a framework of its own! At some point, we are going to have to need higher-level abstractions over native lower-level features! Where there seems to be a bit of a bad news is that the current state of HTML and the DOM makes that happen sooner; it isn't long into the journey before gaps and dips in the platform gets you into forced labour! But we can easily turn this around by investing along two lines:

Low-level tooling: platform-focused initiative to standardise and factor into the platform common web development architectures and paradigms. (We're overdue for native-level reactivity and a more empowering component model - just to mention a few!)
Higher-level tooling: community-focused initiative to extend the platform's low-level capabilities with "web-native" libraries and frameworks. (We need a new wave of modest abstractions that draw on the web platform and let us do the same!)

This is where I put my money where my mouth is! I'd like to take you on a few ideas I've been working on along these lines!

Open to some tooling ideas?

Explore with Me...

Our journey spans a series of posts! We begin with the underlying equations and move on to a showcase of the proposals and polyfils, the userland libraries and framework!

For the curious, here's a sneak peak into this project on github.

Notes

No frameworks were harmed in the creation of this article.

Acknowledgements

Special thanks to Alex Russell for the time spent reviewing this article!

References

Can You Afford It?: Real-world Web Performance Budgets (Alex Russell)
2022 Page Weight Report: JavaScript Bytes (HTTP Archive)
The Cost of JavaScript Frameworks (Tim Kadlec)
What drives Optimal Overhead? (Shawn Wang)
Glossary of Modern JavaScript Concepts: Part 1 (Kim Maida)
Rethinking Best Practices (Pete Hunt)
Why JavaScript is Eating HTML (Mike Turley)
A Unified Styling Language (Mark Dalgleish)
Why I don't use web components (Rich Harris)
Maybe Web Components are not the Future? (Ryan Carniato)
JavaScript Frameworks & The Lost Art of HTML (Mark Steadman)
why we’re breaking up with css-in-js (Brad Frost)
The Great Divide (Chris Coyer)
Web Components: The Long Game (Alex Russell)
Is modern JS tooling too complicated? (JS Party – Episode #89 | Changelog)
The web didn't change; you did (Remy Sharp)
Web Standards: The What, The Why, And The How (Amy Dickens)
From Classicism to Metamodernism — A Short History of Web Development Series' Articles (Frédéric Bonnet)
Hype Driven Development (Marek Kirejczyk)
What Got Me Writing Vanilla JavaScript again (Hajime Yamasaki Vukelic)
Why We're Breaking Up with CSS-in-JS (Sam Magura)
Intro to HTML-first Frontend Frameworks (SitePen Engineering)
Modern web apps without JavaScript bundling or transpiling (David Heinemeier Hansson)

Borrowing Frank Chimero's title: Everything Easy is Hard Again ↩
Defining The Pit of Success ↩
See Fred Brooks: No Silver Bullet (PDF) ↩
See Cultural Erosion ↩
"Web-native" is a recurring theme across multiple initiatives and paradigms: ↩
- Web Native - Ionic Team's presentation of the idea, but scoped to its Capacitor runtime.
- Web-Native - Frédéric Bonnet's presentation of the idea in The Postmodernist Period of The Third Age of Web Development.
- Modern Web - Guides, tools and libraries for modern web development built on web standards.
- The Stackless Way - Daniel Kehoe's optimistic take on web development that proposes we “use the platform” instead of frameworks and build tools.
- Buildless - Pascal Schilp's paradigm for creating production websites without a build process.
Some related tags:
- #buildless - DEV
- #vanilla - DEV
- #useThePlatform - twitter
See Sunk Cost Fallacy ↩

Reflex-Based Web Monetization API - Hackathon Submission

Oxford Harrison — Mon, 15 Jun 2020 05:07:12 +0000

The Web Monetization API has seen a lot of integration patterns that are tuned to many of the current frontend frameworks and developer tools. With each pattern serving only a specific community, I decided to explore a vanilla-level technology for Web Monetization for the rest of the web.

What I built

A Reflex-based Web Monetization API that is consumable as a plain, but reactive JavaScript object instead of an EventTarget interface. This is a new API that promises new foundational possibilities! Here is my introductory DEV post to this project, where I explained key problems solved and a few use cases.

To highlight, the goal of this project is to provide a minimalist API that lets us implement web monetization with zero boiler plate.

// You initialize a WebMonetization instance with a payment pointer
let monetization = WebMonetization.init('$ilp.example.com/me');

That instance object will now be a joy to consume! For example, instead of the mental stress of maintaining the relationship between the standard document.monetization.state property and its three state-change events (monetizationpending, monetizationstart, monetizationstop), we would simply observe this object's .state property directly.

// Observe the state property live
Reflex.observe(monetization, 'state', newState => {
    console.log(newState); // pending, started, stopped
});

If we were to need dynamic start and stop of the payment stream, we wouldn't have to worry about dynamically creating and removing the appropriate <meta> tag. The monetization.start() and monetization.stop() methods automatically take care of that.

And what about tracking progress and deriving totals? The monetization.progress would come into play.

// Observe the progress property live
Reflex.observe(monetization, 'progress', progressDetails => {
    console.log(progressDetails.currentTotal); // Totals since calling the instance's start() method
    console.log(progressDetails.sessionTotal); // Totals since initializing the instance
});

An observable object just gives us only properties to think about - in reading their values both statically and dynamically.

// The totals at any given point in time
console.log(monetization.progress.currentTotal);

// The totals on the fly
Reflex.observe(monetization, 'progress', progressDetails => {
    console.log(monetization.progress.currentTotal);
});

Now, what's even more interesting is that in real life, we can do without observing anything! Observable objects like this are the perfect subjects for the ScopedJS technology! We can simply bind any elements to properties of these object to achieve any UI behaviour. ScopedJS's automatic observability will do the heavy-lifting of keeping everything in sync.

First we would make the WebMonetization instance available to scoped scripts.

let ENV = window.WebNative.ScopedJS.ENV;
ENV.globals.monetization = monetization

And we could simply orchestrate behaviour in the document on the fly!

Here is an example taken from the introductory post where we toggle the visibility of an exclusive section on the page by simply referencing the monetization.state property.

<body>
    <div id="exclusive">
        This is exclusive content!
        <script type="text/scoped-js">
          // This statement will run as a reflex action
          // to each change in monetization.state
          this.classList.toggle('hidden', monetization.state !== 'started');
        </script>
    </div>
</body>

And as other examples in there show, we could go to weave-in more monetization logic as part of our UI's presentational logic.

Submission Category:

Foundational Technology

Demo

The project support banner at web-native.dev featuring Web Monetization was implemented using the Reflex-based Web Monetization API.

Link to Code

This API is currently primarily documented in the introductory post. An official documentation will be arriving soon for the entire list of Reflex-based components at Web-Native.

Repository:

web-native / reflex-components

Reflex Components

Application components and client-side APIs implemented as objects with observable properties.

Documentation

The Reflex Components Official Guide

Issues

To report bugs or request features, please submit an issue to this repository.

License

MIT.

View on GitHub

)

How I built it

I am creating this project as part of the Web-Native project - a larger mission to make certain modern UI development techniques possible at platform level. With the Reflex-based monetization API now part of the Web-Native suite, we now have an all-native way to designing the UI.

Write in Vanilla. - Leave your markup in vanilla HTML - not even a template syntax is required. And yes! none of a framework code (<IfWebMonetized>...</IfWebMonetized>) that would look simple on the eyes but incur unnecessary tooling and engineering, and the inevitable compile step. And, probably one of the most important advantages with a markup-level technology, this implementation makes code more universal! It lets us integrate seamlessly with other wonderful tools that read and write HTML. So, instead of the lock-in with an all-JS stack, a PHP backend that spits out HTML could be everything.
Write your Logic. - The Reflex-based monetization API is not some <custom-element /> that would encapsulate only a predetermined behaviour. It works along with the ScopedJS foundational technology that provides platform-level capabilities for creating anything, offering more flexibility and control.
Maintain the Modern UI Development Pattern. - We simply cannot remember a time without componentization and reactivity anymore! Bringing the Web Monetization API to vanilla HTML does not trade these patterns either. What we get instead is Web Monetization plus componentization plus reactivity - all out of the realm of JavaScript files! (You may want to check out CHTML for the rest of the news.)

Additional Resources/Info

The overall theme of this project is provided by the Web-Native project.

Reflex-Based Web Monetization API

Oxford Harrison — Sat, 06 Jun 2020 04:04:11 +0000

This is a Reflex-based (live) object that wraps the Web Monetization API. Reflex is JavaScript's new observability API that lets us observe JavaScript objects and arrays, combining the magic of the once useful Object.observe() API with other JavaScript reflection APIs - Object and Reflect. Making the Web Monetization API a live, observable object will be opening up new exciting ways to integrate Web Monetization.

Problem Solved

State changes in the Web Monetization API is event-based and this leaves us much engineering work to do. For example, to track changes to the document.monetization.state property, we have to remember three (3) different event names ( monetizationpending, monetizationstart, monetizationstop) and implement three (3) different callbacks. A Reflex-based object would simply offer this property as an observable property.

// The initial state...
console.log(document.monetization.state);

// Subsequent states would now be...
Reflex.observe(document.monetization, 'state', newState => {
    console.log(newState); // pending, started, stopped
});

This is the level of simplicity led by Reflex, underpinning a growing collection of Reflex-based APIs at the Web-Native Project. The long-term promise of this is reactivity without a framework; change-detection in vanilla JavaScript!

The Web Monetization API will now also be reflexive and natively playing with the rest of the Reflex-driven world.

Usage

This library is part of the official Reflex Components package.

import {Money} from '@web-native-js/reflex-components';
import Reflex from '@web-native-js/reflex';

const WebMonetization = Money.WebMonetization;
// Initialize an instance with a payment pointer
let monetization = WebMonetization.init('$ilp.example.com/me');

// Observe state
Reflex.observe(monetization, 'state', newState => {
    // Code here...
});

// Start/stop anytime
monetization.start();
// monetization.stop();

Features

Start/Stop Methods - Calling the .start() method would create the appropriate meta tag in the document head, if not already exists. The .stop() method removes it.
Multiple Pointer Implementations - Multiple instances could be initialized with different payment pointers. Calling .start() on an instance starts a new payment stream and stops any currently active one.
Optional Prompt - Users coming to a monetized site without a supporting browser might need to be informed of the feature. This can be specified in our call to .init().
```
let monetization = WebMonetization.init('$ilp.example.com/me', {prompt: true});
```
The prompt automatically detects the user's browser and asks to take them to the appropriate extension for their browser.
Observable Properties - Now we only need to think of properties, not events and callbacks. These properties are as follows:
- state - (string). This can either be started, pending, stopped. The following three other properties also reflect the state of the instance.
- pending - (object|boolean). This becomes an object when state goes pending, false otherwise. This object maps to the event.detail property of the underlying monetizationpending event.
- started - (object|boolean). This becomes an object when state turns started, false otherwise. This object maps to the event.detail property of the underlying monetizationstart event.
- stopped - (object|boolean). This becomes an object when state gets stopped, false otherwise. This object maps to the event.detail property of the underlying monetizationstopped event.
- progress - (object). This property reflects the event.detail property of the underlying monetizationprogress event. Each progress update also automatically updates the following other properties.
- currentTotal - (object). This object represents the totals between starts and stops on the instance. It begins at zero on each call to .start(). The currentTotal.amount property gives the raw sum of micro payment sent during the stream, while currentTotal.value gives the sum in the format of the active currency.
- sessionTotal - (object). This object represents the overall totals on the instance. The sessionTotal.amount property gives the raw sum, while sessionTotal.value gives the sum in the format of the active currency.
- currency - (string). The payment currency.

Usage in ScopedJS - the Fun Part!

Using ScopedJS, the Reflex-based, micro runtime for JavaScript, We could declaratively create behaviour over a WebMonetization instance right within an HTML markup.

First we would make the WebMonetization instance available to scoped scripts.

let ENV = window.WebNative.ScopedJS.ENV;
ENV.globals.monetization = WebMonetization.init('$ilp.example.com/me');

Now we would be able to reference the monetization object from any scoped script and build behaviour on the fly with any element.

Below we are toggling the visibility of an exclusive section on the page by simply referencing the monetization.state property.

<body>
    <div id="exclusive">
        This is exclusive content!
        <script type="text/scoped-js">
          // This statement will run as a reflex action
          // to each change in monetization.state
          this.classList.toggle('hidden', monetization.state !== 'started');
        </script>
    </div>
</body>

And we can display a live total; and even implement a threshold for our exclusive content. (E.g to show more exclusive content as payment adds up.)

<body>
    <div id="exclusive">
        This exclusive content!
        ...
        And you've streamed us <span id="total"></span> in total.
        <script type="text/scoped-js">
          let elTotal = this.querySelector('#total');
          elTotal.innerHTML = monetization.sessionTotal.value + ' ' + monetization.sessionTotal.currency;
          this.classList.toggle('hidden', monetization.sessionTotal.amount < 10);
        </script>
    </div>
</body>

We could also add a start button to the page.

<body>
    <div id="start">
        Show exclusive content
        <script type="text/scoped-js">
          this.addEventListener('click', () => {
              monetization.start();
          });
        </script>
    </div>
</body>

Working Demo

The web-native.dev's project support banner featuring Web Monetization was implemented using the Reflex-based Web Monetization API.

What's Next for this Project

Good documentation
Bug fixes
More examples
More experiments

Forem: Oxford Harrison

A Fresh Take On Reactivity

webqit / quantum-js

A runtime extension to JavaScript that unlocks Imperative Reactive Programming (IRP) in JavaScript!

Quantum JS

Overview

Laying Reactive Primitives to Rest

Putting the Language to Work in Fun Ways

Reactivity in the Full Range of the JS Language

...and Beyond

Up for a Spin!

And No, We Aren't Done!

Our Star ☆ Button

Much Thanks

Revisiting the HTML Problem Space and Introducing OOHTML

Question 1: How Do You Do Reactivity?

Introducing: Native Data Binding for HTML

Wanna Try?

Introducing: Quantum Reactivity

This Works Today!

Question 2: How Do You Make Components?

Introducing: Namespacing

Good Thinking?

Introducing: Style and Script Scoping

Try Now!

Question 3: How Do You Re-Use Components?

Introducing: HTML Imports

Module Definition

Remote Modules

Module Imports

Scoped Modules

Cool Yet?

Question 4: How Do You Pass Data/Props?

Introducing: The Bindings API

Good Job There?

Introducing: The Context API

Exciting Yet?

webqit / oohtml

Towards a more dynamic and object-oriented HTML.

OOHTML

Status

Implementation

Re-Exploring Reactivity and Introducing the Observer API and Reflex Functions

Update Jan, '24

Re-Exploring Change Detection

Accessors and Proxies; the Tricky Parts

Accessors Got a Flexibility Problem There

Proxies Got an Identity Problem There

The Whole Idea of Internal Traps

The Whole Idea of "Magic Objects"

A Way to Think About the Problem

Rediscovering Prior Art and Nailing the Problem

Introducing the Observer API

An Overview

Featuring Path Observability

Documentation

The Project

webqit / observer

A simple set of functions for intercepting and observing JavaScript objects and arrays.

The Observer API

Re-Exploring the Language of Reactivity

The Toll On Syntax and Mental Model

Solving for Syntax

Solving for Mental Model

Solving for Both

Bringing Reactivity to Imperative JavaScript and Nailing the Problem

Introducing Reflex Functions

An Overview

Change Propagation

Documentation

The Project

webqit / quantum-js

A runtime extension to JavaScript that unlocks Imperative Reactive Programming (IRP) in JavaScript!

Quantum JS

Overview

The Duo and the Prospect for Reactivity

Acknowledgements

References

Rethinking the Modern Web

Table of Contents